A large language model knowledge question answering method based on semantic parsing correction

Abstract

The application discloses a large language model knowledge question and answer method based on semantic analysis correction, and belongs to the natural language reasoning field. The method extracts key entities from a natural language question and matches target entities in a knowledge graph, and mines multiple knowledge paths in the knowledge graph for model training. A prompt instruction and a knowledge path are used to supervise the training of a large language model, so that the large language model has the ability to generate a multi-hop reasoning path that conforms to the structure of the knowledge graph. Based on the trained model, knowledge path decoding is performed, the natural language question is converted into multiple candidate reasoning paths, and is converted into a logical query statement to be executed in the knowledge graph, thereby completing the fact verification and completion of the path. The high-quality path verified by the knowledge graph and the natural language question are input into the large language model to obtain an accurate and interpretable answer. The method realizes the deep fusion of structured knowledge and the language model, and improves the reasoning ability, answer accuracy and interpretability in complex question and answer tasks.

Description

Technical Field

[0001] This invention belongs to the field of natural language reasoning, and more specifically, relates to a knowledge question answering method for large language models based on semantic parsing correction. Background Technology

[0002] With the rapid development of large-scale pre-trained language models, they have demonstrated strong general capabilities in natural language processing tasks such as open-domain question answering, logical reasoning, and complex task understanding. However, current mainstream large language models still face significant challenges when dealing with knowledge-based reasoning, especially in multi-hop question answering scenarios. The main problems include: firstly, the knowledge stored in the parameters of large language models has static and time-sensitive limitations, making it difficult to cover dynamically growing professional domain knowledge systems; secondly, in the absence of structured constraints, the model's reasoning process is prone to "illusion" problems, i.e., generating logically incorrect or factually inaccurate answers, affecting the model's credibility and controllability in practical applications.

[0003] To address this, researchers have attempted to introduce external knowledge sources, particularly well-structured and clearly defined knowledge graphs, to assist models in performing complex reasoning tasks. Knowledge graphs possess explicit entity and relational structures, effectively supporting tasks such as entity linking and path reasoning, and are especially suitable for modeling multi-hop reasoning problems.

[0004] However, using knowledge graphs solely for retrieval-based question answering or logical reasoning often fails to fully grasp the semantic information of natural language questions and lacks flexible language generation capabilities. On the other hand, directly embedding knowledge graph information into prompts to guide large language models in reasoning can still lead to unstable model performance due to improper path selection or redundant path information. Specifically, this manifests in the following ways: 1) There is a lack of effective interaction interfaces between knowledge graphs and large language models, resulting in a disconnect between path generation and natural language understanding; 2) Graph reasoning processes often struggle to structurally decompose complex problems and lack fine-grained modeling capabilities for reasoning sub-tasks; 3) Existing joint models lack a systematic evaluation and backtracking mechanism for the credibility of knowledge paths, easily introducing misleading information; 4) Some methods generate thought chain prompts that lack interpretability and consistency with reasoning paths, making it difficult to achieve controllable optimization of model behavior. Summary of the Invention

[0005] In view of the above-mentioned defects or improvement needs of existing technologies, this invention provides a knowledge question answering method based on semantic parsing correction for large language models, which aims to improve the reasoning efficiency and accuracy of large language models combined with knowledge graphs.

[0006] To achieve the above objectives, according to a first aspect of the present invention, a knowledge question answering method based on semantic parsing and correction for a large language model is provided, comprising: Training phase: S1, preprocess the initial sample set to obtain the fine-tuned sample set; The initial sample set includes: natural language questions, entities involved in the questions, and a knowledge graph for solving the natural language questions; the preprocessing includes: identifying key entities for each natural language question and using them as starting points; locating target entities in the knowledge graph based on the standard answers corresponding to each natural language question and using them as ending points; searching the knowledge graph for all possible paths from the starting point to the ending point and saving them to the fine-tuning sample set; each possible path is represented by the entity nodes it passes through and the relationship types between adjacent entities; S2, take each natural language question in the initial sample set, the entity involved in the question, and the corresponding path in the fine-tuning sample set as a pair of fine-tuning instructions, and save them to the fine-tuning instruction set; construct task prompt instructions and store them in each pair of fine-tuning instructions in the fine-tuning instruction set; S3 uses the fine-tuning instruction set obtained from S2 to supervise the training of the large language model; Application phase: The natural language question to be answered is input into the trained large language model to obtain candidate paths. Each candidate path is converted into a logical query statement, and the logical query statement is converted into a knowledge graph query statement and queried in the knowledge graph to obtain the query result. The logical query statement consists of the starting point and relation sequence in the candidate path, and all entities in the candidate path except the starting point are empty; the query result is not empty if and only if the empty entity in the candidate path is found in the knowledge graph. The query result is concatenated with the natural language question to be answered and then input into the trained large language model to obtain the final answer to the natural language question to be answered.

[0007] According to a second aspect of the present invention, an electronic device is provided, comprising: a computer-readable storage medium and a processor; The computer-readable storage medium is used to store executable instructions; The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the method as described in the first aspect.

[0008] According to a third aspect of the invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to perform the method as described in the first aspect.

[0009] According to a fourth aspect of the invention, a computer program product is provided, comprising a computer program or instructions that, when executed by a processor, implement the method described in the first aspect.

[0010] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects: The knowledge question answering method based on semantic parsing correction provided by this invention first constructs a fine-tuning sample set of the large language model based on knowledge graph semantic parsing, transforming natural language questions, involved entities, and their reachable knowledge paths into structured training data, forming training samples covering various reasoning logics. Then, a structured instruction template is designed to guide the large language model to perform a path generation task from the question entity to the target entity, completing the deep injection of structured knowledge and training of reasoning ability through supervised fine-tuning. Next, after the user inputs a natural language question, the system calls the fine-tuned large language model to generate multiple multi-hop candidate knowledge paths, transforming these paths into formalized logical query statements, and performing verification and completion operations in the knowledge graph based on the logical query statements, eliminating candidate paths containing illusions or inconsistent with facts, and extracting executable, highly credible knowledge paths. Finally, the question and the corrected knowledge paths are merged to form a structured prompt input into the large language model, generating the final answer and natural language explanation by combining structured knowledge with linguistic context information.

[0011] The method provided by this invention not only improves the accuracy and robustness of language models in multi-hop reasoning tasks, but also significantly enhances the interpretability and controllability of generated answers. It achieves deep integration of structured knowledge and language models, improving reasoning ability, answer accuracy, and interpretability in complex question-answering tasks. Based on fine-tuning of a large language model, structured knowledge is injected into the model. By constructing a large language model capable of generating knowledge paths, deep injection of structured knowledge and enhanced semantic parsing capabilities are achieved, enabling the large language model to infer corresponding knowledge graph paths from natural language questions. By effectively guiding the large language model into a controlled generation mode, the generation space is limited to a constrained subset with path-oriented objectives, thereby improving training efficiency and generation accuracy. Attached Figure Description

[0012] Figure 1 A flowchart of a knowledge question answering method for a large language model based on semantic parsing and correction provided in an embodiment of the present invention; Detailed Implementation To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0013] This invention provides a knowledge-based question-answering method for large language models based on semantic parsing and correction, comprising: Training phase: S1, preprocess the initial sample set to obtain the fine-tuned sample set; The initial sample set includes: natural language questions, entities involved in the questions, and a knowledge graph for solving the natural language questions; the preprocessing includes: identifying key entities for each natural language question and using them as starting points; locating target entities in the knowledge graph based on the standard answers corresponding to each natural language question and using them as ending points; searching the knowledge graph for all possible paths from the starting point to the ending point and saving them to the fine-tuning sample set; each possible path is represented by the entity nodes it passes through and the relationship types between adjacent entities; Specifically, step S1 includes S11 and S12.

[0014] S11, Construct an initial sample set, including: natural language questions, entities involved in the questions, and a knowledge graph that can be used to solve the current questions; S12, the initial sample set is preprocessed as follows to obtain the fine-tuning sample set. The preprocessing includes: First, using a named entity recognition tool, key entities with semantic core functions are extracted from the natural language question and used as the starting point for path search; then, based on the standard answer corresponding to the question, the target entity is directly matched in the knowledge graph to determine the endpoint of the path; further, starting from the question entity, multi-hop path search is performed based on the existing triplet information in the knowledge graph, exploring the jump paths between entities layer by layer, and recording the relationships and entity sequences passed through; when a path successfully connects to the target answer entity, the path is determined as a valid knowledge path and retained; if multiple reachable paths exist, all are retained to cover various possible reasoning logics, providing richer training samples for subsequent model fine-tuning, i.e., the fine-tuning sample set.

[0015] In other words, the processing includes the following steps: 1) For the input natural language question, key entity nodes are identified from each natural language question using named entity recognition tools. These entities generally play a core role in the question, and the question entity is determined as the starting point of the path; 2) Based on the standard answer corresponding to each question, the target entity is directly located in the knowledge graph as the endpoint of the path; 3) Starting from the question entity, the system expands outward layer by layer to find all possible entity jump paths. In each jump, the system finds the next entity connected to the current entity based on the existing triple information in the knowledge graph and records the relationship type and entity nodes traversed in between; 4) Once the endpoint of a path matches the standard answer entity, the path is considered a valid path. The system retains it completely as a candidate knowledge path for the current question. If multiple different paths can reach the standard answer, all paths are retained to form a fine-tuning sample set, which is used to train the model to more comprehensively understand various reasoning methods.

[0016] To avoid introducing irrelevant or redundant information through excessively long inference paths, preferably, the path lengths in the fine-tuning sample set are all less than a length threshold, i.e., the lengths are within a controllable range.

[0017] Fine-tuning the sample set, for each natural language question q, the entities involved in q. Locate the target entity in the knowledge graph based on the standard answer to q. The path obtained by searching in the knowledge graph is The elements in the path sequence are connected by arrow symbols to form a linear sequence, which facilitates the model's recognition of the path structure while preserving the entity-relationship connection semantics of the original graph. Each path begins with an entity involved in the problem (denoted as...). ), terminates at the target answer entity (denoted as ), (The middle part is connected by several jump relationships) With intermediate entities , ,..., Series connection.

[0018] S2, take each natural language question in the initial sample set, the entity involved in the question, and the corresponding path in the fine-tuning sample set as a pair of fine-tuning instructions, and save them to the fine-tuning instruction set; construct task prompt instructions and store them in each pair of fine-tuning instructions in the fine-tuning instruction set; S3 uses the fine-tuning instruction set obtained from S2 to supervise the training of the large language model.

[0019] Specifically, with the question , and the corresponding path Construct a pair of fine-tuning instructions and place them into the fine-tuning instruction set; if a problem contains If there are 10 paths, then it will be constructed as follows: For fine-tuning instructions, add them to the fine-tuning instruction set.

[0020] The construction of task prompts aims to enable the model to accurately understand the task objective and focus on the sub-task of "generating a knowledge path starting from the initial entity" during the training phase. This is achieved by clarifying the task intent through natural language templates, thereby controlling the language model's behavior, guiding the model to execute the path generation task, and reducing ambiguity in the training objective. The prompt is: "Please generate a knowledge path that starts from the question entity and reaches the answer entity through a multi-hop path. This path needs to conform to the format requirements of a knowledge graph." This template effectively guides the language model to recognize the current task as a "knowledge path generation task" rather than a "direct answer generation task," and prompts it to perform structured reasoning given the entities involved in the question.

[0021] Task prompts are also included in the fine-tuning instruction set, which is then used to train the large language model. The fine-tuning instruction set ultimately comprises four parts: the natural language question, the entities involved in the question, the task prompts, and the path. .

[0022] By using the fine-tuning instruction set as training samples for a large language model and combining it with a supervised learning mechanism, the parameters of the large language model are fine-tuned, enabling it to reproduce knowledge paths based on natural language problems.

[0023] Specifically, the first step is to deploy an open-source large language model locally on a server. The model architecture can adopt a general large language model, with fine-tuning based on the existing pre-trained parameters. After deployment, the model's training set becomes the fine-tuning instruction set. The input to the large language model is set as a natural language question, the entities involved in the question, and task prompts; the output is set as a path. Set the batch size for fine-tuning to 4 and the learning rate to [value missing]. The warm-up ratio was set to 0.03, the learning rate scheduling method was cosine scheduling, and the total number of training rounds was 3.

[0024] Application phase: The natural language question to be answered is input into the trained large language model to obtain candidate paths. Each candidate path is converted into a logical query statement, and the logical query statement is converted into a knowledge graph query statement and queried in the knowledge graph to obtain the query result. The logical query statement consists of the starting point and relation sequence in the candidate path, and all entities in the candidate path except the starting point (i.e., intermediate entities) are empty; the query result is not empty if and only if the empty entities in the candidate path are found in the knowledge graph. The query result is concatenated with the natural language question to be answered and then input into the trained large language model to obtain the final answer to the natural language question to be answered.

[0025] Specifically, after the structured knowledge injection into the large language model is completed, the fine-tuned large language model is used to begin serving users. Decoding and prediction are performed based on the multi-path knowledge of the large language model, including: (1) For natural language questions input by users, use a large language model to decode multi-hop candidate paths.

[0026] Using a finely tuned large language model, multiple candidate knowledge paths are generated from the natural language questions input by the user. Each path... Each represents a possible reasoning path, starting from the entity involved in the question and gradually connecting to the potential answer entity through multiple relationships. Among them, This relates to natural language issues. Indicate the entities involved in the problem. This represents the candidate answer entity obtained through reasoning. This represents the predicted path.

[0027] Preferably, by setting the decoding strategy of the large language model to beam search, the large language model can generate multi-hop candidate paths at once, thereby improving the answer coverage capability of the large language model for multi-answer questions. For example, a question may correspond to multiple valid answers. Using the beam search decoding strategy can generate multiple reasoning paths (i.e., candidate paths), each leading to a different candidate answer.

[0028] (2) Convert each candidate path into a logical query statement oriented towards the knowledge graph.

[0029] Logical query statements are extracted from each candidate path for subsequent knowledge graph queries: Each candidate path generated by the large language model is further converted into a logical query statement that can be executed on the knowledge graph. The logical query statement consists of a sequence of relations in the path; entities involved in the path are left blank, and wildcards can be used to replace them, such as the wildcard "?". Accordingly, the logical query statement... Understandably, converting knowledge paths into logical query statements is to unify the query interface and simplify subsequent path verification operations in the knowledge graph. Logical query statements allow for efficient checking of whether a path exists in the knowledge graph and the search for more real paths to correct or complete the generated results.

[0030] (3) Use logical query statements to perform fact verification and completion on the knowledge paths generated by the large language model, so as to further improve the accuracy and completeness of the reasoning path.

[0031] Following step (2) above, the candidate paths output by the large language model have been converted into logical query statements, i.e., a formalized structure of "relationship chain + placeholder entity". At this point, it is necessary to convert the logical query statements into knowledge graph query statements that can be directly executed on the knowledge graph.

[0032] Logical query statement It is an abstract path description format, and knowledge graph query is a query language for knowledge graphs that supports triple pattern matching. The knowledge graph query statement used can be any existing knowledge graph query statement, such as SPARQL statements, SQL statements, etc.

[0033] Taking SPARQL statements as an example, the process of converting a logical query statement into a knowledge graph query statement includes: 1) Entity mapping, will Mapped to entity IDs in the knowledge graph; 2) Expanding the relationship chain, Expand into a nested triplet query; 3) Variable definition: using , Variables represent intermediate or target entities; 4) Statement concatenation: Combine the above entity numbers, triple queries, and variables to form a complete knowledge graph query statement.

[0034] For example, with It is the name of an actor. , They are respectively , For example, let's talk about logical query statements. The transformed knowledge graph query statement is: “

[0035] " Through the above transformation, the model's "reasoning path hypothesis" is converted into an executable knowledge graph query language expression.

[0036] (4) Perform a knowledge graph query and return the knowledge path.

[0037] Logical query statements are used to perform fact verification and completion on the knowledge paths generated by the large language model, thereby further improving the accuracy and completeness of the reasoning paths. This involves executing the above knowledge graph queries on an actual knowledge graph (KG) using knowledge graph query statements, and extracting reliable, multi-hop candidate paths as correction results. Specifically, this includes: 1) Submit the knowledge graph query statement to the knowledge graph query interface; 2) Obtain the query results returned by the knowledge graph: a complete knowledge path containing intermediate entities. The query result is only non-empty if all intermediate entities on the candidate path are found in the knowledge graph. If the query result is empty, it indicates that the corresponding logical query (i.e., the candidate path output by the large language model) has problems such as "illusion" or "factual error," and the current logical query is discarded, that is, the candidate path corresponding to the current logical query is considered an invalid path. If the query result is empty, it is determined that the knowledge graph does not contain any knowledge paths that are helpful to the current problem, and the principle of not introducing irrelevant or redundant information is followed. Vacant.

[0038] (5) Integrating knowledge paths Together with the natural language questions input by the user, it completes the answer prediction.

[0039] Will As additional display input, it is concatenated with the natural language question entered by the user and then fed into the large language model for joint reasoning to generate the answer to the natural language question entered by the user and its natural language explanation.

[0040] This invention provides a knowledge-based question answering system for large language models based on semantic parsing and correction, comprising: a computer-readable storage medium and a processor; The computer-readable storage medium is used to store executable instructions; The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the method as described in any of the above embodiments.

[0041] This invention provides a computer-readable storage medium storing computer instructions that cause a processor to perform the method described in any of the above embodiments.

[0042] This invention provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the method described in any of the above embodiments.

[0043] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A knowledge-based question-answering method for large language models based on semantic parsing and correction, characterized in that, include: Training phase: S1, preprocess the initial sample set to obtain the fine-tuned sample set; The initial sample set includes: natural language questions, entities involved in the questions, and a knowledge graph for solving the natural language questions; the preprocessing includes: identifying key entities for each natural language question and using them as starting points; locating target entities in the knowledge graph based on the standard answers corresponding to each natural language question and using them as ending points; searching the knowledge graph for all possible paths from the starting point to the ending point and saving them to the fine-tuning sample set; wherein each possible path is represented by the entity nodes it passes through and the relationship types between adjacent entities; S2, take each natural language question in the initial sample set, the entity involved in the question, and the corresponding path in the fine-tuning sample set as a pair of fine-tuning instructions, and save them to the fine-tuning instruction set; construct task prompt instructions and store them in each pair of fine-tuning instructions in the fine-tuning instruction set; S3 uses the fine-tuning instruction set obtained from S2 to supervise the training of the large language model; Application phase: The natural language question to be answered is input into the trained large language model to obtain candidate paths. Each candidate path is converted into a logical query statement, and the logical query statement is converted into a knowledge graph query statement and queried in the knowledge graph to obtain the query result. The logical query statement consists of the starting point and relation sequence in the candidate path, and all entities in the candidate path except the starting point are empty; the query result is not empty if and only if the empty entity in the candidate path is found in the knowledge graph. The query result is concatenated with the natural language question to be answered and then input into the trained large language model to obtain the final answer to the natural language question to be answered.

2. The method as described in claim 1, characterized in that, The task prompt instruction is: "Please generate a knowledge path that starts from the question entity and reaches the answer entity through a multi-hop path. This path needs to conform to the format requirements of a knowledge graph." 3. The method as described in claim 1 or 2, characterized in that, The logical query statement is a SPARQL statement, and the process of converting the logical query statement into a knowledge graph query statement includes: Entity mapping: Mapping the starting point to an entity number in the knowledge graph; Relationship chain expansion: Expands a sequence of relationships into nested triple queries; Variable definition: Use variables to represent entities other than the starting point; Statement concatenation: Combine entity numbers, nested triple queries, and variables to form a complete knowledge graph query statement.

4. The method as described in claim 1, characterized in that, The path lengths in the fine-tuning sample set are all less than the length threshold.

5. The method as described in claim 1, characterized in that, The decoding strategy of the large language model is beam search.

6. A knowledge-based question-answering system for a large language model based on semantic parsing and correction, characterized in that, include: Computer-readable storage media and processors; The computer-readable storage medium is used to store executable instructions; The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the method as described in any one of claims 1-5.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a processor to perform the method as described in any one of claims 1-5.

8. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the method as described in any one of claims 1-5.

Images