Knowledge graph question and answer enhancement method based on large language model
By constructing a knowledge graph question-and-answer enhancement method based on large language models, high-score pseudo-samples are generated and screened, and the problems of insufficient model training and scarce data are solved, and the accuracy and robustness of the knowledge graph question-and-answer model are improved.
Patent Information
- Application Number
- CN202510587205.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-08
- Publication Date
- 2025-08-19
AI Technical Summary
In the prior art, the knowledge graph question-and-answer method has problems such as insufficient model training, scarce data and high data labeling cost in actual applications, resulting in low accuracy of question-and-answer tasks.
Build a knowledge graph question-and-answer enhancement method based on large language models. By generating semantic annotation large model and semantic evaluation model, generate large-scale label-free data and filter high-score pseudo-samples, build enhanced data sets for model training, reduce manual annotation dependence, and improve the robustness of the model.
It significantly reduces the cost of data acquisition, reduces noise data, and improves the Q&A accuracy of the knowledge graph Q&A model in real scenarios.
Smart Images
Figure CN120508618A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of natural language processing technology, and in particular to a knowledge graph question-answering enhancement method based on a large language model. Background Art
[0002] The knowledge graph question answering method based on semantic parsing is committed to converting natural language questions into knowledge graph structured query statements, and then executing the knowledge graph structured query statements in the knowledge graph to obtain answers to natural language questions.
[0003] Existing technologies typically use generative deep learning models to implement this process, but this approach relies on the effectiveness of deep learning model training. When training samples are insufficient or the knowledge graph is too large for the training set to cover, the deep learning model is undertrained, resulting in low accuracy in question-answering tasks. In many real-world applications, the high cost of labeling question-answering data and the difficulty of obtaining it on a large scale often make question-answering tasks face the dilemma of data scarcity. Due to limited data, deep models are prone to underfitting during training, resulting in poor performance in real-world scenarios.
[0004] Therefore, how to improve the question-answering effect of knowledge graph question-answering methods in actual scenarios is still worth exploring. Summary of the Invention
[0005] The purpose of the present invention is to provide a knowledge graph question answering enhancement method based on a large language model, which solves the problems of insufficient model training, high data annotation cost, data scarcity, underfitting, etc. in the knowledge graph question answering tasks in the prior art.
[0006] To achieve the above objectives, the present invention provides a knowledge graph question answering enhancement method based on a large language model, comprising the following steps:
[0007] Build a semantic annotation model that can fully understand the semantic information contained in structured query statements in the knowledge graph and convert it into natural language questions;
[0008] Construct a semantic evaluation model that can evaluate the annotation effect of a large semantic annotation model;
[0009] Generate large-scale knowledge graph structured query statements to be annotated;
[0010] Use a large semantic annotation model to construct pseudo sample pairs of natural language questions and corresponding knowledge graph structured query statements;
[0011] Use semantic evaluation model to filter out high-scoring pseudo sample pairs;
[0012] Merge the screened high-scoring sample pairs with the original dataset to construct an enhanced dataset;
[0013] Use enhanced datasets to perform enhanced training on the knowledge graph question answering model.
[0014] The specific steps to build a semantic annotation model that can fully understand the semantic information contained in the structured query statements of the knowledge graph and convert it into natural language questions include:
[0015] The knowledge graph structured query statement in the model training set is used as input, and the corresponding natural language question is used as output;
[0016] A sequence-to-sequence generative model is trained in the form of instruction fine-tuning to build a large semantic annotation model that can fully understand the semantic information contained in the structured query statements of the knowledge graph and convert it into natural language questions.
[0017] The specific steps of building a semantic evaluation model that can evaluate the annotation effect of the semantic annotation large model include:
[0018] Construct positive sample sentence pairs using the knowledge graph structured query statements in the model training set and the corresponding natural language questions;
[0019] Construct negative sample sentence pairs using the knowledge graph structured query statements in the model training set and the natural language questions corresponding to the errors;
[0020] The constructed positive and negative sample sentence pairs are used as input, and the positive and negative sample labels are used as output. A ranking model is trained in the form of fine-tuning to build a semantic evaluation model that can evaluate the annotation effect of the semantic annotation large model.
[0021] The specific steps for generating large-scale structured query statements for knowledge graphs to be annotated include:
[0022] Through the knowledge graph structured query statement iterative method, large-scale knowledge graph structured query statements to be annotated are generated.
[0023] Among them, the semantic annotation large model is used to construct pseudo sample pairs of natural language questions and corresponding knowledge graph structured query statements. The specific steps include:
[0024] Taking the structured query sentence of the knowledge graph to be annotated as input, the semantic annotation large model fine-tuned by the instructions is used to perform text reasoning and generate the corresponding pseudo-annotated natural language questions;
[0025] The knowledge graph structured query statements to be annotated are matched with the pseudo-annotated natural language questions, and pseudo-sample pairs of natural language questions and corresponding knowledge graph structured query statements are constructed.
[0026] Among them, the semantic evaluation model is used to screen out high-scoring pseudo sample pairs. The specific steps include:
[0027] The pseudo sample pairs are input into the semantic evaluation model to score the semantic similarity between the two and screen out high-scoring pseudo sample pairs.
[0028] The present invention provides a knowledge graph question-answering enhancement method based on a large language model. It generates a large amount of unlabeled data using the knowledge graph query statement specification, and uses the rich language knowledge and semantic understanding ability of the large model to label the unlabeled data, significantly reducing the reliance on manually labeled data and reducing the cost of data acquisition. When using the large model for semantic annotation, semantic evaluation is used for assistance to screen high-scoring labeled samples, reduce noise data, and improve the robustness of the knowledge graph question-answering model. The high-scoring labeled samples are combined with the original data set to construct an enhanced data set for enhanced training of the knowledge graph question-answering model, thereby improving the accuracy of the knowledge graph question-answering model and further enhancing the question-answering effect of the knowledge graph question-answering model in real-world scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art.
[0030] Figure 1 This is a flowchart of the knowledge graph question answering enhancement method based on a large language model of the present invention.
[0031] Figure 2 It is a diagrammatic annotation diagram of the semantic annotation large model of the present invention.
[0032] Figure 3 Schematic diagram of the evaluation of the semantic evaluation model of the present invention. DETAILED DESCRIPTION
[0033] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, but should not be understood as limiting the present invention.
[0034] See also Figures 1 to 3 ,in, Figure 1 This is a flowchart of the knowledge graph question answering enhancement method based on a large language model of the present invention. Figure 2 It is a diagrammatic annotation diagram of the semantic annotation large model of the present invention. Figure 3 Schematic diagram of the evaluation of the semantic evaluation model of the present invention.
[0035] The present invention provides a knowledge graph question answering enhancement method based on a large language model, comprising the following steps:
[0036] S101: Build a semantic annotation model that can fully understand the semantic information contained in structured query statements in the knowledge graph and convert them into natural language questions;
[0037] Specifically, the knowledge graph structured query statements in the model training set are used as input, and the corresponding natural language questions are used as output;
[0038] For each knowledge graph structured query statement, add a same prefix instruction Prefix at the beginning of the query statement;
[0039] Get a pre-trained large language model as the baseline model, which contains the original weight matrix ;
[0040] Using the obtained model input and output, the baseline model is fine-tuned using the Low-Rank Adaptation (LoRA) method;
[0041] During fine-tuning, the cross entropy loss function is used to minimize the difference between the generated question Y and the real question. The difference between
[0042] in, is the length of the output sequence, It is The output word of the time step, It is before The output word of the time step, is the input sequence;
[0043] During fine-tuning, the original weight matrix is kept fixed;
[0044] Introducing an incremental weight matrix , used to represent the original weight matrix Adjustment;
[0045] The incremental weight matrix Decomposed into two low-rank matrices and The product of
[0046] Among them, the matrix The dimension is , the matrix The dimension is , is the preset rank. Much smaller than the original weight matrix The corresponding dimension ;
[0047] During the fine-tuning training phase, only the low-rank matrix is updated and Parameters;
[0048] Effective weights of the fine-tuned model Expressed as original weight With incremental weight The sum of ;
[0049] After the fine-tuning process of S1.4-S1.9, a large semantic annotation model is obtained that can fully understand the semantic information contained in the structured query statements of the knowledge graph and convert it into natural language questions.
[0050] S102: Constructing a semantic evaluation model capable of evaluating the annotation effect of the semantic annotation large model;
[0051] Specifically, we use the knowledge graph structured query statements in the model training set and the corresponding natural language questions to construct positive sample sentence pairs;
[0052] Construct negative sample sentence pairs using the knowledge graph structured query statements in the model training set and the natural language questions corresponding to the errors;
[0053] Take the constructed positive and negative sample sentence pairs as input and the positive and negative sample labels (positive sample sentence pair labels are 1, negative sample sentence pair labels are 0) as output;
[0054] Select a pre-trained language model as the baseline model, for each pair of samples (knowledge graph structured query statement and natural language problems ), which is input into the model to obtain the embedding vector and , using cosine similarity to embed the vector and The semantic similarity between them is calculated to obtain the similarity between the knowledge graph structured query statement and the natural language question. The calculation formula of cosine similarity is:
[0055]
[0056] in, represents the dot product of vectors, and They represent the norm of the vector respectively.
[0057] Using the obtained input and output, the model is trained using the mean squared error loss (MSE) as the loss function, calculated as:
[0058]
[0059] in represents the number of samples in the training set, Indicates the The cosine similarity of the sentences, Indicates the corresponding golden similarity label (i.e., 1.0 for positive samples and 0.0 for negative samples);
[0060] After the training process, we can obtain structured query statements that can effectively evaluate the knowledge graph. and natural language problems Semantic evaluation model for semantic similarity between.
[0061] S103: Generate a large-scale knowledge graph structured query statement to be annotated;
[0062] Specifically, initialize an empty list Used to store query statements;
[0063] In the model training set, for each entity in the knowledge graph structured query statement, the knowledge graph is queried to obtain the neighborhood entities and relationships within two hops to form a logical path. For each logical path, a new structured query statement is constructed according to the specification of the structured query statement used by the specific knowledge graph. ;
[0064] In the model training set, for each question containing a literal, extract the specific literal, determine whether the literal type is a date type or a numeric type based on regular matching, find the corresponding numeric type relationship in the knowledge graph, and for each pair of literals and corresponding relationships, construct a new structured query statement based on the structured query statement specifications used in the specific knowledge graph. ;
[0065] In the model training set, for each question that contains both entities and literals, use S3.2 and S3.3 to construct a new structured query clause. and , and then use the conjunction function to combine the candidate results of the two types of query statements and Merge to get a new structured query statement .
[0066] Will Add to Get the knowledge graph structured query statement to be annotated .
[0067] S104: Use the semantic annotation model to construct pseudo sample pairs of natural language questions and corresponding knowledge graph structured query statements;
[0068] Specifically, instantiate the semantic annotation large model obtained after training in step S101;
[0069] Use the knowledge graph structured query statement to be annotated obtained in step S103 , for each knowledge graph structured query statement , add the prefix instruction Prefix in step S101 at the beginning to get ;
[0070] Will As input, it is input into the semantic annotation model in step S104, and the questions matching the query statement are output;
[0071] Will One-to-one correspondence with the matching questions, construct pseudo sample pairs of natural language questions and corresponding knowledge graph structured query statements.
[0072] S105: Use the semantic evaluation model to filter out high-scoring pseudo sample pairs;
[0073] Specifically, for each pseudo sample pair obtained in step S104, the question is preprocessed, irrelevant tokens (such as punctuation marks, stop words, etc.) in the question are filtered out through rules, and repeated words and sentences in the question are deleted to obtain a cleaned pseudo sample pair;
[0074] Instantiate the semantic evaluation model trained in step S102;
[0075] Input the cleaned pseudo sample pairs into the semantic evaluation model trained in step S102, and output the semantic similarity between the sentence pairs;
[0076] For each pseudo sample pair, when the semantic similarity is greater than the preset threshold When , it is retained as a high-scoring pseudo sample pair, otherwise the sample pair is discarded;
[0077] Get the high-scoring pseudo sample pairs after evaluation.
[0078] S106: Merge the screened high-scoring sample pairs with the original dataset to construct an enhanced dataset;
[0079] S107: Use enhanced datasets to perform enhanced training on the knowledge graph question answering model.
[0080] Specifically, the high-scoring pseudo sample pairs filtered by S105 are merged with the original training set to construct an enhanced dataset;
[0081] Use the enhanced dataset for enhanced training of subsequent question answering models.
[0082] A large amount of unlabeled data was generated using the knowledge graph query statement specification. This unlabeled data was then annotated using the large model's rich language knowledge and semantic understanding capabilities, significantly reducing reliance on manually annotated data and lowering data acquisition costs. When using the large model for semantic annotation, semantic evaluation was used to assist in screening high-scoring annotated samples, reducing noise data and improving the robustness of the knowledge graph question-answering model. High-scoring annotated samples were combined with the original dataset to construct an enhanced dataset for enhanced training of the knowledge graph question-answering model, improving its accuracy and further enhancing its performance in real-world scenarios.
[0083] The above disclosure is merely one or more preferred embodiments of the present application and is not intended to limit the scope of the present application. A person skilled in the art will understand that all or part of the processes of the above embodiments and equivalent changes made in accordance with the claims of the present application are still within the scope of the present application.
Claims
1. A knowledge graph question answering enhancement method based on a large language model, characterized in that: The following steps are involved: Build a semantic annotation model that can fully understand the semantic information contained in structured query statements in the knowledge graph and convert it into natural language questions; Construct a semantic evaluation model that can evaluate the annotation effect of a large semantic annotation model; Generate large-scale knowledge graph structured query statements to be annotated; Use a large semantic annotation model to construct pseudo sample pairs of natural language questions and corresponding knowledge graph structured query statements; Use semantic evaluation models to filter out high-scoring pseudo-sample pairs; Merge the screened high-scoring sample pairs with the original dataset to construct an enhanced dataset; Use enhanced datasets to perform enhanced training on the knowledge graph question answering model.
2. The knowledge graph question answering enhancement method based on a large language model according to claim 1, characterized in that: Build a semantic annotation model that can fully understand the semantic information contained in the structured query statements of the knowledge graph and convert it into natural language questions. The specific steps include: The knowledge graph structured query statement in the model training set is used as input, and the corresponding natural language question is used as output; A sequence-to-sequence generative model is trained in the form of instruction fine-tuning to build a large semantic annotation model that can fully understand the semantic information contained in the structured query statements of the knowledge graph and convert it into natural language questions.
3. The knowledge graph question answering enhancement method based on a large language model according to claim 2, characterized in that: Construct a semantic evaluation model that can evaluate the annotation effect of the semantic annotation large model. The specific steps include: Construct positive sample sentence pairs using the knowledge graph structured query statements in the model training set and the corresponding natural language questions; Construct negative sample sentence pairs using the knowledge graph structured query statements in the model training set and the natural language questions corresponding to the errors; The constructed positive and negative sample sentence pairs are used as input, and the positive and negative sample labels are used as output. A ranking model is trained in the form of fine-tuning to build a semantic evaluation model that can evaluate the annotation effect of the semantic annotation large model.
4. The knowledge graph question answering enhancement method based on a large language model according to claim 3, characterized in that: Generate large-scale structured query statements for the knowledge graph to be annotated. The specific steps include: Through the knowledge graph structured query statement iterative method, large-scale knowledge graph structured query statements to be annotated are generated.
5. The knowledge graph question answering enhancement method based on a large language model according to claim 4, characterized in that: Use the semantic annotation model to construct pseudo sample pairs of natural language questions and corresponding knowledge graph structured query statements. The specific steps include: Taking the structured query sentence of the knowledge graph to be annotated as input, the semantic annotation large model fine-tuned by the instructions is used to perform text reasoning and generate the corresponding pseudo-annotated natural language questions; The knowledge graph structured query statements to be annotated are matched with the pseudo-annotated natural language questions, and pseudo-sample pairs of natural language questions and corresponding knowledge graph structured query statements are constructed.
6. The knowledge graph question answering enhancement method based on a large language model according to claim 5, characterized in that: Use the semantic evaluation model to filter out high-scoring pseudo sample pairs. The specific steps include: The pseudo sample pairs are input into the semantic evaluation model to score the semantic similarity between the two and screen out high-scoring pseudo sample pairs.