A knowledge question answering method based on large language models and knowledge graphs

Abstract

This invention discloses a knowledge question answering method based on a large language model and a knowledge graph. The method includes: given a question, obtaining a set of candidate answers with the highest scores from the knowledge graph; obtaining a set of documents related to the question from a document library as the context of the question; creating a reading comprehension prompt template; filling the prompt template with the question, the set of documents related to the question, and the set of candidate answers; and inputting the filled prompt template into a large language model (LLM) to output the final set of answers to the question. This invention can provide answers to questions more accurately.

Description

Technical Field

[0001] This invention relates to the field of knowledge question answering technology, and in particular to a knowledge question answering method based on a large language model and knowledge graph. Background Technology

[0002] Knowledge-based question answering technology is an important branch of the question answering field. Early on, it primarily provided knowledge services to users through semantic matching based on traditional knowledge bases or knowledge graphs. With the development of large language models, knowledge-based question answering technology based on large language models is flourishing and has a promising future. However, current knowledge-based question answering technology has the following shortcomings:

[0003] 1) Traditional knowledge graph-based question answering technologies typically include two main approaches: semantic parsing-based and information retrieval-based. The former aims to parse a natural language question into a logical expression, then transform this logical representation into a query language based on the underlying storage structure of the knowledge graph to retrieve the answer. The latter directly retrieves candidate answers from the knowledge graph and ranks them. This method generally first extracts entities from the question, finds subgraphs related to those entities in the knowledge graph, performs semantic representation learning on the question, performs reasoning learning on the entity-related subgraphs, and finds and ranks candidate entities that semantically match the question. These candidate entities are typically neighboring nodes of the entities in the question. This approach often suffers from difficulties in semantic understanding because the given question text is usually short and lacks contextual information. Furthermore, the structured storage of entities in the knowledge graph further reduces semantic information, leading to less accurate answer matching.

[0004] 2) Knowledge-based question answering technology based on large language models. Because large language models are trained on massive corpora, the higher-level model parameters make the model more expressive and more accurate in understanding natural language. However, most large language models are trained on open-source Internet corpora, and the correctness of knowledge cannot be guaranteed. This poses many problems in fields where the accuracy of knowledge is strictly required.

[0005] The knowledge question answering technology that combines large language models and knowledge graphs can make full use of the semantic understanding ability of large language models and the knowledge accuracy of knowledge graphs, overcoming the problems of poor semantic understanding ability and low knowledge quality of traditional knowledge question answering technology. Summary of the Invention

[0006] In view of this, the present invention provides a knowledge question answering method based on a large language model and knowledge graph, which has the advantages of accurate semantic understanding and high correctness of answers.

[0007] This invention discloses a knowledge question answering method based on a large language model and knowledge graph, which includes:

[0008] Step 1: Given a question q, obtain a set A = {a1, ..., a2} from the knowledge graph G, consisting of the k highest-scoring candidate answers to question q. k}; where A is a subset of the entity node set V in the knowledge graph G, a i ∈A represents the k-th highest score candidate answer for question q, and a i ∈V, i.e., a i Belongs to the knowledge graph entity node set V;

[0009] Step 2: Retrieve the collection of documents D related to question q from the document library D. q ={d q,1 ,…,d q,n Using question q as context, create a reading comprehension prompt template, utilizing question q and the set of question-related documents D. q The Prompt template is filled with candidate answer set A, and the filled Prompt template is input into the Large Language Model (LLM) to output the final answer set A for question q. o ={a o,1 ,…,a o,n}, A o It is a subset of the candidate answer set A.

[0010] Further, step 1 includes:

[0011] The knowledge graph embedding representation learning model KGE is used to learn the embedding representation of entity relations in the knowledge graph G in advance; for question q, the entity set E = {e1,…,e...} contained in question q needs to be extracted first using an entity recognition model. n}, the entity set E = {e1,…,e} extracted from problem q n} sequentially link to the corresponding entity set V in the knowledge graph G. e ={v e,1 ,…,v e,n} on; among which, V e It is a subset of the knowledge graph entity node set V;

[0012] The scoring function of the Knowledge Graph Embedded Representation Learning (KGE) model is used to score each candidate entity, and the candidate answer with the highest score is obtained.

[0013] Furthermore, in step 1:

[0014] For each fact triple f in the knowledge graph G i=<h,r,t> Complex embedding representations of head entities, relations, and tail entities generated by the Knowledge Graph Embedding Representation Learning Model (KGE). Let K represent a complex vector space of dimension K, and define a scoring function for the following triplet facts:

[0015]

[0016] Where Re represents the real part of a complex number, and K represents the dimension of the embedding vector. Let e ​​be the conjugate vector of the complex vector e.

[0017] Furthermore, in step 1:

[0018] The knowledge embedding representation module uses the Knowledge Graph Embedding Representation Learning Model (KGE) to learn the representation of each entity and relation in the knowledge graph G. <e h ,e r ,e t >;

[0019] The question embedding representation module passes question q through the encoder. q Encode into a fixed-dimensional K-dimensional embedding representation vector Then it is input into a fully connected network layer, and finally mapped to a complex vector space to obtain vector e. q ;

[0020] Given a question q and a set of candidate answer entities A = {a1, ..., a2} k Extract the topic entity h∈V from question q, and learn the embedding representation of question q;

[0021] The output of the Knowledge Graph Embedding Representation Learning (KGE) model is finally compared with the training sample labels, and the encoder is optimized using a loss function. q Model parameters;

[0022] During the reasoning phase, the Knowledge Graph Embedded Representation Learning Model (KGE) calculates a score by inputting the embedded representations of all entity nodes in the knowledge graph G into a scoring function based on the question q and the topic entity h of question q. The model with the highest score is then taken as the answer to the question.

[0023] Furthermore, the vector e q Represented as:

[0024] e q =encoder q (q)

[0025] The embedding representation of question q is learned in the following way:

[0026]

[0027]

[0028] Where φ is the scoring function of the Knowledge Graph Embedding Representation Learning Model (KGE), and the score of the scoring function is then input into the Sigmoid function. The score for the correct answer is 1, and the score for the incorrect answer is 0.

[0029] The answer to the question is expressed as:

[0030]

[0031] Where V represents all entity nodes in the knowledge graph G.

[0032] Furthermore, in step 2:

[0033] The documents in document library D are pre-segmented, and the segmented documents are stored in a vector retrieval database using a document vectorization tool.

[0034] Using the same vectorization tool, a vector representation of question q is obtained, and documents related to question q are retrieved from a document vector database to form a set D. q .

[0035] Furthermore, Sentence-BERT was chosen as the text2vec document vectorization method. Sentence-BERT borrows the framework of the Siamese network model, inputting different sentences into two BERT models with shared parameters to obtain the sentence representation vector of each sentence, and then using the obtained sentence representation vector for semantic similarity calculation.

[0036] Furthermore, for triples<a,p,n> Sentence 'a' is the anchor sentence, 'p' is an affirmative sentence (representing sentences similar to 'a'), and 'n' is a negative sentence (representing sentences dissimilar to 'a'). Sentence-BERT optimizes its model parameters using the following Triplet loss function:

[0037] Loss = max(|s a -s p |-||s a -s n ||+ε,0)

[0038] Among them, s a ,s p ,s n Let be the sentence embedding representation corresponding to sentences a, p, n, where ||.|| represents distance calculation, ε>0 represents the margin, and s ensures p away from s a Distance compared to s n Nearly ε.

[0039] Furthermore, in step 2:

[0040] The following is a pre-prepared reading comprehension prompt template:

[0041] Given the context, select the answer to the question from the options provided above.

[0042] Then replace the {question} placeholder in the template with question q, and add the relevant document collection D. q By concatenating and replacing the {context} placeholders in the template with semicolons, the candidate answer set A is concatenated and replaced with the {options} placeholders in the template, finally resulting in the Prompt text T. q .

[0043] Furthermore, in step 2:

[0044] The Prompt text T q By inputting into the open-source large language model LLM, and leveraging LLM's language understanding and reasoning capabilities, a more accurate set of answers A can be obtained from the candidate answer options. o ={a o,1 ,…,a o,n}

[0045] Because of the adoption of the above technical solution, the present invention has the following advantages:

[0046] This invention utilizes traditional knowledge graph question-answering technology based on information retrieval to initially recall candidate answers to questions from the knowledge graph, thus limiting the range of possible answers. Then, leveraging the powerful semantic understanding capabilities of the Large Language Model (LLM), it further infers more precise answers within the limited range provided by the knowledge graph. Essentially, it transforms the open-ended question-answering task of traditional large language models into a multiple-choice question-answering task with fixed options based on the knowledge graph. Because of the LLM model's superior semantic understanding capabilities and the fact that all candidate answers entering the LLM model come from a structured and highly accurate knowledge base, the inferred answers are more accurate, avoiding many problems associated with open-ended question answering, such as the inconsistent answer quality caused by the LLM model learning from large-scale open-source internet corpora. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in the embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0048] Figure 1 This is a flowchart illustrating a knowledge question answering method based on a large language model and knowledge graph, according to an embodiment of the present invention. Detailed Implementation

[0049] The present invention will be further described in conjunction with the accompanying drawings and embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art should fall within the protection scope of the present invention.

[0050] As attached Figure 1 As shown, the present invention provides a knowledge question answering technology based on a large language model and knowledge graph, comprising:

[0051] S1: For the question q = "What is the capital of China?", based on the traditional knowledge graph question answering technology, retrieve the candidate answer set A = {Beijing, Chengdu} from the knowledge graph G;

[0052] Extract the entity set E = {“China”};

[0053] S2: For the question q = "What is the capital of China?", retrieve the document set D related to question q based on document retrieval technology. q ={"The capital of China is Beijing", "Beijing is the capital of the People's Republic of China"};

[0054] S3: Develop a Prompt template for the reading comprehension task: P = "Given the context, select the answer to question q from the options provided." and use the question q and the set of relevant documents D. q The Prompt text T is obtained by filling the context portion and answer option portion of the template with the candidate answer set A. q =“Given the context {The capital of China is Beijing; Beijing is part of the People's Republic of China}, choose the answer to the question {Where is the capital of China?} from the following options {Beijing; Chengdu}.”;

[0055] S4: Move the Prompt text T p The input is fed into a large language model (LLM), and the output answer set A is obtained. o = {Beijing};

[0056] As an optimization of the above embodiment, in step S1, the entities and relations in the knowledge graph are pre-learned using a knowledge graph embedding representation learning algorithm. Then, for question q, a knowledge graph-based intelligent question answering method is used to infer candidate answers. This patent uses a knowledge graph-based knowledge question answering method based on information retrieval to obtain candidate answers.

[0057] Preferably, the knowledge graph embedding representation learning algorithm in S1 can be the CompIEx knowledge embedding representation learning algorithm. This algorithm is a semantic matching-based knowledge embedding representation model that can handle a large number of symmetric and asymmetric relations by introducing complex numerical embedding representation. For each fact triple f of the knowledge graph i =<h,r,t> The model generates complex embedding representations of head entities, relations, and tail entities. Let K represent a complex vector space of dimension K, and define a scoring function for the following triplet facts:

[0058]

[0059] Where Re represents the real part of a complex number, and K represents the dimension of the embedding vector. Let e ​​be the conjugate vector of the complex vector e.

[0060] Knowledge graph-based question answering methods for information retrieval typically employ the EmbedKGQA approach. This method utilizes CompIEx knowledge graph embedding representation learning to perform multi-hop reasoning for knowledge answering. The method mainly comprises three modules:

[0061] (1) Knowledge Embedding Representation Module

[0062] This module utilizes the CompIEx knowledge embedding representation learning model to learn the representation of each entity and relation in the knowledge graph G. <e h ,e r ,e t >

[0063] (2) Problem Embedding Representation Module

[0064] This module passes question q through the encoder. q Encode into a fixed-dimensional K-dimensional embedding representation vector The main approach utilizes language models such as BERT to encode the question q, which is then input into four fully connected network layers, ultimately mapping it to a complex vector space to obtain the vector e. q .

[0065] e q =encoder q (q)

[0066] Given a question q and a set of candidate answer entities A = {a1, ..., a2} k Extract the topic entity h∈V from the question. Learn the embedding representation of the question in the following way:

[0067]

[0068]

[0069] Here, φ is the scoring function of the CompIEx model. The score from the scoring function is then input into the Sigmoid function, where a correct answer scores 1 and an incorrect answer scores 0. The model output is finally compared with the training sample labels, and the encoder is optimized using the BCE loss function. q Model parameters.

[0070] (3) Answer selection module

[0071] During the inference phase, the model, based on the question q and the topic entity h, inputs the embedded representations of all entity nodes in the knowledge graph into the scoring function to calculate a score, and selects the highest-scoring answer as the question's answer.

[0072]

[0073] Where V represents all entity nodes in the knowledge graph G.

[0074] As an optimization of the above embodiment, in step S2, the document is vectorized using the text2vec document vectorization method and stored in a vector database such as Faiss. For question q, the question is vectorized using the same text2vec vectorization method, and relevant documents are retrieved from the document vector database.

[0075] Preferably, in step S2, Sentence-BERT is selected as the text2vec document vectorization method. Sentence-BERT borrows the framework of the Siamese network model, inputting different sentences into two BERT models with shared parameters to obtain the sentence representation vector of each sentence, and then using the obtained sentence representation vector for semantic similarity calculation. For triples...<a,p,n> In this model, sentence 'a' is the anchor sentence, 'p' is an affirmative sentence (representing sentences similar to 'a'), and 'n' is a negative sentence (representing sentences dissimilar to 'a'). Sentence-BERT optimizes its model parameters using the following Triplet loss function:

[0076] Loss = max(||s a -s p ||-||s a -sn ||+ε,0)

[0077] Where s a ,s p ,s n Let be the sentence embedding representation corresponding to sentences a, p, n, where ||.|| represents distance calculation, ε>0 represents the margin, and s ensures p away from s a Distance compared to s n Nearly ε.

[0078] As an optimization of the above embodiment, in step S3, the Prompt can be any template for a multiple-choice reading comprehension task, including three parts: question, options, and context.

[0079] Preferably, the present invention uses the template "Given context {context}, select the answer to question {qeustion} from the following options {options} based on the above."

[0080] As an optimization of the above embodiment, in step S4, the filled Prompt text is input into the Large Language Model (LLM), and the answer to the question is inferred based on the capabilities of the Large Language Model.

[0081] Preferably, ChatGLM-6B can be selected as the large language model for LLM. ChatGLM-6B is an open-source dialogue language model that supports both Chinese and English, based on the GLM architecture, with 6.2 billion parameters. Combined with model quantization technology, users can deploy it on consumer-grade graphics cards. However, due to its relatively small size, ChatGLM-6B is known to have many limitations, such as factual / mathematical logic errors, potential generation of harmful / biased content, weak contextual ability, self-awareness confusion, and generation of content that completely contradicts Chinese instructions in response to English instructions. A larger ChatGLM based on the 130 billion parameter GLM-130B is currently under internal testing and development. GLM-130B uses autoregressive whitespace imputation as its primary training objective. In actual training, GLM-130B uses two different mask identifiers, [MASK] and [gMASK], for the generation of short and long texts, respectively.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A knowledge question answering method based on a large language model and knowledge graph, characterized in that, include: Step 1: Given the problem From knowledge graph Obtaining the problem The set of the top k highest-scoring candidate answers ;in, It is a knowledge graph The set of entity nodes in a subset of Representative issue The candidate answer with the highest score, and ,Right now Belongs to the knowledge graph entity node set ; Step 2: From the document library Acquisition and Problems Related document collection As a question Based on the context, create a reading comprehension prompt template and use questions. Collection of documents related to the question and candidate answer set Populate the Prompt template and input the populated Prompt template into the large language model. In the middle, the output problem Final set of answers , It is a set of candidate answers A subset of; In step 2: Pre-segmenting documents in a document repository and storing the segmented documents in a vector search database using a document vectorization tool. The problem is obtained using the same vectorization tools. Vector representation of documents for retrieval in document vector databases Related documents form a collection .

2. The method of claim 1, wherein, Step 1 includes: Pre-learning model using knowledge graph embedding representation Learning knowledge graphs Embedded representation of entity relationships; for question q, the first step is to use an entity recognition model to extract the set of entities contained in question q. , the problem Entity set extracted from Linked sequentially to the knowledge graph The corresponding entity set Above; among them, It is a collection of entity nodes in a knowledge graph. A subset of; Utilizing knowledge graph embedding representation learning models The scoring function scores each candidate entity and obtains a highest-scored candidate answer.

3. The method of claim 2, wherein, In step 1: For each fact triple of a knowledge graph , a knowledge graph embedding representation learning model produces complex embedding representations of the head entity, the relation, and the tail entity , represent a complex vector space of dimension and defines a scoring function for triples of facts as follows:​ wherein, represents the real part of a complex number, K represents the dimension of the embedding representation vector, represents the conjugate vector of a complex vector .

4. The method of claim 3, wherein, In step 1: The knowledge embedding representation module utilizes a knowledge graph embedding representation learning model. Learning knowledge graphs For each entity, the relation is represented as < >; The question embedding representation module embeds the question through an encoder into an embedding representation vector of a fixed dimension K and then inputs a fully connected network layer to finally map to a complex vector space to obtain a vector ; Given problem and a set of candidate answer entities Extraction problem Topic Entities Learning problems Embedded representation; Embedding knowledge graph representation learning model The results of the output of the encoder are finally compared with the training sample labels, and the model parameters of the encoder are optimized by a loss function. During the reasoning phase, the knowledge graph embedding representation learning model Based on the question and the problem Thematic Entities Knowledge graph The embedding of all entity nodes in the input rating function is used to calculate the score, and the highest score is taken as the answer to the question.

5. The method according to claim 4, characterized in that, The vector is represented as: Learn the problem of embedding representations by the following way: in, Embedding representation learning models for knowledge graphs The scoring function is then used to input the scores from the scoring function. For functions, the correct answer scores 1 point, and the incorrect answer scores 0 points; The answer to the question is expressed as: wherein, represent all entity nodes in the knowledge graph .

6. The method according to claim 1, characterized in that, Sentence-BERT was chosen as the text2vec document vectorization method. Sentence-BERT borrows the framework of the Siamese network model, inputting different sentences into two BERT models with shared parameters to obtain the sentence representation vector of each sentence, and then using the obtained sentence representation vector for semantic similarity calculation.

7. The method according to claim 6, characterized in that, For triples , sentence Let p be the anchor sentence, p be an affirmative sentence (representing sentences similar to a), and n be a negative sentence (representing sentences dissimilar to a). Sentence-BERT optimizes the model parameters using the following Triplet loss function: in, For sentences The corresponding sentence embedding representation, Represents distance calculation, , represents an interval, to ensure Leave Distance Nearly at least .

8. The method according to claim 1, characterized in that, In step 2: The following is a pre-prepared reading comprehension prompt template: Given the context, select the answer to the question from the following options. case ； Then use the question Replace the {question} placeholder in the template and add the relevant document collection. Use semicolons to concatenate and replace the {context} placeholders in the template to create a set of candidate answers. Use semicolons to concatenate and replace the {options} placeholders in the template, and you will get the Prompt text. .

9. The method according to claim 8, characterized in that, In step 2: The Prompt text Input open source large language model In this context, by leveraging the language comprehension and reasoning abilities of LLM, a more precise set of answers can be obtained from candidate answer options. .

Images