A knowledge graph-based question answering method, system and storage medium

Abstract

The application discloses a kind of based on knowledge graph's question and answer method, system and storage medium, method includes: the entity identification of input question, forms entity set;First sorting screening is carried out based on entity set and knowledge base, obtains the candidate set containing multiple triples;Extract question feature, based on question, candidate set and question feature carries out second sorting screening, obtains a triple as the initial answer of question;Based on question and the triple as initial answer, output declarative answer as final answer, end this question and answer.The application improves universality, reduces labor cost, improves user experience.

Description

Technical Field

[0001] This invention belongs to the field of knowledge question answering technology, specifically relating to a question answering method, system, and storage medium based on knowledge graphs. Background Technology

[0002] A question-and-answer system (QA) refers to a system where the questioner inputs a question, and the system processes it internally and returns the answer. Based on the different corpora used to answer the questions, QA systems are mainly divided into two types: knowledge-based and text-based. Knowledge-based systems utilize existing knowledge such as knowledge graphs to provide answers; text-based systems, such as Facebook's DrQA, find the answers to questions within a given set of documents or paragraphs.

[0003] In recent years, with the development of knowledge graph technology, various large-scale general-purpose knowledge graphs such as OpenKG have emerged, along with open-source domain knowledge graphs in fields such as finance and healthcare. For question answering tasks, when a domain-specific knowledge graph exists, using knowledge graph-based structured question answering offers advantages such as high accuracy, accurate language description, and low resource consumption. Furthermore, with the advent of encoder-decoder frameworks such as Transformer, text generation tasks, as a branch of natural language processing tasks, have flourished, giving rise to various excellent text generation models such as T5 and GPT, making it possible to generate unstructured natural language from structured text.

[0004] Currently, the main question-answering model based on knowledge graphs is "retrieval-matching". For example, Chinese patent application number CN202111387891.7 discloses a method for constructing a question-answering module for citrus management based on knowledge graphs and a question-answering system. Its dataset has three columns: the first column is the question, the second column is the database query statement, and the third column is the corresponding answer in the knowledge base. This patent's dataset uses knowledge graph query statements. When migrated to other fields, the query statements need to be reconstructed, making a direct workflow impossible and requiring significant manpower for annotation.

[0005] It is evident that the main idea behind existing knowledge graph-based question answering technologies is to traverse all paths and use a sorting algorithm to find a path identical to the query statement to directly obtain the corresponding answer. However, this approach suffers from high complexity when exhaustively enumerating entity and predicate combinations. Furthermore, question answering typically feeds back the results directly to the knowledge graph, which are mostly in the format of [subject, relation, object] triples, constituting structured text and lacking the fluency of natural language. Summary of the Invention

[0006] The purpose of this invention is to provide a question-answering method, system, and storage medium based on knowledge graphs, which improves versatility, reduces labor costs, and enhances user experience.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A question-answering method based on a knowledge graph, wherein the knowledge graph is mapped to a knowledge base composed of triples, the question-answering method based on the knowledge graph includes:

[0009] Entity recognition is performed on the input question to form an entity set;

[0010] Based on the entity set and knowledge base, a first sorting and filtering is performed to obtain a candidate set containing multiple triples;

[0011] Extract problem features, and perform a second sorting and filtering based on the problem, candidate set, and problem features to obtain a triplet as the initial answer to the problem;

[0012] Based on the question and the triplet that serves as the initial answer, output a declarative answer as the final answer to end the question-and-answer session.

[0013] Several alternative methods are provided below, but they are not intended as additional limitations on the overall solution above. They are merely further additions or optimizations. Provided there are no technical or logical contradictions, each alternative method can be combined individually with respect to the overall solution above, or multiple alternative methods can be combined with each other.

[0014] Preferably, the first sorting and filtering based on the entity set and knowledge base yields a candidate set containing multiple triples, including:

[0015] The entity set is queried and matched in the knowledge base to obtain the query results;

[0016] Input the question and query results into the entity chain index model, output the relevance score between each triple in the query results and the question, and select the triples with the highest relevance scores as the candidate set.

[0017] Preferably, the extraction of problem features includes: based on the candidate set, performing equal-frequency discretization on the correlation scores to obtain a candidate feature set.

[0018] Preferably, the second sorting and filtering based on the question, candidate set, and question features to obtain a triplet as the initial answer to the question includes:

[0019] The question and the triples in the candidate set are combined to form multiple "question-triple" data pairs;

[0020] Extract features from the "question-triple" data pairs as combined features;

[0021] Each combined feature is concatenated with the problem feature to obtain the concatenated feature;

[0022] Based on the splicing features, the triples in the candidate set are sorted by matching degree, and the triple with the highest matching degree with the question is taken as the initial answer to the question.

[0023] Preferably, the step of extracting features from the "question-triple" data pairs as combined features includes: using the "question-triple" data pairs as input to the BERT model and taking the final [CLS] feature of the BERT model as the combined features.

[0024] Preferably, the step of ranking the triples in the candidate set based on splicing features includes:

[0025] The concatenated features are input into the feedforward neural network to obtain the output of the feedforward neural network;

[0026] The output of the feedforward neural network is input into the binary classifier, and the probability score output by the binary classifier is taken as the matching degree between the current triple and the problem.

[0027] The triples in the candidate set are sorted by their matching degree.

[0028] Preferably, the output of a declarative answer based on the question and the triplet serving as the initial answer includes:

[0029] Select a predefined conversion template based on the problem;

[0030] Based on the transformation template, the triples are output as declarative statements as declarative answers.

[0031] Preferably, the output of a declarative answer based on the question and the triplet serving as the initial answer includes:

[0032] The question and the triplet that serves as the initial answer are combined to form a "question-answer" data pair;

[0033] Input the "question-answer" data into the text generation model, and use the output of the text generation model as the declarative answer.

[0034] A knowledge graph-based question answering system, wherein the knowledge graph is mapped to a knowledge base composed of triples, the knowledge graph-based question answering system comprising:

[0035] The entity analysis module is configured to identify entities in the input question and form an entity set.

[0036] The first-order sorting module is configured to perform a first sorting and filtering based on the entity set and the knowledge base to obtain a candidate set containing multiple triples;

[0037] The second-order sorting module is configured to extract question features, perform a second sorting and filtering based on the question, candidate set and question features, and obtain a triplet as the initial answer to the question.

[0038] The text generation module is configured to output a declarative answer as the final answer based on the question and the triplet that serves as the initial answer, thus ending the question-and-answer session.

[0039] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the knowledge graph-based question-answering method.

[0040] This invention provides a question-answering method, system, and storage medium based on knowledge graphs. It completes question-answering using only information from the knowledge graph without adding additional query data, making it universally applicable across various fields. The entire question-answering system is streamlined, and the original method of traversing entity predicate combinations is replaced by entity chaining, reducing process complexity and saving manpower for annotation. The answers are further processed by replacing structured answers with declarative statements, resulting in a better user experience. Finally, it achieves end-to-end generation of fluent natural language answers from user-submitted questions. Attached Figure Description

[0041] Figure 1 This is a flowchart of a knowledge graph-based question-answering method according to the present invention;

[0042] Figure 2 This is a schematic diagram illustrating data transmission in a knowledge graph-based question-answering method according to the present invention.

[0043] Figure 3 This is a flowchart of the second sorting and filtering process of the present invention;

[0044] Figure 4 This is a flowchart illustrating the generation of declarative answers in this invention;

[0045] Figure 5 This is a schematic diagram of some prediction results and labels in the experiment of this invention. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0048] Example 1

[0049] This embodiment provides a question-answering method based on a knowledge graph, which outputs a declarative corpus as the answer based on the input question. A knowledge graph consists of entities and entity relations, represented in the form of a graph. The knowledge graph organizes data using triples; in a knowledge graph, a node-edge-node can be considered a record, with the first node as the subject, the edge as the predicate, and the second node as the object.

[0050] Based on the nodes and edges in the knowledge graph, the subject is extracted as the subject, the predicate as the relation, and the object as the object, thus mapping the knowledge graph into a knowledge base composed of triples [subject, relation, object], thereby not being limited by the storage database of the knowledge graph.

[0051] The question-answering method proposed in this embodiment can be a question-answering method for a general domain or a question-answering method for a specific professional domain (such as finance, medicine, etc.). The domain to which it is applied is determined by the domain to which the knowledge graph belongs.

[0052] like Figure 1 As shown, the knowledge graph-based question-answering method in this embodiment includes the following steps:

[0053] Step 1: Perform entity recognition on the input question to form an entity set.

[0054] First, the problem is subjected to named entity recognition to obtain one or more entities, which together form an entity set.

[0055] In this embodiment, the NER model is preferably used for entity recognition. In other embodiments, it can be replaced with other entity recognition models such as LSTM-CRF.

[0056] Step 2: Perform the first sorting and filtering based on the entity set and knowledge base to obtain a candidate set containing multiple triples.

[0057] like Figure 2 As shown, since there are entities with the same name in the knowledge base, it is necessary to first query and match the entity set extracted in step 1 in the knowledge base. During the query and matching, each entity in the entity set is matched with the subject and object in the knowledge base one by one to obtain the query results.

[0058] Then, the question and query results are input into the entity chain index model for sorting. The entity chain index model outputs the query results containing the relevance scores of each triple to the question. Based on the relevance scores, the triples in the query results are sorted from high to low, and the triples with the highest relevance scores are selected as the candidate set. That is, the top N subjects with the highest relevance scores are concatenated with all relations and objects to form the candidate set.

[0059] This embodiment only uses questions and knowledge graphs, without using knowledge graph query statements, making the method universal across various domains and streamlining the entire question-answering system process. Furthermore, by introducing an entity chaining model to replace the "entity-predicate" query method to obtain triples, it can link to the knowledge graph and filter out a large number of irrelevant entities, returning entities strongly related to the question.

[0060] Step 3: Extract problem features. Based on the problem, candidate set, and problem features, perform a second sorting and filtering to obtain a triplet as the initial answer to the problem.

[0061] In this embodiment, when extracting problem features, the relevance scores are discretized using an equal-frequency method based on a candidate set to obtain a candidate feature set as the problem features. By discretizing the relevance scores of the Top N subjects using an equal-frequency method, this embodiment can obtain enhanced features and increase the number of ranking channels.

[0062] In other embodiments, when the complexity of the problem increases, the problem features are not limited to the candidate feature set, but may also include features of the problem itself, such as length, entity features of named entity recognition, such as entity attributes, parts of speech, etc.

[0063] The secondary sorting and filtering process incorporates the problem itself and its characteristics, making the final answer more targeted. Since the candidate set's data structure consists of multiple [subject, relation, object] triples, therefore... Figure 3 As shown, in this embodiment, the problem and the triples in the candidate set are first combined in pairs to form multiple "problem-triple" data pairs, and combined features are extracted based on the "problem-triple" data pairs.

[0064] The combined feature fusion expresses the problem and its relationship with each triple. In this embodiment, the "problem-triple" data pair is used as the input to the BERT model, and the final [CLS] feature of the BERT model is taken as the combined feature.

[0065] It should be noted that this embodiment preferably uses the BERT model, but is not limited to using only the BERT model. It can be replaced by various BERT variants such as ALBERT, DeBERT, and RoBERT.

[0066] The higher the feature dimension, the higher the prediction accuracy tends to be. Therefore, in this embodiment, each combined feature is concatenated with the question feature in pairs to obtain concatenated features. Based on the concatenated features, the triples in the candidate set are sorted by matching degree, and the triple with the highest matching degree with the question is taken as the initial answer to the question.

[0067] When ranking the triples in the candidate set based on their matching degree, the concatenated features are first input into a feedforward neural network to obtain its output. Then, the output of the feedforward neural network is input into a binary classifier, and the probability score output by the binary classifier is taken as the matching degree between the current triple and the question. The triples in the candidate set are then ranked according to their matching degree. Therefore, the triplet ranked first is the one with the highest matching degree to the question.

[0068] The problem features used in this embodiment include a candidate feature set obtained by discretizing the relevance scores. Each candidate triple in the candidate set has a relevance score. The discretized ID of the relevance score is mapped to a high-dimensional vector and concatenated with the [CLS] feature vector of the BERT model. After several layers of MLP and SigmaDose, the output score is used as the matching degree ranking criterion to obtain the triple that best matches the question. This embodiment strengthens features to improve the matching degree ranking effect and improve the question answering accuracy. In addition, the use of pluggable feature modules makes it easy to expand and introduce other new question features such as attributes and parts of speech.

[0069] Step 4: Based on the question and the triplet that serves as the initial answer, output the declarative answer as the final answer to end this question-and-answer session.

[0070] Since triples are structured data and lack the fluency of natural language, this embodiment converts triples into declarative answer outputs to improve the user experience.

[0071] In the transformation of structured data into unstructured data, a final declarative answer can be obtained through multiple modules based on the question and the triplet results. According to prior experience, a transformation template is designed based on the question. After obtaining the triplet with the highest matching degree, a predefined transformation template is selected based on the question. Based on the transformation template, the triplet is output as a declarative statement as the declarative answer. The operation of transforming structured data into unstructured data based on the template can be implemented using existing technologies, such as the method disclosed in Chinese Patent Application No. CN201811244279.2, which will not be detailed in this embodiment.

[0072] In addition, such as Figure 4As shown, for cases where declarative answers cannot be constructed using templates, a text generation model can be used. This involves combining the question and the triples used as the initial answer to form a "question-answer" data pair. The "question-answer" data pair is then input into the text generation model, and the declarative text description output by the model serves as the declarative answer. To avoid information duplication, when constructing the "question-answer" data pair, only the object from the triples can be extracted, and the object and question can be combined to form the "question-answer" data pair.

[0073] The preferred text generation model is the mT5 model, but it is not limited to the mT5 model. It can also be replaced by various pre-trained text generation models such as T5, GPT, BART, and JointGT.

[0074] During training, the text generation model first constructs a corpus of declarative answers, extracts the object part of the triples and combines it with the question to form "question-answer" data pairs. Then, the mT5 model is trained using the "question-answer" data pairs as input and the declarative answer corpus as labels. During prediction, only the "question-answer" data pairs are input; no additional declarative answer corpus is required to obtain the final declarative answer.

[0075] This embodiment introduces a text generation model and combines it with a custom template to flexibly process the optimal triples and generate sentence-type declarative descriptions, ensuring that the answer has the fluency of natural language.

[0076] After receiving a question, this embodiment automatically performs named entity recognition, entity chaining, and sorting modeling, selects the optimal triplet result, and directly generates a sentence-type declarative answer based on the triplet structure. Ultimately, it can achieve the effect of giving a question end-to-end and then displaying the declarative answer on the screen.

[0077] The effectiveness of each step in the method of this embodiment is illustrated by the following experiments.

[0078] 1) Steps 1-3 are used as a whole as a ranking model. The entity recognition model in step 1 uses the NER model, the combined feature extraction in step 2 uses the BERT model, and the feedforward neural network in step 3 uses the Rank model. The OpenKG general knowledge graph is used as the knowledge graph in the experiment.

[0079] A self-made dataset containing manually paired questions and triplet pairs was used to validate the triplet output by the ranking model, with the triplets in the question and triplet pair serving as labels. The validation results are shown in Table 1.

[0080] Table 1. Validation results of the ranking model

[0081] F1 Recall AUC 0.765 0.837 0.903

[0082] In Table 1, the F1 score is used to comprehensively evaluate the accuracy and recall of the ranking model. The Recall score refers to the recall rate of the image model, and the AUC score refers to the area under the ROC curve, which can describe the ranking ability of the ranking model. The higher the score, the stronger the model's ranking ability.

[0083] As can be seen from Table 1, the ranking model in this embodiment achieves a ranking ability of over 0.9 on the self-made dataset, demonstrating strong ranking capabilities and ensuring the accuracy of answer selection.

[0084] 2) Using step 4 as the generation model, we verified the text conversion method based on the text generation model, taking the mT5 model as the text generation model. We used the OpenKG general knowledge graph as the knowledge graph in the experiment.

[0085] A self-made dataset was used, containing manually paired question-answer pairs and declarative answer corpora. The declarative answer corpora were used as labels to validate the text output by the generative model. The validation results are shown in Table 2.

[0086] Table 2. Model Validation Results

[0087] Rouge-1 Rouge-2 Rouge-L BLEU-1 BLEU-2 BLEU-3 BLEU-4 METEOR 100 100 100 0.96 0.95 0.94 0.93 0.963

[0088] In Table 2, the metrics Rouge (Recall-Oriented Understudy for Gisting Evaluation) are based on N-gram and are used to describe the recall of the generative model; the BLEU (Bilingual Evaluation Understudy) metric is based on N-gram and is used to describe the accuracy of the generative model; and the METEOR (Metric for Evaluation of Translation with Explicit Order) metric is used to comprehensively describe the accuracy and recall of the generative model. The higher the scores of these three metrics, the better the quality of the generated text.

[0089] As shown in Table 2, the generative model in this embodiment achieves a BLEU-1 score of 0.96 on the self-made dataset, demonstrating that the generative model of this application has strong text generation capabilities. Figure 5 As shown, the predictions output by the generated model have a high degree of overlap with the labels in the self-made dataset.

[0090] Example 2

[0091] A knowledge graph-based question answering system, wherein the knowledge graph is mapped to a knowledge base composed of triples, the knowledge graph-based question answering system comprising:

[0092] The entity analysis module is configured to identify entities in the input question and form an entity set.

[0093] The first-order sorting module is configured to perform a first sorting and filtering based on the entity set and the knowledge base to obtain a candidate set containing multiple triples;

[0094] The second-order sorting module is configured to extract question features, perform a second sorting and filtering based on the question, candidate set and question features, and obtain a triplet as the initial answer to the question.

[0095] The text generation module is configured to output a declarative answer as the final answer based on the question and the triplet that serves as the initial answer, thus ending the question-and-answer session.

[0096] For specific limitations regarding a knowledge graph-based question-answering system, please refer to the limitations of a knowledge graph-based question-answering method mentioned above, which will not be repeated here. This embodiment uses a sorting module to filter answers from the knowledge graph and combines it with a text generation module to form a complete end-to-end question-answering system.

[0097] Example 3

[0098] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of a knowledge graph-based question-answering method.

[0099] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods.

[0100] Any references to memory, storage, database, or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0101] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0102] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A question-answering method based on knowledge graphs, characterized in that, The knowledge graph is mapped to a knowledge base composed of triples, and the question-answering method based on the knowledge graph includes: Entity recognition is performed on the input question to form an entity set; Based on the entity set and knowledge base, a first sorting and filtering is performed to obtain a candidate set containing multiple triples, as well as the relevance score of each triple to the question; Extracting problem features, the extraction of problem features includes: based on the candidate set, performing equal-frequency discretization on the correlation score to obtain a candidate feature set as the problem features; Based on the aforementioned question, candidate set, and question characteristics, a second sorting and filtering process is performed to obtain a triplet as the initial answer to the question, including: The question and the triples in the candidate set are combined to form multiple "question-triple" data pairs; Extract features from the "question-triple" data pairs as combined features; Each combined feature is concatenated with the problem feature to obtain the concatenated feature; Based on the splicing features, the triples in the candidate set are sorted by matching degree, and the triple with the highest matching degree with the question is taken as the initial answer to the question. Based on the question and the triplet that serves as the initial answer, output a declarative answer as the final answer to end the question-and-answer session.

2. The question-answering method based on knowledge graphs as described in claim 1, characterized in that, The first sorting and filtering based on the entity set and knowledge base yields a candidate set containing multiple triples, including: The entity set is queried and matched in the knowledge base to obtain the query results; Input the question and query results into the entity chain index model, output the relevance score between each triple in the query results and the question, and select the triples with the highest relevance scores as the candidate set.

3. The question-answering method based on knowledge graphs as described in claim 1, characterized in that, The step of extracting features from the "question-triple" data pairs as combined features includes: using the "question-triple" data pairs as input to the BERT model and taking the final [CLS] feature of the BERT model as the combined features.

4. The question-answering method based on knowledge graphs as described in claim 1, characterized in that, The method of ranking the triples in the candidate set based on splicing features includes: The concatenated features are input into the feedforward neural network to obtain the output of the feedforward neural network; The output of the feedforward neural network is input into the binary classifier, and the probability score output by the binary classifier is taken as the matching degree between the current triple and the problem. The triples in the candidate set are sorted by their matching degree.

5. The question-answering method based on knowledge graphs as described in claim 1, characterized in that, Based on the question and the triplet used as the initial answer, a declarative answer is output, including: Select a predefined conversion template based on the problem; Based on the transformation template, the triples are output as declarative statements as declarative answers.

6. The question-answering method based on knowledge graphs as described in claim 1, characterized in that, Based on the question and the triplet used as the initial answer, a declarative answer is output, including: The question and the triples that serve as the initial answer are combined to form a "question-answer" data pair; Input the "question-answer" data pair into the text generation model, and use the output of the text generation model as the declarative answer.

7. A question-answering system based on knowledge graphs, characterized in that, The knowledge graph is mapped to a knowledge base composed of triples, and the question-answering system based on the knowledge graph includes: The entity analysis module is configured to identify entities in the input question and form an entity set. The first-order ranking module is configured to perform a first-order screening based on the entity set and the knowledge base to obtain a candidate set containing multiple triples, and a correlation score between each triple and the question. The second-order sorting module is configured to extract question features. This extraction includes: based on the candidate set, performing equal-frequency discretization on the relevance scores to obtain a candidate feature set as the question features; performing a second sorting and filtering based on the question, the candidate set, and the question features to obtain a triplet as the initial answer to the question, including: constructing multiple "question-triple" data pairs by combining the question with triples in the candidate set; extracting features from the "question-triple" data pairs as combined features; concatenating each combined feature with the question features to obtain a concatenated feature; and sorting the triples in the candidate set based on the concatenated feature according to their matching degree, selecting the triplet with the highest matching degree to the question as the initial answer to the question. The text generation module is configured to output a declarative answer as the final answer based on the question and the triplet that serves as the initial answer, thus ending the question-and-answer session.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images