A generative knowledge question answering method and apparatus

Abstract

A generative knowledge question answering method and apparatus is disclosed. It treats a question and its answer within a specified question-answering application scenario as a question-answer pair. The question-answer pair dataset is proportionally split into a knowledge base construction dataset and a training dataset, used for automated knowledge base construction and question-answering model training, respectively. The question-answering model and the knowledge base then perform generative knowledge question answering on new questions within the specified question-answering application scenario. This invention uses a generative question answering system, resulting in answers that are closer to natural language, more fluent, and more reasonable. This invention does not require a manually constructed knowledge base, enabling automated construction of event graphs from the question set, and the knowledge base construction dataset and training dataset can be easily expanded. This invention surpasses similar existing technologies in the application of real-world domain data, and the entire method offers greater interpretability and modular decoupling compared to end-to-end deep learning models, making it more suitable for practical application scenarios.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and natural language processing technology, and relates to the automated knowledge base construction technology and the generative question answering technology based on the knowledge base in question answering systems. It is a generative knowledge question answering method and device based on automated knowledge base construction. Background Technology

[0002] With the rapid development of the information society, corresponding artificial intelligence technologies are also constantly improving. Question-answering systems in artificial intelligence are a problem with a very wide range of applications, but also one that is extremely challenging and difficult to solve.

[0003] Question answering systems can be categorized based on the format of their answers: choice-based question answering, retrieval-based question answering, and generative question answering. The first two types can be modeled using classification and ranking models, while generative question answering requires a higher level of natural language understanding. Compared to the former two, generative question answering systems can construct answer descriptions that more closely resemble human language, making the answers appear more natural and intelligent.

[0004] Generative question-answering systems typically employ the Seq2Seq framework from deep learning, using language models to generate answers in natural language form, making them suitable for many scenarios. Generative frameworks have achieved good results in machine translation and text summarization tasks within the field of natural language processing. However, knowledge-based question answering requires additional knowledge as a supplement to deep learning, and it demands very precise knowledge, which neural networks struggle to accurately store. Knowledge-based question answering often necessitates the support of a knowledge base to generate appropriate answers.

[0005] Generative knowledge question answering requires first retrieving and partially selecting knowledge from a knowledge base, and then fusing the selected knowledge with a deep learning model so that the model generates meaningful and accurate answers with the support of that knowledge. Recent generative knowledge question answering systems such as GenQA and CoreQA select knowledge points to support a Seq2Seq model by making choices within the knowledge base. This method can only be used to answer complex questions that rely on knowledge within a single hop range of the knowledge base, and it places high demands on the structure and completeness of the knowledge base.

[0006] Traditional knowledge-based question-answering systems have extensive knowledge bases supporting the applicable question range, such as DBpedia, FreeBase, and WikiData, all built on Wikipedia. These knowledge bases effectively support common-sense questions and play a crucial role in traditional knowledge-based question-answering systems, such as Google and Baidu's search engines. However, for specific scenarios or questions that go beyond basic knowledge points, such as those concerning causality, time series, and comparisons—more complex questions requiring a deeper understanding of the issues and the relevant knowledge—these systems struggle. Furthermore, existing methods often rely on manually building knowledge bases to meet specific needs, which is extremely resource-intensive. For domain-specific applications, manually building knowledge bases is a tedious, complex, and difficult-to-expand solution. Summary of the Invention

[0007] The problem this invention aims to solve is to extract knowledge from data and automatically construct a knowledge base based on the answers to questions in the corresponding domain knowledge question-and-answer session when a knowledge base is lacking, and then generate knowledge question-and-answer sessions for new questions based on the constructed knowledge base.

[0008] The technical solution of this invention is as follows: a generative knowledge question answering method, which takes a question and its answer in a specified question-answering application scenario as a question-answer pair, splits the question-answer pair dataset into a database construction dataset and a training dataset according to a certain ratio, and uses them for automated knowledge base construction and question answering model training, respectively. The training dataset is labeled to obtain an event extraction dataset, and an extraction model is trained through the event extraction dataset. Then, events and event relationships are extracted from the database construction dataset to obtain an event graph knowledge base. After the knowledge base is constructed, the question answering model is trained. Events of the question are extracted from the training dataset and mapped to event nodes in the event graph knowledge base according to similarity. Subgraphs on the graph are selected from the event nodes as subgraphs related to the question. The question text and the selected subgraphs are input into the question answering model based on the Seq2Seq framework, and the answer text is output. The standard answer of the training dataset is used to supervise the output, and the question answering model is trained.

[0009] Finally, the answering model and knowledge base generate knowledge-based question answering for new questions in the specified question-answering application scenario.

[0010] Furthermore, the automated construction of the knowledge base and the training of the question-answering model are specifically as follows:

[0011] The steps involved in the automated knowledge base construction are as follows:

[0012] Step 1: The training dataset is manually labeled to obtain the events and event relationships corresponding to the question-answer pairs, which are used as knowledge points to build the knowledge base, resulting in the event extraction dataset; then, the events in the dataset are combined into pairs and semantic relationship annotation is performed to obtain the event pair semantic relationship dataset.

[0013] Step 2: Train a BERT-based joint extraction model using the labeled event extraction dataset. The joint extraction model uses sequence labeling to extract events and relations.

[0014] Step 3: Apply the joint extraction model obtained in Step 2 to the database dataset to extract the event set and the relationship between events in the database dataset, and obtain a basic event graph.

[0015] Step 4: Train a BERT-based sentence pair classification model using the event pair semantic relationship dataset. After pairing the events extracted in Step 3 into event pairs, use the sentence pair classification model to predict the semantic relationship between each event pair. Add the obtained relationship to the basic event graph. Then, add co-occurrence relationships between the events extracted from the same question-answer pair. After enriching the relationship between events, combine all the obtained events and event relationships into a knowledge base event graph, thus constructing the knowledge base.

[0016] The steps for training the answer model are as follows:

[0017] Step 5: Use a joint extraction model to extract events from the training dataset for each question. Use these events as a summary description of the question. Then, use a sentence pair classification model to predict the relationship between each question event and all nodes in the knowledge base event graph. Select event nodes marked with coreference relationships, linking the question's summary description to nodes in the knowledge base event graph. Perform the same process on the answers in the training dataset, linking the answer events to nodes in the knowledge base event graph.

[0018] Step 6: Based on the selected event nodes, run the PersonalizedPageRank algorithm on the knowledge base event graph to select subgraphs on the graph as subgraphs related to the question;

[0019] Step 7: Input the question text and the selected subgraph into the question-answering model based on the Seq2Seq framework, which combines a graph neural network (GNN) and a copy mechanism, and output the answer text.

[0020] The present invention also proposes a generative knowledge question-answering device, wherein a computer program is configured in the device, and the computer program, when executed, implements the above-described method.

[0021] The beneficial effects of this invention are as follows: (1) The method of this invention uses a generative question-answering system, and the answers obtained are closer to natural language, appearing more fluent and reasonable; (2) The method of this invention does not require the aid of a manually constructed knowledge base, and can automatically construct an event graph for the question set. Furthermore, by expanding the question-answering dataset, it is easy to expand the database and training datasets. Compared to manually constructing the entire knowledge base, this invention only needs to annotate some data to achieve automatic construction of the entire knowledge base, and can achieve algorithm iteration that expands the knowledge base by expanding the dataset; (3) In the application of real-world domain data, the method of this invention surpasses a series of the latest retrieval-based and generation-based question-answering technologies. Moreover, the entire method has more interpretability and modular decoupling compared to end-to-end deep learning models, making it more suitable for real-world application scenarios. The ROUGE values ​​of this invention on the test set are compared with those of the retrieval-based question-answering methods BM25 and SRMRS, and the generation-based question-answering method mBART25 as follows:

[0022] method ROUGE-1 ROUGE-2 ROUGE-L BM25 30.3 15.0 18.6 SRMRS 32.2 15.2 18.9 mBART25 34.1 18.7 24.3 This invention 37.1 31.2 26.8

[0023] As can be seen from the test results, the test results of the present invention are significantly better than those of the existing methods. Attached Figure Description

[0024] Figure 1 This is the overall flowchart of the present invention.

[0025] Figure 2 This is a schematic diagram of a graph-based generative question-answering model for the question-answering module. Detailed Implementation

[0026] The present invention will be further described below with reference to embodiments and flowcharts.

[0027] This embodiment is applied to a geography college entrance examination causal question-and-answer dataset. For this scenario, manually constructing a knowledge base is extremely tedious and inefficient. The method of this invention effectively solves this problem. The question and its answer are treated as a question-and-answer pair. The question-and-answer pair dataset is then split into a 3:2 ratio: an automated knowledge base construction part and an answer model training part, i.e., a knowledge base construction dataset and a training dataset. Figure 1As shown, an event extraction dataset is obtained by labeling the training dataset. An extraction model is trained using the event extraction dataset. Events and event relationships are extracted from the database dataset to obtain an event graph knowledge base. After the knowledge base is constructed, a question-answering model is trained. Events for questions are extracted from the training dataset and mapped to event nodes in the event graph knowledge base based on similarity. Subgraphs on the graph are selected from the event nodes as subgraphs related to the question. The question text and the selected subgraphs are input into the question-answering model based on the Seq2Seq framework. Combining graph neural networks (GNNs) and a copy mechanism, the answer text is output. The output is supervised using the standard answers from the training dataset to train the question-answering model.

[0028] The following is a detailed implementation description. First, let's describe the automated knowledge base construction part:

[0029] Step 1: Label 50% of the training dataset with events. For each question-answer pair, label all events in the question and all events in the answer, and indicate the contextual and causal relationships between the events, thus obtaining the event extraction dataset. Then, for the entire labeled event set, use a combination of rules and manual annotation to indicate the semantic relationships between pairs of events, including coreference, relevance, antonyms, and irrelevance, thus obtaining the event pair semantic relationship dataset.

[0030] Step 2: Train a BERT-based joint extraction model using the event extraction dataset. Figure 1 The module, referred to as the "Event Extraction and Relationship Extraction Model," obtains descriptions of events and relationships between events by processing the output of sequence labels. BERT uses model parameters pre-trained on Chinese corpora. The sequence labeling uses double-line BOEs to indicate the beginning and end of events. The embeddings of two events are then input into a single-classification layer to obtain the classification results of the event relationships.

[0031] Step 3: Use the trained joint extraction model to predict each question-answer pair in the dataset, obtaining the events and event relationships in each pair. Transform the obtained data into a graph structure and merge all event nodes with the same text to obtain a preliminary event graph knowledge base, i.e., the basic event graph. Figure 1 This is referred to as the "Events and Event Relationships" module.

[0032] Step 4: Further, a BERT sentence pair multi-classifier is trained using the labeled event pair semantic relationship dataset. The BERT configuration is the same as the joint extraction model described above. The sentence pair multi-classifier is used to concatenate two events and directly input them into BERT to obtain their embedding representations. Then, the embedding representations are input into a linear classification layer to obtain the final classification result. The trained sentence pair classifier is used for the basic event graph, making a prediction for every two events, thereby obtaining the semantic relationship between any two events in the basic event graph. These semantic relationships are then added to the basic event graph to enrich the connectivity between each node in the graph. Specifically, all event pairs labeled with coreference are merged, so that an event with a single meaning is represented as a node in the graph. In addition, a co-occurrence relation is added between events extracted from the same question-answer pair, and the co-occurrence relation is also added to the basic event graph to complete the relation supplementation, resulting in an automatically constructed event graph knowledge base.

[0033] The next part is the training of the answer model:

[0034] Step 5: For each question on the entire training dataset, apply the previously obtained joint extraction model to extract all events mentioned in the question. These events are used as a summary description of the question, and then mapped to nodes in the knowledge graph. Use the previously trained BERT sentence-pair multi-classifier to classify both the summary description events and each node in the graph. Then, among all the explicit coreference results, use a heuristic similarity ranking algorithm that combines fastText similarity and ROUGE values ​​to obtain the graph nodes most relevant to the summary description events; these are the graph nodes linked to the question events. In some cases, these are event nodes with completely identical text. The same process is used to obtain the graph nodes linked to the answer events for the answers on the training dataset.

[0035] Step Six: Starting from the graph nodes most relevant to the event described in the summary, run the Personalized PageRank algorithm across the entire graph to obtain the weight of each point relative to these event nodes. Then, sort them from largest to smallest and select the top 500 nodes as a subgraph representation of the knowledge base relevant to the question. In subsequent processes, only this subgraph needs to be used as the relevant knowledge scope for the question. Steps Five and Six are now complete. Figure 1 The "Subgraph Selection" module works.

[0036] Step 7: Input the question text and the selected subgraph into the Seq2Seq framework-based question-answering model, combining a graph neural network (GNN) and a copy mechanism to output the answer text. Specifically: Input the subgraph obtained in Step 6 and the natural language text of the question into the model. Figure 2 The graph-based generative question-answering model shown is structured as follows: First, the question text and each event node on the subgraph are encoded using a BERT encoder. Then, the encoded results are used as initialization to run a 3-layer graph neural network (GNN) on the subgraph to obtain further encoded results. The output of each event node is passed through a linear answer classifier to pre-determine whether the node is an answer, and the graph nodes linked by the answer events obtained in step five are used for supervision. The subgraph nodes are sorted according to the output of the linear answer classifier, and the top 20 nodes are selected as the final input event nodes to the Seq2Seq decoder. The above results are concatenated, including the GNN embedding representation of the top 20 nodes, the text representation of the nodes, and the question text itself. Finally, the concatenated result is input into a 6-layer decoder based on a Transformer decoder architecture to obtain the final output text. The standard answer texts corresponding to the questions in the training dataset supervise the decoder output, allowing the decoder to directly output the answers obtained by the model.

[0037] Through the above method, the present invention realizes the automatic construction of a knowledge base based on known questions and answers in a specific scenario, and the generative question answering of new questions in the scenario. The present invention is implemented through a generative knowledge question answering device, in which a computer program is configured, and when the computer program is executed, the above-mentioned generative knowledge question answering method is implemented.

Claims

1. A generative knowledge question answering method, characterized by: The question and its answer in a specified question-answering application scenario are taken as a question-answer pair. The question-answer pair dataset is split into a database construction dataset and a training dataset according to the proportion. These are used for automated knowledge base construction and question answering model training, respectively. The training dataset is labeled to obtain an event extraction dataset. An extraction model is trained through the event extraction dataset. Events and event relationships are then extracted from the database construction dataset to obtain an event graph knowledge base. After the knowledge base is constructed, the question-answering model is trained. Events for the questions are extracted from the training dataset and mapped to event nodes in the event graph knowledge base based on similarity. Subgraphs on the graph are selected from the event nodes as subgraphs related to the questions. The question text and the selected subgraphs are input into the question-answering model based on the Seq2Seq framework, which combines graph neural networks (GNN) and a copy mechanism. The output is the answer text. The standard answers from the training dataset are used to supervise the output, and the question-answering model is trained. Finally, the answering model and knowledge base generate knowledge-based question answering for new questions in the specified question answering application scenario; The automated knowledge base construction and question-answering model training are specifically as follows: The steps involved in the automated knowledge base construction are as follows: Step 1: The training dataset is manually labeled to obtain the events and event relationships corresponding to the question-answer pairs, which are used as knowledge points to build the knowledge base, resulting in the event extraction dataset; then, the events in the dataset are combined into pairs and semantic relationship annotation is performed to obtain the event pair semantic relationship dataset. Step 2: Train a BERT-based joint extraction model using the event extraction dataset. The joint extraction model uses sequence labeling to extract events and relations. Step 3: Apply the joint extraction model obtained in Step 2 to the database dataset to extract the event set and the relationship between events in the database dataset, and obtain a basic event graph. Step 4: Train a BERT-based sentence pair classification model using the event pair semantic relationship dataset. After pairing the events extracted in Step 3 into event pairs, use the sentence pair classification model to predict the semantic relationship between each event pair. Add the obtained relationship to the basic event graph. Then, add co-occurrence relationships between the events extracted from the same question-answer pair. After enriching the relationship between events, combine all the obtained events and event relationships into a knowledge base event graph, thus constructing the knowledge base. The steps for training the answer model are as follows: Step 5: Use a joint extraction model to extract events from the training dataset for each question. Use these events as a summary description of the question. Then, use a sentence pair classification model to predict the relationship between each question event and all nodes in the knowledge base event graph. Select event nodes marked with coreference relationships, linking the question's summary description to nodes in the knowledge base event graph. Perform the same process on the answers in the training dataset, linking the answer events to nodes in the knowledge base event graph. Step 6: Based on the selected event nodes, run the Personalized PageRank algorithm on the knowledge base event graph to select subgraphs on the graph as subgraphs related to the question; Step 7: Input the question text and the selected subgraph into the question-answering model based on the Seq2Seq framework, which combines a graph neural network (GNN) and a copy mechanism, and output the answer text.

2. The generative knowledge question answering method according to claim 1 is characterized in that, in step one, 50% of the training dataset is labeled with events, and all events in the question and all events in the answer are labeled for each question-answer pair. The contextual and causal relationships between events are also labeled to obtain an event extraction dataset. Then, for the entire labeled event set, the semantic relationships between pairs of events are marked by a combination of rules and manual annotation, including coreference, relevance, opposite, and irrelevance, to obtain an event pair semantic relationship dataset.

3. The generative knowledge question answering method according to claim 1, characterized in that: In step four, based on the semantic relationships between the obtained event pairs, all event pairs marked with coreference are merged.

4. The generative knowledge question answering method according to claim 1, characterized in that: In step five, for event nodes with coreference relationships, a heuristic similarity ranking algorithm that combines fastText similarity and ROUGE value is used to obtain the graph node most relevant to the summary description event, which is the graph node linked to the problem event.

5. The generative knowledge question answering method according to claim 1, characterized in that: Step six is ​​as follows: For a problem event, starting from the graph node linked to the summary description, run the Personalized PageRank algorithm on the entire graph to obtain the weight of each node relative to the linked graph node. Then sort them from largest to smallest and select the top N nodes as the subgraph representation of the knowledge base related to the problem. In subsequent processes, only this subgraph needs to be used as the relevant knowledge scope of the problem.

6. A generative question answering method according to claim 1, characterized in that in step seven, the answering model is a graph-based generative question answering model, the structure of which is as follows: first, the question text and each event node on the subgraph are encoded using a BERT encoder; then, the encoded result is used as initialization, and a 3-layer graph neural network (GNN) is run on the subgraph to obtain further encoded results; the output of each event node is passed through a linear answer classifier to obtain a pre-judgment of whether the node is an answer, and the answer event link graph nodes obtained in step five are used to supervise it; the subgraph nodes are sorted according to the output of the linear answer classifier, and the first M nodes are selected as the event nodes of the final input Seq2Seq decoder; the results are concatenated, and the results include the graph neural network embedding representation of the first M nodes, the text representation of the nodes, and the question text itself; finally, the concatenated result is input into a 6-layer decoder based on the Transformer decoder architecture to obtain the final output text.

7. A generative knowledge question-answering device, characterized in that: The device is equipped with a computer program that, when executed, implements the method described in any one of claims 1-6.

Images