A method for answering questions based on a Chinese knowledge base using a dual processing system

Abstract

The application discloses a Chinese knowledge base question answering method based on a double processing system. The knowledge base question answering model mainly comprises a coordination perception module (System 1) and an explicit reasoning module (System 2), and each model component is interrelated and closely combined. Under the demand of the existing knowledge base question answering task, the application proposes a Chinese knowledge base question answering model based on the double processing system. Based on the question raised by a user, System 1 firstly learns the representation of the question and predicts the corresponding simple path, then System 2 retrieves the complex path in the knowledge base for the question according to the simple path generated by System 1, and finally the trained model is used to predict the answer. The application is based on artificial intelligence technology and combines the double-process cognitive theory to design a new type of Chinese knowledge base question answering model. The method has wide application prospects in the intelligent question answering field of various industries.

Description

Technical Field

[0001] This invention designs a Chinese knowledge base question answering method based on a dual-processing system, involving deep learning technology, representation learning technology, natural language processing technology and other fields. Background Technology

[0002] Chinese Knowledge Base Question Answering (CKBQA) is a challenging natural language processing task that requires matching nodes in a structured knowledge base (KB) as factual answers to natural language questions. The knowledge base, serving as the knowledge source for Chinese CKBQA, is represented as a directed graph where nodes represent entities and edges represent relationships between entities. Currently popular knowledge bases include English knowledge bases like Freebase, DBpedia, and Wikidata, and Chinese knowledge bases like Zhishi.me and CNDBpedia. With the emergence of many high-quality domain knowledge bases, research on Knowledge Base Question Answering (KBQA) is also increasing.

[0003] As an important branch of KBQA, CKBQA's implementation strategies are mainly derived from the most common KBQA methods. Currently, existing KBQA methods can be divided into three categories: semantic parsing-based methods, information retrieval-based methods, and deep learning-based methods. Semantic parsing-based methods transform the question into a logical form composed of entities and relations, then construct a query statement based on this logical form to obtain the final answer. To obtain rich logical structures, most methods use specific query languages, such as SPARQL and Cypher, to construct the corresponding query statements. Information retrieval-based methods attempt to retrieve candidate paths corresponding to the question by searching a knowledge base, calculate their semantic similarity to the question, and then output the optimal path to achieve the final answer. This method is not only easy to construct training data for but also convenient for retrieving answers, thus attracting significant attention and demonstrating outstanding performance advantages. With the rapid development of deep learning, more and more scholars are integrating deep neural networks into the first and second methods, calling it deep learning-based methods. These methods transform the question and answer into a vector space through representation learning methods, thereby simplifying the complex KBQA task into similarity calculation, classification, or sequence creation tasks. Early strategies for obtaining question and answer feature vectors primarily relied on word embedding models, such as Word2Vec and GloVe. In recent years, with the emergence of pre-trained models, BERT, XLNet, and GPT have achieved good performance. Research shows that combining deep learning with the KBQA task not only simplifies the task's complexity but also significantly enhances its performance.

[0004] Natural language processing (NLP) questions can be constructed in various ways. In knowledge base question-answering tasks, simple questions are those that can be answered accurately based on a single three-element formula. Current research has achieved relatively high accuracy for simple questions. In contrast, complex questions require multiple triples to bridge the query to obtain the correct answer. Due to the complexity and variability of complex questions, they are difficult to distinguish and respond to. However, existing deep learning-based methods suffer from error propagation problems. They first identify the subject entities in the question and then query the correct answer in the knowledge base based on those identified subject entities. Therefore, the subject entities in the question and their corresponding candidate paths are not fully explored. Furthermore, the sequential reasoning process used by humans has not been studied in existing methods.

[0005] For humans, the reasoning process for answering questions is completed in two stages, and the focus changes with each stage. (1) First, our brains quickly obtain the content presented in the question by analyzing it; (2) Then, based on the output of step 1, we conduct in-depth analysis by querying knowledge in memory to infer the answer. This process is explained as the dual-process theory in cognitive science, which posits that there are two different cognitive systems in human reasoning. Our brains first retrieve information after attention through implicit, unconscious, and intuitive processes, which is called System 1. Then, a definite, conscious, and controllable reasoning process, called System 2, involves sequential thinking in working memory. This process is slower but has unique characteristics of humans. From this perspective, the CKBQA task can be processed based on the dual-process theory. System 1 is responsible for quickly retrieving information about the question and simple paths, while System 2 performs in-depth reasoning on the knowledge base to find complex paths.

[0006] Inspired by the dual-process cognitive theory, we propose a new dual-process system framework for CKBQA tasks, such as... Figure 1 As shown in the diagram. Specifically, System 1 in our framework is implemented by an entity extraction module, an entity linking module, a simple path retrieval module, and a simple path matching model, which captures candidate simple paths for the question using the simple path matching model. System 2 uses a complex path retrieval module and a complex path matching model to reason about the knowledge base (PKUBASE) to achieve complex path retrieval. System 2 first executes System 1 to find candidate simple paths; then, it establishes a complex path retrieval module to obtain rich complex paths from the knowledge base. In the path matching process, we propose a hybrid semantic fusion mechanism to extract valuable information from the question and candidate paths. Summary of the Invention

[0007] This invention proposes a Chinese knowledge graph question answering method based on a dual-processing system for Chinese knowledge base question answering tasks. The model includes System 1 (simple path matching) and System 2 (complex path matching), as follows: Figure 1 As shown on the left, System 1 comprises four basic components: an entity extraction module, an entity linking module, a simple path retrieval module, and a simple path matching model. For System 1, we train it to obtain its optimal network parameters, laying the foundation for the next step of predicting simple paths. For System 2, we first use the trained System 1 to capture simple candidate paths. Next, we apply the complex path retrieval module and the complex path matching model to reason about the knowledge base (PKUBASE) to retrieve complex paths. Finally, we use the trained System 1 and System 2 to perform question reasoning to obtain the answer.

[0008] The present invention achieves the above objectives through the following technical solutions:

[0009] 1. In step (i), the question is segmented using System 1 with two word segmentation tools, Jieba and PkuSeg, to obtain the question's tokens. Then, the tokens are obtained by connecting to the knowledge base to query the corresponding candidate entities and candidate paths. The MacBERT pre-trained model is used to obtain the joint word embeddings of the question and candidate paths. The calculation formula is as follows:

[0010] (1)

[0011] 2. Step (II) embeds the joint terms of the problem and candidate paths. Inputting into the GRU model yields the joint feature vector. Then, it is fused with the joint word embedding from step one to obtain the context feature vector. The calculation formula is as follows:

[0012] (2)

[0013] (3)

[0014] 3. In step (iii), the context feature vector After performing convolution operations using convolution kernels of different scales (1,2,3), fine-grained semantic features are obtained. The calculation formula is as follows:

[0015] (4)

[0016] 4. In step (iv), the fine-grained semantic features of the problem and candidate paths are analyzed. Perform the corresponding max pooling operation to obtain the corresponding pooled feature vector. Then, the pooled features at each scale are concatenated to obtain the final semantic fusion features. The calculation formula is as follows:

[0017] (5)

[0018] (6)

[0019] 5. A fully connected network is used to calculate the semantic similarity between the question and candidate paths. Finally, binary cross-entropy is used as the loss function to constrain the model. The calculation formula is as follows:

[0020] (7)

[0021] 6. The implementation of System 2 is consistent with the above approach. The core difference from System 1 lies in the multi-angle convolutional layer. The complex path matching model uses a more refined combination of convolutions (1,2,3,4,5) to obtain more valuable semantic information. The specific calculation formula is as follows:

[0022] (8)

[0023] (9)

[0024] (10)

[0025] (11)

[0026] (12)

[0027] (13)

[0028] By connecting pooled features of different scales, semantic similarity between the problem and candidate paths is calculated, thereby completing the training of a simple path matching system. Attached Figure Description

[0029] Figure 1 It is a Chinese knowledge base question-answering model structure based on a dual-processing system. Detailed Implementation

[0030] The present invention will be further described below with reference to the accompanying drawings:

[0031] Figure 1This is a Chinese knowledge base question answering model based on a dual-processor system. The model mainly consists of a coordinated perception module (System 1) and an explicit reasoning module (System 2). The various model components are interconnected and trained collaboratively to ultimately achieve the task of predicting answers to Chinese knowledge base questions.

[0032] The purpose of System 1 is to lay the foundation for simple path prediction in the overall model by training a simple path matching model. For example... Figure 1 As shown on the left, the question is first processed by the entity extraction module, entity connection module, and simple path retrieval module to obtain its corresponding candidate entities and candidate paths. Then, the question and candidate paths are jointly fed into System 1 to train the simple path matching model and obtain the optimal System 1 parameters to prepare for subsequent modules.

[0033] The purpose of System 2 is to provide strong support for obtaining more complex paths by training a complex path matching model. For example... Figure 1 As shown on the left, the simple path candidate set is first obtained through System 1 with parameters, and then the complex path retrieval module searches for its corresponding complex path candidate set. Finally, the question and complex candidate paths are jointly fed into System 2 to train the complex path matching model and obtain the optimal System 2 parameters to prepare for the question answering model.

[0034] During the training phase, we first process the question through the entity extraction module, entity joining module, and simple path retrieval module to obtain entities and a set of simple path candidates. Then, we use the trained System 1 to predict simple paths, obtaining a set of simple path candidates. Next, we use the complex path retrieval module to perform queries, obtaining a set of complex path candidates. Finally, we use System 2 with parameters to predict the optimal path. After obtaining the optimal path, we create a SPARQL query statement and then query to obtain the answer.

[0035] To demonstrate the practical application capabilities of Chinese knowledge base question answering, this invention uses two publicly available datasets, CKBQA2019 and CKBQA2020, for model performance validation. Both datasets were manually annotated and released by the KBQA task of the Chinese Knowledge Graph and Semantic Computation Conference. Each data sample consists of a question and answer with a standard query statement (SPARQL). In the experiments, this invention uses the large-scale Chinese knowledge base PKUBASE, which contains 66,191,767 triples, 25,437,419 nodes, and 408,261 relations. This knowledge base was constructed by the Institute of Computer Science, Peking University. Furthermore, this invention also conducts comparative experiments with existing methods; the specific experimental results are shown in the table below.

[0036] Table 1. Experimental comparison results of the network model of this invention with other existing models on public datasets.

[0037]

[0038] As shown in Table 1, the experimental results demonstrate that the Chinese knowledge base question-answering method based on a dual-processing system proposed in this invention outperforms the best existing experimental results in terms of average F1 score on public datasets.

[0039] To further verify the effectiveness of System 1 and System 2 proposed in this invention, detailed ablation experiments were conducted. System 1 and System 2 models were replaced respectively under otherwise unchanged conditions. To ensure the fairness of the comparative experiments, the models were trained under the same experimental environment. Detailed results of the ablation experiments are shown in Table 2.

[0040] Table 2 Ablation experiment results among different modules of the network model of this invention

[0041]

[0042] As can be seen from the experimental results in the table, both System 1 and System 2 of the present invention are beneficial to improving model performance. However, System 2 outperforms System 1. The experimental results further verify the effectiveness of the method of the present invention.

Claims

1. A Chinese knowledge base question-answering method based on a dual-processor system, characterized in that... Includes the following steps: Step 1: Using System 1, the question is segmented using Jieba and PkuSeg to obtain question tokens. Then, the tokens are retrieved from the knowledge base to find the corresponding candidate entities and candidate paths. Finally, a MacBERT pre-trained model is used to obtain the joint word embeddings of the question and candidate paths. ; Step 2: Combine the word embeddings of the problem and candidate paths Inputting into the GRU model yields the joint feature vector. Then, it is fused with the joint word embedding from step one to obtain the context feature vector. ; Step 3: Transfer the context feature vector After performing convolution operations using convolution kernels of different scales (1,2,3), the corresponding fine-grained semantic features are obtained. ; Step 4: Fine-grained semantic features of the problem and candidate paths Perform the corresponding max pooling operation to obtain the corresponding pooled feature vector. Then, the pooled features at each scale are concatenated to obtain the final semantic fusion features. ; Step 5: Use a fully connected network to calculate the semantic similarity between the problem and candidate paths, and finally use binary cross-entropy as the loss function to constrain the model; Step Six: The implementation idea of ​​System 2 is the same as the above method. The core difference from System 1 is the multi-angle convolutional layer. The complex path matching model uses a more refined convolution combination (1,2,3,4,5) to obtain more valuable semantic information.

2. The knowledge base question-answering method based on a dual-processing system according to claim 1, characterized in that... In step one, System1 segments the question using two word segmentation tools, Jieba and PkuSeg, and then uses a MacBERT pre-trained model to obtain the joint word embeddings of the question and candidate paths. The specific calculation method is as follows: (1) In the formula A token representing the problem and candidate paths.

3. The knowledge base question-answering method based on a dual-processing system according to claim 1, characterized in that... In step three, a simple path matching method based on hybrid semantics is proposed, which combines the joint contextual feature vectors of the question and candidate paths. Convolution operations are performed using convolution kernels of different scales (1,2,3), and the calculation method is as follows: (2) (3) (4) (5) (6) Fine-grained semantic features are obtained for different questions and candidate paths. Then, the corresponding max pooling operations are performed, and then pooling features of different scales are connected to calculate the semantic similarity between the problem and the candidate path, thereby completing the training of the simple path matching system.

4. A knowledge base question-answering method based on a dual-processing system according to claim 1, characterized in that... In step six, System 2 constructs a complex path matching method based on hybrid semantics. The core innovation lies in the multi-angle convolutional layer. The complex path matching model uses a more refined combination of convolutions (1,2,3,4,5) to obtain more valuable semantic information. The specific calculation method is shown below: (7) (8) (9) (10) (11) (12) By connecting pooled features of different scales, semantic similarity between the problem and candidate paths is calculated, thereby completing the training of a simple path matching system.

Images