A question-answering method and system in a multi-document scenario

Abstract

The present invention relates to a question-answering method in a multi-document scenario, the method comprising: encoding text information of multiple documents to obtain feature vectors of each document; encoding questions to be answered to obtain feature vectors of the questions to be answered; determining N target documents close to the questions to be answered based on the feature vectors of each document and the feature vectors of the questions to be answered; extracting SPO triples from the N target documents; constructing a knowledge graph based on the SPO triples; extracting M subgraphs from the knowledge graph based on the similarity between entities in the questions to be answered and each node in the knowledge graph, the M subgraphs forming an evidence graph set; obtaining a target evidence graph from the evidence graph set using a classifier; and obtaining a target answer to the question to be answered based on the nodes of the target evidence graph. The present invention improves the accuracy and stability of the question-answering system.

Description

Technical Field

[0001] The present invention relates to the field of language processing technology, and in particular to a question-answering method and system in a multi-document scenario. Background Art

[0002] The purpose of intelligent question-answering systems is to automatically provide people with the information they need. They are currently widely used in many fields, such as medicine, finance, and e-commerce. Most methods focus on simple questions, while solving complex problems has always been one of the challenges of question-answering systems. With the widespread use of the Internet, the application of big data has become possible. It has become one of the best sources for obtaining information and has accelerated the development of various question-answering systems. At present, accurate and efficient question-answering systems can intelligently propose multiple solutions based on people's daily needs and allow users to choose, which has positive significance for individuals, enterprises, and governments.

[0003] With the rapid development of deep learning, text-based end-to-end question answering systems have emerged in recent years. The system has achieved good performance in answering complex questions and has good robustness. The system can usually be decomposed into three subtasks: question analysis, document re-tracing, and answer generation: In question analysis, the current common idea is to rely on large-scale supervised reading comprehension data, which can learn to read real documents and answer complex questions without knowing anything about the language structure in advance. In terms of document selection and answer generation, the main solution currently adopted is to use recurrent neural networks to semantically summarize articles and select articles related to the question as knowledge sources, and then obtain answers by comparing the semantic similarity with the question. However, text-based end-to-end question answering systems also have a very obvious weakness, that is, lack of interpretability, that is, when the question answering system gives an answer based on the trained model, it cannot also give the derivation process, which is unacceptable for some fields with high precision requirements such as medicine.

[0004] In recent years, with the popularity of the concept of knowledge graph, many question-answering systems based on knowledge graph have been proposed. For a given natural language question, this type of question-answering system mainly solves the problem through semantic parsing, and then uses the constructed knowledge graph to query and reason to get the answer. According to the structural characteristics of the knowledge graph, this type of method provides the explainability of the answer by outputting a subgraph related to the question and the answer. Question-answering systems based on knowledge graphs can be divided into three categories. The first is based on semantic parsing. Its main idea is to convert natural language into a series of logical forms, express the entire question as semantics, and then query the knowledge graph through the query language to find the answer. These methods rely on the first step of semantic parsing from natural language to logical form, and there is a problem of error transmission, resulting in poor model performance and low recall. At the same time, the exact matching of these methods reduces the reflection coefficient. The second is based on information extraction. This method extracts the entity in the question and queries the knowledge graph to obtain a subgraph centered on the entity. Each node or edge in the subgraph can be used as a candidate answer. Although this method can improve the recall rate, it is still limited by the incompleteness of the graph because it relies on the mapping between natural language and knowledge graph. The third type is based on vector modeling, where questions and candidate answers are embedded in distributed representations. The distributed representatives are trained through training data to make the vector representation of the question and the score of the correct answer as high as possible. This type of method effectively avoids errors in semantic parsing and improves the performance of the model. However, these methods are still limited by the incompleteness of the graph, which reduces the interpretability of the model.

[0005] In recent years, big data is becoming more and more popular, and people's demand for convenient access to information is growing. Intelligent question-answering systems are becoming more and more important. At present, most mobile platforms and large Internet companies are developing intelligent question-answering systems. Intelligent question-answering systems can automatically analyze questions based on people's input and find ideal answers for people, greatly reducing people's time costs in the office. For question-answering systems, their most important work is mainly divided into two aspects: analysis of user needs and extraction of information sources based on needs. The analysis of user questions mainly relies on knowledge of natural language processing, while in recent years, the extraction of information that meets the needs mainly uses methods such as deep learning. With the advancement of models, the accuracy of question-answering is gradually improving, but the biggest problem with deep learning question-answering today is the ambiguity of the derivation process, which has led to the questioning of related models. Summary of the invention

[0006] Based on this, the purpose of the present invention is to provide a question-answering method and system in a multi-document scenario, thereby improving the accuracy and stability of the question-answering system.

[0007] To achieve the above object, the present invention provides the following solutions:

[0008] A question-answering method in a multi-document scenario, the method comprising:

[0009] Encoding text information of multiple documents to obtain a feature vector of each document;

[0010] Encode the question to be answered to obtain a feature vector of the question to be answered;

[0011] Determining N target documents that are close to the question to be answered based on the feature vectors of each of the documents and the feature vectors of the question to be answered;

[0012] Extracting SPO triples from the N target documents;

[0013] Construct a knowledge graph based on the SPO triples;

[0014] Extracting M subgraphs from the knowledge graph according to the similarity between the entity in the question to be answered and each node in the knowledge graph, wherein the M subgraphs constitute an evidence graph set;

[0015] Obtaining a target evidence graph from the evidence graph set using a classifier;

[0016] A target answer to the question to be answered is obtained according to the nodes of the target evidence graph.

[0017] Optionally, before encoding the text information of the multiple documents to obtain the feature vector of each of the documents, the method further includes:

[0018] Get semi-structured data of multiple documents;

[0019] Semi-structured data of multiple documents are processed to obtain text information of the multiple documents.

[0020] Optionally, the method further includes: using the target evidence graph as a source of explainability of the target answer.

[0021] Optionally, the method further comprises:

[0022] The classifier is obtained by training an attention-based graph neural network using data from the HEAD-QA dataset.

[0023] Optionally, extracting SPO triples from the N target documents specifically includes:

[0024] Perform part-of-speech tagging on the N target documents and extract phrases;

[0025] Performing named entity recognition on the N target documents;

[0026] The named entity and the phrase are taken as entity objects of the SPO triple.

[0027] Optionally, encoding the text information of the multiple documents to obtain the feature vector of each of the documents specifically includes:

[0028] The text information of each of the documents is encoded using the TF-IDF method to obtain a feature vector of each of the documents.

[0029] Optionally, encoding the text information of each document using the TF-IDF method to obtain a feature vector of each document specifically includes:

[0030] Eliminating stop words in the text information of each of the documents to obtain each processed document;

[0031] Divide the frequency of each word appearing in the processed document by the total frequency of each word appearing in all documents to obtain the word frequency of each word in each processed document;

[0032] The feature vector of each document is determined according to the word frequency of each word in each processed document.

[0033] Optionally, encoding the question to be answered to obtain a feature vector of the question to be answered specifically includes:

[0034] The TF-IDF method is used to encode the question to be answered to obtain the feature vector of the question to be answered.

[0035] The invention also provides a question-answering system in a multi-document scenario, the system comprising:

[0036] A document feature vector determination and acquisition module is used to encode text information of multiple documents to obtain a feature vector of each document;

[0037] A feature vector acquisition module for the question to be answered, used to encode the question to be answered and obtain the feature vector of the question to be answered;

[0038] A target document determination module, used to determine N target documents close to the question to be answered based on the feature vectors of each of the documents and the feature vectors of the question to be answered;

[0039] An SPO triple extraction module, used for extracting SPO triples from the N target documents;

[0040] A knowledge graph construction module, used to construct a knowledge graph according to the SPO triples;

[0041] An evidence graph set determination module, used to extract M subgraphs from the knowledge graph according to the similarity between the entity in the question to be answered and each node in the knowledge graph, wherein the M subgraphs constitute an evidence graph set;

[0042] A target evidence graph acquisition module, used to acquire a target evidence graph from the evidence graph set using a classifier;

[0043] The target answer acquisition module is used to obtain the target answer to the question to be answered according to the nodes of the target evidence graph.

[0044] Optionally, the system further comprises:

[0045] The target evidence graph is used as a source of explainability for the target answer.

[0046] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0047] The present invention discloses a question-answering method and system in a multi-document scenario, which determines N target documents close to the question to be answered from the feature vectors of multiple documents, extracts SPO triples from the N target documents, and constructs a knowledge graph based on the SPO triples, determines an evidence graph set based on the similarity between the entity in the question to be answered and each node in the knowledge graph, and then uses a classifier to obtain the target answer from the evidence graph set, thereby improving the accuracy and stability of question-answering. In addition, the target evidence graph serves as the interpretability source of the target answer, thereby improving the interpretability of the target answer. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0049] Figure 1 A flowchart of a question-answering method in a multi-document scenario according to the present invention;

[0050] Figure 2 A schematic diagram of the overall process of a question-answering method in a multi-document scenario according to an embodiment of the present invention;

[0051] Figure 3 The effect diagram of the accuracy of answering questions with different methods under different problem complexities;

[0052] Figure 4 This is a schematic diagram of the structure of a question-answering system in a multi-document scenario according to the present invention. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] The purpose of the present invention is to provide a question-answering method and system in a multi-document scenario, thereby improving the accuracy and stability of the question-answering system.

[0055] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0056] Figure 1 FIG. 1 is a flow chart of a question-answering method in a multi-document scenario according to the present invention. Figure 1 As shown, a question-answering method in a multi-document scenario includes:

[0057] Step 101: Encode text information of multiple documents to obtain a feature vector of each document.

[0058] The method before step 101 further includes: obtaining semi-structured data of multiple documents; and processing the semi-structured data of multiple documents to obtain text information of multiple documents, specifically including: using the wikiextractor package to process the semi-structured data of Wikipedia.

[0059] Wherein, step 101 specifically includes:

[0060] The TF-IDF method is used to encode the text information of multiple documents to obtain the feature vector of each document, specifically including:

[0061] Stop words in the text information of each of the documents are removed to obtain each processed document.

[0062] The frequency of each word in the processed document is divided by the total frequency of each word in all documents to obtain the word frequency of each word in each processed document.

[0063] The feature vector of each document is determined according to the word frequency of each word in each processed document, that is, the result vector composed of all word frequencies is used as the feature of the article.

[0064] Step 102: Encode the question to be answered to obtain a feature vector of the question to be answered.

[0065] Wherein, step 102 specifically includes:

[0066] The TF-IDF method is used to encode the question to be answered to obtain the feature vector of the question to be answered.

[0067] Step 103: According to the feature vectors of each document and the feature vector of the question to be answered, determine N target documents that are close to the question to be answered. In this embodiment, the value of N is 5.

[0068] Step 104: Extract SPO triples from the N target documents.

[0069] Wherein, step 104 specifically includes:

[0070] Use the NLTK package to perform part-of-speech tagging on the N target documents and extract phrases.

[0071] The NLTK package is used to perform named entity recognition on the N target documents.

[0072] The named entity and the phrase are taken as entity objects of the SPO triple.

[0073] Step 105: Construct a knowledge graph based on the SPO triples.

[0074] Use graphs to represent the relevant knowledge involved in the problem. A huge original graph (knowledge graph) is constructed based on the SPO triples extracted from the articles related to the problem, and then possible candidate evidence graphs are extracted from it. The original graph consists of nodes and edges: G = (V, E), where G represents the original graph, V represents the set of nodes in the original graph, and E represents the set of edges in the original graph.

[0075] The knowledge graph mainly includes the following contents:

[0076] Nodes are extracted from the information in the SPO triples, as well as directed edges connecting the nodes, where S and O are assigned entity attributes and P is assigned predicate attributes.

[0077] The attribute nodes connected to the Entity node also have directed edges connecting each other, which are mainly extracted based on the large number of is-a sentences about the article topics in Wikipedia.

[0078] A bidirectional edge connecting entity nodes or relationship nodes with the same meaning.

[0079] The specific process of establishing a knowledge graph is as follows:

[0080] S1: Open information extraction is used to extract triples, that is, when the relationship is unknown, the input corpus and a small amount of empirical rules independent of the relationship are used to extract the required entities and relationships.

[0081] S2: In the preprocessing process, the NLTK package is used to perform part-of-speech tagging and named entity recognition on the extracted articles. When processing the SPO triple information, the named entities and the extracted phrases are respectively regarded as entity objects.

[0082] S3: In the process of extracting triples, the pairwise distances between the parts (entities) of the triples are used as the confidence index of each triple relationship.

[0083] S4: The relationship between the predicate in the triple and the other two entities is taken as an edge, and the predicate is taken as a separate node. That is, each SPO triple can form two entity nodes and one predicate node in the knowledge graph, and the confidence index is taken as the weight of the corresponding node.

[0084] S5: A semantic dictionary is used, which is compiled from a large knowledge base and includes the probability that the same text form points to different standard entities. Through this semantic dictionary, the similarity between two nodes can be calculated and entities with similarity above a threshold can be merged.

[0085] S6: Build the processed triples into a graph to form a knowledge graph.

[0086] The entire original graph establishment process is shown in Algorithm 1. When establishing the original graph, the predicate in the SPO triple is not used as an edge but as a separate node, that is, each SPO triple can form two entity nodes and a predicate node in the knowledge graph. If the S and O nodes already exist in the graph, an edge will be extended on the basis of the original triple to connect the new node. At the same time, the present invention calculates the probability that each text form points to the same entity, and connects two nodes whose similarity exceeds the threshold through a bidirectional edge.

[0087]

[0088]

[0089] Step 106: Based on the similarity between the entity in the question to be answered and each node in the knowledge graph, M subgraphs are extracted from the knowledge graph, and the M subgraphs constitute an evidence graph set.

[0090] Among them, step 106 specifically includes taking the target node in the knowledge graph corresponding to the keyword in the question to be answered as an anchor point, and extracting a number of subgraphs from the original knowledge graph as a set of candidate evidence graphs. Among them, the keyword phase is the word in the question to be answered after removing pronouns, modal particles and stop words.

[0091] Suppose there is an entity set {q1, q2, q3, ...q n}, when there is a node V in the original graph and any question entity q i Satisfaction between S is the similarity threshold, and the node V is called an anchor.

[0092] Step 107: Obtain a target evidence graph from the evidence graph set using a classifier.

[0093] The method specifically also includes:

[0094] The classifier is obtained by training an attention-based graph neural network (AGNN) using data from the HEAD-QA dataset.

[0095] The process of training an attention-based graph neural network is:

[0096] S1: In the process of obtaining the vector representation of the problem itself, the TextCNN model is used and its dimension is set to be equal to the subsequent graph vector.

[0097] S2: For each evidence graph, extract all nodes and edges from it, where the edges are encoded in COO format, that is, set two lists, the first list contains the index of the source node, and the index of the target node is specified in the second list. For the nodes, the vector obtained by training Wikipedia data is passed through a linear layer as the initialization state of each node.

[0098] S3: During the training process, when extracting candidate evidence graphs, check the non-anchor nodes of each candidate evidence graph. If a node is very close to the true answer, it is set as positive, otherwise it is negative. In each round of training, the vector V output by each evidence graph is g With the problem vector V q The cosine similarity of is used as an excitation parameter to calculate the features of the graph.

[0099] S4: The Adam optimizer is used to train the model, and its learning rate is set to 0.01. In order to prevent the model from overfitting, a Dropout layer is used in the middle of the model to actively forget certain dimensions of the vector, and the forgetting rate is set to 30%. Since the vector dimension of the node is 300, two linear layers are added to compress the vector.

[0100] Step 108: Obtain a target answer to the question to be answered based on the nodes of the target evidence graph.

[0101] The method also includes: using the target evidence graph as a source of explainability for the target answer.

[0102] TextCNN is used in training to obtain a vector representation of a question. It first encodes the question into a set of features, and then converts each feature into a vector. After the set of word vectors is extracted by a one-dimensional convolutional layer, it enters a MaxPolling layer to make the length of the obtained sentence vector uniform. In this way, the final extracted vector can be used as the question representation V when training the candidate graph later. q As for the graph input, the semantic information represented by the nodes and the spatial information represented by the edges are simultaneously input into the AGNN, and iterated through a custom loss function.

[0103] The following describes in detail a question-answering method in a multi-document scenario of the present invention with a specific embodiment.

[0104] (1) Establishing article word frequency features

[0105] The present invention converts all articles in Wikipedia into vectors of TF-IDF weights. Assuming that m documents constitute a document set: N = {N1, N2, ..., N m}, and N i The number of occurrences of phrase j in the article is represented by C ij , then the tf (Term frequency) and idf (Inverse Document Frequency) of word j are:

[0106]

[0107]

[0108] Where n is N i The number of phrases in the article, h is the total number of documents containing phrase j, from which we can calculate the number of phrase j in N i The overall frequency in the article:

[0109] TF-IDF = TF*IDF, where TF-IDF represents the comprehensive frequency.

[0110] Double counting is used for word counting. The combined frequency of the two words before and after is calculated simultaneously. Then a hash table is used to make each phrase correspond to a hash value. In this way, each document can be encoded into a dictionary containing the frequency of each phrase. By comparing the document with the phrases in the question to be answered, the five closest articles can be selected.

[0111] (2) Constructing the original map

[0112] Each set of information in the document is defined as a triple consisting of two entities and a relationship, which are represented as (S, P, O). The present invention uses open information extraction to extract triples, that is, when the relationship is unknown, input corpus and a small amount of empirical rules independent of the relationship to extract the required entities and relationships. In the preprocessing process, the NLTK package is used to perform part-of-speech tagging and named entity recognition on the extracted articles. When processing the SPO triple information, the named entity and the extracted phrase are respectively used as entity objects. Regarding SPO triples, extraction is mainly based on dependency syntax. First, a child node of the dependency syntax of the word is generated for each entity object, which mainly stores the relationship and the position of the corresponding son word, and then a dependency structure of the parent-child array of the word is generated, which mainly records the part of speech of the word, the part of speech of the parent node and the relationship between them. In the loop process of each entity, the relationship between the subject and the object, the subject and the object relationship after the attributive are found and extracted. After eliminating irrelevant words that do not have a dependency structure, the dependency syntax analysis results and the specific sentences are analyzed together to generate SPO triples.

[0113] In the process of extracting SPO triples, the present invention uses the pairwise distances between the parts of the SPO triples as the confidence index of each SPO triple relationship. Assuming that in an SPO triple, the distances between S and P, and between P and O are d1 and d2 respectively, then the confidence indexes of SP and PO are:

[0114]

[0115]

[0116] When the same pair of SPO triples appears in different documents {S i}, their common confidence index is

[0117]

[0118] When constructing the original knowledge graph, it is necessary to use triples (SPO triples) in multiple documents at the same time. At the same time, in order to determine the importance of different nodes, different weights are assigned to nodes and edges during the construction process. The weight is mainly determined based on the premise that the more relevant the entity mentioned in the problem is, the more important it is to solve the problem. The graph in the present invention does not only treat the predicate in the triple as an edge but as a separate node, that is, each SPO triple can form two entity nodes and predicate nodes in the knowledge graph. If the S and O nodes already exist in the graph, an edge will be extended on the basis of the original triple to connect the new node. Since the same node may appear in different forms in multiple triples, the present invention uses a semantic dictionary, which is compiled from a large knowledge base and includes the probability of the same text form pointing to different standard entities. The similarity of two nodes can be calculated through this dictionary, that is, when the same standard entity e exists in two nodes i and j, the difference in probability is the similarity, otherwise the similarity is 0:

[0119]

[0120] When setting the weight of each node, first observe whether there is a standard entity e that the problem phrase and the node point to together. If so, it is used as the weight and the similarity is higher than the threshold. Otherwise, it is set to 0:

[0121]

[0122] Regarding the edges in the triples, the present invention uses the confidence index obtained in the previous triple extraction process as the weight.

[0123] W e =b v1摮v2

[0124] Among them, v1 and v2 are the two nodes connected by edge e.

[0125] (3) Extracting evidence graph collection

[0126] In order to extract a suitable evidence graph from the original knowledge graph, the present invention first determines several anchors from the question according to the previous definition of anchor points, and then adopts such an algorithm: each anchor node is used as the initial state of a tree to start synchronous growth, and each step selects the edges and nodes with larger weights. When two trees meet, they are merged. When any tree covers at least one node in each anchor node set, the tree stops growing and exists as a candidate evidence graph. The present invention sets a maximum value to limit the number of candidate evidence graphs. When the number of candidate results greater than the limited value appears, the graph with a larger comprehensive weight is selected. Using this algorithm, several candidate evidence graphs can be obtained within a limited time complexity, and the answer node is likely to be hidden in the non-anchor nodes of the evidence graph.

[0127] (4) Training AGNN classifier

[0128] When training the model of the evidence graph, the output result of the evidence graph, i.e., the vector representation, needs to be as close to the problem itself as possible. Therefore, a standard value and the output result of the evidence graph are required to form a loss function. The present invention uses TextCNN in training to obtain a vector representation of a problem. Attention-based Graph Neural Network (AGNN) is a model with advanced effects on processing graphs proposed on the basis of GNN. It replaces the original fully connected layer with an attention mechanism. The attention mechanism allows each node in the graph to dynamically and adaptively learn the features of adjacent nodes during the propagation process, while also reducing many parameters to improve efficiency. AGNN learns which neighboring node is more relevant to the node during each iteration of the graph node, and measures its contribution accordingly.

[0129] For each evidence graph, all nodes and edges are extracted from it, where the edges are encoded in COO format, that is, two lists are set, the first list contains the index of the source node, and the index of the target node is specified in the second list. For the nodes, the vector obtained by training Wikidata is passed through a linear layer as the initial state of each node:

[0130]

[0131] Where W T is the mapping matrix of the linear layer, X i is the input vector, b is the offset parameter, is the node status in the neural network.

[0132] The present invention uses the attention-based propagation layer of AGNN to realize the intersection of information between nodes. Unlike the previous GNN, AGNN uses scalars to transfer nodes and adjacent nodes in each propagation intermediate layer:

[0133] H (t+1) =P (t) H (t)

[0134] Where P (t) It is an n*n matrix, where n represents the number of nodes. Each element in the matrix represents the transfer parameters between two nodes, and 0 is used to represent non-adjacent nodes.

[0135] The propagation of each layer is as follows:

[0136]

[0137]

[0138] Where cos represents the calculation of the cosine similarity of two vectors, and N(i) represents the neighboring nodes of node i in the graph.

[0139] In terms of output, the output vector of the anchor node in the model is passed through the previous TextCNN, and its representation in the hidden layer is used as the vector representation of the final graph. After obtaining the target evidence graph, the entity node after removing the anchor node in the evidence graph is selected as the candidate answer, and then the candidate answers are sorted by analyzing the node type of the candidate answer and the answer type of the question to obtain the answer.

[0140] During the training process of AGNN, a part of the data is used as the training set, and the remaining questions are used as the test set. During the training process, a positive evidence graph and a negative evidence graph are required for a question. Therefore, when extracting candidate evidence graphs from the document of each question, the non-anchor nodes of each candidate evidence graph are checked. If a node that is very close to the true answer appears, the evidence graph is positive, otherwise it is negative. In each round of training, the vector V output by each evidence graph is converted into g With the problem vector V q The cosine similarity of is used as the excitation parameter to calculate the features of the graph, and its loss function is set using MultiMarginLoss as:

[0141]

[0142] Where x is a set of all evidence graphs of a problem after the reward function, y is the distribution of positive graphs in set x, and i is the label distribution of the remaining negative graphs. By calculating the loss function, parameter updates can be implemented for both TextCNN and AGNN at the same time. The present invention uses Adam optimizer to train the model, and its learning rate is set to 0.01. In order to prevent overfitting of the model, a layer of Dropout is used in the middle of the model to actively forget certain dimensions of the vector, and the forgetting rate is set to 30%. Since the vector dimension of the node is 300, two linear layers are added to compress the vector.

[0143] The present invention provides auxiliary evidence of the answer while answering the question, thus supplementing the explainability of the question-answering system. Figure 3 This is a graph showing the accuracy of answering questions using different methods at different question complexities. The horizontal axis is the number of question entities, and the vertical axis is the accuracy rate. Figure 3The relationship curves between the number of question entities and the accuracy of target answers for QUEST (Question answering with Steiner Trees), DrQA (Document Retriever Question Answering), STAGG (Staged query graph generation), OQA (Open QA) and a question answering method in a multi-document scenario of the present invention are given. Figure 3 The relevant parameters of the accuracy of answering questions by different methods under different question complexities are shown in Table 1.

[0144] Table 1

[0145]

[0146] Among them, MRR is the sum of the ranking scores of the correct answers in all the answers to the question, Precision@1 is the accuracy of the first answer, and Hits@5 is the probability that the correct answer is included in the first 5 answers to the question.

[0147] Figure 4 FIG. 1 is a schematic diagram of the structure of a question-answering system in a multi-document scenario according to the present invention. Figure 4 As shown, a question answering system in a multi-document scenario includes:

[0148] The document feature vector determination and acquisition module 201 is used to encode the text information of each document to obtain the feature vector of each document.

[0149] The feature vector acquisition module 202 of the question to be answered is used to encode the question to be answered and obtain the feature vector of the question to be answered.

[0150] The target document determination module 203 is used to determine N target documents that are close to the question to be answered based on the feature vectors of each of the documents and the feature vectors of the question to be answered.

[0151] The SPO triple extraction module 204 is used to extract SPO triples from the N target documents.

[0152] The knowledge graph construction module 205 is used to construct a knowledge graph based on the SPO triples.

[0153] The evidence graph set determination module 206 is used to extract M subgraphs from the knowledge graph according to the similarity between the entities in the question to be answered and the nodes in the knowledge graph, and the M subgraphs constitute an evidence graph set.

[0154] The target evidence graph acquisition module 207 is used to acquire a target evidence graph from the evidence graph set using a classifier.

[0155] The target answer acquisition module 208 is used to obtain the target answer to the question to be answered according to the nodes of the target evidence graph.

[0156] The system also includes: using the target evidence graph as a source of explainability for the target answer.

[0157] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0158] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A question-answering method in a multi-document scenario, characterized in that: The method comprises: Encoding text information of multiple documents to obtain a feature vector of each document; Encode the question to be answered to obtain a feature vector of the question to be answered; Determine N target documents that are close to the question to be answered according to the feature vectors of each of the documents and the feature vectors of the question to be answered; the value of N is 5; Extracting SPO triples from the N target documents; Construct a knowledge graph based on the SPO triples; According to the similarity between the entity in the question to be answered and each node in the knowledge graph, M subgraphs are extracted from the knowledge graph, and the M subgraphs constitute a set of evidence graphs, specifically including: taking the target node in the knowledge graph corresponding to the keyword in the question to be answered as an anchor point, and extracting M subgraphs from the knowledge graph as a set of candidate evidence graphs; wherein the keyword is the word in the question to be answered after removing pronouns, modal particles and stop words; Obtaining a target evidence graph from the evidence graph set using a classifier; Obtaining a target answer to the question to be answered according to the nodes of the target evidence graph; The method specifically further includes: using data in the HEAD-QA dataset to train an attention-based graph neural network to obtain the classifier; Using the target evidence graph as a source of explainability for the target answer; The specific process of establishing a knowledge graph is as follows: S1: Open information extraction is used to extract SPO triples; S2: In the preprocessing process, the NLTK package is used to perform part-of-speech tagging and named entity recognition on the extracted articles. When processing SPO triple information, the named entities and the extracted phrases are respectively regarded as entity objects; S3: In the process of extracting SPO triples, the pairwise distances between the parts of the SPO triples are used as the confidence index of each SPO triple relationship; S4: The relationship between the predicate and the other two entities in the SPO triple is taken as an edge, and the predicate is taken as a separate node. That is, each SPO triple forms two entity nodes and one predicate node in the knowledge graph, and the confidence index is used as the weight of the corresponding node; S5: Calculate the similarity of two nodes through a semantic dictionary and merge entities with similarity above a threshold. The semantic dictionary includes the probability that the same text form points to different standard entities. S6: Build the processed SPO triples into a graph to form a knowledge graph; When establishing the original graph, the predicate in the SPO triple is not treated as an edge but as a separate node, that is, each SPO triple forms two entity nodes and a predicate node in the knowledge graph. If the S and O nodes already exist in the original graph, an edge will be extended on the basis of the original SPO triple to connect the new node. At the same time, the probability of each text form pointing to the same entity is calculated, and the two nodes whose similarity exceeds the threshold are connected through a bidirectional edge.

2. The question-answering method in a multi-document scenario according to claim 1, characterized in that: Before encoding the text information of the plurality of documents to obtain the feature vector of each of the documents, the method further comprises: Get semi-structured data of multiple documents; Semi-structured data of multiple documents are processed to obtain text information of the multiple documents.

3. The question-answering method in a multi-document scenario according to claim 1, characterized in that: The extracting SPO triples from the N target documents specifically includes: Perform part-of-speech tagging on the N target documents and extract phrases; Performing named entity recognition on the N target documents; The named entity and the phrase are taken as entity objects of the SPO triple.

4. The question-answering method in a multi-document scenario according to claim 1, characterized in that: The encoding of the text information of the plurality of documents to obtain the feature vector of each of the documents specifically includes: The text information of each of the documents is encoded using the TF-IDF method to obtain a feature vector of each of the documents.

5. The question-answering method in a multi-document scenario according to claim 4, characterized in that: The adopting the TF-IDF method to encode the text information of each document to obtain the feature vector of each document specifically includes: Eliminating stop words in the text information of each of the documents to obtain each processed document; Divide the frequency of each word appearing in the processed document by the total frequency of each word appearing in all documents to obtain the word frequency of each word in each processed document; The feature vector of each document is determined according to the word frequency of each word in each processed document.

6. The question-answering method in a multi-document scenario according to claim 1, characterized in that: The step of encoding the question to be answered to obtain the feature vector of the question to be answered specifically includes: The TF-IDF method is used to encode the question to be answered to obtain the feature vector of the question to be answered.

7. A question-answering system in a multi-document scenario, characterized in that: The system comprises: A document feature vector determination and acquisition module is used to encode text information of multiple documents to obtain a feature vector of each document; A feature vector acquisition module for the question to be answered, used to encode the question to be answered and obtain the feature vector of the question to be answered; A target document determination module, used to determine N target documents close to the question to be answered according to the feature vectors of each document and the feature vectors of the question to be answered; the value of N is 5; An SPO triple extraction module, used for extracting SPO triples from the N target documents; A knowledge graph construction module, used to construct a knowledge graph according to the SPO triples; The evidence graph set determination module is used to extract M subgraphs from the knowledge graph according to the similarity between the entity in the question to be answered and each node in the knowledge graph, and the M subgraphs constitute the evidence graph set, specifically including: taking the target node in the knowledge graph corresponding to the keyword in the question to be answered as an anchor point, and extracting M subgraphs from the knowledge graph as a set of candidate evidence graphs; wherein the keyword is the word in the question to be answered after removing pronouns, modal particles and stop words; A target evidence graph acquisition module, used to acquire a target evidence graph from the evidence graph set using a classifier; A target answer acquisition module, used to obtain a target answer to a question to be answered according to the nodes of the target evidence graph; Using the data in the HEAD-QA dataset to train an attention-based graph neural network to obtain the classifier; Using the target evidence graph as a source of explainability for the target answer; The specific process of establishing a knowledge graph is as follows: S1: Open information extraction is used to extract SPO triples; S2: In the preprocessing process, the NLTK package is used to perform part-of-speech tagging and named entity recognition on the extracted articles. When processing SPO triple information, the named entities and the extracted phrases are respectively regarded as entity objects; S3: In the process of extracting SPO triples, the pairwise distances between the parts of the SPO triples are used as the confidence index of each SPO triple relationship; S4: The relationship between the predicate and the other two entities in the SPO triple is taken as an edge, and the predicate is taken as a separate node. That is, each SPO triple forms two entity nodes and one predicate node in the knowledge graph, and the confidence index is used as the weight of the corresponding node; S5: Calculate the similarity of two nodes through a semantic dictionary and merge entities with similarity above a threshold. The semantic dictionary includes the probability that the same text form points to different standard entities. S6: Build the processed SPO triples into a graph to form a knowledge graph; When establishing the original graph, the predicate in the SPO triple is not treated as an edge but as a separate node, that is, each SPO triple forms two entity nodes and a predicate node in the knowledge graph. If the S and O nodes already exist in the original graph, an edge will be extended on the basis of the original SPO triple to connect the new node. At the same time, the probability of each text form pointing to the same entity is calculated, and the two nodes whose similarity exceeds the threshold are connected through a bidirectional edge.

Images