A method for accurate question retrieval based on deep semantics

Abstract

The present invention discloses a method for accurate question retrieval based on deep semantics. First, a pre-trained text representation model is used to extract text representation vectors of questions in a knowledge base. Then, the extracted vectors are normalized and persisted to disk, and five similarity features for accurate calculation are calculated. These five similarity features are used as inputs of a classification-discrimination model to accurately judge candidate questions, and finally the answers judged as True are returned to the user. The five external similarity features of the present invention can well capture the detailed features of sentences, and to a certain extent improve the accuracy of accurate retrieval and recognition of user questions; in addition, the classification-discrimination model can predict or judge the question status of candidate questions and return the only correct answer to the user, solving the defect of previous question-answering systems that answer questions irrelevantly.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing and intelligent question answering, and relates to a method for accurate retrieval of deep semantic questions. Background Art

[0002] With the rapid development of deep learning in the field of natural language processing, question-answering systems are gradually becoming intelligent. Whether the question-answering system can quickly and accurately answer user questions and bring convenience to users' lives is an urgent problem to be solved. In vertical field question-answering, the traditional manual method wastes a lot of manpower and material resources to solve user problems, which has limitations. That is, if the user's question is asked in a different way, the returned answer is wrong. The question-answering system needs to quickly and accurately retrieve the answer to the question that is most similar to the user's question from the question-answering data. The existing vertical field question-answering system uses methods based on rule template matching, statistical learning, machine learning and deep learning to solve the question-answering problem. The method based on rule template matching is suitable for building a vertical localized question-answering model, which satisfies the user's questions in the vertical field or a specific range, but it requires manual formulation of rules. The disadvantage is that it is time-consuming and the model cannot be well expanded. The statistical or machine learning-based method has good interpretability, and the features that rely on manual search are more accurate. The key information of the problem can be located during retrieval, making the retrieval results more accurate, but the generalization ability is general and the effect of the model depends largely on the selection of features. The deep learning method based on neural network has strong generalization ability and can capture the semantic relationship between questions and knowledge base questions. However, the existing pre-trained text representation model has anisotropy in the representation of sentences and it is difficult to reflect the semantic similarity between two sentences. Summary of the invention

[0003] The purpose of the present invention is to provide a deep semantic question accurate retrieval method, which integrates the characteristics and representations of questions and knowledge base questions, uses a deep learning-based method to increase the model's generalized retrieval capability, constructs five similarity features by analyzing the semantic relationship between knowledge base questions, and uses a statistical or machine learning-based method to increase the model's accurate retrieval capability, thereby improving the comprehensive performance of the question-answering model.

[0004] The technical solution adopted by the present invention is a method for accurate retrieval of deep semantic questions, which is specifically implemented according to the following steps:

[0005] Step 1: Text semantic feature comparison. In the text semantic feature comparison stage, the question and answer pairs stored in the case base CB (Case Base) are classified and merged to form a knowledge base KB (Knowledge Base). CB is composed of question and answer pairs collected from cultural and tourism institutions, which contains 3217 question and answer pairs. KB is manually inspected and labeled for question and answer pairs. Each data consists of four attributes: major category number, minor category number, question, and answer, with a total of 3217 data. The knowledge base questions are converted into text representation vectors through the pre-trained text representation model, and the output results are persisted to the disk, so that the questions asked by users can be calculated for similarity with the knowledge base questions;

[0006] Step 2: Accurately retrieve feature calculation, use the representation vector of the user's question to retrieve the most relevant k (k>0) questions in the knowledge base, this step is called top-k candidates. Since users usually ask short questions, the questions contain less effective information, resulting in sparse sample features and high dimensionality of feature sets, making it difficult to extract key and accurate sample features from user questions for retrieval of similar questions. Therefore, in view of this characteristic of short text, in the precise retrieval feature calculation stage, based on the top-k candidate question-answer pairs, five external similarity features can be constructed for the short text questions to be retrieved, namely, the text representation similarity Deep_Score between the user question and the top-k candidate question, the feature similarity Sim(Q,KBQ) between the user question and the top-k candidate question, the entity feature similarity Sim(Q,KBQE) between the user question and the top-k candidate question, the answer feature similarity Sim(Q,KBA) between the user question and the top-k candidate question, and the intention feature similarity Sim(Q,KBQR) between the user question and the top-k candidate question.

[0007] Step 3: Determine the status of candidate questions. By obtaining accurate retrieval features from step 2, we designed a classification-discrimination model to determine whether the question can be answered. The classification-discrimination model weights the five similarity features of step 2, downplays unimportant features, and strengthens the main features of the question. The classification-discrimination model outputs True, indicating that the candidate question is similar to the question to be retrieved, and its answer can be used as the answer to the question to be retrieved. The output of False indicates that the candidate question is not similar to the question to be retrieved, and its answer cannot be used as the answer to the question to be retrieved.

[0008] The present invention is also characterized in that:

[0009] Step 1 is implemented according to the following steps:

[0010] Step 1.1, extracting text representation information using a pre-trained text representation model;

[0011] Step 1.2: Use vector processing tools to persist the vectors and calculate the vector similarity.

[0012] Specifically, step 1.1 extracts representation information from knowledge base questions by inputting question sentences in the knowledge base into the pre-trained text representation model.

[0013] Step 1.2 is implemented as follows:

[0014] Step 1.2.1: Get the knowledge base question representation vector through step 1.1, normalize the representation vector, and persist the processed representation vector set to the disk to provide support for subsequent calculations.

[0015] Step 1.2.2: Calculate the similarity between the representation vector obtained in step 1.2.1 and the knowledge base question representation vector, and then efficiently sort the calculation results.

[0016] Step 2 is implemented as follows:

[0017] Step 2.1: Based on the sorting results in step 1, we select the first k (k>0) questions as the top-k candidate set of the current user question.

[0018] Step 2.2: According to step 1.2.2, the similarity between the user question representation vector and the knowledge base question representation vector is used as the deep semantic Deep_Score. where e u , e k Represent the user question representation vector and the knowledge base question representation vector respectively

[0019] Step 2.3, calculate the similarity Sim(Q,KBQ) between the user question and the top-k candidate question features, Sim(Q,KBQ) = max(Q∩KBQ), where Q represents the user question semantic block feature, KBQ represents the candidate question semantic block feature. We use the maximum observed value of Q and KBQ to represent the correlation between the user question and the knowledge base question.

[0020] Step 2.4, calculate the similarity Sim(Q,KBQE) between the user question and the top-k candidate question entity features, Sim(Q,KBQE) = max(Q∩KBQE), where KBQE represents the candidate question entity features obtained using the named entity extraction algorithm. This feature is introduced to calculate the similarity between the question and the top-k candidate question entity features. This feature is a further correction to the result of step 2.3 to solve the problem that the semantic block cannot maximize the representation of the core entity features.

[0021] Step 2.5, calculate the similarity Sim(Q,KBA) between the user question and the top-k candidate question answer features, Sim(Q,KBA) = max(Q∩KBA), where KBA represents the answer semantic block feature of the candidate question. This feature is introduced as a bias feature. Generally speaking, the relationship between the question and the answer is very weak. Introducing this weak feature as a bias feature helps improve the performance of precise retrieval feature calculation.

[0022] Step 2.6, calculate the similarity Sim(Q,KBQR) between the user question and the top-k candidate question intent features, Sim(Q,KBQR) = max(Q∩KBQR), where KBQR represents the candidate question intent feature obtained using the text intent extraction algorithm. The main function of this feature is to judge the user intent and the strength of the intent. Using the intent feature can effectively match the user question with the top-k candidate question intent.

[0023] Step 3 is implemented as follows:

[0024] Step 3.1. In step 2, we constructed the feature values ​​of five dimensions. We used the logistic regression model to model the features of the above five dimensions and obtained a weighted vector group V with a dimension of 5×m (where m≥1). V represents the contribution matrix of the knowledge base candidate questions on the user questions. Through the linear normalization calculation of the user input question and the contribution matrix V, we can obtain the detailed characteristics of the user question.

[0025] Step 3.2: Based on the design of step 3.1, use a nonlinear threshold function to optimize the downstream discrimination task of the overall question answering. We need to discriminate the questions based on the classification, and mark the positive questions as True and the negative questions as False. Its training combines the feature modeling in step 3.1 with the nonlinear activation threshold function to improve the accuracy of the question answering system. We call it a classification-discrimination model, and its training principle is I(V; w, α) = T(soft max(wV+b 1 )α+b 2 ), where V is the input, w, α are the parameters to be learned by the model, and b 1 ,b 2 is introduced as a bias term.

[0026] The beneficial effects of the present invention are as follows: the present invention provides a method for accurate question retrieval based on deep semantics. First, the text representation vectors of the questions asked by the user and the questions in the knowledge base are obtained through a pre-trained text representation model with relatively excellent performance. By calculating the similarity between the text representation vectors of the questions in the knowledge base and the user's questions, the retrieval range is narrowed for accurate calculation. Then, five external similarity features are constructed for the user's questions to assist in accurate question retrieval. Based on the five external similarity features obtained, the detailed features of the sentence can be well captured, which improves the accuracy of accurate retrieval and recognition of user questions to a certain extent; in addition, the classification-discrimination model can predict or judge the question status of the candidate question and return the only correct answer to the user, which solves the defect of the previous question-answering system that answers the wrong question. The accurate retrieval based on deep semantics can be derived to other practical applications of intelligent question-answering systems in vertical fields, such as intelligent question-answering assistants in the medical field, intelligent question-answering assistants in e-commerce, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a flow chart of a method for accurate retrieval of deep semantic questions of the present invention. DETAILED DESCRIPTION

[0028] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] The present invention provides a method for accurately retrieving deep semantic questions, such as Figure 1 As shown in the figure, the text representation vector of the questions in the knowledge base is first extracted using the pre-trained text representation model. The model outputs a vector of 768 dimensions. The extracted vector is then normalized and persisted to disk. Then, five similarity features for accurate calculation are calculated, namely, the similarity between the text representation of the user question and the top-k candidate question, Deep_Score, the feature similarity between the user question and the top-k candidate question, Sim(Q,KBQ), the entity feature similarity between the user question and the top-k candidate question, Sim(Q,KBQE), the answer feature similarity between the user question and the top-k candidate question, Sim(Q,KBA), and the intention feature similarity between the user question and the top-k candidate question, Sim(Q,KBQR). These five similarity features are used as the input of the classification-discrimination model to accurately judge the candidate questions, and finally the answer judged as True is returned to the user.

[0030] Example

[0031] A method for accurate retrieval of deep semantic questions is implemented in the following steps:

[0032] Step 1: mainly completes the text semantic feature extraction. In this stage, the question and answer data pairs stored in the case base CB (Case Base) are sorted and merged to form the knowledge base KB (Knowledge Base). The knowledge base questions are converted into text representation vectors through the pre-trained text representation model, so that the user questions can be similar to the knowledge base questions. The specific steps are as follows;

[0033] (1) Build a knowledge base KB (Knowledge Base). On the one hand, the knowledge base is used to train the pre-trained text representation model, and on the other hand, the knowledge base serves to retrieve related questions.

[0034] (2) Fine-tune using question-answer pairs in the knowledge base

[0035] (3) Training the text representation model: We use the fine-tuned BERT model as the pre-trained text representation model. The input data shape is [batch_size, question], and the output of the model is a representation vector result of [batch_size, 768].

[0036] (4) Use the vector processing tool Faiss to persist the output vector to disk, providing fast and efficient calculation for subsequent retrieval.

[0037] Step 2: Accurate retrieval feature calculation. With the help of the vector processing tool in step 1, assuming that the user's question is "Do students get discounts when buying tickets?", the k candidate questions most relevant to this question can be retrieved from the knowledge base, such as "Preferential policies for students?", "Do teachers get discounts when buying tickets?", "Are there discounts for scenic spot tickets?", etc. The user question text representation vector retrieves the most relevant k questions in the knowledge base, and this step is called top-k candidates. Due to the unique question structure of the user's question, it contains less effective information, resulting in sparse sample features and high dimensionality of the feature set, making it difficult to extract key and accurate sample features from the user's question for retrieval of similar questions. For example, "Are there any discounts for students buying tickets?" and "Are there any discounts for teachers buying tickets?" have strong similarities in sentence structure and intent, and the related entities "teacher" and "student" also belong to the same category of words. The pre-trained text representation model is difficult to distinguish the differences between this category of words. Therefore, five external features can be constructed for the short text questions to be retrieved, namely, the text representation similarity Deep_Score between the user question and the top-k candidate question, the feature similarity Sim(Q,KBQ) between the user question and the top-k candidate question, the entity feature similarity Sim(Q,KBQE) between the user question and the top-k candidate question, the answer feature similarity Sim(Q,KBA) between the user question and the top-k candidate question, and the intention feature similarity Sim(Q,KBQR) between the user question and the top-k candidate question. The specific steps to complete are as follows:

[0038] (1) Based on the sorting results in step 1, we select the first k questions as the top-k candidate sets of the current user question. When the user question q is "Are there any discounts for students buying tickets?", the top-3 candidate sets can be query 1 : "Are there any discounts for teachers when buying tickets?", query 2 : "Preferential policies for students?", query 3 : “Are there any discounts on scenic spot tickets?

[0039] (2) The similarity between the user question representation vector and the knowledge base question representation is used as the deep semantic Deep_Score. i∈[1,k], where B(q) and B(query i ) is the text representation vector obtained by the pre-trained text representation model B for the user question q and the i-th question in the candidate case library.

[0040] (3) The feature similarity between the user question q and the top-k candidate questions is Sim(Q,KBQ). The semantic block extraction algorithm f(x) is used to extract the semantic blocks in the question. Q represents the relevant features of the user question, specifically, Q = f(q), KBQ = f(query), KBQ represents the semantic block features of the candidate question, and then the maximum correlation feature between Q and KBQ is calculated Sim(Q,KBQ) = max(f(q) ∩f(query)).

[0041] (4) The similarity between the entity features of the user question q and the top-k candidate questions is Sim(Q,KBQE), where KBQE is the entity feature of the candidate question obtained using the named entity extraction algorithm E(x). This feature is introduced to calculate the similarity between the entity features of the question and the top-k candidate questions. In particular, Q = E(q), where E(q) corresponds to the student, and E(query 1 ) corresponds to the teacher, E(query 2 ) corresponds to students, E(query 3 ) corresponds to the scenic spot, then Sim(Q,KBQE)=max(E(q)∩E(query 2 )).

[0042] (5) The similarity between the user question q and the top-k candidate question answer features is Sim(Q, KBA), specifically, Q = f(q), KBA = f(answer) represents the answer semantic block features of the candidate question, and finally Sim(Q, KBA) = max(f(q) ∩f(answer)).

[0043] (6) The similarity between the user question q and the top-k candidate question entity features is Sim(Q,KBQR), where KBQR represents the candidate question entity features obtained by the intent extraction algorithm R(x). In particular, Q = R(q), where R(q) corresponds to buying tickets, and R(query 1 ) corresponds to buying a ticket, R(query 2 ) corresponds to buying a ticket, R(query 3 ) corresponds to buying a ticket, through the intention information in the question Sim(Q,KBQR)=max(R(q)∩R(query 1 )), the intent feature can be used to effectively match the intent of user questions with the top-k candidate questions.

[0044] Step 3: Determine the status of candidate questions. Through the above steps, we have obtained the five structural features involved in the method. We designed a classification-discrimination model to determine whether the question can be answered. The classification-discrimination model weights the five retrieval features in step 2. The classification-discrimination model outputs True, which means that the candidate question is similar to the question to be retrieved, and its answer can be used as the answer to the question to be retrieved. The output False means that the candidate question is not similar to the question to be retrieved. The specific steps to complete step 3 are as follows.

[0045] (1) In step 2, we constructed the feature values ​​of the five dimensions. We used the logistic regression model to classify and discriminate the features of the five dimensions and obtained a set of weighted vectors V = {v 1 , v 2 ,v 3 ,v 4 ,v 5}, whose dimension is 5×m (where m≥1). The preliminary classification task is completed by linear normalization calculation of user input question q and V. In particular, user question q and knowledge base question query are calculated through similarity features to form V matrix, which is described in detail in step 2.

[0046] (2) Based on the above design, a nonlinear threshold function is used to optimize the downstream discrimination task of the overall question-answering. We need to discriminate the questions based on feature extraction. The cases that can answer the user's questions are marked as True, and the algorithm returns the answers corresponding to the case questions to the user. On the contrary, the counterexample questions are marked as False. Therefore, the threshold correlation coefficient α is introduced to more accurately identify the answer status of the question. We call this whole a classification-discrimination model, which is a fusion of classification feature extraction and discrimination algorithm. Its training principle is I(V; w, α) = T(soft max(wV+b 1 )α+b 2 ), where V is the input, w, α, is the learning parameter of the model, and b 1 ,b 2 is the bias term.

Claims

1. A method for accurate question retrieval based on deep semantics, characterized in that: Follow the steps below to implement it: Step 1: Text semantic feature comparison In the text semantic feature comparison stage, the collected user question and answer data pairs stored in the case base CB are sorted and merged to form a knowledge base KB; the knowledge base questions are converted into text representation vectors through the pre-trained text representation model, and the output results are persisted to the disk, so that the user questions can be similar to the knowledge base questions. Step 2: Accurate retrieval feature calculation Use the text representation vector to retrieve the most relevant k questions in the case base CB, k>1, this step is called top-k candidates; for short texts, in the precise retrieval feature calculation stage, based on the top-k candidate question-answer pairs, five similarity features are constructed for the short text questions to be retrieved, namely, the text representation similarity Deep_Score between the user question and the top-k candidate question, the feature similarity Sim(Q,KBQ) between the user question and the top-k candidate question, the entity feature similarity Sim(Q,KBQE) between the user question and the top-k candidate question, the answer feature similarity Sim(Q,KBA) between the user question and the top-k candidate question, and the intention feature similarity Sim(Q,KBQR) between the user question and the top-k candidate question; The step 2 is specifically implemented according to the following steps: Step 2.1, calculate the feature similarity Sim(Q,KBQ) between the user question and the top-k candidate question, where Q represents the semantic block feature of the user question, KBQ represents the semantic block feature of the candidate question, and the maximum observed value of Q and KBQ is taken as the feature similarity Sim(Q,KBQ), that is, Sim(Q,KBQ) = max(Q∩KBQ); Step 2.2, calculate the similarity Sim(Q,KBQE) between the user question and the top-k candidate question entity features, Sim(Q,KBQE) = max(Q∩KBQE), where KBQE represents the candidate question entity features obtained using the named entity extraction algorithm; Step 2.3, calculate the similarity between the user question and the top-k candidate question answer features Sim(Q, KBA), Sim(Q, KBA) = max(Q∩KBA), where KBA represents the answer semantic block feature of the candidate question; Step 2.4, calculate the similarity Sim(Q,KBQR) between the user question and the top-k candidate question intent features, Sim(Q,KBQR) = max(Q∩KBQR), where KBQR represents the candidate question intent features obtained using the text intent extraction algorithm; Step 2.5: Calculate the similarity between the user question and the top-k candidate question text representation Deep_Score where e u , e k Represent the user question text representation vector and the knowledge base question text representation vector respectively; The user question semantic block feature Q uses a semantic block extraction algorithm f(q) to extract the semantic block in the question; Step 3: Determine the status of candidate questions A classification-discrimination model is constructed based on the precise retrieval feature similarity obtained in step 2. The classification-discrimination model is used to determine whether the question can be answered. The classification-discrimination model models the five similarity features in step 2 and strengthens the main features of the question. If the classification-discrimination model outputs True, it means that the candidate question is similar to the user's question, and its answer can be returned as the final answer. If it outputs False, it means that the candidate question is not similar to the user's question, and its answer cannot be returned as the final answer.

2. According to the method of claim 1, the method is characterized in that: Step 1 is as follows: Step 1.1, input the case library data into the pre-trained text representation model to extract the feature vector of the text; Step 1.2: Use vector processing tools to persist feature vectors; details are as follows: Step 1.2.1, obtain the feature vector through step 1.1, normalize the feature vector, and use the vector processing tool Faiss to persist the processed text representation vector set to the disk; Step 1.2.2: Then, the similarity between the user question text representation vector and the knowledge base representation vector is calculated, and the calculation results are efficiently sorted.

3. The method for accurate question retrieval based on deep semantics according to claim 1, characterized in that: The step 3 is specifically as follows: Step 3.1: In step 2, the feature similarity values ​​of the five dimensions are constructed, and the features of the five dimensions are modeled using a logistic regression model to obtain a matrix V with a dimension of 5×m, where m≥1; V represents the contribution matrix of the candidate questions in the knowledge base to the user questions, and the detailed features of the user questions are obtained by linear normalization calculation of the user input questions and the contribution matrix V; Step 3.2: Based on the design of step 3.1, a nonlinear threshold function is used to optimize the classification-discrimination model. The classification-discrimination model outputs True, which means that the candidate question is similar to the question to be retrieved, and its answer can be used as the answer to the question to be retrieved. The output of False means that the candidate question is not similar to the question to be retrieved. The algorithm returns the first question marked as True to the user.

4. The method for accurate question retrieval based on deep semantics according to claim 1, characterized in that: The classification-discrimination model in step 3 is a model that integrates classification feature extraction and discrimination algorithm. The training principle of the classification-discrimination model is: I(V; w, α) = T(softmax(wV+b1)α+b2). Where V is the input, w, α, are the learning parameters of the model, and b1, b2 are bias terms.

5. The method for accurate question retrieval based on deep semantics according to claim 1, characterized in that: Step 1: The case library CB is composed of question-answer pairs collected from cultural and tourism institutions, including 3,217 question-answer pairs; the knowledge base KB is manually inspected and labeled by the question-answer pairs, and each piece of data consists of four attributes: major category number, minor category number, question, and answer, with a total of 3,217 pieces of data.

Images