A question generation method for machine reading comprehension

Abstract

The invention relates to a method for generating questions for machine reading comprehension, the method comprising: using a constructed reading comprehension data set, including questions, paragraphs and answers; extracting triple knowledge related to the answers in the data set; using a seq2seq model, taking the paragraphs, answers and triple knowledge as input data of the model, obtaining a coding vector and a prediction result of question generation; using the prediction result of question generation to calculate a loss function for question generation; using an auxiliary classification model to predict the question word type of a target question according to the coding vector, obtaining the question word type and the prediction result of the auxiliary classification model; using the question word type and the prediction result of the auxiliary classification model to calculate the loss function of the auxiliary classification model; combining two loss functions to obtain a total loss function, using the total loss function to iteratively train the seq2seq model and the auxiliary classification model to obtain a trained seq2seq model; and using the trained seq2seq model to generate questions.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and in particular to a method for generating questions for machine reading comprehension. Background Art

[0002] In recent years, with the rapid development of artificial intelligence technology, research related to natural language generation has received more and more attention. Among them, question generation is an important research direction of natural language generation. The current research on question generation mainly focuses on the generation of questions with answers. According to the given context information and the specified answer information, the machine can understand the given context information and generate questions for the specified answer. Question generation technology has a wide range of applications in academia and industry. On the one hand, question generation can be used as a data enhancement technology to generate training data for question-answering systems and reading comprehension tasks. Studies have shown that using automatically generated questions to supplement data, or training both answer extraction and question generation tasks at the same time can significantly improve the performance of question-answering systems. On the other hand, question generation can simulate user questions in the field of education, guide machines to actively ask questions in the field of dialogue, etc. Embedding a question generation model in a dialogue system can strengthen the interaction between the system and humans, help the system to more accurately judge human intentions, and thus make human-computer dialogue more fluent.

[0003] Early research on question generation was mainly based on rules and templates. This method defines some heuristic rules on the syntax tree to convert a sentence into a question sentence. The advantage of this method is that it can ensure the relevance and fluency of the generated questions, but this type of method strictly relies on manually set conversion and generation rules. At the same time, the diversity of generated questions is largely limited, and the portability is poor. With the emergence of some large-scale, high-quality machine reading comprehension datasets, such as SQuAD and MARCO, question generation based on neural networks has made great progress. In subsequent research, question generation based on neural networks has become the mainstream research method, and a large number of high-quality research has emerged. At present, the construction of datasets for reading comprehension mainly adopts manual methods, which requires a lot of manpower and material resources and has high labor costs.

[0004] The question generation task requires understanding the context and generating appropriate questions based on the answers. The accuracy of the question words in the question directly determines the accuracy of the question. Therefore, the accuracy of the question words in the question is crucial. A statistical analysis of the accuracy of the questions generated by the current question generation model found that more than half of the question words in the generated questions did not match the question type, which seriously affected the effect of question generation. At the same time, the generated questions also have the problem of not being closely related to the original text and the answer. Summary of the invention

[0005] The purpose of the present invention is to provide a method for generating questions for machine reading comprehension, aiming to solve the problem of lack of reading comprehension data sets, and reduce labor costs by effectively utilizing deep learning methods and techniques. In view of the problem that more than half of the question words in the generated questions do not match the question type, the present invention proposes to classify the input paragraphs and answers, and classify the question types according to the categories of question words. In view of the problem that the generated questions have little relevance to the original text and the answer, the present invention extracts the triple knowledge in the data set according to the given answer, and fuses the triple knowledge with the original input and inputs them into the encoder together to obtain a contextual representation with answer awareness, and generates questions related to the answer and the original text through the decoder.

[0006] The technical solution adopted by the present invention to solve the above technical problems is to provide a method for generating questions for machine reading comprehension, the method comprising:

[0007] Use an already constructed reading comprehension dataset, including questions, passages, and answers;

[0008] Extracting triple knowledge related to the answer in the data set;

[0009] Using a seq2seq model, taking the paragraph, answer, and triple knowledge as input data of the model, encoding and decoding the input data, and obtaining a coding vector and a prediction result of question generation;

[0010] Using the question as a label, and using the prediction result generated by the question to calculate the loss function generated by the question;

[0011] Using the auxiliary classification model to predict the interrogative word type of the target question according to the encoding vector, and obtaining the interrogative word type and the prediction result of the auxiliary classification model;

[0012] Taking the question as a label, using the question word type and the prediction result of the auxiliary classification model to calculate the loss function of the auxiliary classification model;

[0013] The loss function generated by the problem is combined with the loss function of the auxiliary classification model to obtain a total loss function, and the seq2seq model and the auxiliary classification model are iteratively trained using the total loss function to obtain a trained seq2seq model;

[0014] The trained seq2seq model is used for question generation.

[0015] In a possible implementation, the seq2seq model uses a bidirectional LSTM based on a self-attention mechanism as an encoder and an LSTM based on an attention mechanism as a decoder.

[0016] In a possible implementation, the auxiliary classification model is a BERT classification model, and a cross entropy loss function is used to calculate the loss function generated by the problem and the loss function of the auxiliary classification model.

[0017] The present invention adopts seq2seq architecture to automatically construct a reading comprehension dataset, generates questions for the answers based on given text and answers, and designs an auxiliary task of question word classification to classify question types in order to improve the accuracy of question words in generated questions in order to improve the accuracy of question words in generated questions. In addition, the triple knowledge in paragraphs and answers is extracted and input into the model together, thereby providing the model with more information to improve the quality of generated questions. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 The present invention discloses a flowchart of a method for generating questions for machine reading comprehension. DETAILED DESCRIPTION

[0019] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments.

[0020] The task definition of question generation based on the seq2seq architecture is: given a paragraph D and an answer A, the model generates a question This question can be answered based on the given paragraph and answer, as shown in formula (1):

[0021]

[0022] Figure 1 The present invention discloses a flowchart of a method for generating questions for machine reading comprehension.

[0023] In step S101, the training set data required for training the model is obtained, including questions, paragraphs and answers.

[0024] In step S102, the triplet knowledge related to the answer in the data set is t ,r,e> to extract and obtain the answer A t Related keywords r and e, among which A t is entity one, r is the entity relationship, and e is entity two. Keywords r and e are used to assist the model in generating questions related to the answer. By extracting triplet knowledge related to the answer in the original paragraph, the relevance of the generated question and the answer can be improved.

[0025] ​In step S103, using the seq2seq model, the encoder uses a bidirectional LSTM to input paragraphs, answers, and triples of knowledge. t ,r,e> is encoded as shown in formulas (2)-(3):

[0026] h t =LSTM(h t-1 ,[D t ,A t , t ,r,e>]) (2)

[0027]

[0028] Among them, h t-1 is the hidden state at time step t-1, and They represent the encoding from left to right and from right to left respectively, and H is the encoding vector after bidirectional LSTM encoding.

[0029] The encoded information is processed using the self-attention mechanism, which allows the model to dynamically assign weights to different information, as shown in formulas (4)-(6):

[0030] f(H,h t )=H T W a h t (4)

[0031] α i =softmax(f(H,h t )) (5)

[0032]

[0033] Among them, W a is the learnable parameter matrix, α i That is the weight coefficient corresponding to the word, and then the weighted sum is performed to get the attention value a t .

[0034] Then decode it. First, after LSTM decoding, y t-1 is the output result of LSTM at the previous moment, s t is the hidden state at time t, W u is a learnable parameter matrix. Then the attention mechanism is used to process the decoded information and calculate the attention score c i , normalize it to get the attention value u t , as shown in formulas (7)-(10):

[0035] s​​t =LSTM(s t-1 ,y t-1 ) (7)

[0036]

[0037] c i =softmax(α t ) (9)

[0038]

[0039] In step S104, the probability P of generating each word is calculated using the cross entropy loss function. QG , using the question as the label, calculate the loss function for question generation, as shown in formula (11), where y <t is the output result before time t, y t The predictions generated for the question.

[0040]

[0041] In steps S105-S106, the auxiliary classification model is used to predict the interrogative word type of the target question based on the hidden layer state during the encoding process, and then the prediction will also use the attention-based decoder to guide question generation. Using the question as the label, the loss function L of the auxiliary classification model of the classification model is calculated using the cross entropy loss function c , C is the classification result, y i is the prediction result of the auxiliary classification model, and the calculation formula is shown in (12)-(13):

[0042] C = softmax(W c H) (12)

[0043]

[0044] In step S107, the loss function generated by the problem is combined with the loss function of the auxiliary classification model to obtain the total loss function L all , as shown in formula (14).

[0045] L all =L QG +αL c (14)

[0046] The seq2seq model and the auxiliary classification model are iteratively trained using the total loss function to obtain a trained model.

[0047] In step S108, the trained seq2seq model is used to generate questions.

[0048] At the same time, the classification accuracy of question types also has a positive impact on the quality of questions, so choosing a classification model with good performance is also crucial. The present invention uses the BERT classification model with the highest accuracy as the auxiliary classification model for auxiliary tasks to help the question generation model generate questions with accurate question words.

[0049] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for generating questions for machine reading comprehension, characterized in that: The method comprises: Use an already constructed reading comprehension dataset, including questions, passages, and answers; Extracting triple knowledge related to the answer in the data set; extracting triple knowledge related to the answer in the paragraph; Using a seq2seq model, taking the paragraph, answer, and triple knowledge as input data of the model, encoding and decoding the input data, and obtaining a coding vector and a prediction result of question generation; Using the question as a label, and using the prediction result generated by the question to calculate the loss function generated by the question; Using the auxiliary classification model to predict the interrogative word type of the target question according to the encoding vector, and obtaining the interrogative word type and the prediction result of the auxiliary classification model; Taking the question as a label, using the question word type and the prediction result of the auxiliary classification model to calculate the loss function of the auxiliary classification model; The loss function generated by the problem is combined with the loss function of the auxiliary classification model to obtain a total loss function, and the seq2seq model and the auxiliary classification model are iteratively trained using the total loss function to obtain a trained seq2seq model; Use the trained seq2seq model to generate questions; The seq2seq model uses a bidirectional LSTM based on a self-attention mechanism as an encoder; The method to get the encoding vector is as follows: Using the seq2seq model, the encoder uses a bidirectional LSTM to encode the input paragraph, answer, and triple knowledge to obtain the encoded vector.

2. The method according to claim 1, characterized in that The seq2seq model uses LSTM based on attention mechanism as decoder.

3. The method according to claim 1, characterized in that The auxiliary classification model is a BERT classification model, and a cross entropy loss function is used to calculate the loss function generated by the problem and the loss function of the auxiliary classification model.

Images