A self-training method for machine reading comprehension models for refusal questions

Abstract

The present invention belongs to the field of natural language processing, and discloses a self-training method for a reading comprehension model for refusal to answer questions, including the following steps: S1. Constructing a machine reading comprehension model for refusal to answer questions, including a text encoding module and an answer prediction module; S2. Inputting text and questions into the text encoding module for encoding, and then inputting them into the answer prediction module for processing; training and adjusting the parameters of the machine reading comprehension model with the goal of minimizing the training loss function to obtain an initial machine reading comprehension model; S3. Iteratively training the initial machine reading comprehension model on an unlabeled data set using a self-training method to obtain an ideal machine reading comprehension model. The verification function introduced in the present invention detects the prediction effect of the machine reading comprehension model, and the model training can be completed well using unlabeled data, obtaining better results than traditional training methods.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing machine reading comprehension, and in particular relates to a self-training method for a reading comprehension model for refusal to answer questions. Background Art

[0002] Machine reading comprehension aims to teach machines to answer questions after understanding a given paragraph, which has long been the basic goal of natural language understanding (NLU). Early MRC (machine reading comprehension) systems were designed on the underlying assumption that all questions can be answered based on the given text. However, when the question cannot be answered under the current given text, the system needs to be able to refuse to give an answer. This requires that the machine reading comprehension model must be able to distinguish those unanswerable questions to avoid giving credible answers. With the widespread application of natural language processing technology in the judicial field, applications such as legal judgment prediction, legal intelligent question answering, and case label prediction have made good progress. In addition, in the field of tourism, the combination of artificial intelligence technology and user behavior data has also spawned a broad application demand, such as personalized travel guides, tourism public opinion analysis, and other applications.

[0003] Pre-trained models have been a hot topic in the field of natural language processing in recent years. Thanks to its rapid development, more and more deep learning models combined with pre-trained models have achieved good performance in reading comprehension tasks. Traditional rule-based models are not only time-consuming, but also require specific feature engineering for the task. The deep pre-trained model obtained through a large amount of corpus training can replace the role of feature engineering and through the pre-training of massive corpus, the model has a stronger generalization ability for downstream tasks. However, there is not enough labeled data to train the model for specific fields such as law and finance, and the potential of deep pre-trained models has not been brought into play. In traditional text data enhancement methods, synonym replacement technology has very similar word vectors, so two sentences will be treated as almost the same sentences during model training, and the data set is not actually effectively expanded. Back-translation methods can often increase the diversity of text data. Compared with replacing words, they can sometimes change syntactic structures and retain semantic information. However, the data generated by the back-translation method depends on the quality of the translation, and most of the translation results may not be so accurate. If you use the interface of some translation software, you may also encounter account restrictions.

[0004] The present invention proposes a self-training method for a machine reading comprehension model for refusal to answer questions, which belongs to deep learning data enhancement technology. First, we build a machine reading comprehension model based on a pre-trained language model. The model is trained with existing annotated data to obtain an initial machine reading comprehension model. Afterwards, we use the initial pre-trained model to predict the unlabeled data, and use a soft labeling method to process the predicted value to obtain pseudo-annotated data to expand the annotated training data set. Further training of the model improves the accuracy of the model. Summary of the invention

[0005] The purpose of the present invention is to provide a self-training method for a reading comprehension model for refusal questions, so that the trained model can understand the refusal questions more accurately.

[0006] A self-training method for a reading comprehension model for refusal to answer questions, comprising the following steps:

[0007] S1. Build a machine reading comprehension model for refusal questions, including a text encoding module and an answer prediction module;

[0008] S2. Input the text and question into the text encoding module for encoding, and then input into the answer prediction module for processing; train and adjust the parameters of the machine reading comprehension model with the goal of minimizing the training loss function to obtain an initial machine reading comprehension model;

[0009] S3. The initial machine reading comprehension model is iteratively trained on an unlabeled dataset using a self-training method to obtain an ideal machine reading comprehension model.

[0010] Furthermore, the steps of inputting the text and question into the text encoding module for encoding are as follows:

[0011] S21. Select a preset number of label samples as a label training set;

[0012] S22. Concatenate the question and document data in the label training set to form "question-document" text data, and use it as the input of the tokenizer to obtain a text token sequence;

[0013] S23. Use the BERT pre-trained model to process the text token sequence to obtain the contextual representation vector x∈R of the text data b×c×d , where b is the amount of input text data, c is the length of the text, and d is the dimension of the encoding.

[0014] Furthermore, according to the design of the BERT pre-training model, the first value of the text token sequence is the [CLS] tag value, and the representation vector h corresponding to the [CLS] tag value in the context representation vector is used.[cls] ∈R b×d Used as the input of the subsequent answer prediction module, where b is the number of input text data and d is the dimension of the encoding.

[0015] Furthermore, the answer prediction module processing method is implemented as follows:

[0016] S24. The representation vector h corresponding to the [CLS] tag value obtained by the text encoding module [cls] After overfitting using dropout, the final answer probability is recorded as logit as the input of the classifier. cls ;

[0017] S25. Taking the cross entropy loss function as the target, train and adjust the parameters of the machine reading comprehension model to obtain the initial machine reading comprehension model;

[0018] The cross entropy loss function The calculation formula is:

[0019]

[0020] in The answer probability logit predicted by the machine reading comprehension model cls ,y i is the true answer indicating whether the question is answerable or unanswerable; N is the number of label samples;

[0021] S26. Using a preset evaluation function and a verification label data set, evaluate the accuracy of the answers output by the trained initial machine reading comprehension model;

[0022] The preset evaluation function adopts the accuracy evaluation index, and its calculation formula is:

[0023]

[0024] Where f(x i ) is the model prediction label, y i is the true label, and the I function is the indicator function. When the input is True, the output is 1, and when the input is False, the output is 0;

[0025] S27. After a preset number of evaluations, determine whether the evaluation index has been improved. If not, stop training the machine reading comprehension model to obtain an initial machine reading comprehension model for self-training.

[0026] Furthermore, the classifier described in step S24 is a linear regression classifier, and the function calculation formula is:

[0027] y=β0+β1x

[0028] Among them, x is the predicted value of the model, y is the true label value; β0 and β1 are the parameters learned by model training.

[0029] Furthermore, the implementation method of iteratively training the initial machine reading comprehension model using the self-training method on the unlabeled data set to obtain the ideal machine reading comprehension model is as follows:

[0030] S31. Collect unlabeled data samples for self-training; use the training labeled data set to disrupt the sequence and remove the label items to form an unlabeled data sample set sample;

[0031] S32. Inputting the unlabeled data sample set into the initial reading comprehension model to obtain a predicted value, and obtaining a pseudo label that can be used for self-training by using a soft label method, thereby constructing a pseudo label sample set for self-training;

[0032] S33. Input the pseudo-label sample training set into the model obtained in the previous round of iteration for training, take the relative entropy loss function as the target, adjust the parameters of the model obtained in the previous round of iteration, and obtain a new round of machine reading comprehension model;

[0033] Among them, the calculation formula of the relative entropy loss function is as follows:

[0034]

[0035] where q ij is the pseudo label in the dataset, p ij is the predicted value of the model;

[0036] S34. Evaluate the accuracy of the answers of the machine reading comprehension model using a preset evaluation function and verification label sample data;

[0037] The preset evaluation function adopts the accuracy evaluation index, and its calculation formula is:

[0038]

[0039] Where f(x i ) is the model prediction label, y i is the true label, and the I function is the indicator function. When the input is True, the output is 1, and when the input is False, the output is 0;

[0040] S35. After a preset number of evaluations, the reading comprehension model with the highest accuracy is the ideal reading comprehension model.

[0041] Furthermore, the soft label method described in step S32 obtains a pseudo label by enhancing high-confidence predictions and degrading low-confidence predictions by squaring and normalizing the current predictions. The calculation formula is:

[0042]

[0043] Among them, q ij represents the pseudo-label obtained by the soft labeling method, which is obtained by applying the classifier trained by the machine reading comprehension model to the context representation vector corresponding to the [CLS] tag value of each text;

[0044] p ij = p(c j |h [CLS] )

[0045] where c j Indicates the classification of the true answer: yes or no; h [cls] Represents the context representation vector corresponding to the [CLS] tag value in the input document.

[0046] Furthermore, the soft label method can also be replaced by a hard label method, which converts high confidence predictions exceeding a threshold into a one-hot label, i.e., q ij =I(p ij >threshold), where I is the indicator function, when the input is True, the output is 1, and when the input is False, the output is 0.

[0047] Compared with the prior art, the present invention introduces a loss function and a verification function to detect the prediction effect of the reading comprehension model. The use of unlabeled data can better improve the effect of model training and obtain better results than traditional training methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 The present invention is a flowchart of a self-training method for a reading comprehension model for refusal to answer questions according to an embodiment of the present invention. DETAILED DESCRIPTION

[0049] To further disclose the present invention, exemplary embodiments will be described in detail below with reference to the accompanying drawings.

[0050] The self-training method of the reading comprehension model for the refusal to answer question provided in this embodiment specifically includes the following steps:

[0051] S1: Establish a reading comprehension model for refusal to answer questions, design the reading comprehension task to be processed by two sub-modules, and conduct training. The sub-modules include a text encoding module and an answer classification module:

[0052] S2: Input the text and question into the text encoding module, which encodes them and then inputs them into the answer classification module, which outputs the answer to the question:

[0053] S3: Self-train the trained reading comprehension model:

[0054] The present invention can obtain the correct answer through the machine reading comprehension model based on the question and related text, and is used in the legal field to improve reading efficiency and quickly feedback the answer. The invention proposes a self-training method for the reading comprehension model for the refusal to answer question task in reading comprehension, which can effectively use the unlabeled legal data, expand the labeled training data set, and ultimately improve the performance of the final model.

[0055] S1: In this embodiment, the steps of the text encoding module processing text and questions are as follows:

[0056] (1) Select a preset number of labeled samples as a labeled training set;

[0057] In this embodiment, a sample consists of a document, a question, and the answer to the question. The training set is a collection of selected samples. When selecting samples, the subject matter of the samples is selected based on the application field of the model. For example, if it is the field of machine reading of smart justice, documents of legal data are selected.

[0058] (2) The question and document data in the label training set are concatenated to form a new "question-document" text data, which is used as the input of the tokenizer to obtain a text token sequence, where the token sequence vector P of a given passage is T and the token sequence vector Q of the question T , the “question-document” text can be represented as:

[0059] [CLS]Q T [SEP]P T [SEP]

[0060] Among them, [CLS] and [SEP] are special flags required by the BERT model. The [CLS] flag is placed at the beginning of the first sentence. The representation vector C obtained by BERT can be used for subsequent classification tasks. The [SEP] flag is used to separate two input sentences. For example, if you input sentences A and B, you need to add the [SEP] flag after sentences A and B. The tokenizer is the text encoding of the bert model, which is used to convert text into a sequence vector, that is, a list of subscripts of words in the dictionary.

[0061] (3) Use the BERT (BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding) pre-trained model to encode the "question-document" text and obtain the vectorized representation of the text h∈R c×d , where C is the length of the text and d is the dimension of the encoding;

[0062] BERT is a deep bidirectional, unsupervised language representation model that is pre-trained using only a plain text corpus. Context-independent models (such as word2vec or GloVe) generate a word vector representation for each word in the vocabulary, so it is easy to have word ambiguity. BERT takes into account the context in which the word appears. For example, the word2vec word vector of the word "water" is the same in "plants need to absorb water" and "there is water in financial statements", but BERT provides different word vectors based on different contexts, and the word vector is related to the meaning of the sentence. The "question-document" text is encoded using BERT. Through this process, deeper feature representations of the passage and the question can be learned.

[0063] For natural language processing tasks, computers cannot directly understand human language, so the text needs to be encoded into a digital format that the computer can understand. The text encoding module mainly encodes questions and text as input for subsequent modules.

[0064] S2: In this embodiment, the answer prediction module predicts the answer and performs training, and the steps are as follows:

[0065] a represents the vector h corresponding to the [CLS] tag value obtained by the text encoding module [cls] Use dropout to perform overfitting and then use it as the input of the classifier to get the final answer probability, which is recorded as logit. cls ;

[0066] The classifier is a linear regression, and the function calculation formula of the linear regression is:

[0067] y=β0+β1x

[0068] Among them, x is the predicted value of the model, y is the true label value; β0 and β1 are the parameters learned by the model.

[0069] Dropout is a regularization technique used to combat overfitting in artificial neural networks. Dropout is effective because it avoids complex co-adaptation on the training data. The term dropout refers to discarding some neurons (both hidden and visible) in a neural network. During the training phase, dropout allows only part of the network structure to be updated each time, making it an efficient way to average neural network models.

[0070] b. Taking the cross entropy loss function as the target, adjust the parameters of the machine reading comprehension model to obtain a basic machine reading comprehension model.

[0071] Among them, the cross entropy loss function The calculation formula is:

[0072]

[0073] in The answer probability logit predicted by the machine reading comprehension model class ,y i is the true answer indicating whether the question is answerable or unanswerable. N is the number of labeled samples.

[0074] After adjusting the parameters of the basic machine reading comprehension model with the goal of minimizing the training loss function, the method further includes:

[0075] c. After the parameters of the machine reading comprehension model have been adjusted a preset number of times, the accuracy of the answers output by the trained basic machine reading comprehension model is evaluated using a preset evaluation function and a verification label data set:

[0076] Among them, the preset evaluation function adopts the accuracy evaluation index, and its calculation formula is:

[0077]

[0078] In the above calculation formula, f(x i ) is the model prediction label, y i is the true label, and the I function is the indicator function. When the input is True, the output is 1, and when the input is False, the output is 0.

[0079] d. determining whether the evaluation index is improved after a preset number of trainings; if not, stopping the training of the machine reading comprehension model to obtain an initial model for self-training;

[0080] After the above steps are trained for a preset number of times, and after the preset number of training, the above preset evaluation function is used for evaluation, so that after the preset number of evaluations, it is determined whether the evaluation index has been improved.

[0081] S3: As a self-training method for the reading comprehension model for refusal to answer questions, the initial model mentioned above is used for self-training. The idea of ​​self-training is to repeatedly use the current prediction value of the model to calculate the target distribution and guide the model to improve. The steps are as follows:

[0082] 1) Constructing self-training pseudo-label data samples for self-training of the above initial model;

[0083] Optionally, we shuffle the labeled dataset used for training and remove the label items to form an unlabeled data sample set.

[0084] 2) Input the above unlabeled data sample set into the model obtained in the previous iteration to obtain the predicted value, and use the soft label to obtain its target distribution, so as to construct a pseudo-label sample set for self-training;

[0085] There are two main choices for target distribution: hard labels and soft labels. The hard label method converts high-confidence predictions exceeding a threshold into one-hot labels, i.e., q ij =I(p ij > threshold), where I is the indicator function, when the input is True, the output is 1, and when the input is False, the output is 0. The soft label method obtains the target distribution by enhancing high-confidence predictions and demoting low-confidence predictions by squaring and normalizing the current predictions.

[0086] The calculation formula of the soft label function is:

[0087]

[0088] Among them, q ij represents the pseudo-label obtained by the soft labeling method, which is obtained by applying the classifier trained by the machine reading comprehension model to the context representation vector corresponding to the [CLS] tag value of each text;

[0089] p ij = p(c j |h [CLS] )

[0090] where c j Indicates the classification of the true answer: yes or no, h [cls] Embedding vector representing the input document.

[0091] 3) Input the above pseudo-label sample set into the model obtained in the previous round of iteration for training, take the relative entropy loss function as the target, adjust the parameters of the model obtained in the previous round of iteration, and obtain a new round of machine reading comprehension model;

[0092] Among them, the calculation formula of the relative entropy loss function is as follows:

[0093]

[0094] Relative entropy, also known as Kullback-Leibler divergence or information divergence, is a measure of the asymmetry of the difference between two probability distributions, where q ij is the pseudo label in the dataset, p ij is the predicted value of the model.

[0095] 4) Using the preset evaluation function and verification label sample data, the accuracy of the answers of the above machine reading comprehension model is evaluated.

[0096] Among them, the preset evaluation function adopts the accuracy evaluation index, and its calculation formula is:

[0097]

[0098] In the above calculation formula, f(x i ) is the model prediction label, y i is the true label, and the I function is the indicator function. When the input is True, the output is 1, and when the input is False, the output is 0.

[0099] 5) After a preset number of trainings, and after a preset number of trainings, the preset evaluation function is used for evaluation again, so that after the preset number of evaluations, a reading comprehension model with the highest accuracy is obtained which can be used for the next round of iterations.

[0100] Using the method provided in the above embodiment, it has been verified that the accuracy of the test results obtained using the BERT-based reading comprehension model is 80.54%, while the accuracy of the test obtained using the method of this embodiment is 81.69%. It is proved that the reading comprehension model trained by the method provided in this embodiment is better than the BERT-based reading comprehension model.

Claims

1. A self-training method for a reading comprehension model for refusal questions, characterized in that: The steps include: S1. Build a machine reading comprehension model for refusal questions, including a text encoding module and an answer prediction module; S2. Input the text and question into the text encoding module for encoding, and then input into the answer prediction module for processing; specifically: S21. Select a preset number of label samples as a label training set; S22. Concatenate the question and document data in the label training set to form "question-document" text data, and use it as the input of the tokenizer to obtain a text token sequence; according to the design of the BERT pre-training model, the first value of the text token sequence is the [CLS] tag value, and the representation vector h corresponding to the [CLS] tag value in the context representation vector is used. [cls] ∈R b×d Used as input for the subsequent answer prediction module, where b is the number of input text data and d is the dimension of the encoding; S23. Use the BERT pre-trained model to process the text token sequence to obtain the contextual representation vector x∈R of the text data b×c×d , where b is the amount of input text data, c is the length of the text, and d is the dimension of the encoding; S24. The representation vector h corresponding to the [CLS] tag value obtained by the text encoding module [cls] After overfitting using dropout, the final answer probability is recorded as logit as the input of the classifier. cls ; S25. Taking the cross entropy loss function as the target, train and adjust the parameters of the machine reading comprehension model to obtain the initial machine reading comprehension model; The cross entropy loss function The calculation formula is: in The answer probability logit predicted by the machine reading comprehension model cls ,y i is the true answer indicating whether the question is answerable or unanswerable; N is the number of label samples; S26. Using a preset evaluation function and a verification label data set, evaluate the accuracy of the answers output by the trained initial machine reading comprehension model; The preset evaluation function adopts the accuracy evaluation index, and its calculation formula is: Where f(x i ) is the model prediction label, y i is the true label, and the I function is the indicator function. When the input is True, the output is 1, and when the input is False, the output is 0; S27. After a preset number of evaluations, determine whether the evaluation index has been improved. If not, stop training the machine reading comprehension model to obtain an initial machine reading comprehension model for self-training; S3. The initial machine reading comprehension model is iteratively trained on an unlabeled dataset using a self-training method to obtain an ideal machine reading comprehension model. The implementation method is as follows: S31. Collect unlabeled data samples for self-training; use the training labeled data set to disrupt the sequence and remove the label items to form an unlabeled data sample set sample; S32. Input the unlabeled data sample set into the initial machine reading comprehension model to obtain a predicted value, and obtain a pseudo label for self-training by using a soft label method, thereby constructing a pseudo label sample set for self-training; wherein the soft label method enhances high-confidence predictions and degrades low-confidence predictions by squaring and normalizing the current predictions, thereby obtaining a pseudo label that can be used for self-training, and the calculation formula is: Among them, q ij represents the pseudo-label obtained by the soft labeling method, which is obtained by applying the classifier trained by the machine reading comprehension model to the context representation vector corresponding to the [CLS] tag value of each text; p ij =p(c j |h [CLS] ); where c j Indicates the classification of the true answer: yes or no; h [cls] Represents the context representation vector corresponding to the [CLS] tag value in the input document; S33. Input the pseudo-label sample training set into the model obtained in the previous round of iteration for training, take the relative entropy loss function as the target, adjust the parameters of the model obtained in the previous round of iteration, and obtain a new round of machine reading comprehension model; Among them, the calculation formula of the relative entropy loss function is as follows: where q ij is the pseudo label in the dataset, p ij is the predicted value of the model; S34. Evaluate the accuracy of the answers of the machine reading comprehension model using a preset evaluation function and verification label sample data; The preset evaluation function adopts the accuracy evaluation index, and its calculation formula is: Where f(x i ) is the model prediction label, y i is the true label, and the I function is the indicator function. When the input is True, the output is 1, and when the input is False, the output is 0; S35. After a preset number of evaluations, the reading comprehension model with the highest accuracy is the ideal reading comprehension model.

2. The self-training method for a reading comprehension model for refusal to answer questions according to claim 1, characterized in that: The classifier described in step S24 is a linear regression classifier, and the function calculation formula is: y=β0+β1x Among them, x is the predicted value of the model, y is the true label value; β0 and β1 are the parameters learned by model training.

Images