Machine reading comprehension model training method based on contrastive learning distant supervision

Abstract

The application discloses a machine reading comprehension model training method based on contrast learning long-distance supervision. The method comprises the following steps: 1) for each given <question, answer> pair, first, a text paragraph containing the <question, answer> pair is retrieved from a text corpus, and an answer string is found and positioned in the text paragraph as an annotation of an answer position, thereby obtaining a training instance bag for the <question, answer> pair; and a text paragraph not containing the answer in the <question, answer> pair is taken as a negative example text paragraph; 2) inputting the question and the text paragraph in the training instance into a machine reading comprehension model, obtaining an answer position of the training instance according to a predicted output, and updating a confidence of the training instance according to a probability distribution of an answer start and end position; and 3) training the machine reading comprehension model based on the training instance after the confidence is updated. The application can avoid the prediction shortcut and semantic drift problems caused by the wrong annotated instance.

Description

TECHNICAL FIELD

[0001] The application relates to a machine reading comprehension model training method, in particular to a machine reading comprehension model training method based on contrastive learning distant supervision, and belongs to the technical field of natural language processing. BACKGROUND

[0002] Machine reading comprehension technology refers to giving a text paragraph and a related question, training a model by using a machine learning method, so that the model can extract the answer to the question from the text paragraph.

[0003] Generally, a supervised machine reading comprehension model is trained using artificially labeled <question, answer position> data. However, labeling training data consumes a lot of manpower and time. In a low-resource environment, a machine reading comprehension model is often learned using distant supervision technology. Distant supervision uses <question, answer> data, which is easier to obtain than accurately labeling the answer position in the document. Specifically, given a <question, answer> pair, the distant supervision machine reading comprehension model system retrieves the relevant text paragraph, finds a string matching the answer in the text paragraph as the answer position, and trains the machine reading comprehension model using the noisy answer position data.

[0004] However, distant supervision data contains many incorrectly labeled training instances. As shown in Table 1, the string "1990" in paragraph 2 is consistent with the correct answer, but its context is irrelevant to the question, so the paragraph is not a correct training instance. The incorrectly labeled problem in the distant supervision data will bring noise and bias to the training of the machine reading comprehension model. Figure 1

[0005] On the one hand, due to the inconsistency between the content of the incorrectly labeled paragraph and the question and the content of the correctly labeled paragraph, the machine reading comprehension model tends to learn a shortcut for the answer position, and cannot obtain the ability to accurately match the context semantics. This is because the most obvious commonality between the context of the correctly and incorrectly labeled training instances is the answer itself. For example, as shown in Table 2, both paragraph 1 and paragraph 2 in the question are labeled as correct paragraphs for the question by the distant supervision system, so the model tends to directly use "1990" as the answer to the question in the answer prediction. Figure 1

[0006] On the other hand, the machine reading comprehension model will establish an incorrect semantic association between the question and the context of the incorrectly labeled instance, causing the semantic representation of the incorrectly labeled instance to be too close to the question semantics and far away from the representation of the negative example paragraph that is similar in content but does not contain the answer. For example, as shown in Table 3, the context of the question "1990" is "1990" and "1990", which is very close to the question semantics, and the semantic representation of the incorrectly labeled paragraph is very close to the question semantics, which is far away from the semantic representation of the negative example paragraph. Figure 1 ​​As shown, paragraph 2 and paragraph 3 contain similar information. However, since paragraph 2 contains the answer and paragraph 3 does not, the representation of paragraph 2 will gradually approach the representation of the question and paragraph 1 during the learning process, while the representation of paragraph 3 will deviate from the representation of the question and paragraph 1. Therefore, although paragraph 2 and paragraph 3 describe similar content, the distance between their representations will be widened.

[0007] Contrastive learning refers to a technique for learning the distribution of representations of samples by expanding the similarity between positive samples and minimizing the similarity between positive samples and negative samples. SUMMARY

[0008] To more effectively train a distant supervision machine reading comprehension model, the present application provides a machine reading comprehension model training method based on distant supervision of contrastive learning.

[0009] The present application mainly includes: (1) a training instance confidence estimation method for distant supervision machine reading comprehension; and (2) a confidence-based distant supervision machine reading comprehension contrastive learning technique. The training instance confidence estimation method for distant supervision machine reading comprehension takes all <question, text paragraph, answer position> training instances corresponding to a question as an instance bag, and can automatically estimate the confidence of the training instances in the instance bag, thereby distinguishing between correctly and incorrectly labeled instances in the instance bag. The confidence-based distant supervision machine reading comprehension contrastive learning technique reduces noise in the instance bag by contrasting high-confidence and low-confidence training instances within the instance bag during model training; and eliminates the bias of the instance context representation by confusing the low-confidence training instances within the instance bag with text paragraphs outside the instance bag that do not contain the answer.

[0010] Because incorrectly labeled instances are used as correct instances during model training, the model will incorrectly establish a connection between the context semantics of the question and the incorrectly labeled instances. This incorrect connection causes the semantic representation of the incorrectly labeled sample to be biased: the semantic representation of the incorrectly labeled sample is biased towards the question semantics and away from text paragraphs outside the instance bag that express similar content but do not contain the answer. The present application narrows the distance between the semantic representations of the incorrectly labeled sample and the text paragraphs that do not contain the answer by confusing the low-confidence training instances within the instance bag with the text paragraphs outside the instance bag that do not contain the answer, thereby eliminating the bias in the semantic representation caused by incorrect labeling. The distribution of positive and negative example sample representations is adjusted during model training through contrastive learning, i.e., the model is trained using a contrastive learning loss function.

[0011] The technical solutions adopted by the present application are summarized as follows:

[0012] A machine reading comprehension model training method based on distant supervision of contrastive learning, the steps of which include:

[0013] 1) For each given <question, answer> pair, first retrieve the text passage containing the <question, answer> pair from the text corpus, and find and locate the answer string in the text passage as the annotation of the answer position, obtaining the training instance bag for the <question, answer> pair; the text passage not containing the answer in the <question, answer> pair is taken as the negative example text passage; each training instance in the training instance bag includes <question, text passage, answer position>;

[0014] 2) Input the question and text passage in the training instance into the machine reading comprehension model, obtain the answer position of the training instance according to the predicted output of the start and end position probability distribution of the answer in the training instance, and update the confidence of the training instance according to the start and end position probability distribution of the answer;

[0015] 3) Train the machine reading comprehension model based on the training instance whose confidence is updated; wherein the loss function used in the training process is L CDS =L CE +μL SCL ; wherein, is the semantic representation of the negative example text passage, represents the set of negative example text passages corresponding to the i-th question q i , and F(·) is a function for adjusting the sensitivity of the representation, μ is a hyperparameter, and N is the total number of <question, answer> pairs, is the answer start position word of the j-th training instance in the i-th instance bag, is the answer end position word of the j-th training instance in the i-th instance bag, r i,j is the vector representation of the j-th training instance in the i-th instance bag, r i,k is the vector representation of the k-th training instance in the i-th instance bag, c i,j represents the confidence of the j-th training instance corresponding to the i-th question q i , c i,k represents the confidence of the k-th training instance corresponding to the i-th question q i , and J is the total number of training instances in the i-th instance bag, and respectively represent the annotated start and end positions of the answer in the j-th training instance corresponding to the i-th question q i , represents the probability predicted by the machine reading comprehension model as the start of the answer, represents the probability predicted by the machine reading comprehension model as the end of the answer,

[0016] ​The probability of the answer ending.

[0017] Further, the method for updating the confidence of the training instance is: the updated confidence c i,j = λc i,j ′ + (1- λ)z i,j ; wherein, λ∈(0,1) represents a sliding average weighting constant, c i,j ′ is the confidence of the jth training instance in the ith instance bag before updating.

[0018] Further, the method for updating the confidence of the training instance is: the method for updating the confidence of the training instance is: the updated confidence c i,j = λc i,j ′ + (1- λ)z i,j ; wherein, λ∈(0,1) represents a sliding average weighting constant, c i,j ′ is the confidence of the jth training instance in the ith instance bag before updating, and respectively represent the start and end positions of the answer labeled in the kth training instance corresponding to the ith question q i , represent the probability of the machine reading comprehension model predicting as the start of the answer, represent the probability of the machine reading comprehension model predicting as the end of the answer.

[0019] Further, the method for updating the confidence of the training instance is: the method for updating the confidence of the training instance is: the updated confidence c i,j = λc i,j ′ + (1- λ)z i,j ; wherein, λ∈(0,1) represents a sliding average weighting constant, c i,j ′ is the confidence of the jth training instance in the ith instance bag before updating, P start (c) represents the probability of the machine reading comprehension model predicting the position c as the start of the answer, P end (c) represents the probability of the machine reading comprehension model predicting c as the end of the answer.

[0020] Further, in step 1), the confidence score of each training instance is initialized according to the retrieval matching degree between the question and the paragraph.

[0021] Further, the vector representation of the [CLS] token of the negative example text passage is used as r - .

[0022] Further, the machine reading comprehension model includes but is not limited to a machine reading comprehension model based on a BERT pre-training language model.

[0023] An answer prediction method is characterized by inputting a target question and a selected text passage into the machine reading comprehension model obtained by the above method, and outputting an answer corresponding to the target question in the selected text passage.

[0024] A server is characterized by including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing steps in the above method.

[0025] A computer-readable storage medium having a computer program stored thereon is characterized in that the computer program is executed by a processor to implement the steps of the above method.

[0026] A machine reading comprehension model training method based on contrast learning and distant supervision includes the following key technologies:

[0027] 1) Training instance confidence estimation method for distant supervision machine reading comprehension: all <question, text passage, answer position> training instances corresponding to a question are taken as an instance bag, and the confidence of each training instance in the instance bag is estimated by iteratively using the answer start and end position probability predicted by the current machine reading comprehension model, and the model is trained based on the estimated confidence.

[0028] 2) Confidence-based distant supervision machine reading comprehension contrast learning model training technology: in the model training process, the training instances with high and low confidence in the instance bag are compared to reduce noise in the instance bag; and the low-confidence training instances in the instance bag are mixed with text passages that do not contain answers outside the instance bag to eliminate the bias of the instance context representation.

[0029] Further, the confidence of the training instance is initialized using the retrieval matching degree of the question and the text passage in part 1).

[0030] Further, part 1) of the technology contains three methods for updating the confidence of the training instance in the instance bag by the model output result:

[0031] 1) Soft Weighting: according to the prediction probability of the machine reading comprehension model for the answer, a continuous confidence score is calculated for each training instance.

[0032] 2) Hard Max: Assuming that only one instance in each instance bag is correctly labeled, the instance with the highest probability is selected as the correct instance; the confidence scores of other training instances in the instance bag are all labeled as 0.

[0033] 3) Positive Average: If the model's predicted answer position is consistent with the marked answer position, the instance is marked as correct, and the confidence scores of the correct training instances are evenly distributed, i.e., the confidence scores of the correct training instances are all set to the same value.

[0034] Further, the technology 2) part includes a loss function composed of two parts:

[0035] 1) Confidence-based distant supervision machine reading comprehension contrastive learning loss function.

[0036] 2) Confidence-weighted machine reading comprehension cross-entropy loss function.

[0037] Further, the technology 2) part includes denoising the instance bag:

[0038] By minimizing the similarity of the semantic representations of high-confidence and low-confidence training examples in the instance bag, the instance bag is denoised to avoid learning the wrong shortcut of predicting the answer position.

[0039] Further, the technology 2) part includes eliminating the bias of instance context representation in the training process:

[0040] By maximizing the similarity of the semantic representations of low-confidence training instances in the instance bag and negative example text passages that do not contain answers outside the instance bag, the low-confidence training instances are confused with negative example text passages, avoiding the problem of semantic drift of training instance context.

[0041] Compared with the prior art, the beneficial effects of the present application are:

[0042] 1) For low-resource supervised machine reading comprehension, a contrastive learning distant supervision method is proposed, which automatically discriminates incorrectly labeled instances through instance bag-level instance contrast strategy, and denoises and debiases the distant supervision process.

[0043] 2) A confidence-based contrastive learning strategy is designed to align the distance between training instances according to the estimated confidence score, thereby optimizing the distribution of training instance semantic representations and avoiding the prediction shortcut and semantic drift problems caused by incorrectly labeled instances. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 Distant supervision machine reading comprehension training instance graph.

[0045] Figure 2 A principle flow chart of the distant supervision method of comparative learning. DETAILED DESCRIPTION

[0046] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below.

[0047] Given a <question, answer> pair, first retrieve the relevant text passage from the text corpus, and find and locate the answer string in the text passage as the annotation of the answer position, so as to construct the training instance bag for the <question, answer> pair. For the text passage that does not contain the answer, it is regarded as a negative example text passage. Subsequently, the present method iterates between training instance confidence estimation and confidence-based distant supervision machine reading comprehension comparative learning.

[0048] The machine reading comprehension model in the present application can adopt a variety of different structures of models, including but not limited to machine reading comprehension models based on pre-trained language models such as BERT (Bidirectional Encoder Representations from Transformers). The reading comprehension model is defined as:

[0049] P start ,P end =MRC(q,p)

[0050] Where q and p are the question and the text passage respectively, and the model output is two probability distributions P start and P end , which respectively represent the probability of each word in the text passage p being the start and end position. The answer position in the training instance is the position of the correct answer located in the text passage, which is used for model training and is the target of the model output, which is used when calculating the loss function; the confidence of the training instance is calculated according to the answer position probability distribution output by the model and the answer position in the training instance.

[0051] Training instance confidence estimation:

[0052] The instance confidence estimation method estimates a confidence score for each training instance in the instance bag, which represents the probability of the training instance being correct.

[0053] First, according to the retrieval matching degree between the question and the passage, the confidence score of each training instance is initialized. The present application uses the BM25 matching score, and the score is normalized:

[0054]

[0055] Where c i,jdenotes the initial confidence of the jth training instance of the ith question q i , J is the total number of training instances corresponding to the ith question q i . i,j denotes the text passage of the jth training instance of the ith question q i . i,k denotes the text passage of the kth training instance of the ith question q i , sim() denotes the BM25 matching function.

[0056] The present application uses the output of the machine reading comprehension model as the basis for confidence estimation, updating the confidence c i,j of each instance. The confidence vector is updated using a sliding average method:

[0057] c i,j = λc i,j + (1 - λ)z i,j

[0058] where λ ∈ (0, 1) represents the sliding average weighting constant, z i,j is the confidence calculated according to the output of the machine reading comprehension model. The present application designs three methods to calculate the confidence from the model output:

[0059] 1) Soft Weighting: According to the predicted probability of the answer by the machine reading comprehension model, a continuous confidence score is calculated for each training instance, p i,j , the corresponding confidence score is z i,j :

[0060]

[0061]

[0062] where p i,j represents the predicted answer probability, P start and P end are the probability distributions output by the machine reading comprehension model, representing the probability of each word in the text passage as the start and end position of the answer, and represent the start and end positions of the answer labeled by distant supervision in the jth training instance corresponding to the ith question, represent the probability of the word in the text passage as the start of the answer, represent the probability of the word in the text passage as the end of the answer.

[0063] 2) Hard Max: Assume that only one instance in each instance bag is correctly labeled, and select the instance with the highest probability as the correct instance:

[0064]

[0065] 3) Positive Average: If the model's predicted answer position is consistent with the marked answer position, the instance is marked as correct, and the confidence score is evenly distributed to the correct training instances:

[0066]

[0067]

[0068] By matching the score to initialize the confidence and using the model output to iteratively update the confidence score, this method can gradually optimize the confidence score of the training instance; arg max c P start (c) refers to the position of the maximum value in P start .

[0069] Confidence-based distant supervision machine reading comprehension contrastive learning:

[0070] Given the estimated confidence score, the present application realizes the denoising and debiasing of the machine reading comprehension model through the confidence-based contrastive learning strategy. Specifically, by contrasting the high-confidence and low-confidence training instances within the instance bag, the instance bag is denoised; and by mixing the low-confidence training instances within the instance bag with the text passages outside the instance bag that do not contain the answer, the bias of the instance context representation is eliminated.

[0071] Specifically, the present application regards the low-confidence instances in the instance bag as potential mislabeled instances, and performs contrastive learning centered on the potential mislabeled instances. The representation of the low-confidence instances should be far away from the high-confidence instances, and should be confused with the negative sample text passages outside the instance bag. Therefore, the present application designs a contrastive learning-based method that minimizes to expand the distance between the high-confidence instances and the low-confidence instances in the instance bag, and to reduce the distance between the low-confidence instances and the negative sample text passages, i.e., to minimize and maximize

[0072] However, the confidence score is a continuous variable rather than a 0 / 1 category label, so there is no hard distinction between correct and potentially mislabeled instances. Simply using the InfoNCE loss function will result in inaccurate confidence estimation. To address this challenge, the present application designs a confidence-based weighted contrastive loss function.

[0073] Example bag noise reduction comparison (inside the bag):

[0074] This invention reduces noise in the instance bag by minimizing the similarity between the semantic representations of high-confidence and low-confidence training instances, thus avoiding the model's erroneous shortcut in learning to predict the answer position.

[0075] First, the starting positional term is represented by the answer of the j-th training instance in the i-th instance bag. and the end position word The summation of the vectors yields the vector representation r of the j-th training instance in the i-th instance bag. i,j :

[0076]

[0077] For each potentially mislabeled instance, minimize the similarity between its representation and the high-confidence instances in the instance bag:

[0078]

[0079] Where the confidence score c i,k It can select high-confidence instances from the instance bag. For high-confidence instances, c i,k Greater weight, for The confidence level plays a decisive role, while for low-confidence instances, c i,k The lower the weight, the smaller the impact on the formula. By using confidence weighting, high-confidence training instances are selected for calculation, thereby minimizing the similarity between high-confidence and low-confidence training instances. F(·) is a function that adjusts the sensitivity to similarity; in this invention, F(·) = exp(·).

[0080] The training process for negative sample confusion involves bias removal:

[0081] This invention avoids the problem of semantic drift in training instance context by maximizing the similarity of the semantic representations of low-confidence training instances in the instance bag and negative sample text paragraphs outside the instance bag that do not contain answers, thereby confusing low-confidence training instances and negative sample text paragraphs at the semantic representation level.

[0082] For each potentially mislabeled instance, i.e. a low-confidence training instance, maximize the similarity between its representation and the representation of the negative sample text paragraph:

[0083]

[0084] in This is the semantic representation of the negative sample text paragraph. denotes the set of negative example text passages corresponding to the i-th question, and the vector representation of the [CLS] tag of the negative example text passage is used as

[0085] The long-distance supervised machine reading comprehension contrast learning loss function is:

[0086] The loss function of contrast learning is defined as:

[0087]

[0088] Where N is the total number of question and answer pairs in the training data. The (1-c i,j ) term can select low-confidence training instances from the instance bag as the center of contrast learning, and reduce the weight of high-confidence instances in the loss function.

[0089] In order to learn the ability of the machine reading comprehension model to select answers, the present application designs a machine reading comprehension cross-entropy loss function based on confidence weighting:

[0090]

[0091] Where the confidence c i,j The weighted average of the training instances in the instance bag is performed, thereby reducing the influence of low-confidence training instances on model training.

[0092] The overall training loss function is represented as:

[0093] L CDS =L CE +μL SCL

[0094] Where μ is a hyperparameter. The model optimization method uses the gradient descent method.

[0095] Answer prediction:

[0096] After model training, the input question and text passage can predict the position of the answer to the question in the text passage, thereby answering the question.

[0097] The above embodiments are only used to illustrate the technical solutions of the present application and not to limit them, and those skilled in the art can modify or equivalently replace the technical solutions of the present application without departing from the principles and scope of the present application, and the protection scope of the present application should be subject to the description of the claims.

Claims

1. A method for training a machine reading comprehension model based on contrastive learning with long-range supervision, comprising the following steps: 1) For each given <question, answer> pair, first retrieve the text paragraphs containing the <question, answer> pair from the text corpus, and find and locate the answer string in the text paragraphs as the label of the answer position to obtain a training instance bag for the <question, answer> pair; text paragraphs that do not contain the answer in the <question, answer> pair are taken as negative sample text paragraphs; each training instance in the training instance bag includes <question, text paragraph, answer position>. 2) Input the questions and text paragraphs from the training instance into the machine reading comprehension model, and obtain the answer position of the training instance by predicting the probability distribution of the start and end positions of the answer in the training instance; and update the confidence of the training instance according to the probability distribution of the start and end positions of the answer. 3) The machine reading comprehension model is trained based on the training instances with updated confidence levels; wherein the loss function used in the training process is L. CDS =L CE +μL SCL ;in, This is the semantic representation of the negative sample text paragraph. Let q represent the i-th question. i The set of corresponding negative sample text paragraphs, F(·) is a function that adjusts the sensitivity of similarity, μ is a hyperparameter, and N is the total number of <question, answer> pairs. Let the word be the starting position of the answer for the j-th training instance in the i-th instance bag. r represents the end-of-response word of the answer for the j-th training instance in the i-th instance bag. i,j Let r be the vector representation of the j-th training instance in the i-th instance bag. i,k Let c be the vector representation of the k-th training instance in the i-th instance bag. i,j Let q represent the i-th question. i The confidence level of the j-th training instance, c i,k Let q represent the i-th question. i The confidence score of the corresponding k-th training instance, where J is the total number of training instances in the i-th instance bag. and Let q represent the i-th question respectively. i The start and end positions of the answer are marked in the j-th training instance. Represents machine reading comprehension model prediction The probability of starting with the answer. Represents machine reading comprehension model prediction The probability of ending as the answer.

2. The method according to claim 1, characterized in that, The method for updating the confidence of the training instance is as follows: the updated confidence c of the j-th training instance in the i-th instance bag. i,j =λc i,j ′+(1-λ)z i,j ;in, c represents the moving average weighting constant. i,j ' represents the confidence level of the j-th training instance in the i-th instance bag before the update.

3. The method according to claim 1, characterized in that, The method for updating the confidence of the training instance is as follows: The method for updating the confidence of the training instance is as follows: The updated confidence c of the j-th training instance in the i-th instance bag. i,j =λc i,j ′+(1-λ)z i,j ;in, c represents the moving average weighting constant. i,j ' represents the confidence level of the j-th training instance in the i-th instance bag before the update. and Let q represent the i-th question respectively. i The corresponding start and end positions of the answer are marked in the k-th training instance. Represents machine reading comprehension model prediction The probability of starting with the answer. Represents machine reading comprehension model prediction The probability of ending as the answer.

4. The method according to claim 1, characterized in that, The method for updating the confidence of the training instance is as follows: The method for updating the confidence of the training instance is as follows: The updated confidence c of the j-th training instance in the i-th instance bag. i,j =λc i,j ′+(1-λ)z i,j ;in, c represents the moving average weighting constant. i,j Let P' be the confidence level of the j-th training instance in the i-th instance bag before the update. start (c) represents the probability that the machine reading comprehension model predicts position c as the starting point of the answer, P end (c) represents the probability that the machine reading comprehension model predicts c as the answer.

5. The method according to any one of claims 1 to 4, characterized in that, In step 1), the confidence score of each training instance is initialized based on the retrieval matching degree between the question and the paragraph.

6. The method according to claim 1, characterized in that, The vector representation of the [CLS] label of the negative sample text paragraph is used as r. - .

7. The method according to claim 1, characterized in that, The machine reading comprehension model includes, but is not limited to, a machine reading comprehension model based on the BERT pre-trained language model.

8. A method for predicting answers, characterized in that, The target question and the selected text paragraph are input into the machine reading comprehension model obtained by the method described in claim 1, and the answer corresponding to the target question in the selected text paragraph is output.

9. A server, characterized in that, The method includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing each step of the method of any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images