Influence function interpretation for sample-based sequence-to-sequence tasks in NLP

Abstract

A method for interpreting an influence function of a sample-based sequence-to-sequence task in NLP comprises the following steps: 1) selecting a sequence-to-sequence task to be interpreted, and selecting a natural language processing model and a data set to be tested for the sequence-to-sequence task; 2) encoding the corpus in the data set, dividing each piece of encoded data into different intervals to obtain a plurality of pieces of data and using them as new samples; 3) sending the new sample obtained in step 2) to the model to be tested for training, and ending the training when the loss tends to be stable; among all the checkpoints output during the training process of the model to be tested, selecting a checkpoint close to the final parameters of the model to be tested for calculating an influence score; 4) calculating the loss of the training sample and the test sample based on the parameters of the model to be tested in the checkpoint selected in step 3); 5) using the sample loss obtained in step 4), calculating a gradient vector based on the parameters of the model to be tested, and calculating the influence score according to the influence function formula.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing (NLP), and more specifically, to an influence function interpretation method for a sample-based sequence-to-sequence task. Background Art

[0002] As complex models become indispensable tools in many fields of artificial intelligence, people are increasingly interested in explaining the working mechanisms of these "black box" models and are committed to developing methods that can explain model decisions. However, the birth of deep learning has made models more and more complex, and the size of data sets is also increasing day by day, which has challenged the ability of explanation methods in terms of interpretability and fidelity.

[0003] At present, considerable progress has been made in the interpretation methods of machine learning models. In the existing technology, the influence function based on training samples to explain the behavior of the model has been recognized by most. Unlike other methods such as input erasure, saliency map or attention matrix, the influence function understands the decision of the model by explaining how a specific input causes the model to produce such a result based on the training examples. When this interpretation method was first designed, it was mainly applicable to image processing tasks and cannot be fully applied to natural language processing tasks.

[0004] Natural Language Processing (NLP) is the use of computers to process, understand and apply human languages ​​(such as Chinese, English, etc.). It is a branch of artificial intelligence and an interdisciplinary subject between computer science and linguistics. It is also known as computational linguistics.

[0005] In existing research, BERT-based text classification and natural language inference models have begun to use this method to try to explain decisions. These works have proved that influence functions are useful in the field of natural language processing. However, the application of existing influence function methods in sequence-to-sequence tasks in the field of natural language processing (for example, dialogue generation, automatic summarization, etc.) is still blank, and many targeted problems need to be adaptively solved. First, following the original method of influence function IF (Influence Function), for modern natural language processing models and large-scale data sets, the application of influence function may lead to excessively high computational costs due to the inverse Hessian matrix calculation in the original formula. The newly introduced TracIn method has no Hessian matrix calculation, but it may make false explanations for the participation of suboptimal checkpoints and lead to infidelity to the model in question; secondly, when the influence function is applied to sequence-to-sequence tasks, it is also necessary to solve the problem of how to calculate multiple labels in a sample; finally, for long natural language texts that are common in sequence-to-sequence tasks, the decision of the model may only depend on a specific part of the sample, and it is not accurate enough to use the entire original sample as an explanation unit.

[0006] Therefore, the influence function interpretation method for sample-based sequence-to-sequence tasks in NLP still needs to be improved. Summary of the invention

[0007] In view of the existing problems, the present invention proposes an influence function interpretation method for sample-based sequence-to-sequence tasks in NLP, so as to measure the influence of a specific training sample on a test sample based on a model to be tested (natural language processing model).

[0008] The present invention comprises the following steps:

[0009] An influence function interpretation method for a sample-based sequence-to-sequence task in NLP, comprising the steps of:

[0010] 1) Select a sequence-to-sequence task that needs to be explained, and select a natural language processing model and dataset to be tested for the sequence-to-sequence task;

[0011] 2) Encode the corpus in the data set, divide each encoded data into different intervals to obtain multiple data as new samples;

[0012] 3) Send the new samples obtained in step 2) to the model to be tested for training, and end the training when the loss tends to be stable;

[0013] Among all the checkpoints output during the training process of the model to be tested, select the checkpoints that are close to the final parameters of the model to be tested to calculate the impact score;

[0014] 4) Based on the parameters of the model to be tested in the checkpoint selected in step 3), calculate the loss of the training sample and the test sample;

[0015] 5) Using the loss obtained in step 4), the gradient vector is calculated based on the parameters of the model to be tested, and the influence score is calculated according to the influence function formula.

[0016] The technical improvements of the present invention mainly include:

[0017] (1) For sequence-to-sequence tasks, formulate a data processing strategy and set a sliding window to obtain the appropriate sample interval granularity as the interpretation unit;

[0018] (2) For the multi-label problem in sequence-to-sequence tasks where the number of labels is equal to the number of vocabulary, the loss of each token in the sample is summed as the loss of the sample;

[0019] (3) Select appropriate model output checkpoints as records of the model training process and participate in the calculation.

[0020] The technical effects of the present invention mainly include:

[0021] (1) Adopting appropriate data processing methods can both improve the model function as much as possible and fully realize the interpretability of the method;

[0022] (2) It eliminates the obstacle of multi-label problem to sample loss calculation in sequence-to-sequence tasks;

[0023] (3) It reduces the computational cost and ensures that the impact score results are faithful to the final model. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a flowchart of the influence function of the sample-based sequence-to-sequence task in NLP of the present invention. DETAILED DESCRIPTION

[0025] The following is combined with Figure 1 The process of describing in detail the specific implementation of the influence function of the sample-based sequence-to-sequence task in NLP of the present invention.

[0026] A sample-based influence function interpretation method for sequence-to-sequence tasks, comprising the following steps:

[0027] Step 1: Select the sequence-to-sequence (seq2seq) task that needs to be explained, and select the model and dataset to be tested for the specific sequence-to-sequence task;

[0028] Step 2: Encode the corpus in the selected data set, divide each piece of encoded data into different intervals according to the strategy, and obtain multiple pieces of data as new samples, and use the set of new samples as the data set to be used;

[0029] Step 3: Send the data set to the model (natural language processing model) for training, and end the training when the loss tends to be stable. During the training process, select some of the checkpoints output by the model to affect the calculation of the score;

[0030] Step 4: Calculate the loss of specific training samples and test samples based on the model parameters in the checkpoint of the selected model to be tested;

[0031] Step 5: Calculate the gradient vector of the loss of the specific sample obtained based on the parameters of the model to be tested, calculate the influence score according to the set influence function formula, and generate an influence score file.

[0032] Specifically in this specific implementation mode:

[0033] In step 1, select the appropriate language model and dataset as follows:

[0034] Step 1.1: The sequence-to-sequence task is a natural language processing task that realizes the conversion from one sequence to another. There are many choices for sequence-to-sequence tasks, such as machine translation, automatic summarization, and dialogue generation. Compared with applications such as machine translation and text summarization where humans can clearly understand the reasons for the result generation, it is relatively more meaningful to obtain explanations for the completion of the dialogue generation task. Based on this, this embodiment selects the dialogue generation task as the basic task;

[0035] Step 1.2: When dealing with generation tasks such as dialogue generation tasks, the model generates sequences in order.

[0036] When actually generating content, the dialogue generation task is from left to right, and the autoregressive language model predicts the following content based on the previous content, which naturally matches this process. Based on this embodiment, the autoregressive language model GPT-2 is selected as the model to be tested;

[0037] Step 1.3: The selection of dialogue generation tasks can be divided into three categories according to the application scenario: task-based, chat-based, and question-answering-based.

[0038] These three categories are widely used in different application scenarios and businesses. Compared with task-based conversations that require background knowledge, people need to understand the reasons for model decisions in daily chat conversations more, and the use of explanation methods is more meaningful. Based on this, this embodiment selects chat conversations as the data set.

[0039] In step 2, the method of encoding the corpus is as follows:

[0040] Step 2.1: The dataset contains multiple dialogue samples, each of which contains multiple rounds of speeches by two characters, and each character's single round of speech contains multiple sentences. Encode each dialogue sample in the dataset according to the sequence number of the word in the model vocabulary;

[0041] Step 2.2: Insert a coding symbol representing the speaking role before the speech of each role in the dialogue sample, and insert a coding symbol representing the start and end of each dialogue round at the beginning and end of each speech round;

[0042] Step 2.3: Set a sliding window and select each encoded dialogue sample. Slide the window back by the length of a single speech of a character according to the character coding symbol. Set the sliding window length to the length of 5 speeches. Save the content in the window after each sliding. Cut the dialogue sample into multiple new dialogue samples of different lengths.

[0043] In this example, the sliding window length is preferably set to the length of 5 speeches, which can effectively shorten the sample length, ensure that the data is not too redundant, and retain the relationship between the conversation contexts.

[0044] Step 2.4: Keep all samples with the same last speaking role in the new dialogue sample as a new dataset.

[0045] In step 3, the model checkpoint acquisition process is as follows:

[0046] Step 3.1: Assuming that the training samples are accessed every time during the training process, accessing a specific training sample during training will change the parameters of the model, and this change will affect the loss on the test sample. In order to obtain the impact of a specific training sample on a specific test sample, it is necessary to track the training process and capture the changes in the loss of the test sample after each training sample is accessed, that is, the influence of the training sample will be attributed to the cumulative value of each access to the training sample. This process is replaced by recording the checkpoints output during the model training process;

[0047] Step 3.2: Select a set of checkpoints output by the model. Among the multiple checkpoints output during the model training process, select the checkpoints that are close to the final model to ensure that the influence function calculation process and the influence score results are faithful to the final model parameters. That is, for the parameters in the checkpoints output by the model i is the traversal of the selected checkpoints, and a small enough δ is selected so that Are approaching

[0048] In step 4, the sample loss is calculated as follows:

[0049] Step 4.1: Calculate the relative contribution of a specific training instance at each checkpoint. When calculating the loss gradient point by point, for a specific training sample z, let the sample sequence be represented as z = {t 0 , ..., t m}, t is each token in the sample.

[0050] Step 4.2: Based on model parameters The cross entropy loss function is used to calculate the loss of the token. The loss calculation method is as follows:

[0051]

[0052] Among them, C is the number of label categories, that is, the number of words in the vocabulary, p = [p 0 , ..., p C-1 ] is a probability distribution, each element p i Indicates t m The probability of belonging to category i, y = [y 0 , ..., y C-1 ], is t m The one hot representation of the label means that each element y i In t m The label of is 1 when it is category i, otherwise it is 0;

[0053] Step 4.3: The loss of the generation mechanism of the sequence-to-sequence task is calculated at the token level. To obtain the loss of each sample segment, the loss of each token in each sample segment is summed up. The calculation method is as follows:

[0054]

[0055] Step 4.4: Similarly, for a specific test sample z′, the sample sequence is represented as z′={t′ 0 , ..., t′ n}, t′ is each token in the sample. The sample loss calculation method is as follows:

[0056]

[0057] In step 5, the impact score is calculated as follows:

[0058] Step 5.1: Based on parameters The loss gradient expression for calculating the sample is as follows:

[0059]

[0060] Step 5.2: According to the influence function TracInS formula, based on the parameters of each checkpoint According to step 5.1, calculate the test and training sample loss gradients, do a simple dot product, sum them up at each checkpoint, and calculate the influence score of the training sample z on the test sample z′:

[0061]

[0062] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in the relevant technical field, are also included in the patent protection scope of the present invention.

Claims

1. A method for interpreting influence functions for sample-based sequence-to-sequence tasks in NLP, characterized by The following steps are involved: Step 1: Select the sequence-to-sequence task that needs to be explained, and select the natural language processing model and dataset to be tested for the specific sequence-to-sequence task; Step 2: Encode the corpus in the selected data set, divide each piece of encoded data into different intervals, and use the multiple pieces of data obtained as new samples, and use the set of new samples as the data set to be used; Step 3: Send the data set to be used obtained in step 2 to the model to be tested for training, and end the training when the loss tends to be stable; For all the checkpoints of the model output during the training process, select the checkpoints that are close to the final parameters of the model to be tested to calculate the impact score; Step 4: Based on the parameters of the model to be tested in the checkpoint selected in step 3, calculate the loss of the training sample and the test sample, and add the loss of each token in the two types of samples as the loss of each sample; Step 5: Using the loss obtained in step 4, calculate the gradient vector based on the parameters of the model to be tested, calculate the influence score according to the set influence function formula, and generate an influence score file; Step 2 includes the following steps: Step 2.1: The dataset contains multiple dialogue samples, each of which contains multiple rounds of speeches by two characters, and each character's single round of speech contains multiple sentences; encode each dialogue sample in the dataset according to the sequence number of the word in the vocabulary of the model to be tested; Step 2.2: In the dialogue sample, insert the coding symbol representing the speaking role before the speaking role in each round, and insert the coding symbol representing the start and end of each round of dialogue at the beginning and end of each round of speech; Step 2.3: Set a sliding window and select each encoded dialogue sample; slide the window back by the length of a single round of speech of a character according to the character's encoding symbol, and save the content in the corresponding window; Cut the conversation sample into multiple new conversation samples of different lengths; Step 2.4: In the new dialogue samples, take all samples with the same last speaking role as the new data set, which is the data set to be used in step 3.

2. The influence function interpretation method for sample-based sequence-to-sequence tasks in NLP according to claim 1, characterized in that In the step 1: selecting a dialogue generation task as a basic task; Select the autoregressive language model GPT-2 as the model to be tested; select the chat dialogue as the data set.

3. The influence function interpretation method for sample-based sequence-to-sequence tasks in NLP according to claim 1, characterized in that In step 2.3, the sliding window length is set to the length of 5 speeches.

4. The influence function interpretation method for sample-based sequence-to-sequence tasks in NLP according to claim 1, wherein in step 3: The steps to select some checkpoints include: Step 3.1: The influence of the training sample is the cumulative value of each visit to the training sample, and the checkpoints output during the model training process are used as the training process record instead; Step 3.2: Select a set of checkpoints that are close to the final model, that is, the parameters within the checkpoints output by the model i is the traversal of the selected checkpoints, and a small enough δ is selected so that Are approaching 5. The influence function interpretation method for sample-based sequence-to-sequence tasks in NLP according to claim 4, wherein in step 4: The sample loss is calculated as follows: 4.1: The steps to calculate the loss of training samples include: Step 4.1.1: When calculating the loss gradient at each checkpoint, for a training sample z, let the training sample sequence be represented as z = {t0, ..., t m }, t is each token in the sample; Step 4.1.2: Based on the parameters of the model to be tested The cross entropy loss function is used to calculate the loss of the token. The calculation method is as follows: Where C is the number of label categories, that is, the number of words in the vocabulary, p = [p0, ..., p C-1 ] is a probability distribution, each element p i Indicates t m The probability of belonging to category i, y = [y0, ..., y C-1 ], is t m The one hot representation of the label means that each element y i In t m The label of is 1 when it is category i, otherwise it is 0; Step 4.1.3: Add up the losses of each token in each training sample. The calculation method is as follows: 4.2: The steps to calculate the test sample loss include: Step 4.2.1: When calculating the loss gradient at each checkpoint, for the test sample z′, let the test sample sequence be represented as z′={t′0,...,t′ n }, t′ is each token in the sample; Step 4.2.2: Based on the parameters of the model to be tested The cross entropy loss function is used to calculate the loss of the token. The calculation method is as follows: Where C is the number of label categories, that is, the number of words in the vocabulary, p = [p0, ..., p C-1 ] is a probability distribution, each element p i Indicates t′ n The probability of belonging to category i, y = [y0, ..., y C-1 ], is t′ n The one hot representation of the label means that each element y i At t′ n The label of is 1 when it is category i, otherwise it is 0; Step 4.2.3: Add up the losses of each token in each test sample. The calculation method is as follows:

6. The influence function interpretation method for sample-based sequence-to-sequence tasks in NLP according to claim 5, wherein in step 5: The impact score is calculated as follows: Step 5.1: Based on parameters The loss gradient expression for calculating the sample is as follows: Step 5.2: According to the influence function TracInS formula, based on the parameters of each checkpoint According to step 5.1, calculate the test and training sample loss gradients, do a simple dot product, sum them up at each checkpoint, and calculate the influence score of the training sample z on the test sample z′:

Images