Sample-based sequence-to-sequence task influence function interpretation method in NLP

Abstract

A sample-based sequence-to-sequence task influence function interpretation method in NLP comprises the following steps: 1) selecting a sequence-to-sequence task needing to be interpreted, and selecting a to-be-tested natural language processing model and a data set for the sequence-to-sequence task; 2) encoding corpora in the data set, dividing each piece of encoded data into different intervals to obtain a plurality of pieces of data, and taking the plurality of pieces of data as new samples; 3) sending the new sample obtained in the step 2) into a to-be-tested model for training, and ending the training when the loss tends to be stable; in all check points output in the training process of the to-be-tested model, the check points approaching the final parameters of the to-be-tested model are selected to be used for influence score calculation; 4) calculating the loss of the training sample and the test sample based on the to-be-tested model parameters in the check points selected in the step 3); and 5) calculating a gradient vector based on a to-be-measured model parameter by using the sample loss obtained in the step 4), and calculating according to an influence function formula to obtain an influence score.

Description

technical field [0001] The present invention relates to the field of natural language processing (NLP), and more particularly, the present invention relates to a sample-based influence function interpretation method for sequence-to-sequence tasks. Background technique [0002] As complex models become an indispensable tool in many fields of artificial intelligence, there is growing interest in explaining how these "black-box" models work, and efforts to develop methods that can explain the model's decisions. However, the birth of deep learning has made the development of models more and more complex, and the scale of data sets has also increased day by day, which has challenged the ability of interpretation methods in terms of interpretability and fidelity. [0003] At present, considerable progress has been made in the interpretation methods of machine learning models. In the prior art, the influence function that explains the behavior of the model based on training samples...

AI Technical Summary

Problems solved by technology

But at the same time, the application of existing influence function methods in sequence-to-sequence tasks in the field of natural language processing (for example, dialogue generation, automatic summarization, etc.) is still blank, and many targeted problems still need to be solved adaptively.

First, following the original method of the influence function, IF (Influence Function), for modern natural language processing models and large-scale data sets, due to the inverse Hessian matrix calculation in the original formula, the application of the influence function may lead to excessive computational costs, the new The introduced TracIn method has no Hessian matrix calculation, but it may falsely explain the participation of suboptimal checkpoints and lead to being unfaithful to the model in question; secondly, the influence function is applied to the sequence-to-sequence task, and it is also necessary to solve a sample How to calculate multiple labels in the sequence; finally, for long natural language texts that are common in sequence-to-sequence tasks, the decision of the model may only depend on a specific part of the sample, and it is not accurate enough to use the entire original sample as the interpretation unit

[0006] Therefore, the influence function interpretation method for sample-based sequence-to-sequence tasks in NLP is still to be perfected

Images