Model parameter importance degree evaluation method and device and electronic equipment

Abstract

The embodiment of the invention provides a model parameter importance degree evaluation method and device and electronic equipment, and relates to the technical field of computers. The method comprises the steps of training a neural machine translation NMT model based on a training data set, and obtaining parameter value variation of each model parameter before and after each training; sampling the training data set to obtain a sampling data set; determining the gradient of the loss function of the NMT model corresponding to each time of training relative to each model parameter based on the sampling data set; and determining the importance degree of each model parameter based on the gradient and parameter value variation corresponding to each training of each model parameter. According tothe technical scheme, the gradient of each model parameter is determined based on the sampling data set, so that the data calculation amount is reduced; based on the gradient of each model parameterand the parameter value variation before and after training, the importance degree of each model parameter is determined, and the contribution of each model parameter in the loss function convergenceprocess can be clarified.

Description

technical field [0001] The present application relates to the field of computer technology, and in particular, the present application relates to a method, device and electronic equipment for evaluating the importance of model parameters. Background technique [0002] Since the neural machine translation (Neural Machine Translation, NMT) model was proposed, it has rapidly become the focus of translation research. Not only can the NMT model produce impressive translation results, but it is also inherently advantageous in its translation model structure. Compared with traditional statistical machine translation, its model can model language model, translation model and alignment model in a unified way instead of a pipelined way, which can reduce the side effects of error accumulation. [0003] As a complex neural network, the NMT model can have up to 108 million model parameters in the entire network, and all model parameters must be iteratively trained until convergence. Ho...

AI Technical Summary

Problems solved by technology

However, the current NMT model is trained as a black box, and it is impossible to understand the role of each model parameter in the loss function during the training process. This increases the difficulty of understanding the model training mechanism, which is not conducive to further targeted Improve model performance

Images