Sentence translation model training method and device, sentence translation method and device

Abstract

The present application provides a sentence translation model training method and device, a sentence translation method and device, wherein the sentence translation model includes an encoder and a decoder, and the sentence translation model training method includes: obtaining the sentence to be translated and the sentence to be translated The target sentence corresponding to the sentence, the vector to be translated is obtained according to the sentence to be translated and input to the encoder, the coding vector output by each coding layer is obtained, the target vector is obtained according to the target sentence and input to the decoder A device; input the coding vector output by each coding layer to each of the decoding layers, and obtain the decoding vector corresponding to each coding vector output by each of the decoding layers in the decoder; according to each Calculate corresponding loss values ​​for each of the decoding vectors and the target vector, and determine the target loss value; adjust the parameters of the sentence translation model according to the target loss value to train the sentence translation model.

Description

technical field [0001] The present application relates to the field of Internet technologies, and in particular to a sentence translation model training method and device, a sentence translation method and device, computing equipment, and a computer-readable storage medium. Background technique [0002] With the improvement of Internet technology, the application of neural networks is becoming more and more extensive, such as building neural machine translation models to realize the conversion of sentences to be translated into target sentences. The neural machine translation model is an end-to-end network structure, and the architecture includes: encoder (encoder)-decoder (decoder). The encoder encodes the input information to obtain the encoding result, and then the decoder simultaneously receives the input of the decoder and the encoding result of the encoder for decoding, and inputs the decoded information into the linear layer, normalizes the layer, and finally obtains ...

AI Technical Summary

Problems solved by technology

[0003] At present, when the end-to-end neural machine translation model builds a loss function during the training process, the encoder and decoder only rely on the output of the top layer of the encoder in the subsequent processing, and update the model parameters through backward propagation at each step. The encoder , The parameters of all layers of the decoder will be updated. In fact, in the process of backpropagation, the gradients of all parameters of the model are quite different. The parameters with larger gradients should be updated less frequently, and the parameters with smaller gradients should be updated less frequently. It should be larger, but in the current model update parameter method, the parameters of all layers of the encoder and decoder will be updated, making the training speed of the model slower, which is not conducive to the rapid training of the model

Images