Translation model training method and device

Abstract

The invention provides a translation model training method and device. The translation model comprises an encoder and a decoder. The encoder comprises n encoding layers which are connected in sequence, the decoder comprises n decoding layers which are connected in sequence, a self-attention sub-layer of the ith encoding layer and a self-attention sub-layer of the ith decoding layer share a self-attention parameter, n is greater than or equal to 1, and i is greater than or equal to 1 and less than or equal to n. The method comprises the following steps of: receiving a training statement and a target statement corresponding to the training statement; obtaining a training statement vector corresponding to the training statement and a target statement vector corresponding to the target statement; inputting the training statement vector into the encoder, and performing encoding processing to obtain an encoding vector; inputting the encoding vector and the target statement vector into the decoder, decoding the encoding vector and the target statement vector to obtain a decoding vector, and calculating a loss value according to the decoding vector; and adjusting parameters of the translation model according to the loss value.

Description

technical field [0001] The present application relates to the technical field of artificial intelligence, and in particular to a translation model training method and device, computing equipment, and a computer-readable storage medium. Background technique [0002] With the improvement of computer computing power, the application of neural networks is becoming more and more extensive, such as building translation models to realize the conversion of sentences to be translated into target sentences. [0003] The translation model is an end-to-end network structure, including an encoder and a decoder. The encoder includes multiple encoding layers, and the decoder includes multiple decoding layers. Each encoding layer includes a self-attention sublayer and a feedforward neural network. Network sub-layers, including self-attention sub-layer, encoding-decoding attention sub-layer and feed-forward neural network sub-layer in each decoding layer, each sub-layer has its own parameter...

AI Technical Summary

Problems solved by technology

The existing translation model has a complex structure, a large amount of parameters for each sub-layer, and a large model volume. When training the translation model, the training cycle is long and consumes high computing resources.

Images