Neural machine translation method based on pre-training bilingual word vector

Abstract

The invention discloses a neural machine translation method based on a pre-training bilingual word vector. The method comprises the following steps of: performing'source language-target language 'splicing on labeled and aligned parallel corpora as input of an XLM (X-Language Model) for pre-training; training: taking a bilingual word vector matrix obtained by pre-training to initialize a translation model; inputting a source language into an encoder, inputting vector representation of source language encoding and a corresponding target language into a decoder, outputting a prediction sequence, comparing the prediction sequence with a corresponding target sequence, calculating a loss value, and inputting the loss value into an optimizer to optimize translation model parameters; predicting: in a certain time step, inputting a source language into an optimized encoder, outputting corresponding vector representation by the encoder, inputting the vector representation and a target language word translated by the previous time step into a decoder, outputting a target word of the time step by the decoder, splicing the target words translated by different time steps according to a time sequence, and obtaining a source language translation result. Machine translation effect of low-resource languages is improved.

Description

technical field [0001] The invention relates to the technical field of neural machine translation, in particular to a neural machine translation method based on deep learning and bilingual word vectors. Background technique [0002] Neural machine translation (Neural machine translation, NMT) is a machine translation technology that introduces artificial neural networks for translation. Compared with traditional statistical machine translation (Statistical Machine Translation, SMT), neural machine translation uses an end-to-end "encoder-decoder" architecture. Specifically, this architecture can be divided into the following three categories: [0003] (1) Neural machine translation model based on recurrent neural network. This model uses Long Short-Term Memory (LSTM) or Gated Recurrent Units (GRU) as the unit to build the "encoder-decoder" architecture. At the same time, it introduces an attention mechanism, so that when each target word is generated, the decoder can focus...

AI Technical Summary

Problems solved by technology

[0009] (2) The word vector matrix of the neural machine translation model is often initialized randomly, and as the training process progresses, the parameters in the word vector matrix are learned and updated. The initial quality of the word embedding matrix is ​​poor and it is easier to overfit during the training update process

Images