Scarce resource neural machine translation training method based on pre-training

Abstract

The invention discloses a scarce resource neural machine translation training method based on pre-training, which comprises the following steps of: constructing massive monolingual corpora, and performing word segmentation and sub-word segmentation preprocessing flows to obtain converged model parameters; constructing parallel corpora, randomly initializing parameters of a neural machine translation model, and enabling the sizes of a word embedding layer and a hidden layer of the neural machine translation model to be the same as that of a pre-trained language model; integrating the pre-training model into a neural machine translation model; training the neural machine translation model through the parallel corpora, so that a generated target statement is more similar to a real translationresult, and the training process of the neural machine translation model is completed; and sending the source statement input by the user into a neural machine translation model, and generating a translation result by the neural machine translation model through greedy search or bundle search. According to the method, knowledge in the monolingual data is fully utilized, and compared with a randomly initialized neural machine translation model, the translation performance can be obviously improved.

Description

technical field [0001] The invention relates to a neural machine translation training method, in particular to a pre-training-based scarce resource neural machine translation training method. Background technique [0002] Nowadays, neural machine translation technology has made great progress. Compared with previous rule-based and statistical-based machine translation models, neural machine translation can achieve better translation quality and more fluent translation results. However, a problem with neural machine translation is that it is extremely dependent on data. If there is enough training data, that is, bilingual corpus of source language and target language, then the translation model can achieve better translation quality, but in training In the case of scarce data, the translation model cannot achieve the desired effect. Scarce resource scenarios can generally be divided into language data scarcity and domain data scarcity. Languages ​​with sufficient bilingual ...

AI Technical Summary

Problems solved by technology

[0008] In view of the lack of sufficient bilingual corpus in many actual scenarios in the machine translation system in the prior art, which leads to the poor translation effect of the neural machine translation model, the technical problem to be solved by the present invention is to provide a pre-trained neural machine based on scarce resources. The translation training method can make full use of monolingual corpus pre-training to shield the language model and other tasks in the case of insufficient bilingual corpus, and then integrate the information extracted by the pre-training model into the neural machine translation model to significantly improve the translation quality of the model

Images