Training method and training system for non-autoregressive machine translation model based on task-level curriculum learning

Abstract

The invention discloses a training method and a training system of a non-autoregressive machine translation model based on task level curriculum type learning, and belongs to the field of non-autoregressive machine translation. In the present invention, the method comprises the following steps: firstly, establishing a Transformer-based machine translation model; replacing a multi-head self-attention mechanism in the Transformer model decoder with a causal-k self-attention mechanism according to the multi-head self-attention mechanism of the Transformer model decoder; obtaining a TCL-NAT model;then, adjusting a parameter k in a causal-k self-attention mechanism; dividing the training process into AT training stages (k=1) in sequence; an SAT training stage (1<k<N) and an NAT training stage(k=N), introducing a task window concept in an SAT training stage, and simultaneously training a plurality of tasks with different parallelism degrees in the same stage, so that the model can stably transit from one training stage to another training stage, and the accuracy of the non-autoregressive machine translation model is effectively improved.

Description

technical field [0001] The invention relates to the field of non-autoregressive machine translation, in particular to a training method and system for a non-autoregressive machine translation model based on task-level curriculum learning. Background technique [0002] In recent years, neural machine translation (NMT) has developed rapidly. Usually, NMT adopts the encoder-decoder framework. At present, the mainstream method for the decoder to generate the target sentence is the autoregressive method. The characteristic of the autoregressive method is that the generation of the current word depends on the prediction results of the previous words and the source context from the encoder. . Although the accuracy of NMT using the autoregressive method has reached the human level, because the autoregressive method must be translated word by word, that is, in the reasoning process, the following words must wait for all the previous words to be inferred before reasoning can be perfo...

AI Technical Summary

Problems solved by technology

[0005] In order to solve the problem of low accuracy of the existing non-autoregressive machine translation model, the present invention starts with the training method of the model, and proposes a training method and training system of a non-autoregressive machine translation model based on task-level curriculum learning , the present invention uses a task-level course-based learning method to gradually transfer the model from AT to NAT

Images