Non-autoregressive neural machine translation method based on auxiliary representation fusion

Abstract

The invention discloses a non-autoregressive neural machine translation method based on auxiliary representation fusion. The method comprises the following steps: constructing an autoregressive neural machine translation model; building a training parallel corpus, and training a model with only one layer of decoder; constructing a non-autoregressive neural machine translation model; carrying out weighted fusion on the output after the feedforward neural network on the topmost layer of the autoregressive neural machine translation model decoder and the top layer representation of the non-autoregressive neural machine translation model encoder, and taking the fusion result as the input of the non-autoregressive neural machine translation model decoder; enabling the encoder to extract source statement sub-information, and enabling the decoder to predict a corresponding target statement according to the source statement sub-information; completing the training of a non-autoregressive neural machine translation model; and sending the source statements into the non-autoregressive neural machine translation model, and decoding translation results with different lengths. According to the method, the advantages of the autoregression model and the non-autoregression model are combined, and the speed can be increased by 7-9 times under the condition of low performance loss.

Description

technical field [0001] The invention relates to a neural machine translation inference acceleration method, in particular to a non-autoregressive neural machine translation method based on auxiliary representation fusion. Background technique [0002] Machine translation is the technique of translating one natural language into another. Machine translation is a branch of natural language processing, one of the ultimate goals of artificial intelligence, and has important scientific research value. At the same time, with the rapid development of Internet technology, machine translation technology has played an increasingly important role in people's daily life and work. [0003] From the rule-based method in the 1970s to the instance-based method in the 1980s, the statistical method in the 1990s, and the neural network-based method today, machine translation technology has achieved good results after years of development. more widely used in daily life. [0004] At present,...

AI Technical Summary

Problems solved by technology

[0008] Aiming at the problem of translation quality degradation due to weak target information in the non-autoregressive machine translation model, the present invention provides a non-autoregressive neural machine translation method based on auxiliary representation fusion, which can make the non-autoregressive machine translation obtain the same level as the automatic neural machine translation. Return to machine translation equivalent performance with higher response speed and better practical application

Images