Neural machine translation system training acceleration method based on stacking algorithm

Abstract

The invention discloses a training acceleration method of a deep neural machine translation system based on a stacking algorithm. The method comprises the following steps: constructing a coding end comprising a coding block, a decoding end and a preceding Transformer model; inputting sentences expressed by dense vectors into the coding end and the decoding end, and writing the input of the codingend into a memory network; writing the output vector into the memory network after completing the operation of each coding block, and accessing the memory network to perform linear aggregation to obtain the output of the current coding block; training a current model; copying the coding block parameters of the top layer to construct a new coding block and stacking the new coding block on the current coding end to construct a model containing two coding blocks; repeating the process to construct a neural machine translation system with a deeper coding end, and training the neural machine translation system to a target layer number until convergence; and performing translating by using the trained model. According to the method, the network with 48 coding layers can be trained, and the performance of the model is improved while 1.4 times of the speed-up ratio is obtained.

Description

technical field [0001] The invention relates to a neural machine translation technology, in particular to a training acceleration method for a neural machine translation system based on a stacking algorithm. Background technique [0002] Machine translation (English: Machine Translation, often abbreviated as MT) belongs to the category of computational linguistics, is an important branch of computational linguistics, and has very important scientific research value. It is the process of converting one language into another using a computer. At the same time, machine translation has important practical value. With the rapid development of economic globalization and the Internet, machine translation technology plays an increasingly critical role in promoting political, economic, and cultural exchanges. [0003] The development of machine translation technology has been closely following the development of computer technology, information theory, linguistics and other discipl...

AI Technical Summary

Problems solved by technology

[0010] In view of the shortcomings of existing deep neural machine translation system training, such as high equipment requirements, long training time and slow convergence speed, the technical problem to be solved by the present invention is to provide a training acceleration method for neural machine translation system based on stacking algorithm

Images