Deep neural machine translation method based on dynamic linear aggregation

Abstract

The invention discloses a deep neural machine translation method based on dynamic linear aggregation. Based on a Transformer model, a memory network is added to a coding end and a decoding end of theTransformer model at the same time to store the output of a previous middle lamination. The memory network is accessed before the next laminated network calculation. A linear multi-step method based on an ordinary differential equation aggregates dense vectors stored in a memory network and semantic vectors of all previous laminates to obtain a hidden layer representation for aggregating characteristics of each layer, and the hidden layer representation obtains a semantic vector of which parameters obey standard regularization as input of a next lamination through layer regularization operation. In this way, the feature vectors extracted by all previous laminated networks are fully considered when the current laminated network is calculated, so that a deeper model is trained to improve therepresentation ability of the model, and the performance of machine translation is improved.

Description

technical field [0001] The invention belongs to the technical field of machine translation and relates to a deep neural machine translation method based on dynamic linear aggregation. Background technique [0002] Neural machine translation technology usually uses a neural network-based encoder-decoder framework to model the entire translation process end-to-end, and learns the mapping relationship between the source language and the target language through a combination of linear and nonlinear functions. The Transformer model with self-attention mechanism has achieved the best translation performance in multiple languages. Among them, the encoder is responsible for encoding the input source language sentences into dense semantic vectors, and the decoder will generate translation results corresponding to the source language through the semantic vectors. Usually, the input sentence is segmented to obtain multiple word fragments, and then the word fragments are converted into...

AI Technical Summary

Problems solved by technology

[0006] In order to solve the above technical problems, the present invention provides a deep neural machine translation method based on dynamic linear aggregation, to solve the problem of neural machine translation model Transformer in the prior art in training The underlying network cannot be fully trained in the deep network, resulting in the inability to effectively train the deeper network structure, thus failing to further improve translation performance

Images