Neural machine translation inference acceleration method based on attention mechanism

Abstract

The invention discloses a neural machine translation inference acceleration method based on an attention mechanism. The neural machine translation inference acceleration method comprises the steps: constructing and training a training parallel corpus and a multilayer neural machine translation model based on the attention mechanism to obtain model parameters after training convergence; calculatingthe parameter similarity between any two layers of the decoding end self-attention, the coding end self-attention and the coding and decoding attention in the model; if the similarity between the upper layers and the bottom layers of the coding end and the decoding end is higher than a threshold value, enabling the upper layers to directly use attention weight parameters of the bottom layers forcalculation; if the similarity between the upper layer and the bottom layer is higher than the threshold value, enabling the upper layer to directly use the attention calculation result of the bottomlayer; and inputting the vocabulary into the model for calculation to obtain probability distribution of the machine translation vocabulary, and selecting the highest word as a translation result. Onthe basis of the latest implementation of rapid reasoning, the neural machine translation inference acceleration method can averagely obtain 1.3 times of acceleration ratio, and meanwhile, hardly reduces the model performance.

Description

technical field [0001] The invention relates to a neural machine translation inference acceleration technology, in particular to an attention mechanism-based neural machine translation inference acceleration method. Background technique [0002] Machine translation (Machine Translation or MT) is an experimental subject that uses computers to translate between natural languages. In layman's terms, it is the process of using a computer to convert a natural language (source language) into another natural language (target language). For a long time, machine translation has been regarded as one of the ultimate technical means to solve translation problems, and the application demand is very strong. For example, the Chinese government includes research on natural language understanding, including machine translation technology, in the national medium and long-term scientific and technological development plan; it is reported that Google Translate provides services to more than 20...

AI Technical Summary

Problems solved by technology

At the same time, in the neural machine translation system based on the self-attention mechanism, due to the frequent attention alignment operations within the sentence and between sentences during the calculation process, this method of machine translation is more likely to be used in practice. The decoding speed in use is difficult to meet the needs of real-time response

Images