Multi-mechanism attention merging multi-path neural machine translation method

Abstract

The invention relates to a multi-mechanism attention merging multi-path neural machine translation method, and belongs to the field of natural language processing. According to the invention, a CNN translation mechanism, a Transformer translation mechanism and a Tree-Transformer translation mechanism independently generate attention values of themselves, weighted accumulation is carried out on thecalculated attention values, then alignment and normalization are carried out to form new attention values, the new attention values are transmitted to a Dec-Enc attention layer of a decoder, and allthe translation mechanisms complete the subsequent machine translation process to obtain a decoded key-value matrix. And weighted stacking and normalization are performed by adopting a decoding key-value matrix generated by each mechanism, and thus generating a target translation through a linear transformation layer and a softmax layer. According to the multi-mechanism attention superposition and normalization process, the analysis capability of multiple algorithms can be effectively integrated, the formed attention is closer to the real attention of the theory, so that a better translationeffect is obtained, and the translation accuracy can be effectively improved.

Description

technical field [0001] The invention relates to a multi-path neural machine translation method with multi-mechanism combined attention, and belongs to the field of natural language processing. Background technique [0002] Machine translation refers to the process of using computers to translate sentences in one language (sentences in the source language) into sentences in another language (sentences in the target language) with the same meaning, and has become an important research direction in the field of artificial intelligence. [0003] In the prior art, Gehring et al. proposed a CNN translation mechanism to realize machine translation, which fully utilizes a convolutional neural network to realize machine translation, and uses a convolutional neural network as a working unit of an encoder and a decoder, wherein the encoder and decoder All devices are composed of multi-layer convolutional neural network stacks. On the encoding side, the input sequence is encoded using ...

AI Technical Summary

Problems solved by technology

Under different attention generation mechanisms, the calculation results of their attention are inconsistent, and the attention generated by any single mechanism cannot fully and accurately reflect the theoretical attention in language

Images