Multi-field neural machine translation method based on self-attention mechanism

Abstract

The invention discloses a multi-field neural machine translation method based on a self-attention mechanism. The invention discloses a multi-field neural machine translation method based on a self-attention mechanism. The multi-field neural machine translation method comprises the following steps: carrying out two important changes on a Transformer, wherein the first change is a self-attention mechanism based on domain perception, and the domain representation is added to a key and a value vector of the original self-attention mechanism, the weight of the attention mechanism is the degree of correlation of the query and domain aware keys, the second change is to add a domain representation learning module to learn a domain vector. The method has the beneficial effects that a domain-aware NMT model architecture is provided on the basis of a neural network architecture Transformer representing the most advanced level at present. A self-attention mechanism based on domain awareness is provided for multi-domain translation. It is known that this is a first attempt on a multi-domain NMT based on a self-attention mechanism. Meanwhile, experiments and analysis also verify that the model can significantly improve the translation effect of each field and can learn the field information of training data.

Description

technical field [0001] The invention relates to the field of neural machine translation, in particular to a multi-domain neural machine translation method based on a self-attention mechanism. Background technique [0002] With the improvement of computer computing power and the application of big data, deep learning has been further applied. Neural Machine Translation based on deep learning has attracted more and more attention. In the field of NMT, the current state-of-the-art neural network architecture Transformer is an attention-based encoder-decoder model. The main idea is to encode the sentence to be translated (collectively referred to as 'source sentence' hereinafter) into a vector representation through an encoder, and then use a decoder to decode the vector representation of the source sentence and translate it into its Corresponding translations (collectively referred to as 'target sentences' hereinafter). [0003] Neural machine translation has made significan...

AI Technical Summary

Problems solved by technology

[0005] The first problem is a well-known problem in domain adaptation in MT: translation performance is limited by the matching between training and test data

[0006] The second problem is more practical: in the case of model inflexibility, multi-domain translation scenarios require multiple domain-specific system architectures, and each time a new domain is involved, a dedicated domain-specific data must be used to retrain a model, lacks architectural scalability and causes high maintenance costs

However, this solution has two obvious disadvantages: 1) domain-specific models can only be adapted to sentence inputs with known domain information, so that each sentence is processed by the corresponding domain model, and 2) each time involves a new domain, a dedicated model must be retrained using domain-specific data

However, in real-world application scenarios, translation requests rarely carry domain information, the concept of domain itself is vague, and domain-specific data is difficult to obtain

Images