A machine translation model optimization method based on transformer model

Abstract

The invention discloses a machine translation model optimization method based on the Transformer model, which allows Transformer individuals to have different structures and parameters by designing variable-length codes and candidate blocks, and provides a variety of candidate models for word vector learning; and then designs cross mutation The strategy enables Transformer individuals to communicate with each other, so that the excellent structure or parameters for processing word vectors can be passed on to the next generation; then an environment selection strategy is designed to generate the next generation of Transformer individuals, eliminating models with relatively poor learning word vectors, and retaining the learned word vectors. The model with better vector effect; after that, iterative evolution search to find the Transformer model with the best word vector learning effect, which is used to finally solve the machine translation task, so that the Transformer model can better learn the word vector expression in the machine translation task, and improve the performance of the machine. Accuracy of the translation task.

Description

technical field [0001] The invention relates to the field of evolutionary computing, in particular to a method for optimizing a machine translation model based on a Transformer model. Background technique [0002] Transformer is a sequence-to-sequence proposed by Google in 2017 to solve machine translation tasks. Before Transformer was proposed, machine translation models can be divided into two categories: models based on feedback neural networks or convolution-based sequence regression Model. Most of the models based on the feedback neural network are composed of RNN or LSTM structure. The input of each layer in the model depends on the output state of the previous layer. The process requires a lot of time and computing resources; the convolution-based network model is composed of a multi-layer convolutional neural network, and the number of convolution operations in this model will increase rapidly when calculating the relationship of long-distance information, such as i...

AI Technical Summary

Problems solved by technology

Although the Transformer model has achieved good results in machine translation, there are still several problems: 1. The arrangement mode of the MHA layer and the FFN layer in the Transformer model with different network layers is fixed, and existing studies have shown that Transformer's different layer arrangement modes have better performance than the base Transformer model on other natural language processing tasks

3. The number of layers and hyperparameters of the Transformer model are set by experts combined with domain knowledge. If non-professionals want to use the Transformer model to solve machine translation tasks, it is difficult to independently design a model that meets expectations

Images