Translation model optimization method for dynamically adjusting length punishment and translation length

Abstract

The invention discloses a translation model optimization method for dynamically adjusting length punishment and translation length. The method comprises the steps of obtaining standard data in a specified language direction as a standard bilingual data set for various index prediction; performing word segmentation operation on the standard bilingual data set, and performing further training to obtain a new training data set; modifying a neural machine translation model decoder part, and automatically predicting the optimal length punishment value of the current sentence pair; performing lengthstatistics to obtain a target statement sub-length; preparing an independent feedforward neural network model so that a translation finally predicted by the model tends to a translation result with the optimal length; and enabling the Transformer neural machine translation model to dynamically adjust the length penalty and the optimal translation sentence length for different sentences. Accordingto the method, the length punishment and the dynamic adjustment of the translation length in the model translation process are realized, the realization is simple, the method is effective, the practicability is high, and the model translation quality improvement effect is obvious.

Description

technical field [0001] The invention relates to the field of machine translation, in particular to a translation model optimization method for dynamically adjusting length penalty and translation length. Background technique [0002] In recent years, many natural language processing tasks constructed using neural network technology have achieved optimal results, such as neural machine translation. Machine translation models using neural network technology architecture, also known as neural machine translation. As a machine translation model with super learning ability, it often needs to use large-scale high-quality bilingual parallel corpus for training support. The neural machine translation model is actually a model structure that can automatically translate a sentence of a certain length in a certain language into a sentence of a certain length in another language through a computer. It is mainly composed of an encoder and a decoder. The role of the device is to encode ...

AI Technical Summary

Problems solved by technology

[0010] In view of the disadvantages of the existing method that the length penalty value in the training process of the neural machine translation model will have a certain degree of impact on the quality of the translated sentences generated by the model, the method of the present invention proposes a translation model optimization that dynamically adjusts the length penalty and the translation length method, so that the neural machine translation model can obtain the optimal length penalty value and the optimal translation sentence length value during the training process, and make these two values ​​​​participate in the model translation generation process

Images