Machine translation engine training system and method and trained machine translation engine

Abstract

The invention discloses a machine translation engine training system and method and a trained machine translation engine. The machine translation engine training system comprises a preprocessing replacement module, a replacement information storage module, a sub-word segmentation keeping module, a translation engine training module and a replacement information restoration module. Wherein the preprocessing replacement module is used for replacing received bilingual sentence pairs or label pairs in an original text to be translated with special label pairs, and only the label pairs with the same label content are replaced with the same special label pairs. And the replacement information restoration module restores the special mark in the received translation in the training effect verification mode or when the engine executes the translation task so as to evaluate the training effect of the machine translation engine or output the final translation. According to the method and the device, the processing capability of the machine translation engine on formatted information, especially tags, in the translated text can be effectively improved, so that the machine translation engine can process superscript information and subscript information in the translated file with high reliability.

Description

Technical field [0001] The present invention relates to the field of computer technology, specifically to machine translation engine training systems, methods and trained machine translation engines. Background [0002] Current mainstream machine translation engines, usually built on neural network models, use the Encoder-Decoder framework. The input of the encoder module is a matrix composed of word vectors of words (sub-words) in the sentence, and the matrix undergoes a series of transformations to output a new matrix representing the entire sentence information, and each vector in the new matrix is the encoding of each word in the original text and its context information. The decoder module receives the original vector encoded by the encoder module, combines the vector of the translated word that the decoder has output, calculates the vector representing the next translation word, and finally determines the next word through the softmax (normalization index) layer. [0003] ...

AI Technical Summary

Problems solved by technology

[0004] However, on the one hand, subword segmentation may separate tags, which may easily lead to translation errors when machine translation engines translate complex sentences; on the other hand, adding tags to the original text actually adds It is easy to cause interference to the machine translation engine (especially the machine translation engine without special training in this area), and the label may be translated correctly, but due to the influence of the label information, the translation quality of other content in the translation may decline

Images