Mongolian-Chinese machine translation system based on byte pair coding technology

Abstract

The invention discloses a Mongolian-Chinese machine translation system based on a byte pair coding technology. Firstly, English-Chinese parallel corpora and Mongolian-Chinese parallel corpora are preprocessed through the BPE technology, English words, Mongolian words and Chinese words are all divided into single characters, then the occurrence frequency of character pairs is counted within the range of the words, and the character pair with the highest occurrence frequency is stored every time till the circulation frequency is finished. Secondly, the preprocessed English-Chinese parallel corpus is used for training based on a neural machine translation framework; and then, the preprocessed translation model parameter weight trained by the English-Chinese parallel corpus is migrated into aMongolian-Chinese neural machine translation framework, and a neural machine translation model is trained by utilizing the preprocessed Mongolian-Chinese parallel corpus to obtain a Mongolian-Chineseneural machine translation prototype system based on a byte pair coding technology. And finally, the BLEU value of the translation of the system and the BLEU value of the translation of the statistical machine are compared and evaluated to achieve the purpose of finally improving the translation performance of the Mongolian-Chinese machine.

Description

technical field [0001] The invention belongs to the technical field of neural machine translation, in particular to a Mongolian-Chinese machine translation system based on byte pair coding technology. Background technique [0002] Machine translation refers to the process of using computers to automatically convert one natural language into another natural language with exactly the same meaning. With the rapid development of economic globalization and the Internet, machine translation technology plays an increasingly important role in promoting political, economic, and cultural exchanges. Neural machine translation adopts a novel system for solving machine translation problems, and it has developed rapidly in recent years and achieved many important results. Especially in terms of fluency and accuracy of translation, neural machine translation has smoother translation results than traditional statistical machine translation. [0003] However, neural machine translation als...

AI Technical Summary

Problems solved by technology

This greatly preserves the structural features and fluency of sentences, reduces the number of unregistered words in Mongolian in the Mongolian-Chinese translation, and solves the problem of a large number of unregistered words in Mongolian

Images