Neural machine translation system based on Roman Uygur language

Abstract

The invention discloses a neural machine translation system based on Romatized Uygur language, which provides bilingual data with segmented words and proper format for subsequent word alignment processing, for Chinese-Uygur language before constructing a translation system; according to the preprocessing of the system, different preprocessing processes are carried out on corpora according to different characteristics of the corpora, namely Jieba word segmentation is carried out on Chinese corpora, words are continuously cut by using BPE coding, root + affix form word segmentation is firstly carried out on Uygur language corpora, then Roman processing is carried out on Uygur language after word segmentation, and finally processing is carried out by using BPE coding; a translation model is trained by using a Tansformer translation model to obtain a final translation model; for a translation process, firstly, Uygur language is also preprocessed, namely, word segmentation and Roman processing of'word roots + affixes' are carried out, then the processed corpus is translated by using the trained model, and finally translated Chinese is obtained.

Description

technical field [0001] The invention relates to the technical field of machine translation, and mainly relates to a neural machine translation system based on Romanized Uighur language. Background technique [0002] Machine translation is a technology that uses computers to automatically convert a source language into a target language. Currently commonly used translation methods are statistical machine translation and neural machine translation. Neural Machine Translation (NMT) has achieved impressive results in previous years, outperforming traditional Phrase-Based Statistical Machine Translation (PBSMT) methods. State-of-the-art NMT systems rely on encoder-decoder architectures and introduce attention mechanisms to simulate word alignment; the model then encodes the source sentence into a fixed-length vector, which is then decoded word-by-word to output the target string from the vector representation. [0003] NMT and Statistical Machine Translation (SMT) models are se...

AI Technical Summary

Problems solved by technology

[0010] 2) For languages ​​with high deformation and morphological changes (such as Uyghur), BPE cannot correctly segment sentences, thereby reducing translation quality;

[0011] 3) There are few studies on Chinese-Uyghur neural machine translation

Images