Neural Machine Translation Encoder Training with Pseudo Parallel Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing neural machine translation using encoder/decoder architecture faces challenges in enhancing encoder accuracy due to training with pseudo-original texts containing errors, which limits the overall translation accuracy.
Innovation Solution
A pseudo parallel translation data generation apparatus and method that includes a back-translation unit and a pseudo parallel translation data obtaining unit, which generate diverse pseudo parallel data by back-translating target language monolingual data and pairing it with the original data, and a machine translation processing apparatus that uses these data with adjusted learning rates to enhance encoder performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If pseudo parallel translation data is generated by back-translating target language monolingual corpus, then the quantity of training data is increased, but the accuracy of encoder is degraded due to errors in pseudo-original texts
Solution Approach 1:
The training process is segmented into two distinct phases: first training the decoder using pseudo parallel data generated from target language monolingual corpus, then training the encoder using accurate source-language parallel corpus. This segmentation allows each component to be trained on appropriate data types, preventing error propagation to the encoder
Solution Approach 2:
The decoder is preliminarily trained using pseudo parallel translation data before encoder training begins. This preliminary action allows the decoder to learn from diverse pseudo data while the encoder is subsequently trained on high-quality parallel data, ensuring encoder accuracy is not compromised
2Measurement precision
If only accurate parallel corpus is used for training, then the encoder accuracy is maintained, but the diversity of training data is insufficient
Solution Approach 1:
The training corpus is segmented into two types: accurate source-language parallel corpus for encoder training, and diverse pseudo parallel corpus (generated from target language monolingual data) for decoder training. This segmentation allows the system to leverage both accuracy and diversity without contamination
Solution Approach 2:
Different quality requirements are applied locally to different components: the encoder requires high-quality accurate parallel data for precise source language understanding, while the decoder can utilize diverse pseudo parallel data for learning target language generation, with each component receiving data optimized for its specific function
3Quantity of substance
If pseudo parallel translation data is generated using conventional methods, then training data is obtained, but the diversity of pseudo source language data is limited
Solution Approach 1:
The back-translation process is made dynamic by performing multiple back-translations of the same target language sentence and selecting diverse results. This dynamic approach increases the variety of pseudo source language data generated, providing more diverse training examples for the decoder
Data Source
AI summary
Provided are a model training method for neural machine translation that enhances an encoder using a monolingual corpus of a target language and improves the accuracy of the entire translator, and a machine translation system for performing the model training method. The machine translation system 1000 uses a monolingual corpus of the target language to obtain multiple pieces of pseudo source language data, thus allowing for obtaining a large amount of pseudo parallel corpus data having diversity. Further, the machine translation system 1000 uses both the pseudo parallel corpus data having diversity, which has been obtained in large quantities, and the base parallel corpus data in a small quantity but with high accuracy, with the applied learning rates changed accordingly, to perform the learning process (training process) for the machine translation model. This allows the machine translation system 1000 to obtain a learned model (machine translation model) with very high accuracy.