Pseudo-Parallel Data Generation for Domain-Adaptive Machine Translation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural machine translation systems require large-scale parallel translation data sets for high-quality translation, which are costly and difficult to obtain, especially when adapting to new domains without existing data.
Innovation Solution
A method for generating pseudo parallel translation data using monolingual and parallel data sets from other domains to train a machine translation model, involving initialization and optimization steps, including auto-encoding, zero-shot round-trip translation, and supervised translation, to create reliable pseudo parallel data for new domains.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If large-scale parallel translation data sets are used to train neural machine translation systems, then translation performance is improved, but data acquisition cost and difficulty increase
Solution Approach 1:
The patent creates pseudo-parallel translation data by copying and transforming monolingual data from the target domain through machine translation and back-translation processes. Instead of requiring expensive human-translated parallel data, the system generates synthetic parallel data copies that mimic the statistical properties of real parallel corpora, thereby reducing data acquisition costs while maintaining translation performance.
Solution Approach 2:
The patent performs preliminary training of the machine translation model using available parallel data from other domains before adapting it to the target domain. This preliminary action establishes a baseline model that can then be fine-tuned with pseudo-parallel data generated from monolingual target domain data, reducing the need for extensive target domain parallel data while achieving domain adaptation.
2Adaptability or versatility
If parallel translation data sets from other domains are used for domain adaptation, then machine translation in target domain can be achieved, but translation accuracy for target domain decreases
Solution Approach 1:
The patent applies local quality by generating pseudo-parallel translation data specifically tailored to the target domain using monolingual data from that domain. Instead of using generic parallel data from other domains, the system creates domain-specific training data that preserves the linguistic characteristics and terminology of the target domain, thereby maintaining both adaptability and translation accuracy.
Solution Approach 2:
The patent changes the parameters of the training data by transforming monolingual target domain data into pseudo-parallel format through machine translation and back-translation. This parameter transformation allows the model to learn domain-specific patterns while maintaining the structural requirements for supervised training, thus improving translation accuracy for the target domain while preserving domain adaptation capability.
3Ease of manufacture
If monolingual data sets are used instead of parallel translation data sets, then data acquisition cost decreases, but machine translation performance deteriorates
Solution Approach 1:
The patent introduces an intermediary process that transforms monolingual data into pseudo-parallel translation data through machine translation and back-translation. This intermediary transformation allows the system to use inexpensive monolingual data while achieving training effects similar to parallel data, thereby maintaining machine translation performance while reducing data acquisition costs.
4Reliability
If pseudo parallel translation data is generated using initialization and optimization steps, then translation performance in target domain is improved, but training complexity increases
Solution Approach 1:
The patent segments the training process into distinct initialization and optimization steps, each with specific objectives and data requirements. The initialization phase uses available parallel data to establish baseline parameters, while the optimization phase generates and uses pseudo-parallel data for fine-tuning. This segmentation makes the complex training process more manageable and systematic, reducing the perceived complexity while maintaining high translation performance.
Data Source
AI summary
Provided is a pseudo parallel translation data generation apparatus for generating pseudo parallel translation data for accurately performing machine translation in an adaptation target domain even when there exists no parallel translation data for the adaptation target domain. Using other-domains parallel translation data D0(L1-L2), other-domains first language data D0(L1), other-domains second language D0(L2), adaptation target domain first language data D0(R1), and adaptation target domain second language data D0(R2), the pseudo parallel translation data generation apparatus 100 performs optimization processing for a cross-lingual language model including an input data embedding unit 2 and an XLM processing unit 3, and performs parameter optimization processing for a pseudo parallel translation data generation NMT model including the input data embedding unit after the optimization processing and a machine translation processing unit 5. Performing processing using the pseudo parallel translation data generation machine translation model obtained by the parameter optimization processing allows for obtaining pseudo parallel translation data for the adaptation target domain for which no parallel translation data sets exist.