Multilingual Semantic Parsing with Active Learning Translation Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multilingual semantic parsers face challenges due to data imbalance and low quality of machine-translated utterances, leading to performance degradation in low-resource languages, as human translation is costly and machine translation introduces errors and biases.
Innovation Solution
An active learning approach that iteratively selects and incorporates human translations into machine-translated training data using acquisition functions to mitigate translation bias and errors, enhancing the parser's performance by leveraging both human and machine translations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If machine translation is used to generate training data, then annotation cost is reduced, but translation quality deteriorates due to errors and biases
Solution Approach 1:
The patent segments the training data into machine-translated portions and human-translated portions. The acquisition function selectively identifies which utterances require human translation versus those that can rely on machine translation, creating a segmented approach that optimizes both cost and quality
Solution Approach 2:
The patent implements feedback through the acquisition function that evaluates translation quality metrics and uses this feedback to determine which utterances need human translation. The system continuously monitors translation quality and adjusts human intervention accordingly
2Reliability
If human translation is used for all utterances, then translation quality is improved, but annotation cost and time consumption increase significantly
Solution Approach 1:
The patent applies partial action by having human translators work on only a subset of utterances identified by the acquisition function as needing human translation. This partial approach maintains quality for critical cases while preserving overall productivity
Solution Approach 2:
The system dynamically changes the parameter of human translation coverage based on the acquisition function's evaluation. The proportion of human-translated utterances adjusts according to the measured quality metrics and resource constraints
3Quantity of substance
If machine-translated data is used for low-resource languages, then data availability is improved, but parsing accuracy deteriorates due to lexical distribution discrepancy
Solution Approach 1:
The patent applies local quality by ensuring that human translation is applied locally to specific utterances where lexical distribution accuracy is critical. The acquisition function identifies regions of the data space that require higher quality translations to maintain lexical distribution consistency
Solution Approach 2:
The training dataset becomes a composite of machine-translated and human-translated utterances. This composite approach leverages the quantity advantage of machine translation while incorporating the quality advantage of human translation in strategic portions
Data Source
AI summary
Described is a system and method for training a multilingual semantic parser. A method includes receiving, by a multilingual semantic parser, a multilingual training dataset, wherein the multilingual training dataset includes pairs of utterances and meaning representations from at least one high-resource language and at least one low-resource language and wherein the multilingual training dataset is initially a machine-translated dataset, training, the multilingual semantic parser, by translating the utterances in the multilingual training dataset to a target language; and iteratively performing selecting, by an acquisition functions estimator, a subset of the multilingual training dataset for human translation, updating the multilingual training dataset with the human-translated subset of the multilingual training dataset with, and retraining, the multilingual semantic parser, with the updated multilingual training dataset.