Multi-Model Denoising for Scarce-Language NLU Training Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Natural Language Understanding (NLU) tasks for scarce-resource languages face performance limitations due to the lack of reliable training data, leading to poor-quality training samples synthesized through translation or generation methods.
Innovation Solution
A multi-model joint denoising approach that utilizes multiple models to denoise training samples synthesized through both translation and generation methods, employing mechanisms like co-training, weight determination, and label updating to improve sample quality and model robustness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If training data is synthesized through translation or generation methods for scarce-resource languages, then the quantity of training data increases, but the quality and reliability of training samples deteriorates
Solution Approach 1:
The patent introduces an uncertainty estimation mechanism as an intermediary between the synthesized training data and the NLU model training process. This mechanism estimates the reliability of each synthesized training sample and uses this information to weight or filter samples during training, thereby maintaining high quality while utilizing large quantities of synthesized data.
Solution Approach 2:
The patent changes the parameter of sample weighting by introducing uncertainty estimates. Instead of treating all synthesized training samples equally, the system dynamically adjusts the weight of each sample based on its estimated uncertainty, allowing the model to learn more from high-quality samples while still benefiting from the overall increased data quantity.
2Reliability
If multiple models are used for joint denoising training, then the quality of training samples improves, but the device complexity increases
Solution Approach 1:
The patent segments the complex multi-model system into distinct functional components: uncertainty estimation modules for each model, a coordination mechanism for joint training, and a sample weighting system. This segmentation makes the complex system more manageable and allows each component to be optimized independently while working together to improve training sample quality.
Data Source
AI summary
The present disclosure proposes a method and apparatus for multi-model joint denoising training. Multiple models may be obtained. A set of training samples may be denoised through the multiple models. The multiple models may be trained with the set of denoised training samples.