NL2LF Model Robustness Through Keyword and Template Augmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep-learning based natural language to logical form (NL2LF) models require a large amount of training data and struggle with high error rates when encountering underrepresented keywords or phrasings, necessitating costly and time-consuming manual data curation.
Innovation Solution
A framework is developed to improve model robustness by generating additional training examples through keyword augmentation and synthetic template-based methods, enhancing the training data set to include underrepresented terms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If deep-learning models are trained with limited training data, then training time and computational resources are reduced, but model robustness and accuracy deteriorate
Solution Approach 1:
The patent creates synthetic copies of training examples by applying data augmentation techniques. Templates with placeholder variables are instantiated multiple times with different parameter values to generate diverse training samples. This allows the model to learn from synthesized data copies rather than requiring extensive manual data collection, thereby improving model robustness while maintaining manageable data volumes.
Solution Approach 2:
The patent systematically varies parameters in training data templates to generate diverse examples. By changing parameters such as entity names, relationships, and query structures within templates, the system creates parameter-varied training instances that improve model generalization without requiring proportional increases in manual data collection effort.
2Reliability
If manual data curation is performed to improve training data quality, then model accuracy improves, but time and cost increase
Solution Approach 1:
Instead of manually curating diverse training examples, the patent uses templates to automatically generate synthetic copies of training data. This automated template instantiation process replaces time-consuming manual data curation while maintaining data quality through structured template design and controlled parameter variation.
Solution Approach 2:
The system performs self-service data generation by automatically instantiating templates with appropriate parameters. Rather than requiring manual intervention to create training examples, the system autonomously generates diverse training data through template-based synthesis, significantly reducing the time and human resources required for data curation.
3Adaptability or versatility
If the model is trained to handle diverse keywords and phrasings, then coverage of user intents improves, but model complexity increases
Solution Approach 1:
The patent employs universal templates that can generate multiple types of training examples through parameter substitution. A single template structure serves multiple functions by instantiating different entity types, relationships, and query patterns, allowing the model to learn diverse intent representations without requiring separate specialized components for each keyword or phrasing variation.
Data Source
AI summary
Techniques are disclosed herein for improving model robustness on operators and triggering keywords in natural language to a meaning representation language system. The techniques include augmenting an original set of training data for a target robustness bucket by leveraging a combination of two training data generation techniques: (1) modification of existing training examples and (2) synthetic template-based example generation. The resulting set of augmented data examples from the two training data generation techniques are appended to the original set of training data to generate an augmented training data set and the augmented training data set is used to train a machine learning model to generate logical forms for utterances.