Semantic Parsing Training Data Augmentation With Logical Form Perturbation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep-learning systems for translating natural language to machine-oriented languages require large amounts of labeled training data, which are costly and difficult to obtain, especially for tasks like semantic parsing.
Innovation Solution
A method for augmenting training data by procedurally perturbing and generating new data points using linguistics-based rules to maintain semantic coherence, reducing the need for extensive labeled data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If deep-learning systems are used for translating natural language to machine-oriented languages, then the system can generalize across tasks and domains, but the system requires a large amount of labeled training data which is costly and difficult to obtain
Solution Approach 1:
The patent creates synthetic copies of existing training data by procedurally generating new data points from templates. The system takes existing natural language-to-logical-form pairs and generates variations by substituting entities, relationships, and values while maintaining semantic coherence, thereby creating additional training data without requiring human annotation for each example
Solution Approach 2:
The system modifies parameters of existing training data by varying entities, relationships, and values in procedurally generated examples. By changing these parameters systematically while preserving the underlying semantic structure, the system creates diverse training data that maintains generalization capability without requiring proportional increases in labeled data
2Reliability
If more training data is collected to improve model performance, then the model accuracy improves, but the cost and time required to obtain labeled data increases
Solution Approach 1:
The system performs self-service by automatically generating training data from existing data templates without requiring external human annotation. The procedural generation process uses predefined rules and templates to create new natural language-to-logical-form pairs, eliminating the time-consuming manual labeling process while maintaining data quality for model training
3Measurement precision
If human annotations are used to create training data, then the data quality is high, but the cost becomes expensive
Solution Approach 1:
Instead of paying humans to annotate data, the system copies and transforms existing data through procedural generation. It creates synthetic training examples by substituting entities, relationships, and values in templates, achieving high data quality at minimal cost since the process is automated and requires no human intervention
Solution Approach 2:
The system uses disposable procedural generation rules and templates to create training data, replacing the expensive human annotation process. The generation rules can be easily created, modified, and discarded without cost, providing an economical alternative to hiring annotators while maintaining sufficient data quality for model training
Data Source
AI summary
Techniques for augmenting training data include accessing training data comprising a plurality of training examples comprising a first training example comprising a first natural language utterance and a first logical form for the first natural language utterance. A second natural language utterance is generated by adding or replacing one or more values in the first natural language utterance. A logical form for the second natural language utterance is generated. A second training example is generated, comprising the second natural language utterance and the logical form for the second natural language utterance. The training data is augmented by adding the second training example to the plurality of training examples to generate an augmented training data set. A machine learning model is trained to generate logical forms for utterances using the augmented training data set.