NL2SQL Long-Context Fine-Tuning with Schema Augmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing large language models (LLMs) struggle with effectively handling long contexts in natural language to SQL (NL2SQL) tasks, leading to degraded performance when context lengths increase, making it challenging for non-technical users to interact efficiently with databases.
Innovation Solution
A data augmentation method is employed to create long-context benchmarking and fine-tuning datasets, enhancing LLMs by iteratively combining and shuffling database schemas, thereby improving their context length, robustness, and generalization in SQL generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional LLMs are used for NL2SQL tasks, then the model structure is simple and training is straightforward, but the effective context length is limited and performance degrades with increased context
Solution Approach 1:
The patent applies preliminary action by pre-processing training data to create augmented datasets with varied context lengths before model training. The method generates synthetic training examples with different context configurations in advance, allowing the model to be pre-exposed to long-context scenarios during training, thereby improving its ability to handle varying context lengths without architectural modifications.
2Adaptability or versatility
If the database schema is extended to include more tables for long context training, then the model's context length capability improves, but the training data complexity and processing difficulty increase
Solution Approach 1:
The patent applies segmentation by dividing the database schema into multiple independent table components. Instead of treating the entire schema as a single complex structure, the method segments tables into separate entities that can be independently selected, shuffled, and combined during data augmentation. This segmentation simplifies the training data structure by allowing modular assembly of context examples from discrete table units.
Solution Approach 2:
The patent applies parameter changes by dynamically varying the context length parameter during data augmentation. The method generates training examples with different numbers of tables (context lengths) by sampling from the segmented schema, thereby creating a diverse dataset that covers a range of context lengths. This parameter variation allows the model to learn robustness across different context scales without requiring a fixed complex structure.
3Adaptability or versatility
If more training examples with long contexts are generated, then the model's generalization ability improves, but the data processing time and computational resources increase
Solution Approach 1:
The patent applies periodic action by implementing iterative rounds of data augmentation with varying parameters. Instead of generating all possible long-context examples at once, the method performs multiple passes through the training data, each time augmenting with different table combinations and context lengths. This periodic augmentation approach distributes the computational workload over time, improving generalization while managing processing resources through staged generation.
Data Source
AI summary
The present disclosure relates to manufacturing training and testing data by leveraging data augmentation techniques to generate examples of long context database schemas. Aspects are directed towards accessing a training dataset comprising training examples where each training example may include i) a prompt including a natural language utterance and a database schema having one or more tables, and ii) a gold logical form corresponding to the natural language utterance, combining the tables from the database schemas in the training examples may generate a combined database schema set, generating a set of long context training examples based on the training dataset and the combined database schema set, and incorporating the long context database schema into the selected training example to generate a long context training example to train a generative artificial intelligence model with at least the set of long context training examples to generate a trained generative artificial intelligence model.