Automated Question-SQL Pair Generation for Semantic Parsing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional semantic parsing models require extensive manual labeling of training data for new databases, leading to inefficiencies in generating SQL query statements and corresponding questions, which is time-consuming and labor-intensive.
Innovation Solution
A method for automatically generating SQL query statements and their corresponding questions by dividing SQL query statements into clauses, using pre-trained models to obtain question descriptions, and combining them to form high-quality question-SQL pairs, thereby reducing manual effort and improving efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If manual labeling is used to build training data for new databases, then training data quality can be ensured, but the process consumes excessive manpower and time resources
Solution Approach 1:
The patent uses template matching to copy and adapt existing SQL query patterns from the training set to new databases. By identifying similar query templates and replacing database-specific identifiers, the system automatically generates high-quality training data for new databases without manual labeling, thus copying proven patterns to new contexts while maintaining quality standards.
Solution Approach 2:
The system performs self-service by automatically generating training data for new databases using the established templates and similarity matching algorithms. The semantic parsing model and template matching mechanism work autonomously to create question-SQL pairs without requiring manual intervention for each new database, enabling the system to serve itself in data generation tasks.
2Adaptability or versatility
If conventional semantic parsing models are used for new databases, then generalization capability is maintained to some extent, but desirable performance cannot be achieved
Solution Approach 1:
The patent applies preliminary action by pre-processing new database schemas to identify similar templates from the training set before actual query generation. The system performs template matching and similarity computation in advance, preparing adapted SQL templates that are then ready for immediate use. This preliminary adaptation ensures both generalization to new databases and reliable performance by pre-establishing the mapping between new schemas and proven query patterns.
3Reliability
If more training data is generated manually for new databases, then model performance can be improved, but the time and labor costs increase significantly
Solution Approach 1:
Instead of manually creating training data from scratch for new databases, the system copies and adapts existing SQL query templates from the training set. By matching new database schemas with similar templates and automatically generating corresponding question-SQL pairs, the system produces sufficient training data with high model performance while avoiding the time-consuming manual labeling process.
Data Source
AI summary
The present disclosure provides a data pair generating method, apparatus, electronic device and storage medium, and the field of artificial intelligence such as natural language processing and deep learning. The method may include: generating M SQL query statements for a given database, where M is a positive integer greater than one; performing the following processing for each SQL query statement: dividing the SQL query statement into at least one SQL clause; obtaining a question description corresponding to each SQL clause; combining the question descriptions to obtain a question corresponding to the SQL query statement. The solution of the present disclosure may be applied to save manpower and time costs.