Multi-Task Learning for Natural Language to SQL Translation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional approaches for transforming natural language to SQL queries face challenges due to the need for large training datasets for each task, leading to poor performance and inefficiency in non-technical users' interaction with databases.

Innovation Solution

A multi-task learning strategy is employed to jointly train a machine learning model for translating natural language utterances to logical forms, sharing model parameters across tasks and reducing the reliance on extensive training data.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If conventional approaches (rule-based or statistical-based systems) are used for natural language to SQL translation, then the system can be implemented with simpler architecture, but the performance and accuracy of translation deteriorates due to inability to generalize to new use cases and limited parsing capabilities

Engineering Contradiction:

Improvesystem architecture complexityVSAvoidtranslation accuracy and generalization capability

Core Design Contradiction:

Device complexityVSReliability

Solution Approach 1:

The patent replaces rule-based mechanical translation systems with a deep learning-based semantic parsing system. The neural network model learns to translate natural language to SQL queries by processing semantic roles and relationships, eliminating the need for complex hand-crafted rules while achieving superior accuracy and generalization across different domains and use cases.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent changes the fundamental parameters of the translation approach by using statistical-based token labeling and deep learning models instead of rule-based systems. The model learns from training data to map semantic roles to SQL components, enabling flexible adaptation to new domains without requiring manual rule updates, thus improving both accuracy and generalization.

Inventive Principle:

Principle #35Parameter changes

2Adaptability or versatility

If deep-learning based systems are used for natural language to SQL translation, then the generalization to new use cases improves, but the requirement for large training datasets increases, leading to poor performance when data is limited

Engineering Contradiction:

Improvegeneralization to new use casesVSAvoidtraining data quantity required

Core Design Contradiction:

Adaptability or versatilityVSQuantity of substance

Solution Approach 1:

The patent creates a universal semantic parsing model that can handle multiple types of SQL queries and domains through a single trained system. The model learns general patterns in natural language to SQL translation that apply across different use cases, eliminating the need for separate models for each domain and reducing the total training data required compared to multiple specialized models.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent uses partial training on representative samples to achieve effective performance. Instead of requiring extensive training data covering all possible scenarios, the model learns from a curated set of examples that capture the essential patterns, then generalizes to new use cases through its learned semantic understanding, avoiding the data hunger problem of conventional deep learning approaches.

Inventive Principle:

Principle #16Partial or excessive action

3Adaptability or versatility

If statistical-based systems are used for natural language to SQL translation, then the system can process various sentence structures, but the limitation on sentence types (must be representable as parse tree) reduces flexibility and adaptability

Engineering Contradiction:

Improvesentence structure processing capabilityVSAvoidflexibility in handling diverse sentence types

Core Design Contradiction:

Adaptability or versatilityVSEase of operation

Solution Approach 1:

The patent replaces the mechanical parse tree requirement with a neural network-based semantic role labeling approach. The deep learning model directly processes natural language sentences and maps them to SQL queries without requiring explicit parse tree representations, enabling flexible handling of diverse sentence structures including those that cannot be easily represented as traditional parse trees.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS12430329B2Transforming natural language to structured query language based on multi- task learning and joint training

Publication Date: 2025.09.30 ORACLE INT CORP

AI summary

Techniques are disclosed for training a model, using multi-task learning, to transform natural language to a logical form. In one particular aspect, a method includes accessing a first set of utterances that have non-follow-up utterances and a second set of utterances that have initial utterances and associated one or more follow-up utterances and training a model for translating an utterance to a logical form. The training is a joint training process that includes calculating a first loss for a first semantic parsing task based on one or more non-follow-up utterances from the first set of utterances, calculating a second loss for a second semantic parsing task based on one or more initial utterances and associated one or more follow-up utterances from the second set of utterances, combining the first and second losses to obtain a final loss, and updating model parameters of the model based on the final loss.