Visualization Training Data Pipeline for NL2LF Models

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing natural language to logical form (NL2LF) models struggle to convert natural language utterances into visualization representations due to a lack of large-scale and high-quality benchmark training data.

Innovation Solution

A data manufacturing pipeline is developed to generate visualization training data through augmentation and synthesis techniques, including modifying existing training datasets to include visualization actions, and using incremental and manipulation visualization datasets to create new examples. These techniques are used to train machine learning models to convert natural language into meaning representation languages (MRL) that include visualization actions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If deep learning approaches are used to convert natural language queries to SQL queries, then the system can generalize across tasks and domains, but a large amount of labelled training data is required which is challenging to obtain

Engineering Contradiction:

Improvegeneralization capabilityVSAvoidtraining data volume

Core Design Contradiction:

Adaptability or versatilityVSQuantity of substance

Solution Approach 1:

The patent applies preliminary action by pre-processing natural language queries through semantic parsing to extract structured information and generate SQL queries before training the deep learning model. This pre-computation of training labels from unlabelled data reduces the need for manually annotated training examples while maintaining generalization capability.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces an intermediary semantic representation layer between natural language input and SQL output. This intermediary structure serves as a bridge that enables the model to learn from unlabelled data by parsing semantics and generating SQL through a structured intermediate format, reducing dependency on large labelled datasets.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Ease of manufacture

If rule-based systems are used to translate natural language sentences to SQL queries, then the translation process is straightforward, but the systems are domain-specific and do not generalize well to new use cases

Engineering Contradiction:

Improvetranslation simplicityVSAvoiddomain generalization

Core Design Contradiction:

Ease of manufactureVSAdaptability or versatility

Solution Approach 1:

The patent replaces the mechanical rule-based translation system with a deep learning-based semantic parsing system. This substitution allows the system to move from rigid, domain-specific rules to a flexible, data-driven approach that can generalize across domains while maintaining translation capability through learned semantic representations.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent changes the parameters of the translation system from fixed rules to learned parameters through deep learning. By training the model on diverse datasets including unlabelled data, the system adapts its parameters to handle multiple domains and use cases, achieving both translation simplicity and domain generalization.

Inventive Principle:

Principle #35Parameter changes

3Measurement precision

If statistical-based slot-filling systems are used to parse natural language sentences, then the system can label tokens according to their semantic role, but the system has limitations on the types of sentences that can be parsed

Engineering Contradiction:

Improvesemantic role labeling accuracyVSAvoidsentence type coverage

Core Design Contradiction:

Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent applies dynamics by using a flexible deep learning architecture that can adapt to different sentence structures and types. Unlike rigid slot-filling systems, the neural model dynamically adjusts its parsing strategy based on the input sentence characteristics, enabling it to handle diverse sentence types while maintaining semantic role labeling accuracy.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent creates a universal semantic parsing system that can handle multiple sentence types through a single deep learning model. The model is designed to perform multiple functions including token labeling, dependency parsing, and SQL generation, making it versatile across different sentence structures while maintaining precision in semantic role identification.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS20250068626A1Techniques for manufacturing training data to transform natural language into a visualization representation

Publication Date: 2025.02.27 ORACLE INT CORP

AI summary

The present disclosure relates to manufacturing training data by leveraging an automated pipeline that manufactures visualization training datasets to train a machine learning model to convert a natural language utterance into meaning representation language logical form that includes one or more visualization actions. Aspects are directed towards accessing an original training dataset, a visualization query dataset, an incremental visualization dataset, a manipulation visualization dataset, or any combination thereof. One or more visualization training datasets are generated by: (i) modifying examples in the original training dataset, the visualization query dataset, or both to include visualization actions, (ii) generating examples, using the incremental visualization dataset, the manipulation visualization dataset, or both, that include visualization actions, or (iii) both (i) and (ii). An augmented training dataset is generated by adding the one or more visualization training datasets to the original training dataset and then used to train the machine learning model.