Example-Based Transformation Program Generation for Data Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data transformation processes are often time-consuming and difficult due to the need for manual search and application of different transformations for diverse data formats, leading to inefficiencies in converting data to a consistent format for analysis.
Innovation Solution
A transformation engine dynamically generates transformation programs based on example input and output values, searching a collection of tools to identify relevant functions, functions, tables, and services, and applies supplemental tools to achieve desired data transformations, with suggestions provided for user selection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual search and application of different transformations is performed for diverse data formats, then data transformation accuracy can be achieved, but time consumption and operational difficulty increase significantly
Solution Approach 1:
The system performs self-service by automatically generating transformation programs without requiring manual user intervention. The transformation engine autonomously analyzes input data characteristics, searches for appropriate transformation tools, and composes transformation programs based on example input-output pairs, eliminating the need for users to manually search and apply transformations.
Solution Approach 2:
The system performs preliminary action by pre-processing and analyzing example input and output values before generating transformation programs. The transformation engine examines the characteristics of example data pairs in advance to identify appropriate transformation tools and parameters, which accelerates the overall transformation process when actual data needs to be transformed.
2Measurement precision
If manual search and application of different transformations is performed for diverse data formats, then data transformation accuracy can be achieved, but operational complexity increases
Solution Approach 1:
The system performs self-service by automatically generating transformation programs without requiring manual user intervention. The transformation engine autonomously analyzes input data characteristics, searches for appropriate transformation tools, and composes transformation programs based on example input-output pairs, eliminating the need for users to manually search and apply transformations.
Solution Approach 2:
The system implements universality through a single transformation engine that can handle diverse data formats and transformation types. The engine maintains a collection of multiple transformation tools and can automatically select and compose appropriate tools based on the specific transformation requirements, providing a unified interface for various transformation operations.
3Adaptability or versatility
If a collection of multiple transformation tools is maintained, then transformation capability and versatility improve, but system complexity increases
Solution Approach 1:
The transformation engine acts as an intermediary between the user and the collection of transformation tools. It manages the complexity by automatically selecting, composing, and coordinating multiple transformation tools based on the specific transformation requirements, shielding users from the underlying system complexity while maintaining high transformation capability.
Solution Approach 2:
The system implements dynamics by enabling the transformation engine to adaptively select and compose different transformation tools based on the characteristics of the input data and desired output. The engine dynamically adjusts its transformation strategy rather than following a fixed sequence, optimizing the use of available transformation tools for each specific transformation task.
Data Source
AI summary
Methods, computer systems, computer-storage media, and graphical user interfaces are provided for facilitating data transformations, according to embodiments of the present invention. In one embodiment, a transformation function is executed using an example input value to obtain an initial output value. Thereafter, a plurality of supplemental transformation tools is applied to the initial output value to generate a plurality of intermediary output values. Based on a comparison of each of the intermediary output values to an example output value, the supplemental transformation tool that generated an intermediary output value having a greatest extent of similarity to the example output values is identified. The identified supplemental transformation tool and the transformation function are used to generate a transformation program that transforms the example input values to the desired form in which to transform data.