Spreadsheet Data Transformation via Automatic Data Flow Graph Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data transformation processes are inefficient and burdensome, requiring IT expertise to move data from transactional databases to analytic databases, making it difficult for business users to access and analyze data due to differing storage structures and the spread of important data across an enterprise.
Innovation Solution
A computer-implemented system allows business users to specify data sources, manipulate data in a spreadsheet-like interface, which is automatically translated into data flow objects and executed as a data flow graph, enabling direct access and transformation of data without IT intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If data is moved from transactional databases to analytic databases using traditional data warehousing methods, then data can be accessed for analysis, but the process becomes time-consuming and requires IT expertise
Solution Approach 1:
The system enables business users to perform their own data transformations using a visual spreadsheet interface without requiring IT department assistance. Users can directly manipulate data, apply transformations, and generate analytics independently, eliminating the need to submit requests to IT staff and wait for complex extraction and transformation processes.
Solution Approach 2:
The patent replaces traditional mechanical data transformation tools (complex ETL software requiring IT training) with a visual spreadsheet interface that business users already understand. The system translates familiar spreadsheet operations into automated data flow graphs, substituting complex mechanical transformation processes with user-friendly visual operations.
2Adaptability or versatility
If complex extraction and transformation tools are used to move data, then data can be transformed according to needs, but the process becomes burdensome and requires special software training
Solution Approach 1:
The system uses spreadsheets as a familiar copy or representation of data that business users already understand from everyday business operations. Instead of requiring users to learn complex transformation tools, the system presents data in a familiar spreadsheet format and translates user manipulations into automated transformations, leveraging existing user knowledge rather than requiring new skills.
Solution Approach 2:
The spreadsheet interface serves multiple functions simultaneously: it displays data, allows manipulation, defines transformations, and generates analytics. This universal interface replaces multiple specialized tools (data extraction utilities, transformation software, loading tools) with a single multi-functional platform that business users can operate without specialized training.
3Ease of operation
If data is stored in spreadsheets across an enterprise, then data can be easily manipulated, but tracking and accessing data becomes difficult
Solution Approach 1:
The system provides automated feedback mechanisms that track data origins, transformations, and usage. When users manipulate data in spreadsheets, the system automatically records the data flow, transformation steps, and lineage information, making it easy to track where data comes from, how it's been transformed, and where it's being used without requiring manual documentation.
Data Source
AI summary
The present disclosure includes techniques pertaining to computer implemented systems and methods for transforming data. In one embodiment, data sources are specified by a user and copies of specified data are received in local storage. Data is presented to the user as a spreadsheet of columns and rows and the user manipulates the data in the spreadsheet. The user manipulations are automatically translated into data flow objects and a data flow graph, which is executed to transform the data. The modified data is presented to the user.