Data Flow Graph Automation for Spreadsheet Data Transformation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for making transactional data available to business users are inefficient and burdensome, requiring complex data preparation by IT staff and involving multiple iterations to move data from transactional databases to analytic databases, which can be time-consuming and difficult to manage, especially when data is spread across an enterprise.
Innovation Solution
A computer-implemented system that allows business users to manipulate data in a user interface, automatically generating data transformations through data flow objects and graphs, enabling direct access to various data sources without requiring IT expertise, with the system performing complex data processing steps in the background.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is moved from transactional databases to analytic databases using traditional extraction, transformation, and loading tools, then data can be accessed for analysis, but the process becomes burdensome, time-consuming, and requires multiple iterations
Solution Approach 1:
The system enables business users to directly access and manipulate transactional data through a user interface without requiring IT staff involvement. Users can perform data operations independently, eliminating the need for complex extraction and transformation processes that previously required specialized tools and multiple iterations.
Solution Approach 2:
The patent replaces traditional mechanical data movement processes (using ETL tools and manual data preparation) with an automated system that uses data flow graphs and virtual tables to dynamically access and transform data on-demand, significantly reducing the time and effort required.
2Ease of manufacture
If IT staff use complex extraction, transformation, and loading tools to move data, then data can be transferred to analytic databases, but the process becomes burdensome and requires special software training
Solution Approach 1:
The system introduces virtual tables as intermediaries between users and the underlying transactional data. These virtual tables provide a simplified interface that automatically handles the complex extraction, transformation, and loading processes, eliminating the need for users to work with complex ETL tools while maintaining data integrity.
Solution Approach 2:
The patent replaces complex mechanical data preparation processes with automated system-level operations. The data flow graph engine automatically executes transformations and data movement, substituting manual IT staff operations with automated processes that are transparent to users.
3Ease of operation
If data is stored in spreadsheets across an enterprise, then data is accessible to business users, but it becomes difficult to track, access, and load by IT organization
Solution Approach 1:
The system provides a universal interface that allows users to access data from multiple sources (spreadsheets, databases, etc.) through a single user interface. The virtual table mechanism handles different data sources uniformly, making it easier to track and manage data regardless of its original location while maintaining ease of access.
Solution Approach 2:
The system implements feedback mechanisms that track data access patterns, usage statistics, and transformation operations. This feedback enables the system to automatically optimize data retrieval, monitor data flow, and provide insights into data usage across the enterprise, making data management more transparent and controllable.
4Adaptability or versatility
If multiple iterations are required to meet business analyst needs, then data can be transformed to match requirements, but the process becomes time-consuming
Solution Approach 1:
The system uses dynamic data flow graphs that can automatically adapt to changing data requirements. When business analysts modify their needs, the system dynamically reconfigures the data flow and transformations without requiring manual rework, enabling rapid adaptation while reducing the time needed for iterative transformations.
Solution Approach 2:
The system implements feedback loops that monitor data usage patterns and automatically adjust transformations based on actual requirements. This feedback mechanism enables the system to learn from previous iterations and automatically optimize data transformations, reducing the overall time required to meet business needs.
Data Source
AI summary
The present disclosure includes techniques pertaining to computer implemented systems and methods for automatic generation of data transformations. In one embodiment, a user manipulates a spreadsheet of data in a user interface. On a backend, the user's manipulations trigger actions that cause the data to be modified. Actions may automatically cause data flow objects and data flow graphs to be produced. The transformations defined by the data flow graphs are automatically executed by a software engine and the results are displayed to the user. The user may access and manipulate data from a variety of data sources while the underlying complexities of the transformation process are performed in an automated manner.