Spreadsheet Formula Conversion to Distributed Job Flows
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in converting spreadsheet data structures with formulae into objects suitable for distributed processing, lacking user-friendly mechanisms for complex calculations and data organization, and requiring advanced programming skills.
Innovation Solution
An apparatus and method that processes requests to generate a set of objects within a federated area for performing job flows, including data objects, task routines, and job flow definitions, which correlate input and output data, identify parallel tasks, and generate visual representations of task dependencies, enabling efficient distributed analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If spreadsheet data structures with formulae are used for analysis, then user-friendliness is improved, but the ability to perform complex distributed processing deteriorates
Solution Approach 1:
The patent introduces an intermediary conversion process that transforms spreadsheet data structures into a hybrid representation suitable for distributed processing. The system converts spreadsheet data and formulae into a structured format that maintains user-friendliness while enabling distributed computation across multiple processing cores, thus bridging the gap between ease of use and processing capability.
Solution Approach 2:
The patent segments the analysis routine into discrete tasks that can be independently processed. By dividing the spreadsheet-based analysis into separable computational units, the system enables distributed processing across multiple cores while preserving the original spreadsheet structure for user interaction, thus resolving the contradiction between user-friendliness and distributed processing capability.
2Productivity
If analysis routines are written for distributed processing, then processing capability is improved, but programming skill requirements increase
Solution Approach 1:
The patent creates a copy of the analysis routine in two representations: the original spreadsheet format for user-friendly definition and a converted format for distributed processing execution. This copying approach allows users to define routines using familiar spreadsheet interfaces while the system automatically generates the distributed processing version, eliminating the need for users to learn complex programming skills.
Solution Approach 2:
The conversion system acts as an intermediary that automatically translates between spreadsheet-based definitions and distributed processing implementations. This mediator handles the complexity of distributed programming internally, allowing users to work with simple spreadsheet interfaces while achieving high processing capability through multi-core execution.
3Productivity
If data sets are broken up for distributed processing, then processing efficiency is improved, but data organization complexity increases
Solution Approach 1:
The patent segments data sets into portions that can be distributed across multiple processing cores, improving processing efficiency. Simultaneously, it maintains the original spreadsheet data organization structure, allowing users to work with data in their familiar format while the system handles the segmentation and distribution transparently, thus resolving the contradiction between efficiency and organizational complexity.
Data Source
AI summary
An apparatus includes a processor to: receive a request to generate a DAG of a job flow of multiple tasks of an analysis based on data table(s) and formulae of a spreadsheet data structure; correlate each indication of data required as input or output to at least a subpart of a data table; identify data dependencies and determine an order of performance among the multiple tasks based on the formulae; generate, within the specified federated area, a job flow definition that specifies the order of performance of the multiple tasks; for each task of the multiple tasks, generate, within the specified federated area, a corresponding macro data structure of multiple macro data structures; and generate the requested visualization based on the job flow definition and the multiple macro data structures.