Data Manipulation Program Generation via Input-Output Examples
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing spreadsheet systems face challenges in efficiently transforming large datasets due to limitations in manual processing, user familiarity with features, and complexity of macro programming, leading to poor user experience and limited applicability of macro recording techniques.
Innovation Solution
A method and system for automatically creating data manipulation programs using input-output examples, which generates subprograms with selection conditions to apply appropriate operations to new input items, integrated with spreadsheet functionality and utilizing an expression language for parsing and transforming data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If manual data transformation is used, then user control and flexibility are maintained, but processing efficiency deteriorates for large datasets
Solution Approach 1:
The system enables self-service automation by allowing users to record their manual data transformation operations and automatically replay them. The macro recording mechanism captures user actions and generates automated programs that can process large datasets without continuous manual intervention, thus maintaining user control while dramatically improving processing efficiency.
Solution Approach 2:
The system performs preliminary action by recording user operations during an initial phase and storing them as reusable macro programs. These pre-recorded macros can then be applied to large datasets in advance or on-demand, eliminating the need for repeated manual processing and significantly reducing time loss.
2Extent of automation
If macro recording is used, then automation capability is provided, but applicability deteriorates for input items not matching recorded patterns
Solution Approach 1:
The system implements dynamics by making macro programs adaptive and modifiable. Users can edit and customize recorded macros to accommodate varying input patterns, and the system allows dynamic adjustment of transformation rules based on different data scenarios, thereby maintaining both automation and versatility.
Solution Approach 2:
The macro recording system provides universality by designing transformation rules that can handle multiple input patterns. The recorded macros are structured to work with various data formats and transformations, making them applicable beyond the exact recorded scenarios and enhancing their versatility across different datasets.
3Adaptability or versatility
If custom macro programming is used, then precise control over transformation logic is achieved, but user accessibility deteriorates due to programming complexity
Solution Approach 1:
The system introduces an intermediary layer between user intent and program execution. The macro recording feature acts as a mediator that translates user actions into structured transformation programs automatically, eliminating the need for users to write complex code directly while still providing precise control over transformation logic through the recorded macros.
4Adaptability or versatility
If manual macro modification is used, then program adaptability to changing needs is achieved, but user burden and complexity increase
Solution Approach 1:
The system enables self-service adaptation by allowing users to easily modify and edit recorded macros through simplified interfaces. Users can adjust transformation parameters and logic without dealing with complex programming structures, making program adaptation to changing needs straightforward and reducing the perceived complexity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A program creation system is described which generates a data manipulation program based on input-output examples. The created program may include a collection of subprograms together with a collection of corresponding selection conditions. When a new input item is received, a program execution module uses the selection conditions to select one of the subprograms. The program execution module then applies the selected subprogram to generate a new output item. The program creation system generates the program using a three-part approach, involving: generating sets of subprograms for the respective input-output examples; grouping the sets of programs into partitions and choosing representative subprograms for the partitions; and determining the selection conditions. A user interaction module provides various mechanisms which allow a user to interact with the program creation system and thereby improve the performance of the created program.