Data Preparation Interface With Multi-Pane Flow and Data Views
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data visualization tools struggle to balance clarity in data flow representation with visibility of actual data, leading to confusion in complex data manipulation tasks, whether in data flow style systems that obscure data or Potter's Wheel style systems that obscure flow structure.
Innovation Solution
A user interface with concurrent display of a data flow pane, tool pane, and profile pane, allowing users to interact directly with data elements to modify the flow diagram, with each node having a hash value that recomputes operations only when necessary, and supporting direct data manipulation to add new nodes based on user input.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If data flow style systems are used to provide clarity on overall structure, then flow representation is improved, but data visibility deteriorates
Solution Approach 1:
The interface is segmented into multiple panes: a flow diagram pane for high-level structure, a profile pane for data element statistics, and a data pane for actual data rows. Each pane serves a specific function, allowing users to navigate between flow structure and data details without confusion.
Solution Approach 2:
The profile pane acts as an intermediary between the flow diagram and the data pane. It provides intermediate data representations (distributions, counts) that bridge the gap between abstract flow nodes and concrete data rows, making data visibility accessible without overwhelming users.
2Loss of information
If Potter's Wheel style systems are used to present concrete data interface, then data visibility is improved, but flow structure clarity deteriorates
Solution Approach 1:
The interface is segmented into multiple panes: a flow diagram pane for high-level structure, a profile pane for data element statistics, and a data pane for actual data rows. Each pane serves a specific function, allowing users to navigate between flow structure and data details without confusion.
Solution Approach 2:
The system adds a dimensional layer by displaying the same data from multiple perspectives simultaneously: the flow diagram shows structural relationships, the profile pane shows statistical characteristics, and the data pane shows actual values. This multi-dimensional view resolves the contradiction between structure clarity and data visibility.
3Manufacturing precision
If each small operation gets its own node in the flow diagram, then operation precision is improved, but device complexity worsens due to rat's nest of nodes
Solution Approach 1:
Multiple related operations are merged into single nodes in the flow diagram. For example, multiple filtering operations or transformations are combined into one node, reducing the number of individual nodes while maintaining operational precision through the node's internal structure.
Solution Approach 2:
Complex operations are nested within nodes, allowing detailed operations to be contained within a single visual element. The node can expand to show internal structure when needed, providing both simplicity in the overview and precision in the details.
4Loss of information
If data is displayed in the flow diagram, then data visibility is improved, but flow structure clarity deteriorates due to occlusion
Solution Approach 1:
The interface is segmented into multiple panes: a flow diagram pane for high-level structure, a profile pane for data element statistics, and a data pane for actual data rows. Each pane serves a specific function, allowing users to navigate between flow structure and data details without confusion.
Solution Approach 2:
Data display is extracted from the flow diagram pane and placed in separate panes (profile pane and data pane). This extraction prevents data from obscuring flow structure while maintaining data visibility through dedicated display areas that do not interfere with the flow diagram's structural clarity.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
A computer system displays a user interface that includes a flow pane, a tool pane, a profile pane, and a data pane. The flow pane displays a node/link flow diagram that identifies data sources, operations, and output datasets. The tool pane includes a data source selector that enables users to add data sources to the flow diagram, and includes an operation palette that enables users to insert nodes into the flow diagram for performing specific transformation operations. The profile pane displays schemas corresponding to selected nodes in the flow diagram, including information about data fields and statistical information about data values for the data fields and enables users to modify the flow diagram by interacting with individual data elements. The data pane displays rows of data corresponding to selected nodes in the flow diagram, and enables users to modify the flow diagram by interacting with individual data values.