Data Lineage Visualization with Hierarchical Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems face challenges in efficiently navigating and analyzing large-scale, complex data lineage due to overwhelming amounts of end-to-end lineage information, making it difficult to search, retrieve, and evaluate data lineage effectively in big data analytics.
Innovation Solution
A system that processes data lineage by displaying limited portions of upstream and downstream relationships in a centered, carousal manner, allowing users to focus on specific levels and navigate through lineage data efficiently, with features like prioritization and additional information display upon cursor interaction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If complete end-to-end lineage information is displayed, then comprehensive data coverage is achieved, but information overload and difficulty in navigation occur
Solution Approach 1:
The patent segments the complete data lineage into hierarchical levels (current level, upstream levels, downstream levels) and displays only the necessary portions at each level. The visualization system divides the overwhelming end-to-end lineage into manageable segments that can be explored interactively, allowing users to focus on specific portions without being overwhelmed by the entire data set.
Solution Approach 2:
The patent introduces a new dimensional approach by organizing lineage data in a hierarchical multi-level structure rather than a flat linear display. This dimensional transformation allows users to navigate through lineage information across different levels (upstream, current, downstream) and selectively expand or collapse portions, effectively managing information density while preserving completeness.
2Measurement precision
If detailed lineage data is processed and displayed, then analysis accuracy is improved, but processing time and system resources increase
Solution Approach 1:
The patent implements partial action by initially displaying only the essential current level lineage information and allowing users to progressively expand to upstream and downstream levels as needed. This on-demand expansion approach processes and renders detailed lineage data only when specifically requested, rather than pre-processing and displaying all possible lineage information simultaneously, thereby reducing initial processing time while maintaining analytical accuracy when needed.
3Loss of information
If comprehensive data assets are displayed at all levels, then complete lineage visibility is achieved, but display complexity and user confusion increase
Solution Approach 1:
The patent applies local quality by providing different levels of detail and organization for different portions of the lineage visualization. The current level receives prominent display with detailed information, while upstream and downstream levels are organized hierarchically with progressively aggregated information. This localized differentiation in information presentation reduces overall visualization complexity while maintaining complete lineage visibility.
Data Source
AI summary
A system displays summaries of relationships of a selected data asset with other data assets at a limited number of levels upstream and downstream from the selected data asset in rows above and below the selected data asset. In each row, data assets are arranged in order of usage, with most used data asset displayed directly above or below the selected data asset. The user views grandparent-level data assets of a parent-level data asset that is directly above the selected data asset. The system includes a carousal feature to further navigate the lineage data upstream or downstream. By selecting a new data asset in the parent row, the user can view grandparent-level data assets of the newly selected data asset. The user can view multiple upstream or downstream levels arranged in respective rows displayed above or below the selected data asset. The system can analyze data from any application.