Analytics Network Graphs for Artifact Lineage Visualization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data analytics environments lack tools to effectively visualize relationships and lineage between various artifacts, making it difficult for users to understand how their data is being used and transported within the system.
Innovation Solution
A system and method for generating a network graph from analytic artifacts, using metadata to create a visualization of relationships and lineage between artifacts such as User, Role, DV Project, Dataset, Connection, Dataflow, Sequence, and ML Model, providing insights through a user interface.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If data analytics environments provide comprehensive data examination capabilities, then users can derive conclusions from large amounts of data, but users lack tools to effectively visualize relationships and lineage between various artifacts
Solution Approach 1:
The patent introduces an intermediary visualization system that mediates between the complex analytics artifacts and the user. This system automatically generates visual representations of data lineage and relationships, allowing users to understand data flow without directly interacting with the complex underlying artifacts. The intermediary translation layer preserves complete lineage information while presenting it in an easily digestible visual format.
2Loss of information
If the system provides detailed information about analytic artifacts, then users can understand data usage, but the complexity of tracking relationships between multiple artifacts increases
Solution Approach 1:
The system employs self-service mechanisms where the visualization tool automatically discovers, extracts, and organizes relationship information between artifacts without requiring manual configuration. The system autonomously traverses the analytics environment, identifies connections between data sources, transformations, and destinations, and generates visual lineage representations. This self-service approach maintains complete relationship information while avoiding the complexity burden on users.
Solution Approach 2:
The patent implements preliminary action by pre-computing and storing relationship metadata between artifacts before users need to query them. The system proactively builds and maintains a knowledge graph of data relationships, so when users request lineage information, it is already prepared and can be quickly rendered. This preliminary computation preserves comprehensive relationship information while reducing the apparent system complexity during user interactions.
Data Source
AI summary
In accordance with an embodiment, described herein is a system and method for providing a network-visualization browser extension in an analytics environment. An analytic applications environment can be provided by, or otherwise operate at, a computer system providing access to a data warehouse, or data warehouse instance. The system is adapted for generating a network graph from analytic artifacts, including display of one or more dataflows as dataflow constellation visualizations that illustrates the dataflows and a lineage or other relationship information. The systems and methods disclosed herein can provide tools to provide insights for users of an analytics environment with regard to the users' analytic artifacts and relationships among the same.