Insight Lineage Tracking via Automated Graph Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems lack effective methods for tracking the lineage of insights derived from data analytics, which are crucial for understanding the dependencies and creation processes of valuable knowledge within organizations, hindering the ability to maintain metadata and visualize the evolution of insights.
Innovation Solution
A method and system for tracking insight lineage by detecting initiation and interactions with insight editing programs, generating and updating insight lineage graphs, and mapping these graphs to files to record the creation and editing processes of insights, enabling the tracking of insight lineage and dependencies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If insight lineage tracking is implemented through manual documentation, then implementation simplicity is maintained, but tracking accuracy and completeness deteriorate
Solution Approach 1:
The system automatically generates and updates lineage graphs by monitoring user interactions with insight editing programs. The tracking mechanism serves itself by detecting engagement actions and creation actions without requiring manual intervention, thereby maintaining implementation simplicity while achieving high tracking accuracy through automated self-documentation of the insight creation process
Solution Approach 2:
The patent replaces manual documentation mechanisms with automated computer-based monitoring systems. The system uses software to detect user actions, generate lineage graphs, and update tracking data automatically, substituting the mechanical process of manual record-keeping with an automated digital system that ensures both accuracy and ease of implementation
2Loss of information
If comprehensive interaction monitoring is performed to track all insight creation steps, then lineage tracking completeness is improved, but system complexity increases
Solution Approach 1:
The system segments the insight creation process into distinct detectable actions: engagement actions (opening files, loading data) and creation actions (generating insights, saving results). By dividing the monitoring task into these discrete segments, the system achieves complete lineage tracking while managing complexity through modular detection and processing of individual action types
Solution Approach 2:
The insight service performs multiple functions through a unified system: it monitors user interactions, generates lineage graphs, updates tracking data, and visualizes insight relationships. This multi-functional approach reduces overall system complexity by consolidating what could be separate complex systems into a single integrated platform that handles all lineage tracking operations
3Loss of information
If real-time lineage graph updates are performed during insight creation, then information freshness is improved, but processing time increases
Solution Approach 1:
The system performs lineage graph updates at periodic intervals corresponding to key user actions (engagement actions and creation actions) rather than continuously. This periodic updating approach maintains information freshness by capturing all significant events while reducing processing time by avoiding unnecessary continuous monitoring and updates during intermediate states
Data Source
AI summary
A method and system for insight lineage tracking. Any insight may be defined as a finding (or more broadly, as useful knowledge) gained through data analytics or, more precisely, through the discovery of patterns and/or relationships amongst any given assortment of data/information. Accordingly, any insight may be derived, or inferred, from any number of data/information items, including any number of other insights, as well as any number of employed analytic algorithms or models. In aiming to track these insight-pertinent objects, as well as the dependencies there-between, embodiments disclosed herein create insight lineage graphs visually conveying said objects and dependencies and, moreover, maintain metadata descriptive of each step in an insight creation process leading to the production of insights.