Intermediate Data Layer for Provenance Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional approaches to accessing data integrity lack interactivity and intuitiveness, making it difficult for users to identify the correspondence between static data and its sources, thereby failing to accurately provide provenance information and degrading user experience.
Innovation Solution
A computing system processes data from sources into JSON objects, organizing them into a tree data structure as an intermediate layer, allowing for the recreation of content and metadata information and providing a user interface to display provenance information, enabling users to interactively visualize relationships between data analysis objects and their sources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a static version of data (e.g., PDF) is accessed by users, then data integrity is preserved, but interactivity and intuitiveness are lost, making it difficult for users to identify correspondence between the static version and data sources
Solution Approach 1:
The patent introduces an intermediate data layer that acts as a mediator between the static data version and the data sources. This layer contains structured data records with identifiers that can be traced back to original sources, enabling both data integrity preservation and user interactivity through runtime reconstruction of source correspondence
Solution Approach 2:
The system performs preliminary action by creating and storing an intermediate data layer in advance, which captures the state of data sources at a specific point in time. This pre-prepared structure enables rapid retrieval and reconstruction of provenance information without requiring access to original data sources during user interaction
2Ease of manufacture
If conventional static data access methods are used, then implementation is simple, but provenance information cannot be accurately provided and user experience degrades
Solution Approach 1:
The patent segments the data access system into distinct components: the original data sources, an intermediate data layer with structured records, and a runtime reconstruction mechanism. Each segment handles specific functions, allowing accurate provenance tracking while maintaining implementation feasibility through modular architecture
3Measurement precision
If data is stored in a structured intermediate layer with traceability to sources, then provenance information accuracy improves, but system complexity increases
Solution Approach 1:
The patent applies local quality by implementing structured data records with source traceability specifically at the intermediate data layer, while keeping other parts of the system relatively simple. The complexity is localized to where it provides maximum benefit (provenance tracking) rather than distributed throughout the entire system
Data Source
AI summary
Systems, methods, and non-transitory computer-readable media can obtain data form one or more data sources. The data from the one or more data sources can be processed into one or more data records. Each data record can include at least an identification, a data type, and data to be stored. The data records can be organized into a tree data structure. The tree data structure can be stored as an intermediate data layer in a data analysis platform.