Faceted Metadata Storage with Source Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems lack an efficient method to store and provide metadata items corresponding to multiple data objects while ensuring access permissions and time frames are considered, leading to incomplete or inaccurate data lineage and impact analyses.
Innovation Solution
A system that receives metadata from multiple sources, stores each item as a facet in a repository, tags each facet with a source identifier and time frame, and provides the relevant facets to requesters based on their access permissions and specified criteria, including time frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If metadata from multiple sources is stored in a unified repository, then data lineage and impact analysis capabilities are improved, but access control and data security become more complex
Solution Approach 1:
The patent segments metadata storage into separate facet tables (e.g., table 104 for first source, table 106 for second source) rather than using a single unified table. Each facet table stores metadata from a specific source with its own access permissions. This segmentation allows the system to maintain comprehensive metadata for improved data lineage analysis while keeping access control logic simpler by source, rather than managing complex multi-source access permissions in a single table.
2Loss of information
If all metadata facets are retrieved and provided to requesters, then completeness of data lineage information is improved, but access permissions and security requirements are violated
Solution Approach 1:
The patent introduces an intermediary component (metadata service 102) that sits between the facet repositories and requesters. This intermediary retrieves metadata from multiple facet tables, applies access permission filters based on the requester's authorization level, and provides only the appropriate subset to the requester. This mediator approach ensures both metadata completeness (by having access to all facets) and permission enforcement (by filtering before delivery).
3Measurement precision
If metadata is stored with detailed source identifiers and time frames, then accuracy of data lineage tracking is improved, but storage complexity and query processing time increase
Solution Approach 1:
The patent applies preliminary action by pre-tagging each metadata facet with source identifiers (e.g., source1 108, source2 110) and time frame information during the storage phase. This pre-processing of metadata with complete tracking information allows the system to maintain high data lineage tracking accuracy. When queries are executed, the pre-tagged facets can be quickly filtered and retrieved based on these pre-established tags, significantly reducing query processing time compared to computing lineage information at query time.
Data Source
AI summary
A system and method for storing and providing metadata. Metadata may be retrieved from multiple sources. The metadata is stored in facets in a repository and tagged to indicate one or more of the source, a time frame, or an associated data object. In response to receiving a request for metadata, a system selects metadata based on the specified object, source, or time frame. Access permissions corresponding to the requester are used to select and provide metadata for which the requester has permissions.