Database Subset Extraction via Entity Graph Traversal
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for extracting a subset of data from databases with complex schemas require a large number of joins, leading to resource-intensive processes and potential selection of unnecessary data, which is not scalable for large schemas and can result in storage limitations.
Innovation Solution
A method that minimizes parent-to-child joins by condensing entity graphs to remove cycles, determining directed edges for bidirectional traversal, and generating an ordered list of edges to efficiently extract a referentially intact subset, reducing the overall number of joins required.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional traversal schemes are used to extract data subsets from complex schemas, then referential integrity can be maintained, but the number of joins increases significantly leading to resource-intensive processes
Solution Approach 1:
The patent segments the schema graph into multiple traversal phases, processing different types of relationships (parent-to-child, child-to-parent) in separate passes. This segmentation allows the system to maintain referential integrity by systematically traversing relationship edges while avoiding redundant joins, thereby improving extraction efficiency without sacrificing data integrity
Solution Approach 2:
The patent performs preliminary actions by first identifying all relationship edges in the schema graph before executing the actual data extraction. The system pre-processes the graph to determine the optimal traversal sequence, marking edges that require bidirectional traversal. This preliminary analysis enables the extraction process to proceed with minimal redundant joins while maintaining referential integrity
2Quantity of substance
If bidirectional traversal is performed on all relationship edges, then complete data coverage is achieved, but unnecessary data is selected increasing storage requirements
Solution Approach 1:
The patent applies local quality by differentiating the traversal behavior for different types of relationship edges. Instead of uniformly traversing all edges bidirectionally, the system identifies specific edges that require bidirectional traversal based on the schema structure and data dependencies. This localized approach ensures complete data coverage for necessary relationships while avoiding unnecessary data selection, thereby improving storage efficiency
3Productivity
If the number of joins is reduced to improve extraction speed, then productivity increases, but referential integrity may be compromised
Solution Approach 1:
The patent introduces dynamics by adaptively adjusting the traversal strategy based on the specific schema structure and relationship types encountered. The system dynamically determines which edges require bidirectional traversal versus unidirectional traversal, optimizing the join operations for each context. This dynamic approach maintains referential integrity where necessary while minimizing joins elsewhere, achieving both high productivity and reliability
Data Source
AI summary
An apparatus, computer-readable medium, and computer-implemented method for data subsetting, including receiving a request comprising a criterion indicating a criterion table in a plurality of tables of a database, as schema of the database corresponding to an entity graph, the entity graph comprising a plurality of entities corresponding to the plurality of tables and a plurality of directed edges connecting the plurality of entities, determining directed edges in the plurality of directed edges that must be traversed in both directions in order to traverse all entities in the entity graph starting from a criterion entity corresponding to the criterion table, generating, an ordered list of edges for the entity graph based on the directed edges that must be traversed in both directions and topological ordering, and generating a subset of data from the plurality of tables based on the ordered list of edges for the entity graph and the request.