Bootstrapped Graph Structure for Lateral Data Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Complex systems with diverse components and stakeholders face challenges in efficiently sharing accurate and updated information, leading to difficulties in diagnosing and addressing problems due to overwhelming volumes of data.
Innovation Solution
Implementing global and lateral data filtering techniques using semantic filter context objects and bootstrapped graph structures, which allow for efficient data communication and optimized SQL query generation across multiple databases without duplicating data, reducing payload size and enhancing communication speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If all data is maintained and duplicated into a graph database for efficient querying, then data retrieval speed is improved, but system complexity and data storage requirements increase significantly
Solution Approach 1:
The patent segments the data retrieval process into two distinct phases: (1) a bootstrapping phase that pre-calculates and stores only the graph structure metadata (table names, primary keys, foreign keys, relationship types) in a lightweight graph database, and (2) a query execution phase that uses this metadata to dynamically generate optimized SQL queries against the original relational databases. This segmentation avoids duplicating actual data while providing fast path calculation capabilities.
Solution Approach 2:
The patent introduces a bootstrapped graph structure as an intermediary layer between the relational databases and the query interface. This intermediary contains only structural metadata (nodes representing tables, edges representing relationships with types like PK-FK, One-to-Many, etc.) and enables rapid path calculation without requiring full data duplication. The intermediary translates high-level graph queries into optimized SQL statements that execute against the source databases.
2Loss of information
If data is shared across multiple services with full context, then information accuracy is improved, but payload size and communication overhead increase
Solution Approach 1:
The patent extracts only the essential structural metadata from the relational databases and places it in the bootstrapped graph structure. This extracted metadata includes table names, primary key columns, foreign key columns, and relationship types—sufficient information for path calculation but excluding actual data values. This extraction maintains information accuracy for routing purposes while dramatically reducing payload size.
Solution Approach 2:
The patent creates a lightweight copy of the database schema in the form of a bootstrapped graph structure. This copy contains only the topological relationships and metadata needed for query optimization, not the actual data. The copy enables fast service-to-service communication about data relationships while the full data remains in the original databases, minimizing payload size during communication.
3Adaptability or versatility
If a complete graph database is maintained to support lateral filtering across multiple databases, then query flexibility is improved, but data storage and maintenance complexity increase
Solution Approach 1:
The patent implements a dynamic bootstrapped graph structure that can be regenerated or updated based on changes in the underlying relational databases. The graph structure is built on-demand using metadata extraction, allowing it to adapt to schema changes without requiring full data reimportation. This dynamic approach maintains query flexibility across multiple databases while avoiding the storage overhead of static, complete graph databases.
Data Source
AI summary
A global filter allows data filtering using attributes across multiple Analysis Tools (ATs), by broadcasting semantic filter context objects. Upon selecting object attribute values, the filter context object is created with attribute names and values. A processing engine resolves the filter context object to a data object, and then subsequently to target data. A lateral filter finds related entities in a relational database, without having to maintain and/or duplicate all of the data into a graph database. The processing engine resolves lateral filters using an entity graph path calculation conducted in conjunction with the generation of a bootstrapped graph structure. That graph structure is constructed (bootstrapped) utilizing available database schematic information—e.g., pre-calculated (key) relations and metadata read from the relational database. From that information, relationships in the bootstrapped graph structure are determined. Possible paths between entities are used to generate an optimized SQL query to reach target data.