Concurrent Adaptive Graph Storage for Path-Based Dataset Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems, both relational and graph-based, face inefficiencies in executing path-oriented searches, particularly in large or complex datasets, due to differences in schema and the inability of traditional indexes to handle relationship information between records.
Innovation Solution
A method is implemented where data objects are stored concurrently in both a non-graph and a graph data repository, with metadata being transformed into nodes and edges in the graph repository, allowing concurrent searches in both systems to optimize path-based queries, and results are mapped back to the non-graph format for display.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If data is stored only in a non-graph database system, then storage simplicity is maintained, but path-based query efficiency deteriorates
Solution Approach 1:
The system segments data storage into two separate repositories: a non-graph database for general data storage and a graph database specifically for storing relationship data. This segmentation allows each repository to be optimized for its specific function, with the graph database handling path-based queries efficiently while the non-graph database maintains storage simplicity.
Solution Approach 2:
The graph database acts as an intermediary between the non-graph database and path-based query requirements. It receives data objects from the non-graph database, extracts relationship information, and stores it in graph format, enabling efficient path-based queries without requiring the primary database to change.
2Productivity
If data is stored concurrently in both non-graph and graph repositories, then path-based query efficiency is improved, but system complexity increases
Solution Approach 1:
The system merges two different database technologies (non-graph and graph databases) into a unified federated system that handles both general data storage and relationship-based queries. This combination allows the system to leverage the strengths of both approaches while managing complexity through standardized interfaces and automated data synchronization.
3Ease of manufacture
If traditional database indexes are used, then storage simplicity is maintained, but relationship search capability deteriorates
Solution Approach 1:
The graph database serves as an intermediary that specifically handles relationship search operations. It receives data objects, extracts relationship information, and stores it in a graph structure optimized for traversing connections between entities, thereby enhancing relationship search capability without affecting the simplicity of the primary storage system.
Data Source
AI summary
A method of managing digital entities in data repositories comprises storing one or more data objects in a non-graph data repository into one or more nodes and edges of a graph, comprising transforming an access control list (ACL) of a first data object into an ACL node and transforming a version of a second data object into a version node in a graph data repository; electronically receiving a search query associated with a user account for a shortest path between two specified nodes of the graph; executing the search query against the graph data repository to generate a result set of nodes including only nodes corresponding to most recent versions of the one or more data objects that are visible to the user account under applicable ACLs.