Distributed Directed Acyclic Graph Storage via Object Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing source code repository storage models, particularly those using directed acyclic graphs, face challenges in durability and accessibility due to their centralized storage and replication, especially when frequent changes and traversals are required, as seen in version control systems.
Innovation Solution
A mechanism for distributed storage and retrieval of directed acyclic graphs, where objects are grouped into chunks using relatedness heuristics and stored across multiple computing nodes, with a distributed key/value data storage service creating an index for efficient retrieval and optimization of chunk contents.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If source code repositories are stored in a single location and replicated to other locations, then availability is improved, but durability and accessibility are limited
Solution Approach 1:
The patent segments the directed acyclic graph into individual objects that can be independently stored, retrieved, and replicated across distributed locations. Each object represents a discrete unit of version control data that can be managed separately, enabling both high availability through replication and durability through distributed storage without the limitations of traditional centralized replication models.
2Adaptability or versatility
If source code repositories are stored in a single location and replicated, then availability is improved, but accessibility is limited
Solution Approach 1:
By segmenting the repository into individual objects with unique identifiers, the system enables selective retrieval of specific objects or chunks rather than requiring access to entire replicated copies. This improves accessibility by allowing users to retrieve only the data they need from any distributed location, reducing access complexity and improving ease of operation.
3Adaptability or versatility
If directed acyclic graphs are stored in a replicated system, then availability is improved, but storage complexity increases when new data is added frequently
Solution Approach 1:
The patent divides the directed acyclic graph into discrete objects that can be independently added, updated, and replicated. When new data is added, only the specific new objects need to be processed and replicated, rather than managing entire graph structures. This segmentation dramatically reduces storage complexity for systems with frequent data additions.
Solution Approach 2:
The system pre-computes and stores metadata about object relationships and dependencies before replication occurs. This preliminary organization of data allows for efficient replication and retrieval operations, reducing the complexity of managing frequent data additions in a replicated environment.
4Adaptability or versatility
If directed acyclic graphs are stored in a replicated system, then availability is improved, but traversal efficiency decreases when frequent partial traversals are required
Solution Approach 1:
By organizing the graph as discrete, addressable objects rather than monolithic replicated structures, the system enables efficient partial traversals. Users can retrieve and traverse only the specific objects and relationships they need, rather than loading entire replicated graph copies. This segmentation maintains traversal efficiency even in replicated systems with frequent partial access patterns.
Data Source
AI summary
Technologies are described herein for distributed storage and retrieval of directed acyclic graphs, such as version control graphs maintained by a version control system. In order to store data contained in a directed acyclic graph, objects stored in the graph may be grouped into chunks utilizing a relatedness heuristic. The chunks may then be stored in a distributed object store. An index to the objects stored in the chunks may also be created that includes an index entry for each object. The index entry for each object includes a unique identifier for the object and data identifying the location of the object in the distributed object store. The index may be utilized to traverse the directed acyclic graph and to obtain all or a portion of the objects in the directed acyclic graph from the distributed object store.