Distributed Graph Storage with Global Address Space

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Relational storage systems are more commonly used than graph storage systems, which limits the ability to efficiently handle complex data structures and scalable, fault-tolerant storage for graph applications, particularly in supporting interactive queries and real-time analysis.

Innovation Solution

A distributed graph storage system that employs a global address space, buddy memory allocator, and minitransactions to manage memory allocation and ensure concurrency-safe access, allowing for online data migration and fault tolerance, thereby supporting scalable and high-throughput storage for interactive graph queries.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of manufacture

If relational storage systems are used, then common utilization and simplicity are improved, but the ability to efficiently handle complex data structures and graph applications deteriorates

Engineering Contradiction:

Improvecommon utilizationVSAvoidability to handle complex data structures

Core Design Contradiction:

Ease of manufactureVSAdaptability or versatility

Solution Approach 1:

The system segments graph data into vertices and edges stored in separate hash tables, allowing independent optimization of each data structure. This segmentation enables efficient handling of complex graph relationships while maintaining the simplicity of relational storage operations.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent changes the storage parameter from traditional relational tables to a hybrid structure combining hash tables for vertices/edges with property graphs for attributes. This parameter change enables efficient graph traversals and complex query handling while preserving ease of data insertion and retrieval.

Inventive Principle:

Principle #35Parameter changes

2Adaptability or versatility

If graph storage systems are used, then direct mapping to object oriented applications is improved, but common utilization and adoption deteriorates

Engineering Contradiction:

Improvemapping to object oriented applicationsVSAvoidcommon utilization

Core Design Contradiction:

Adaptability or versatilityVSEase of manufacture

Solution Approach 1:

The system implements a universal storage interface that can handle both traditional relational queries and graph-specific operations. The property graph model serves multiple functions by storing both structural relationships and attribute data in a unified framework, enabling broad applicability across different application types.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent introduces an intermediary layer that translates between relational query interfaces and graph storage operations. This intermediary enables common utilization of graph storage systems by providing familiar relational access patterns while leveraging the underlying graph structure for efficient complex data handling.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Reliability

If distributed storage is implemented, then scalability and fault tolerance are improved, but system complexity and coordination overhead deteriorates

Engineering Contradiction:

Improvefault toleranceVSAvoidsystem complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The distributed system segments the graph into partitions distributed across multiple nodes, with each node maintaining independent vertex and edge hash tables. This segmentation reduces coordination complexity by allowing autonomous operation of individual nodes while providing fault tolerance through distributed redundancy.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system implements copying mechanisms for data replication across distributed nodes, ensuring fault tolerance through redundant copies. This copying approach simplifies the complexity of distributed consistency by using standard replication techniques rather than complex consensus protocols for all operations.

Inventive Principle:

Principle #26Copying

4Adaptability or versatility

If online data migration is implemented, then data balancing and adaptability are improved, but transaction overhead and coordination deteriorates

Engineering Contradiction:

Improvedata balancingVSAvoidtransaction overhead

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The system performs preliminary actions by pre-computing migration candidates and preparing data transfer batches before actual migration occurs. This preliminary action reduces transaction overhead during execution by minimizing the number of individual coordination operations needed during the actual data migration process.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

Online data migration is implemented as a continuous process that operates alongside normal graph queries and updates. The migration process maintains continuity by allowing read/write operations to proceed while data is being migrated, reducing the time loss associated with stopping the system for data balancing operations.

Inventive Principle:

Principle #20Continuity of useful action

Data Source

PatentUS8938599B2Distributed graph storage system

Publication Date: 2015.01.20 HEWLETT PACKARD ENTERPRISE DEV LP

AI summary

In a method of implementing a graph storage system, the graph storage system is stored on a plurality of computing systems. A global address space is provided for distributed graph storage. The global address space is managed with graph allocators, in which a graph allocator allocates space from a block of the distributed global memory in order to store a plurality of graphs.