Graph data storage method and system based on distributed memory cloud

Abstract

The invention relates to a graph data storage method and system based on distributed memory cloud. The system comprises a graph service layer which is used for allocating virtual nodes to each serverin the distributed memory cloud, mapping the virtual nodes to a Hash ring according to Hash values of the virtual nodes, storing a mapping relationship as a Hash mapping table, obtaining graph data stored by each server, searching the Hash values of the graph data on the Hash mapping table, storing the graph data to the corresponding virtual nodes, allocating new virtual nodes to new serves in thedistributed memory cloud when the new servers are increased, updating the Hash mapping table, adding the new virtual nodes to the Hash ring according to the Hash values of the new virtual nodes, andcarrying out data migration on each virtual node in the Hash ring according to the Hash values of the graph data stored by the virtual nodes and the updated Hash mapping table. Through utilization ofa consistent Hash algorithm, the method and the system support dynamic expansion, the nodes can be increased or deleted, and the loss can be reduced as much as possible when faults occur in the nodes.

Description

technical field [0001] The invention relates to the field of graph data storage, in particular to a graph data storage method and system based on a distributed memory cloud. Background technique [0002] In the past ten years, with the popularization of Internet technology and the promotion of Web2.0 technology, Internet social relationship data has grown rapidly. According to the "36th China Internet Development Status Report Technology" released by CNNIC, as of June 2015, micro The number of blog users reached 162 million, and the utilization rate reached 27.3%. The storage and computing problems brought about by the explosive growth of relational network data have become increasingly prominent, and the storage and analysis of massive data relational network graphs are facing great challenges. [0003] Graph storage system, where graph storage refers to the storage of structured data (called a graph from a mathematical point of view) on the network, and can be divided int...

AI Technical Summary

Problems solved by technology

[0022] When investigating memory-based graph databases, it was found that Microsoft has a closed-source implementation of Trinity, which is a solution for storing graph data in memory, but the frequency of use is not high, and this technology cannot be dynamically expanded. The study found that the solution to this defect can be achieved through a consistent hash algorithm

[0023] When studying the distributed computing of the graph engine, it was found that large network transmission between computing nodes affects computing efficiency, but there is locality in distributed graph computing. In order to solve this problem, the inventor investigated some graph computing framework, and finally use the memory cloud subgraph partition algorithm to solve the problem

Images