Large-scale social network service-oriented graph data storage and query method

Abstract

The invention discloses a large-scale social network service-oriented graph data storage and query method. A data storage manager stores received graph data in a Key-Value way, wherein the vertex ID (Identity) of the graph data is taken as the Key, a vertex neighbourhood is taken as the Value, and the data of each vertex neighbourhood is stored; and a plurality of edges connected with the vertex neighbourhood are orderly stored in a memory block with a fixed size via a timestamp, a double linked list is formed, and the attribution information and the index information of the vertex are stored in a data structure. When the data storage manager receives an access request that the vertex v is accessed, the data storage manager transmits the vertex v and a k-order neighbourhood of the vertex v to a requester; and the requester caches stored data locally, firstly checks local cache during next query, and sends the access request to the data storage manager if no queried vertexes exist. The graph data storage and query method can meet a scene which is dynamically updated and is suitable for processing data sparseness, and random access.

Description

technical field [0001] The invention relates to a large-scale social network-oriented graph data storage and query method, which belongs to the field of software technology. Background technique [0002] At present, the mainstream method of graph data storage is to preprocess the graph data into records of edges and vertices, and store them in large files in the distributed file system in the form of sequential data sets. When accessing graph data, the large files storing the graph data are accessed in a sequential scan. This organization method cannot provide effective data storage and access performance for multi-round iterative graph computing applications. In order to improve the access performance of graph data, the memory management technology of graph data has become an important trend, such as Trinity, Giraph, etc. [0003] Neo4j is a graph database using the Key-Value storage model. The most basic storage units are vertices and edges. When the breadth traverses BFS...

AI Technical Summary

Problems solved by technology

The data division of Redis is completed by the client, and the server program cannot proxy the request of different map data

[0009] From the above research work, it can be seen that the data access requirements of graph computing and the characteristics of graph data pose challenges to the storage of graph data. Traditional data storage methods such as file systems or distributed file systems and graph databases with Key-Value storage models are difficult. Efficient support for graph data access and update, and thus cannot effectively support graph calculations

Images