Graph calculation method based on key value pair storage

Abstract

The invention provides a graph calculation method based on key value pair storage, which comprises the following steps that: a server loads an original graph data set and stores the original graph data set into a memory according to a key value pair mode; for a graph calculation task, a traversal index is added to key value pair storage. The server receives the graph calculation request sent by the client, analyzes the graph calculation request and sends the graph calculation request to the graph calculation engine for execution. The graph calculation engine accesses graph data through the traversal index, updates key vertexes belonging to keys in the local key value storage, and sends the updated key vertexes to a remote server; and update data sent by other servers is received, and thenthe local data is updated. The above steps are repeated until all calculations are completed, and a calculation result is returned to the client. According to the method, traversal indexes are used, the graph data traversal speed is increased, meanwhile, the distribution characteristics of the key value pairs are fully utilized for data propagation and updating, communication expenditure is reduced, and efficient graph calculation can be conducted in the storage mode of the key value pairs.

Description

technical field [0001] The invention relates to a communication method for distributed computing, in particular to a communication method for processing query tasks in a distributed system. Background technique [0002] With the development of the Internet and the advent of the era of big data, more and more attention has been paid to the research of graph data. Many relational data can be abstracted into graph-structured data, including social network data, knowledge graphs, and links between web pages. At present, the research on graph data mainly includes two aspects: graph computing system and graph query system. [0003] Graph computing systems usually run complex algorithms on the whole graph, such as Pagerank, SSSP (single-source shortest path), etc., need to traverse all vertices of the whole graph, and return the result after multiple rounds of iterations. Therefore, in order to obtain better performance, graph computing uses a traversal access-friendly compressed...

AI Technical Summary

Problems solved by technology

However, implementing efficient graph computing directly in the key-value pair storage faces many challenges: first, when the graph computing engine traverses and accesses all vertices, it needs to search through the hash table, which will cause a lot of performance overhead; second, In the key-value pair storage mode, how does the graph computing engine perform efficient data dissemination and update

[0006] Therefore, how to design a graph computing method based on key-value pair storage, which can overcome the shortcomings of the original key-value pair storage that is not efficient enough for graph data traversal, and reduce the overhead of data transmission in the graph computing process, has become a key-value pair storage. Graph Computing Problems

Images