Method and device for processing graph data

Abstract

The embodiments of the invention disclose a method and a device for processing graph data, and relates to the technical field of data mining for increasing graphic calculation efficiency. The method comprises: obtaining edge data in the graph data and a computational algorithm for conducting graphic calculation on the graph data; based on the computational algorithm, determining a partitioning algorithm for partitioning the edge data in the graph data, the partitioning algorithm referring to an algorithm which partitions the edge data in the graph data in accordance with nodes which constitute the edge data, the nodes constituting the edge data referring to source nodes and / or destination nodes that constitute the edge data; partitioning the edge data in the graph data into N initial edge data blocks in accordance with the partitioning algorithm and memory capacity of local memory, each initial edge data block of the N initial edge data blocks having a data capacity less than the memory capacity, wherein N is equal to or greater than 1 and is an integer; arranging the edge data of each initial edge data block of the N initial edge data blocks in accordance with the nodes the constitute the edge data, and obtaining N target edge data blocks.

Description

technical field [0001] The invention relates to the technical field of data mining, in particular to a method and device for processing graph data. Background technique [0002] In the era of big data mining, graphs can not only directly describe many practical applications in the fields of computer science, chemistry, and bioinformatics, such as social networks, web (web) graphs, chemical substances, and biological structures, but also can be used to describe various A data mining algorithm, such as matrix factorization or shortest path, etc. Among them, the graph includes a plurality of nodes and the edges connecting each node, the graph data includes the node data of each node and the edge data of the edges connecting each node, and the edge data of an edge includes the source node, the destination node and the The weight of the edge. In a stand-alone graph computing processing platform (that is, a processing platform that uses a single computer for graph computing), du...

AI Technical Summary

Problems solved by technology

In this case, after the computer reads an edge data block into the local memory, it is necessary to obtain all incoming edge data of multiple destination nodes in the edge data block (that is, the source node is other nodes, and the destination node is the edge data of this node. ) and all outgoing edge data (that is, the source node is the node, and the destination node is the edge data of other nodes), so as to calculate the node data of multiple destination nodes in the edge data block in turn, in order to quickly obtain the destination node. To input all edge data, it is necessary to reorder the edge data in the edge data block according to the order of the destination node ID from small to large, and the reordering process takes a long time, making the efficiency of graph calculation low

Images