Cluster-based graph data division method

Abstract

The invention discloses a cluster-based graph data division method. The method comprises the steps of determining a degree value of each vertex in V vertexes comprised in graph data, wherein the graph data comprises E edges; according to the degree value of each vertex, sorting the V vertexes according to a degree value sequence from big to small, and determining M vertexes with the degree values greater than a first threshold; according to the V vertexes, the E edges and the M vertexes, determining T paths; attributing the vertexes which T paths do not comprise in the V vertexes to the T paths, forming the T paths after the attribution, or creating N paths by the T paths; according to the M vertexes, the T paths and the N paths, determining M clusters; and according to an association degree among the M clusters and a preset divided block number P, dividing the M clusters of the graph data into P divided blocks. According to the method, the vertexes with the relatively big degree values serve as end pintos of the paths, so that the repetition rate of graph data division can be reduced; the graph scale is reduced by taking the paths as units, so that the memory overhead is reduced; and large-scale graph data can be processed.

Description

technical field [0001] The invention belongs to the field of big data processing, and more specifically relates to a method for dividing graph data based on clusters. Background technique [0002] Data mining of real-world relational graphs is a hot topic nowadays, such as social network analysis, web page ranking. Whether it is multi-threaded parallel processing on a single machine or using a distributed system to process these graphs, the key step of graph data partitioning cannot be avoided. However, graph data has some characteristics such as large data scale and complex relationship, so it is a very challenging task to properly divide it. The graph size reflects the number of vertices and edges included in the graph data. If the graph data includes a large number of vertices and edges, it is large-scale graph data. [0003] In recent years, with the popularization of graph computing, graph data partitioning algorithms have emerged in an endless stream. Random hash (...

AI Technical Summary

Problems solved by technology

[0005] In view of the defects of the prior art, the purpose of the present invention is to solve the technical problem that the prior art graph data division method has a high repetition rate, the division result is not ideal, or the memory overhead is large, and the division of large-scale graph data cannot be handled well.

Images