k-bisimulation calculation method based on GPS (global positioning system) platform

Abstract

The invention discloses a k-bisimulation calculation method based on a GPS (Global Positioning System) platform. The method comprises the following steps of graph pre-division processing and the distributed calculation of k-bisimulation features, wherein the graph pre-division processing is used for reducing an number of edges which are distributed to graphs on different working nodes, and guaranteeing the scale of a point processed by each working node to be equivalent; the distributed calculation of k-bisimulation features comprises the following steps: updating point feature numbers, calculating the numbers of node features and calculating the k-bisimulation division feature of each point in the graph; and the graph pre-division processing and the distributed calculation of the k-bisimulation features are both based on a GPS distributed graph data processing platform of node programming and are used for stipulating space locality. Algorithm operation efficiency is drastically improved by the frame and the programming thought of a distributed graph processing platform, a network data transmission amount can be only 1 / 2-1 / 12 of the network data transmission amount of a MapReduce calculation model algorithm. In addition, for the calculation of the local bisimulation division of a big graph which contains several hundred millions of edges and points, the algorithm disclosed by the invention is characterized in that required calculation time is only 1 / 7-1 / 16 of the required calculation time of the MapReduce calculation model algorithm.

Description

technical field [0001] The invention belongs to the technical field of image data platform processing, in particular to a k-bisimulation calculation algorithm based on a GPS platform. Background technique [0002] Among numerous graph-based studies, mutual-modular partitioning plays a vital role in many application domains. Intuitively, the mutual modeling partition of a graph refers to the operation of dividing the points in the graph according to the predefined "features", and the points with the same "features" are divided into the same set. "Features" can be flexibly defined according to needs; mutual model division has a wide range of applications. In the field of data compression, points in the same collection use the same number; through appropriate definitions, mutual model partition can be used to build structured indexes for XML and RDF databases; in addition, mutual model partition in data query optimization and analysis There are also clear applications. With ...

AI Technical Summary

Problems solved by technology

The local mutual modeling calculation algorithm of the graph needs to use multiple rounds of iterative operations. Using Hadoop platform processing requires careful design of multiple rounds of MapReduce operations. The Map operation in each round of operation will store the intermediate results on the local disk, and the Reduce operation will save the intermediate calculations. The results are stored on HDFS, and read and write operations on disk or HDFS will cause performance bottlenecks

In addition, each MapReduce iterative operation will transmit the state of all vertices and edges in the entire graph through the network, and re-allocate computing nodes for vertices and edges, which will inevitably bring a heavy network communication burden

Although the scalability problem of mutual modulus partition can be solved based on the MapReduce model, using the k-mutual modulus partition feature of Hadoop platform calculation points will bring heavy global network communication costs, which will affect the efficiency of the algorithm itself

Images