Directed digraph strongly-connected component analysis method based on MapReduce

Abstract

The invention discloses a directed digraph strongly-connected component analysis method based on MapReduce. The method includes the following steps that firstly, a sub-node is randomly selected in a digraph, a label of an output edge of the sub-node and a label of an input edge of the sub-node are set as +1 and -1 respectively, a to-be-processed state is set, and the edges are represented according to four-tuples; then, labels of neighbor nodes of the edges in the to-be-processed state are set to be +1 and -1 by a Map function according to the type of the output edge and the type of the input edge; the edges with the same node identifier are recorded and form a set by a Reduce function, the states of the edges are updated into non-processed labels and states; the Map function and the Reduce function are executed circularly till no edges in the to-be-processed state exist, and corresponding nodes containing the labels of +1 and -1 belong to the same strongly-connected component; nodes of the component are deleted from the digraph, and the steps are repeated till the digraph is empty. In the computing process, the labels of the output edge and the input edge are processed at the same time, the states of the edges are set, reverse operation and repeated calculation are avoided, and therefore the calculation quantity is reduced, and the analysis efficiency is improved.

Description

technical field [0001] The invention relates to the technical field of parallel computing, in particular to a method for analyzing strongly connected components of directed graphs based on MapReduce. Background technique [0002] With the rise of network science, analysis methods based on network and graph mining have received more and more attention, and have been widely used in various fields of physics, biology, political economy, Internet, engineering development, and social life. In the past decade, researchers have abstracted real-world data into graphs, and used methods from graph theory, data mining, machine learning, pattern classification, information retrieval, and statistical inference to reveal hidden patterns and interaction rules behind the data. , thus providing people with an unprecedented understanding of cognitive objects. At the same time, with the development of the information industry and the whole society, more and more data are collected. Another ch...

AI Technical Summary

Problems solved by technology

This algorithm effectively solves the problem of strongly connected components of large-scale directed graphs, but each iteration requires reverse operations on the graph, and at the same time needs to calculate and update node label information twice. Many scholars believe that this will affect the performance of the algorithm. Computational efficiency, longer processing time

Images