Calculation method and system for complex network topology characteristic parameters based on mapreduce

Abstract

The present invention provides a method and system for calculating complex network topology characteristic parameters based on MapReduce, adopting an algorithm parallelization method based on message passing, including: step 1, generating an update message; step 2, transmitting the message; step 3, updating the interior of the node status information. Aiming at the low efficiency of traditional stand-alone algorithms when calculating large-scale network topology characteristic parameters, the present invention proposes a method for parallel transplantation of stand-alone algorithms for network topology characteristic parameters to the MapReduce computing framework, which overcomes the problem of parallel transplantation of current stand-alone algorithms to MapReduce To solve the existing problems, the Hadoop computing platform is used to realize the parallel calculation of the network topology characteristic parameters, which improves the calculation efficiency of the network topology characteristic parameters.

Description

technical field [0001] The present invention relates to complex networks, in particular to a method and system for calculating topology characteristic parameters of complex networks based on MapReduce. Background technique [0002] First, the relevant terms are explained. [0003] Complex network: A network with some or all of the properties of self-organization, self-similarity, attractor, small world, and scale-free is called a complex network. Including WWW network, Internet, social relationship network, economic network, power network and so on in reality. [0004] Network topology characteristic parameters: Due to the complex network structure, researchers have proposed many concepts and methods to describe the statistical characteristics of complex network structures, which are called network topology characteristic parameters. It mainly includes degree, clustering coefficient, network diameter, average path length, maximum connected subgraph size, number of cores an...

AI Technical Summary

Problems solved by technology

However, in a distributed environment, each node information is stored in a separate text, and searching or modifying neighbor node information requires traversing the entire graph file, which is very inefficient

[0015] 2. The stand-alone algorithm lacks parallelism

[0017] 3. Additional overhead will be generated when parallel transplantation of stand-alone algorithms

In particular, when the number of iterations in the graph algorithm is too large, the MapReduce framework needs to start several jobs in succession to complete the iterative processing of the graph. Each job will perform operations such as starting jobs, task scheduling, and reading and writing disks, especially adjacent jobs. All data is exchanged through the distributed file system, resulting in a large amount of additional overhead, resulting in a decrease in the computational efficiency of the algorithm

Images