Complex network topology characteristic parameter calculation method and system based on MapReduce

Abstract

The invention provides a complex network topology characteristic parameter calculation method and system based on MapReduce. An algorithm parallel method based on message transmission is employed. The method comprises the steps of S1, generating update messages; S2, transmitting the messages; and S3, updating internal state information of nodes. For the problem that the efficiency is relatively low when conventional stand-alone algorithms are used for calculating large-scale network topology characteristic parameters, the invention provides a method for transplanting the stand-alone algorithms for the network topology characteristic parameters to a MapReduce calculation framework in parallel, the problem occurred in the process of transplanting the stand-alone algorithms to the MapReduce in parallel is overcome, the network topology characteristic parameters are calculated in parallel through utilization of a Hadoop calculation platform, and the calculation efficiency of the network topology characteristic parameters is improved.

Description

technical field [0001] The present invention relates to complex networks, in particular to a method and system for calculating topology characteristic parameters of complex networks based on MapReduce. Background technique [0002] First, the relevant terms are explained. [0003] Complex network: A network with some or all of the properties of self-organization, self-similarity, attractor, small world, and scale-free is called a complex network. Including WWW network, Internet, social relationship network, economic network, power network and so on in reality. [0004] Network topology characteristic parameters: Due to the complex network structure, researchers have proposed many concepts and methods to describe the statistical characteristics of complex network structures, which are called network topology characteristic parameters. It mainly includes degree, clustering coefficient, network diameter, average path length, maximum connected subgraph size, number of cores an...

AI Technical Summary

Problems solved by technology

However, in a distributed environment, each node information is stored in a separate text, and searching or modifying neighbor node information requires traversing the entire graph file, which is very inefficient

[0015] 2. The stand-alone algorithm lacks parallelism

[0017] 3. Additional overhead will be generated when parallel transplantation of stand-alone algorithms

In particular, when the number of iterations in the graph algorithm is too large, the MapReduce framework needs to start several jobs in succession to complete the iterative processing of the graph. Each job will perform operations such as starting jobs, task scheduling, and reading and writing disks, especially adjacent jobs. All data is exchanged through the distributed file system, resulting in a large amount of additional overhead, resulting in a decrease in the computational efficiency of the algorithm

Images