Distributed large-scale complex community detection method and device based on node degree

Abstract

The invention discloses a distributed large-scale complex community detection method and device based on node degree. The method comprises the following implementation steps: preparing relational network data; Loading the relational network data to a distributed file system for network segmentation, customizing a message transfer function, a message fusion function and a message processing function based on a label propagation algorithm of the node degree, and then loading the message transfer function, the message fusion function and the message processing function into the distributed file system; Configuring operation parameters of the distributed file system; And calling the distributed file system to execute community detection and outputting a community detection result. The method is suitable for community detection of a large-scale complex network, and has the advantages of high accuracy, good stability, strong parallel computing capability, good community detection universality and adaptability, and simple and efficient development process.

Description

technical field [0001] The invention relates to the field of large-scale complex network data mining and processing, in particular to a distributed large-scale complex community detection method and device based on node degree. Background technique [0002] Complex networks are abstract representations of complex systems. Many complex systems in real society can be abstracted in the form of networks, such as social networks, transportation networks, and molecular networks. With a large number of studies on complex networks, community structure is a common feature of these networks. Exploring the characteristics of communities in complex networks will help to better understand and master the functions and structures of complex systems. [0003] The community structure means that the vertices in the network can be divided into groups, the connections between the vertices in the group are relatively dense, and the connections between the vertices in the groups are relatively sp...

AI Technical Summary

Problems solved by technology

However, this algorithm also has some disadvantages: (1) The update order of nodes is random, and when there are multiple labels (maximum value labels) that appear most frequently in adjacent nodes, a label will be randomly selected; (2) There are different Necessary label update process; (3) When updating labels, only consider the number of label occurrences in adjacent nodes, ignoring the local structure information of adjacent nodes

For social networks, the number of nodes is tens of millions and the structure of nodes is intricate. Traditional community discovery algorithms can no longer effectively mine the community structure.

Images