A distributed social network structure fast mining system for large-scale network traffic

Abstract

The invention provides a distributed social network structure rapid mining system facing large-scale network traffic. The distributed social network structure rapid mining system is used for carryingout distributed allocation and storage on the large-scale network traffic generated at a high speed and mining a topological structure in a social network, and comprises three data processing, networkstructure mining and data storage subsystems; the distributed social network structure rapid mining system faces the large-scale network traffic and proposes an original distributed data allocation and storage model; by proposing the original distributed data allocation and storage model, the topological structure in the social network can be subjected to different particle sizes of measurement;the distributed social network structure rapid mining system is applicable to large-scale streaming data and can be used for network topological structure measurement; and user identity prediction, community detection, abnormal behavior mining and the like can also be carried out on the basis of a measurement result of the network topological structure.

Description

technical field [0001] The invention belongs to the technical field of data mining, in particular to a distributed social network structure rapid mining system for large-scale network traffic. Background technique [0002] With the rapid development and wide application of Internet technology, the social network composed of users is becoming larger and larger. For the analysis of a single user, the network structure information that can be obtained is relatively limited. Mining the topological structure of the social network where the user lives can provide a deeper understanding of the network environment where the user lives. It is useful for understanding user characteristics from multiple dimensions, predicting user identities, mining abnormal groups in social networks, and maintaining public security. Network security and building a harmonious network environment are of great significance. [0003] In the social network built by users, network traffic is the main medi...

AI Technical Summary

Problems solved by technology

On the one hand, some methods are based on pool sampling or sampling with a fixed probability, which can perform real-time measurement of dynamically generated stream data, but the error is high, and no data distribution scheme for measurement on a distributed system is proposed; on the other hand , some methods first sample large network data to form sub-networks, and then use the parallel and fast computing power of distributed systems to mine and count the triangular topology of sub-networks, but they cannot be directly migrated to social networks formed by large-scale network traffic. Real-time mining of network structure

Images