A distributed data stream clustering method and system

Abstract

The invention discloses distributed data stream clustering method and system and overcomes the defect that the existing most data steam clustering algorithms are unable to run in the distributed cloud environment, unable to easily extend and low in operational time efficiency. The method includes: summarizing data streams to obtain a plurality of eigenvectors of the data streams; performing locality-sensitive hashing algorithm to obtain a plurality of clusters with each comprising at least one eigenvector, and selecting at least one cluster as a candidate cluster; periodically using the candidate cluster to cluster eigenvectors of newly arrived data streams. The real-time performance better than that of the prior art is guaranteed by the use of the method and system based on the locality-sensitive hashing algorithm.

Description

technical field

[0001] The invention relates to a data stream clustering technology, in particular to a distributed data stream clustering method and system. Background technique

[0002] In recent years, with the wide application of computer technology and network technology in industrial production, information processing and other fields, data is no longer limited to traditional static forms such as files and databases. A continuous, unbounded, and variable-speed streaming data has appeared in more and more application fields. These application areas are usually systems with multiple data sources, such as intrusion detection systems, e-commerce, telecommunications, distributed sensor networks, meteorological monitoring, real-time analysis of scientific data, and peer-to-peer (P2P) computing and other application scenarios. In these applications, a large amount of high-dimensional data flows to the data collection center at a high speed, and clustering such data in real t...

Images