Distributed-structure-based big data clustering method and device

Abstract

The invention provides a distributed-structure-based big data clustering method. The distributed-structure-based big data clustering method comprises the following steps: S100, preprocessing big data; S200, segmenting and managing the big data; S300, establishing a clustering hypergraph model; S400, mapping the big data, specifically, respectively mapping segmented data blocks to the hypergraphs H=(V, E), namely mapping each data block to one hypergraph; S500, clustering all the data blocks respectively by using the hypergraphs; S600, reclustering clustering results of all the data blocks, obtained in the step S500, to obtain a final clustering result. According to the distributed-structure-based big data clustering method, by using a cloud platform in combination with a hypergraph theory, the big data is mined and clustered, so that fast, real-time and accurate analysis and processing of the big data are achieved.

Description

technical field [0001] The invention relates to the field of data mining, in particular to a large data clustering method and device based on a distributed structure. Background technique [0002] Over the past half century, with the full integration of computer technology into social life, the information explosion has accumulated to a degree that has begun to trigger changes. Not only is it flooding the world with more information than ever before, but its growth rate is accelerating. The subject of information explosion, such as astronomy and genetics, created the concept of "big data". Today, this concept is applied to almost all areas of human intelligence and development. The 21st century is an era of great development of data and information. Mobile Internet, social network, e-commerce, etc. have greatly expanded the boundaries and application scope of the Internet, and various data are rapidly expanding and becoming larger. Internet (social, search, e-commerce), m...

AI Technical Summary

Problems solved by technology

In the face of big data processing, the method based on sample sampling probability is generally adopted, but the sampling method does not consider the overall relative distance between data points or intervals and the uneven distribution of data, resulting in the problem of hard division of intervals

Although later, clustering, fuzzy concepts, and cloud models were introduced to improve the problem of interval division and achieved good results, but these methods did not consider the different effects of big data data points on knowledge discovery tasks

Images