Method for efficiently clustering massive data

Abstract

The invention provides a method for efficiently clustering massive data. The method comprises the following steps of: performing graded sampling on original data, wherein the number of grades for sampling is determined by the number of grades to be clustered, the data quantity of sampled data is increased grade by grade, and the magnitude of the last grade of data needs to satisfy two conditions of being more than 5% of the original data and being more than 30 times of the number of total clustering centers; clustering a first grade of sampled data by using a K-mean value clustering algorithm; quantizing a next grade of sampled data to all the centers of the current grade by using a rapid quantification method; respectively clustering the current grade of grouped data subjected to quantification by using a self-adaptive K-mean value clustering algorithm; and converging the clustering center of each group in the current grade into a large center. In the invention, a graded sampling method, a small center number grading and self-adaptive clustering method and a rapid quantification method are used, so that the clustering time is shortened.

Description

technical field [0001] The invention relates to the technical fields of cluster analysis, data mining, knowledge discovery in data, etc., and particularly relates to a large-scale data high-efficiency clustering method. Background technique [0002] In recent years, the rapid development of computer technology and communication technology has brought tremendous changes to human social civilization. People can obtain and store data in a faster, more convenient and cheaper way; the scale, scope and depth of database applications are also increasing. Constantly expanding, a large number of databases are used in business management, government office, scientific research and engineering development; and this momentum will continue to develop rapidly, making the amount of data and information grow exponentially. However, while having large-scale data, we lack sufficient understanding and application of the information contained in the data, and the application of traditional data...

AI Technical Summary

Problems solved by technology

However, there are still many problems in the application of existing clustering methods to large-scale data, and the main problems are timeliness and accuracy.

Images