A clustering method based on vision principle to solve big data clustering

Abstract

The invention discloses a visual principle-based clustering method for solving big data clustering. By performing lossless multi-scale encoding on the original data with a given precision, the multi-scale and multi-dimensional grid storage of the data is realized. Each scale code judges the similarity between codes and neighborhood codes, and uses connectivity analysis to realize multi-scale clustering and provide multi-scale clustering results. In the process of data encoding, the principle of vision is used, which conforms to Weber's law, that is, the difference threshold of sensation changes with the change of the original stimulus amount.

Description

technical field [0001] The invention belongs to the field of big data clustering, and in particular relates to a visual principle-based clustering method for solving big data clustering. Background technique [0002] Clustering is a knowledge discovery method that divides data into different groups according to some similarity (such as structure or trend) of data. Measuring the similarity between data is the basis of clustering. Usually, the similarity between each point is stored in the form of a matrix. For large-scale or distributed data, this method will lead to a huge amount of data transmission, slow calculation efficiency, and even due to the huge matrix Unable to store the problem. [0003] The reason for these problems is that the similarity is stored in a dense matrix, and the data volume increases at the square speed of the original data volume. [0004] There are currently two types of big data clustering algorithms: [0005] A divisional clustering method wit...

AI Technical Summary

Problems solved by technology

This is difficult to meet in big data and distributed situations, so this method is also difficult to meet the needs of clustering

[0007] Clustering problems are the basis of information processing methods such as artificial intelligence and machine learning. There are many excellent clustering algorithms, but they are difficult to implement in the environment of big data computing, and the existing big data clustering methods are difficult to meet the needs of use.

Images