Text Data Stream Clustering Algorithm Based on Neighbor Propagation

Abstract

The invention discloses a text data stream clustering algorithm based on neighbor propagation, which is characterized in that the steps are as follows: 1. Perform dimensionality reduction processing on text data sets to obtain corresponding text vector sets; 2. Obtain cluster centers at all times , and complete the clustering algorithm. The present invention can improve the accuracy and robustness of the algorithm without specifying the number of clusters in advance, so as to meet the needs of solving practical problems.

Description

technical field [0001] The invention relates to a clustering algorithm of text data flow based on neighbor propagation. Background technique [0002] With the advent of the big data era, a large amount of unstructured data has been generated on the network. Faced with these unstructured data, which are generated in real time, have a huge amount of data, and have complex structures, people urgently need to extract valuable information and knowledge from them. Text data stream clustering technology is a common method for analyzing these unstructured data. It has achieved good application results in news filtering, topic detection and tracking (TDT), user feature recommendation, etc., and has quickly become a current research hotspot. Since text data has high-dimensional sparse features, how to improve the efficiency and accuracy of clustering algorithms is very important. In 2005, Shi Zhong proposed the OSKM algorithm, which is an extension of the k-means algorithm. It divid...

AI Technical Summary

Problems solved by technology

[0003] This algorithm also has the following disadvantages: the number of clusters needs to be determined in advance for each clustering, and the number of clusters cannot be changed as the category changes.

The algorithm can achieve better clustering results for spherical clusters, but it is difficult to cluster into clusters of arbitrary shapes.

There are also studies that propose an HPStream algorithm, which uses high-dimensional projection technology to select subspaces for clustering, and uses decay functions to represent evolution information, but the prior parameter—the average clustering dimension is difficult to determine

The above improvement studies have adapted to the problems of flow clustering to a certain extent, but the accuracy and robustness of the clustering results have not been well resolved, and further improvement is needed.

Images