Method and device for text clustering

Abstract

Embodiments of the present invention provide a method and a device for text clustering, and relates to the technical field of text processing. After determining k text data which was obtained text feature vector concentrated as the center point of k cluster, and traversing the text feature vectors set, dividing each text datum into the cluster with the highest similarity, and then executing for each cluster: calculating a mutual information between each word in a text data of a cluster and the clustered data in the cluster, and ranking the calculated amount of mutual information, based on a preset evaluation function of the mutual information according to the method. Extracting T words with high mutual information quantity as the feature vector of the text data, and determining an averagevalue of the feature vectors of all text data in the calculated cluster as a new center point of the cluster, until the preset condition is satisfied. The method and device for text clustering reducing the influence of an isolated data point on the calculation of the new center point, and the method is more accurately, and in the condition of cluster characteristics are ensured to be complete, reducing the dimension, so that the calculation efficiency is improved.

Description

technical field [0001] The present invention relates to the technical field of text processing, in particular to a text clustering method and device. Background technique [0002] Text clustering (Text clustering) document clustering is mainly based on the well-known clustering assumption: documents of the same type have a greater similarity, while documents of different types have a smaller similarity. As an unsupervised machine learning method, clustering has certain flexibility and high automatic processing ability because it does not require training process and manual labeling of documents in advance, and has become an effective method for text information. An important means of organizing, summarizing and navigating, attracting more and more researchers' attention. The center point calculated by the existing K-means conventional text clustering algorithm has high dimensions, low calculation efficiency, and is sensitive to noise and isolated data points. A small amount...

AI Technical Summary

Problems solved by technology

The center point calculated by the existing K-means conventional text clustering algorithm has high dimensions, low calculation efficiency, and is sensitive to noise and isolated data points. A small amount of noise and isolated data will have a great impact on the average value , affects the speed of cluster convergence, and also increases the computational cost

Images