Short text clustering method based on deep semantic feature learning

Abstract

The invention discloses a short text clustering method based on deep semantic feature learning. The method includes the steps that dimensionality reduction representation is performed on original features under the restraint of local information preservation through traditional feature dimensionality reduction, binarization is performed on an obtained low-dimension actual value vector, and error back propagation is performed with the binarized vector being supervisory information of a convolutional neural network structure to train a model; non-supervision training is performed on a term vector through an outer large-scale corpus, vectorization representation is performed on all words in text according to the word order, and the vectorized words serve as implicit semantic features of initial input feature learning text of the convolutional neural network structure; after deep semantic feature representation is obtained, a traditional K-means algorithm is adopted for performing clustering on the text. By means of the method, extra natural language processing and other specialized knowledge are not needed, design is easy, deep semantic features can be learnt, besides, the learnt semantic features have unbiasedness, and good clustering performance can be achieved more effectively.

Description

technical field [0001] The invention relates to the field of vectorized representation of text features, in particular to a short text clustering method based on deep semantic feature learning. Background technique [0002] With the widespread popularity of social media, short text clustering has increasingly become an important task, and its main challenge lies in the sparsity of text representation. In order to overcome this difficulty, some researchers try to enrich and expand short text data through Wikipedia or ontology database. However, these methods require a lot of natural language processing knowledge, and still use high-dimensional feature representation, which is easy to waste storage and computing time. Other researchers have tried to develop complex models for clustering short text data. But how to design an effective model is an open problem, and most previous methods are based on the latent layer model of bag-of-words features. [0003] With the rise of de...

AI Technical Summary

Problems solved by technology

However, the recurrent neural network needs to build an additional syntax tree and has high complexity, and the text semantic representation of the recurrent neural network is more focused on the representation information of the last word

Images