Dynamic short text cluster searching method

Abstract

The invention discloses a dynamic short text cluster searching method. According to the method, short text stream data are used for building a short-term topic model and a long-term historic topic model is synthesized to amend the short-term topic model in a data stream to obtain the probability distribution of topics and feature words, clustering is performed by the conditional probability of thetext and the topics, and dynamic accurate searching of the keywords is formed. The dynamic topic model is built, the keyword searching function changing along the time is realized, the problems of sparsity of the short text data, information loss and the like are solved by a polynomial mixed topic model, and the efficiency and the performance of information searching are improved.

Description

technical field [0001] The invention belongs to the field of document retrieval, and specifically relates to a clustering retrieval algorithm based on a dynamic polynomial mixed topic model. Background technique [0002] The multinomial mixed topic model is an important method for topic detection in the text mining neighborhood. Among them, the latent Dirichlet distribution model (LDA) is one of the most famous and simplest methods in the topic model. It uses the probability distribution of words in the document to input The document set is decomposed into potential topic sets. The multinomial mixed topic model assumes that each document is generated by a mixed model, and there is a one-to-one correspondence between each mixed part and a cluster, and iteratively traverses all documents in the text stream and calculates the conditional probability of the document and each cluster , to re-partition the document to obtain the final topic model. [0003] Clustering algorithm i...

AI Technical Summary

Problems solved by technology

[0005] Aiming at the problem of accurate retrieval of massive short text streams in social media, the present invention proposes a method based on dynamic polynomial hybrid Clustering Retrieval Method of Topic Model

The present invention realizes the keyword retrieval function that changes with time by establishing a dynamic topic model, and solves the problems of sparsity of short text data and lack of information through the polynomial mixed topic model, and improves the efficiency and performance of information retrieval

Images