Topic mining method and apparatus

Abstract

The invention provides a topic mining method and apparatus. When an iterative process is performed each time, a target message vector is determined from a message vector according to a residual error of the message vector; a current document-topic matrix and a current word-topic matrix are updated only according to the target message vector; and a target element in a word-document matrix corresponding to the target message vector is only calculated according to the current document-topic matrix and the current word-topic matrix, so that the calculation of all non-zero elements in the word-document matrix in each iterative process is avoided, the update of the current document-topic matrix and the current word-topic matrix according to all message vectors is avoided, the calculation amount is greatly reduced, the topic mining speed is increased, and the topic mining efficiency is improved.

Description

technical field [0001] Embodiments of the present invention relate to information technology, and in particular, to a topic mining method and device. Background technique [0002] Topic mining is the process of clustering semantically related words in a large-scale document set by using the machine learning model of the Latent Dirichlet Allocation (LDA) model, so as to obtain each word in the large-scale document set in the form of a probability distribution. The topic of a document, that is, the theme expressed by the author through the document. [0003] Topic mining in the prior art needs to first train the LDA model based on the training documents using the Belief Propagation (BP) algorithm, and determine the model parameters of the trained LDA model, namely the word-topic matrix Φ and the document-topic matrix θ, and then input the word-document matrix of the document to be tested into the trained LDA model for topic mining, so as to obtain the document-topic matrix θ'...

AI Technical Summary

Problems solved by technology

Since the BP algorithm contains a large number of iterative calculations, that is to say, the current document-topic matrix and the current word-topic matrix of the LDA model are repeated multiple times to calculate each non-zero element in the word-document matrix to obtain the word-document After the message vector of each non-zero element in the matrix, the process of updating the current document-topic matrix and the current word-topic matrix according to all the above-mentioned message vectors, until the message vector,...

Images