Subtopic Discovery Method in Semi-structured Short Text Collection Based on Mutually Constrained Topic Model

Abstract

The invention relates to a method for discovering subtopics in a semi-structured short text collection based on a mutual constraint topic model. Its main technical features are: data cleaning of short text collections containing topic tags; A short text containing a specified seed topic tag; generate an input file from the cleaned data; input the input file into a mutually constrained topic model for model training; obtain the semantic vector representation of the topic tag in the collection and the average semantic vector representation of the text And the lexical vector representation of the text where the topic tag is located; the three vector representations are sequentially connected as a complete semantic representation of a topic tag; the Kmeans clustering method is used for clustering, and the centroid of the clustered category is output as a subtopic. The invention has a reasonable design, adopts mutual constraint latent topic modeling, and solves the problems of high sparseness and high noise faced by the existing semi-structured short text topic semantic modeling technology.

Description

technical field [0001] The invention belongs to the technical field of data mining, in particular to a method for discovering subtopics in a semi-structured short text set based on a mutual constraint topic model. Background technique [0002] The exploration and automatic modeling of the topic structure of microblog short texts has increasingly become a hot research topic, and this technology is very important for the acquisition of automatic information knowledge. However, due to the short length of microblog short text itself, sparse vocabulary, and irregular writing, etc., the serious problems of high sparsity and high noise in the data make it difficult for traditional topic models (such as LDA, PLSA) to directly model microblog short texts. Semantic information on topics in this book. In response to the above problems, researchers use data augmentation methods to convert short texts into long texts for modeling. The typical technical solutions are as follows: gather s...

AI Technical Summary

Problems solved by technology

[0003] The purpose of the present invention is to overcome the deficiencies of the prior art, and propose a method for discovering subtopics in semi-structured short texts based on a mutual constraint topic model. Semantic means to obtain an effective topic structure in short text collections, which solves the problems of high sparsity and high noise faced by existing semi-structured short text topic semantic modeling techniques

Images