Subtopic discovery method for semi-structure short text set based on mutual-constraint topic model

Abstract

The invention relates to a subtopic discovery method for a semi-structure short text set based on a mutual constraint topic model. The main technical characteristics are as follows: applying data cleaning to a short text set containing topic tags; extracting a short text containing specified seed topic tags for a certain topic based on seed topic tags; generating an input file for the cleaned data; inputting the input file to a mutual constraint topic model for model training; obtaining a semantic vector representation of the topic tags in the set, an average semantic vector representation of the text where the topic tags are and a lexical vector representation of the text where the topic tags are; orderly connecting the three vector representations as a complete semantic representation for one topic tag; and clustering by use of a Kmeans clustering method and outputting the centroid of a class obtained by clustering as a subtopic. According to the invention, the design is rational; mutual constraint potential topic modelling is adopted to solve the problems of high sparsity and great noise existing in the traditional semi-structure short text topic semantic modelling technology.

Description

Technical field [0001] The invention belongs to the field of data mining technology, and in particular is a method for discovering subtopics in a semi-structured short text set based on a mutual constraint topic model. Background technique [0002] The exploration and automatic modeling of the topic structure of short microblog texts has increasingly become a hot research topic. This technology is very important for the acquisition of automatic information and knowledge. However, due to the short length of Weibo short text itself, sparse vocabulary, and irregular writing, etc., serious problems of high sparseness and high noise in the data have been caused, making it difficult for traditional topic models (such as LDA, PLSA) to directly model and get Weibo short texts The topic semantic information in this book. In response to the above problems, the researchers adopted a data expansion method to transform short texts into long texts for modeling. The typical technical solutions...

AI Technical Summary

Problems solved by technology

[0003] The purpose of the present invention is to overcome the deficiencies of the prior art, and propose a method for discovering subtopics in semi-structured short texts based on a mutual constraint topic model. Semantic means to obtain an effective topic structure in short text collections, which solves the problems of high sparsity and high noise faced by existing semi-structured short text topic semantic modeling techniques

Images