Short text topic distribution reasoning method and system, computer equipment and storage medium

Abstract

The invention relates to a short text topic distribution reasoning method and system, computer equipment and a storage medium. The method comprises the steps: extracting co-occurrence word pairs appearing in a short text in unit time, and integrating the co-occurrence word pairs to obtain a word group set; associating the phrase set according to the semantic similarity and the historical co-occurrence degree to obtain a dynamic association degree of the phrase set, and storing the dynamic association degree in a phrase matrix form; extracting theme names from the phrase set, and correcting thetheme names according to the dynamic correlation degree; counting theme names in the corrected short text, and obtaining theme distribution of the short text. Through the index of the designed dynamic correlation degree, each co-occurrence word is endowed with different importance. Moreover, the extraction of the topic names in the method has a biased topic model, so various more continuous and compact topic names can be extracted, and the topic distribution of each short text can be reasoned more accurately.

Description

technical field [0001] This application relates to the field of big data, in particular to a reasoning method, system, computer equipment and storage medium for topic distribution of short texts. Background technique [0002] Topic model is a kind of topic mining and extraction from text data. In order to design a topic model suitable for short text data, researchers usually use several useful strategies. The first one is to limit the number of topics in each short text, which is obtained through the Dirichlet Multinomial Mixture Model (DMM) model, in which it is assumed that each short text contains only one topic through this model. This strategy further simplifies the topic model by limiting the topic information in the target data, in order to more accurately mine, extract and assign topics. Second, build topic models on word patterns that contain sufficient topic information. A typical representative is the Attentional Segments Topic Model (ASTM). ASTM will extract th...

AI Technical Summary

Problems solved by technology

The problem with this strategy lies in two points: 1. How to reasonably use external information to guide the work of topic models; 2. Existing topic models usually only consider semantic information as external information, while ignoring some other important information

Images