Reasoning methods, systems, computer equipment and storage media of short text theme distribution

Abstract

The present application relates to a reasoning method, system, computer equipment and storage medium for topic distribution of short texts. The method includes: extracting the co-occurrence word pairs appearing in the short text within a unit time, integrating the co-occurrence word pairs to obtain a phrase set; associating the phrase sets according to the semantic similarity and historical co-occurrence degree, and obtaining the dynamic association of the phrase sets degree, and store the dynamic association degree in the form of a phrase matrix; extract the subject name from the phrase set, and modify the subject name according to the dynamic association degree; count the subject name in the short text after the correction, and obtain Topic distribution of the short text. Through the designed index of dynamic correlation, different importance is given to each co-occurrence word pair. In addition, the topic name extraction in this method has a biased topic model, so that more continuous and compact topic names can be extracted, and the topic distribution of each short text can be inferred more accurately.

Description

technical field [0001] This application relates to the field of big data, in particular to a reasoning method, system, computer equipment and storage medium for topic distribution of short texts. Background technique [0002] Topic model is a kind of topic mining and extraction from text data. In order to design a topic model suitable for short text data, researchers usually use several useful strategies. The first one is to limit the number of topics in each short text, which is obtained through the Dirichlet Multinomial Mixture Model (DMM) model, in which it is assumed that each short text contains only one topic through this model. This strategy further simplifies the topic model by limiting the topic information in the target data, in order to more accurately mine, extract and assign topics. Second, build topic models on word patterns that contain sufficient topic information. A typical representative is the Attentional Segments Topic Model (ASTM). ASTM will extract th...

AI Technical Summary

Problems solved by technology

The problem with this strategy lies in two points: 1. How to reasonably use external information to guide the work of topic models; 2. Existing topic models usually only consider semantic information as external information, while ignoring some other important information

Images