Short Text Topic Model Mining Method Based on Feature Expansion Based on Word Network

A topic model and short text technology, applied in the field of feature extension of short text, can solve the problems of extension result impact, short text data sparseness, model quality dependent extension strategy, etc., and achieve the effect of solving the problem of feature sparsity

Active Publication Date: 2019-08-27
NANJING UNIV
View PDF9 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Another way to expand short texts is to use search engines or wiki databases to find words or sentences related to the current text in search engines or wiki databases to expand the original text. This method can to a certain extent Solve the data sparsity problem of short text, but on the one hand, external databases will have a great impact on the expansion results, on the other hand, the final model quality depends on the quality of the expansion strategy

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Short Text Topic Model Mining Method Based on Feature Expansion Based on Word Network
  • Short Text Topic Model Mining Method Based on Feature Expansion Based on Word Network
  • Short Text Topic Model Mining Method Based on Feature Expansion Based on Word Network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0053] In order to better understand the technical content of the present invention, specific embodiments are given together with the attached drawings for description as follows.

[0054] As shown in Figure 1, the present invention will set up the weighted word network diagram according to the training corpus before implementation,

[0055] Step 0 is to establish the initial state of the network graph of weighted words.

[0056] Step 1 is to use an open source word segmentation tool to perform Chinese word segmentation on the documents in the corpus, and convert each document into a collection of words.

[0057]Step 2 is the operation of removing stop words for word segmentation. Since stop words have no meaning for topic modeling, after the word segmentation is completed, stop words in the word set are removed against the stop word vocabulary.

[0058] Step 3 is to establish nodes in the weighted word network, and each word after step 2 to stop word processing is used to es...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

A short text topic model mining method based on a word network to extend characteristics comprises a weighted word network construction step, a short text characteristics extending step, and a topic mining step. The weighted word network construction step comprises preprocessing a text, performing Chinese words segmentation on the text in a short text corpus, and deleting stop words; establishing a weighted word network from a document after the Chinese words segmentation is performed, wherein nodes in the weighted word network are words, each edge between the nodes is cooccurrence relation of two words in the same document, and the weight of the edge is the cooccurrence time of the two words in the whole corpus; and ending. The short text characteristics extending step comprises using the word nodes included by each short text after the Chinese words segmentation is performed as a community of the established weighted word network. According to the short text characteristics sparsity solution method based on word network community module degree, the problem that the effect of applying an LDA topic model to the short text is poor is solved. Accuracy of a short text topic model is increased.

Description

technical field [0001] The invention relates to the field of short text text topic models and complex network analysis, and is a method for expanding short text features by using weighted word networks, so as to solve the feature sparsity problem existing in the application of LDA topic models to short texts. Background technique [0002] The features of short texts are very sparse, resulting in the poor performance of generative models such as LDA for topic modeling based on word co-occurrence relations on short texts. In the current environment, short texts have incomparable advantages over long texts. Compared with long texts, short texts express concise semantics and transmit information quickly. People are more and more inclined to use short texts to transmit information. , short texts are becoming one of the most important information carriers in today's society, such as online advertisements, short messages, and popular social media such as Weibo, Twitter, etc. These ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & AuthorityPatents(China)
IPC IPC(8): G06F16/335G06F16/35
Inventor张雷戴恒宇蔡洋王陆霞陆恒杨徐鸣王崇骏
OwnerNANJING UNIV