A Short Text Feature Extraction Method

Abstract

The invention discloses a short text feature extraction method, which is based on a knowledge base and a syntax analysis method to perform feature extraction on short texts, including a model training process and a feature extraction process; training is performed on training set data; The weight group W corresponding to the highest accuracy rate and the training model M corresponding to the highest accuracy rate; after the feature extraction process processes the test set data, each category is assigned a weight group W; the ESA algorithm is used to map the short text to the concept space, Get the explanation vector of the short text; get the topic vector through LDA, as the final feature vector of the short text, as the feature of the short text. The method provided by the invention can solve the problems of sparse short text features and unclear topics of short texts; reduce the processing difficulty of short text feature extraction, improve the results of short text feature extraction, and improve the accuracy of text classification.

Description

technical field [0001] The invention relates to text feature extraction and text classification methods, in particular to a short text feature extraction method. Background technique [0002] With the development of applications such as Weibo, social networking sites, and hotlines, more and more information is presented in the form of short texts, and the growth is explosive. Text mining technology can help people quickly and effectively obtain key information from massive data, and text feature extraction is a key step in text mining. [0003] Most of the existing text feature extraction methods use the method based on the Bag of Words (bag of words) model. This method usually achieves better results when used in long texts, but often does not work well when used in short texts. The main reason is that compared with long texts, short texts have the characteristics of sparse features and unclear topics. First of all, due to the limitation of the length of the short text, t...

AI Technical Summary

Problems solved by technology

[0006] A feature extraction method for short texts. This method extracts features from short texts based on the knowledge base and syntactic analysis methods. By calculating the weight of each topic, the topic vector is used as the final feature vector of the short text to solve the problem of short text feature sparsity and Issues with short texts with unclear topics; including model training process and feature extraction process

Images