Mixed text feature word extraction method

Abstract

The invention discloses a mixed text feature word extraction method comprising the steps of performing word segmentation on a text by using a word segmentation technology; removing stop words by matching a stop word list; acquiring a series of word position and property weight values according to research statistics; extracting the first word c(w1), and the first m-1 words of which the RE (ci,c(w1) values are bigger according to information quantity of the words in the text based on the abovementioned two factors; building a word network model based on a word semantic similarity method; and at last finding a text feature word set meeting an important degree condition according to a neighborhood method. Compared with traditional text feature word extraction method, the method provided by the invention has higher accuracy, harsher conditions, and higher application value; according to the method, the feature of the text can be extracted under the condition that only a text condition is provided without a text set or without advanced classification, contribution degrees of the different words to a text thought are computed, and a good theoretical basis is provided for subsequent text similarity and text clustering.

Description

technical field [0001] The invention relates to the technical field of semantic network, in particular to a method for extracting mixed text feature vocabulary. Background technique [0002] Text features refer to the collection of words that can best represent the subject of the text. Text features can not only summarize the main content and subject of the text well, but also reduce the complexity of text processing. Currently commonly used text feature extraction methods include word frequency-inverse document frequency method, information gain and other methods. The simple structure of the word frequency-inverse document frequency method cannot effectively reflect the importance of words or phrases and the distribution of feature values, so the accuracy of TF-IDF is not very high. The information gain method is only suitable for extracting text features of one category, but cannot be used for extracting text features of multiple categories. The above two text feature ex...

AI Technical Summary

Problems solved by technology

[0003] For the commonly used text feature extraction methods require a large number of training sets as the premise of extraction, if only a text is given, the text feature will not be able to be extracted and the commonly used text feature extraction methods have low precision, the present invention provides a A Hybrid Text Feature Vocabulary Extraction Method

Images