Improved algorithm for extracting text feature by small-world model

Abstract

The invention disclosed an improved algorithm for extracting a text feature by a small-world model. According to a Chinese word segmentation preprocessing process, vocabulary position weight and part-of-speech weight are determined, and the semantic relativity function R (c1,c2) is determined by combining a relevancy algorithm of two vocabularies on the basis of the HowNet with a vocabulary to text important degree method. Since a polymerism can construct a vocabulary semantic network model map, two parameters, including a density parameter and an edge weight parameter, are set, proper threshold values are independently set, and the threshold values are taken as text feature vocabularies if a condition is met. The algorithm exhibits better accuracy and application value, the deficiency that an information gaining method is only suitable for extracting text features of one category, the contribution degrees of different vocabularies for text thought can be accurately calculated, and meanwhile, a good theoretical basis is provided for subsequent text clustering.

Description

technical field [0001] The invention relates to the technical field of semantic network, in particular to an improved small-world model algorithm for extracting text features. Background technique [0002] In the era of information explosion, the amount of information increases exponentially. In the face of massive texts, how to quickly grasp the theme of an article and grasp the author's thoughts has become a key issue in saving readers' time and improving their reading speed. Text features as the embodiment of the theme of the article and the author's thoughts can effectively solve this problem. However, the vast majority of articles on the Internet do not provide keywords. If manual indexing is used to mark these texts, it will not only be time-consuming, laborious, inefficient, but also relatively subjective. [0003] Currently commonly used text feature extraction methods include word frequency-inverse document frequency method—TF-IDF, information gain method, mutual ...

AI Technical Summary

Problems solved by technology

However, the commonly used text feature extraction methods do not consider the semantic status of the feature vocabulary and its contribution to the expression of the text, and ignore the semantic information and structural information of the document, resulting in the lack of semantic and structural information of keywords.

Images