A text keyword weight calculation method integrating a word position factor and a word frequency factor

Abstract

The invention relates to a text keyword weight calculation method integrating a word position factor and a word frequency factor, and the method comprises the following specific steps of (1) opening asingle text, and recombining paragraphs of the text to form a new text; (2) preprocessing the new text, including word segmentation and stop word removal, and constructing a candidate keyword matrixby taking the rest words as candidate keywords; (3) calculating the weight of each candidate keyword by using the position factors and the word frequency factors of the harmonious series comprehensivewords; and (4) outputting the weight corresponding to each candidate keyword. According to the method, the text structure information is fully utilized, that is, word position factors and word frequency factors in the text are fused, and the weight of a keyword can be calculated only for a single text on the premise of not depending on a field text set. Compared with TFIDF and TEXTRANK, the method has the advantages that the operation is simple and easy, the effect is good, and the functions of the TFIDF and the TEXTRANK can be realized.

Description

technical field [0001] The present invention relates to a kind of text key word weight calculation method of comprehensive word position factor and word frequency factor, specifically relate to adopting harmonic progression comprehensive word position factor and word frequency factor to calculate the weight of word, improve title and first and last two paragraphs of words Weight, and make each word as the word frequency increases, the weight of the position where the word appears decreases. Background technique [0002] The most widely used keyword extraction algorithm is vector space model. The vector space model represents the text as a weight vector, each item in the vector is composed of a word, and the weight of each word is determined by the TFIDF method. Among them, the TFIDF method uses the word weight formula to calculate the importance of a word to a single text in the corpus. The word weight of the TFIDF method is the product of the term frequency TF (Term Frequ...

AI Technical Summary

Problems solved by technology

[0003] (1) The vector space model regards the text as a collection of words, and regards the relationship between words as independent, thus losing the information reflected in the text paragraph structure

[0004] (2) The TFIDF method does not consider the influence of their location factors on their weight when calculating word frequency, but considering the number of occurrences or co-occurrence times alone is not enough to express its actual weight

[0005] (3) When the TFIDF method calculates the inverse document frequency of a word, it needs to rely on the domain text collection and cannot target a single text, and the quality and scale of the domain collection have a huge impact on word weight calculation and keyword extraction

Images