Method for extracting key words of single text

Abstract

The invention discloses a method for extracting key words of single text, especially comprising the following steps: (1) opening the single text in the field collection; (2) pre-processing the content of the text; (3) extracting the meaningful notional word; (4) making statistic of the word frequency of the notional word; (5) opening all the texts in the field collection; (6) making statistic of the message frequency of the notional word in the field collection; (7) making statistic of the returning pages of search engine retrieving the notional word; (8) using the developed TFIDF word right formula to calculate the weights of all the notional words in the single text to extract a certain percentage of the key words. Besides, the method can compensate the insufficient of the TFIDF algorithm and can prevent the impacts of the irrelevant field connection to extract the key words, thereby improving the extracting precision of the key words and maintaining the field features of the extracting result for the key words.

Description

technical field [0001] The invention relates to a method for extracting keywords of a single text, in particular to an improved method for extracting keywords of a single text in a field corpus using the TFIDF method. Background technique [0002] Single text keywords are the basic elements of text representation in text knowledge flow generation, semantic chain network construction, text context complexity and information volume. The extraction accuracy of keywords in a single text directly affects the quality and effect of text information processing such as text classification, clustering, word association analysis, automatic text summarization, text filtering, information retrieval, topic detection, and web page annotation. At present, the research on single text keyword extraction technology mainly includes: TFIDF method, naive Bayesian classification method, mutual information method, maximum entropy model method, maximum likelihood and prefix tree method, etc. [000...

AI Technical Summary

Problems solved by technology

[0007] (1) The domains and total number of texts involved in the irrelevant domain corpus in the TFIDF method will reduce the quality of keyword extraction for a single text

[0008] (2) Since the inverse document frequency of a word is inversely proportional to the frequency of words appearing in the corpus, the TFIDF word weight will tend to be low-frequency words, resulting in a decrease in the extraction accuracy of keywords in a single text

[0009] (3) When the domain anthology belongs to a single domain, some high-frequency articles representing the domain as a whole will also be filtered out, and the final keyword extraction results can only retain text characteristics and not domain characteristics

Images