Chinese text keyword extraction method based on document theme structures and semantics

Abstract

The invention discloses a Chinese text keyword extraction method based on document theme structures and semantics, and relates to keyword extraction. The method includes the steps: text preprocessing;Chinese segmentation and part-of-speech tagging; stop word filtering and part-of-speech filtering; keyword extraction. The basic conception of text keyword extraction, Chinese segmentation and English segmentation differences and a common Chinese text keyword extraction method are introduced. A method based on the document theme structures and a method based on semantics are researched, and the principle and an existing implementation scheme are analyzed. In order to overcome difficulty in new word identification in Chinese segmentation, Chinese segmentation effects are continuously improvedby the aid of a dynamically updated segmentation dictionary. The method based on the document theme structures is improved, and global keywords are extracted. Semantic similarities of Chinese words are taken into account, and an algorithm is further improved. The improved algorithm is verified in a self-built data set, good results are acquired by verification experiments and comparison experiments, and keyword extraction effects can be improved by the improved algorithm.

Description

technical field [0001] The invention relates to keyword extraction, in particular to a Chinese text keyword extraction method based on document topic structure and semantics. Background technique [0002] Entering the 21st century, with the continuous advancement of science and technology and the rapid development of the Internet, various information resources have doubled and rapidly increased. People are eager to quickly and accurately find information that is really useful to them from huge information sources. Keywords can highly summarize the content of the document and reflect the theme of the document, providing powerful help for people to find resources. [0003] In a document, a keyword is an in-depth refinement of the content of the document, which is generally represented by several words or phrases. Through the keywords of the document, you can gain insight into the main content described in the document and quickly determine whether it is a required resource. ...

AI Technical Summary

Problems solved by technology

However, due to the different knowledge reserves, understanding of keywords, and ability to summarize and generalize, the annotators have strong subjectivity, and the extracted keywords are not the same.

What's more, using manpower to mark keywords in the text will take more energy to read and understand the text content, which obviously cannot satisfy the current situation that the number of information resources is constantly doubling

Images