Term extraction method and system oriented to academic paper

Abstract

The invention discloses a term extraction method and system oriented to academic paper. The term extraction method comprises the following steps: performing preprocessing, including text block annotation, text block screening, word segmentation and part-of-speech tagging and noise word removal, on an academic paper corpus; performing candidate term extraction on a title, an abstract and keyword text blocks to form a candidate term set; screening and filtering word type terms and compound word type terms in the candidate term set respectively to obtain a new candidate term set; and determining weights of positions according to position information of candidate terms by an analytic hierarchy process, performing overall rating, ranking the candidate terms according to ratings, and taking TopN candidate terms or candidate terms of which the ratings are greater than a threshold value as extracted terms. Through adoption of the term extraction method and system, the term distribution characteristic of the academic paper and class information of the academic paper are considered fully, and the accuracy and recall rate of term extraction of the academic paper are increased.

Description

technical field [0001] The invention belongs to the technical field of computer natural language processing or pattern recognition, and more specifically relates to a term extraction method for academic papers. Background technique [0002] The existing linguistic term extraction method based on Chinese word formation rules extracts and filters candidate terms for the entire free text, and summarizes the part-of-speech matching template of the term by analyzing the collocation and occurrence rules of different words, and then uses the template to extract Extract candidate terms; among them, the characteristics of terms between different types of text blocks are ignored, the position information of terms cannot be fully utilized, and the quality of term extraction is reduced; on the other hand, in the case of some specific corpora, such as academic papers Contains different text blocks, such as titles, abstracts, keywords, etc., and the distribution of terms in different text...

AI Technical Summary

Problems solved by technology

[0002] The existing linguistic term extraction method based on Chinese word formation rules extracts and filters candidate terms for the entire free text, and summarizes the part-of-speech matching template of the term by analyzing the collocation and occurrence rules of different words, and then uses the template to extract Extract candidate terms; among them, the characteristics of terms between different types of text blocks are ignored, the position information of terms cannot be fully utilized, and the quality of term extraction is reduced; on the other hand, in the case of some specific corpora, such as academic papers Contains different text blocks, such as titles, abstracts, keywords, etc., and the distribution of terms in different text blocks is different. Using the same term extraction method for the entire academic paper makes the accuracy of term extraction not high

[0003] After the candidate term base is extracted, it needs to be screened to obtain the correct term; there

Images