Term extraction method and system oriented to academic paper

Abstract

The invention discloses a term extraction method and system oriented to academic paper. The term extraction method comprises the following steps: performing preprocessing, including text block annotation, text block screening, word segmentation and part-of-speech tagging and noise word removal, on an academic paper corpus; performing candidate term extraction on a title, an abstract and keyword text blocks to form a candidate term set; screening and filtering word type terms and compound word type terms in the candidate term set respectively to obtain a new candidate term set; and determining weights of positions according to position information of candidate terms by an analytic hierarchy process, performing overall rating, ranking the candidate terms according to ratings, and taking TopN candidate terms or candidate terms of which the ratings are greater than a threshold value as extracted terms. Through adoption of the term extraction method and system, the term distribution characteristic of the academic paper and class information of the academic paper are considered fully, and the accuracy and recall rate of term extraction of the academic paper are increased.

Description

technical field [0001] The invention belongs to the technical field of computer natural language processing or pattern recognition, and more specifically relates to a term extraction method for academic papers. Background technique [0002] The existing linguistic term extraction method based on Chinese word formation rules extracts and filters candidate terms for the entire free text, and summarizes the part-of-speech matching template of the term by analyzing the collocation and occurrence rules of different words, and then uses the template to extract Extract candidate terms; among them, the characteristics of terms between different types of text blocks are ignored, the position information of terms cannot be fully utilized, and the quality of term extraction is reduced; on the other hand, in the case of some specific corpora, such as academic papers Contains different text blocks, such as titles, abstracts, keywords, etc., and the distribution of terms in different text...

AI Technical Summary

Problems solved by technology

[0002] The existing linguistic term extraction method based on Chinese word formation rules extracts and filters candidate terms for the entire free text, and summarizes the part-of-speech matching template of the term by analyzing the collocation and occurrence rules of different words, and then uses the template to extract Extract candidate terms; among them, the characteristics of terms between different types of text blocks are ignored, the position information of terms cannot be fully utilized, and the quality of term extraction is reduced; on the other hand, in the case of some specific corpora, such as academic papers Contains different text blocks, such as titles, abstracts, keywords, etc., and the distribution of terms in different text blocks is different. Using the same term extraction method for the entire academic paper makes the accuracy of term extraction not high

[0003] After the candidate term base is extracted, it needs to be screened to obtain the correct term; there are many filtering methods, mainly through the unit of term and field correlation; for example: TF-IDF (term frequency-inverse document frequency, TermFrequency- Inverse Document Frequency) method is to use the frequency of candidate terms appearing in this article and the frequency in the entire corpus to judge the domain relevance of candidate terms; SCP (Symmetrical Conditional Probability, symmetric conditional probability) is used to judge the collocation rationality of compound terms ; C-VALUE (C value, Content-Value) is used to judge the field relevance of composite terms; these methods have a good effect on the screening of candidate terms, but under some specific corpora, such as academic papers, their The category attribute is very clear, but these methods do not take this attribute into account, do not use category information when screening terms for academic papers, and do not fully consider the domain relevance of candidate terms, resulting in low term extraction accuracy

Images