Document keyword extraction method and device based on BERT model

Abstract

A document keyword extraction method based on a BERT model comprises the following steps that each document in a document set is coded through the BERT model, and the attention weight of document semantics generated by the BERT model to each sub-word is extracted; restoring the sub-words into words, and aggregating the attention weights of the sub-words into the attention weight of the words; the attention weights of the same word at different positions in the document are aggregated into the attention weight, irrelevant to the position, of the word, and the attention weight is recorded as p (wordweight '2jeemaa2' doc); calculating the attention weight of each word on the document set, and recording the attention weight of each word on the document set as p (wordweight '2jeemaa2' corpus); and combining the p (wordweight '2jeemaa2' doc) and the p (wordweight '2jeemaa2' corpus), and selecting N words with the highest final attention weight as the keyword of the document. According to the method, the BERT model is used for extracting the document semantic representation to calculate the word attention weight distribution, the keyword extraction is finally realized, the word frequency information is considered, the problem that the semantics is ignored by the traditional unsupervised algorithm is effectively solved, and the keyword extraction accuracy and recall rate are improved.

Description

technical field [0001] The present invention relates to the technical field of information processing, in particular to a document keyword extraction method and device based on a BERT model. Background technique [0002] With the vigorous development of Internet technology and the rapid growth of network information, document keyword extraction technology can be used to index document content characteristics, construct information retrieval, quickly extract document center content, improve reader review efficiency, and deal with information overload. [0003] Keywords are words that can express the core content of a document, including words, terms, and phrases, which contain a certain amount of information and can promote the understanding of text content. From a technical point of view, document keyword extraction is the basic work of text mining research such as text retrieval, document comparison, abstract generation, document classification and clustering; Whether the ...

AI Technical Summary

Problems solved by technology

[0011]Aiming at the deficiencies of the prior art, the purpose of the present invention is to provide a document keyword extraction method and device based on the BERT model, through semantic information and document collection information extraction The keywords that can accurately and comprehensively express the content of the document center have good transferability between different document collections, without the need for manual word segmentation and data labeling, reducing human workload, and solving the problem that existing keyword extraction methods are easily affected by ambiguous words 、 Unable to solve the shortcomings of synonyms and different words and the extraction results are not accurate enough

Images