Keyword extraction method, apparatus and device, and storage medium

Abstract

The invention relates to the technical field of text processing, in particular to a keyword extraction method, apparatus and device and a storage medium. The method includes: obtaining corpus text tobe extracted; inputting the to-be-extracted corpus text into a text labeling model for character type labeling processing to obtain a label corresponding to each character in the to-be-extracted corpus text, wherein the text labeling model is determined by performing supervised training based on a preset neural network model by using a training corpus text with a sample label, and the preset neural network model comprises a semantic representation model, a full connection layer connected with the semantic representation model, a conditional random field connected with the full connection layerand an output layer connected with the conditional random field; obtaining a character corresponding to a preset label in the corpus text to be extracted; and determining a keyword of the corpus textto be extracted according to the character corresponding to the preset label. According to the method, the accuracy and recall rate of keyword extraction can be improved.

Description

technical field [0001] The present invention relates to the technical field of text processing, in particular to a keyword extraction method, device, equipment and storage medium. Background technique [0002] With the development of the Internet, the amount of online text information has exploded, and it is increasingly difficult to manually obtain the required text information. Therefore, how to quickly and effectively summarize the key information of texts in a certain field or topic has become an important issue. [0003] In order to effectively deal with massive text data, researchers have done a lot of research in the directions of text classification, text clustering, automatic summarization and information retrieval, and these researches all involve the problem of how to obtain the keywords in the text. Keywords are the refinement of text topic information, which highly summarizes the main content of the text and can help users quickly understand the gist of the tex...

AI Technical Summary

Problems solved by technology

However, using this method to extract keywords requires word segmentation for the text to be extracted, so the effect of extracted keywords is very dependent on the accuracy of word segmentation. When the accuracy of word segmentation is poor, the accuracy of keyword extraction is low; in addition, this The method does not consider the semantic features of the candidate words and may perform poorly in keyword extraction in proprietary domains

Images