Automatic keyword extraction method and device, equipment and storage medium

Abstract

The invention relates to the field of artificial intelligence, and particularly discloses an automatic keyword extraction method and device, equipment and a storage medium. The method comprises: obtaining a sample text used for training, extracting keywords from the sample text, sorting the keywords according to the keyword extraction sequence, and sequentially performing label processing on all the keywords to obtain a keyword set containing labels; performing association processing on the sample text and the keyword set to form a training data set; training a pre-constructed language prediction model based on the training data set and the cross entropy loss function, and iteratively optimizing the language prediction model by using a gradient descent method to obtain a target language prediction model; and obtaining a to-be-processed text, inputting the to-be-processed text into the target language prediction model, outputting a target keyword set containing the tag, and extracting a target keyword of the to-be-processed text from the target keyword set according to the tag. In this way, the keyword obtaining efficiency and accuracy can be improved.

Description

technical field [0001] The invention relates to the technical field of natural language of artificial intelligence, in particular to a keyword automatic extraction method, device, equipment and storage medium. Background technique [0002] As an important technology in text mining, keyword extraction is the basic and necessary work of information retrieval, text classification and recommendation system. Text keywords condense the gist and gist of the entire text, and are a high-level summary of a text. By extracting words or phrases with a summary function from a text, it can help users quickly retrieve the information they need and quickly judge the need for the text, and effectively solve the problem of information overload. [0003] There are many methods for extracting keywords at present, and the main methods used are the algorithm based on statistical TF-IDF, the TextRank algorithm based on the graph model, and the LDA algorithm based on the topic model. However, mos...

AI Technical Summary

Problems solved by technology

[0005] The present invention provides a keyword automatic extraction method, device, equipment and storage medium, which can improve the efficiency and accuracy of keyword acquisition, and solve the current problem of low efficiency and inaccuracy caused by extracting keywords from texts by extracting abstracts

Images