A keyword extraction method, device, equipment and storage medium

Abstract

The present application provides a keyword extraction method, apparatus, device, and storage medium. The method includes: obtaining text data for which keywords are to be extracted; using a pre-trained keyword extraction model to extract key sentences from the text data, and using the keyword extraction model to extract keywords from the key sentences. The keyword extraction model in the present application uses training text and key sentence sets, non-key sentence sets, keyword sets, and non-keyword sets obtained from the training text as training data, with the goal of making the probability of predicting a key sentence in the key sentence set as a key sentence greater than the probability of predicting a non-key sentence in the non-key sentence set as a key sentence, and making the probability of predicting a keyword in the keyword set as a keyword greater than the probability of predicting a non-keyword in the non-keyword set as a keyword. This training strategy can train a model with better performance. Using the model with better performance to extract keywords from text data can obtain better extraction results.

Description

Technical Field

[0001] This application relates to the technical field of natural language processing, and in particular, to a keyword extraction method, apparatus, device, and storage medium. Background Art

[0002] Natural language processing is an important direction in the fields of computer science and artificial intelligence, and keyword extraction is an important basic technology in natural language processing. The keyword extraction method can extract important keywords from the text, enabling readers to quickly obtain the core content of the text, and thus is widely used in fields such as information retrieval, text classification, and text clustering.

[0003] The current keyword extraction methods are mainly model-based keyword extraction methods, that is, using manually labeled training texts (i.e., training texts with manually labeled keywords) to train a keyword extraction model, and then using the trained keyword extraction model to extract keywords from the text data to be extracted with keywords.

[0004] However, due to the subjectivity of manual annotation itself, it is very difficult to obtain a sufficient amount of high-quality labeled data. It is difficult to train a keyword extraction model with better performance using lower-quality labeled data. When using a keyword extraction model with poor performance for keyword extraction, it is difficult to obtain a good extraction effect. Summary of the Invention

[0005] In view of this, this application provides a keyword extraction method, apparatus, device, and storage medium, which are used to solve the problem that the existing keyword extraction methods are difficult to obtain a sufficient amount of high-quality labeled data, resulting in difficulty in training a keyword extraction model with better performance, and further resulting in difficulty in obtaining a good extraction effect when using the keyword extraction model to extract keywords from the text data to be extracted with keywords. The technical solutions are as follows:

[0006] A keyword extraction method includes:

[0007] Obtain text data to be extracted with keywords;

[0008] Use a pre-trained keyword extraction model to extract key sentences from the text data, and use the keyword extraction model to extract keywords from the extracted key sentences;

[0009] Among them, the keyword extraction model uses training texts, as well as key sentence sets, non-key sentence sets, keyword sets, and non-keyword sets obtained from the training texts, with the goal of training to make the probability of predicting the key sentences in the key sentence set as key sentences greater than the probability of predicting the non-key sentences in the non-key sentence set as key sentences, and the probability of predicting the keywords in the keyword set as keywords greater than the probability of predicting the non-keywords in the non-keyword set as keywords.

[0010] Optionally, extracting keywords from the extracted key sentences using the keyword extraction model includes:

[0011] Using the keyword extraction model to obtain a number of candidate keywords from the extracted key sentences, where each candidate keyword is a sentence fragment in the extracted key sentences with a length within a preset length range;

[0012] Using the keyword extraction model to predict the probabilities of the number of candidate keywords being keywords respectively, and determining keywords from the number of candidate keywords according to the probabilities of the number of candidate keywords being keywords respectively.

[0013] Optionally, predicting the probabilities of the number of candidate keywords being keywords respectively using the keyword extraction model includes:

[0014] For each candidate keyword among the number of candidate keywords, using the keyword extraction model and combining the information of the already determined keywords, predicting the probability of this candidate keyword being a keyword.

[0015] Optionally, predicting the probabilities of the number of candidate keywords being keywords respectively using the keyword extraction model includes:

[0016] Using the keyword extraction model, based on the importance of the extracted key sentences, sorting the number of candidate keywords to obtain a candidate keyword sequence;

[0017] Using the keyword extraction model to obtain the context vectors corresponding to each candidate keyword in the candidate keyword sequence, where the context vector contains the context information of the corresponding candidate keyword in the candidate keyword sequence;

[0018] For each candidate keyword in the candidate keyword sequence, using the keyword extraction model, based on the context vector corresponding to this candidate keyword, predicting the probability of this candidate keyword being a keyword.

[0019] Optionally, using the keyword extraction model to obtain the context vectors corresponding to each candidate keyword in the candidate keyword sequence includes:

[0020] Using the keyword extraction model, obtain the representation vectors of each candidate keyword in the candidate keyword sequence, where the representation vector of a candidate keyword includes the context information of each sub-word included in the candidate keyword in the training text;

[0021] Using the keyword extraction model and the representation vectors of each candidate keyword in the candidate keyword sequence, obtain the context vectors corresponding to each candidate keyword in the candidate keyword sequence.

[0022] Optionally, using the keyword extraction model, predicting the probability that the candidate keyword is a keyword based on the context vector corresponding to the candidate keyword includes:

[0023] Using the keyword extraction model, predicting the probability that the candidate keyword is a keyword based on the context vector corresponding to the candidate keyword and the historical keyword information vector, where the historical keyword information vector is the fusion result of the context vectors corresponding to each determined keyword;

[0024] The keyword extraction method further includes:

[0025] For each candidate keyword in the candidate keyword sequence, if it is determined that the candidate keyword is a keyword according to the probability that the candidate keyword is a keyword, then fuse the context vector corresponding to the candidate keyword with the historical keyword information vector, and use the fused vector as the new historical keyword information vector.

[0026] Optionally, the training text has a corresponding reference keyword set;

[0027] Obtaining the key sentence set and non-key sentence set from the training text includes:

[0028] Perform sentence splitting on the training text to obtain each sentence of the training text;

[0029] For each sentence of the training text, determine the score of the sentence according to the occurrence situation of the common words between the reference keyword set and the sentence in the sentence and in the corpus;

[0030] Determine the key sentences according to the scores of each sentence of the training text, and use the sentence set composed of the determined key sentences as the key sentence set obtained from the training text;

[0031] Use the sentence set composed of several sentences that have no intersection with the reference keyword set as the non-key sentence set obtained from the training text.

[0032] Optionally, obtaining the keyword set and non-keyword set from the training text includes:

[0033] Obtain a number of candidate keywords from the key sentences in the key sentence set;

[0034] Determine the candidate keywords that exactly match the keywords in the reference keyword set among the number of candidate keywords, and use the word set composed of the determined keywords as the keyword set obtained from the training text;

[0035] Determine the candidate keywords that do not exactly match the keywords in the reference keyword set among the number of candidate keywords, and use the word set composed of the determined non-keywords as the non-keyword set obtained from the training text.

[0036] Optionally, the determining the candidate keywords that do not exactly match the keywords in the reference keyword set as non-keywords includes:

[0037] Determine the candidate keywords that do not exactly match the keywords in the reference keyword set and have an intersection with the keywords in the reference keyword set as non-keywords.

[0038] Optionally, the training process of the keyword extraction model includes:

[0039] Use the keyword extraction model to predict the probability that each sentence in the training text is a key sentence, so as to obtain the probability that each sentence in the key sentence set is a key sentence and the probability that each sentence in the non-key sentence set is a key sentence;

[0040] Determine the key sentence prediction loss according to the probability that each sentence in the key sentence set is a key sentence and the probability that each sentence in the non-key sentence set is a key sentence;

[0041] Predict the probability that each of the number of candidate keywords obtained from the key sentences included in the key sentence set is a keyword, so as to obtain the probability that each word in the keyword set is a keyword and the probability that each word in the non-keyword set is a keyword;

[0042] Determine the keyword prediction loss according to the probability that each word in the keyword set is a keyword and the probability that each word in the non-keyword set is a keyword;

[0043] Update the parameters of the keyword extraction model according to the key sentence prediction loss and the keyword prediction loss.

[0044] A keyword extraction device includes: a text acquisition module and a text processing module;

[0045] The text acquisition module is used to acquire the text data for which keywords are to be extracted;

[0046] The text processing module is used to extract key sentences from the text data by using a pre-trained keyword extraction model, and extract keywords from the extracted key sentences by using the keyword extraction model;

[0047] Wherein, the keyword extraction model uses training texts and key sentence sets, non-key sentence sets, keyword sets, and non-keyword sets obtained from the training texts, so that the probability of predicting the key sentences in the key sentence set as key sentences is greater than the probability of predicting the non-key sentences in the non-key sentence set as key sentences, and the probability of predicting the keywords in the keyword set as keywords is greater than the probability of predicting the non-keywords in the non-keyword set as keywords for training.

[0048] Optionally, when the text processing module extracts keywords from the extracted key sentences by using the keyword extraction model, it is specifically used for:

[0049] Obtain a number of candidate keywords from the extracted key sentences by using the keyword extraction model, wherein each candidate keyword is a sentence fragment with a length within a preset length range in the extracted key sentences;

[0050] Predict the probabilities that the number of candidate keywords are keywords respectively by using the keyword extraction model, and determine keywords from the number of candidate keywords according to the probabilities that the number of candidate keywords are keywords respectively.

[0051] Optionally, when the text processing module predicts the probabilities that the number of candidate keywords are keywords respectively by using the keyword extraction model, it is specifically used for:

[0052] For each candidate keyword in the number of candidate keywords, use the keyword extraction model and combine the information of the already determined keywords to predict the probability that the candidate keyword is a keyword.

[0053] A keyword extraction device includes: a memory and a processor;

[0054] The memory is used to store programs;

[0055] The processor is used to execute the program to implement each step of the keyword extraction method described in any one of the above.

[0056] A computer-readable storage medium stores a computer program thereon, and when the computer program is executed by a processor, each step of the keyword extraction method described in any one of the above is implemented.

[0057] The keyword extraction method, device, equipment, and storage medium provided by this application, after obtaining the text data for which keywords are to be extracted, first uses a pre-trained keyword extraction model to extract key sentences from the text data, and then uses the keyword extraction model to extract keywords from the extracted key sentences. Different from the prior art, the keyword extraction model in this application is not trained with training texts annotated with keywords, aiming to make the predicted keywords for the training texts tend to be the same as the annotated keywords in the training texts. Instead, it uses the training texts, as well as the key sentence set, non-key sentence set, keyword set, and non-keyword set obtained from the training texts as training data, aiming to make the probability of predicting the key sentences in the key sentence set as key sentences greater than the probability of predicting the non-key sentences in the non-key sentence set as key sentences, and make the probability of predicting the keywords in the keyword set as keywords greater than the probability of predicting the non-keywords in the non-keyword set as keywords. This training strategy enables the training of a keyword extraction model with better performance without the need to obtain high-quality annotated data. Using the keyword extraction model with better performance to extract keywords from the text data for which keywords are to be extracted can achieve better extraction results. In addition, this application does not directly extract keywords from the text data, but first extracts key sentences from the text data and then further extracts keywords from the extracted key sentences. This extraction strategy has higher extraction efficiency compared to directly extracting keywords from the text data. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on the provided accompanying drawings.

[0059] Figure 1 It is a flowchart of the keyword extraction method provided by the embodiment of this application;

[0060] Figure 2 It is a flowchart of obtaining the key sentence set and non-key sentence set from the training text provided by the embodiment of this application;

[0061] Figure 3 It is a flowchart of obtaining the keyword set and non-keyword set from the training text provided by the embodiment of this application;

[0062] Figure 4 It is a flowchart of training the keyword extraction model using the training text, as well as the key sentence set, non-key sentence set, keyword set, and non-keyword set obtained from the training text provided by the embodiment of this application;

[0063] Figure 5It is a structural example of the keyword extraction model provided by the embodiment of the present application;

[0064] Figure 6 It is a schematic structural diagram of the keyword extraction device provided by the embodiment of the present application;

[0065] Figure 7 It is a schematic structural diagram of the keyword extraction device provided by the embodiment of the present application. Detailed implementation manners

[0066] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0067] The applicant found during the implementation of the present application that: most of the current keyword extraction methods are strong supervision methods, that is, a keyword extraction model is trained using training texts with labeled keywords, and then the trained keyword extraction model is used to extract keywords from the text data to be extracted. However, the cost of keyword annotation is relatively high, and the subjectivity of manual annotation itself makes it difficult to obtain a sufficient number of high-quality annotated data. Although there are some annotated data on the Internet currently, the quality of these annotated data is relatively poor. That is to say, whether it is to obtain annotated data by manual annotation or directly obtain annotated data from the Internet, the obtained annotated data cannot meet the model training requirements. The applicant also found that there are currently some unsupervised methods, such as keyword extraction methods based on statistical quantization features and keyword extraction methods based on topic models. However, the keyword extraction method based on statistical quantization features will miss keywords with unobvious statistical features, and the accuracy is not good, while the keywords obtained by the keyword extraction method based on the topic model are too broad and cannot fully reflect the text theme. Therefore, the accuracy of the extracted keywords is also insufficient.

[0068] In view of the problems existing in the existing keyword extraction methods, the applicant attempts to propose a keyword extraction method with better effects. For this purpose, the applicant has conducted in-depth research. Through continuous research, a keyword extraction method is finally proposed, which perfectly solves the problems existing in the existing keyword extraction methods. The keyword extraction method proposed in this application can be applied to an electronic device with data processing capabilities. This electronic device can be a terminal used by the user side, such as a smart phone, a PC, a notebook, a PAD, etc. This electronic device can also be a server on the network side (which can be a single server, or multiple servers or a server cluster). The electronic device can extract keywords from the text data from which keywords are to be extracted according to the keyword extraction method provided in this application. Next, the keyword extraction method provided in this application will be introduced through the following embodiments.

[0069] The First Embodiment

[0070] Please refer to Figure 1 , which shows a schematic flowchart of the keyword extraction method provided in the embodiment of this application. The method may include:

[0071] Step S101: Obtain the text data from which keywords are to be extracted.

[0072] The keyword extraction method provided in this application is applicable to any scenario that requires keyword extraction, such as a speech recognition scenario, an information retrieval scenario, a text classification scenario, a text clustering scenario, etc. Correspondingly, the text data from which keywords are to be extracted is the text data that needs to be extracted for keywords in the applicable scenarios of the keyword extraction method provided in this application.

[0073] Step S102: Extract key sentences from the text data by using a pre-trained keyword extraction model.

[0074] The process of extracting key sentences from the text data by using a pre-trained keyword extraction model may include: First, perform sentence splitting on the text data to obtain each sentence of the text data, then use the pre-trained keyword extraction model to predict the probability that each sentence of the text data is a key sentence, and finally determine the key sentence according to the probability that each sentence of the text data is a key sentence.

[0075] Step S103: Extract keywords from the extracted key sentences by using the keyword extraction model.

[0076] The process of extracting keywords from the extracted key sentences using a keyword extraction model may include: First, use the keyword extraction model to obtain candidate keywords from the extracted key sentences to obtain a number of candidate keywords, and then use the keyword extraction model to predict the probability of each candidate keyword being a keyword, and determine the keywords from the number of candidate keywords according to the probability of each candidate keyword being a keyword. Among them, each candidate keyword is a sentence fragment in the extracted key sentence with a length within a preset length range.

[0077] It should be noted that the keyword extraction model in this embodiment uses training texts and key sentence sets, non-key sentence sets, keyword sets, and non-keyword sets obtained from the training texts, so that the probability of predicting the key sentences in the key sentence set as key sentences is greater than the probability of predicting the non-key sentences in the non-key sentence set as key sentences, and the probability of predicting the keywords in the keyword set as keywords is greater than the probability of predicting the non-keywords in the non-keyword set as keywords for target training.

[0078] Among them, the key sentence set includes a number of key sentences obtained from the training text, and the key sentence set is the positive sample set of key sentences for model training. The non-key sentence set includes a number of non-key sentences obtained from the training text, and the non-key sentence set is the negative sample set of key sentences for model training. The keyword set includes a number of keywords obtained from the training text, and the keyword set is the positive sample set of keywords for model training. The non-keyword set includes a number of non-keywords obtained from the training text, and the non-keyword set is the negative sample set of keywords for model training.

[0079] The keyword extraction method provided by the embodiments of the present application, after obtaining the text data of the keywords to be extracted, first uses a pre-trained keyword extraction model to extract key sentences from the text data, and then uses the keyword extraction model to extract keywords from the extracted key sentences. The keyword extraction model in the embodiments of the present application is not trained as in the prior art by using training texts annotated with keywords, with the goal of making the predicted keywords for the training texts tend to be the same as the annotated keywords in the training texts. Instead, it uses the training texts and the key sentence set, non-key sentence set, keyword set, and non-keyword set obtained from the training texts as training data, with the goal of making the probability of predicting the key sentences in the key sentence set as key sentences greater than the probability of predicting the non-key sentences in the non-key sentence set as key sentences, and making the probability of predicting the keywords in the keyword set as keywords greater than the probability of predicting the non-keywords in the non-keyword set as keywords. This training strategy enables the training of a keyword extraction model with better performance without the need to obtain high-quality annotated data. Using the keyword extraction model with better performance to extract keywords from the text data of the keywords to be extracted can obtain better extraction results. In addition, the present application does not directly extract keywords from the text data, but first extracts key sentences from the text data and then further extracts keywords from the extracted key sentences. This extraction strategy has higher extraction efficiency compared to directly extracting keywords from the text data.

[0080] Second Embodiment

[0081] As mentioned in the above embodiments, the extraction of keywords is implemented based on a pre-trained keyword extraction model. Next, the training process of the keyword extraction model will be introduced.

[0082] Since the keyword extraction model is trained using training texts and the key sentence set, non-key sentence set, keyword set, and non-keyword set obtained from the training texts, before introducing the training process of the keyword extraction model, first the process of obtaining the key sentence set, non-key sentence set, keyword set, and non-keyword set from the training texts will be introduced.

[0083] Since the keyword set and non-keyword set are obtained based on the key sentence set, this embodiment first introduces the process of obtaining the key sentence set and non-key sentence set from the training texts.

[0084] Please refer to Figure 2 , which shows a schematic flowchart of obtaining the key sentence set and non-key sentence set from the training texts, and may include:

[0085] Step S201: Perform sentence segmentation on the training text to obtain each sentence of the training text.

[0086] The process of clause segmentation for the training text may include: First, tokenize the training text to obtain the tokenization result of the training text, and then use a window with a specified window length to perform clause segmentation on the tokenization result of the training text.

[0087] Exemplarily, if the specified window length is 30, the starting position of the training text (usually the start symbol here) is used as the start position of the window, the 29th word of the training text is used as the end position of the window, and there are 30 words in the window. The sentence composed of these 30 words is used as the first sentence divided. Then, move the window backward to the next word after the end position of the previous window, that is, use the 30th word of the training text as the start position of the window and the 59th word of the training text as the end position of the window. In this way, a sentence composed of the 30th word to the 59th word of the training text can be obtained, that is, the second sentence divided, and so on.

[0088] Preferably, to ensure the integrity of the sentence, if the end position of the window is not a punctuation mark (such as a comma, a pause mark, a colon, a semicolon, a period, a question mark, an exclamation mark, etc.), then move the end position of the window backward to the first punctuation mark.

[0089] This embodiment does not limit the use of the above-mentioned clause segmentation method to perform clause segmentation on the training text, and other clause segmentation methods can also be used. For example, clause segmentation can be performed according to the punctuation marks in the training text.

[0090] After performing clause segmentation on the training text, a sentence sequence corresponding to the training text can be obtained. The sentence sequence corresponding to the training text can be expressed as para = {(s1, s2,..., s n )|s i =(s i,1 , s i,2 ,..., s i,m )}, where s i is the i-th sentence in the training text, and s i,j is the j-th word included in the i-th sentence.

[0091] Step S202: For each sentence of the training text, determine the score of the sentence according to the occurrence situation of the common words between the reference keyword set corresponding to the training text and the sentence in the sentence and in the corpus.

[0092] It should be noted that news corpora usually include the main content of the news and several keywords related to the main content of the news. The training text in this application can be the main content of the news in the news corpus, and the reference keyword set corresponding to the training text in this application can be a word set composed of the keywords related to the main content of the news in the news corpus.

[0093] In addition, it should be noted that several keywords related to the main content of the news in the news corpus may include keywords that appear in the main content of the news, or may include keywords that do not appear in the main content of the news. In view of this, keywords that do not appear in the main content of the news among several keywords related to the main content of the news can be deleted, and the set of words composed of the remaining keywords is used as the reference keyword set corresponding to the training text, that is, the reference keyword set obtained in this way only includes keywords that appear in the training text.

[0094] Optionally, for each sentence of the training text, the process of determining the score of the sentence according to the co-occurrence of the common words between the reference keyword set corresponding to the training text and the sentence in the sentence and in the corpus can include: calculating the TF-IDF value corresponding to the sentence according to the co-occurrence of the common words between the reference keyword set corresponding to the training text and the sentence in the sentence and in the corpus, and using the TF-IDF value of the sentence as the score of the sentence.

[0095] Among them, when calculating the TF-IDF value corresponding to a sentence, first calculate the TF value corresponding to the sentence according to the co-occurrence of the common words between the reference keyword set corresponding to the training text and the sentence in the sentence, and calculate the IDF value corresponding to the sentence according to the co-occurrence of the common words between the reference keyword set corresponding to the training text and the sentence in the corpus, and then calculate the TF-IDF value corresponding to the sentence according to the TF value corresponding to the sentence and the IDF value corresponding to the sentence. It should be noted that the score of a sentence (such as the TF-IDF value) can represent the importance of the sentence.

[0096] Step S203: Determine the key sentences according to the scores of the sentences of the training text, and the set of sentences composed of the determined key sentences is used as the key sentence set obtained from the training text.

[0097] There are various ways to determine the key sentences according to the scores of the sentences of the training text. In one possible implementation, the preset number (such as 10) of sentences with the highest scores can be determined as the key sentences. In another possible implementation, the sentences with scores greater than the preset score threshold can be determined as the key sentences.

[0098] Step S204: Use the set of sentences composed of several sentences that have no intersection with the reference keyword set corresponding to the training text as the non-key sentence set obtained from the training text.

[0099] It should be noted that a sentence that has no intersection with the reference keyword set refers to a sentence that does not contain the keywords in the reference keyword set.

[0100] Next, the process of obtaining the keyword set and non-keyword set from the training text will be introduced.

[0101] Please refer to Figure 3 , which shows a schematic flowchart of obtaining a keyword set and a non-keyword set from training text, and may include:

[0102] Step S301: Obtain a number of candidate keywords from the key sentences in the key sentence set.

[0103] Among them, each candidate keyword is a sentence fragment in the key sentence with a length within a preset length range.

[0104] Specifically, for each key sentence in the key sentence set, all possible sentence fragments with a length within the preset length range can be obtained from the key sentence.

[0105] Exemplarily, a key sentence is "The method can be applied to the post-processing module in a speech recognition system and an information retrieval system", and the preset length range is [2, 4]. Then the sentence fragments obtained from this sentence may include "The method", "The method can", "The method can be applied", "method can", "method can be applied", "method can be applied to",..., and all possible sentence fragments with a length within the preset length range are obtained from this key sentence, and finally 48 sentence fragments will be obtained.

[0106] In a possible implementation manner, after obtaining a number of sentence fragments from each key sentence in the key sentence set, each obtained sentence fragment can be used as a candidate keyword to obtain a number of candidate keywords; in order to reduce the subsequent calculation amount, in another possible implementation manner, after obtaining a number of sentence fragments from each key sentence in the key sentence set, some sentence fragments that are obviously not keywords (such as sentence fragments containing stop words, sentence fragments containing punctuation judgments, etc.) can be filtered out. Considering that in most cases, keywords appear more than once in the text, in view of this, the number of occurrences of each obtained sentence fragment in the training text can be counted, and the sentence fragments that appear only once in the training text can be filtered out, and each of the finally remaining sentence fragments can be used as a candidate keyword to obtain a number of candidate keywords.

[0107] Step S302: Determine the candidate keywords that exactly match the keywords in the reference keyword set corresponding to the training text among the number of candidate keywords, and the word set composed of the determined keywords is used as the keyword set obtained from the training text.

[0108] It should be noted that two words exactly match means that the two words are exactly the same. For example, a candidate keyword is "speech recognition", and assuming that a keyword in the reference keyword set corresponding to the training text is "speech recognition", then this candidate keyword exactly matches the keyword "speech recognition" in the reference keyword set corresponding to the training text.

[0109] Step S303: Determine the candidate keywords that do not exactly match the keywords in the reference keyword set corresponding to the training text among a number of candidate keywords, and use the set of words composed of the determined non-keywords as the non-keyword set obtained from the training text.

[0110] It should be noted that two words do not exactly match means that the two words are completely different, or only partially the same. For example, "can be applied" and "speech recognition" are completely different, and "speech recognition" and "speech recognition system" are only partially the same.

[0111] In a possible implementation, all or part of the candidate keywords that do not exactly match the keywords in the reference keyword set corresponding to the training text among a number of candidate keywords can be used to form a non-keyword set; in order to improve the model training effect, in another possible implementation, the candidate keywords that do not exactly match the keywords in the reference keyword set corresponding to the training text and have an intersection with the keywords in the reference keyword set corresponding to the training text can be used to form a non-keyword set. It should be noted that the non-keywords in the non-keyword set obtained through the second implementation are difficult negative samples. Exemplarily, a candidate keyword is "speech recognition system", and a keyword in the reference keyword set corresponding to the training text is "speech recognition". Since "speech recognition system" does not exactly match "speech recognition" and "speech recognition system" has an intersection "speech recognition" with "speech recognition", "speech recognition system" is a difficult negative sample. Subsequently, using the difficult negative samples to train the keyword extraction model can improve the decision-making ability of the keyword extraction model for keyword boundaries.

[0112] Through the implementation methods provided above, a key sentence set, a non-key sentence set, a keyword set, and a non-keyword set can be obtained from the training text. Next, the process of training a keyword extraction model using the training text and the key sentence set, non-key sentence set, keyword set, and non-keyword set obtained from the training text will be introduced.

[0113] Please refer to Figure 4 , which shows a schematic flowchart of training a keyword extraction model using the training text and the key sentence set, non-key sentence set, keyword set, and non-keyword set obtained from the training text, and may include:

[0114] Step S401: Use the keyword extraction model to predict the probability that each sentence in the training text is a key sentence, so as to obtain the probability that each sentence in the key sentence set is a key sentence and the probability that each sentence in the non-key sentence set is a key sentence.

[0115] The process of using a keyword extraction model to predict the probability of each sentence in a training text being a key sentence may include: for each sentence in the training text, first using the keyword extraction model to obtain the context vector corresponding to each word contained in the sentence, then using the keyword extraction model and the context vectors corresponding to all words contained in the sentence to determine the representation vector of the sentence, thereby obtaining the representation vector of each sentence in the training text; using the keyword extraction model and the representation vectors of all sentences contained in the training text to determine the context vector corresponding to each sentence contained in the training text; for each sentence in the training text, predicting the probability of the sentence being a key sentence based on the context vector corresponding to the sentence, thereby obtaining the probability of each sentence in the training text being a key sentence. The context vector corresponding to a word contains the context information of the word in its sentence, and the context vector corresponding to a sentence contains the context information of the sentence in the training text.

[0116] Optional, such as Figure 5 As shown, the keyword extraction model may include a key sentence extraction module, which can be used to predict the probability of each sentence in the training text being a key sentence. Specifically, the key sentence extraction module may include a word encoding module, a sentence representation vector acquisition module, a sentence encoding module, and a key sentence prediction module. For each sentence in the training text, the representation vectors of each word contained in the sentence are first input into the word encoding module for encoding. The word encoding module outputs the context vectors corresponding to each word contained in the sentence ( Figure 5 w in s1 1,1 、w 1,2 、…w 1,n Represents the representation vector of each word contained in the first sentence of the training text, h 1,1 、h 1,2 、…h 1,n Represents the context vectors corresponding to each word in the first sentence of the training text, ..., Figure 5 s in m w in m,1 、w m,2 、…w m,n Represents the representation vector of each word contained in the mth sentence of the training text, h m,1 、h m,2 、…h m,n The context vectors corresponding to the words in the mth sentence of the training text are input into the sentence representation vector acquisition module, and the sentence representation vector acquisition module outputs the representation vector of the sentence ( Figure 5 S1 in the training text represents the representation vector of the first sentence, S2 represents the representation vector of the second sentence of the training text, and S m(representing the representation vector of the m-th sentence of the training text). Optionally, the sentence representation vector acquisition module can be a pooling layer, which performs a max-pooling operation on the context vectors corresponding to all the words included in the sentence and outputs the representation vector of the sentence. Through the above process, the representation vectors of each sentence of the training text can be obtained. Next, the representation vectors of all the sentences of the training text are input into the sentence encoding module for encoding, and the sentence encoding module outputs the context vector corresponding to each sentence of the training text( Figure 5 In Figure 5 , S1′ represents the context vector corresponding to the first sentence of the training text, S2′ represents the context vector corresponding to the second sentence of the training text, and S m ′ represents the context vector corresponding to the m-th sentence of the training text). Finally, the context vector corresponding to each sentence of the training text is input into the key sentence prediction module, and the key sentence prediction module outputs the probability that each sentence of the training text is a key sentence( Figure 5 In Figure 5 , y1 represents the probability that the first sentence of the training text is a key sentence, y2 represents the probability that the second sentence of the training text is a key sentence, and y m represents the probability that the m-th sentence of the training text is a key sentence). Optionally, the key sentence prediction module can be a classification layer, which predicts the probability that each sentence of the training text is a key sentence according to the context vector corresponding to each sentence of the training text and outputs it. It should be noted that the structure of the key sentence extraction module given above is only an example, and this embodiment is not limited thereto. As long as it can predict the probability that each sentence in the training text is a key sentence, the structure belongs to the scope protected by this application.

[0117] Step S402: Determine the key sentence prediction loss of the keyword extraction model according to the probability that each sentence in the key sentence set is a key sentence and the probability that each sentence in the non-key sentence set is a key sentence.

[0118] Specifically, the key sentence prediction loss of the keyword extraction model can be calculated according to the following formula:

[0119]

[0120] where P represents the key sentence set, N represents the non-key sentence set, |N| represents the number of sentences in the non-key sentence set, and y i represents the probability that the i-th sentence in the key sentence set P is a key sentence, and y j represents the probability that the j-th sentence in the non-key sentence set N is a key sentence.

[0121] It should be noted that in the traditional training method, the prediction loss is calculated for each sentence, with the aim of making the correct classification probabilities of all sentences approach 1. However, considering that the key sentences in this application are obtained through simple rules and may be incorrect, if trained in the above manner, it will cause difficulties in model learning. Therefore, this application proposes to obtain a negative sample set of key sentences (i.e., the non-key sentence set), and constrain the probability of predicting positive samples in the positive sample set of key sentences (i.e., the key sentence set) as key sentences to be higher than the probability of predicting negative samples in the negative sample set of key sentences (i.e., the non-key sentence set), so as to improve the stability of model training under inaccurate data.

[0122] Step S403: Use the keyword extraction model to predict the probabilities that several candidate keywords obtained from the key sentences included in the key sentence set are keywords respectively, so as to obtain the probabilities that each word in the keyword set is a keyword, and the probabilities that each word in the non-keyword set is a keyword.

[0123] The process of using the keyword extraction model to predict the probabilities that several candidate keywords obtained from the key sentences included in the key sentence set are keywords respectively may include:

[0124] Step S4031: Use the keyword extraction model to rank several candidate keywords based on the importance of each key sentence included in the key sentence set, so as to obtain a candidate keyword sequence.

[0125] When ranking several candidate keywords, the candidate keywords obtained from the same key sentence can be ranked first to obtain a candidate keyword sequence corresponding to each key sentence. When ranking the candidate keywords obtained from the same key sentence, the candidate keywords can be ranked in descending order of the number of occurrences in the training text. After obtaining the candidate keyword sequences corresponding to each key sentence included in the key sentence set, the candidate keyword sequences corresponding to each key sentence are ranked in descending order of the importance of each key sentence included in the key sentence set, so as to obtain the final candidate keyword sequence.

[0126] Exemplarily, the key sentence set includes 3 key sentences s1, s2, s3, and the candidate keywords extracted from s1 include c 11 、c 12 、c 13 ,the candidate keywords extracted from s2 include c 21 、c 22 、c 23 、c 24 ,the candidate keywords extracted from s3 include c 31 、c 32 、c 33 ,assuming that c 11 appears more times in the training text than c12 ,c 12 appears more than c times in the training text 13 ,c 21 appears more than c times in the training text 22 ,c 22 appears more than c times in the training text 23 ,c 23 appears more than c times in the training text 24 ,c 31 appears more than c times in the training text 32 ,c 32 appears more than c times in the training text 33 ,then the candidate keyword sequence corresponding to s1 is {c 11 ,c 12 ,c 13}, the candidate keyword sequence corresponding to s2 is {c 21 ,c 22 ,c 23 ,c 24}, the candidate keyword sequence corresponding to s3 is {c 31 ,c 32 ,c 33}, assuming the importance of s2 is higher than that of s1, and the importance of s1 is higher than that of s3, then the candidate keyword sequence corresponding to s2 is ranked in front of the candidate keyword sequence corresponding to s1, and the candidate keyword sequence corresponding to s3 is ranked at the end, and finally the candidate keyword sequence {c 21 ,c 22 ,c 23 ,c 24 ,c 11 ,c 12 ,c 13 ,c 31 ,c 32 ,c 33} is obtained.

[0127] Step S4032: Use the keyword extraction model to obtain the context vectors corresponding to each candidate keyword in the candidate keyword sequence.

[0128] Among them, the context vector corresponding to a candidate keyword includes the context information of the candidate keyword in the candidate keyword sequence.

[0129] Specifically, the process of using the keyword extraction model to obtain the context vectors corresponding to each candidate keyword in the candidate keyword sequence may include:

[0130] Step a1: Use the keyword extraction model to obtain the representation vectors of each candidate keyword in the candidate keyword sequence.

[0131] Among them, the representation vector of a candidate keyword includes the context information of each sub-word included in the candidate keyword in the training text;

[0132] Specifically, the process of using the keyword extraction model to obtain the representation vectors of each candidate keyword in the candidate keyword sequence may include: for each candidate keyword, using the keyword extraction model to obtain the context vectors corresponding to each sub-word included in the candidate keyword, and determining the representation vector of the candidate keyword according to the context vectors corresponding to each sub-word included in the candidate keyword. Among them, the representation vector of the candidate keyword can be obtained by performing a max-pooling operation on the context vectors corresponding to all sub-words included in the candidate keyword.

[0133] It should be noted that when predicting the probability that each sentence in the training text is a key sentence using the keyword extraction model, the context vectors corresponding to each word included in each sentence in the training text are obtained. When obtaining the context vectors corresponding to each sub-word included in a candidate keyword here, it only needs to be obtained from the previously obtained context vectors according to the start position and end position of the candidate keyword in the sentence. Exemplarily, a sentence is "The method can be applied to the post-processing module in a speech recognition system and an information retrieval system.", when predicting the probability that this sentence is a key sentence, the context vectors corresponding to "the", "method", "can",... are obtained. If a candidate keyword is "speech recognition", then the context vectors corresponding to "speech" and "recognition" obtained previously can be obtained.

[0134] Step a2: Use the keyword extraction model to obtain the context vectors corresponding to each candidate keyword in the candidate keyword sequence based on the representation vectors of each candidate keyword in the candidate keyword sequence.

[0135] The purpose of step a2 is to perform information interaction between candidate keywords so that the model can learn the context information of candidate keywords in the candidate keyword sequence.

[0136] Step S4033: For each candidate keyword in the candidate keyword sequence, use the keyword extraction model to predict the probability that the candidate keyword is a keyword based on the context vector corresponding to the candidate keyword.

[0137] There are various implementation methods for using the keyword extraction model to predict the probability that the candidate keyword is a keyword based on the context vector corresponding to the candidate keyword:

[0138] In a possible implementation, a keyword extraction model can be used to predict the probability that a candidate keyword is a keyword based only on the context vector corresponding to the candidate keyword. Specifically, the probability that a candidate keyword is a keyword can be calculated according to the following formula:

[0139] y i = softmax(Wk i + b) (2)

[0140] where W and b are model training parameters, and y i is the probability that the i-th candidate keyword in the candidate keyword sequence is a keyword, and k i is the context vector corresponding to the i-th candidate keyword.

[0141] To avoid finally extracting keywords with the same or similar semantics, in another possible implementation, a keyword extraction model can be used to predict the probability that a candidate keyword is a keyword based on the context vector corresponding to the candidate keyword and the historical keyword information vector, where the historical keyword information vector is the fusion result of the context vectors corresponding to the determined keywords. Specifically, the probability that a candidate keyword is a keyword can be calculated according to the following formula:

[0142] y i = softmax(W1k i - W2Mem i + b) (3)

[0143] Mem i = tanh(Mem i-1 + W3(sign(y i-1 - 0.5)k i-1 )) (4)

[0144] where W1, W2, W3, and b are model training parameters, y i is the probability that the i-th candidate keyword is a keyword, k i is the context vector corresponding to the i-th candidate keyword, sign is the sign function, Mem o is a vector of all zeros, and Mem i is the historical keyword information vector corresponding to the i-th candidate keyword.

[0145] It should be noted that if it is determined that the i-th candidate keyword is a keyword according to y i (for example, if y i is greater than a preset keyword probability threshold (such as 0.5), then it can be determined that the i-th candidate keyword is a keyword), then the context vector corresponding to the i-th candidate keyword is fused with Mem i to obtain Mem i+1(The fusion method can refer to the above formula (4).) The fused vector is used as the historical keyword information vector Mem corresponding to the (i + 1)-th candidate keyword. i+1 When predicting the probability that the (i + 1)-th candidate keyword is a keyword, the context vector corresponding to the (i + 1)-th candidate keyword and the historical keyword information vector Mem i+1 are used for prediction.

[0146] It should be noted that in this application, the probability of each candidate keyword in the candidate keyword sequence is predicted one by one. For the above second implementation method, when predicting the probability of each candidate keyword, the information of the already determined keywords is combined for prediction, which can suppress the prediction of keywords with the same or similar semantics as the already determined keywords.

[0147] Step S404: Determine the keyword prediction loss of the keyword extraction model according to the probability that each word in the keyword set is a keyword and the probability that each word in the non-keyword set is a keyword.

[0148] Specifically, the keyword prediction loss of the keyword extraction model can be calculated according to the following formula:

[0149]

[0150] where Q represents the keyword set, M represents the non-keyword set, |M| represents the number of words in the non-keyword set, p i represents the probability that the i-th word in the keyword set Q is a keyword, and p j represents the probability that the j-th word in the non-keyword set M is a keyword.

[0151] Optionally, the keyword extraction model may include a keyword extraction module, and the keyword extraction module of the keyword extraction model can be used to predict the probability that each sentence in the training text is a key sentence according to the implementation method provided above. Specifically, such as Figure 5As shown in the figure, the keyword extraction module may include a candidate keyword representation vector acquisition module, a candidate keyword encoding module, and a keyword prediction module. Among them, the candidate keyword representation vector acquisition module acquires the representation vectors of each candidate keyword in the candidate keyword sequence and outputs them. The representation vectors of each keyword in the candidate keyword sequence are input into the candidate keyword encoding module for encoding. The candidate keyword encoding module outputs the context vectors corresponding to each candidate keyword in the candidate keyword sequence. Optionally, the candidate keyword encoding module can be, but is not limited to, a multi-layer Transformer (such as a three-layer Transformer). The context vectors corresponding to each keyword in the candidate keyword sequence are input into the keyword prediction module one by one for probability prediction. The keyword prediction module outputs the probability that the corresponding candidate keyword is a keyword. Optionally, the keyword extraction module may further include a historical keyword information processing and recording module. After each keyword is determined, the historical keyword information processing and recording module fuses the context vector corresponding to the currently determined keyword with the recorded historical key information vector and records the fusion result. It should be noted that the initial historical key information vector is a vector of all zeros. After the first keyword is determined, the context vector corresponding to the first keyword is fused with the initial historical key information vector to obtain a first fusion result. After the second keyword is determined, the context vector corresponding to the second keyword is fused with the above first fusion result to obtain a second fusion result. After the third keyword is determined, the context vector corresponding to the third keyword is fused with the above second fusion result, and so on. It should be noted that the structure of the keyword extraction module given above is only an example, and this embodiment is not limited thereto. As long as the structure for keyword probability prediction is in the manner of steps S4031 to S4031, it belongs to the scope protected by this application.

[0152] Step S405: Update the parameters of the keyword extraction model according to the key sentence prediction loss of the keyword extraction model and the keyword prediction loss of the keyword extraction model.

[0153] According to the above training method, different training data are used for multiple iterative trainings until the training end condition is satisfied. After the training is completed, the trained keyword extraction model can be used to extract keywords from the text data to be extracted for keywords.

[0154] The embodiments of the present application use training texts, as well as key sentence sets, non-key sentence sets, keyword sets, and non-keyword sets obtained from the training texts, as training data, with the goal of training to make the probability of predicting a key sentence in the key sentence set as a key sentence greater than the probability of predicting a non-key sentence in the non-key sentence set as a key sentence, and making the probability of predicting a keyword in the keyword set as a keyword greater than the probability of predicting a non-keyword in the non-keyword set as a keyword. This training strategy enables the training of a keyword extraction model with better performance without the need to obtain high-quality labeled data.

[0155] The third embodiment

[0156] Based on the above embodiments, this embodiment introduces the specific implementation processes of "Step S102: Extract key sentences from text data using a pre-trained keyword extraction model" and "Step S103: Extract keywords from the extracted key sentences using the keyword extraction model" in the above embodiments.

[0157] The process of extracting key sentences from text data using a pre-trained keyword extraction model may include:

[0158] Step b1: Use the keyword extraction model to predict the probability of each sentence in the text data being a key sentence.

[0159] Specifically, the process of using the keyword extraction model to predict the probability of each sentence in the text data being a key sentence may include: for each sentence in the text data, first use the keyword extraction model to obtain the context vector corresponding to each word included in the sentence (the context vector corresponding to a word includes the context information of the word in its sentence), and then use the keyword extraction model and the context vectors corresponding to all the words included in the sentence to determine the representation vector of the sentence, so as to obtain the representation vector of each sentence in the text data; use the keyword extraction model and the representation vectors of all the sentences included in the text data to determine the context vector corresponding to each sentence included in the text data (the context vector corresponding to a sentence includes the context information of the sentence in the text data); predict the probability of each sentence in the training text being a key sentence according to the context vector corresponding to each sentence included in the text data.

[0160] It should be noted that the implementation process of using the keyword extraction model to predict the probability of each sentence in the text data being a key sentence is the same as the implementation process of using the keyword extraction model to predict the probability of each sentence in the training text being a key sentence in the above embodiments. For a more specific implementation process of using the keyword extraction model to predict the probability of each sentence in the text data being a key sentence, reference can be made to the specific implementation process of using the keyword extraction model to predict the probability of each sentence in the text data being a key sentence in the above embodiments, and this embodiment will not elaborate here.

[0161] Step b2: Determine the key sentences based on the probability of each sentence in the text data being a key sentence, so as to obtain the key sentences extracted from the text data.

[0162] There are multiple ways to determine the key sentences based on the probability of each sentence in the text data being a key sentence. In one possible implementation, the preset number (such as 10) of sentences with the highest probability can be determined as the key sentences. In another possible implementation, the sentences with a probability greater than the preset key sentence probability threshold can be determined as the key sentences.

[0163] After extracting the key sentences from the text data, the keyword extraction model can be further used to extract keywords from the extracted key sentences. The implementation process of using the keyword extraction model to extract keywords from the extracted key sentences can include:

[0164] Step c1: Use the keyword extraction model to obtain a number of candidate keywords from the extracted key sentences.

[0165] Each candidate keyword is a sentence fragment within the preset length range in the extracted key sentence.

[0166] The implementation process of using the keyword extraction model to obtain a number of candidate keywords from the extracted key sentences is the same as the implementation process of "obtaining a number of candidate keywords from the key sentences in the key sentence set" in the above embodiment. For details, please refer to the relevant parts in the above embodiment, and this embodiment will not be elaborated here.

[0167] Step c2: Use the keyword extraction model to predict the probabilities of the candidate keywords being keywords respectively, and determine the keywords from the candidate keywords according to the probabilities of the candidate keywords being keywords respectively.

[0168] Specifically, the implementation process of step c2 can include:

[0169] Step c21: Use the keyword extraction model to sort the candidate keywords based on the importance of the extracted key sentences to obtain a candidate keyword sequence.

[0170] Specifically, the candidate keywords obtained from the same key sentence can be sorted in descending order of the number of occurrences in the text data to obtain the candidate keyword sequence corresponding to each key sentence. Then, the candidate keyword sequences corresponding to the key sentences can be sorted in descending order of the importance of the key sentences to obtain the final candidate keyword sequence. It should be noted that the higher the prediction probability corresponding to a key sentence, the higher the importance of the key sentence. The prediction probability corresponding to a key sentence refers to the probability that the keyword extraction model predicts the sentence as a key sentence.

[0171] Step c22: Use the keyword extraction model to obtain the context vectors corresponding to each candidate keyword in the candidate keyword sequence.

[0172] Among them, the context vector corresponding to a candidate keyword contains the context information of the corresponding candidate keyword in the candidate keyword sequence.

[0173] Using the keyword extraction model to obtain the context vectors corresponding to each candidate keyword in the candidate keyword sequence includes: using the keyword extraction model to obtain the representation vectors of each candidate keyword in the candidate keyword sequence to obtain a candidate keyword representation vector sequence, where the representation vector of a candidate keyword contains the context information of each sub-word included in the candidate keyword in the training text; using the keyword extraction model, based on the candidate keyword representation vector sequence, to obtain the context vectors corresponding to each candidate keyword in the candidate keyword sequence.

[0174] Step c23: For each candidate keyword in the candidate keyword sequence, use the keyword extraction model, based on the context vector corresponding to the candidate keyword, to predict the probability that the candidate keyword is a keyword, and determine whether the candidate keyword is a keyword according to the probability that the candidate keyword is a keyword.

[0175] Among them, there are various implementation methods for using the keyword extraction model to predict the probability that the candidate keyword is a keyword based on the context vector corresponding to the candidate keyword:

[0176] In a possible implementation method, the keyword extraction model can be used to predict the probability that the candidate keyword is a keyword only based on the context vector corresponding to the candidate keyword; in order to avoid finally extracting keywords with the same or similar semantics, in another possible implementation method, the keyword extraction model can be used to predict the probability that the candidate keyword is a keyword based on the context vector corresponding to the candidate keyword and the historical keyword information vector, where the historical keyword information vector is the fusion result of the context vectors corresponding to the determined keywords. In the above second implementation method, for each candidate keyword, the keyword extraction model is used to predict the probability that the candidate keyword is a keyword by combining the information of the determined keywords. Combining the information of the determined keywords for probability prediction can suppress the prediction of keywords with the same or similar semantics as the determined keywords.

[0177] If the above second implementation method is adopted, for each candidate keyword in the candidate keyword sequence, if it is determined that the candidate keyword is a keyword according to the probability that the candidate keyword is a keyword, then fuse the representation vector of the candidate keyword with the historical keyword information vector, and the fused vector is used as the new historical keyword information vector.

[0178] It should be noted that the specific implementation process of "using the keyword extraction model to predict the probability that the candidate keyword is a keyword based on the context vector corresponding to the candidate keyword" in step c23 is the same as that of "step S4033 in the above embodiment, for each candidate keyword in the candidate keyword sequence, using the keyword extraction model to predict the probability that the candidate keyword is a keyword based on the context vector corresponding to the candidate keyword". For a more specific implementation process, reference can be made to the relevant parts in the above embodiment, and details will not be elaborated in this embodiment.

[0179] When determining whether a candidate keyword is a keyword based on the probability that the candidate keyword is a keyword, it can be determined whether the probability that the candidate keyword is a keyword is greater than a preset keyword probability threshold (such as 0.5). If the probability that the candidate keyword is a keyword is greater than the preset keyword probability threshold, it is determined that the candidate keyword is a keyword; otherwise, it is determined that the candidate keyword is not a keyword.

[0180] Fourth Embodiment

[0181] The embodiments of the present application further provide a keyword extraction device. The keyword extraction device provided in the embodiments of the present application will be described below. The keyword extraction device described below can be correspondingly referred to the keyword extraction method described above.

[0182] Please refer to Figure 6 , which shows a schematic structural diagram of the voice endpoint detection device provided by the embodiments of the present application, and may include: a text acquisition module 601 and a text processing module 602.

[0183] The text acquisition module 601 is used to acquire text data of keywords to be extracted;

[0184] The text processing module 602 is used to extract key sentences from the text data by using a pre-trained keyword extraction model, and extract keywords from the extracted key sentences by using the keyword extraction model.

[0185] Among them, the keyword extraction model uses training text and key sentence sets, non-key sentence sets, keyword sets, and non-keyword sets obtained from the training text, and is trained with the goal of making the probability of predicting the key sentences in the key sentence set as key sentences greater than the probability of predicting the non-key sentences in the non-key sentence set as key sentences, and making the probability of predicting the keywords in the keyword set as keywords greater than the probability of predicting the non-keywords in the non-keyword set as keywords.

[0186] Optionally, when the text processing module 602 extracts keywords from the extracted key sentences using the keyword extraction model, it specifically is used for:

[0187] Using the keyword extraction model to obtain a number of candidate keywords from the extracted key sentences, where each candidate keyword is a sentence fragment in the extracted key sentence with a length within a preset length range;

[0188] Using the keyword extraction model to predict the probabilities that the number of candidate keywords are keywords respectively, and determining keywords from the number of candidate keywords according to the probabilities that the number of candidate keywords are keywords respectively.

[0189] Optionally, when the text processing module 602 predicts the probabilities that the number of candidate keywords are keywords respectively using the keyword extraction model, it specifically is used for:

[0190] For each candidate keyword among the number of candidate keywords, using the keyword extraction model, combining the information of the keywords that have been determined, to predict the probability that this candidate keyword is a keyword.

[0191] Optionally, when the text processing module 602 predicts the probabilities that the number of candidate keywords are keywords respectively using the keyword extraction model, it specifically is used for:

[0192] Using the keyword extraction model, based on the importance of the extracted key sentences, sorting the number of candidate keywords to obtain a candidate keyword sequence;

[0193] Using the keyword extraction model to obtain the context vectors corresponding to the candidate keywords in the candidate keyword sequence respectively, where the context vector contains the context information of the corresponding candidate keyword in the candidate keyword sequence;

[0194] For each candidate keyword in the candidate keyword sequence, using the keyword extraction model, based on the context vector corresponding to this candidate keyword, to predict the probability that this candidate keyword is a keyword.

[0195] Optionally, when the text processing module 602 uses the keyword extraction model to obtain the context vectors corresponding to the candidate keywords in the candidate keyword sequence respectively, it specifically is used for:

[0196] Using the keyword extraction model to obtain the representation vectors of the candidate keywords in the candidate keyword sequence respectively, where the representation vector of a candidate keyword contains the context information of each sub-word included in this candidate keyword in the training text;

[0197] The keyword extraction model and the representation vector of each candidate keyword in the candidate keyword sequence are used to obtain the context vector corresponding to each candidate keyword in the candidate keyword sequence.

[0198] Optionally, the text processing module 602 uses the keyword extraction model and the context vector corresponding to the candidate keyword to predict the probability that the candidate keyword is a keyword, including:

[0199] Using the keyword extraction model, based on the context vector corresponding to the candidate keyword and the historical keyword information vector, the probability of the candidate keyword being a keyword is predicted, wherein the historical keyword information vector is a fusion result of the context vectors corresponding to each determined keyword;

[0200] The text processing module 602 is further configured to:

[0201] For each candidate keyword in the candidate keyword sequence, if the candidate keyword is determined to be a keyword based on the probability that the candidate keyword is a keyword, the context vector corresponding to the candidate keyword is fused with the historical keyword information vector, and the fused vector is used as a new historical keyword information vector.

[0202] Optionally, the training text has a corresponding reference keyword set.

[0203] The keyword extraction device provided in the embodiment of the present application may further include a sentence set acquisition module, which is used to acquire key sentence sets and non-key sentence sets from the training text.

[0204] When the sentence set acquisition module acquires the key sentence set and the non-key sentence set from the training text, it is specifically used to:

[0205] Segmenting the training text to obtain each sentence of the training text;

[0206] For each sentence of the training text, determine a score for the sentence based on the occurrence of the common words of the reference keyword set and the sentence in the sentence and in the corpus;

[0207] Determining key sentences according to the scores of the sentences in the training text, and using a sentence set consisting of the determined key sentences as the key sentence set obtained from the training text;

[0208] A sentence set consisting of several sentences that have no intersection with the reference keyword set is used as a non-key sentence set obtained from the training text.

[0209] The keyword extraction device provided in the embodiment of the present application may further include a word set acquisition module, which is used to acquire a keyword set and a non-keyword set from the training text.

[0210] When the word set acquisition module acquires the keyword set and the non-keyword set from the training text, it is specifically used to:

[0211] Acquire several candidate keywords from the key sentences in the key sentence set;

[0212] Determining candidate keywords among the plurality of candidate keywords that completely match keywords in the reference keyword set as keywords, and using a word set consisting of the determined keywords as a keyword set obtained from the training text;

[0213] The candidate keywords among the candidate keywords that do not completely match the keywords in the reference keyword set are determined as non-keywords, and a word set consisting of the determined non-keywords is used as a non-keyword set obtained from the training text.

[0214] When determining a candidate keyword among the candidate keywords that does not completely match a keyword in the reference keyword set as a non-keyword, the word set acquisition module is specifically configured to:

[0215] The candidate keywords among the candidate keywords that do not completely match the keywords in the reference keyword set and have an intersection with the keywords in the reference keyword set are determined as non-keywords.

[0216] The keyword extraction device provided in the embodiment of the present application may further include a model training module. The model training module is used to:

[0217] Using a keyword extraction model, predicting the probability that each sentence in the training text is a key sentence, so as to obtain the probability that each sentence in the key sentence set is a key sentence and the probability that each sentence in the non-key sentence set is a key sentence;

[0218] Determining a key sentence prediction loss based on a probability that each sentence in the key sentence set is a key sentence and a probability that each sentence in the non-key sentence set is a key sentence;

[0219] Predicting the probability that several candidate keywords obtained from the key sentences included in the key sentence set are keywords, so as to obtain the probability that each word in the keyword set is a keyword, and the probability that each word in the non-keyword set is a keyword;

[0220] determining a keyword prediction loss based on a probability that each word in the keyword set is a keyword and a probability that each word in the non-keyword set is a keyword;

[0221] Update the parameters of the keyword extraction model according to the loss predicted by the key sentence and the loss predicted by the keyword.

[0222] After obtaining the text data of the keyword to be extracted, the keyword extraction device provided in the embodiment of the present application first extracts the key sentence from the text data by using a pre-trained keyword extraction model, and then extracts the keyword from the extracted key sentence by using the keyword extraction model. The keyword extraction model in the embodiment of the present application is not trained as in the prior art, which uses the training text marked with keywords and aims to make the predicted keywords for the training text tend to be consistent with the keywords marked in the training text. Instead, it uses the training text and the key sentence set, non-key sentence set, keyword set, and non-keyword set obtained from the training text as the training data, and aims to make the probability of predicting the key sentence in the key sentence set as a key sentence greater than the probability of predicting the non-key sentence in the non-key sentence set as a key sentence, and make the probability of predicting the keyword in the keyword set as a keyword greater than the probability of predicting the non-keyword in the non-keyword set as a keyword. This training strategy enables the training of a keyword extraction model with better performance without obtaining high-quality labeled data. Using the keyword extraction model with better performance to extract keywords from the text data of the keyword to be extracted can obtain better extraction results. In addition, the keyword extraction device provided in the embodiment of the present application does not directly extract keywords from the text data, but first extracts the key sentence from the text data, and then further extracts the keyword from the extracted key sentence. This extraction strategy has higher extraction efficiency compared to directly extracting keywords from the text data.

[0223] The fifth embodiment

[0224] The embodiment of the present application also provides a keyword extraction device. Please refer to Figure 7 , which shows a schematic structural diagram of the keyword extraction device. The keyword extraction device may include: at least one processor 701, at least one communication interface 702, at least one memory 703, and at least one communication bus 704;

[0225] In the embodiment of the present application, the number of the processor 701, the communication interface 702, the memory 703, and the communication bus 704 is at least one, and the processor 701, the communication interface 702, and the memory 703 complete mutual communication through the communication bus 704;

[0226] The processor 701 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention, etc.;

[0227] The memory 703 may include high-speed RAM memory and may also include non-volatile memory, such as at least one disk memory;

[0228] Among them, the memory stores a program, and the processor can call the program stored in the memory. The program is used for:

[0229] Obtain the text data of the keywords to be extracted; use the pre-trained keyword extraction model to extract key sentences from the text data, and use the keyword extraction model to extract keywords from the extracted key sentences; among them, the keyword extraction model uses the training text and the key sentence set, non-key sentence set, keyword set, and non-keyword set obtained from the training text, so that the probability of predicting the key sentences in the key sentence set as key sentences is greater than the probability of predicting the non-key sentences in the non-key sentence set as key sentences, and the probability of predicting the keywords in the keyword set as keywords is greater than the probability of predicting the non-keywords in the non-keyword set as keywords for target training.

[0230] Optionally, the refinement function and extension function of the program can refer to the above description.

[0231] Sixth Embodiment

[0232] The embodiment of the present application also provides a computer-readable storage medium, which can store a program suitable for execution by a processor. The program is used for:

[0233] Obtain the text data of the keywords to be extracted; use the pre-trained keyword extraction model to extract key sentences from the text data, and use the keyword extraction model to extract keywords from the extracted key sentences; among them, the keyword extraction model uses the training text and the key sentence set, non-key sentence set, keyword set, and non-keyword set obtained from the training text, so that the probability of predicting the key sentences in the key sentence set as key sentences is greater than the probability of predicting the non-key sentences in the non-key sentence set as key sentences, and the probability of predicting the keywords in the keyword set as keywords is greater than the probability of predicting the non-keywords in the non-keyword set as keywords for target training.

[0234] Optionally, the refinement function and extension function of the program can refer to the above description.

[0235] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising said element.

[0236] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other.

[0237] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.

Claims

1. A keyword extraction method, characterized in that Including: Obtain the text data of the keywords to be extracted; Use a pre-trained keyword extraction model to extract key sentences from the text data, and use the keyword extraction model to obtain several candidate keywords from the extracted key sentences, where each candidate keyword is a sentence fragment within a preset length range in the extracted key sentences; Use the keyword extraction model to sort the several candidate keywords based on the importance of the extracted key sentences to obtain a candidate keyword sequence; Use the keyword extraction model to obtain the context vectors corresponding to each candidate keyword in the candidate keyword sequence, where the context vector contains the context information of the corresponding candidate keyword in the candidate keyword sequence; For each candidate keyword in the candidate keyword sequence, use the keyword extraction model to predict the probability that the candidate keyword is a keyword based on the context vector corresponding to the candidate keyword, and determine the keyword from the several candidate keywords; Among them, the keyword extraction model uses training texts and the key sentence set, non-key sentence set, keyword set, and non-keyword set obtained from the training texts, so that the probability of predicting the key sentences in the key sentence set as key sentences is greater than the probability of predicting the non-key sentences in the non-key sentence set as key sentences, and the probability of predicting the keywords in the keyword set as keywords is greater than the probability of predicting the non-keywords in the non-keyword set as keywords, and is trained with this as the goal.

2. The keyword extraction method according to claim 1, wherein It also includes: For each candidate keyword among the several candidate keywords, use the keyword extraction model to predict the probability that the candidate keyword is a keyword in combination with the information of the already determined keywords.

3. The keyword extraction method according to claim 1, wherein The step of using the keyword extraction model to obtain the context vectors corresponding to each candidate keyword in the candidate keyword sequence includes: Use the keyword extraction model to obtain the representation vectors of each candidate keyword in the candidate keyword sequence, where the representation vector of a candidate keyword contains the context information of each sub-word included in the candidate keyword in the training text; Use the keyword extraction model and the representation vectors of each candidate keyword in the candidate keyword sequence to obtain the context vectors corresponding to each candidate keyword in the candidate keyword sequence.

4. The keyword extraction method according to claim 1, wherein The step of using the keyword extraction model to predict the probability that the candidate keyword is a keyword based on the context vector corresponding to the candidate keyword includes: Use the keyword extraction model to predict the probability that the candidate keyword is a keyword based on the context vector corresponding to the candidate keyword and the historical keyword information vector, where the historical keyword information vector is the fusion result of the context vectors corresponding to the already determined keywords; The keyword extraction method also includes: For each candidate keyword in the candidate keyword sequence, if it is determined that the candidate keyword is a keyword according to the probability that the candidate keyword is a keyword, the context vector corresponding to the candidate keyword is fused with the historical keyword information vector, and the fused vector is used as the new historical keyword information vector.

5. The keyword extraction method according to claim 1, characterized in that The training text has a corresponding reference keyword set; Obtaining a key sentence set and a non-key sentence set from the training text, including: Performing sentence splitting on the training text to obtain each sentence of the training text; For each sentence of the training text, determine the score of the sentence according to the occurrence of the common words between the reference keyword set and the sentence in the sentence and in the corpus; Determine the key sentences according to the scores of the sentences of the training text, and the sentence set composed of the determined key sentences is used as the key sentence set obtained from the training text; The sentence set composed of several sentences that have no intersection with the reference keyword set is used as the non-key sentence set obtained from the training text.

6. The keyword extraction method according to claim 5, characterized in that Obtaining a keyword set and a non-keyword set from the training text, including: Obtaining several candidate keywords from the key sentences in the key sentence set; Determine the candidate keywords that exactly match the keywords in the reference keyword set among the several candidate keywords, and the word set composed of the determined keywords is used as the keyword set obtained from the training text; Determine the candidate keywords that do not exactly match the keywords in the reference keyword set among the several candidate keywords, and the word set composed of the determined non-keywords is used as the non-keyword set obtained from the training text.

7. The keyword extraction method according to claim 6, wherein The determination of the candidate keywords that do not exactly match the keywords in the reference keyword set among the several candidate keywords as non-keywords includes: Determine the candidate keywords that do not exactly match the keywords in the reference keyword set and have an intersection with the keywords in the reference keyword set as non-keywords.

8. The keyword extraction method according to claim 1, characterized in that The training process of the keyword extraction model includes: Using the keyword extraction model to predict the probability that each sentence in the training text is a key sentence, so as to obtain the probability that each sentence in the key sentence set is a key sentence and the probability that each sentence in the non-key sentence set is a key sentence; Determine the key sentence prediction loss according to the probability that each sentence in the key sentence set is a key sentence and the probability that each sentence in the non-key sentence set is a key sentence; Predict the probability that each of the several candidate keywords obtained from the key sentences included in the key sentence set is a keyword, so as to obtain the probability that each word in the keyword set is a keyword and the probability that each word in the non-keyword set is a keyword; Determine the keyword prediction loss according to the probability that each word in the keyword set is a keyword and the probability that each word in the non-keyword set is a keyword; Update the parameters of the keyword extraction model according to the key sentence prediction loss and the keyword prediction loss.

9. A keyword extraction device, characterized in that, Including: A text acquisition module and a text processing module; The text acquisition module is used to acquire the text data for which keywords are to be extracted; The text processing module is used to extract key sentences from the text data by using a pre-trained keyword extraction model, and obtain a number of candidate keywords from the extracted key sentences, where each candidate keyword is a sentence fragment with a length within a preset length range in the extracted key sentences; use the keyword extraction model to sort the number of candidate keywords based on the importance of the extracted key sentences to obtain a candidate keyword sequence; use the keyword extraction model to obtain context vectors corresponding to each candidate keyword in the candidate keyword sequence, where the context vector contains context information of the corresponding candidate keyword in the candidate keyword sequence; for each candidate keyword in the candidate keyword sequence, use the keyword extraction model to predict the probability that the candidate keyword is a keyword based on the context vector corresponding to the candidate keyword, and determine the keyword from the number of candidate keywords; Among them, the keyword extraction model uses training texts and key sentence sets, non-key sentence sets, keyword sets, and non-keyword sets obtained from the training texts, so as to train with the goal of making the probability of predicting the key sentences in the key sentence set as key sentences greater than the probability of predicting the non-key sentences in the non-key sentence set as key sentences, and the probability of predicting the keywords in the keyword set as keywords greater than the probability of predicting the non-keywords in the non-keyword set as keywords.

10. The keyword extraction device according to claim 9, wherein The text processing module is further used for: For each candidate keyword among the number of candidate keywords, use the keyword extraction model to predict the probability that the candidate keyword is a keyword in combination with the information of the determined keywords.

11. A keyword extraction device, characterized in that, Including: A memory and a processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the keyword extraction method described in any one of claims 1 to 8.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, each step of the keyword extraction method described in any one of claims 1 to 8 is implemented.

Images