Keyword extraction method, device and electronic device

Abstract

The embodiment of the present application discloses a keyword extraction method, device and electronic device. The method includes: obtaining a text to be processed; performing word segmentation processing on the text to be processed to obtain a word segmentation result corresponding to the text to be processed; inputting the text to be processed into a labeling model to obtain an output result corresponding to the text to be processed, the output result including a labeling result, the labeling result characterizing the keyword distribution of the text to be processed; obtaining a first keyword based on the labeling result; obtaining a target keyword based on the word segmentation result and the first keyword. Through the above method, the labeling result corresponding to the text to be processed can be obtained based on the labeling model, and the first keyword can be obtained based on the labeling result, and then the target keyword can be obtained based on the word segmentation result corresponding to the text to be processed and the first keyword, so as to obtain sufficient target keywords.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to a keyword extraction method, device, and electronic device. Background Art

[0002] Text keyword extraction technology can extract the most relevant words from a text. These words can be used for subsequent scenarios such as information flow recommendations and knowledge base construction. However, among related methods, text keyword extraction technology still suffers from insufficient keyword extraction. For example, methods based on statistical analysis and graph models do not fully consider the context of the text, making it difficult to extract key but low-frequency words in the text. For another example, methods based on sequence models often fail to extract keywords in extremely short text scenarios such as short video titles, news headlines, and product titles. Summary of the Invention

[0003] In view of the above problems, the present application proposes a keyword extraction method, device and electronic device to improve the above problems.

[0004] In the first aspect, the present application provides a keyword extraction method, which includes: obtaining a text to be processed; performing word segmentation processing on the text to be processed to obtain a word segmentation result corresponding to the text to be processed; inputting the text to be processed into a labeling model to obtain an output result corresponding to the text to be processed, the output result including a labeling result, and the labeling result characterizing the keyword distribution of the text to be processed; obtaining a first keyword based on the labeling result; and obtaining a target keyword based on the word segmentation result and the first keyword.

[0005] In the second aspect, the present application provides a keyword extraction device, which includes: a text to be processed acquisition unit, used to acquire the text to be processed; a word segmentation result acquisition unit, used to perform word segmentation processing on the text to be processed to obtain the word segmentation result corresponding to the text to be processed; an annotation result acquisition unit, used to input the text to be processed into an annotation model to obtain an output result corresponding to the text to be processed, the output result includes a annotation result, and the annotation result represents the keyword distribution of the text to be processed; a first keyword acquisition unit, used to obtain a first keyword based on the annotation result; a target keyword acquisition unit, used to obtain a target keyword based on the word segmentation result and the first keyword.

[0006] In a third aspect, the present application provides an electronic device comprising one or more processors and a memory; one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the above-mentioned method.

[0007] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores program code, wherein the above method is executed when the program code is run.

[0008] The present application provides a keyword extraction method, device, electronic device, and storage medium. After obtaining a text to be processed, the text to be processed is segmented to obtain a segmentation result corresponding to the text to be processed, and the text to be processed is input into a labeling model to obtain an output result of a labeling result corresponding to the text to be processed, including a result of a keyword distribution representing the text to be processed. A first keyword is obtained based on the labeling result; and a target keyword is obtained based on the segmentation result and the first keyword. Through the above-mentioned method, the labeling result corresponding to the text to be processed can be obtained based on the labeling model, and the first keyword can be obtained based on the labeling result. Then, the target keyword can be obtained based on the segmentation result corresponding to the text to be processed and the first keyword, so as to obtain sufficient target keywords. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 A flowchart of a keyword extraction method proposed in an embodiment of the present application is shown;

[0011] Figure 2 Shows this application Figure 1 A flowchart of an embodiment of S150;

[0012] Figure 3 A flowchart of a keyword extraction method proposed in another embodiment of the present application is shown;

[0013] Figure 4 A schematic diagram of a training annotation model proposed in this application is shown;

[0014] Figure 5 A schematic diagram of another annotation model to be trained proposed in this application is shown;

[0015] Figure 6 The following is a schematic diagram showing the basic flow of the keyword extraction method proposed in this application;

[0016] Figure 7 A structural block diagram of a keyword extraction device proposed in an embodiment of the present application is shown;

[0017] Figure 8 Shows a structural block diagram of an electronic device proposed in this application;

[0018] Figure 9 It is a storage unit for storing or carrying program codes for implementing the keyword extraction method according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0020] As the demand for text processing increases, it is required in many scenarios, such as information flow recommendation and knowledge base construction.

[0021] The inventors discovered during their research that text keyword extraction techniques still suffer from insufficient keyword extraction. For example, statistical analysis-based and graph-based methods fail to fully consider the context of the text, making it difficult to extract key but low-frequency words. Another example is that sequence model-based methods, when used in contexts with extremely short text, such as short video titles, news headlines, and product titles, often yield few or no keywords at all.

[0022] Therefore, the inventors proposed a keyword extraction method, device, and electronic device in this application. After obtaining the text to be processed, the text to be processed is segmented to obtain the segmentation result corresponding to the text to be processed, and the text to be processed is input into the annotation model to obtain the output result of the annotation result corresponding to the text to be processed, including the keyword distribution result characterizing the text to be processed, and the first keyword is obtained based on the annotation result; the target keyword is obtained based on the segmentation result and the first keyword. Through the above method, the annotation result corresponding to the text to be processed can be obtained based on the annotation model, and the first keyword can be obtained based on the annotation result. Then, the target keyword can be obtained based on the segmentation result corresponding to the text to be processed and the first keyword, so as to obtain sufficient target keywords.

[0023] In order to better understand the solutions of the embodiments of the present application, the technical terms used in the embodiments of the present application are explained below.

[0024] HMM (Hidden Markov Model) is a statistical model that can be used to describe a Markov process with hidden unknown parameters. It is mainly used for modeling time series data and is widely used in fields such as speech recognition and natural language processing. In word segmentation algorithms, Hidden Markov is often used as an algorithm to discover new words. By learning from massive amounts of data, it can identify names of people, places, new words on the Internet, etc., and has a wide range of application scenarios.

[0025] A Conditional Random Field (CRF) is a conditional probability distribution model that models a set of output random variables given a set of input random variables. Its characteristic is that it assumes that the output random variables form a Markov random field. CRFs are often used in sequence labeling problems, such as named entity recognition.

[0026] SVM (Support Vector Machine) is a binary classification model. This binary classification model can be a linear classifier with the largest margin defined in the feature space. The learning strategy of SVM is to maximize the margin, which can be formalized as a problem of solving a convex quadratic programming, which is equivalent to minimizing the regularized hinge loss function.

[0027] BERT (Bidirectional Encoder Representation from Transformers) is a pre-trained language representation model that can generate deep bidirectional language representations.

[0028] LSTM (Long Short-Term Memory) is a time-recurrent neural network designed to solve the long-term dependency problem of general RNN.

[0029] GRU (Gate Recurrent Unit), a gating mechanism designed to solve the gradient vanishing or exploding problem in standard RNN while retaining the long-term information of the sequence, is a variant of the LSTM network.

[0030] The embodiments of the present application will be described below with reference to the accompanying drawings.

[0031] See also Figure 1 The present application provides a keyword extraction method, which is applied to electronic devices and includes:

[0032] S110: Obtain the text to be processed.

[0033] The text to be processed refers to the text from which keywords are to be extracted, and keywords may refer to words that can represent important information in the text. Optionally, the text to be processed may be in Chinese or English.

[0034] As a way, the application scenario of the keyword extraction method of the present application can be determined first, and then the text to be processed can be obtained based on the determined application scenario through business collection, crawler technology, public data sets, etc. Among them, business collection refers to the case where the keyword extraction method of the present application needs to be applied to the application scenario corresponding to a certain specific business (such as: product search, news title keyword extraction, etc.), and the text to be processed is collected for this specific business. For example, when the application scenario is news title keyword extraction, news data on the business side can be collected as the text to be processed, or news data from other news publishing platforms can be crawled from the network through crawler technology as the text to be processed, and data can also be obtained from public data sets (such as: TNews data sets, etc.) as the text to be processed.

[0035] S120: Perform word segmentation processing on the text to be processed to obtain a word segmentation result corresponding to the text to be processed.

[0036] As one approach, for example, if the text to be processed is Chinese, the text can be segmented using character matching, statistical methods, deep learning algorithms, or a combination of machine learning algorithms (HMM, CRF, SVM, etc.) and statistical methods to obtain a segmentation result corresponding to the text to be processed. The segmentation result can include multiple candidate words. For example, the text to be processed can be segmented using open source Chinese word segmentation tool libraries such as Jieba (statistical methods, HMM), THULAC, pkuseg, and FoolNLTK.

[0037] Optionally, different granularities can be set for word segmentation of the text to be processed based on different business needs and the type of business data to obtain the word segmentation results corresponding to the text to be processed. By setting different granularities for word segmentation of the text to be processed, the word segmentation results obtained may be different, and the range of keywords obtained may also be different, thereby making the range corresponding to the output results of subsequent tasks (such as business recommendations, etc.) may also be different. Among them, the coarser the granularity of the word segmentation result, the longer the keywords obtained, the smaller the range of keywords included, and the more accurate the output results of subsequent tasks (such as business recommendations, etc.) can be.

[0038] For example, in a recommendation business scenario, suppose the text to be processed is: "Xiao Ming is studying at the University of the Chinese Academy of Sciences, China's largest national research institution." To provide more accurate recommendation information, coarse-grained word segmentation can be performed on the text to be processed. The resulting coarse-grained word segmentation results may be: "Xiao Ming," "study at," "Chinese Academy of Sciences," "university," ",","China," "largest," "national," "research institution." In this case, if the keyword corresponding to the coarse-grained word segmentation result is "Chinese Academy of Sciences," information and products related to the Chinese Academy of Sciences can be recommended. To provide as much recommended information as possible, fine-grained word segmentation can be performed on the text to be processed. The resulting fine-grained word segmentation results may be: "Xiao Ming," "study at," "China," "Academy of Sciences," "university," ",","China," "largest," "national," "research institution." In this case, if the keyword corresponding to the fine-grained word segmentation result is "Academy of Sciences," information and products related to academies of sciences in various countries and regions can be recommended, rather than just those related to the Chinese Academy of Sciences.

[0039] For example, if most of the data in the recommendation business scenario is XXX University, the keyword corresponding to the coarse-grained word segmentation result can be the full name of the university, such as Princeton University. In this case, the relevant information of Princeton University will be recommended; while the keyword corresponding to the fine-grained word segmentation result can be XXX+University, such as Princeton+University. In this case, information about Princeton and the university will be recommended.

[0040] Optionally, the text to be processed is segmented according to the preset part of speech of the target keyword to obtain a segmentation result corresponding to the text to be processed, and the segmentation result may include candidate words with the same part of speech as the target keyword. For example, the text to be processed is: "Xiao Ming is studying at the University of the Chinese Academy of Sciences, China's largest national research institution". When the part of speech of the target keyword is set to noun, the keywords obtained based on the segmentation result may be Xiao Ming, Chinese Academy of Sciences, and research institution; and when the part of speech is set to noun and verb, the keywords obtained based on the segmentation result may be Xiao Ming, studying, Chinese Academy of Sciences, and research institution. At this time, it can be further judged based on the keywords that Xiao Ming is a student rather than a researcher, thereby making relevant recommendations. However, when too many parts of speech are set, it may result in too many candidate keywords, thereby affecting the accuracy of the keyword extraction method proposed in this application. Therefore, the part of speech of the candidate words can be set based on actual needs and specific scenarios.

[0041] S130: Inputting the text to be processed into a labeling model to obtain an output result corresponding to the text to be processed, wherein the output result includes a labeling result, and the labeling result represents a keyword distribution of the text to be processed.

[0042] Among them, the text to be processed may include multiple tokens, and each token may correspond to category information. A token may refer to the smallest constituent element of the text to be processed. For example, in the text to be processed of Chinese type, each Chinese character can be understood as a token. Exemplarily, if the text to be processed is: "Xiaoming studies at the University of Chinese Academy of Sciences, the largest national scientific research institution in China", then this text to be processed can be composed of multiple tokens such as "小", "明", "就",..., "机", "构".

[0043] In the embodiments of the present application, the annotation result may include the category information corresponding to each of the multiple tokens. An annotation model refers to a model that can be used to annotate the category information of the tokens included in the text to be processed.

[0044] As a way, multiple tokens can be input into the annotation model to obtain the category information corresponding to each of the multiple tokens. This category information can characterize whether the corresponding token has a first keyword, and in the case where the first keyword exists, the position information of the first keyword.

[0045] Optionally, the category information can be divided into 5 categories based on "IOB" annotation: B, O, I, S, PAD. Among them, B can indicate that the token is a keyword and is at the beginning position of the keyword, I can indicate that the token is a keyword and is in the middle position of the keyword, O can indicate that the token is a non-keyword, S can indicate a single-word keyword (for example, the character "龙" can be represented by S), and PAD can indicate a character that appears outside the sentence.

[0046] S140: Obtain the first keyword based on the annotation result.

[0047] Among them, as a way, if there is category information indicating that the corresponding token has a first keyword, then the corresponding first keyword can be extracted based on the category information indicating the existence of the first keyword. Exemplarily, if the text to be processed is: "Xiaoming studies at the University of Chinese Academy of Sciences, the largest national scientific research institution in China", and the annotation result output by step S130 is: B I O O O B I I I I O O O O O O O O O O B I I I, then the first keywords that can be obtained are: "Xiaoming", "Chinese Academy of Sciences", "scientific research institution".

[0048] S150: Obtain the target keyword based on the word segmentation result and the first keyword.

[0049] The word segmentation result may include multiple candidate words. In this embodiment of the present application, the candidate word may refer to a word that may become the second keyword. As a method, the importance score corresponding to each of the multiple candidate words can be obtained based on the attention weight, and the candidate word whose importance score meets the preset conditions is used as the second keyword, so that the target keyword can be obtained based on the first keyword and the second keyword.

[0050] The output of the annotation model in step S130 may further include an attention weight, which may represent the semantic contribution of each of the multiple word-grams corresponding to the text to be processed to the text to be processed. The semantic contribution may be the extent to which a certain word-gram helps the annotation model correctly understand the semantics of the text to be processed. The importance score may represent the extent to which each of the multiple candidate words included in the word segmentation result helps the annotation model correctly understand the semantics of the text to be processed. The preset condition may be that the candidate words are ranked in the top K (K is a positive integer) positions in importance score.

[0051] Optionally, the attention weights of the multiple word-grams corresponding to the text to be processed can be obtained based on step S130.

[0052] Optionally, the word-grams included in each of the multiple candidate words can be obtained, and the average of the attention weights corresponding to the word-grams can be used as the importance scores corresponding to the multiple candidate words. Assume that a single candidate word and the word-grams included in the candidate word can be represented as: w = {token1, token2, ..., token n}, the calculation formula for the importance score of a single candidate word is as follows:

[0053]

[0054] Among them, att i It can represent the attention weight of a single word, n can represent the total number of words included in the candidate word, att w It can represent the importance score of the candidate word.

[0055] Exemplarily, the text to be processed may be: "Xiao Ming is studying at the University of Chinese Academy of Sciences, China's largest national research institution". The candidate words included in the word segmentation result obtained by step S120 may be: "Xiao Ming", "studied at", "Chinese Academy of Sciences", "university", ",", "China", "largest", "of", "national", "research institution". The attention weights obtained by step S130 may be: {a1, a2, a3, ..., a24}, where a1 is the attention weight corresponding to the word "small", a2 is the attention weight corresponding to the word "Ming", and the importance score corresponding to the candidate word "Xiao Ming" may be: A1 = (a1+a2) / 2.

[0056] As a way, such as Figure 2 As shown, obtaining the target keyword based on the first keyword and the second keyword includes:

[0057] S151: Obtain the number of the first keywords.

[0058] As one approach, the number of first keywords may be determined through the output result of step S140.

[0059] S152: Sort the second keywords from high to low according to the importance scores, and compare them one by one with the importance score threshold.

[0060] The importance score threshold may refer to a value used to measure whether the second keyword can be output as a target keyword.

[0061] For example, assuming that the second keywords are: w1, w2, w3, the importance scores of w1, w2, w3 are: A1, A2, A3, where A1>A3>A2, then the result of sorting the second keywords from high to low according to the importance score is: w1, w3, w2.

[0062] S153: During the current comparison process, the importance score of the second keyword participating in the current comparison is compared with the current importance score threshold. If the importance score of the second keyword participating in the current comparison is greater than the current importance score threshold, the second keyword participating in the current comparison is added as a supplementary keyword, and the next comparison process is entered; otherwise, the comparison is ended; wherein the current importance score threshold is obtained based on the first preset value, the second preset value, the number of the first keywords, and the current number of supplementary keywords, wherein the importance score of the second keyword participating in the comparison in the latter comparison process is ranked after the second keyword participating in the comparison in the previous comparison process.

[0063] The second keyword involved in the comparison in the latter comparison process is adjacent to the second keyword involved in the comparison in the previous comparison process. The calculation formula for the current importance score threshold is as follows:

[0064] H=α-n×β

[0065] Wherein, α may represent an importance reference value, β may represent a penalty coefficient for the number of target keywords, and n may represent the total number of the first keyword and the current supplementary keywords.

[0066] Optionally, the specific values of α and β can be preset based on requirements. Exemplarily, assuming x1 < x2 and y1 < y2, when more supplementary keywords need to be extracted, the value of α can be set to x2 and the value of β can be set to y1, so as to reduce the current importance score threshold to obtain more supplementary keywords; when not too many supplementary keywords need to be extracted, the value of α can be set to x1 and the value of β can be set to y2, so as to increase the current importance score threshold to obtain fewer supplementary keywords.

[0067] Exemplarily, assume the number of the first keywords is 3 and the number of supplementary keywords before the first comparison process is 0. The second keywords and their corresponding importance scores are: {(w1, A1), (w3, A3), (w2, A2)}, where A1 > A3 > A2. Then, in the first comparison process, A1 can be compared with the current score threshold (H1 = α - 3×β). If A1 > H1, w1 can be added as a supplementary keyword. Here, "added as" can be understood as adding a new supplementary keyword to the existing supplementary keywords. Correspondingly, after this comparison, the current supplementary keywords include w1 and the number of supplementary keywords is 1, and then enter the second comparison process. In the second comparison process, A3 corresponding to w3 adjacent to w1 can continue to be compared with the current score threshold (H2 = α - (3 + 1)×β); if A1 < H1, since A1 is the maximum value of the importance scores of the second keywords, when A1 < H1, the importance scores of other second keywords will also be less than H1. Therefore, the importance scores of other second keywords do not need to be compared with the current importance score threshold anymore, and there will be no second keyword that can be added as a supplementary keyword.

[0068] In the embodiments of the present application, through the attention weights and word segmentation results output by the annotation model, additional keywords can be obtained without significantly increasing the computing cost.

[0069] S154: Use the current supplementary keywords and the first keywords as the target keywords.

[0070] Wherein, the current supplementary keywords can be the supplementary keywords obtained after the foregoing comparison process. As a way, the obtained first keywords and supplementary keywords can be used as the target keywords for output.

[0071] Exemplarily, Table 1 shows the target keywords output by the keyword extraction method proposed in the present application.

[0072] Table 1

[0073]

[0074]

[0075] The keyword extraction method provided in this embodiment performs word segmentation processing on the text to be processed after obtaining the text to be processed to obtain a word segmentation result corresponding to the text to be processed, and inputs the text to be processed into a labeling model to obtain an output result of a labeling result corresponding to the text to be processed, including a result representing the keyword distribution of the text to be processed, and obtains a first keyword based on the labeling result; and obtains a target keyword based on the word segmentation result and the first keyword. Through the above method, the labeling result corresponding to the text to be processed can be obtained based on the labeling model, and the first keyword can be obtained based on the labeling result. Then, the target keyword can be obtained based on the word segmentation result corresponding to the text to be processed and the first keyword, so as to obtain sufficient target keywords.

[0076] See also Figure 3 The present application provides a keyword extraction method, which is applied to electronic devices and includes:

[0077] S210: Obtain designated training data and the labeling results corresponding to the designated training data, wherein the labeling results corresponding to the designated training data represent the keyword distribution of the designated training data. S220: Train a to-be-trained labeling model based on the designated training data and the labeling results corresponding to the designated training data to obtain the labeling model.

[0078] Among them, the specified training data may refer to the training data obtained for the keyword extraction method proposed in this application. Specifically, the specified training data may be business data (for example, data corresponding to the search business for e-commerce, etc.), data in an open source data set (such as data in an entity recognition task data set), etc. The specified training data may include multiple training samples, and each of the multiple training samples corresponds to multiple word units. The annotation results corresponding to the specified training data may include annotations corresponding to each of the multiple word units. The annotations may represent the category information of each of the multiple word units. The annotations corresponding to each of the multiple word units may be obtained by manual annotation based on "IOB". The category information corresponding to the annotations may be divided into 5 categories: B, O, I, S, and PAD, wherein B may indicate that the word unit is a keyword and is located at the beginning of the keyword, I may indicate that the word unit is a keyword and is located in the middle of the keyword, O may indicate that the word unit is a non-keyword, S may indicate a single-word keyword (for example, the word "dragon" can be represented by S), and PAD may indicate a word that appears outside a sentence.

[0079] Optionally, the annotation model to be trained may include a pre-trained feature extraction network and a classification network, as a way, such as Figure 4As shown, multiple training samples can be first input into a pre-trained feature extraction network to obtain feature vectors of multiple word units corresponding to each of the multiple training samples; the feature vectors can then be input into a classification network to obtain predicted labels corresponding to each of the multiple word units; thus, the labeling model to be trained can be trained based on the predicted labels and labels to obtain a labeling model.

[0080] The predicted labels corresponding to the multiple tokens refer to the category information of the multiple tokens predicted by the trained labeling model. The pre-trained feature extraction network can be a feature extraction model with an attention mechanism that has been pre-trained based on a large-scale public dataset and has sequence data feature extraction capabilities. Examples include a pre-trained LSTM model with an attention mechanism, a pre-trained model with an attention mechanism, and a pre-trained BERT model, among others. The BERT model itself has an attention mechanism. The classification network may include fully connected layers and activation functions.

[0081] When the pre-trained feature extraction network is a BERT model, for example, consider the input training sample: "Xiao Ming is a student at the University of the Chinese Academy of Sciences, China's largest national research institution." Before feeding this training sample into the BERT model, the sentence start and end markers [CLS] and [SEP] can be added to the training sample. [CLS] can be considered to contain the semantic information of the entire sentence (or multiple sentences), while [SEP] can be used to separate multiple sentences. After adding these markers, the training sample can be divided into tokens using a dictionary and represented. Among them, a single word can be taken as a word unit, and the vector representation of each word unit can be found based on the dictionary, so as to convert the input sequence: "[CLS]Xiao Ming is studying at the University of Chinese Academy of Sciences, China's largest national research institution [SEP]" into a vector sequence: "w[CLS]w1 w2 w3 w4 w5 w6 w7 w8 w9...w[SEP]", so that the vector sequence can be input into the BERT model to obtain the feature vectors of multiple word units corresponding to the training sample; then the feature vectors are input into the classification network to obtain the predicted labels corresponding to the multiple word units; thus, the to-be-trained annotation model can be trained based on the predicted labels and labels, and the model that meets the target training conditions can be used as the annotation model, wherein the target training conditions can refer to the convergence of the to-be-trained annotation model, the accuracy of the predicted labels reaching the target value, or the preset number of training times, etc.

[0082] Among them, optionally, when performing word-to-word operations on the input samples to be trained, the dictionary used can be an open-source, trained dictionary, or a customized initialization dictionary based on the keyword extraction method proposed in this application, and the dictionary and the annotation model to be trained can be trained simultaneously. From the training samples to the output prediction annotation is an end-to-end process. The customized initialization dictionary can set word-unit division methods of different granularities based on actual needs, thereby improving the annotation accuracy of the annotation model.

[0083] Optionally, you can obtain open-source pre-trained BERT models of varying sizes, such as BERT-large, BERT-base, and BERT-small, based on your actual needs and hardware requirements. Different BERT models (BERT-large, BERT-base, and BERT-small) have different model sizes. Larger BERT models (such as BERT-large) achieve better annotation accuracy, but require more computing resources (such as GPUs (Graphics Processing Units)) and dedicated training data for offline training and online inference. For example, if you only plan to provide a single GPU for inference and require a query-per-second (QPS) greater than 10, while using a BERT-large model will yield more accurate predictions and annotations, achieving a QPS greater than 10 on a single GPU is difficult. In this case, BERT-base or BERT-small might be an option.

[0084] In the embodiments of this application, by using a pre-trained BERT model as the feature extraction network, a large number of keywords can be obtained using only a small amount of specified training data, thereby improving the training efficiency of the training annotation model and the annotation accuracy of the annotation model. Furthermore, BERT models of different model sizes can be selected based on actual needs and hardware conditions, increasing the flexibility and applicability of the keyword extraction method proposed in this application.

[0085] Optionally, the annotation model to be trained includes a pre-trained feature extraction network, conditional random field, and classification network, as a way, such as Figure 5 As shown, multiple training samples can be first input into a pre-trained feature extraction network to obtain reference feature vectors of multiple word units corresponding to each of the multiple training samples; the reference feature vectors can be input into a classification network to obtain feature vectors of multiple word units corresponding to each of the multiple training samples; the feature vectors can be input into a conditional random field to obtain predicted labels corresponding to each of the multiple word units; and the labeling model to be trained can be trained based on the predicted labels and the labels to obtain a labeling model.

[0086] In the embodiment of the present application, inputting the feature vector into the conditional random field can further learn the dependency relationship between adjacent elements, further improving the annotation accuracy of the annotation model. For example, when the character "中" appears at the i-th position in the sequence corresponding to the feature vector, the probability that the character "国" appears at position i + 1 is relatively high, and the probability that the character "心" appears is also relatively high, etc. That is, after determining the character at position i, the probabilities of all characters at i + 1 are not equal. The conditional random field can model the dependency relationship between sequence elements, thereby improving the annotation accuracy of the annotation model.

[0087] It should be noted that compared with the to-be-trained annotation model that only includes a pre-trained feature extraction network and a classification network, the to-be-annotated model that includes a pre-trained feature extraction network, a conditional random field, and a classification network has a larger computational amount, thus reducing the output speed of the predicted annotation result. One of the above two to-be-trained annotation models can be selected for training based on actual needs.

[0088] S230: Obtain the text to be processed.

[0089] S240: Perform word segmentation on the text to be processed to obtain the word segmentation result corresponding to the text to be processed.

[0090] S250: Input the text to be processed into the annotation model to obtain the output result corresponding to the text to be processed. The output result includes an annotation result, and the annotation result represents the keyword distribution of the text to be processed.

[0091] S260: Obtain the first keyword based on the annotation result.

[0092] S270: Obtain the target keyword based on the word segmentation result and the first keyword.

[0093] The keyword extraction method provided in this embodiment enables, through the above method, training the to-be-trained annotation model based on specified training data to obtain an annotation model, improving the annotation accuracy of the annotation model, and then obtaining the target keyword based on the word segmentation result and the annotation result corresponding to the text to be processed, so as to obtain sufficient target keywords.

[0094] To better understand the solutions of all embodiments of the present application, the basic process of the keyword extraction method of the present application is introduced below.

[0095] Please refer to Figure 6, the basic business process of the keyword extraction method mainly includes the acquisition of the first keyword and the second keyword. After obtaining the text to be processed, the text to be processed can be input into the annotation model to obtain the annotation results and attention weights corresponding to the text to be processed. At this time, the first keyword of the text to be processed can be obtained based on the annotation results. At the same time, after obtaining the text to be processed, the text to be processed can also be segmented to obtain a segmentation result including multiple candidate words, and then the importance score of each candidate word is obtained based on the segmentation result and the attention weight output by the annotation model, and the candidate words ranked in the top K positions from high to low in importance score are used as the second keyword, and then the importance score of the second keyword is compared with the importance score threshold, and the supplementary keyword is obtained based on the comparison result. Finally, the first keyword and the supplementary keyword are merged and output as the target keyword.

[0096] Among them, the annotation model may include a pre-trained BERT model. Since the keywords of the data set for training the BERT model are basically named entities, for example, the keywords corresponding to a piece of news obtained from the Internet are: "World Cup Qualifiers, Poland_Sports, Play-offs", which makes the BERT model more inclined to obtain named entities in the text. The combination of word segmentation processing and attention weighting can better supplement the more important non-entity nouns or verbs in the text, and can also extract supplementary keywords in short texts. Therefore, in the embodiment of the present application, the target keywords obtained by the word segmentation results and the first keyword can enrich the types of keywords obtained.

[0097] See also Figure 7 The present application provides a keyword extraction device 400, which runs on an electronic device. The device 400 includes:

[0098] The to-be-processed text obtaining unit 410 is configured to obtain the to-be-processed text.

[0099] The word segmentation result obtaining unit 420 is configured to perform word segmentation processing on the text to be processed to obtain a word segmentation result corresponding to the text to be processed.

[0100] The annotation result acquisition unit 430 is configured to input the text to be processed into an annotation model to obtain an output result corresponding to the text to be processed, wherein the output result includes an annotation result, and the annotation result represents the keyword distribution of the text to be processed.

[0101] The first keyword obtaining unit 440 is configured to obtain a first keyword based on the annotation result.

[0102] The target keyword acquisition unit 450 is configured to obtain a target keyword based on the word segmentation result and the first keyword.

[0103] Wherein, the device further comprises:

[0104] The annotation model generation unit 460 is used to obtain specified training data and the annotation results corresponding to the specified training data, where the annotation results corresponding to the specified training data represent the keyword distribution of the specified training data; and train the to-be-trained annotation model based on the specified training data and the annotation results corresponding to the specified training data to obtain the annotation model.

[0105] In one embodiment, the text to be processed includes multiple tokens, and the annotation results include category information corresponding to each of the multiple tokens. The annotation result acquisition unit 430 is specifically configured to input the multiple tokens into the annotation model to obtain category information corresponding to each of the multiple tokens, wherein the category information indicates whether the corresponding token contains a first keyword, and, if the first keyword exists, location information of the first keyword. The first keyword acquisition unit 440 is specifically configured to extract the corresponding first keyword based on the category information indicating the presence of the first keyword in the corresponding token, if category information indicates the presence of the first keyword in the corresponding token.

[0106] As another way, the output result also includes attention weights, the word segmentation result includes multiple candidate words, the attention weights represent the semantic contribution of each of the multiple word elements to the text to be processed, and the target keyword acquisition unit 450 is specifically used to obtain the importance scores corresponding to each of the multiple candidate words based on the attention weights; use the candidate words whose importance scores meet preset conditions as the second keywords; and obtain the target keywords based on the first keyword and the second keyword.

[0107] Optionally, the target keyword acquisition unit 450 is specifically used to obtain word-grams included in each of the multiple candidate words, and use the average of the attention weights corresponding to each of the word-grams as the importance scores corresponding to each of the multiple candidate words.

[0108] Optionally, the target keyword acquisition unit 450 is specifically used to obtain the number of the first keywords; sort the second keywords from high to low according to the importance score, and compare them one by one with the importance score threshold; in the current comparison process, compare the importance score of the second keyword participating in the comparison with the current importance score threshold, if the importance score of the second keyword participating in the comparison is greater than the current importance score threshold, add the second keyword participating in the comparison as a supplementary keyword, and enter the next comparison process, otherwise, end the comparison; wherein the current importance score threshold is obtained based on the first preset value, the second preset value, the number of the first keywords and the current number of supplementary keywords, wherein the importance score of the second keyword participating in the comparison in the latter comparison process is ranked after the second keyword participating in the comparison in the previous comparison process; use the current supplementary keyword and the first keyword as the target keywords.

[0109] As another embodiment, the designated training data includes multiple training samples, each of the multiple training samples corresponds to multiple word-grams, and the labeling results corresponding to the designated training data include labels corresponding to each of the multiple word-grams, and the labels represent the category information of each of the multiple word-grams. The labeling model to be trained includes a pre-trained feature extraction network and a classification network. The labeling model generation unit 460 is specifically used to input the multiple training samples into the pre-trained feature extraction network to obtain feature vectors of the multiple word-grams corresponding to each of the multiple training samples; input the feature vectors into the classification network to obtain predicted labels corresponding to each of the multiple word-grams; and train the labeling model to be trained based on the predicted labels and the labels to obtain the labeling model.

[0110] As another embodiment, the designated training data includes multiple training samples, each of the multiple training samples corresponds to multiple word-grams, and the labeling results corresponding to the designated training data include labels corresponding to each of the multiple word-grams, and the labels represent the category information of each of the multiple word-grams. The labeling model to be trained includes a pre-trained feature extraction network, a conditional random field, and a classification network. The labeling model generation unit 460 is specifically used to input the multiple training samples into the pre-trained feature extraction network to obtain reference feature vectors of the multiple word-grams corresponding to each of the multiple training samples; input the reference feature vectors into the classification network to obtain feature vectors of the multiple word-grams corresponding to each of the multiple training samples; input the feature vectors into the conditional random field to obtain predicted labels corresponding to each of the multiple word-grams; and train the labeling model to be trained based on the predicted labels and the labels to obtain the labeling model.

[0111] The following will be combined Figure 8 An electronic device provided by this application is described.

[0112] See also Figure 8 Based on the above-mentioned keyword extraction method and apparatus, the present application also provides another electronic device 100 that can execute the above-mentioned keyword extraction method. The electronic device 100 includes one or more (only one is shown in the figure) processors 102 and a memory 104 coupled to each other. The memory 104 stores a program that can execute the content of the above-mentioned embodiment, and the processor 102 can execute the program stored in the memory 104.

[0113] The processor 102 may include one or more processing cores. The processor 102 utilizes various interfaces and circuits to connect various components within the electronic device 100. It executes instructions, programs, code sets, or instruction sets stored in the memory 104, and accesses data stored in the memory 104 to perform various functions and process data for the electronic device 100. Optionally, the processor 102 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 102 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the modem handles wireless communications. It is understood that the modem may not be integrated into the processor 102 and may be implemented separately via a communication chip.

[0114] The memory 104 may include a random access memory (RAM) or a read-only memory (ROM). The memory 104 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 104 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data created by the terminal 100 during use (such as a phone book, audio and video data, chat history data), etc.

[0115] Please refer to Figure 9 , which shows a block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable storage medium 800 stores program code, which can be called by a processor to execute the method described in the above method embodiment.

[0116] The computer-readable storage medium 1000 can be an electronic memory such as a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an EPROM, a hard disk, or a ROM. Alternatively, the computer-readable storage medium 800 comprises a non-transitory computer-readable storage medium. The computer-readable storage medium 800 has storage space for program code 810 for executing any of the method steps described above. These program codes can be read from or written to one or more computer program products. The program code 810 can be compressed, for example, in a suitable form.

[0117] In summary, the present application provides a keyword extraction method, device, and electronic device. After obtaining the text to be processed, the text to be processed is segmented to obtain the segmentation result corresponding to the text to be processed, and the text to be processed is input into the annotation model to obtain the output result of the annotation result corresponding to the text to be processed, including the keyword distribution result characterizing the text to be processed, and the first keyword is obtained based on the annotation result; the target keyword is obtained based on the segmentation result and the first keyword. Through the above-mentioned method, the annotation result corresponding to the text to be processed can be obtained based on the annotation model, and the first keyword can be obtained based on the annotation result. Then, the target keyword can be obtained based on the segmentation result corresponding to the text to be processed and the first keyword, so as to obtain sufficient target keywords.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A keyword extraction method, characterized in that: Applied to electronic equipment, the method includes: Acquire a text to be processed, wherein the text to be processed includes a plurality of word units; Performing word segmentation processing on the text to be processed to obtain a word segmentation result corresponding to the text to be processed, wherein the word segmentation result includes a plurality of candidate words; Inputting the text to be processed into a tagging model to obtain an output result corresponding to the text to be processed, the output result including a tagging result, the tagging result representing a keyword distribution of the text to be processed, the tagging result including category information corresponding to each of the multiple word-grams, the category information representing whether the corresponding word-gram contains a first keyword, and, if the first keyword exists, location information of the first keyword, the tagging result also including an attention weight, the attention weight representing a semantic contribution of each of the multiple word-grams to the text to be processed; If the first keyword exists in a word-gram corresponding to the category information representation, extracting the corresponding first keyword based on the category information representing the existence of the first keyword; Obtaining importance scores corresponding to each of the plurality of candidate words based on the attention weights, and selecting the candidate words whose importance scores meet a preset condition as second keywords; Obtaining the number of the first keywords, sorting the second keywords from high to low according to the importance scores, and comparing each of the second keywords with an importance score threshold; During the current comparison process, the importance score of the second keyword participating in the current comparison is compared with the current importance score threshold; if the importance score of the second keyword participating in the current comparison is greater than the current importance score threshold, the second keyword participating in the current comparison is added as a supplementary keyword, and the next comparison process is entered; otherwise, the comparison is terminated; wherein the current importance score threshold is obtained based on a first preset value, a second preset value, the number of the first keywords, and the number of current supplementary keywords, wherein the importance score of the second keyword participating in the comparison in the latter comparison process is ranked after the second keyword participating in the comparison in the previous comparison process; The current supplementary keyword and the first keyword are used as target keywords.

2. The method according to claim 1, characterized in that Obtaining the importance scores corresponding to the plurality of candidate words based on the attention weights includes: The word-grams included in each of the multiple candidate words are obtained, and the average of the attention weights corresponding to each of the word-grams is used as the importance score corresponding to each of the multiple candidate words.

3. The method according to claim 1, characterized in that Before obtaining the text to be processed, the method further includes: Obtaining specified training data and a labeling result corresponding to the specified training data, wherein the labeling result corresponding to the specified training data represents a keyword distribution of the specified training data; The to-be-trained annotation model is trained based on the designated training data and the annotation results corresponding to the designated training data to obtain the annotation model.

4. The method according to claim 3, characterized in that The designated training data includes a plurality of training samples, each of the plurality of training samples corresponds to a plurality of word-grams, the labeling results corresponding to the designated training data include labels corresponding to each of the plurality of word-grams, the labels represent category information of each of the plurality of word-grams, the labeling model to be trained includes a pre-trained feature extraction network and a classification network, and the labeling model to be trained is trained based on the designated training data and the labeling results to obtain the labeling model, including: Inputting the plurality of training samples into the pre-trained feature extraction network to obtain feature vectors of a plurality of word units corresponding to the plurality of training samples; Inputting the feature vector into the classification network to obtain predicted labels corresponding to each of the multiple word-units; The to-be-trained annotation model is trained based on the predicted annotation and the annotation to obtain the annotation model.

5. The method according to claim 4, characterized in that The designated training data includes a plurality of training samples, each of the plurality of training samples corresponds to a plurality of word-grams, the labeling results corresponding to the designated training data include labels corresponding to each of the plurality of word-grams, the labels representing category information of each of the plurality of word-grams, the labeling model to be trained includes a pre-trained feature extraction network, a conditional random field, and a classification network, and the labeling model to be trained is trained based on the designated training data and the labeling results to obtain the labeling model, including: Inputting the plurality of training samples into the pre-trained feature extraction network to obtain reference feature vectors of a plurality of word units corresponding to the plurality of training samples; Inputting the reference feature vector into the classification network to obtain feature vectors of a plurality of word units corresponding to each of the plurality of training samples; Inputting the feature vector into the conditional random field to obtain predicted labels corresponding to each of the multiple word-units; The to-be-trained annotation model is trained based on the predicted annotation and the annotation to obtain the annotation model.

6. A keyword extraction device, characterized in that: The device comprises: A to-be-processed text acquisition unit, configured to acquire a to-be-processed text, wherein the to-be-processed text includes a plurality of word units; A word segmentation result obtaining unit, configured to perform word segmentation processing on the text to be processed to obtain a word segmentation result corresponding to the text to be processed, wherein the word segmentation result includes a plurality of candidate words; a tagging result acquisition unit, configured to input the text to be processed into a tagging model to obtain an output result corresponding to the text to be processed, wherein the output result includes a tagging result, wherein the tagging result represents the keyword distribution of the text to be processed, wherein the tagging result includes category information corresponding to each of the multiple word-grams, wherein the category information represents whether the corresponding word-gram contains a first keyword, and, if the first keyword exists, location information of the first keyword, and wherein the tagging result also includes an attention weight, wherein the attention weight represents the semantic contribution of each of the multiple word-grams to the text to be processed; a first keyword acquiring unit configured to extract the corresponding first keyword based on the category information representing the presence of the first keyword if the first keyword exists in a word-gram corresponding to the category information representation; A target keyword acquisition unit is used to obtain the importance scores corresponding to each of the multiple candidate words based on the attention weight, and use the candidate words whose importance scores meet the preset conditions as the second keywords; based on the number of the first keywords obtained, the second keywords are sorted from high to low according to the importance scores, and compared with the importance score threshold one by one; in the current comparison process, the importance score of the second keyword participating in the comparison is compared with the current importance score threshold, if the importance score of the second keyword participating in the comparison is greater than the current importance score threshold, the second keyword participating in the comparison is added as a supplementary keyword, and enter the next comparison process, otherwise, the comparison is ended; wherein the current importance score threshold is obtained based on the first preset value, the second preset value, the number of the first keywords and the current number of supplementary keywords, wherein the importance score of the second keyword participating in the comparison in the latter comparison process is ranked after the second keyword participating in the comparison in the previous comparison process; the current supplementary keyword and the first keyword are used as the target keywords.

7. An electronic device, characterized in that: including one or more processors and memory; One or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program code, wherein when the program code is run, the method according to any one of claims 1 to 5 is executed.

Images