Keyword extraction method

Abstract

The embodiment of the invention discloses a keyword extraction method, which comprises the following steps of: inputting to-be-extracted power data and a plurality of candidate keywords in the to-be-extracted power data into a preset BERT model to obtain an embedded vector of each candidate keyword and an embedded vector of the to-be-extracted power data, then determining a correlation value between each candidate keyword and other candidate keywords and a correlation value between each candidate keyword and the to-be-extracted power data according to the embedding vector of each candidate keyword and the embedding vector of the to-be-extracted power data, and finally aggregating the two correlation values, according to the method and the device, the importance score of each candidate keyword is obtained, and then the required keyword is extracted, so that the deviation problem that only long keywords can be extracted in the prior art can be effectively solved, and the similarity matching effectiveness problem caused by lack of context information can be effectively solved.

Description

Technical Field

[0001] The present invention relates to the technical field of power systems, and in particular to a keyword extraction method. Background Art

[0002] Keyword extraction is an important task that can automatically extract a set of phrases from a document to provide readers with a concise summary of the core content, and is also beneficial to many downstream tasks such as information retrieval and summarization.

[0003] The current keyword extraction method mainly uses similarity to extract keywords from documents, but this method has two main disadvantages: first, since documents are usually much longer than keywords and contain multiple keywords, in order to reliably measure the similarity between them, this method can only extract longer keywords; second, keywords are extracted without contextual information, which limits the effectiveness of similarity matching. Summary of the invention

[0004] Based on this, it is necessary to propose a keyword extraction method to address the above problems, which can effectively solve the bias problem of only extracting longer keywords in the prior art, as well as the effectiveness problem of similarity matching due to lack of context information.

[0005] To achieve the above object, the present invention provides a keyword extraction method in a first aspect, the method comprising:

[0006] Obtain the power data to be extracted;

[0007] Performing keyword recognition on the power data to be extracted to obtain a plurality of candidate keywords;

[0008] Inputting a plurality of candidate keywords and the power data to be extracted into a preset BERT model to obtain an embedding vector of each candidate keyword and an embedding vector of the power data to be extracted;

[0009] Determine, according to the embedding vector of each candidate keyword and the embedding vector of the power data to be extracted, a correlation value between each candidate keyword and the remaining candidate keywords, and a correlation value between each candidate keyword and the power data to be extracted;

[0010] Determine the importance score of each candidate keyword according to the correlation value between each candidate keyword and the remaining candidate keywords, and the correlation value between each candidate keyword and the power data to be extracted;

[0011] Determine the target keyword based on the importance scores of all candidate keywords.

[0012] Optionally, determining the correlation value between each candidate keyword and other candidate keywords, and the correlation value between each candidate keyword and the power data to be extracted according to the embedding vector of each candidate keyword and the embedding vector of the power data to be extracted, includes:

[0013] Perform a maximum pooling operation on the embedding vector of each candidate keyword to obtain the representation vector of each candidate keyword;

[0014] Performing an average pooling operation on the embedded vector of the power data to be extracted to obtain a representation vector of the power data to be extracted;

[0015] The correlation value between each candidate keyword and the remaining candidate keywords, and the correlation value between each candidate keyword and the power data to be extracted are determined according to the representation vector of each candidate keyword and the representation vector of the power data to be extracted.

[0016] Optionally, determining the correlation value between each candidate keyword and other candidate keywords, and the correlation value between each candidate keyword and the power data to be extracted according to the representation vector of each candidate keyword and the representation vector of the power data to be extracted, includes:

[0017] Get the weight value of each candidate keyword;

[0018] Determine the correlation value between each candidate keyword and the remaining candidate keywords according to the representation vector of each candidate keyword and the representation vectors of the remaining candidate keywords;

[0019] Determine an initial correlation value between each candidate keyword and the power data to be extracted according to the representation vector of each candidate keyword and the representation vector of the power data to be extracted;

[0020] According to the initial correlation value between each candidate keyword and the power data to be extracted, and the weight value of each candidate keyword, the correlation value between each candidate keyword and the power data to be extracted is determined.

[0021] Optionally, determining the correlation value between each candidate keyword and the remaining candidate keywords according to the representation vector of each candidate keyword and the representation vectors of the remaining candidate keywords includes:

[0022] Using the formula Determine a correlation value between each candidate keyword and the remaining candidate keywords;

[0023] in, is the correlation value between the ith candidate keyword and the other candidate keywords, N is the total number of candidate keywords, h piis the representation vector of the i-th candidate keyword, h pj is the representation vector of the jth candidate keyword, T is the transposition operation, λ is the preset hyperparameter, δ i is the denoising factor of the i-th candidate keyword, and Mean() is the averaging operation.

[0024] Optionally, determining the initial correlation value between each candidate keyword and the power data to be extracted according to the representation vector of each candidate keyword and the representation vector of the power data to be extracted includes:

[0025] Using the formula Determining an initial correlation value between each candidate keyword and the power data to be extracted;

[0026] The step of determining the correlation value between each candidate keyword and the power data to be extracted according to the initial correlation value between each candidate keyword and the power data to be extracted and the weight value of each candidate keyword includes:

[0027] Using the formula Determine a correlation value between each candidate keyword and the power data to be extracted;

[0028] in, is the initial correlation value between the ith candidate keyword and the power data to be extracted, h s is the representation vector of the power data to be extracted, h pi is the representation vector of the i-th candidate keyword, |||| 1 is the Manhattan distance operation, is the correlation value between the i-th candidate keyword and the power data to be extracted, ρ i is the weight value of the i-th candidate keyword.

[0029] Optionally, obtaining the weight value of each candidate keyword includes:

[0030] Using the formula Get the weight value of each candidate keyword;

[0031] Among them, ρ i is the weight value of the i-th candidate keyword, softmax() is a regularization operation, and e is a natural constant.

[0032] Optionally, determining the importance score of each candidate keyword according to the correlation value between each candidate keyword and other candidate keywords, and the correlation value between each candidate keyword and the power data to be extracted, includes:

[0033] Using the formula Determine the importance score for each candidate keyword;

[0034] Among them, r i is the importance score of the i-th candidate keyword, is the correlation value between the ith candidate keyword and the other candidate keywords, is the correlation value between the ith candidate keyword and the power data to be extracted.

[0035] Optionally, the embedding vector of each candidate keyword is subjected to a maximum pooling operation to obtain a representation vector of each candidate keyword, including:

[0036] Using the formula h pi =Max-Pooling({h 1,pi ,h 2,pi ,h 3,pi ,...,h k,pi ,...,h K,pi}) Get the representation vector of each candidate keyword;

[0037] The performing an average pooling operation on the embedded vector of the power data to be extracted to obtain a representation vector of the power data to be extracted includes:

[0038] Using the formula h s =Avg-Pooling({h 1,s ,h 2,s ,h 3,s ,...,h m,s ,...,h M,s}) obtaining a representation vector of the power data to be extracted;

[0039] Among them, h s is the representation vector of the ith candidate keyword, Max-Pooling() is the maximum pooling operation, {h 1,pi ,h 2,pi ,h 3,pi ,...,h k,pi ,...,h K,pi} is the embedding vector of the i-th candidate keyword, h k,pi is the kth word related to the ith candidate keyword in the power data to be extracted, h s is the representation vector of the power data to be extracted, Avg-Pooling() is the average pooling operation, {h 1,s ,h 2,s ,h 3,s ,...,h m,s ,...,h M,s} is the embedding vector of the power data to be extracted, h m,sis the mth word of the power data to be extracted.

[0040] Optionally, obtaining the power data to be extracted includes:

[0041] Get raw power data;

[0042] The original power data is sequentially cleaned, deduplicated, missing valued and standardized to obtain the power data to be extracted;

[0043] The keyword recognition is performed on the power data to be extracted to obtain a plurality of candidate keywords, including:

[0044] An unsupervised machine learning model is used to perform keyword recognition on the power data to be extracted to obtain a plurality of candidate keywords.

[0045] Optionally, determining the target keyword according to the importance scores of all candidate keywords includes:

[0046] Get the preset extraction number;

[0047] All candidate keywords are sorted according to their importance scores, and the preset extracted candidate keywords with higher importance scores are used as the target keywords.

[0048] To achieve the above object, the present invention provides a keyword extraction device in a second aspect, the device comprising:

[0049] An acquisition module, used for acquiring power data to be extracted;

[0050] An identification module, used for performing keyword identification on the power data to be extracted to obtain a plurality of candidate keywords;

[0051] An embedding vector extraction module, used for inputting a plurality of candidate keywords and the power data to be extracted into a preset BERT model to obtain an embedding vector of each candidate keyword and an embedding vector of the power data to be extracted;

[0052] A correlation determination module, used to determine the correlation value between each candidate keyword and the remaining candidate keywords, and the correlation value between each candidate keyword and the power data to be extracted according to the embedding vector of each candidate keyword and the embedding vector of the power data to be extracted;

[0053] An importance determination module, used to determine the importance score of each candidate keyword according to the correlation value between each candidate keyword and other candidate keywords, and the correlation value between each candidate keyword and the power data to be extracted;

[0054] The target determination module is used to determine the target keyword according to the importance scores of all candidate keywords.

[0055] To achieve the above-mentioned object, the present invention provides in a third aspect a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the method as described in any one of the first aspects.

[0056] To achieve the above-mentioned purpose, the present invention provides a computer device in a fourth aspect, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the method as described in any one of the first aspects.

[0057] The embodiment of the present invention has the following beneficial effects: the above method obtains the power data to be extracted and performs keyword recognition on the power data to be extracted to obtain multiple candidate keywords, and then inputs the multiple candidate keywords and the power data to be extracted into the preset BERT model to obtain the embedding vector of each candidate keyword and the embedding vector of the power data to be extracted, and then determines the correlation value between each candidate keyword and the remaining candidate keywords, and the correlation value between each candidate keyword and the power data to be extracted according to the embedding vector of each candidate keyword and the embedding vector of the power data to be extracted, and then determines the importance score of each candidate keyword according to the correlation value between each candidate keyword and the remaining candidate keywords, and the correlation value between each candidate keyword and the power data to be extracted, and finally The target keyword is determined according to the importance scores of all candidate keywords; that is: by inputting the power data to be extracted and multiple candidate keywords in the power data to be extracted into the preset BERT model, the embedding vector of each candidate keyword and the embedding vector of the power data to be extracted are obtained, and then the correlation value between each candidate keyword and the remaining candidate keywords, as well as the correlation value between each candidate keyword and the power data to be extracted are determined according to the embedding vector of each candidate keyword and the embedding vector of the power data to be extracted, and finally the two correlation values ​​are aggregated to obtain the importance score of each candidate keyword, and then the required keywords are extracted, which can effectively solve the bias problem of only extracting longer keywords in the prior art and the effectiveness problem of similarity matching due to lack of context information. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0059] in:

[0060] Figure 1 A schematic diagram of a keyword extraction method in an embodiment of the present application;

[0061] Figure 2 A schematic diagram of a keyword extraction device in an embodiment of the present application;

[0062] Figure 3 1 is a diagram of the internal structure of a computer device in some embodiments. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] Keyword extraction is an important task that can automatically extract a set of phrases from a document to provide readers with a concise summary of the core content, and is also beneficial to many downstream tasks such as information retrieval and summarization.

[0065] The current keyword extraction method mainly uses similarity to extract keywords from documents, but this method has two main disadvantages: first, since documents are usually much longer than keywords and contain multiple keywords, in order to reliably measure the similarity between them, this method can only extract longer keywords; second, keywords are extracted without contextual information, which limits the effectiveness of similarity matching.

[0066] In response to the above problems, the present application proposes a keyword extraction method that can effectively solve the bias problem in the prior art that only longer keywords can be extracted, as well as the effectiveness problem of similarity matching due to lack of context information. The specific implementation principle will be described in detail in the following embodiments.

[0067] In a first aspect, the present application provides a keyword extraction method.

[0068] See also Figure 1 , is a schematic diagram of a keyword extraction method in an embodiment of the present application, the method comprising:

[0069] Step 110: Obtain power data to be extracted.

[0070] The power data to be extracted refers to some data related to the power system. Of course, the operator can also replace the power data to be extracted with any other data according to actual needs, and this application does not limit it here.

[0071] In some embodiments, the power data to be extracted may include, but is not limited to, power production data, supply chain data, energy consumption data, weather condition data, energy price data, and the like.

[0072] In some embodiments, before obtaining the power data to be extracted, the original power data may be obtained first, and then the original power data may be preprocessed to obtain standard power data, and finally the standard power data may be used as the power data to be extracted.

[0073] It should be noted that by preprocessing the original power data to obtain the power data to be extracted, some abnormal data in the original power data can be removed to ensure the accuracy of the data.

[0074] Step 120: Perform keyword recognition on the power data to be extracted to obtain a plurality of candidate keywords.

[0075] In some embodiments, an existing machine learning model is used to perform keyword recognition on the power data to be extracted to obtain multiple candidate keywords, that is, a preset prompt template can be set to guide the machine learning model to recognize keywords of the power data to be extracted to obtain multiple candidate keywords.

[0076] Step 130: Input multiple candidate keywords and the power data to be extracted into a preset BERT model to obtain an embedding vector of each candidate keyword and an embedding vector of the power data to be extracted.

[0077] Among them, the preset BERT model here is a pre-trained one for predicting and outputting the embedding vector of each candidate keyword and the embedding vector of the power data to be extracted based on multiple candidate keywords and power data to be extracted.

[0078] As for the method of obtaining the preset BERT model, in some embodiments, a large amount of historical power data to be extracted and the corresponding multiple historical candidate keywords, as well as the embedding vectors of the historical power data to be extracted and the corresponding multiple historical candidate keywords are collected, and then each historical power data to be extracted and the corresponding multiple historical candidate keywords, as well as the embedding vectors of each historical power data to be extracted and the corresponding multiple historical candidate keywords are sequentially input into the initial BERT model for training. After training to a certain extent, a trained preset BERT model is obtained; wherein, the embedding vector of each historical power data to be extracted and the corresponding multiple historical candidate keywords can be used as the true value of the initial BERT model, that is, by comparing the true value with the embedding vector of each historical power data to be extracted and the corresponding multiple historical candidate keywords output during the training process, it can be determined whether the BERT model is well trained and has met the expected requirements.

[0079] Regarding the method of obtaining embedding vectors of a large amount of historical power data to be extracted and the corresponding embedding vectors of multiple historical candidate keywords, in some embodiments, the historical power data to be extracted can be word segmented into multiple words and / or sub-words of words, and then each word and / or sub-word of word is converted into word embedding according to the context information of each word and / or sub-word of word in the historical power data to be extracted, so as to obtain the embedding vector of the historical power data to be extracted; and each word and / or sub-word of word is converted into word embedding according to the context information of each word and / or sub-word of word in the historical power data to be extracted and the semantic relationship between each word and / or sub-word of word and each historical candidate keyword, so as to obtain the embedding vector of each historical candidate keyword.

[0080] It can be understood that the embedding vectors of the historical power data to be extracted and the corresponding embedding vectors of multiple historical candidate keywords obtained in this way are used as training sets for training, so that the preset BERT model obtained by training can capture the contextual information of each word and / or sub-word in the power data to be extracted, and generate a richer and more semantic embedding vector representation. The word embeddings in these embedding vectors not only retain the semantic and contextual relationships between words, but also can help subsequent processing tasks better process text data.

[0081] Of course, the above embodiment is only one of the embodiments provided by the present application. In other embodiments, as long as the BERT model can predict and output the embedding vector of each candidate keyword and the embedding vector of the power data to be extracted based on multiple input candidate keywords and the power data to be extracted, it can be used as the preset BERT model of the present application, and the present application does not impose any restrictions on this.

[0082] Step 140: Determine the correlation value between each candidate keyword and the remaining candidate keywords, and the correlation value between each candidate keyword and the power data to be extracted according to the embedding vector of each candidate keyword and the embedding vector of the power data to be extracted.

[0083] It should be noted that since the word embeddings in the embedding vector of the candidate keywords and the word embeddings in the embedding vector of the power data to be extracted both retain the semantic relationship and contextual relationship between words, in some embodiments, the correlation value between each candidate keyword and the remaining candidate keywords, as well as the correlation value between each candidate keyword and the power data to be extracted can be determined based on the word embeddings in the embedding vector of each candidate keyword and the word embeddings in the embedding vector of the power data to be extracted.

[0084] Step 150: Determine the importance score of each candidate keyword according to the correlation value between each candidate keyword and other candidate keywords, and the correlation value between each candidate keyword and the power data to be extracted.

[0085] In some embodiments, the correlation values ​​between each candidate keyword and the remaining candidate keywords and the correlation values ​​between each candidate keyword and the power data to be extracted can be aggregated to obtain the aggregated result of each candidate keyword, and then the aggregated result of each candidate keyword is used as the importance score of each candidate keyword.

[0086] For example, when the aggregation method is multiplication, the correlation value between each candidate keyword and the remaining candidate keywords and the correlation value between each candidate keyword and the power data to be extracted can be multiplied to obtain the product result of each candidate keyword, and then the product result of each candidate keyword can be used as the importance score of each candidate keyword.

[0087] Step 160: Determine the target keyword according to the importance scores of all candidate keywords.

[0088] Among them, the target keyword refers to the keyword extracted from the power data to be extracted in this application.

[0089] It should be noted that the higher the importance score, the more the candidate keyword meets the keywords required for extracting the power data to be extracted, that is, the target keyword is a keyword with a higher importance score.

[0090] In an embodiment of the present application, the power data to be extracted and multiple candidate keywords in the power data to be extracted are input into a preset BERT model to obtain an embedding vector of each candidate keyword and an embedding vector of the power data to be extracted, and then the correlation value between each candidate keyword and the remaining candidate keywords, as well as the correlation value between each candidate keyword and the power data to be extracted are determined according to the embedding vector of each candidate keyword and the embedding vector of the power data to be extracted. Finally, the two correlation values ​​are aggregated to obtain the importance score of each candidate keyword, and then the required keywords are extracted. This can effectively solve the bias problem in the prior art that only longer keywords can be extracted, as well as the effectiveness problem of similarity matching due to lack of contextual information.

[0091] In addition, this application avoids the bias problems existing in the methods previously studied in this application by comprehensively considering the correlation between candidate keywords and the correlation between candidate keywords and the power data to be extracted, rather than relying solely on the similarity, length or frequency of keywords, thereby ensuring that the extracted keywords are more comprehensive and accurate; by utilizing the embedding vectors generated by the BERT model that contain rich semantic information and contextual information, the effectiveness problem caused by the lack of contextual information can be avoided.

[0092] In a feasible implementation, step 140 in the above embodiment determines the correlation value between each candidate keyword and the remaining candidate keywords, as well as the correlation value between each candidate keyword and the power data to be extracted based on the embedding vector of each candidate keyword and the embedding vector of the power data to be extracted, including: performing a maximum pooling operation on the embedding vector of each candidate keyword to obtain a representation vector of each candidate keyword; performing an average pooling operation on the embedding vector of the power data to be extracted to obtain a representation vector of the power data to be extracted; determining the correlation value between each candidate keyword and the remaining candidate keywords, as well as the correlation value between each candidate keyword and the power data to be extracted based on the representation vector of each candidate keyword and the representation vector of the power data to be extracted.

[0093] It should be noted that the maximum pooling operation is performed on the embedding vector of each candidate keyword, which can capture the most significant features of all word embeddings in the embedding vector of each candidate keyword to obtain the representation vector of the candidate keyword; the average pooling operation is performed on the embedding vector of the power data to be extracted, which can capture the average global features of all word embeddings in the embedding vector of the power data to be extracted to obtain the representation vector of the power data to be extracted.

[0094] In an embodiment of the present application, by introducing maximum pooling operations and average pooling operations to respectively determine the correlation values ​​between each candidate keyword and the remaining candidate keywords, as well as the correlation values ​​between each candidate keyword and the power data to be extracted, it is possible to effectively reduce the dimension and extract key information, improve computing efficiency, and enhance the accuracy of correlation calculation.

[0095] It can be understood that reducing dimension and extracting key information: by performing maximum pooling operation on the embedding vector of each candidate keyword, the most significant features of all word embeddings of the embedding vector of each candidate keyword can be captured, which helps to remove redundant information and reduce the dimension of the vector while retaining the most important information. By performing average pooling operation on the embedding vector of the power data to be extracted, the average global features of all word embeddings of the embedding vector of the power data to be extracted can be captured, which can reflect the average information of the power data to be extracted and help the subsequent correlation calculation; improving computational efficiency and interpretability: the pooling operation simplifies the representation of the vector, making the subsequent correlation calculation more efficient; enhancing the accuracy of correlation calculation: by using the pooled representation vector for correlation calculation, the intrinsic connection between the candidate keywords and the correlation between the candidate keywords and the power data to be extracted can be more accurately reflected. This is because the pooling operation extracts the key information in the vector and reduces the interference of noise and redundancy.

[0096] In addition, the embedding vectors generated by the BERT model contain rich semantic and contextual information, which can be further extracted through pooling operations, so that contextual factors can be better considered when calculating relevance, thereby improving the accuracy of keyword extraction.

[0097] In a feasible implementation, the above-mentioned embodiment determines the correlation value between each candidate keyword and the remaining candidate keywords, as well as the correlation value between each candidate keyword and the power data to be extracted according to the representation vector of each candidate keyword and the representation vector of the power data to be extracted, including: obtaining the weight value of each candidate keyword; determining the correlation value between each candidate keyword and the remaining candidate keywords according to the representation vector of each candidate keyword and the representation vectors of the remaining candidate keywords; determining the initial correlation value between each candidate keyword and the power data to be extracted according to the representation vector of each candidate keyword and the representation vector of the power data to be extracted; determining the correlation value between each candidate keyword and the power data to be extracted according to the initial correlation value between each candidate keyword and the power data to be extracted, and the weight value of each candidate keyword.

[0098] Regarding the method of obtaining the weight value of the candidate keyword, in some embodiments, the weight value of each candidate keyword can be determined according to the position information of each candidate keyword in the power data to be extracted.

[0099] In an embodiment of the present application, by determining the correlation value between each candidate keyword and the remaining candidate keywords based on the representation vector of each candidate keyword and the representation vectors of the remaining candidate keywords, and introducing the weight value of the candidate keyword for optimization, the correlation value between each candidate keyword and the power data to be extracted is determined based on the representation vector of each candidate keyword and the representation vector of the power data to be extracted, which can improve the accuracy of the correlation calculation, enhance the robustness of the model, optimize the efficiency of power data extraction, and support more complex extraction requirements, etc.

[0100] It can be understood that the accuracy of correlation calculation is improved: by considering the weight value of each candidate keyword, the importance of different keywords in the extraction of power data to be extracted can be more accurately reflected. When the correlation calculation is performed with the power data to be extracted or other keywords, the influence of the keywords with higher weights will be correspondingly amplified, thereby improving the accuracy of the overall calculation; the robustness of the model is enhanced: after the introduction of weight values, it can respond to different situations more flexibly. Even if the representation vectors of some candidate keywords are higher in some dimensions, due to their lower weights, these candidate keywords may not dominate the final correlation calculation, thereby avoiding the overall deviation caused by individual outliers and enhancing the robustness of the model; the efficiency of power data extraction is optimized: by pre-determining the weights of candidate keywords and taking them into account in the correlation calculation, the keywords most relevant to the power data to be extracted can be extracted more quickly, which helps to reduce unnecessary calculations and improve the efficiency of data extraction; support more complex extraction requirements: when dealing with complex power data extraction tasks, it is often necessary to consider the combined influence of multiple factors. By introducing weight values, more complex extraction requirements can be supported (for example, different weights can be set for different candidate keywords according to different application scenarios and requirements to achieve more accurate data extraction).

[0101] In a feasible implementation, the method of determining the correlation value between each candidate keyword and the other candidate keywords according to the representation vector of each candidate keyword and the representation vectors of the other candidate keywords in the above embodiment includes:

[0102] Using the formula Determine a correlation value between each candidate keyword and the remaining candidate keywords;

[0103] in, is the correlation value between the ith candidate keyword and the other candidate keywords, N is the total number of candidate keywords, h pi is the representation vector of the i-th candidate keyword, h pj is the representation vector of the jth candidate keyword, T is the transposition operation, λ is the preset hyperparameter, δ i is the denoising factor of the i-th candidate keyword, and Mean() is the averaging operation.

[0104] Among them, the preset hyperparameters can be obtained by the operator based on a large number of experiments, statistics or experience. Of course, they can also be set by the operator according to actual needs, and there is no limitation here.

[0105] In some embodiments, the preset hyperparameter of the present application is an experimental value obtained through a large number of experiments.

[0106] In the embodiments of the present application, a rigorous calculation formula for the correlation value between each candidate keyword and the other candidate keywords is provided from a mathematical perspective. The accuracy of the calculated correlation value can be ensured from the rigor of mathematical logic, and the above calculation formula is preferably shown to provide reference, understanding and calculation for technical personnel; in addition, by adopting the above calculation formula to calculate the correlation value, a more accurate correlation value can be obtained to improve the accuracy of keyword extraction.

[0107] In addition, by introducing a denoising factor, the influence of noise on the correlation calculation can be reduced.

[0108] In a feasible implementation, the method of determining the initial correlation value between each candidate keyword and the power data to be extracted according to the representation vector of each candidate keyword and the representation vector of the power data to be extracted in the above embodiment includes:

[0109] Using the formula Determine an initial correlation value between each candidate keyword and the power data to be extracted;

[0110] In the above embodiment, the correlation value between each candidate keyword and the power data to be extracted is determined according to the initial correlation value between each candidate keyword and the power data to be extracted, and the weight value of each candidate keyword, including:

[0111] Using the formula Determine a correlation value between each candidate keyword and the power data to be extracted;

[0112] in, is the initial correlation value between the ith candidate keyword and the power data to be extracted, h s is the representation vector of the power data to be extracted, h piis the representation vector of the i-th candidate keyword, |||| 1 is the Manhattan distance operation, is the correlation value between the i-th candidate keyword and the power data to be extracted, ρ i is the weight value of the i-th candidate keyword.

[0113] For the Manhattan distance operation, it can be understood that, in some embodiments, assuming h s The coordinates of (x 1 ,y 1 ), h pi The coordinates of (x 2 ,y 2 ), then ||h s -h pi || 1 for|x 2 -x 1 |+|y 2 -y 1 |; where is the absolute value.

[0114] In the embodiments of the present application, a rigorous calculation formula for the initial correlation value and the correlation value between each candidate keyword and the power data to be extracted is provided from a mathematical perspective. The accuracy of the calculated initial correlation value and the correlation value can be ensured from the rigor of mathematical logic, and the above-mentioned calculation formula is preferably shown to provide reference, understanding and calculation for technical personnel; in addition, by adopting the above-mentioned calculation formula to calculate the initial correlation value and the correlation value, a more accurate correlation value can be obtained to improve the accuracy of keyword extraction.

[0115] In addition, by introducing the Manhattan distance to calculate the distance between representation vectors, the similarity or difference between the candidate keywords and the power data to be extracted can be quantitatively represented, thereby obtaining a specific initial correlation value, which is more accurate and reliable than qualitative judgment. By introducing the weight value of each candidate keyword on the basis of the initial correlation value to calculate the final correlation value, the final correlation value can be made more in line with actual needs, thereby improving the accuracy and pertinence of keyword extraction.

[0116] In a feasible implementation, obtaining the weight value of each candidate keyword in the above embodiment includes:

[0117] Using the formula Get the weight value of each candidate keyword;

[0118] Among them, ρ i is the weight value of the i-th candidate keyword, softmax() is a regularization operation, and e is a natural constant.

[0119] It should be noted that the i in the i-th candidate keyword in the present application not only represents the number information of the candidate keyword, but also represents the position information of the i-th candidate keyword in the power data to be extracted.

[0120] In the embodiments of the present application, a rigorous calculation formula for the weight value of each candidate keyword is provided from a mathematical perspective. The accuracy of the calculated weight value can be ensured from the rigor of mathematical logic, and the above calculation formula is preferably shown to provide reference, understanding and calculation for technical personnel; in addition, by adopting the above calculation formula to calculate the weight value, a more accurate weight value can be obtained to improve the accuracy of keyword extraction.

[0121] In addition, by introducing the regularization operation, the weight distribution can be smoothed, that is, in the context of weight calculation, it can help smooth the weight distribution, which means that the weight value will not be too extreme (that is, not too high or too low), thereby helping to maintain the relative balance of weights between candidate keywords; calculating the weight value through mathematical formulas reduces the need for human intervention and subjective judgment, which helps to maintain the objectivity and consistency of weight distribution.

[0122] In a feasible implementation, step 150 in the above embodiment, according to the correlation value between each candidate keyword and other candidate keywords, and the correlation value between each candidate keyword and the power data to be extracted, determines the importance score of each candidate keyword, including:

[0123] Using the formula Determine the importance score for each candidate keyword;

[0124] Among them, r i is the importance score of the i-th candidate keyword, is the correlation value between the ith candidate keyword and the other candidate keywords, is the correlation value between the i-th candidate keyword and the power data to be extracted.

[0125] In the embodiments of the present application, a rigorous calculation formula for the importance score of each candidate keyword is provided from a mathematical perspective. The accuracy of the calculated importance score can be ensured from the rigor of mathematical logic, and the above calculation formula is preferably shown to provide reference, understanding and calculation for technical personnel; in addition, by adopting the above calculation formula to calculate the importance score, a more accurate importance score can be obtained to improve the accuracy of keyword extraction.

[0126] In addition, by simultaneously considering the internal correlation between candidate keywords and the external correlation between candidate keywords and the power data to be extracted, the importance of each candidate keyword in the power data to be extracted can be evaluated more accurately. This method helps to reduce the risk of misextraction or omission of important keywords. Since multiple correlation factors are taken into account, this method has a certain tolerance for slight changes or noise in the data, which also improves the robustness of the algorithm.

[0127] In a feasible implementation, the embedding vector of each candidate keyword in the above embodiment is subjected to a maximum pooling operation to obtain a representation vector of each candidate keyword, including:

[0128] Using the formula h pi =Max-Pooling({h 1,pi ,h 2,pi ,h 3,pi ,…,h k,pi ,…,h K,pi}) Get the representation vector of each candidate keyword;

[0129] In the above embodiment, the embedding vector of the power data to be extracted is averaged and pooled to obtain a representation vector of the power data to be extracted, including:

[0130] Using the formula h s =Avg-Pooling({h 1,s ,h 2,s ,h 3,s ,…,h m,s ,...,h M,s}) obtain the representation vector of the power data to be extracted;

[0131] Among them, h s is the representation vector of the ith candidate keyword, Max-Pooling() is the maximum pooling operation, {h 1,pi ,h 2,pi ,h 3,pi ,...,h k,pi ,…,h K,pi} is the embedding vector of the i-th candidate keyword, h k,pi is the kth word related to the ith candidate keyword in the power data to be extracted, h s is the representation vector of the power data to be extracted, Avg-Pooling() is the average pooling operation, {h 1,s ,h 2,s ,h 3,s ,…,h m,s ,…,h M,s} is the embedding vector of the power data to be extracted, h m,s is the mth word of the power data to be extracted.

[0132] In the embodiments of the present application, rigorous calculation formulas for the representation vectors of each candidate keyword and the representation vectors of the power data to be extracted are provided from a mathematical perspective. The accuracy of the calculated representation vectors can be ensured from the rigor of mathematical logic, and the above-mentioned calculation formulas are preferably shown to provide reference, understanding and calculation for technical personnel; in addition, by adopting the above-mentioned calculation formulas to calculate the representation vectors, a more accurate representation vector can be obtained to improve the accuracy of keyword extraction.

[0133] In addition, the maximum pooling operation can retain the features most relevant to the candidate keywords, thereby better representing their importance in the power data to be extracted; the average pooling operation can obtain the average global features in the power data to be extracted, thereby effectively capturing the overall semantics of the entire power data to be extracted.

[0134] In a feasible implementation, step 110 in the above embodiment, obtaining the power data to be extracted, includes: obtaining the original power data; performing cleaning, deduplication, missing value processing and standardization processing on the original power data in sequence to obtain the power data to be extracted.

[0135] In step 120 of the above embodiment, keyword recognition is performed on the power data to be extracted to obtain multiple candidate keywords, including: using an unsupervised machine learning model to perform keyword recognition on the power data to be extracted to obtain multiple candidate keywords.

[0136] It should be noted that cleaning processing refers to removing noise, outliers, irrelevant information, etc.; deduplication processing refers to deleting duplicate data; missing value processing refers to filling or deleting data containing missing values; standardization processing refers to converting data into a unified format or range.

[0137] In an embodiment of the present application, by sequentially performing cleaning, deduplication, missing value processing and standardization processing on the original power data to obtain the power data to be extracted, the quality of the power data to be extracted can be significantly improved, and the accuracy and consistency of the power data to be extracted can be ensured; by using an unsupervised machine learning model for keyword recognition to obtain multiple candidate keywords, that is, the unsupervised machine learning model can automatically process a large amount of data and quickly generate a list of candidate keywords, which greatly improves the efficiency and automation of candidate keyword recognition.

[0138] In addition, since unsupervised machine learning models do not require manually labeled data for training, they can more flexibly adapt to power data in different fields and formats, reducing dependence on domain expert knowledge and manual intervention.

[0139] In a feasible implementation, step 160 in the above embodiment determines the target keyword based on the importance scores of all candidate keywords, including: obtaining a preset extraction number; sorting all candidate keywords according to their importance scores, and taking the preset extraction number of candidate keywords with higher importance scores as the target keyword.

[0140] Among them, the preset extraction number can be set by the operator according to actual needs and is not limited here.

[0141] For example, when the preset extraction number is 5, after all candidate keywords are sorted according to their importance scores, the top 5 candidate keywords with the highest importance scores can be used as target keywords to be extracted.

[0142] In an embodiment of the present application, by presetting the extraction number, the number of target keywords to be extracted can be clearly specified, making the entire process more controllable, which helps to meet the specific requirements of different application scenarios for the number of keywords; by sorting the candidate keywords according to the importance score, it can be ensured that the selected target keywords have high representativeness and importance in the power data. This method is more efficient and accurate than random selection or methods based on other non-quantitative standards.

[0143] In a second aspect, the present application provides a keyword extraction device.

[0144] See also Figure 2 , is a schematic diagram of a keyword extraction device in an embodiment of the present application, the device 210 includes:

[0145] An acquisition module 211 is used to acquire power data to be extracted;

[0146] An identification module 212 is used to perform keyword identification on the power data to be extracted to obtain a plurality of candidate keywords;

[0147] The embedding vector extraction module 213 is used to input multiple candidate keywords and the power data to be extracted into a preset BERT model to obtain the embedding vector of each candidate keyword and the embedding vector of the power data to be extracted;

[0148] A correlation determination module 214, configured to determine a correlation value between each candidate keyword and other candidate keywords, and a correlation value between each candidate keyword and the power data to be extracted, based on the embedding vector of each candidate keyword and the embedding vector of the power data to be extracted;

[0149] The importance determination module 215 is used to determine the importance score of each candidate keyword according to the correlation value between each candidate keyword and other candidate keywords, and the correlation value between each candidate keyword and the power data to be extracted;

[0150] The target determination module 216 is used to determine the target keyword according to the importance scores of all candidate keywords.

[0151] In the embodiment of the present application, the relevant contents of the acquisition module 211, the identification module 212, the embedding vector extraction module 213, the correlation determination module 214, the importance determination module 215 and the target determination module 216 can be referred to. Figure 1 The contents in the illustrated embodiment will not be described in detail here.

[0152] It should be noted that the device 210 of the present application also includes some other modules. It can be understood that the method of the present application and the device 210 have a one-to-one correspondence. Therefore, some other modules of the device 210 of the present application are the contents corresponding to the method of the present application in the above-mentioned embodiment.

[0153] In an embodiment of the present application, the power data to be extracted and multiple candidate keywords in the power data to be extracted are input into a preset BERT model to obtain an embedding vector of each candidate keyword and an embedding vector of the power data to be extracted, and then the correlation value between each candidate keyword and the remaining candidate keywords, as well as the correlation value between each candidate keyword and the power data to be extracted are determined according to the embedding vector of each candidate keyword and the embedding vector of the power data to be extracted. Finally, the two correlation values ​​are aggregated to obtain the importance score of each candidate keyword, and then the required keywords are extracted. This can effectively solve the bias problem in the prior art that only longer keywords can be extracted, as well as the effectiveness problem of similarity matching due to lack of contextual information.

[0154] In addition, the present application avoids the bias problems existing in the methods previously studied in the present application by comprehensively considering the correlation between candidate keywords and the correlation between candidate keywords and the power data to be extracted, rather than relying solely on the similarity, length or frequency of keywords, thereby ensuring that the extracted keywords are more comprehensive and accurate; by utilizing the embedding vectors generated by the BERT model that contain rich semantic information and contextual information, the effectiveness problem caused by the lack of contextual information can be avoided.

[0155] In a third aspect, the present application further provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes a keyword extraction method in the above method embodiment.

[0156] In a fourth aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes a keyword extraction method in the above method embodiment.

[0157] Figure 3 The internal structure diagram of a computer device in some embodiments is shown. The computer device may be a terminal, a server, or a gateway. Figure 3 As shown, the computer device includes a processor, a memory and a network interface connected through a system bus.

[0158] The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement each step in the above method embodiment. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can implement each step in the above method embodiment. Those skilled in the art can understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0159] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods.

[0160] Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0161] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0162] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A keyword extraction method, characterized in that: The method comprises: Obtain the power data to be extracted; Performing keyword recognition on the power data to be extracted to obtain a plurality of candidate keywords; Inputting a plurality of candidate keywords and the power data to be extracted into a preset BERT model to obtain an embedding vector of each candidate keyword and an embedding vector of the power data to be extracted; Determine, according to the embedding vector of each candidate keyword and the embedding vector of the power data to be extracted, a correlation value between each candidate keyword and the remaining candidate keywords, and a correlation value between each candidate keyword and the power data to be extracted; Determine the importance score of each candidate keyword according to the correlation value between each candidate keyword and the remaining candidate keywords, and the correlation value between each candidate keyword and the power data to be extracted; Determine the target keyword based on the importance scores of all candidate keywords.

2. The method according to claim 1, characterized in that The step of determining the correlation value between each candidate keyword and other candidate keywords, and the correlation value between each candidate keyword and the power data to be extracted according to the embedding vector of each candidate keyword and the embedding vector of the power data to be extracted, comprises: Perform a maximum pooling operation on the embedding vector of each candidate keyword to obtain the representation vector of each candidate keyword; Performing an average pooling operation on the embedded vector of the power data to be extracted to obtain a representation vector of the power data to be extracted; The correlation value between each candidate keyword and the remaining candidate keywords, and the correlation value between each candidate keyword and the power data to be extracted are determined according to the representation vector of each candidate keyword and the representation vector of the power data to be extracted.

3. The method according to claim 2, characterized in that The step of determining the correlation value between each candidate keyword and other candidate keywords, and the correlation value between each candidate keyword and the power data to be extracted according to the representation vector of each candidate keyword and the representation vector of the power data to be extracted, comprises: Get the weight value of each candidate keyword; Determine the correlation value between each candidate keyword and the remaining candidate keywords according to the representation vector of each candidate keyword and the representation vectors of the remaining candidate keywords; Determine an initial correlation value between each candidate keyword and the power data to be extracted according to the representation vector of each candidate keyword and the representation vector of the power data to be extracted; According to the initial correlation value between each candidate keyword and the power data to be extracted, and the weight value of each candidate keyword, the correlation value between each candidate keyword and the power data to be extracted is determined.

4. The method according to claim 3, characterized in that The step of determining the correlation value between each candidate keyword and the remaining candidate keywords according to the representation vector of each candidate keyword and the representation vectors of the remaining candidate keywords includes: Using the formula Determine a correlation value between each candidate keyword and the remaining candidate keywords; in, is the correlation value between the ith candidate keyword and the other candidate keywords, N is the total number of candidate keywords, h pi is the representation vector of the i-th candidate keyword, h pj is the representation vector of the jth candidate keyword, T is the transposition operation, λ is the preset hyperparameter, δ i is the denoising factor of the i-th candidate keyword, and Mean() is the averaging operation.

5. The method according to claim 3, characterized in that: The step of determining the initial correlation value between each candidate keyword and the power data to be extracted according to the representation vector of each candidate keyword and the representation vector of the power data to be extracted comprises: Using the formula Determining an initial correlation value between each candidate keyword and the power data to be extracted; The step of determining the correlation value between each candidate keyword and the power data to be extracted according to the initial correlation value between each candidate keyword and the power data to be extracted and the weight value of each candidate keyword includes: Using the formula Determine a correlation value between each candidate keyword and the power data to be extracted; in, is the initial correlation value between the ith candidate keyword and the power data to be extracted, h s is the representation vector of the power data to be extracted, h pi is the representation vector of the i-th candidate keyword, || ||1 is the Manhattan distance operation, is the correlation value between the i-th candidate keyword and the power data to be extracted, ρ i is the weight value of the i-th candidate keyword.

6. The method according to claim 3, characterized in that The step of obtaining the weight value of each candidate keyword includes: Using the formula Get the weight value of each candidate keyword; Among them, ρ i is the weight value of the i-th candidate keyword, softmax() is a regularization operation, and e is a natural constant.

7. The method according to claim 1, characterized in that Determining the importance score of each candidate keyword according to the correlation value between each candidate keyword and other candidate keywords, and the correlation value between each candidate keyword and the power data to be extracted, includes: Using the formula Determine the importance score for each candidate keyword; Among them, r i is the importance score of the i-th candidate keyword, is the correlation value between the ith candidate keyword and the other candidate keywords, is the correlation value between the ith candidate keyword and the power data to be extracted.

8. The method according to claim 2, characterized in that: The embedding vector of each candidate keyword is subjected to a maximum pooling operation to obtain a representation vector of each candidate keyword, including: Using the formula h pi =Max-Pooling({h 1,pi ,h 2,pi ,h 3,pi ,...,h k,pi ,...,h K,pi }) Get the representation vector of each candidate keyword; The performing an average pooling operation on the embedded vector of the power data to be extracted to obtain a representation vector of the power data to be extracted includes: Using the formula h s =Avg-Pooling({h 1,s ,h 2,s ,h 3,s ,...,h m,s ,...,h M,s }) obtaining a representation vector of the power data to be extracted; Among them, h s is the representation vector of the ith candidate keyword, Max-Pooling() is the maximum pooling operation, {h 1,pi ,h 2,pi ,h 3,pi ,...,h k,pi ,...,h K,pi } is the embedding vector of the i-th candidate keyword, h k,pi is the kth word related to the ith candidate keyword in the power data to be extracted, h s is the representation vector of the power data to be extracted, Avg-Pooling() is the average pooling operation, {h 1,s ,h 2,s ,h 3,s ,...,h m,s ,...,h M,s } is the embedding vector of the power data to be extracted, h m,s is the mth word of the power data to be extracted.

9. The method according to claim 1, characterized in that: The obtaining of the power data to be extracted comprises: Get raw power data; The original power data is sequentially cleaned, deduplicated, missing valued and standardized to obtain the power data to be extracted; The keyword recognition is performed on the power data to be extracted to obtain a plurality of candidate keywords, including: An unsupervised machine learning model is used to perform keyword recognition on the power data to be extracted to obtain a plurality of candidate keywords.

10. The method according to claim 1, characterized in that Determining the target keyword according to the importance scores of all candidate keywords includes: Get the preset extraction number; All candidate keywords are sorted according to their importance scores, and the preset extracted candidate keywords with higher importance scores are used as the target keywords.

Images