A Method for Constructing an Electricity Keyword Dictionary Based on Attention Mechanism

Abstract

This invention discloses a method for constructing an electricity keyword dictionary based on an attention mechanism. Guided by a classification model, a corresponding keyword evaluation neural network model is designed to obtain the attention weights of each keyword and label. This method filters out keywords that are useless to the classification results while minimizing text classification error, selecting a set of relatively important keywords. This accelerates the training speed of the text classification model, optimizes the training process, and improves the accuracy of the classification model. By extracting the most representative and informative keywords, this method achieves significant advantages in dimensionality reduction, noise reduction, interpretability, and model training efficiency.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing text classification technology, and specifically relates to a method for constructing an electricity keyword dictionary based on an attention mechanism. Background Technology

[0002] Text classification, a key task in natural language processing, aims to categorize text data into predefined classes. Feature selection is a crucial step in building text classification models, and keyword selection, as an important method, plays a key role in improving model performance and interpretability. With the advent of the information explosion era, raw power industry text data is often massive and complex, containing a large amount of redundant information and noise. Power industry keyword dictionaries built from this raw data, or those from Baidu's open-source power industry keyword dictionary, also contain many keywords that are meaningless for power industry text classification. Therefore, before inputting these keywords into the classification model, a clever and effective keyword selection process is needed to extract the most representative and informative keywords, thereby achieving significant advantages in dimensionality reduction, noise reduction, interpretability, and model training efficiency.

[0003] Keyword selection in the power industry is a keyword selection problem within text classification. The most commonly used existing method is the Term Frequency-Inverse Document Frequency (TF-IDF) method. Its main idea is that if a word appears frequently in one document but rarely in others, it is considered to represent the meaning of the current document well. This method primarily focuses on the frequency of words within a document, ignoring the semantic relationships between words. Similar words may have different TF-IDF weights, leading to inaccurate reflection of word importance in some cases. Other methods for selecting power industry keywords include Mutual Information, Chi-Square Test, and Information Gain, but these methods cannot accurately and effectively obtain the relative importance relationship between keywords and classification results.

[0004] The patent "A Semi-Supervised Learning Method for Constructing a Big Data Keyword Dictionary" (Publication No.: CN115270774A) describes a keyword extraction model based on the BERT model, BiLSTM network, and CRF network. This model extracts keyword fragments from text corpora, filters out keywords, and adds them to the keyword dictionary to obtain an expanded keyword dictionary. However, this method relies on existing deep learning methods for keyword extraction and cannot better adapt to the feature extraction process using classification models. Furthermore, it does not simplify the keyword dictionary to improve model training speed. Summary of the Invention

[0005] To overcome the shortcomings of the prior art, the present invention aims to provide a method for constructing an electricity keyword dictionary based on an attention mechanism. This method can better adapt to the process of extracting keyword features by the classification model, and filter out keywords that are useless to the classification results as much as possible while minimizing the text classification error, thereby selecting a set of keywords with relatively high importance, thus accelerating the training speed of the text classification model and optimizing the training process.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] In a first aspect, the present invention provides a method for constructing an electricity keyword dictionary based on an attention mechanism, comprising the following steps:

[0008] S1, based on the initial electricity keyword dictionary A, encode each keyword into an m-dimensional vector, where m is the number of keywords in A;

[0009] S2, obtain the power text dataset B and its label dataset Y;

[0010] S3. Based on the encoding of keywords in A, encode the words in text B that appear in A to obtain the text vector dataset E corresponding to B;

[0011] S4, construct a keyword evaluation neural network model, which includes an input layer, a fully connected layer, an attention layer, and a classification model layer. The input layer is used to input text vectors and corresponding labels. The fully connected layer includes two parallel and identically structured fully connected neural networks. The inputs of the two fully connected neural networks are respectively connected to the outputs of the input layer, and two feature vectors are obtained using activation function f1. The attention layer sums the two feature vectors and uses activation function f2 to generate attention weights corresponding to keywords and labels. Each keyword in A is weighted based on its corresponding attention weight to generate a keyword weighted set D. The classification model layer is used for text classification.

[0012] S5, divide E into training set and test set;

[0013] S6, input the training set into the keyword evaluation neural network model, use the attention weights to calculate the importance of each keyword relative to the label, and generate a set of relative importance;

[0014] S7, sort the keywords in A from highest to lowest relative importance, and select the keywords with high relative importance to generate q keyword combinations;

[0015] S8. Use q keyword combinations to re-encode B to obtain new text vector datasets, input them into classification models to obtain the output results of q classification models, and select the keyword combination with the highest classification accuracy as the reconstructed simplified keyword dictionary.

[0016] Furthermore, the initial power keyword dictionary A is obtained through one of the following methods:

[0017] Method 1: Preprocess the Chinese power text by performing text segmentation and stop word removal, deleting duplicate or irrelevant data, and selectively filling in incomplete data information to transform the Chinese power text into a structured data form that can be recognized and processed by computers, thus serving as a dictionary of power keywords.

[0018] Method 2: Use an open-source electricity keyword dictionary.

[0019] Furthermore, in S1, each keyword is encoded into an m-dimensional vector using one-hot encoding.

[0020] Furthermore, in S4, the two feature vectors Y1 1×(m+1) and Y2 1×(m+1) The calculation formula is as follows:

[0021] Y1 1×(m+1) =f1(W1) dense *Y 1×(m+1) +b1 dense )

[0022] Y2 1×(m+1) =f1(W2) dense *Y 1×(m+1) +b2 dense )

[0023] Among them, W1 dense W2 dense Y represents the weight parameters of two fully connected neural networks. 1×(m+1) b1 is the output of the input layer. dense b2 dense These represent the offset parameters of the two fully connected neural networks.

[0024] Furthermore, in step S7, the keywords in A are sorted from highest to lowest relative importance, and the sorted keyword set is {a}. r1 ,a r2 ,...,a rl ,...a rm}, where a rlLet l represent the keyword ranked l in relative importance. The process of selecting keywords with high relative importance involves: selecting keywords q times from the sorted keyword set. In the p-th selection, the top mp features are taken, where p is an integer greater than or equal to 1, and q ≥ p ≥ 1. The final q combinations of keywords are:

[0025] {a r1 ,a r2 ,...a rm-1},{a r1 ,a r2 ,...a rm-2},...,{a r1 ,a r2 ,...a rm-q}

[0026] Furthermore, S8 is expressed using MAE, RMSE, or R. 2 The accuracy of the indicator evaluation classification model is calculated using the following formulas:

[0027]

[0028]

[0029]

[0030] Among them, P act,i This represents the actual value of the label. P represents the average value of the actual labels. pre,i represents the predicted value of the label, and n represents the number of samples; based on one of the above indicators, the feature combination with the highest classification accuracy is selected as the reconstructed keyword dictionary.

[0031] In another aspect, the present invention provides a method for classifying power text, which preprocesses the original power text data and then uses the keyword dictionary and text classification model reconstructed by the present invention to achieve text classification.

[0032] Compared with existing technologies, this invention proposes a method for constructing an electricity keyword dictionary based on an attention mechanism. A corresponding keyword evaluation neural network model is designed, including an input layer, a fully connected layer, an attention layer, and a classification model layer. This model can not only classify electricity text using electricity keywords but also obtain the attention weights corresponding to each electricity keyword and label. This combines keyword evaluation with the classification model to form a hybrid model that can automatically learn abstract features from electricity keyword data without the need for manual feature engineering. Given that electricity text data often involves a large amount of technical terminology, industry standards, and technical content, the model designed in this invention can automatically discover complex relationships and patterns between keywords, better capturing the semantic information of the text. This aims to provide theoretical support and practical guidance for building more accurate and efficient text classification models. Attached Figure Description

[0033] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a flowchart illustrating a method for constructing a power keyword dictionary as described in this invention.

[0035] Figure 2 This is a feature evaluation neural network model diagram of the power keyword dictionary construction method described in this invention.

[0036] Figure 3 These are the accuracy, recall, and F1 score of the Fast Text, BERT, and TextCNN models in various categories in the embodiments of this invention.

[0037] Figure 4 This invention compares the Fast Text, BERT, and TextCNN models using the initial keyword dictionary and the simplified keyword dictionary Micro-F1 in this embodiment. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0039] Raw power text data is often large and complex, containing a lot of redundant information and noise. Existing keyword selection and keyword dictionary construction methods are data-driven, and the resulting keyword dictionary is not suitable for the classification model used. This leads to a difference between the actual classification information and the predicted classification results, reducing the accuracy of power text classification and the model training speed.

[0040] like Figure 1 The diagram shown is a flowchart illustrating a method for constructing an electricity keyword dictionary based on an attention mechanism according to the present invention, which may include:

[0041] S101, based on the initial power text keyword dictionary A, encode each keyword into an m-dimensional vector, where m is the number of keywords in A.

[0042] In practical applications, several methods are provided for obtaining the initial electricity text keyword dictionary A, such as:

[0043] Method 1: Preprocess the Chinese power text by performing text segmentation and stop word removal, deleting duplicate or irrelevant data, and selectively filling in incomplete data information to transform the Chinese power text into a structured data form that can be recognized and processed by computers, thus serving as a keyword dictionary for power text.

[0044] Method 2: Use an open-source electricity keyword dictionary.

[0045] S102, Obtain the power text dataset B and its label dataset Y;

[0046] S103. Based on the encoding of keywords in A, encode the words in text B that appear in A to obtain the corresponding text vector dataset E;

[0047] S104, Construct a keyword evaluation neural network model, such as Figure 2As shown, the model comprises an input layer, a fully connected layer, an attention layer, and a classification model layer. The input layer is used to input the text vector and label corresponding to a text. The fully connected layer includes two parallel and identically structured fully connected neural networks. The inputs of the two fully connected neural networks are respectively connected to the outputs of the input layer, and two feature vectors are obtained using the activation function f1. The attention layer sums the two feature vectors and uses the activation function f2 to generate attention weights corresponding to keywords and labels. A is then weighted based on the attention weights to generate a keyword weighted set D. The classification model layer is used for text classification.

[0048] S105, divide E into training set and test set;

[0049] S106, Input the training set into the keyword evaluation neural network model, use the attention weights to calculate the importance of each keyword relative to the label, and generate a set of relative importance;

[0050] In practical applications, using a keyword evaluation neural network model, attention layers can be used to output the attention weights corresponding to each keyword and tag during keyword selection. Furthermore, the keyword evaluation neural network model can be trained based on a complete keyword evaluation neural network model structure.

[0051] S107, sort the keywords in A from highest to lowest relative importance, and select keywords with high relative importance to generate q keyword combinations; the selection of keywords with high relative importance is as follows: select keywords q times from the sorted keyword set, and when selecting for the p-th time, take out the first mp features, where p is an integer greater than or equal to 1, and q≥p≥1.

[0052] In practical applications, the specific values ​​of p and q can be selected according to actual needs, and this application does not impose any restrictions.

[0053] S108: Use q keyword combinations to re-encode B to obtain new text vector datasets, input them into classification models to obtain the output results of q classification models, and select the keyword combination with the highest classification accuracy as the reconstructed simplified keyword dictionary.

[0054] As a preferred embodiment of the attention mechanism-based method for constructing an electricity keyword dictionary according to the present invention, it may include the following steps:

[0055] S201, Obtain the initial electricity text keyword dictionary A = {a1, a2, ..., a i ,…,a m}, where a iLet \(k_i\) be the \(i\)-th keyword, where \(i\) represents the position index of the keyword and \(m\) represents the number of keywords. Encoding technology is adopted to encode each keyword into an \(m\)-dimensional vector.

[0056] In the embodiments of the present invention, several methods for obtaining the initial power text keyword dictionary \(A\) are provided, as follows:

[0057] Method 1: Preprocess the Chinese power text. By performing text tokenization and stop word removal operations, delete duplicate or power-field-irrelevant data, and specifically fill in incomplete data information, so as to convert the Chinese power text into a structured data form that can be recognized and processed by a computer, and thus construct a power keyword dictionary.

[0058] Method 2: Use an open-source power keyword dictionary.

[0059] In this embodiment, Method 1 is adopted. According to the directory of the core electrical engineering category of Peking University, obtain the public summary information of power system patents, journal papers, etc. for 3 - 5 years. A total of 21,352 pieces of power Chinese journal text information are obtained. For each piece of information, select the article name, article abstract, and article keywords as the components of a text record. Since the obtained text has a high degree of fit with the power industry, the following operations are used to preliminarily process the text:剔除 and power-field-irrelevant data, delete duplicates, and完善 incomplete information through auxiliary means such as manual search, and convert it into structured data that can be processed on a large scale. After cleaning, 14,699 pieces of effective Chinese power text are obtained.

[0060] Adopt the basic Chinese word segmentation model component ECWS for the power field. The goal of this component is to build the basic natural language processing ability in the power field, and accurately segment power text into phrases according to the installation packages torch and AllenNLP. In the actual use process, adopt the web api call method of the model component to segment the power text into individual phrases and return them in the form of a dictionary. For example, segment the phrase "Soft-switching type high-efficiency photovoltaic grid-connected micro-inverter applicable to microgrids", and the result is: applicable / to / microgrid / of / soft-switching type / high-efficiency photovoltaic grid-connected / micro-inverter.

[0061] It should be noted that there is an unclear symbol "剔除" in the original text. I have left it as it is for now. You may need to clarify it for a more accurate translation. Also, the description "完善 incomplete information" is a bit odd in the context, and you might want to check if there are more accurate expressions.However, the segmented phrases still contain many meaningless words. By removing duplicate words from the segmentation results and stop word list, these phrases were removed, resulting in 40,332 phrases with individual meanings. These phrases are numerous and diverse, including not only phrases closely related to the power industry such as "step-down transformer," "recurrent neural network," and "electromagnetic bearing," but also many commonly used phrases in Chinese expression, such as "system," "method," "impact," and "internal." The latter type of phrases is not only numerous but also less helpful for mining power industry keywords. This invention will remove stop words from the segmentation results based on Baidu's stop word list, and manually screen, add, and delete phrases from the segmentation results, selecting phrases with a frequency greater than 10. These phrases will be combined with keywords that can represent the central idea of ​​the article and have strong summarization, thus serving as the source of phrases for constructing a keyword dictionary for the power industry.

[0062] In this embodiment, after the above processing, m = 4196 is obtained. The encoding method adopts one-hot encoding, encoding each keyword into a 4196-dimensional vector. The resulting initial power text keyword dictionary A = {a1, a2, ..., a...} i ,…,a 4196 This dictionary is a series of words related to the power grid, ordered alphabetically by their initials. When classifying words, this dictionary can be used to find the words you need more accurately.

[0063] S202, Obtain the power text dataset B = {b1,b2,...,b k ,...,b L}, b k Let L represent the number of texts, where L represents the k-th text. We set category labels for the electricity text data to construct the corresponding label dataset Y = {y1, y2, ..., y...}. k ,...,y L}, where y k This represents the k-th text b. k The corresponding tags.

[0064] In this embodiment of the invention, after processing in S201, 14699 valid Chinese power-related texts are obtained. Therefore, L = 14699, and B = {b1, b2, ..., b}. k ,...,b 14699}, Y = {y1, y2, ..., y k ,...,y 14699}, y k It is a 26-dimensional vector representing 26 fields of power research.

[0065] S203, based on the keyword encoding in the power text keyword dictionary A, process each text in the power text dataset B separately. Specifically, encode the words that appear in the power text keyword dictionary A; skip words that do not appear. Concatenate the words according to the original text word order to form a text vector, resulting in a text vector dataset E = {e1, e2, ..., e...}. k ,...,e 14699}, where e k b k The corresponding text vector.

[0066] In this embodiment of the invention, for a text b k "The current water intake and drainage situation of thermal power plants differs somewhat from the requirements of the discharge permit, making wastewater discharge control imperative. This paper compares relevant laws, regulations, and technical standards for water conservation and wastewater treatment, and, based on the actual situation of water intake and drainage, proposes targeted wastewater discharge control technologies, including the objectives and principles for thermal power plant wastewater discharge control, raw water pretreatment, desulfurization wastewater treatment, and other wastewater treatment technologies. Furthermore, it provides end-of-pipe wastewater treatment technologies, offering a basis and framework for related upgrades to thermal power plants." After processing as described in S201, the result is: "Thermal power plant / water intake / drainage / discharge permit / thermal power plant / wastewater / discharge / control / water conservation / wastewater". The keyword encoding vectors for "thermal power plant, water intake, drainage, discharge permit, wastewater, discharge, control, water conservation, wastewater treatment, laws and regulations, technical standards, water intake, drainage, raw water, pretreatment, desulfurization wastewater treatment, end-of-pipe treatment" are "k1, k2, k3, k4, k5, k6, k7, k8, k9, k10, k11, k12, k13, k14, k15, k16, k17" respectively. k The corresponding text vector is "k1 / k2 / k3 / k4 / k1 / k5 / k6 / k7 / k8 / k9 / k10 / k11 / k12 / k13 / k5 / k6 / k7 / k1 / k5 / k6 / k7 / k14 / k15 / k16 / k9 / k17 / k9 / k1", which can also be represented by b. k The corresponding text vectors are represented as “k1 / k2 / k3 / k4 / k5 / k6 / k7 / k8 / k9 / k10 / k11 / k12 / k13 / k14 / k15 / k16 / k17” and their corresponding word frequency forms.

[0067] S204, Construct a neural network model for keyword evaluation.

[0068] The keyword evaluation neural network model of this invention comprises an input layer, a fully connected layer, an attention layer, and a classification model layer. Wherein:

[0069] The input layer takes a text vector and label corresponding to a text as input.

[0070] The fully connected layer consists of two parallel, identically structured fully connected neural networks. The inputs of the two fully connected neural networks are connected to the outputs of the input layer, and the activation function f1 is used to obtain two feature vectors Y1. 1×(m+1) and Y2 1×(m+1) The calculation is as follows:

[0071] Y1 1×(M+1) =f1(W1) dense *Y 1×(m+1) +b1 dense )

[0072] Y2 1×(m+1) =f1(W2) dense *Y 1×(m+1) +b2 dense )

[0073] Among them, W1 dense W2 dense Y represents the weight parameters of two fully connected neural networks. 1×(m+1) b1 is the output of the input layer. dense b2 dense These represent the offset parameters of the two fully connected neural networks. In this embodiment, the activation function f1 is the tanh activation function.

[0074] The attention layer focuses on two feature vectors Y1 1×(m+1) With Y2 1×(m+1) Summation is performed, and attention weights {I} are generated using activation function f2. a1 ,I a2 ,...,I ai ,...,I am ,I y}, where I ai Indicates a i The corresponding attention weight, I y This represents the attention weight corresponding to the label; finally, the keyword dictionary A for electricity text is weighted based on this attention weight to generate a keyword weighted set D = {e1×I}. a1 e2×I a2 ,...,e m ×I am In this embodiment, the activation function f2 is the sigmoid activation function.

[0075] The model's classification layer is used for text classification, outputting the keyword classification result y′.k In this embodiment, the classification model layer can be a CNN, textCNN, Bi-LSTM, Transformer, or BERT classification model.

[0076] S205, the text vector dataset E is divided into a training set and a test set according to a certain ratio. In this embodiment, the first 70% of the dataset is used as the training set, and the remaining 30% is used as the test set; the training set is used to train the model; the test set is used to evaluate the training effect of the model.

[0077] S206, input each piece of text data from the training set into the keyword evaluation neural network model to obtain the attention weights {I} in the attention layer. a1 ,I a2 ,...,I a4196 ,I y This serves as a set of importance for power-related keywords and tags, and the importance RI of each keyword relative to the tags is calculated. ax =I ax / I y , where x∈[1,4196], and x is a positive integer;

[0078] S207: Sort the keywords in A from highest to lowest relative importance, and select the keywords with high relative importance to generate q keyword combinations.

[0079] In an embodiment of the present invention, the keywords in A are sorted from highest to lowest relative importance, and the sorted keyword set is {a}. r1 ,a r2 ,...,a rl ,...a r4196}, where a rl The keyword ranked l in relative importance is represented by the keyword 'l'. The process of selecting keywords with high relative importance involves: selecting keywords 2196 times from the sorted keyword set. In the p-th selection, the first 4196-p features are taken, where p is an integer greater than or equal to 1, and 2196 ≥ p ≥ 1. The final 2196 keyword combinations are:

[0080] {a r1 ,a r2 ,...a r4195},{a r1 ,a r2 ,...a r4194},...,{a r1 ,a r2 ,...a r2000}

[0081] S208: Using 2196 keyword combinations, B is re-encoded to obtain a new text vector dataset. This dataset is then input into a classification model, yielding 2196 classification model outputs. The keyword combination with the highest classification accuracy is selected as the reconstructed, simplified keyword dictionary. MAE, RMSE, or R... 2 The accuracy of the model classification is evaluated using various metrics. The calculation formulas for the first three metrics are as follows:

[0082]

[0083]

[0084]

[0085] Among them, P act,i This represents the actual value of the label. P represents the average value of the actual labels. pre,i represents the predicted value of the label, and n represents the number of samples. Table 1 shows some of the keyword combination index values ​​obtained.

[0086] Table 1. Keyword Combination Index Values

[0087] Keyword combination number MAE RMSE <![CDATA[R 2 ]]> 1 2.856 4.365 0.862 2 2.851 4.363 0.863 … … … … 1954 2.267 4.041 0.889 1955 2.257 4.034 0.894 1956 2.249 4.023 0.891 1957 2.219 4.022 0.893 1958 2.215 4.013 0.899 1959 2.228 4.059 0.901 1960 2.242 4.065 0.889 … … … … 2195 2.312 4.059 0.887 2196 2.289 4.067 0.895

[0088] The keyword combination with the lowest MAE value, i.e., the 1958th keyword combination, was selected as the basis for the reconstructed simplified keyword dictionary. This simplified keyword dictionary contains a total of 2238 keywords, including keywords such as "AC / DC circuit breaker, switch, electric arc, insulating material, lightning strike, insulator, discharge characteristics, electromagnetic field, electromagnetic force, magnetic material, space charge, voltage and current, electrical engineering technology, energy storage system, energy storage unit, battery energy storage, storage battery, fuel cell, charging and discharging, solar power generation, photovoltaic system, photovoltaic grid connection, wind power system, wind turbine, thermal power plant, heating unit, waste heat power generation, coal, heat dissipation, power generation unit, power plant, power generation system, etc." Electric motor sets, power supply, power quality, electricity consumption, electricity meters, electricity prices, power outages, electrical equipment, traction power supply, electrical appliances, electric vehicles, AC / DC power transmission, power transmission lines and cables, power distribution systems, power equipment, smart substations, electricity markets, spot markets, electricity charges, electricity prices, network losses, power grid systems, microgrids, power flow calculation, load types, model management, power grid dispatching, power devices, converters, inverters, rotors, electric motors, generators, windings, power transformers, grid side, losses, energy management, renewable energy, new energy power generation, inspection, fault diagnosis, live-line working, control systems, robustness, neural networks, big data, artificial intelligence, power communication networks, etc. These keywords were selected by the method of this invention and were found to be of relatively high importance. For example, the relative importance of "AC / DC circuit breaker, switch, electric arc, insulating material, lightning strike, insulator, discharge characteristics" were 1.38, 1.42, 1.11, 1.36, 1.05, 1.29, and 1.07, respectively. The relative importance of keywords such as "model, method, power, series, parallel" were 0.13, 0.11, 0.26, 0.18, and 0.17, respectively. These keywords with low relative importance were deleted.

[0089] The power keyword dictionary constructed by the method of this invention can be used for Chinese power text classification. After obtaining the original power text data and preprocessing it, the reconstructed keyword dictionary and text classification model are used to achieve text classification.

[0090] Specifically, in this embodiment of the invention, 3-5 years of publicly available summary information on power system patents, journal articles, etc., is obtained to acquire raw Chinese power text data. The data is preprocessed through text segmentation and stop word removal, deleting duplicate or irrelevant data and selectively filling in incomplete information. This transforms the Chinese power text into a structured data format that can be recognized and processed by a computer. Using the reconstructed keyword dictionary of this invention combined with text classification models such as BERT, FastCNN, and TextCNN, the text classification accuracy can be effectively improved, the model simplified, and the model speed optimized. The distribution of model accuracy, recall, and F1 score for the three models across 26 power system research areas is shown below. Figure 3As shown in the figure, the Micro-F1 scores of the three models using the initial keyword dictionary and the simplified keyword dictionary are compared as follows: Figure 4 As shown, the Micro-F1 scores of BERT, FastCNN, and TextCNN using the initial keyword dictionary are 0.856, 0.816, and 0.837, respectively, while the Micro-F1 scores using the simplified keyword dictionary are 0.932, 0.889, and 0.916, respectively, representing improvements of 10.17%, 8.95%, and 9.44%.

[0091] In summary, this invention, guided by a classification model, extracts the most representative and informative keywords through keyword screening before inputting them into the model. This results in significant advantages in dimensionality reduction, noise reduction, interpretability, and model training efficiency. This method adapts well to the keyword feature extraction process of classification models, filtering out keywords useless to the classification results while minimizing text classification error, and selecting a set of relatively important keywords. This accelerates the training speed of text classification models and optimizes the training process.

[0092] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for constructing an electricity keyword dictionary based on an attention mechanism, characterized in that, Includes the following steps: S1, based on the initial electricity keyword dictionary A Encode each keyword into a m 3D vector m for A The number of keywords in the text; S2, Obtain the electricity text dataset B and its label dataset Y ; S3, according to A Encoding of keywords in Chinese B Chinese text A The words appearing in the text are encoded to obtain the text vector dataset corresponding to B. E ; S4, construct a keyword evaluation neural network model, which includes an input layer, a fully connected layer, an attention layer, and a classification model layer; the input layer is used to input text vectors and corresponding labels; the fully connected layer includes two parallel and identically structured fully connected neural networks, the inputs of which are respectively connected to the outputs of the input layer, and activation functions are used. Two feature vectors are obtained; the attention layer sums the two feature vectors and applies an activation function. Generate attention weights for keywords and tags, and... A Each keyword in the dataset is weighted based on its corresponding attention weight to generate a weighted keyword set. D The classification model layer is used for text classification. S5, E Divided into training set and test set; S6, input the training set into the keyword evaluation neural network model, use the attention weights to calculate the importance of each keyword relative to the label, and generate a set of relative importance; S7, for A The keywords in the text are sorted from highest to lowest relative importance, and the keywords with the highest relative importance are selected to generate the final product. q Keyword combinations; S8, using q Each keyword combination is respectively for B The text vector dataset is re-encoded and then input into the classification model to obtain... q The output of each classification model is used to select the keyword combination with the highest classification accuracy as the reconstructed simplified keyword dictionary.

2. The method for constructing an electricity keyword dictionary based on an attention mechanism according to claim 1, characterized in that, In step S1, an initial power keyword dictionary is obtained through one of the following methods. A : Method 1: Preprocess the Chinese power text by performing text segmentation and stop word removal, deleting duplicate or irrelevant data, and selectively filling in incomplete data information to transform the Chinese power text into a structured data form that can be recognized and processed by computers, thus serving as the initial power keyword dictionary; Method 2: Use an open-source electricity keyword dictionary.

3. The method for constructing an electricity keyword dictionary based on an attention mechanism according to claim 1, characterized in that, In step S1, each keyword is encoded into a single key using one-hot encoding. m A dimensional vector.

4. The method for constructing an electricity keyword dictionary based on an attention mechanism according to claim 1, characterized in that, S4, two feature vectors and The calculation formula is as follows: in, , These represent the weight parameters of the two fully connected neural networks. The output of the input layer, , These represent the offset parameters of the two fully connected neural networks.

5. The method for constructing an electricity keyword dictionary based on an attention mechanism according to claim 1, characterized in that, The activation function Choose the tanh activation function. Choose the sigmoid activation function; the classification model layer can be a CNN, textCNN, Bi-LSTM, Transformer, or BERT classification model.

6. The method for constructing an electricity keyword dictionary based on an attention mechanism according to claim 1, characterized in that, In step S7, the keywords in A are sorted from highest to lowest relative importance, resulting in a set of sorted keywords. ,in Ranking by relative importance Keywords of a certain importance; the selection of relatively important keywords is as follows: from the sorted keyword set... q Secondary keyword selection, the first p During the next selection, retrieve the previous... mp One feature, among which p For integers greater than or equal to 1, q ≥ p ≥1, the final result q The keyword combination is as follows: , ,..., 。 7. The method for constructing an electricity keyword dictionary based on an attention mechanism according to claim 1, characterized in that, The S8 is expressed using MSE, RMSE, or R. 2 The accuracy of the indicator evaluation classification model is calculated using the following formulas: in, This represents the actual value of the label. This represents the average of the actual values ​​of the labels. This represents the predicted value of the label. This represents the number of samples; based on one of the above indicators, the feature combination with the highest classification accuracy is selected as the reconstructed keyword dictionary.

8. A method for classifying electrical data, characterized in that, After preprocessing the original power text data, text classification is achieved by reconstructing a keyword dictionary and a text classification model based on the attention mechanism-based power keyword dictionary construction method described in any one of claims 1 to 7.

Images