A method, system and storage medium for identifying electric power domain entities
Through the power-oriented domain entity recognition method, combined with the two-way long and short-term memory network and conditional random field model, the entity recognition problem in the construction of the power industry's knowledge graph is solved, and the accurate identification of the fields of power scheduling and operation inspection is achieved, ensuring knowledge Intelligent application of map.
Patent Information
- Application Number
- CN202010625052.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-06-29
- Filing Date
- 2020-07-01
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2040-07-01
AI Technical Summary
The power industry has not yet established a complete domain knowledge graph, and has not achieved comprehensive intelligence of power knowledge, especially in entity recognition, which has affected the construction and application of knowledge graphs.
A domain entity recognition method for power-oriented power is proposed. The entity recognition is performed through data extraction, corpus annotation, a model combining two-way long and short-term memory network and conditional random field. The model is evaluated and evaluated by weighted harmonization averages of accuracy and recall. Correct to ensure recognition effect.
It realizes entity recognition in the fields of power scheduling and operation inspection, provides a basis for building a knowledge graph, avoids model overfitting, and improves recognition accuracy and reliability.
Smart Images

Figure CN111783464B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of electric power artificial intelligence technology, and in particular to an entity recognition method, system and storage medium for the electric power field. Background Art
[0002] In recent years, with the development of smart grids, traditional power grid facilities have been continuously upgraded and transformed, and various information systems have been widely used. Smart grids have generated and accumulated massive multi-source heterogeneous data. With the construction of ubiquitous power Internet of Things, artificial intelligence based on massive power data will play an important role in supporting the construction of professional applications such as marketing, operation and inspection, materials, dispatching, and safety supervision. The in-depth construction of artificial intelligence applications will also promote the continuous emergence of innovations in new businesses and new application models. At present, the power industry has not yet established a complete domain knowledge graph, and has not yet realized the comprehensive intelligence of power knowledge support.
[0003] The so-called knowledge graph describes concepts, entities and their relationships in the objective world in a structured form. It is a structured semantic knowledge base used to describe concepts and their relationships in the physical world in a symbolic form. Its basic components are "entity-relationship-entity" triples, as well as entities and their related attribute-value pairs. Entities are interconnected through relationships to form a network of knowledge structures. Therefore, in the field of power dispatching, knowledge graphs can solidify dispatching procedures and dispatchers' experience knowledge, and assist in supporting applications such as power grid operation monitoring, abnormal handling, and mode adjustment. In the field of power operation and inspection, knowledge graphs can store knowledge such as equipment, faults, and handling methods, and can support intelligent equipment operation, maintenance, and inspection.
[0004] To build a domain knowledge graph, we first need to extract power entities. Entity recognition is a basic task in natural language processing and has a wide range of applications. Named entity recognition in a specific domain aims to identify domain-specific entities and their categories, and plays an important role in domain document classification, retrieval, and content analysis. It is the basis for deep and complex information extraction tasks and the cornerstone of the knowledge computing process of converting data into machine-readable knowledge.
[0005] Therefore, it is extremely necessary to study a domain entity recognition method for electricity to construct a knowledge graph in the electricity field. Summary of the invention
[0006] The object of the present invention is to solve at least one of the above-mentioned technical drawbacks.
[0007] To this end, an object of the present invention is to propose a method for identifying domain entities for electric power, comprising the following steps:
[0008] S1. Extract the acquired power grid data to form a data set, and annotate the corpus of the training data. S2. Divide the annotated corpus into a test set, input the test set into the constructed power field entity recognition model for recognition, reverse decode the recognition result, and obtain the recognized domain entity; S3. Calculate the weighted harmonic mean of the accuracy and recall rate according to the recognized domain entities, and use the weighted harmonic mean to evaluate the power field entity recognition algorithm model. When the obtained evaluation score meets the business requirements, output the recognized domain entity; when the obtained evaluation score does not meet the business requirements, correct the power field entity recognition model and repeat the above steps S2-S3.
[0009] Preferably, in S1, BIE is used to annotate the corpus of training data, wherein the letter B is used to represent the first character of the electric entity, the letter I is used to represent the internal characters of the electric entity, the letter E is used to represent the ending characters of the electric entity, and the letter O is used to represent other characters that do not belong to the named entity.
[0010] In any of the above embodiments, preferably, a method combining a bidirectional long short-term memory network and a conditional random field is used to construct an entity recognition algorithm model in the power field; in S2, the annotated corpus is divided into a test set and a training set, and the constructed domain entity recognition algorithm model is trained using the training set.
[0011] Preferably, in any one of the above embodiments, the electric power field entity recognition algorithm model includes a first layer, which is used to map each word in a sentence from a one-hot vector to a low-dimensional dense word vector; a second layer, a bidirectional LSTM layer, is used to automatically extract sentence features from the word vectors of the first layer; a third layer, a CRF layer, is used to perform sentence-level sequence annotation on the extracted sentence features; and classify the sentence labels formed by the annotation.
[0012] Preferably, in any of the above embodiments, in the CRF layer, the sentence labels are classified using the following formula:
[0013]
[0014] Among them, score(x,y) represents the score of sentence x with label y.
[0015] Preferably, in any of the above embodiments, when training the electric power field entity recognition algorithm model, the log-likelihood value of the training sample is solved by maximizing the log-likelihood function;
[0016] log P(y x |x)=score(x,y x )-log(∑ y′exp(score(x,y′)));
[0017] In the formula, score(x, y x ) means the label of sentence x is y x score; (x, y x ) are training samples.
[0018] Preferably, in any of the above embodiments, in step S3, the power field entity recognition algorithm model uses the Viterbi algorithm to solve the optimal path in the prediction process when solving the log-likelihood value, and obtains the position corresponding to the recognized entity.
[0019] In any of the above embodiments, preferably, in S3, the accuracy is calculated by the following formula:
[0020]
[0021] The recall rate is calculated using the following formula:
[0022]
[0023] When evaluating the entity recognition algorithm model, the weighted harmonic mean of the accuracy and recall is calculated using the following formula:
[0024]
[0025] The present invention also provides a domain entity recognition system for electric power, comprising:
[0026] The data acquisition module is used to extract the acquired power grid data to form a data set and to annotate the training data corpus;
[0027] The domain entity recognition module is used to divide the annotated corpus into a test set, input the test set into the constructed power domain entity recognition model for recognition, reversely decode the recognition result, and obtain the recognized domain entity;
[0028] The evaluation module calculates the weighted harmonic mean of the accuracy and recall rate based on the identified domain entities, and uses the weighted harmonic mean to evaluate the entity recognition algorithm model. When the obtained evaluation score meets the business requirements, the identified domain entities are output; when the obtained evaluation score does not meet the business requirements, the power domain entity recognition model is corrected and recognized again.
[0029] The present invention also provides an electric power-oriented domain entity recognition device, comprising a memory for storing a computer program; and a processor for implementing the steps of the above-mentioned electric power domain entity recognition method when executing the computer program.
[0030] The present invention also provides a computer storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the domain entity recognition method are implemented.
[0031] The entity recognition method for the power field provided by the present invention has at least the following advantages compared with the prior art:
[0032] By marking the corpus of power grid data and then using the power field entity recognition model to identify entities, it is helpful to achieve accurate information collection in the huge power dispatch information, provide a solid foundation for the future construction of knowledge graphs, realize the extraction of entities in the power field, and support the construction of power knowledge graphs. The power field entity recognition model is evaluated by accuracy and recall rate, which facilitates timely correction of the model and prevents the problem of overfitting in the power field entity recognition model. The bidirectional long short-term memory network and conditional random field are used to construct the entity recognition model, which combines the sequence modeling capabilities of the bidirectional long short-term memory network, can capture distant context information, and has the ability of neural network fitting nonlinearity. At the same time, the conditional random field is used to optimize the entire sequence, which effectively solves the problem of gradient disappearance or gradient explosion in traditional recurrent neural networks. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings;
[0034] Figure 1 A flowchart of a method for identifying electric power-oriented domain entities provided by the present invention;
[0035] Figure 2 A unit structure diagram of an LSTM model in a method for identifying electric power-oriented domain entities provided by the present invention;
[0036] Figure 3 A unit structure diagram of a BLSTM model in a power-oriented domain entity recognition method provided by the present invention;
[0037] Figure 4 A method for identifying electric power-oriented entities provided by the present invention includes a bidirectional long short-term memory network and a conditional random field electric power-oriented entity identification model structure diagram; DETAILED DESCRIPTION
[0038] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present invention can be combined with each other without conflict.
[0039] The following detailed description is an exemplary description, which is intended to provide further detailed description of the present invention. Unless otherwise specified, all technical terms used in the present invention have the same meaning as those generally understood by those skilled in the art to which the present invention belongs. The terms used in the present invention are only for describing specific embodiments, and are not intended to limit the exemplary embodiments according to the present invention.
[0040] like Figure 1 As shown, the present invention proposes a method for identifying domain entities in the electric power sector, comprising the following steps:
[0041] S1. Obtain power grid data, extract data from the power grid data to form a data set, and use BIE to annotate the training data corpus;
[0042] When annotating the corpus, the letter B is used to represent the first character of the power entity, the letter I is used to represent the internal characters of the power entity, the letter E is used to represent the end character of the power entity, and the letter O is used to represent other characters that do not belong to the named entity. For example, in the text data "Analysis of the impact of DC reactors on the stability of the transmission system", the power entities are "DC reactors" and "transmission systems", "DC reactors" are annotated as "BIIIE", and "transmission systems" are annotated as "BIIE". In the sentence as shown in the following table
[0043] Table 1 Corpus annotation examples
[0044] Sentences containing electrical entity nouns Corpus annotation results Analysis of the impact of DC reactors on the stability of power transmission systems OOBI II EOBIIEO OOOOO Capacitors at the converter station exit BIEO OOO BIE DC transmission technology based on voltage source converter OOBI IIII EOBI III E
[0045] S2. Divide the annotated corpus into a test set, input the test set into the constructed power domain entity recognition model for recognition, reversely decode the recognition result, and obtain the recognized domain entity;
[0046] Use bidirectional long short-term memory network and conditional random field to construct the algorithm model of entity recognition in the power field;
[0047] It should be noted that the conditional random field (CRF) is a discriminative model that combines the characteristics of the ME model and the HMM model, and has good results in natural language processing sequence labeling such as part-of-speech tagging and entity recognition.
[0048] The CRF model is a statistical model based on probability graphs. It can use contextual information to add multiple features and achieve global normalization. It can obtain the global optimal solution and solve the problem of labeling bias.
[0049] In general, an input sequence is represented by x={x1, x2, ..., xn}, where xi represents the vector of the i-th input character and y={y1, y2, ..., yn} represents the possible label sequence of input x. Y(x) represents the possible label sequence of X. The sequence CRF probability model defines a series of conditional probabilities P(y|z; W, b), and the probability of all possible sequences y given x is calculated by formula (1).
[0050]
[0051] in is the potential function, WTy', yzi and by', y are the weight vector and bias for the label pair (y', y), respectively.
[0052] For CRF model training, maximum likelihood estimation is used. For the training set {(zf, Yf)), the likelihood estimation algorithm is shown in (Formula 2).
[0053] L(W, b) = ∑log p(y|z; W, b) (Formula 2)
[0054] The maximum likelihood training process is to select parameters to maximize L(w, b). The decoding process is to find the labeled sequence y* with the maximum conditional probability as shown in formula (3).
[0055] y * = argmax y∈Y(Z) p(y|z; w, b) (Formula 3)
[0056] For the sequential CIU model, only the interaction between two consecutive labels is considered, and the Viterbi algorithm can achieve better results in the training and decoding process.
[0057] LSTM (Long Short-Term Memory) is a special recurrent neural network (RNN). It solves the RNN gradient vanishing problem by introducing memory units and threshold mechanisms. The LSTM unit structure is shown in the figure. x represents the input of the network at different times, y is the output of the network, h is the hidden layer, u is the weight from the input layer to the hidden layer, w is the weight from the previous node hidden layer to the current node hidden layer, and v is the weight from the hidden layer to the output layer. Figure 2 As shown, this is the structure diagram of LSTM
[0058] Bidirectional long short-term memory network (BLSTM); its structure is as follows Figure 3As shown in the figure, when processing sequence data, BLSTM adds a reverse calculation process compared to the general LSTM. This process can utilize the context information of the sequence, and finally output the values calculated in both the forward and reverse directions to the output layer at the same time. In this way, all the information of a sequence is obtained from two directions, which can be applied to various natural language processing tasks.
[0059] BLSTM has strong sequence modeling capabilities, can capture distant context information, and has the ability to fit nonlinear neural networks. CRF calculates a joint probability and optimizes the entire sequence rather than splicing the best at each moment, so CRF is better than LSTM. Both models have their own unique advantages. In order to integrate the advantages of these two models, the BiLSTM-CRF model is used in power entity recognition. Adding a CRF linear layer after the output layer of the BLSTM network effectively solves the problem of gradient disappearance or gradient explosion in traditional recurrent neural networks.
[0060] like Figure 4 As shown in the figure, a bidirectional long short-term memory network and conditional random field are used to build an entity recognition algorithm model in the power field. The entity recognition model in the power field includes
[0061] The first layer is used to map each word in the sentence from a one-hot vector to a low-dimensional dense word vector; (x1, x2, ..., xn)
[0062] The second layer, the bidirectional LSTM layer, is used to automatically extract sentence features from the word vectors of the first layer;
[0063] Specifically, the word vector is used as the input layer of the bidirectional LSTM, and the bidirectional LSTM automatically extracts sentence features. The LSTM layer converts the hidden state sequence output by the forward LSTM into Hidden state sequence with reverse LSTM output Splice to get the complete hidden state sequence h t , as the output layer of LSTM, where m is the number of units in the hidden layer.
[0064]
[0065]
[0066]
[0067] The hidden state sequence is connected to the CRF linear layer, and the hidden state is mapped from m dimensions to k dimensions (k is the number of labels in the annotation set), so as to obtain the automatically extracted sentence features and form the feature matrix P.
[0068] P=(p1,p2,...,p n)∈R n×k (Formula 7)
[0069] Each dimension of the feature matrix P can be regarded as the word x i Classify to jth i However, when labeling each position, the already labeled information cannot be used, so a CRF layer will be connected for labeling.
[0070] The third layer, the CRF layer, is used to perform sentence-level sequence annotation on the extracted sentence features and classify the sentence labels formed by the annotation.
[0071] The CRF layer is used for sentence-level sequence labeling. Its parameter is the (k+2)×(k+2) matrix A. Aij represents the transfer score from the i-th label to the j-th label, so that when labeling a position, the previously labeled label can be used. For sentence x, the score calculation of its label y=(y1, y2, ..., yn) is shown in the following formula (Formula 8).
[0072]
[0073] It can be seen that the score of the entire sequence is equal to the sum of the scores of each position, and the score of each position is obtained from two parts, one of which is p output by LSTM i The other part is determined by the transfer matrix A of CRF.
[0074] Since the feature matrix is k-dimensional (k is the number of labels in the annotation set), if Softmax is performed on the matrix P, it is equivalent to classifying each position of the sentence into k categories.
[0075] Therefore, Softmax is used to obtain the normalized probability: as shown in the following formula (Formula 9).
[0076]
[0077] After Softmax normalization, the results of classification of each position according to K labels are obtained.
[0078] The annotated corpus is divided into a training set and a test set. The test set is used to train the constructed entity recognition algorithm model. The annotated data is used as the corpus required for entity recognition supervision training, of which 80% is used as training and the remaining 20% is used as test data. In order to speed up the model learning speed, the neural network model uses the Adam optimizer. The parameters of the model are shown in the following table.
[0079] Table 3 Model training parameter settings
[0080] parameter Parameter Value batch_size 64 epoch 10 hidden_dim 300 optimizer Adam CRF True learning rate 0.001 Gradient clipping 5.0 dropout keep_prob 0.5 update embedding True Pretrain embedding random embedding_dim 300 shuffle True
[0081] When the model is trained, the log-likelihood function is maximized. Formula (10) gives the probability that a training sample (x, y x ) is:
[0082] log P(y x |x)=score(x,y x )-log(∑ y′ exp(score(x,y′))) (Formula 10)
[0083] This formula is used to sort the classified annotations. In the prediction process, the Viterbi algorithm is used to solve the optimal path, and the decoding process is to solve the annotation sequence with the highest probability, which is the position corresponding to the recognized entity.
[0084] In one embodiment of the present invention, for the given text data "Two sets of safety automatic devices are installed in Hexi Substation and Wali Substation. After the device detects that the Wali double-circuit line is tripped, it will connect and cut the switch on the east side of the river", after the power domain entity recognition, the domain entities "Hexi Substation", "Wali Substation", "Safety Automatic Device", "Wali Double-circuit Line", "Hedong Side Switch" can be obtained;
[0085] S3. Calculate the weighted harmonic mean of the accuracy and recall rate, and evaluate the entity recognition algorithm model. If the evaluation meets the business requirements, it can be used for power entity recognition. The evaluation indicators used in power entity recognition are mainly accuracy and recall rate, and their definitions are shown in formula (Formula 11) and (Formula 12) respectively.
[0086]
[0087]
[0088] Both values are between 0 and 1. The closer the value is to 1, the higher the accuracy or recall rate is. Accuracy and recall rates sometimes conflict with each other, and their weighted harmonic mean, i.e., F value, needs to be considered comprehensively, as shown in formula (13).
[0089]
[0090] The present invention also provides a domain entity recognition system for electric power, comprising:
[0091] The data acquisition module is used to obtain power grid data, extract data from the power grid data to form a data set, and use BIE to annotate the training data corpus; refer to the above
[0092] The domain entity recognition module is used to divide the annotated corpus into a test set, input the test set into the constructed power domain entity recognition model for recognition, reversely decode the recognition result, and obtain the recognized domain entity;
[0093] The evaluation module calculates the weighted harmonic mean of the accuracy and recall rate based on the identified domain entities, and uses the weighted harmonic mean to evaluate the entity recognition algorithm model. When the evaluation score meets the business requirements, the identified domain entities are output;
[0094] When the evaluation score obtained does not meet the business requirements, the entity recognition model in the power field is modified and recognized again.
[0095] The present invention also provides a computer storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the domain entity recognition method are implemented.
[0096] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0097] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0098] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0099] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.
Claims
1. A method for identifying domain entities in the electric power sector. Features: The following steps are involved: S1. Extract the acquired power grid data to form a data set, and annotate the training data corpus; S2. Divide the annotated corpus into a test set, input the test set into the constructed power field entity recognition algorithm model for recognition, reversely decode the recognition result, and obtain the recognized field entity; S3. Calculate the weighted harmonic mean of the accuracy and recall rate based on the identified domain entities, and use the weighted harmonic mean to evaluate the electric power domain entity recognition algorithm model. When the evaluation score meets the business requirements, output the identified domain entities; When the evaluation score obtained does not meet the business requirements, the electric power field entity recognition algorithm model is corrected and the above steps S2-S3 are repeated; The method of combining bidirectional long short-term memory network with conditional random field is used to construct an entity recognition algorithm model in the power field; in S2, the annotated corpus is divided into a test set and a training set, and the constructed domain power field entity recognition algorithm model is trained using the training set; In S3, The accuracy is calculated using the following formula: ; The recall rate is calculated using the following formula: ; When evaluating the entity recognition algorithm model in the electric power field, the weighted harmonic mean F value of the accuracy and recall rate is calculated using the following formula: ; The electric power field entity recognition algorithm model includes: The first layer is used to map each word in the sentence from a one-hot vector to a low-dimensional dense word vector; The second layer, the bidirectional LSTM layer, is used to automatically extract sentence features from the word vectors of the first layer; The third layer, the CRF layer, is used to perform sentence-level sequence annotation on the extracted sentence features and classify the sentence labels formed by the annotation.
2. According to claim 1, the method for identifying domain entities for electric power, Features: In S1, BIE is used to annotate the corpus of training data, where the letter B represents the first character of the power entity, the letter I represents the internal character of the power entity, the letter E represents the end character of the power entity, and the letter O represents other characters that do not belong to the named entity.
3. The entity recognition method for the electric power field according to claim 1, Features: In the CRF layer, the following formula is used to classify sentence labels: in, Indicates the score of sentence x with label y.
4. The entity recognition method for the electric power field according to claim 3, Features: When training the electric power field entity recognition algorithm model, the log-likelihood value of the training sample is solved by maximizing the log-likelihood function; ; In the formula, The label of sentence x is score; ) are training samples.
5. The entity recognition method for the electric power field according to claim 1, Features: In step S3, the power field entity recognition algorithm model uses the Viterbi algorithm to solve the optimal path in the prediction process when solving the log-likelihood value, and obtains the position corresponding to the recognized entity.
6. An entity recognition system for the power sector, It is characterized in that Based on the method described in any one of claims 1 to 5, the system comprises: The data acquisition module is used to extract the acquired power grid data to form a data set and to annotate the training data corpus; The domain entity recognition module is used to divide the annotated corpus into a test set, input the test set into the constructed power domain entity recognition algorithm model for recognition, reversely decode the recognition result, and obtain the recognized domain entity; The evaluation module calculates the weighted harmonic mean of the accuracy and recall rate based on the identified domain entities, and uses the weighted harmonic mean to evaluate the electric power domain entity recognition algorithm model. When the obtained evaluation score meets the business requirements, the identified domain entities are output; when the obtained evaluation score does not meet the business requirements, the electric power domain entity recognition algorithm model is corrected and recognized again.
7. A computer storage medium, It is characterized in that The computer storage medium stores a computer program, and when the computer program is executed by a processor, the steps of an entity identification method for the power field as claimed in any one of claims 1 to 5 are implemented.
