A method and device for identifying entities in power dispatching text by integrating multiple features

Abstract

The present invention discloses a method and device for entity recognition in electric power dispatch text that integrates multiple features. The method comprises: performing noise reduction and cleaning processing on unstructured electric power dispatch text, performing word-granular entity tagging and part-of-speech feature tagging, and obtaining a word-level corpus. The corpus contains three columns of sequences, namely, word, entity label, and part-of-speech; pre-training the first column of the obtained corpus at word granularity to obtain a vector representation of the word, and obtaining a part-of-speech feature vector representation of the second and third columns using a one-hot vector representation method; adding the left-neighboring word entropy value obtained from the first column to the word vector and part-of-speech feature vector to perform multi-feature fusion; inputting the vector obtained after feature fusion into a two-layer neural network model to obtain an output; and inputting the output of the two-layer neural network model into a conditional random field model to obtain a predicted label sequence, thereby realizing named entity recognition of the sequence. The present invention improves the effect of named entity recognition in electric power dispatch text.

Description

Technical Field

[0001] The present invention relates to the technical field of text recognition, and in particular to a method and device for recognizing entities in electric power dispatching text by integrating multiple features. Background Art

[0002] In the information age, smart grids have become an inevitable trend in the development of power systems, and data analysis in power systems has garnered significant attention. Routine power system dispatching accumulates a vast amount of textual information, such as outage maintenance application forms for decommissioning and recommissioning approvals. Parsing this unstructured data can extract useful information. Named entity recognition is a key step in parsing power dispatch text. Named entities in power dispatch text refer to meaningful nouns, such as the names of electrical equipment.

[0003] Named entity recognition (NER) is a hot topic in natural language processing, and numerous methods have emerged. Currently, most traditional approaches rely on deep learning, some of which employ supervised deep learning methods. These methods often rely on training data by annotating and segmenting the data into training sets. However, due to the imperfections of current segmentation technology, these methods generate noise, which can affect recognition accuracy. Some approaches are based on the BERT model developed by Google. These methods are often fine-tuned based on models that have been trained on large-scale corpora.

[0004] However, electric power texts, especially electric power dispatching texts, have extremely strong domain characteristics. Directly using the above method to accurately identify entities is difficult to meet the needs of serving downstream applications. Therefore, how to reflect the domain characteristics of electric power dispatching texts and at the same time propose a method to improve the accuracy of entity recognition by relying on domain characteristics is an urgent problem to be solved. Summary of the Invention

[0005] Purpose of the invention: The purpose of the present invention is to provide a method and device for entity recognition of electric power dispatching text that integrates multiple features, introduces multiple features to characterize the domain characteristics of electric power dispatching text, and solves the problem in the prior art of directly using traditional domain methods resulting in poor accuracy and other effects.

[0006] Technical solution: To achieve the above-mentioned purpose, the present invention proposes a method for identifying entities in electric power dispatching text by integrating multiple features, which includes the following steps:

[0007] The unstructured power dispatch text is cleaned and de-noised, and then labeled with word-level entities and part-of-speech features to obtain a word-level corpus. The corpus contains three columns: word, entity tag, and part-of-speech.

[0008] The first column of the obtained corpus is pre-trained at word granularity to obtain word vector representations, and the second and third columns are represented by part-of-speech feature vectors using the one-hot vector representation method;

[0009] On the basis of word vector and part-of-speech feature vector, the left neighbor word entropy value obtained from the first column sequence is added to perform multi-feature fusion;

[0010] The vector obtained after feature fusion is input into the two-layer neural network model to obtain the output;

[0011] The output results of the two-layer neural network model are input into the conditional random field model to obtain the predicted label sequence, thereby realizing named entity recognition of the sequence.

[0012] Furthermore, the entropy value of the left neighbor word of the first column sequence is obtained according to the following formula:

[0013]

[0014] Where C is each word in the first column of the corpus, aC is the word that appears to the left of word C, A is the set of all words to the left of word C in the corpus, and P(aC|C) is the probability that word aC appears to the left of C.

[0015] Furthermore, the multi-feature fusion method includes:

[0016] The word vector sequence X obtained from a sentence i , part-of-speech feature vector sequence β, left neighbor word entropy value E L Divided into two groups: one group input is a sequence of word vectors and a sequence of part-of-speech features, and the other group input is a sequence of word vectors and a sequence of entropy values ​​of the corresponding left neighboring words;

[0017] Concatenate the two sequences in the two inputs to obtain two new vectors as the fused features, expressed as: 1,j ={x j β j}, input 2,j ={x j ;E L (i)}, where j is the index of the jth word in the i-th sentence, x j The vector representing the jth word in the i-th sentence.

[0018] Furthermore, the word vector representation of the first column of the corpus is obtained by pre-training the word granularity, including: constructing a word vector pre-training model of character-level control text based on Skip-gram. The word vector of each sentence in the first column sequence after pre-training is: X i ={x i,1 ,x i,2 ,x i,3 ,…,xi,n},in x i,n is the vector of the nth word in the i-th sentence, n is the number of characters in the sequence, and de is the dimensional digital vector representation of the input vector.

[0019] Furthermore, the part of speech of each word is represented by a 10-dimensional vector. The part of speech and its annotation in the corpus are shown in Table 1:

[0020] Table 1 Parts of speech and their corresponding annotations

[0021]

[0022]

[0023] The part-of-speech feature vector represented by the unique hot vector is expressed as: β i =(0,0,0,…,1,0,0…),

[0024] Furthermore, the two-layer neural network model is a two-layer long short-term memory neural network. The two sets of vectors after feature fusion are input into two LSTM networks respectively, and the hidden layer states are obtained and then weighted fused. The new hidden layer state vector of the weighted result is then passed through a neuron to obtain the predicted label probability matrix.

[0025] Furthermore, the predicted label sequences obtained using the conditional random field model include:

[0026] The score of sequence prediction is obtained based on the label probability matrix P obtained from the two-layer neural network model:

[0027]

[0028] in, l is the number of types of labels in the annotation set; A is the transition probability matrix of labels at each adjacent position, where the element A i,j It represents the probability of transferring from label i to label j, that is, the probability of transferring from the current word annotation result to the next word annotation result, and Y is the true label value;

[0029] The probability value of the correct prediction of the sequence in all prediction results is obtained according to the following formula:

[0030]

[0031] Training using maximum likelihood estimation: The maximum prediction probability output matrix of the sequence obtained by decoding is: Y X is the sequence of all labels of the input X;

[0032] Based on the maximum prediction probability output matrix of the sequence, the label of the sequence is obtained in combination with the annotation set.

[0033] The present invention also provides a multi-feature integrated power dispatching text entity recognition device, comprising:

[0034] The corpus construction module is used to perform noise reduction and cleaning on the unstructured power dispatch text, perform word-level entity tagging and part-of-speech feature tagging, and obtain a word-level corpus. The corpus contains three columns: word, entity tag, and part-of-speech.

[0035] The vector acquisition module is used to pre-train the first column of the obtained corpus at word granularity to obtain word vector representations, and to obtain part-of-speech feature vector representations of the second and third columns using the one-hot vector representation method;

[0036] The feature fusion module is used to add the left neighbor word entropy value obtained from the first column sequence to the word vector and part-of-speech feature vector to perform multi-feature fusion;

[0037] The first prediction module is used to input the vector obtained after feature fusion into the double-layer neural network model to obtain the output;

[0038] The second prediction module is used to input the output results of the two-layer neural network model into the conditional random field model to obtain the predicted label sequence, thereby realizing named entity recognition of the sequence.

[0039] The present invention also provides a computer device, comprising:

[0040] one or more processors;

[0041] Memory; and

[0042] One or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the power dispatch text entity recognition method integrating multiple features as described above are implemented.

[0043] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned method for entity recognition of electric power dispatching text by integrating multiple features.

[0044] Beneficial effects: The present invention introduces multiple features to characterize the domain characteristics of electric power dispatching texts. On the basis of semantic features and part-of-speech features, the left-adjacent character entropy feature is introduced to characterize the domain characteristics. The left-adjacent character entropy is used to measure the richness of the collocation of characters on the left side of each character, which can help identify entities more accurately. Based on the fusion of multiple features to characterize domain characteristics, the present invention provides a two-layer BiLSTM model multi-feature training method, which solves the problem of poor migration and low accuracy of the current traditional domain method when directly applied to the electric power dispatching text field. The present invention improves the effect of named entity recognition in electric power dispatching texts. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flowchart of the method for identifying entities in electric power dispatching text by integrating multiple features provided by the present invention;

[0046] Figure 2 A schematic diagram of feature fusion provided by an embodiment of the present invention;

[0047] Figure 3 Schematic diagram of the principle of a double-layer BiLSTM provided in an embodiment of the present invention;

[0048] Figure 4 A schematic diagram of the internal structure of the LSTM layer provided in an embodiment of the present invention;

[0049] Figure 5 A schematic diagram of the tag prediction module flow provided by an embodiment of the present invention;

[0050] Figure 6 This is a comparison chart of the overall performance of the method of the present invention and other methods provided in the embodiments of the present invention. DETAILED DESCRIPTION

[0051] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0052] The texts in the power system, especially the power dispatch texts, have strong domain characteristics, including: (1) the power dispatch text sentences have irregular characteristics. The power dispatch texts do not have a very strict sentence rule constraint. The dispatch instruction sentences issued by different dispatchers have different styles, and the dispatch sentences in different regions also have differences; (2) there are a large number of abbreviations and abbreviations in the dispatch sentences, and there are many simplified expressions; (3) unlike the sentences in the traditional text field, the length of each sentence in the power dispatch text is relatively different, and the length of each sentence can even differ by more than 5 times; (4) the entity structure is complex. Chinese characters, English letters, Arabic numerals, Roman characters, etc. exist in the power dispatch text entities, and special characters also constitute entities. The accuracy of the existing entity recognition method is difficult to meet the needs of downstream applications. In order to further improve the effect of named entity recognition, the present invention adopts word-granular training corpus to avoid the noise generated by word segmentation in model training, and proposes a method for entity recognition using a two-layer BiLSTM neural network model and the fusion of multiple features, which improves the recognition effect of power dispatch text entity recognition and solves the problem of poor transferability of traditional methods in the power dispatch text field.

[0053] Named entity recognition refers to identifying nouns with practical meaning in a text. In an embodiment of the present invention, for example, in the text "Xinjiekou Substation: 1# substation 1# main transformer is supplied by 888 line." "Xinjiekou Substation", "888 line" and so on are all named entities in the power dispatching text.

[0054] The present invention provides a method for identifying entities in electric power dispatching texts by integrating multiple features. The identification process is as follows: Figure 1 As shown. It includes the following steps:

[0055] In step S001, the unstructured power dispatch text is processed, noise reduction and cleaning are performed based on the characteristics of domain entities, and word-level entity annotation and feature annotation are performed to obtain a word-level corpus. The corpus contains three columns of sequences, namely, characters, entity labels, and parts of speech. The label results of the second column of sequences can be predicted by training the first column of sequence data.

[0056] In an embodiment of the present invention, data cleaning and denoising include word segmentation and stop word removal. In order to avoid noise caused by word segmentation to downstream tasks, this application does not adopt the traditional word segmentation method, but segments the sentences by word; after segmentation by character, there are punctuation marks that have no practical meaning, so they need to be removed. Combined with some existing stop word lists (such as the Harbin Institute of Technology stop word list, the Sichuan University Machine Intelligence Laboratory stop word library, Baidu stop word list, etc.), a stop word list suitable for power dispatching text natural language processing tasks is constructed to filter and reduce noise characters in text data.

[0057] In the three-column sequence: character, entity label, and part of speech, the first column is the character to be predicted, and the second column is its actual correct label value. The purpose of training is to predict the corresponding second column value through the first column.

[0058] Each word in the first column of the corpus is used to characterize the features of the sentence, also known as semantic features. In the embodiment of the present invention, word-granular annotation is adopted, and the "BIO" annotation method is adopted. The specific annotations are shown in Table 1.

[0059] Table 1 Entity annotation description

[0060] Entity Class Annotation Station location B-LOC, I-LOC equipment B-EQU、I-EQU line B-LIN、I-LIN transformer B-TRA、I-TRA

[0061] For example, for the sentence “Xinjiekou substation: the adjustment is provided by Zhangxin 888 line.”, the annotation results are shown in Table 2.

[0062] Table 2 Annotation examples

[0063] character Entity labeling new B-LOC street I-LOC mouth I-LOC Change I-LOC tune O Depend on O open B-LIN new I-LIN 8 I-LIN Wire I-LIN for O

[0064] After word tagging, part-of-speech tagging is performed. The part-of-speech tags in the third column of the corpus are the part-of-speech features. The present invention processes the part-of-speech tags according to word granularity to obtain part-of-speech features. The specific parts of speech and their annotations in the corpus are shown in Table 3.

[0065] Table 3 Parts of speech and their corresponding annotations

[0066]

[0067]

[0068] Continuing with the example of “Xinjiekou change: tune supplied by Zhangxin 888 line.”, the results of part-of-speech tagging are shown in Table 4.

[0069] Table 4 Part-of-speech tagging examples

[0070] character Entity labeling Part-of-speech tags new B-LOC NN street I-LOC NN mouth I-LOC NN Change I-LOC NN tune O NV Depend on O NV open B-LIN NN new I-LIN NN 8 I-LIN NN Wire I-LIN NN for O NV

[0071] Step S002: pre-train the first column of the corpus obtained above at word granularity to obtain word vector representations, and use the one-hot vector representation method to obtain part-of-speech feature vector representations for the second and third columns;

[0072] Specifically, the vector representations of various inputs include the following:

[0073] In order to represent context information, the present embodiment adopts a character-level embedding method and adopts character-level word vector pre-training. The skip-gram model in the Word2vec algorithm can pre-train word vectors with domain characteristics. The present invention constructs a character-level word vector pre-training model for regulating text based on Skip-gram. Here, the character-level word vector pre-training model for regulating text refers to the present invention converting the input in Skip-gram into characters for pre-training. The process of the Skip-gram algorithm is prior art and is not the core of the present invention, so it will not be elaborated here. The word vector of each sentence after pre-training is: X i ={x i,1 ,x i,2 ,x i,3 ,…,x i,n},in x i,n is the vector of the nth word in the i-th sentence, and de is the dimension of the input vector.

[0074] The vector representation of part-of-speech features uses the One-hot encoding format, and the part-of-speech of each word is represented by a 10-dimensional vector, namely β i =(0,0,0,…,1,0,0…),

[0075] Step S003, adding the left neighbor word entropy value obtained from the first sequence to the above word vector and part-of-speech feature vector to perform feature fusion;

[0076] The multiple features introduced in the present invention include semantic features, part-of-speech features, and left-adjacent character entropy features. The semantic features and part-of-speech features are obtained in step S002. On the basis of these two types of features, the left-adjacent character entropy feature is added to characterize the domain features, which is defined as follows:

[0077] Entropy is a measure used to represent the uncertainty of random variables. In information science, it is believed that all information has redundancy, and the amount of redundancy is related to the probability of each character appearing. Therefore, borrowing the concept of thermodynamics, the average amount of information after excluding redundancy is called "information entropy", which is defined as:

[0078] H(X)=-∑ x∈X P(x)log2(P(x)) (1)

[0079] Where P(x) is the probability of x occurring. In natural language processing, x is a character or a word. In the definition of this invention, x is a character to reflect the uncertainty of information.

[0080] Information entropy is a measure of the overall amount of information, while the concept of left-right entropy is used to measure the richness of word combinations. Left-right entropy refers to the entropy of the left and right boundaries of word expression.

[0081] Specifically, the calculation process of the entropy value of the left neighboring word based on the first sequence is as follows:

[0082] In this embodiment, the character-level left neighbor entropy is proposed to measure the richness of the collocations on the left side of each word. The formula is as follows:

[0083]

[0084] Where C is each word in the first column of the corpus in step S001, aC is the word that appears on the left side of word C, A is the set of all words on the left side of word C in the corpus, and P(aC|C) is the probability that word aC appears on the left side of C. The larger the left neighbor word entropy is, the richer the words on the left side of the word are, which means that the two words may be a group of entity words. For E L If (C)=0, it means that it may be a boundary.

[0085] like Figure 2 As shown, the steps of the multi-feature fusion method in this embodiment are as follows:

[0086] First, the word vector sequence X of a sentence obtained in step S002 is i , part-of-speech feature vector sequence β, left neighbor word entropy value E L Divided into two groups: one group input is a sequence of word vectors and a sequence of part-of-speech features, and the other group input is a sequence of word vectors and a sequence of entropy values ​​of the corresponding left neighboring words;

[0087] Then, the two sequences in the two sets of input are concatenated;

[0088] Finally, we get two sets of input values: input 1,j ={x j β j}, input 2,j ={x j ;E L (i)}, where j is the index of the jth word in the i-th sentence. j The vector representing the jth word in the i-th sentence.

[0089] Step S004: After feature fusion, the feature is input into a two-layer neural network model to obtain an output;

[0090] In the embodiment, step S003 obtains two sets of input values. In order to fully explore the feature information of the introduced features, such as Figure 3As shown, this embodiment provides a two-layer BiLSTM (Bi-directional Long Short-Term Memory) model. According to the above steps, a text sequence input word vector is The word vector of a word The part-of-speech embedding vector is The entropy of the left neighbor of this word is n is the number of characters in the sequence, then the two sets of input are:

[0091]

[0092] The two-layer bidirectional LSTM training process in this embodiment is to input the word vector through two layers of bidirectional LSTM to obtain the hidden layer state and then perform weighted fusion. Then, the new hidden layer state vector of the weighted result is passed through a neuron to obtain the predicted label probability matrix. The specific steps are as follows:

[0093] First, two sets of inputs are passed through the BiLSTM model to get the output: k is the hidden layer dimension of LSTM. h 1,t ,h 2,t The arrows above represent the hidden layer states obtained by forward propagation and backward propagation respectively.

[0094] Secondly, h 1,t ,h 2,t Output to the same hidden layer, we get: t =(h 1,t W L1 +b L1 )+(h 2,t W L2 +b L2 );

[0095] Finally, after the hidden layer fusion, the output prediction probability vector of the network model is obtained: s = o t W s +b s ;

[0096] Among them, W L1 ,W L2 ,b L1 ,b L2 and W s ,b s These are all parameters that need to be learned. l is the number of types of labels in the annotation set.

[0097] RNN is an algorithm for processing a series of long sequence information. However, traditional RNN cannot handle long-distance dependencies well when processing long sequence text. Therefore, Bidirectional Long Short-Term Memory Network (BiLSTM) was proposed to solve this problem. In this embodiment, the LSTM structure is as follows: Figure 4 As shown, the calculation formula is:

[0098]

[0099] Among them, σ is the sigmoid function, i t ,f t ,o t ,c t They are the input gate, forget gate, output gate and neuron state vectors, h t is the hidden state vector of the t-th input, i t ,f t ,o t ,c t With h t The dimensions of are consistent. W is the weight matrix, and the subscripts refer to the various parts. The weight W of each element and the bias b are the parameters to be learned.

[0100] Step S005: Input the above output results into the conditional random field model to obtain a predicted label sequence, thereby realizing named entity recognition of the sequence.

[0101] In the embodiment, the hidden state h is obtained by BiLSTM. t In order to avoid the loss of contextual meaning in label prediction, for example, it is obvious that the label “I” cannot be preceded by the label “O”, and “I-EQU” cannot be followed by “I-LIN”. To avoid the above situation, this embodiment adopts the conditional random field model (CRF) to obtain the predicted label sequence, such as Figure 5 As shown, the specific process is:

[0102] First, according to the label probability obtained in step S004, the score matrix obtained by outputting a word sequence in step S004 is P, l is the number of types of labels in the annotation set;

[0103] Next, we get the score for the sequence prediction:

[0104]

[0105] Among them, A is the transition probability matrix of the labels at each adjacent position, where the element A i,j It represents the probability of transferring from label i to label j, that is, the probability of transferring from the current word annotation result to the next word annotation result, and Y is the true label value;

[0106] Then, we get the probability value of the correct prediction of the sequence among all the prediction results:

[0107]

[0108] Finally, in order to maximize the probability, the maximum likelihood estimation is used for training. The formula is:

[0109] The maximum prediction probability output matrix of the sequence obtained by decoding is: where Y X It is the sequence of all labels (including those that do not conform to the labeling rules) of the input X. After the maximum probability matrix of the sequence prediction result is obtained from the conditional random field output, the label of the sequence is output in combination with the label set.

[0110] The above examples describe the implementation of the proposed method for entity recognition in electric power dispatch text, which integrates multiple features. To verify the performance of the present invention, the following comparative experiments were conducted. Comparisons with several traditional methods demonstrate that the proposed method for entity recognition in electric power dispatch text, which integrates multiple features, achieves superior recognition results.

[0111] Specifically, on the corpus constructed by the present invention, the present invention sets up four experimental groups to demonstrate the superiority of the model proposed by the present invention. The first group is the baseline group (Baseline), which adopts the BiLSTM+CRF model with only character embedding; the second group adds the embedding of part-of-speech features to the input layer on the basis of Baseline, in order to compare the effect of the added part-of-speech features on entity recognition; the third group adds the left neighboring word entropy features to the single-layer input on the basis of the second group; the fourth group is the double-layer neural network model that integrates multiple features proposed by the present invention. In order to evaluate the performance of the model, a general evaluation system, namely the precision rate P, recall rate R, and F1 value, is adopted. Set TP as the number of entities identified and correct, NP as the number of entities identified but wrong, and N as the number of all entities that should be identified. The precision rate can reflect the accuracy of entity recognition in the identified entities, and the statistical process ignores those parts that are not identified as entities, while the recall rate can reflect the ability of the model to extract entities, that is, whether the entities can be identified as much as possible. F1 is a comprehensive evaluation index of the above two indicators, and the calculation formula is as follows:

[0112]

[0113]

[0114]

[0115] The experimental results show that the overall indicators of each group are compared. Figure 6 The recognition of various entities is shown in Table 5. Figure 6It can be seen that after adding part-of-speech features, the recognition accuracy is improved from 89.43% to 90.32%, and the model proposed in this invention improves the overall accuracy by 2.78%, and the recall rate and F1 value are significantly improved.

[0116] Table 5 Recognition of various entities

[0117]

[0118]

[0119] The present invention also provides a multi-feature integrated power dispatching text entity recognition device, comprising:

[0120] The corpus construction module is used to perform noise reduction and cleaning on the unstructured power dispatch text, perform word-level entity tagging and part-of-speech feature tagging, and obtain a word-level corpus. The corpus contains three columns: word, entity tag, and part-of-speech.

[0121] The vector acquisition module is used to pre-train the first column of the obtained corpus at word granularity to obtain word vector representations, and to obtain part-of-speech feature vector representations of the second and third columns using the one-hot vector representation method;

[0122] The feature fusion module is used to add the left neighbor word entropy value obtained from the first column sequence to the word vector and part-of-speech feature vector to perform multi-feature fusion;

[0123] The first prediction module is used to input the vector obtained after feature fusion into the double-layer neural network model to obtain the output;

[0124] The second prediction module is used to input the output results of the two-layer neural network model into the conditional random field model to obtain the predicted label sequence, thereby realizing named entity recognition of the sequence.

[0125] It should be understood that the power dispatching text entity recognition device integrating multiple features in the embodiment of the present invention can implement all the technical solutions in the above method embodiment, and the functions of its various functional modules can be specifically implemented according to the method in the above method embodiment. The specific implementation process can refer to the relevant description in the above embodiment, and will not be repeated here.

[0126] The present invention also provides a computer device, comprising:

[0127] one or more processors;

[0128] Memory; and

[0129] One or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the power dispatch text entity recognition method integrating multiple features as described above are implemented.

[0130] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned method for entity recognition of electric power dispatching text by integrating multiple features.

[0131] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0132] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0133] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0134] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1A step that specifies a function in one or more boxes.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the scope of protection of the claims of the present invention.

Claims

1. A method for identifying entities in power dispatch text by integrating multiple features, characterized in that: The following steps are involved: The unstructured power dispatch text is cleaned and de-noised, and then labeled with word-level entities and part-of-speech features to obtain a word-level corpus. The corpus contains three columns: word, entity tag, and part-of-speech. The first column of the obtained corpus is pre-trained at word granularity to obtain word vector representations, and the second and third columns are represented by part-of-speech feature vectors using the one-hot vector representation method; On the basis of the word vector and part-of-speech feature vector, the left neighbor word entropy value obtained from the first column sequence is added to perform multi-feature fusion. The left neighbor word entropy value of the first column sequence is obtained according to the following formula: Where C is each word in the first column of the corpus, aC is the word that appears to the left of word C, A is the set of all words to the left of word C in the corpus, and P(aC|C) is the probability that word aC appears to the left of C; The method of multi-feature fusion includes: converting the word vector sequence X obtained from a sentence into i , part-of-speech feature vector sequence β, left neighbor word entropy value E L Divide into two groups: one group of input is a sequence of word vectors and part-of-speech features, and the other group of input is a sequence of word vectors and a sequence of entropy values ​​of the corresponding left neighboring words. The two sequences in the two groups of input are concatenated to obtain two new sets of vectors as the fused features, expressed as: input 1,j ={x j β j }, input 2,j ={x j ;E L (i)}, where j is the index of the jth word in the i-th sentence, x j The vector representing the jth word in the i-th sentence; The vector obtained after feature fusion is input into the two-layer neural network model to obtain the output; The output results of the two-layer neural network model are input into the conditional random field model to obtain the predicted label sequence, thereby realizing named entity recognition of the sequence.

2. The method for identifying entities in electric power dispatching text by integrating multiple features according to claim 1 is characterized in that: The word vector representation of the first column in the corpus is obtained by pre-training the word granularity. This includes building a word vector pre-training model for character-level control text based on Skip-gram. The word vector of each sentence in the first column after pre-training is: X i ={x i,1 ,x i,2 ,x i,3 ,…,x i,n },in x i,n is the vector of the nth word in the i-th sentence, n is the number of characters in the sequence, and de is the dimensional digital vector representation of the input vector.

3. The method for identifying entities in electric power dispatching text by integrating multiple features according to claim 1 is characterized in that: The part of speech of each word is represented by a 10-dimensional vector. The correspondence between the part of speech and its annotation in the corpus is as follows: verbs, marked as NV; Nouns, marked as NN; Conjunction, standard is NL; Particles, marked as NP; Adverbs, marked as NF; Prepositions, marked as PR; Numerals, marked as NU; Numbers, marked as SZ; Valid characters are marked as X; Invalid symbol, marked as U; The part-of-speech feature vector represented by the unique hot vector is expressed as: β i =(0,0,0,…,1,0,0…), 4. The method for identifying entities in electric power dispatching text by integrating multiple features according to claim 1 is characterized in that: The two-layer neural network model is a two-layer long short-term memory neural network. The two sets of vectors after feature fusion are input into two LSTM networks respectively, and the hidden layer states are obtained and then weighted fused. The new hidden layer state vector of the weighted result is then passed through a neuron to obtain the predicted label probability matrix.

5. The method for identifying entities in electric power dispatching text by integrating multiple features according to claim 4 is characterized in that: The predicted label sequences obtained using the conditional random field model include: The score of sequence prediction is obtained based on the label probability matrix P obtained from the two-layer neural network model: in, l is the number of types of labels in the annotation set; A is the transition probability matrix of labels at each adjacent position, where the element A i,j It represents the probability of transferring from label i to label j, that is, the probability of transferring from the current word annotation result to the next word annotation result, and Y is the true label value; The probability value of the correct prediction of the sequence in all prediction results is obtained according to the following formula: Training using maximum likelihood estimation: The maximum prediction probability output matrix of the sequence obtained by decoding is: Y X is the sequence of all labels of the input X; Based on the maximum prediction probability output matrix of the sequence, the label of the sequence is obtained in combination with the annotation set.

6. A multi-feature integrated power dispatch text entity recognition device, characterized in that: include: The corpus construction module is used to perform noise reduction and cleaning on the unstructured power dispatch text, perform word-level entity tagging and part-of-speech feature tagging, and obtain a word-level corpus. The corpus contains three columns: word, entity tag, and part-of-speech. The vector acquisition module is used to pre-train the first column of the obtained corpus at word granularity to obtain word vector representations, and to obtain part-of-speech feature vector representations of the second and third columns using the one-hot vector representation method; The feature fusion module is used to add the left neighbor word entropy value obtained from the first column sequence to the word vector and part-of-speech feature vector to perform multi-feature fusion. The left neighbor word entropy value of the first column sequence is obtained according to the following formula: Where C is each word in the first column of the corpus, aC is the word that appears to the left of word C, A is the set of all words to the left of word C in the corpus, and P(aC|C) is the probability that word aC appears to the left of C; The method of multi-feature fusion includes: converting the word vector sequence X obtained from a sentence into i , part-of-speech feature vector sequence β, left neighbor word entropy value E L Divide into two groups: one group of input is a sequence of word vectors and part-of-speech features, and the other group of input is a sequence of word vectors and a sequence of entropy values ​​of the corresponding left neighboring words. The two sequences in the two groups of input are concatenated to obtain two new sets of vectors as the fused features, expressed as: input 1,j ={x j β j }, input 2,j ={x j ;E L (i)}, where j is the index of the jth word in the i-th sentence, x j The vector representing the jth word in the i-th sentence; The first prediction module is used to input the vector obtained after feature fusion into the double-layer neural network model to obtain the output; The second prediction module is used to input the output results of the two-layer neural network model into the conditional random field model to obtain the predicted label sequence, thereby realizing named entity recognition of the sequence.

7. A computer device, characterized in that: include: one or more processors; Memory; as well as One or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the power dispatch text entity recognition method integrating multiple features as described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for entity recognition of electric power dispatching text integrating multiple features are implemented as described in any one of claims 1 to 5.

Images