Power equipment triplet construction method based on deep learning algorithm

Abstract

The present invention relates to the field of artificial intelligence. A method for constructing a power equipment triple based on a deep learning algorithm is disclosed. The method for constructing a power equipment triple based on a deep learning algorithm proposed in the present invention provides a more complete power equipment knowledge graph triple extraction scheme, that is, constructing a pattern layer in a top-down construction manner, and under the guidance of the pattern layer, constructing a data layer in a bottom-up manner; this scheme clearly reflects the characteristics of power equipment text, can improve the design scheme of power equipment triples, and thus improve the extraction efficiency and accuracy of power equipment triples; the present invention also proposes an entity relationship extraction model including a bidirectional recurrent network, an expansion gate convolutional neural network and a self-attention model; the model is constructed based on a bottom-up constructed data layer, and the idea of ​​a probability graph is used to extract power equipment triples, which further improves the efficiency and accuracy of power equipment triple extraction.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method for constructing a triplet of power equipment based on a deep learning algorithm. Background Art

[0002] Knowledge graph is a knowledge representation method, which is essentially a structured semantic knowledge base. It models entities in the objective world and their relationships in the form of triples (i.e., head entity h, relation r, and tail entity t). These triples are connected to each other through common entities or attributes to form a network knowledge structure.

[0003] Compared with traditional knowledge organization and management methods, the graph-based data organization structure of knowledge graph triples supports more efficient data retrieval, can handle complex and diverse association representations, and can simulate the human thinking process for semantic analysis.

[0004] In the field of power equipment knowledge graph technology, constructing knowledge graph triples of power equipment lays the foundation for intelligent operation and maintenance of power equipment; one of the difficulties in constructing professional knowledge graphs in specific fields is the acquisition of knowledge data. For example, in the field of power equipment, there are few sources of structured and semi-structured professional knowledge, resulting in the lack of a complete solution to extract knowledge graph triples of power equipment. Summary of the invention

[0005] The main purpose of the present invention is to provide a method for constructing triples of power equipment based on a deep learning algorithm, aiming to solve the problem that there is currently no perfect solution to extract triples of power equipment knowledge graphs.

[0006] The technical solution proposed by the present invention is:

[0007] A method for constructing a power equipment triplet based on a deep learning algorithm, comprising:

[0008] A triple extraction model based on data flow programming: a pattern layer is constructed in a top-down manner, and under the guidance of the pattern layer, a data layer is constructed in a bottom-up manner, wherein the pattern layer is a knowledge organization framework for knowledge extraction, and is a data model that describes entities, relationships between entities, and attributes, and the data layer includes a named entity extraction model and an entity relationship extraction model;

[0009] Using the named entity extraction model to perform electric power equipment named entity recognition on the text to be extracted, so as to realize electric power equipment named entity extraction, and annotate to obtain annotated entities;

[0010] Based on the labeled entities, the entity relationship extraction model is used to extract the power equipment entity relationship of the text to be extracted to extract the power equipment triples, wherein the entity relationship extraction model includes a bidirectional recurrent network, an expansion gate convolutional neural network and a self-attention model.

[0011] Preferably, based on the annotated entity, extracting the power equipment entity relationship of the text to be extracted by the entity relationship extraction model to extract the power equipment triples includes:

[0012] The entities annotated by the named entity extraction model are used as an input of the entity relationship extraction model to obtain a first result, and the feature vector obtained by retraining the text to be extracted is used as another input of the entity relationship extraction model to obtain a second result. The first result and the second result are spliced ​​and passed into a convolutional neural network to extract the power equipment triple.

[0013] Preferably, the named entity extraction model is used to perform electric power equipment named entity recognition on the text to be extracted to realize the extraction of electric power equipment named entities, and annotates to obtain annotated entities, including:

[0014] Build a vocabulary database in the field of electricity;

[0015] Get the training set text sequence;

[0016] Based on the training set text sequence, the spatial vector of each word in the power field vocabulary is calculated by the word2vec model, and the word vector of the training set text sequence is calculated by the word embedding layer;

[0017] Obtaining the word mixed Embedding vector sequence encoding by a word mixed Embedding method;

[0018] Add a Position Embedding vector with the same dimension as the word vector sequence to make the position information of the encoded vector sequence more obvious;

[0019] Inputting the word vector sequence encoding into a 12-layer expansion gate convolutional neural network with expansion rates of 1, 2, 5, 1, 2, 5, 1, 2, 5, 1, 1, 1 for learning, so as to output a first sequence;

[0020] Pass the first sequence into a self-attention layer to obtain a second sequence;

[0021] The second sequence is passed into the output of the fully connected layer, and the first position of the entity and the tail position of the entity are predicted using a half-pointer and half-annotation structure to obtain a labeled entity.

[0022] Preferably, the construction of the electric power field vocabulary includes:

[0023] Establish a dictionary tree index, count the frequency of each character and bigram in the text to be extracted, and mark it as word frequency, and take bigrams as candidate words in descending order of word frequency, where the bigram consists of each character and its right adjacent character;

[0024] Get the point mutual information threshold, left and right information entropy threshold, word frequency threshold, and maximum word length threshold;

[0025] Calculate the point mutual information of candidate words;

[0026] When the point mutual information of a candidate word is greater than the point mutual information threshold, the candidate word is marked as a to-be-selected word;

[0027] Calculating the left information entropy and the right information entropy of the words to be selected;

[0028] If the left information entropy of the word to be selected is greater than the left and right information entropy thresholds, and the right information entropy of the word to be selected is greater than the left and right information entropy thresholds, and the word frequency of the word to be selected is greater than the word frequency threshold, the word to be selected is stored in the electric power field word library;

[0029] If the right information entropy of the candidate word is less than the left and right information entropy thresholds, the candidate word is expanded to the right to obtain a right-expanded word, and the length of the right-expanded word is less than the maximum word length threshold, and then the calculation of the left information entropy and the right information entropy of the candidate word and the subsequent steps are performed again;

[0030] If the left information entropy of the candidate word is less than the left and right information entropy thresholds, expand the candidate word to the left to obtain a left-expanded word, and the length of the left-expanded word is less than the maximum word length threshold, and then perform the calculation of the left information entropy and the right information entropy of the candidate word again, and the subsequent steps.

[0031] Preferably, the calculation formula for calculating the point mutual information of the candidate words is:

[0032]

[0033] Among them, PMI (a i ,b j ) is the candidate word a in the text to be extracted i b j Point mutual information; P(a i ,b j ) is the adjacent character a of the candidate word in the text to be extracted i and character b j Combination of characters a i b j The probability of occurrence, P(a i) is the character a i The probability of appearing in the text to be extracted, P(b j ) is the character b j The probability of appearing in the text to be extracted.

[0034] Preferably, a Position Embedding vector with the same dimension as the word vector sequence is added to make the position information of each word in the power field vocabulary more obvious, and the calculation formula of Position Embedding is:

[0035]

[0036] The above formula maps the position with position id p to a d pos The i-th value of the position vector is PE i (p).

[0037] Preferably, the encoding of the vector sequence is input into a 12-layer expansion gate convolutional neural network with expansion rates of 1, 2, 5, 1, 2, 5, 1, 2, 5, 1, 1, 1 for learning to obtain a first sequence, including:

[0038] Set the vector sequence to be processed to z, and z = [z1, z2, ..., z n ];

[0039] The vector sequence encoding is input into the convolutional layer of the convolutional neural network, and a gate mechanism is added to the convolution during the convolution to obtain the first sequence of output results, wherein the expression of the first sequence is:

[0040]

[0041] Wherein, Y represents the first sequence, σ is the Sigmoid activation function, Conv1D1 and Conv1D2 are two one-dimensional convolutions of the same form, one of which is activated by the sigmoid activation function, and the other is not activated.

[0042] Preferably, the entity annotated by the named entity extraction model is used as an input of the entity relationship extraction model to obtain a first result, the feature vector obtained by retraining the text is used as another input of the entity relationship extraction model to obtain a second result, the first result and the second result are concatenated and passed into a convolutional neural network to extract a triple of power equipment, including:

[0043] Perform head entity prediction for each sentence in the extracted text;

[0044] The tail entity corresponding to the head entity is predicted through the head entity, and then the relationship between the input head entity and the tail entity is predicted through the head entity and the tail entity.

[0045] Preferably, predicting the tail entity corresponding to the head entity through the head entity, and then predicting the relationship between the input head entity and the tail entity through the head entity and the tail entity, includes:

[0046] Taking the labeled entity as input, randomly sampling a labeled entity;

[0047] Retraining the text to be extracted corresponding to the labeled entity, using a vector concatenated from word mixed Embedding and position Embedding to pass into a 12-layer dilation gate convolutional neural network with dilation rates of 1, 2, 5, 1, 2, 5, 1, 2, 5, 1, 1, 1, and a self-attention layer for learning, so as to obtain a third sequence;

[0048] The encoding vectors in the third sequence corresponding to the randomly sampled labeled entities are passed into the bidirectional recurrent network for encoding, and the encoding results of the relative positions are added to obtain a vector sequence of the same length as the input sequence;

[0049] Passing the third sequence into another self-attention layer to obtain an output result, and adding the output result to the first vector sequence to obtain a concatenation result;

[0050] The concatenation result is passed to the fully connected layer for output. When the relationship type is determined, for each relationship, the head position and the tail position of the tail entity are predicted respectively, so as to simultaneously predict the relationship between the head entity and the tail entity and output the result.

[0051] Preferably, based on the annotated entity, the entity relationship extraction model is used to extract the power equipment entity relationship of the text to be extracted to extract the power equipment triple, and then further includes:

[0052] Get the preset learning rate and preset times;

[0053] The accuracy of the triplet extraction model is obtained based on the preset learning rate and the preset number of times, and the triplet extraction model is evaluated based on the accuracy.

[0054] Through the above technical solution, the following beneficial effects can be achieved:

[0055] The method for constructing power equipment triples based on deep learning algorithm proposed in the present invention provides a more complete triple extraction scheme for power equipment knowledge graph, that is, constructing the pattern layer in a top-down manner, and under the guidance of the pattern layer, constructing the data layer in a bottom-up manner; this scheme clearly reflects the characteristics of the power equipment text, can enrich the power equipment triples, and thus improve the extraction efficiency of the power equipment triples.

[0056] In addition, the present invention also proposes an entity relationship extraction model including a bidirectional recurrent network, an expansion gate convolutional neural network and a self-attention model; the model is based on a top-down constructed pattern layer and adopts the idea of ​​probability graph to extract power equipment triplets, further improving the efficiency of power equipment triple extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0058] Figure 1 A flowchart of a first embodiment of a method for constructing a power equipment triplet based on a deep learning algorithm proposed by the present invention;

[0059] Figure 2 A schematic diagram of the structure of the mode layer and the data layer in the first embodiment of a method for constructing a power equipment triplet based on a deep learning algorithm proposed by the present invention;

[0060] Figure 3 This is a diagram showing the core points of the top-down construction mode layer in the first embodiment of the method for constructing a power equipment triplet based on a deep learning algorithm proposed by the present invention;

[0061] Figure 4 A schematic diagram of named entity extraction of electric power equipment in a fifth embodiment of a method for constructing electric power equipment triples based on a deep learning algorithm proposed by the present invention;

[0062] Figure 5 A schematic diagram of word mixed Embedding encoding in the fifth embodiment of a method for constructing a power equipment triplet based on a deep learning algorithm proposed by the present invention;

[0063] Figure 6 A schematic diagram of a model of an expansion gate convolutional up / down network (DGCNN model layer) in a fifth embodiment of a method for constructing a power equipment triplet based on a deep learning algorithm proposed by the present invention;

[0064] Figure 7 This is a basic structure diagram of a residual structure gate convolutional neural network in the seventh embodiment of a method for constructing a power equipment triplet based on a deep learning algorithm proposed by the present invention;

[0065] Figure 8 This is a basic network architecture diagram of the attention mechanism in the fifth embodiment of the method for constructing a triplet of power equipment based on a deep learning algorithm proposed in the present invention. DETAILED DESCRIPTION

[0066] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0067] The present invention proposes a method for constructing a triplet of power equipment based on a deep learning algorithm.

[0068] As attached Figure 1 As shown, in a first embodiment of a method for constructing a power equipment triplet based on a deep learning algorithm proposed by the present invention, this embodiment includes the following steps:

[0069] Step S110: A triple extraction model constructed based on data flow programming: a pattern layer is constructed in a top-down manner, and under the guidance of the pattern layer, a data layer is constructed in a bottom-up manner, wherein the pattern layer is a knowledge organization framework for knowledge extraction, and is a data model that describes entities, relationships between entities, and attributes, and the data layer includes a named entity extraction model and an entity relationship extraction model.

[0070] Specifically, as attached Figure 2 As shown in the figure, this method adopts a combination of top-down and bottom-up methods to construct the power equipment triple. By analyzing the content of the text to be extracted (power equipment text), the pattern layer of the triple is designed in a top-down manner; then, under the guidance of the pattern layer, the data layer is constructed in a bottom-up manner, and a suitable extraction method is designed according to the characteristics of the text to be extracted. The three knowledge elements of entity, relationship and attribute are extracted to form a series of high-quality factual expressions, which are then mapped to the pattern layer.

[0071] As attached Figure 3 As shown in the figure, the power equipment triplet pattern layer is constructed by five core elements: equipment name, fault name, fault defect phenomenon, fault cause, and fault handling method. The relationships between them include: "type", "includes", "occurrence", "includes", "cause is", and "handling measures", forming a (head entity h, relationship t, tail entity r) triplet.

[0072] Step S120: using the named entity extraction model to perform power equipment named entity recognition on the text to be extracted to achieve power equipment named entity extraction, and annotate to obtain annotated entity h.

[0073] Step S130: Based on the labeled entity h, the entity relationship extraction model is used to extract the power equipment entity relationship of the text to be extracted to extract the power equipment triples, wherein the entity relationship extraction model includes a bidirectional recurrent network, an expansion gate convolutional neural network and a self-attention model.

[0074] Specifically, according to the entity types and entity relationship types defined in the model layer, appropriate named entity extraction models and entity relationship extraction models are designed to build the data layer. Entity relationship extraction is a step to obtain structured knowledge such as entities, relationships between entities, and attributes from unstructured (semi-)structured data through a series of information extraction methods under the guidance of the knowledge organization architecture of the model layer.

[0075] The method for constructing power equipment triples based on deep learning algorithm proposed in the present invention provides a more complete triple extraction scheme for power equipment knowledge graph, that is, constructing the pattern layer in a top-down manner, and under the guidance of the pattern layer, constructing the data layer in a bottom-up manner; this scheme clearly reflects the characteristics of the power equipment text, can enrich the power equipment triples, and thus improve the extraction efficiency of the power equipment triples.

[0076] In addition, the present invention also proposes an entity relationship extraction model including a bidirectional recurrent network, an expansion gate convolutional neural network and a self-attention model; the model is based on a top-down constructed pattern layer and adopts the idea of ​​probability graph to extract power equipment triplets, further improving the efficiency of power equipment triple extraction.

[0077] In a second embodiment of a method for constructing a power equipment triplet based on a deep learning algorithm proposed by the present invention, based on the first embodiment, step S130 includes the following steps:

[0078] Step S210: Use the entity h annotated by the named entity extraction model as an input of the entity relationship extraction model to obtain a first result, use the feature vector obtained by retraining the text to be extracted as another input of the entity relationship extraction model to obtain a second result, concatenate the first result and the second result, and pass them into a convolutional neural network to extract the power equipment triple.

[0079] Specifically, the present invention designs an entity relationship extraction model including a bidirectional recurrent network, an expansion gate convolutional neural network and a self-attention model to realize entity relationship extraction of the text to be extracted.

[0080] As attached Figure 4 As shown, in a third embodiment of a method for constructing a power equipment triplet based on a deep learning algorithm proposed by the present invention, based on the second embodiment, step S120 includes the following steps:

[0081] Step S310: Text preprocessing: preprocess the text to be extracted to delete pictures, tables, clauses and sentences irrelevant to the extraction of power equipment triples in the text to be extracted.

[0082] Step S320: constructing a vocabulary database in the electric power field.

[0083] Step S330: Obtain training set text sequence.

[0084] Step S340: Calculate the spatial vector of each word in the power field vocabulary based on the training set text sequence through the word2vec model.

[0085] Specifically, the Word2vec model is a group of related models used to generate word vectors. These models are shallow, two-layer neural networks that are trained to reconstruct linguistic word text. The network is represented by words and needs to guess the input words in adjacent positions. Under the word bag model assumption in word2vec, the order of words is not important. After training, the word2vec model can be used to map each word to a vector, which can be used to represent the relationship between words. The vector is the hidden layer of the neural network.

[0086] Step S350: Obtain the word mixing Embedding vector sequence encoding through word mixing Embedding.

[0087] Specifically, in order to make the encoding more effectively incorporate semantic information, this application selects the method of mixed word embedding to obtain the vector encoding, the mixed encoding of characters + words, and accurate word segmentation is very important, so the above-mentioned power field vocabulary is added during text segmentation to improve the accuracy of power field word segmentation.

[0088] As attached Figure 5 As shown, taking "medium voltage side winding is seriously deformed" in the text to be extracted as an example, "medium voltage side winding" and "severe deformation" are both segmentation result words.

[0089] First, the word embedding layer is used to obtain the word vector sequence of each word in the sentence. Then, the word segmentation results "medium voltage side winding" and "severe deformation" are used through the Word2Vec model to obtain the corresponding word vectors. The "medium voltage side winding" word vector is repeated 5 times, and the "severe deformation" word vector is repeated 4 times. Then, through a matrix transformation, each word vector is converted into the same dimension as the corresponding word vector, and then the two are added together with the separately encoded word vector to obtain the final encoding result.

[0090] Step S360: adding a Position Embedding vector with the same dimension as the word vector sequence to make the position information of each word in the power field vocabulary more obvious.

[0091] Step S370: Input the word vector sequence encoding into a 12-layer dilation gate convolutional neural network (DGCNN model layer) with dilation rates of 1, 2, 5, 1, 2, 5, 1, 2, 5, 1, 1, 1 for learning, so as to output a first sequence Y.

[0092] Specifically, the main computational task of the convolutional neural network CNN is to perform convolution operations. The difference between dilated convolution and ordinary convolution is that dilated convolution can perform convolution operations on longer input sequences by skipping δ input sequence widths.

[0093] Compared with the two, the dilated convolution uses the dilation rate parameter to indicate the size of the expansion. When the sizes of the convolution kernels of the two are the same, the dilated convolution will have a larger receptive field because it skips a certain sequence width at each layer, which means that it can expand the character context width and thus better extract the global information of the entire text sequence.

[0094] As attached Figure 6 As shown in the figure, dilated convolution can obtain more distant information in the input sequence with fewer parameters and computation. Therefore, in natural language processing tasks, using dilated convolution can improve computational efficiency. For input sequences of the same length, using dilated convolution can reduce the number of neural network layers and simplify the model. Therefore, this paper selects dilated convolution as the convolution model, reduces the number of model layers by adjusting the dilation rate, and extracts the features of the entire sequence with fewer convolution layers.

[0095] Step S380: Pass the first sequence Y into a self-attention layer to obtain a second sequence Y'.

[0096] Specifically, the basic network architecture of the attention mechanism in step S560 is as shown in the attached figure. Figure 8 shown.

[0097] The general expression of the attention mechanism is as follows:

[0098]

[0099] Among them, d k is the dimension of the input information. When this dimension is large, the dot product will also be large, which may cause the gradient of the sigmoid activation function to disappear. Therefore, this formula is factored To adjust, use Scale the dot product to get the attention score function in scaled dot product form.

[0100] The attention mechanism can be regarded as a focus on the input weight distribution. The text to be extracted is regarded as a sequence of key-value pairs, with K = (k1,...,k N ) and V=(v1,...,v N ) represent the key sequence and value sequence respectively, and Q=(q1,...,q N ) represents the query sequence, where Q corresponds to query, K corresponds to key, and V corresponds to value.

[0101] The self-attention model can be regarded as a special form of the attention model. The self-attention model is the case when the query, key and value are equal. The input sequence is the output sequence. The attention mechanism is performed within the sequence to calculate the weight of the sequence itself, so as to find the connection within the sequence.

[0102] The self-attention model can directly mine the semantic combination relationship of words within a sentence, obtain the syntactic and semantic characteristics between words, and better utilize the information of word combinations or even phrases in natural language processing tasks, which significantly improves the effect of neural network processing text information, and thus is recognized in various natural language processing tasks. Therefore, this application selects the Self-Attention mechanism for word vector processing, fully considering the semantic and grammatical connections between sentences and different words, so as to improve the accuracy and efficiency of triple extraction.

[0103] Step S390: passing the second sequence Y' to the output of the fully connected layer (Dense layer), and using the half-pointer and half-annotation structure to predict the first position of the entity h and the tail position of the entity h to obtain the annotated entity h.

[0104] Specifically, the output labeled entity h is saved and output as the input of the subsequent entity relationship extraction model.

[0105] Specifically, through the above technical solution, the present invention proposes a method for extracting named entity recognition of power equipment based on expansion gate convolutional neural network and self-attention mechanism algorithm. This method is based on the electric power field vocabulary, combined with word mixed vector, position vector and deep learning algorithm, which lays the foundation for the extraction of entity relationship of power equipment, improves the efficiency of triple extraction of power equipment, and improves the accuracy of triple extraction.

[0106] In a fourth embodiment of a method for constructing a power equipment triplet based on a deep learning algorithm proposed by the present invention, based on the third embodiment, step S320 includes the following steps:

[0107] Step S401: Establish a dictionary tree index, count the frequency of each character and bigram in the text to be extracted, and mark it as word frequency, and take out bigrams as candidate words in descending order of word frequency, where the bigram consists of each character and its right adjacent character.

[0108] Step S402: Obtain the point mutual information threshold PMI, the left and right information entropy threshold AE, the word frequency threshold WF, and the maximum word length threshold LW.

[0109] Step S403: Calculate the point mutual information of the candidate words.

[0110] Specifically, the point mutual information of each character segment in the text to be extracted is used to determine whether the character segment forms a word, that is, the point mutual information is used as the criterion for whether it can form a word. The point mutual information represents the connection between two Chinese characters. If the two characters are closely connected, they can form a word.

[0111] Step S404: When the point mutual information of a candidate word is greater than the point mutual information threshold, the candidate word is marked as a to-be-selected word.

[0112] Step S405: Calculate the left information entropy and the right information entropy of the word to be selected.

[0113] Specifically, left-right information entropy is to reflect whether a word has rich left-right collocations by calculating the information entropy of the left and right sides of a character segment. If it reaches a certain threshold, it can be considered that the character segment can become a word. Information entropy is defined as follows:

[0114]

[0115] Where X represents a character segment, x represents the left or right collocation of the word, and P(x) represents the probability of the collocation.

[0116] Step S406: If the left information entropy of the candidate word is greater than the left and right information entropy thresholds AE, and the right information entropy of the candidate word is greater than the left and right information entropy thresholds AE, and the word frequency of the candidate word is greater than the word frequency threshold WF, the candidate word is stored in the electric power field vocabulary.

[0117] Step S407: If the right information entropy of the candidate word is less than the left and right information entropy threshold AE, expand the candidate word to the right to obtain a right-expanded word, and the length of the right-expanded word is less than the maximum word length threshold LW, and then execute step S307 and subsequent steps again.

[0118] Step S408: If the left information entropy of the candidate word is less than the left and right information entropy threshold AE, expand the candidate word to the left to obtain a left-expanded word, and the length of the left-expanded word is less than the maximum word length threshold LW, and then execute step S307 and subsequent steps again.

[0119] At the same time, the electric power field vocabulary can also be completed using the Hanlp Chinese natural language processing word segmentation method and the manual extraction method.

[0120] Specifically, Hanlp Chinese natural language processing word segmentation methods include: standard word segmentation, NLP word segmentation, index word segmentation, N-shortest path word segmentation, CRF word segmentation and ultra-fast dictionary word segmentation.

[0121] Manual extraction methods are also indispensable for establishing domain dictionaries, which can improve the accuracy of domain dictionaries and enrich domain dictionaries. They can also filter the word segmentation results of the information entropy segmentation algorithm and the HanLP word segmentation system to filter out words that do not conform to the power domain dictionary, and can also artificially add domain words to enrich the power domain vocabulary.

[0122] In a fifth embodiment of a method for constructing a power equipment triplet based on a deep learning algorithm proposed by the present invention, based on the fourth embodiment, in step S403, the calculation formula for calculating the point mutual information of the candidate words is:

[0123]

[0124] Among them, PMI (a i ,b j ) is the candidate word a in the text to be extracted i b j Point mutual information; P(a i ,b j ) is the adjacent character a of the candidate word in the text to be extracted i and character b j Combination of characters a i b j The probability of occurrence, P(a i ) is the character a i The probability of appearing in the text to be extracted, P(b j ) is the character b j The probability of appearing in the text to be extracted.

[0125] In the sixth embodiment of a method for constructing a power equipment triplet based on a deep learning algorithm proposed by the present invention, based on the third embodiment, the calculation formula of Position Embedding in step S360 is:

[0126]

[0127] The above formula maps the position with position id p to a d pos The i-th value of the position vector is PE i (p).

[0128] In a seventh embodiment of a method for constructing a power equipment triplet based on a deep learning algorithm proposed by the present invention, based on the third embodiment, step S370 includes the following steps:

[0129] Step S710: Set the vector sequence to be processed as z, where z=[z1, z2, ..., z n ].

[0130] Step S710: Input the vector sequence encoding into the convolutional neural network convolutional layer, and add a gate mechanism to the convolution during convolution to obtain the output result first sequence Y, wherein the expression of the first sequence Y is:

[0131]

[0132] Wherein, Y represents the first sequence; σ is the Sigmoid activation function, Conv1D1 and Conv1D2 are two one-dimensional convolutions of the same form, one of which is activated with the sigmoid activation function, and the other is not activated, and then tensor multiplication operation is performed on them.

[0133] Specifically, most convolution operations in neural networks appear in the form of two-dimensional convolution (Conv2), but because the objects processed by natural language processing tasks are mostly text, convolution operations in natural language processing tasks appear in the form of one-dimensional convolution (Conv1).

[0134] Since the value range of the activation function sigmoid is between 0 and 1, it is equivalent to adding a gate mechanism that can control the flow to each output of Conv1D. Because there is a convolution without any activation function, this can reduce the possibility of gradient disappearance.

[0135] In addition, as attached Figure 7 As shown in the figure, since the dimensions of the input and output are the same, the residual structure can be used to combine the residual with the gated convolutional neural network, and the input and output can be added using the residual structure. This structure can be regarded as an activation function as a whole, called a Gated Linear Unit. The information of the input sequence has a 1-δ probability of being transmitted directly, and has a δ probability of being transmitted after being transformed, thereby achieving the effect of multi-channel information transmission.

[0136] In an eighth embodiment of a method for constructing a power equipment triplet based on a deep learning algorithm proposed by the present invention, based on the seventh embodiment, step S210 includes the following steps:

[0137] Step S810: Predict the head entity h for each sentence in the extracted text.

[0138] Specifically, the present invention extracts triples based on a probability graph idea to predict the head entity h of each sentence in the extracted text.

[0139] Step S820: predict the tail entity t corresponding to the head entity h through the head entity h, and then predict the relationship r between the input head entity h and the tail entity t through the head entity h and the tail entity t, and the calculation formula is:

[0140] P(h,r,t)=P(h)P(t|h)P(r|h,t),

[0141] Among them, P(h) represents the probability of entity h, P(t|h) represents the probability that t is in the relationship of entity h, and P(r / h,t) represents the probability that r is the tail entity of h,t. Since there are multiple entities to be extracted for one input, the sigmoid function is chosen to activate it during the extraction process.

[0142] In a ninth embodiment of a method for constructing a power equipment triplet based on a deep learning algorithm proposed by the present invention, based on the eighth embodiment, step S820 includes the following steps:

[0143] Step S910: taking the annotated entity h as input, and randomly sampling a annotated entity h.

[0144] Step S920: Retrain the text to be extracted corresponding to the labeled entity h, and use the vector concatenated from the word mixed Embedding and the position Embedding to pass into a 12-layer dilation gate convolutional neural network with dilation rates of 1, 2, 5, 1, 2, 5, 1, 2, 5, 1, 1, 1, and a self-attention layer for learning to obtain a third sequence H.

[0145] Step S930: the encoding vector in the third sequence H corresponding to the randomly sampled labeled entity h is passed into the bidirectional recurrent network for encoding, and the encoding result of the relative position is added to obtain a vector sequence of the same length as the input sequence.

[0146] Step S940: pass the third sequence H into another self-attention layer to obtain an output result, and add the output result to the first vector sequence to obtain a concatenation result.

[0147] Step S950: passing the concatenation result to the fully connected layer for output. When the type of relationship r is determined, for each type of relationship r, the head position and tail position of the entity h are predicted respectively, so as to simultaneously predict the head entity t and the relationship r and output the result.

[0148] In the tenth embodiment of the method for constructing a power equipment triplet based on a deep learning algorithm proposed by the present invention, based on the first embodiment, step S130 further includes the following steps:

[0149] Step S1010: Obtain a preset learning rate and a preset number of times.

[0150] Specifically, in this embodiment, the preset learning rate is 0.0001, and the preset number of times is 500 times.

[0151] Step S1020: Obtaining the accuracy of the triplet extraction model based on the preset learning rate and the preset number of times, and evaluating the triplet extraction model based on the accuracy.

[0152] Specifically, machine learning mainly includes the following steps: setting the number of training times, iteratively training the model on task-related data sets, and obtaining the fitted model; and then testing the model's error size by applying the model to actual scenarios.

[0153] The data set is usually divided randomly. To prevent data snooping bias, the data set can be randomly divided into two parts of 8:2, where 80% of the data is used as the training set and 20% of the data is used as the test set. The model is first trained on the training set, and then tested on the test set.

[0154] In this embodiment, the triple extraction model finally uses accuracy as the evaluation index for the extraction result. Accuracy refers to the proportion of individuals belonging to a certain category in the predicted results that actually belong to the category, that is, the proportion of truly relevant content among all predicted relevant content.

[0155] In the triplet extraction model constructed based on data flow programming in this embodiment, the learning rate is set to a preset learning rate of 0.0001, and the number of training times of the triplet extraction model is adjusted. When the number of training times reaches a preset number (500 times), the result curve tends to be stable. At this time, the accuracy of the triplet extraction model is obtained; and the triplet extraction model is evaluated based on the accuracy.

[0156] In this embodiment, the highest accuracy of the training set is 96.34%, and the highest accuracy of the test set is 85.61%.

[0157] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0158] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), including a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0159] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. A method for constructing a power equipment triplet based on a deep learning algorithm, characterized in that: include: Triple extraction model based on data flow programming: construct the pattern layer in a top-down manner, and under the guidance of the pattern layer, construct the data layer in a bottom-up manner, wherein the pattern layer is the knowledge organization framework for knowledge extraction, and is a data model that describes entities, relationships between entities, and attributes. The data layer includes a named entity extraction model and an entity relationship extraction model. According to the characteristics of the text to be extracted, a suitable extraction method is designed to extract the three knowledge elements of entity, relationship, and attribute, forming a series of high-quality factual expressions, which are then mapped to the pattern layer; Using the named entity extraction model to perform electric power equipment named entity recognition on the text to be extracted, so as to realize electric power equipment named entity extraction, and annotate to obtain annotated entities; Based on the labeled entity, the entity relationship extraction model is used to extract the power equipment entity relationship of the text to be extracted to extract the power equipment triple, wherein the entity relationship extraction model includes a bidirectional recurrent network, an expansion gate convolutional neural network and a self-attention model; The extracting the entity relationship of electric power equipment from the text to be extracted by the entity relationship extraction model based on the annotated entity to extract the electric power equipment triples includes: Using the entity annotated by the named entity extraction model as an input of the entity relationship extraction model to obtain a first result, using the feature vector obtained by retraining the text to be extracted as another input of the entity relationship extraction model to obtain a second result, concatenating the first result and the second result, and passing them into a convolutional neural network to extract a triplet of power equipment; The method of performing electric power equipment named entity recognition on the text to be extracted by the named entity extraction model to realize electric power equipment named entity extraction, and annotating to obtain annotated entities, includes: Build a vocabulary database in the field of electricity; Get the training set text sequence; Based on the training set text sequence, the spatial vector of each word in the power field vocabulary is calculated by the word2vec model, and the word vector of the training set text sequence is calculated by the word embedding layer; Obtaining the word mixed Embedding vector sequence encoding by a word mixed Embedding method; Add a Position Embedding vector with the same dimension as the word vector sequence to make the position information of the encoded vector sequence more obvious; Inputting the vector sequence encoding into a 12-layer expansion gate convolutional neural network with expansion rates of 1, 2, 5, 1, 2, 5, 1, 2, 5, 1, 1, 1 for learning, so as to output a first sequence; Pass the first sequence into a self-attention layer to obtain a second sequence; The second sequence is passed into the output of the fully connected layer, and the first position of the entity and the tail position of the entity are predicted using a half-pointer and half-annotation structure to obtain a labeled entity.

2. According to a method for constructing a power equipment triplet based on a deep learning algorithm according to claim 1, it is characterized in that: The construction of the electric power field vocabulary includes: Establish a dictionary tree index, count the frequency of each character and bigram in the text to be extracted, and mark it as word frequency, and take bigrams as candidate words in descending order of word frequency, where the bigram consists of each character and its right adjacent character; Get the point mutual information threshold, left and right information entropy threshold, word frequency threshold, and maximum word length threshold; Calculate the point mutual information of candidate words; When the point mutual information of a candidate word is greater than the point mutual information threshold, the candidate word is marked as a to-be-selected word; Calculating the left information entropy and the right information entropy of the words to be selected; If the left information entropy of the word to be selected is greater than the left and right information entropy thresholds, and the right information entropy of the word to be selected is greater than the left and right information entropy thresholds, and the word frequency of the word to be selected is greater than the word frequency threshold, the word to be selected is stored in the electric power field word library; If the right information entropy of the candidate word is less than the left and right information entropy thresholds, the candidate word is expanded to the right to obtain a right-expanded word, and the length of the right-expanded word is less than the maximum word length threshold, and then the calculation of the left information entropy and the right information entropy of the candidate word and the subsequent steps are performed again; If the left information entropy of the candidate word is less than the left and right information entropy thresholds, expand the candidate word to the left to obtain a left-expanded word, and the length of the left-expanded word is less than the maximum word length threshold, and then perform the calculation of the left information entropy and the right information entropy of the candidate word again, and the subsequent steps.

3. The method for constructing a power equipment triplet based on a deep learning algorithm according to claim 2, characterized in that: The formula for calculating the point mutual information of candidate words is: Among them, PMI (a i ,b j ) is the candidate word a in the text to be extracted i b j Point mutual information; P(a i ,b j ) is the adjacent character a of the candidate word in the text to be extracted i and character b j Combination of characters a i b j The probability of occurrence, P(a i ) is the character a i The probability of appearing in the text to be extracted, P(b j ) is the character b j The probability of appearing in the text to be extracted.

4. The method for constructing a power equipment triplet based on a deep learning algorithm according to claim 1, characterized in that: A Position Embedding vector with the same dimension as the word vector sequence is added to make the position information of the encoded vector sequence more obvious. The calculation formula of Position Embedding is: The above formula maps the position with position id p to a d pos The i-th value of the position vector is PE i (p).

5. The method for constructing a power equipment triplet based on a deep learning algorithm according to claim 1, characterized in that: The encoding of the vector sequence is input into a 12-layer expansion gate convolutional neural network with expansion rates of 1, 2, 5, 1, 2, 5, 1, 2, 5, 1, 1, 1 for learning to obtain a first sequence, including: Set the vector sequence to be processed to z, and z = [z1, z2, ..., z n ]; The vector sequence encoding is input into the convolutional layer of the convolutional neural network, and a gate mechanism is added to the convolution during the convolution to obtain the first sequence of output results, wherein the expression of the first sequence is: Wherein, Y represents the first sequence, σ is the Sigmoid activation function, Conv1D1 and Conv1D2 are two one-dimensional convolutions of the same form, one of which is activated by the sigmoid activation function, and the other is not activated.

6. A method for constructing a power equipment triplet based on a deep learning algorithm according to claim 5, characterized in that: The entity annotated by the named entity extraction model is used as an input of the entity relationship extraction model to obtain a first result, the feature vector obtained by retraining the text is used as another input of the entity relationship extraction model to obtain a second result, the first result and the second result are concatenated and passed into a convolutional neural network to extract a triple of power equipment, including: Perform head entity prediction for each sentence in the extracted text; The tail entity corresponding to the head entity is predicted through the head entity, and then the relationship between the input head entity and the tail entity is predicted through the head entity and the tail entity.

7. The method for constructing a power equipment triplet based on a deep learning algorithm according to claim 6, characterized in that: The step of predicting the tail entity corresponding to the head entity through the head entity, and then predicting the relationship between the input head entity and the tail entity through the head entity and the tail entity, includes: Taking the labeled entity as input, randomly sampling a labeled entity; Retraining the text to be extracted corresponding to the labeled entity, using a vector concatenated from word mixed Embedding and position Embedding to pass into a 12-layer dilation gate convolutional neural network with dilation rates of 1, 2, 5, 1, 2, 5, 1, 2, 5, 1, 1, 1, and a self-attention layer for learning, so as to obtain a third sequence; The encoding vectors in the third sequence corresponding to the randomly sampled labeled entities are passed into the bidirectional recurrent network for encoding, and the encoding results of the relative positions are added to obtain a vector sequence of the same length as the input sequence; Passing the third sequence into another self-attention layer to obtain an output result, and adding the output result to the first vector sequence to obtain a concatenation result; The concatenation result is passed to the fully connected layer for output. When the relationship type is determined, for each relationship, the head position and tail position of the entity are predicted respectively, so as to achieve simultaneous prediction of the head entity and the relationship and output the result.

8. The method for constructing a power equipment triplet based on a deep learning algorithm according to claim 1, characterized in that: Based on the annotated entity, the entity relationship extraction model is used to extract the power equipment entity relationship of the text to be extracted to extract the power equipment triple, and then the method further includes: Get the preset learning rate and preset times; The accuracy of the triplet extraction model is obtained based on the preset learning rate and the preset number of times, and the triplet extraction model is evaluated based on the accuracy.

Images