Knowledge base construction method and system for regulating and controlling business behaviors based on bidirectional word lattice graph

Abstract

The invention discloses a knowledge base construction method and system for regulating and controlling business behaviors based on a bidirectional word lattice graph, and the method comprises the steps: carrying out the combined training through employing a trained power grid field BERT, and obtaining a high-quality feature vector; reserving all matching results by constructing a word lattice graph, and constructing a bidirectional word lattice graph attention network by using a graph attention network; constructing a power grid regulation and control operation ticket sentence vector on the basis of Sentence-BERT; the result of the two-way word lattice graph attention network processing and the sentence vector obtained through the SenteceBERT are handed over to a Transformer to be processed; and completing joint knowledge extraction of behaviors in the power grid regulation and control business field and constructing a knowledge base. According to the method, the error rate of text word segmentation in the field of power grid regulation and control business can be reduced, and the problem of low knowledge extraction accuracy caused by word segmentation error propagation is avoided, so that the utilization efficiency of a knowledge base of regulation and control business behaviors is improved, and the intelligent process of regulation and control business is accelerated.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence and power grid control business, and in particular to a method and system for constructing a knowledge base of control business behavior based on a bidirectional word grid graph. Background Art

[0002] As an important part of the power grid, dispatching and control technology provides strong technical support for the dispatching and operation of the power system. However, with the continuous expansion of the scale and complexity of the power system, the operation and maintenance of dispatching and control has become more and more complex and arduous. The traditional operation and maintenance method mainly relies on manual operation and ensures the correctness of the operation through management means. Although it has certain anti-error measures, it still has great limitations and risks. Operators need to have rich experience and skills in actual work to effectively cope with various complex operation and maintenance tasks. However, manual operation is bound to have omissions and errors, especially when faced with heavy and complex operation and maintenance tasks, system failures caused by operational errors often occur. At present, the operation and maintenance of dispatching and control mainly includes the generation, execution, verification and result inspection of operation tickets. The operation ticket is the core of the operation and maintenance work. Through the standardized management and execution of the operation ticket, the operation risk can be effectively reduced and the safe operation of the system can be ensured. However, the traditional operation ticket management and execution method has problems such as complex operation, easy to make mistakes, dependence on manual work, low efficiency, insufficient anti-error measures and high risk.

[0003] With the development of smart grid, the position of smart grid dispatching and control technology in power system is becoming more and more important. In order to solve the above problems, it is urgent to study the knowledge base of grid dispatching and control business behavior. Summary of the invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: a method for constructing a knowledge base for regulating business behaviors based on a bidirectional word grid graph, comprising:

[0006] Use the trained BERT in the power grid domain to perform joint training to obtain high-quality feature vectors;

[0007] All matching results are retained by constructing a word lattice graph, and a bidirectional word lattice graph attention network is constructed by using a graph attention network;

[0008] Constructing grid control operation sentence vectors based on Sentence-BERT;

[0009] The results of the bidirectional word lattice graph attention network and the sentence vector obtained by SentenceBERT are handed over to Transformer for processing;

[0010] Complete the joint knowledge extraction of behaviors in the power grid control business field and build a knowledge base.

[0011] As a preferred solution of the method for constructing a knowledge base of regulation business behavior based on a bidirectional word lattice graph according to the present invention, wherein: the power grid field BERT includes: BERT is pre-trained using unlabeled raw data, and the pre-training adopts the pre-training method of the mask language model in BERT;

[0012] According to the BERT model, the unlabeled grid control operation ticket data and MLM are finally trained to become BERT in the field of grid control;

[0013] The original power grid control operation ticket data is associated and matched with the power grid facility list, and the power grid equipment in the power grid control operation ticket data is marked with the equipment list to generate marked power grid control operation ticket data;

[0014] The labeled power grid regulation operation ticket data is read into the trained power grid regulation domain BERT to generate weak labels, and weak labels are added to the original power grid regulation operation ticket data; the masked language model head is replaced with the conditional random field head to obtain a vector trained with the labeled power grid regulation operation ticket data.

[0015] As a preferred solution of the method for constructing a knowledge base of regulatory business behaviors based on a bidirectional word lattice graph as described in the present invention, the bidirectional word lattice graph attention network includes constructing a power grid regulation word lattice graph based on the obtained vector, constructing a power grid regulation bidirectional word lattice graph, and embedding the bidirectional word lattice into the graph attention network.

[0016] As a preferred solution of the method for constructing a knowledge base of regulation business behavior based on a bidirectional word lattice graph described in the present invention, wherein: the construction of a power grid regulation operation ticket sentence vector based on Sentence-BERT includes the steps of segmenting and encoding the power grid regulation operation ticket to be processed; processing the input text through a neural network to generate a vector representation of each word;

[0017] These word vectors are processed using a max pooling strategy to generate a vector representation of the entire sentence;

[0018] The model is constructed by fine-tuning the pre-trained Sentence-BERT model.

[0019] Use the fine-tuned Sentence-BERT model to generate grid regulation operation sentence vectors.

[0020] As a preferred solution of the method for constructing a knowledge base of regulation business behavior based on a bidirectional word lattice graph according to the present invention, wherein: in the Transformer, the vector generated by the bidirectional word lattice graph attention network and the power grid regulation operation ticket sentence vector constructed based on Sentence-BERT are constructed into a joint embedding vector, which is input into the Transformer embedding layer;

[0021] In the Transformer generation layer, the Transformer encoder is composed of multiple layers with the same structure, and each layer contains two sub-layers: a self-attention layer and a feedforward neural network;

[0022] The decoder combines the output of the self-attention layer with the encoder for decoding. During the decoding process, a real vector is output at each time step, which is mapped to the output vector of a log-probability regression function after a fully connected layer for knowledge extraction.

[0023] As a preferred solution of the method for constructing a knowledge base for regulating business behavior based on a bidirectional word lattice graph of the present invention, the word lattice graph construction process includes two steps of text segmentation and word lattice graph generation; the text segmentation step includes two parts of character segmentation and word segmentation, wherein the character segmentation is to segment the text in units of characters and obtain a character sequence, while the word segmentation is to segment the text using a word segmentation tool and obtain a word sequence;

[0024] The original grid control operation ticket data is first matched with the grid facility list. Different Chinese word segmentation tools are used to segment the unmatched text. Only Chinese words with a length greater than 1 are retained. The word segmentation results and the grid facility list are combined to generate a dictionary to obtain a word sequence.

[0025] Word lattice graph generation: concatenate and merge the character sequence and word sequence in the text segmentation; make each word lattice graph have only one head node and one tail node, and add extra characters [START] and at the beginning and end of the merged result as the head and tail nodes respectively;

[0026] The bidirectional word lattice graph for power grid regulation is a topological graph. When used in practice, a forward word lattice graph and a reverse word lattice graph are constructed according to the topological graph. The forward word lattice graph construction process includes: firstly, specifying the direction of each word lattice node as positive, and each word lattice node points to the word lattice node at the next position according to its position in the text; the initial embedding of each word lattice node is generated by the multi-granularity embedding module in the next section, including byte nodes and word nodes;

[0027] When constructing the reverse word lattice embedding graph, the direction of the word lattice nodes is opposite to that of the forward word lattice graph, and the other construction steps are the same as the forward word lattice graph construction process.

[0028] As a preferred solution of the method for constructing a knowledge base for regulating business behavior based on a bidirectional word lattice graph according to the present invention, wherein: the bidirectional word lattice embedding graph attention network includes aggregating and updating word lattice nodes through the bidirectional word lattice embedding graph attention network to realize mutual interaction between word nodes and word nodes themselves and nodes;

[0029] The graph attention network uses the attention mechanism to assign different weights to the aggregation between different nodes, so that the weight assigned to the nodes with incorrect word segmentation is small during aggregation, thereby preventing the propagation of word segmentation errors from affecting the model performance.

[0030] In the bidirectional word embedding graph attention network, the initial context embedding of each word lattice node vi Message aggregation process from multi-granularity embedding module; forward word lattice graph:

[0031] First, define the positive word grid graph G f =(V f ,E f ), where V f ={v1,v2,…,v n-1 ,v n} represents the forward word node set, that is, the word node set composed of the characters or word segmentation results in the short text, E f ={e 11 ,e 12 ,…,e i,j ,…,e n-1,n} represents the forward edge set, that is, the edge set formed between word lattice nodes;

[0032] The side e i,j Represents word case node v i and word case node v j There is a one-way connection between them;

[0033] When aggregating neighbor node information at layer l, in addition to aggregating word node v i In addition to its own features, it also aggregates its forward neighbor node information. The aggregation formula is shown in formula (1):

[0034]

[0035] in, Represents the l-th layer node v i The forward node embedding of i Aggregate word node, fw(l) represents the forward node of layer l, R d represents a real vector of length d, and fw(i) represents any forward node v in the lth layer. i ;σ(·) represents the activation function; Represents node v iAll forward neighbors of ij Represents the slave node v j To node v i The normalized weight of the edge; W fw represents the trainable parameters of the positive word grid graph, fw represents the positive direction;

[0036]

[0037] Among them, f(h i ,h j ) is a scoring function; a, W and V are trainable parameters; represents splicing; in the graph attention network, the normalized weight a of the edge ij The calculation formula is shown in formula (3);

[0038] The multi-granularity embedding module is a combined embedding of characters, words, and characters and words;

[0039] The attention type in the attention network is dynamic attention, and its scoring function is shown in formula (4):

[0040]

[0041] Among them, W and V are two trainable parameters, which play the role of linear transformation;

[0042] Use the same parameter U for linear transformation to achieve information sharing, let U = W = V; the final scoring function is simplified as shown in formula (5):

[0043]

[0044] For the reverse word lattice graph, it uses the same mechanism as the forward word lattice graph for message aggregation, except that the direction of the edge is opposite to that of the forward word lattice graph. The node aggregation formula is shown in formula (6):

[0045]

[0046] in is the node v at the lth layer i Backward node embedding of i Aggregate word node, bw(l) represents the reverse node of the lth layer, R d represents a real vector of length d, bw(i) represents any forward node v in the lth layer i ;σ(·) represents the activation function; is node v i All reverse neighbors of W fw represents the trainable parameters of the reverse word lattice graph, bw represents reverse;

[0047] Finally, the forward word node is embedded and backward word lattice node embedding Splice them together to get the word node v i The final context representation

[0048] A knowledge base construction system for regulating business behaviors based on a bidirectional word grid graph using the method of the present invention is characterized by:

[0049] The training unit uses the trained power grid domain BERT to perform joint training to obtain high-quality feature vectors;

[0050] The model building unit retains all matching results by building a word lattice graph and builds a bidirectional word lattice graph attention network by using a graph attention network;

[0051] Vector generation unit, which constructs the grid control operation sentence vector based on Sentence-BERT;

[0052] The processing unit passes the results of the bidirectional word lattice graph attention network and the sentence vector obtained by SentenceBERT to the Transformer for processing;

[0053] The output unit completes the joint knowledge extraction of behaviors in the power grid regulation business field and builds a knowledge base.

[0054] A computer device comprises: a memory and a processor; the memory stores a computer program, wherein: the processor implements the steps of any one of the methods of the present invention when executing the computer program.

[0055] A computer-readable storage medium stores a computer program, wherein: when the computer program is executed by a processor, the steps of any one of the methods of the present invention are implemented.

[0056] Beneficial effects of the present invention: The knowledge base construction method of the regulation business behavior based on the bidirectional word lattice graph provided by the present invention can reduce the error rate of text segmentation in the field of power grid regulation business, and avoid the problem of low accuracy of knowledge extraction due to the propagation of word segmentation errors, thereby improving the utilization efficiency of the knowledge base of the regulation business behavior and accelerating the intelligent process of the regulation business. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0058] Figure 1 An overall flow chart of a method for constructing a knowledge base for regulating business behaviors based on a bidirectional word lattice graph provided in the first embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0060] Example 1, reference Figure 1 , is an embodiment of the present invention, and provides a method for constructing a knowledge base for regulating business behavior based on a bidirectional word grid graph, comprising:

[0061] S1: Training BERT in the field of power grid regulation.

[0062] The BERT model is used for pre-training on the original grid control operation ticket. BERT uses the unlabeled original data for pre-training. The pre-training adopts the pre-training method of the masked language model (MLM) in BERT. According to the existing BERT model, the unlabeled grid control operation ticket data and MLM are finally trained to become the grid control field BERT.

[0063] S2: Automatic marking of power grid control operation tickets.

[0064] The original power grid regulation operation ticket data is associated with the power grid facility list, and the power grid equipment in the power grid regulation operation ticket data is marked using the equipment list to generate marked power grid regulation operation ticket data. The matching process is specifically: an algorithm for finding a pattern string (any facility in the power grid facility list) in a text string (the original power grid regulation operation ticket data). That is, compare the characters of the pattern string one by one in the text string. If they match, continue to compare the next character. If they do not match, move the pattern string one position to the right and continue to compare until all text strings are compared. Then use the same method to compare the next facility until all facilities are compared. In this way, the marking of the original power grid regulation operation ticket data using the power grid facility list is realized.

[0065] S3: Joint training of labeled and unlabeled power grid regulation operation tickets.

[0066] The labeled grid control operation ticket data is read into the trained grid control domain BERT, and weak labels are generated according to certain rules, and weak labels are added to the original grid control operation ticket data. The masked language model header is replaced with the conditional random field header. The vector is obtained by training the combined labeled grid control operation ticket data.

[0067] Furthermore, the replacement result of "replacing the masked language model header with the conditional random field header" is as follows Figure 1 As shown in the execution diagram of the forward word graph, the head position uses the conditional random field head to replace the mask language model head. The execution diagram of the reverse word graph is similar.

[0068] S4 builds a bidirectional lattice graph attention network for power grid control.

[0069] S4.1 Construct a grid control word grid diagram.

[0070] Based on the vector established by S3, a word lattice graph is constructed. The original grid control operation ticket data is matched with the words in the grid facility list, and multiple word segmentation tools are further used to produce multiple results, and finally a word sequence is obtained. In the subsequent graph attention network, different weights are assigned to the aggregation between different nodes through the attention mechanism, thereby reducing the weight of incorrect word segmentation. In the word lattice graph generation step, the character sequence and word sequence in the text segmentation are mainly spliced ​​and merged (the purpose is to extract features from different aspects). At the same time, in order to ensure that each word lattice graph has only one head node and one tail node, additional characters [START] and are added at the beginning and end of the merged result as head and tail nodes.

[0071] S4.2 Construction of a bidirectional word grid graph for power grid control

[0072] The word lattice graph constructed by the above steps is a topological graph, which is mainly used to save the character sequence of the text and all the word segmentation paths. In specific use, the forward word lattice graph and the reverse word lattice graph will be constructed according to the topological graph. The construction process of the forward word lattice graph is as follows: First, specify the direction of each word lattice node as forward, that is, each word lattice node points to the word lattice node at the next position according to its position in the text. The initial embedding of each word lattice node is generated by the multi-granularity embedding module in the next section, including byte nodes and word nodes. The reverse word lattice embedding graph, except that the direction of the word lattice node is opposite to that of the forward word lattice graph, has the same construction steps.

[0073] Among them, the multi-granularity embedding module is the combined embedding of characters, words, and characters and words, so it is a multi-granularity embedding. It mainly solves the problem that general word segmentation is prone to word segmentation errors in solving domain problems, which leads to error propagation and amplification of such errors, resulting in inaccurate information extraction.

[0074] S4.3 Constructing a bidirectional word embedding graph attention network

[0075] The bidirectional word lattice embedding graph attention network is used to learn the forward and backward context information of short texts. The bidirectional word lattice embedding graph attention network aggregates and updates word lattice nodes to achieve mutual interaction between byte nodes, word nodes themselves, and nodes. At the same time, the graph attention network assigns different weights to the aggregation between different nodes through the attention mechanism, which makes the weight assigned to the wrong word segmentation node smaller during aggregation, thereby avoiding the propagation of word segmentation errors affecting the model performance to a certain extent. The bidirectional word lattice embedding graph attention network consists of two separate word lattice embedding graph attention networks, namely the forward word lattice graph and the reverse word lattice graph.

[0076] In the bidirectional word lattice embedding graph attention network, each word lattice node v i Initial context embedding From the multi-granularity embedding module. Next, we take the forward word lattice graph as an example to introduce the specific message aggregation process. For the reverse word lattice graph, it uses the same mechanism as the forward word lattice graph for message aggregation, except that the direction of the edge is opposite to that of the forward word lattice graph.

[0077] First, define the positive word grid graph G f =(V f, E f ), where Vf = {v 1, v 2, v 3, v 4, v 5, v 6, v7} represents the forward word node set, E f ={e 1,1 ,e 2,2 ,…e 7,7 ,e 1,2 ,e 2,3 ,e 3,4 ,e 4,5 ,e 4,6 ,e 5,7 ,e 6,7} represents the forward edge set. i,j Represents word case node v i and word case node v j There is a unidirectional connection between them. Taking the word case node v3 as an example, its forward neighbor node is v4. When aggregating neighbor node information at the lth layer, in addition to aggregating the word case node v i In addition to its own features, it also aggregates its forward neighbor node information. The aggregation formula is shown in formula (1):

[0078]

[0079] In formula 1, Represents the l-th layer node v i The forward node embedding of ; σ(·) represents the activation function; Represents node v i All forward neighbors of a ij Represents the slave node v j To node v i The normalized weight of the edge; W fw Represents the trainable parameters of the positive word lattice graph.

[0080]

[0081]

[0082] In formula (2), f(h i ,h j ) is a scoring function; a, W and V are trainable parameters; In the graph attention network, the normalized weight a of the edge ij The calculation formula of is shown in formula (3).

[0083] The attention type designed in the present invention is dynamic attention, and its scoring function is shown in formula (4):

[0084]

[0085] The two trainable parameters W and V in formula 4 mainly play a role of linear transformation. Therefore, the same parameter U can be used for linear transformation to achieve information sharing, that is, let U = W = V. The final scoring function is simplified as shown in formula (5):

[0086]

[0087] The graph attention network is used to realize that word lattice nodes simultaneously aggregate the information of word nodes and byte nodes, and different attention weights are assigned to different positive neighbor nodes through the attention mechanism. Specifically, the attention mechanism can assign lower weights to incorrect word segmentation nodes and higher weights to correct word segmentation nodes, so that the network pays more attention to the correct word nodes when aggregating word node information to byte nodes. This realizes the explicit use of word information and can prevent the propagation of word segmentation errors from affecting model performance.

[0088] For the reverse word grid graph, its node aggregation formula is shown in formula (6):

[0089]

[0090] in is the node v at the lth layer i The backward node embedding of ; σ(·) represents the activation function; is node v i All backward neighbors of W fw Represents the trainable parameters of the reverse lattice graph.

[0091] Finally, the forward word node is embedded and backward word lattice node embedding Splice them together to get the word node v i The final context representation

[0092] S5 builds sentence vectors for power grid regulation operation based on Sentence-BERT.

[0093] The grid regulation operation ticket vectors are constructed based on the Sentence-BERT sentence vector technology, and these vectors are used to construct the grid regulation operation ticket vectors according to the order of operations.

[0094] S5.1 Input processing.

[0095] First, the grid regulation operation tickets to be processed are input into the model, which includes preprocessing steps such as word segmentation and encoding of the labeled grid regulation operation tickets so that the model can understand and process the data.

[0096] S 5.2 Feature extraction.

[0097] Inside the model, the input text is processed through specific neural network layers (such as Transformer layers) to generate vector representations for each word.

[0098] S 5.3 Lexicalization strategies.

[0099] These word vectors are processed using a max pooling strategy to generate a vector representation of the entire sentence. This is done in the last layer of the model.

[0100] S 5.4 outputs the sentence vector and performs fine-tuning.

[0101] The model outputs a vector of fixed dimension, which represents the semantic information of the entire sentence. The pre-trained Sentence-BERT model needs to be fine-tuned to adapt to this task.

[0102] S6 Transformer-based power grid regulation business behavior knowledge extraction.

[0103] like Figure 1The overall process. The vector generated by the bidirectional word lattice graph attention network and the power grid regulation operation ticket sentence vector based on Sentence-BERT are constructed into a joint embedding vector and input into the Transformer embedding layer. Since the input information not only contains the word information in the sentence, but also contains structural information such as the location of entities such as equipment and operations, it is necessary to represent this information so that the model can learn the relationship between them. In the input embedding layer (Embedding), the present invention encodes the word information and the location information of the entity into an embedded vector representation, which is called a word vector and a position vector. These vectors are then concatenated to obtain a joint embedding vector. Such a word embedding representation contains richer information in the power grid regulation business data as an input code.

[0104] In the Transformer generation layer, the Transformer encoder is composed of multiple layers with the same structure, each of which contains two sublayers: a self-attention layer and a feed-forward neural network. The self-attention layer is mainly used to learn the business behavior embedded in the input data, which is externally reflected in the weights between words in the power grid regulation operation ticket. In addition, it can also help the model learn the syntax and grammar of the data in the power grid regulation operation ticket. The decoder combines the output of the self-attention layer with the encoder for decoding. During the decoding process, a real number vector is output at each time step, which is mapped to a logarithmic probability vector for knowledge extraction after a simple fully connected layer.

[0105] S 6.1 Initialize all module parameters, including the embedding layer and various parameters of the Transformer model.

[0106] The embedding layer parameters include the length of the word segmentation dictionary for the training data, the word vector dimension of the preset value, the position information vector dimension, etc.; the Transformer layer mainly includes some model hyperparameters such as the number of codec layers, and trainable parameters such as the multiple sets of parameter matrices Wq, Wk and Wv in the multi-head self-attention in the codec and the parameters W1, W2, b1, b2 in the feedforward neural network.

[0107] S6.2 constructs a joint vector.

[0108] For each sentence vector d, the corresponding word information w (from the bidirectional word lattice attention network) and entity position information position in d are sent to the embedding layer respectively, and after encoding, they are converted into word vectors and position vector representations, which are then concatenated as the complete information embedding representation X of a piece of data. Thus, a joint vector is constructed.

[0109] S6.3 Forward propagation.

[0110] The Transformer model inputs X into the encoder part. The self-attention part of each layer of the encoder will calculate the query, key, and value matrices Q (query), K (key value), and V (value) based on the parameter matrices Wq, Wk, and Wv and the input X, and then calculate the attention matrix Z for each step. Z is linearly transformed by the feedforward neural network connected to each self-attention layer and propagated to the next layer as output. The encoder passes the output of the last layer to the decoder, and the decoder calculates the decoding result with the same architectural process. The calculation of each layer will cause the Transformer to adjust its attention distribution to different information, and then learn the entity and behavior knowledge in the power grid regulation operation ticket.

[0111] The calculation process involved in the multi-layer self-attention sub-layer is as follows:

[0112] O=XW o (7)

[0113] P=XW p (8)

[0114] S=XW s (9)

[0115]

[0116] W in formula (7) o , W in formula (8) p and W in formula (9) s is a trainable attention parameter matrix; X is the input from the previous sub-layer, which is the embedded representation of the power grid regulation operation ticket data provided by the embedding layer at the model entrance. After obtaining the matrices O, P, and S, the output value of Self-Attention can be calculated using formula (10), where softmax is a normalization function used to calculate the distribution of scores. p is the dimension of the P matrix, which is mainly used to stabilize training.

[0117] S6.4 Back propagation.

[0118] The final output of the decoder passes through the fully connected layer at the top of the decoder and then passes through the softmax function to obtain the predicted distribution, and the index with the largest proportion is used as the relationship classification result r^. The predicted result is used to calculate the loss value loss with the labeled result r, and then back-propagated. During the propagation process, the model will adjust the trainable parameters to fit the input and the correct result.

[0119] S6.5 Output and regulate the business behavior knowledge base.

[0120] Repeat S6.3-S6.4 until the iteration is completed, or terminate early because it is below the convergence threshold (Early-Stopping).

[0121] Z=concat(Z1,Z2,Z3,...,Z n )(11)

[0122] output=relu(ZW1+b1)W2+b2(12)

[0123] As shown in formula (11), Transformer is a multi-head attention mechanism. Each layer has multiple attention heads working together. Therefore, the input Z received by the feedforward neural network layer comes from multiple attention heads Z1, Z2, Z3, ..., Z n The concatenation of the calculation results, concat is the concatenation function. The final output output is shown in formula (12). In the feedforward neural network layer, two training parameter matrices W1 and W2 and their biases b1 and b2 are used to linearly transform the results to generate output output and send it to the next layer. The activation function is relu, which is the linear rectification function (Linearrectification function), also known as the rectified linear unit, which is a commonly used activation function in artificial neural networks. It usually refers to nonlinear functions represented by ramp functions and their variants.

[0124] During the continuous iterative training process, all trainable parameters will be continuously adjusted so that the Transformer model can learn the complex connections between different entities and behaviors in the power grid regulation operation ticket in terms of structure, semantics, etc., and finally build a knowledge base of power grid regulation business behavior.

[0125] Embodiment 2 is an embodiment of the present invention, which provides a knowledge base construction system for regulating business behavior based on a bidirectional word grid graph.

[0126] The training unit uses the trained power grid domain BERT to perform joint training to obtain high-quality feature vectors.

[0127] The model building unit retains all matching results by building a word lattice graph and builds a bidirectional word lattice graph attention network by using a graph attention network.

[0128] The vector generation unit builds the sentence vector of the power grid regulation operation based on Sentence-BERT.

[0129] The processing unit passes the results of the bidirectional word lattice attention network and the sentence vector obtained by SentenceBERT to the Transformer for processing.

[0130] Output unit, completes the joint knowledge extraction of grid control business behaviors and builds a knowledge base

[0131] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0132] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0133] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0134] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0135] Example 3 is an embodiment of the present invention, which provides a knowledge base construction method for regulating business behavior based on a bidirectional word lattice graph. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0136] The verification experiment is carried out on the Southwest Power Grid Control Dataset from 2015 to 2022. Table 1 shows the statistical information of the dataset.

[0137] Table 1 Detailed information of the power grid regulation operation ticket dataset

[0138] Dataset Number of texts Average text length Southwest Power Grid Control Operation Ticket Dataset 9674 1190.45

[0139] This dataset contains 9674 texts.

[0140] Evaluation indicators:

[0141] The present invention uses three commonly used indicators, namely precision (Precision, P), recall (Recall, R) and F1 value, to comprehensively evaluate the method proposed in the present invention.

[0142] The calculation formulas for the three indicators are as follows:

[0143]

[0144]

[0145]

[0146] Where i represents the i-th category, TP represents the number of samples that are both predicted and true, FP represents the number of samples that are predicted to be true but actually false, FN represents the number of samples that are predicted to be false but actually true, and TN represents the number of samples that are predicted to be false but actually false.

[0147] Precision P refers to the average value of the ratio of the number of TPs in each category among n categories to the number of samples (including TP and FP) judged to be true by the model, for a data set of power grid regulation operation tickets in a province in East China. P is a statistic for some samples, focusing on the statistics of data judged to be true by the model. Recall R is the average value of the ratio of the number of TPs in each category among n categories to the number of truly true samples (including TP and FN). R is also a statistic for some samples. Macro-F1 is a comprehensive evaluation indicator, which first calculates the P and R values ​​of each category separately, and then averages them.

[0148] In order to verify the effectiveness and advancement of the proposed method, experiments were conducted using the power grid regulation operation ticket dataset. The specific comparison models and experimental results are shown in Table 2.

[0149] Table 2 Experimental results of the method proposed in this invention and the comparative method on the grid control operation ticket dataset

[0150] method P(％) R(％) F1(％) Regular Expressions 75.87 67.90 71.66 TextCNN-CRF 77.43 70.91 74.03 word2vec+BiLSTM-CRF 81.23 76.39 78.74 ChineseBERT-CRF 83.76 84.27 84.01 Span-BERT-CRF 84.87 86.79 85.82 The method proposed by the present invention 89.67 92.31 90.97

[0151] As shown in Table 3, the method proposed in the present invention surpasses all the comparison methods in multiple evaluation indicators. Compared with the regular expression method, the method proposed in the present invention has a relative improvement of 18.19% in precision P value, 35.95% in recall R value, and 26.95% in F1 value, with significant improvements in all indicators. Compared with the optimal Span-BERT-CRF model, the method proposed in the present invention has improved 5.66%, 6.36% and 6.00% in the three evaluation indicators of precision P, recall R and F1 value, respectively, which has significant advantages. This illustrates the advancement and effectiveness of the method proposed in the present invention.

[0152] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for constructing a knowledge base for regulating business behavior based on a bidirectional word grid graph, characterized in that: include: Use the trained BERT in the power grid domain to perform joint training to obtain high-quality feature vectors; All matching results are retained by constructing a word lattice graph, and a bidirectional word lattice graph attention network is constructed by using a graph attention network; Constructing grid control operation sentence vectors based on Sentence-BERT; The results of the bidirectional word lattice graph attention network and the sentence vector obtained by SentenceBERT are handed over to Transformer for processing; Complete the joint knowledge extraction of behaviors in the power grid control business field and build a knowledge base.

2. The method for constructing a knowledge base for regulating business behavior based on a bidirectional word grid graph as claimed in claim 1, characterized in that: The power grid field BERT includes: BERT uses unlabeled raw data for pre-training, and the pre-training adopts the pre-training method of the mask language model in BERT; According to the BERT model, the unlabeled grid control operation ticket data and MLM are finally trained to become BERT in the field of grid control; The original power grid control operation ticket data is associated and matched with the power grid facility list, and the power grid equipment in the power grid control operation ticket data is marked with the equipment list to generate marked power grid control operation ticket data; The labeled power grid regulation operation ticket data is read into the trained power grid regulation domain BERT to generate weak labels, and weak labels are added to the original power grid regulation operation ticket data; the masked language model head is replaced with the conditional random field head to obtain a vector trained with the labeled power grid regulation operation ticket data.

3. The method for constructing a knowledge base for regulating business behavior based on a bidirectional word grid graph as claimed in claim 2, characterized in that: The bidirectional word lattice graph attention network includes: after constructing a power grid regulation word lattice graph based on the obtained vector, constructing a power grid regulation bidirectional word lattice graph, and embedding the bidirectional word lattice into the graph attention network.

4. The method for constructing a knowledge base for regulating business behavior based on a bidirectional word grid graph as claimed in claim 3, characterized in that: The construction of the power grid regulation operation ticket sentence vector based on Sentence-BERT includes the steps of segmenting and encoding the power grid regulation operation ticket to be processed; processing the input text through a neural network to generate a vector representation of each word; These word vectors are processed using a max pooling strategy to generate a vector representation of the entire sentence; The model is constructed by fine-tuning the pre-trained Sentence-BERT model. Use the fine-tuned Sentence-BERT model to generate grid regulation operation sentence vectors.

5. The method for constructing a knowledge base for regulating business behavior based on a bidirectional word grid graph as claimed in claim 4, characterized in that: In the Transformer, the vector generated by the bidirectional word grid graph attention network and the power grid regulation operation ticket sentence vector constructed based on Sentence-BERT are constructed into a joint embedding vector, which is input into the Transformer embedding layer; In the Transformer generation layer, the Transformer encoder is composed of multiple layers with the same structure, and each layer contains two sublayers: a self-attention layer and a feedforward neural network; The decoder combines the output of the self-attention layer with the encoder for decoding. During the decoding process, a real vector is output at each time step, which is mapped to the output vector of a log-probability regression function after a fully connected layer for knowledge extraction.

6. The method for constructing a knowledge base for regulating business behavior based on a bidirectional word grid graph as claimed in claim 5, characterized in that: The word lattice graph construction process includes two steps: text segmentation and word lattice graph generation; the text segmentation step includes two parts: character segmentation and word segmentation, wherein character segmentation is to segment the text in units of characters and obtain a character sequence, while word segmentation is to segment the text using a word segmentation tool and obtain a word sequence; The original grid control operation ticket data is first matched with the grid facility list. Different Chinese word segmentation tools are used to segment the unmatched text. Only Chinese words with a length greater than 1 are retained. The word segmentation results and the grid facility list are combined to generate a dictionary to obtain a word sequence. Word lattice graph generation: concatenate and merge the character sequence and word sequence in the text segmentation; make each word lattice graph have only one head node and one tail node, and add extra characters [START] and at the beginning and end of the merged result as the head and tail nodes respectively; The bidirectional word lattice graph for power grid regulation is a topological graph. When used in practice, a forward word lattice graph and a reverse word lattice graph are constructed according to the topological graph. The forward word lattice graph construction process includes: firstly, specifying the direction of each word lattice node as positive, and each word lattice node points to the word lattice node at the next position according to its position in the text; the initial embedding of each word lattice node is generated by the multi-granularity embedding module in the next section, including byte nodes and word nodes; 7. The method for constructing a knowledge base for regulating business behavior based on a bidirectional word grid graph as claimed in claim 6, characterized in that: When constructing the reverse word lattice embedding graph, the direction of the word lattice nodes is opposite to that of the forward word lattice graph, and the other construction steps are the same as the forward word lattice graph construction process. The bidirectional word lattice embedding graph attention network includes aggregating and updating word lattice nodes through the bidirectional word lattice embedding graph attention network to achieve mutual interaction between word nodes and word nodes themselves and nodes; In the bidirectional word lattice embedding graph attention network, each word lattice node v i Initial context embedding Message aggregation process from multi-granularity embedding module; forward word lattice graph: First, define the positive word lattice graph G f =(V f ,E f ), where V f ={v1,v2,…,v n-1 ,v n } represents the forward word node set, that is, the word node set composed of the characters or word segmentation results in the short text, E f ={e 11 ,e 12 ,…,e i,j ,…,e n-1,n } represents the forward edge set, that is, the edge set formed between word lattice nodes; The side e i,j Represents word case node v i and word case node v j There is a one-way connection between them; When aggregating neighbor node information at layer l, in addition to aggregating word node v i In addition to its own features, it also aggregates its forward neighbor node information. The aggregation formula is shown in formula (1): in, Represents the l-th layer node v i The forward node embedding of i Aggregate word node, fw(l) represents the forward node of layer l, R d represents a real vector of length d, and fw(i) represents any forward node v in the lth layer. i ;σ(·) represents the activation function; Represents node v i All forward neighbors of a i j represents the slave node v j To node v i The normalized weight of the edge; W fw represents the trainable parameters of the positive word grid graph, fw represents the positive direction; Among them, f(h i ,h j ) is a scoring function; a, W and V are trainable parameters; represents splicing; in the graph attention network, the normalized weight a of the edge ij The calculation formula is shown in formula (3); The graph attention network uses the attention mechanism to assign different weights to the aggregation between different nodes, so that the weight assigned to the nodes with incorrect word segmentation is small during aggregation, thereby preventing the propagation of word segmentation errors from affecting the model performance. The multi-granularity embedding module is a combined embedding of characters, words, and characters and words; The attention type in the attention network is dynamic attention, and its scoring function is shown in formula (4): Among them, W and V are two trainable parameters, which play the role of linear transformation; Use the same parameter U for linear transformation to achieve information sharing, let U = W = V; the final scoring function is simplified as shown in formula (5): in is the node v at the lth layer i Backward node embedding of i Aggregate word node, bw(l) represents the reverse node of the lth layer, R d represents a real vector of length d, bw(i) represents any forward node v in the lth layer i ;σ(·) represents the activation function; is node v i All reverse neighbors of W fw represents the trainable parameters of the reverse word lattice graph, bw represents reverse; Finally, the forward word node is embedded and backward word lattice node embedding Splice them together to get the word node v i The final context representation For the reverse word lattice graph, it uses the same mechanism as the forward word lattice graph for message aggregation, except that the direction of the edge is opposite to that of the forward word lattice graph. The node aggregation formula is shown in formula (6):

8. A knowledge base construction system for regulating business behavior based on a bidirectional word grid graph using the method according to any one of claims 1 to 7, characterized in that: The training unit uses the trained power grid domain BERT to perform joint training to obtain high-quality feature vectors; The model building unit retains all matching results by building a word lattice graph and builds a bidirectional word lattice graph attention network by using a graph attention network; Vector generation unit, which constructs the grid control operation sentence vector based on Sentence-BERT; The processing unit passes the results of the bidirectional word lattice graph attention network and the sentence vector obtained by SentenceBERT to the Transformer for processing; The output unit completes the joint knowledge extraction of behaviors in the power grid regulation business field and builds a knowledge base.

9. A computer device comprising: Memory and processor; The memory stores a computer program, characterized in that: when the processor executes the computer program, the steps of a method for constructing a knowledge base for regulating business behaviors based on a bidirectional word lattice graph are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a method for constructing a knowledge base for regulating business behaviors based on a bidirectional word lattice graph are implemented.

Images