A power entity recognition method and system fusing domain knowledge and attention mechanism
By constructing a domain corpus in the power field and combining it with an attention mechanism, the BiLSTM-CRF model solves the problems of poor recognition performance and insufficient long-distance dependency in existing technologies, achieving higher entity recognition accuracy and complex entity boundary identification capabilities.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NARI INFORMATION & COMM TECH
- Filing Date
- 2026-03-13
- Publication Date
- 2026-07-21
AI Technical Summary
In existing technologies, the BiLSTM-CRF named entity recognition method based on general word vectors lacks domain knowledge in the application of the power industry, resulting in poor recognition performance. Furthermore, the traditional BiLSTM model cannot effectively capture the long-distance dependencies of complex entities, leading to incorrect boundary judgments.
By collecting documents in the power industry to build a domain corpus, training a domain-specific word vector model, and combining it with an attention-based BiLSTM-CRF model, we can enhance the feature capture capability of power texts, dynamically focus on key information, and establish long-distance semantic dependencies.
It improves the accuracy of power entity recognition and the ability to identify complex entity boundaries, solving the problems of low recognition accuracy and weak long-distance capture capability in existing technologies.
Smart Images
Figure CN122433729A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence and natural language processing technology, and in particular relates to a method and system for recognizing electric entities that integrates domain knowledge and attention mechanisms. Background Technology
[0002] Currently, a large amount of unstructured text data exists in professional fields such as power systems, healthcare, and law. Accurately and automatically extracting key entities (such as power equipment, disease names, and legal clauses) from this text is fundamental to achieving intelligent analysis and application. Taking the power sector as an example, the accurate identification of equipment names in operation ticket texts directly affects the safety and efficiency of power grid operation and maintenance.
[0003] In existing technologies, the BiLSTM-CRF named entity recognition method based on general word vectors is currently a relatively mature sequence labeling technique in the industry. This method first prepares a batch of manually labeled text data (such as the BIO system) as a training set. Then, it constructs a deep learning model consisting of a word embedding layer, a bidirectional long short-term memory (BiLSTM) network layer, and a conditional random field (CRF) layer connected in series. The word embedding layer is typically initialized in one of two ways: either completely randomly initialized, or pre-trained general word vectors (such as Google News word vectors) on a large-scale corpus of general domains (such as news or encyclopedias). Finally, the training data is input into the model for training. After training, the model can predict new text and output entities of predefined categories in the text.
[0004] However, existing technologies have the following problems:
[0005] 1. Existing technologies use general word vectors, which result in insufficient understanding of the "prior knowledge" in specialized fields. General word vectors are trained on general corpora and cannot capture the strong semantic correlation between "circuit breaker" and "disconnect switch" in the power field. This leads to inherent deficiencies in the model when processing specialized texts, resulting in poor recognition performance.
[0006] 2. Traditional BiLSTM models give equal weight to all words in the context. When identifying a core word (such as "lightning arrester"), they cannot focus on the most relevant qualifier in the sentence (such as "II segment busbar"), which makes it easy to make mistakes in the boundary judgment of complex entities. Summary of the Invention
[0007] Purpose of the invention: This invention provides a method and system for recognizing electrical entities that integrates domain knowledge and attention mechanisms, aiming to solve the problems of low entity recognition accuracy and weak long-distance dependent capture capability caused by the lack of domain knowledge and the single weight of contextual features in the prior art.
[0008] Technical solution: This invention provides a method for recognizing electrical entities that integrates domain knowledge and attention mechanisms, comprising:
[0009] A domain corpus is constructed by collecting plain text documents from a preset domain, and the domain corpus is used as input to a word vector model to train the word vector model and obtain a dictionary and a word vector matrix.
[0010] The plain text document is annotated to obtain text containing entity tags; by querying the dictionary, the text containing entity tags is mapped to a sequence of text character IDs containing entity tags.
[0011] An entity recognition model is constructed, comprising a word embedding module, a context encoding module, an attention weighting and feature fusion module, and a label decoding module. The word embedding module converts a text character ID sequence into a dense feature vector sequence; the context encoding module extracts features from the dense feature vector sequence; the attention weighting and feature fusion module uses an attention mechanism to process the extracted features and fuses the processed features with the extracted features to obtain fused features; the label decoding module parses the fused features after fully connected processing to obtain an entity label sequence.
[0012] The text character ID sequence containing entity labels is input into the entity recognition model to train the entity recognition model and obtain the trained entity recognition model.
[0013] Obtain the text sequence to be recognized, and obtain the text character ID sequence to be recognized by querying the dictionary mapping;
[0014] Input the sequence of character IDs of the text to be recognized into the trained entity recognition model to obtain the entity label sequence of the text to be recognized, and decode the final entity text and its position in the text sequence to be recognized.
[0015] Furthermore, the trained word vector model includes:
[0016] The domain corpus is segmented and stop words are filtered out to obtain a structured domain preprocessed corpus. A unique text character ID is assigned to each word in the structured domain preprocessed corpus, establishing a mapping relationship between words and text character IDs to form a dictionary. An unsupervised learning approach is used to construct a Skip-gram-based Word2Vec neural network architecture. The maximum number of iterations and training hyperparameters are set, including word vector dimension, context sliding window size, and minimum word frequency filtering threshold, and the word vector matrix weights are initialized. The structured domain preprocessed corpus is input into the Skip-gram-based Word2Vec neural network architecture. The target characters in the structured domain preprocessed corpus are used to predict the feature characters within the context window range of the target characters, and the stochastic gradient descent algorithm is used to iteratively optimize the training hyperparameters. When the training loss function of the Skip-gram-based Word2Vec neural network architecture converges to the target range or reaches the maximum number of iterations, training stops, and the hidden layer weight matrix is extracted as the word vector matrix.
[0017] Furthermore, the word embedding module takes the text character ID sequence as input, uses the word vector matrix as the weight parameters of the preset neural network model, and outputs a dense feature vector sequence.
[0018] The context encoding module takes the dense feature vector sequence as input, uses a bidirectional long short-term memory network to capture the contextual temporal features of the dense feature vector sequence, and generates a sequence containing bidirectional hidden state features.
[0019] The attention weighting and feature fusion module takes the sequence containing bidirectional hidden state features as input and obtains the fused feature sequence through dot product attention mechanism, masking, weight normalization and feature fusion.
[0020] The tag decoding module uses the Viterbi algorithm to parse the fused feature sequence after fully connected processing to obtain the entity tag sequence.
[0021] Furthermore, the attention weighting and feature fusion module specifically includes:
[0022] The dot product attention mechanism is used to perform batch matrix multiplication on the sequence containing bidirectional hidden state features and its own transpose matrix to calculate the semantic association strength between any two positions in the sequence and obtain the original self-attention score matrix.
[0023] Introduce a mask vector of the same length as the sequence containing bidirectional hidden state features. Fill the positions of the characters to be filled in the original self-attention score matrix with minimum values. Perform masking processing on the original self-attention score matrix to obtain the masked score matrix.
[0024] Perform row-wise operations on the score matrix after masking. The normalization process transforms the original score matrix into an attention weight matrix in the form of a probability distribution.
[0025] The attention weight matrix is multiplied again with the sequence containing bidirectional hidden state features to obtain the context vector sequence after global feature weighting.
[0026] The sequence containing bidirectional hidden state features is concatenated and fused with the context vector sequence after global feature weighting on a preset feature dimension to obtain a fused feature sequence.
[0027] Furthermore, the label decoding module employs a conditional random field algorithm to learn the transition constraint rules between entity label sequences, ensuring that the output entity label sequence is globally optimal and conforms to the syntax rules.
[0028] Furthermore, training the entity recognition model includes:
[0029] Constructing a negative log-likelihood loss function based on conditional random fields;
[0030] The text character ID sequence containing entity labels is divided into training set and validation set according to the preset data partitioning ratio; a mask vector of the same length as the training set is constructed; and the maximum number of iterations is set.
[0031] In each iteration, the loss value between the predicted label sequence output by the entity recognition model and the preset standard label sequence is calculated based on the negative log-likelihood loss function. Invalid gradient information generated by the loss value is filtered out based on the mask vector. The Adam optimizer is used to backpropagate and update the weight parameters of the word embedding module, context encoding module, attention weighting and feature fusion module based on the valid gradient information generated by the loss value. After each training iteration, the validation loss value of the current iteration is calculated using the validation set, and the entity recognition model of the current iteration is saved. If the number of iterations reaches the maximum number of iterations or the entity recognition model has converged, training is stopped, and the entity recognition model with the optimal validation loss value of the current iteration is output as the trained entity recognition model.
[0032] Furthermore, decoding the final entity text and its position in the text sequence to be identified includes: using the BIO decoding method to decode from the entity tag sequence of the text to be identified to obtain the final entity text and its position in the text sequence to be identified.
[0033] This invention also provides an electrical entity recognition system that integrates domain knowledge and attention mechanisms, comprising:
[0034] The collection and pre-training module is used to collect plain text documents in a preset domain to form a domain corpus, and use the domain corpus as input to a word vector model to train the word vector model and obtain a dictionary and a word vector matrix.
[0035] The data annotation and mapping module is used to annotate the plain text document to obtain text containing entity tags; by querying the dictionary, the text containing entity tags is mapped to a sequence of text character IDs containing entity tags.
[0036] An entity recognition model building module is used to build an entity recognition model. It includes a word embedding module, a context encoding module, an attention weighting and feature fusion module, and a label decoding module. The word embedding module converts a text character ID sequence into a dense feature vector sequence. The context encoding module extracts features from the dense feature vector sequence. The attention weighting and feature fusion module uses an attention mechanism to process the extracted features and fuses the processed features with the extracted features to obtain fused features. The label decoding module parses the fully connected fused features to obtain an entity label sequence.
[0037] The model training module is used to input the text character ID sequence containing entity labels into the entity recognition model, train the entity recognition model, and obtain the trained entity recognition model.
[0038] The text mapping module is used to obtain the text sequence to be recognized and to obtain the text character ID sequence to be recognized by querying the dictionary mapping.
[0039] The model entity recognition module is used to input the text character ID sequence to be recognized into the trained entity recognition model, obtain the entity label sequence of the text to be recognized, and decode the final entity text and its position in the text sequence to be recognized.
[0040] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method.
[0041] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described method.
[0042] Beneficial effects: Compared with the prior art, the present invention has the following advantages:
[0043] 1. Solving the problem of low recognition accuracy caused by the uniqueness of domain terminology: In view of the lack of knowledge injection into the operation ticket maintenance domain in existing technologies, this invention injects rich domain prior knowledge into the model by pre-training and loading domain-specific word vectors, so that the model has a deeper semantic understanding of the equipment professional terms in the operation ticket, thereby improving the accuracy of entity recognition.
[0044] 2. Enhanced ability to capture long-distance dependency features: In view of the shortcomings of existing technologies, such as single context feature weights and inability to effectively handle long sentences, this invention introduces a self-attention mechanism, which enables the model to dynamically focus on key information in the sentence, effectively establish long-distance semantic dependencies, and improve the model's ability to capture context features.
[0045] 3. Improve the accuracy of identifying complex entity boundaries: This invention integrates the original context features that retain the adjacent relationships with the weighted context features that establish long-distance dependencies, providing a more comprehensive and robust judgment basis for the final tag decoding, thereby enhancing the ability to identify the boundaries of power grid equipment name entities in long and difficult operation tickets. Attached Figure Description
[0046] Figure 1 This is a flowchart of the overall process of the method of the present invention.
[0047] Figure 2 This is a diagram of the core model architecture of the system of this invention. Detailed Implementation
[0048] like Figure 1 As shown, the present invention provides a method for recognizing electrical entities that integrates domain knowledge and attention mechanisms, comprising:
[0049] S1. Collect plain text documents (such as historical power operation tickets, dispatching procedures, technical documents, etc.) in a preset domain to form a domain corpus, and use the domain corpus as input to a word vector model to train the word vector model and obtain a dictionary and a word vector matrix.
[0050] The trained word vector model specifically includes:
[0051] The domain corpus is segmented and stop words are filtered out to obtain a structured domain preprocessed corpus.
[0052] Assign a unique text character ID to each word in the structured domain preprocessing corpus, establish a mapping relationship between words and text character IDs, and form a dictionary;
[0053] An unsupervised learning approach was used to construct a Word2Vec neural network architecture based on Skip-gram;
[0054] Set the maximum number of iterations, set the training hyperparameters, including word vector dimension, context sliding window size, and minimum word frequency filtering threshold, and initialize the word vector matrix weights;
[0055] The structured domain preprocessed corpus is input into the Skip-gram-based Word2Vec neural network architecture. The target characters in the structured domain preprocessed corpus are used to predict the feature characters within the context window of the target characters in order to learn the semantic associations between characters. The training hyperparameters are iteratively optimized using the stochastic gradient descent algorithm.
[0056] When the training loss function of the Skip-gram-based Word2Vec neural network architecture converges to the target range or reaches the maximum number of iterations, training stops, and the hidden layer weight matrix is extracted as the word vector matrix, thereby mapping prior knowledge in the power field to a high-dimensional continuous vector space.
[0057] The word vector matrix can capture the unique semantics of terms within a domain. For example, in this vector space, the vector representations of "transformer" and "circuit breaker" are closer to those of "computer" than they are to "transformer". In this embodiment, the word vector dimension is set to 128 dimensions.
[0058] S2. Use the BIO annotation method to annotate the plain text document to obtain text containing power entity tags; by querying the dictionary, map the text containing power entity tags to a text character ID sequence containing power entity tags, so as to complete the conversion of the original text information and annotation information into a digital matrix.
[0059] S3. Construct an entity recognition model (Embedding + BiLSTM + Attention + CRF). This model is a multi-layered neural network model, including a word embedding module, a context encoding module, an attention weighting and feature fusion module, and a label decoding module, such as... Figure 2 As shown;
[0060] The word embedding module (Embedding layer) takes the text character ID sequence as input, uses the word vector matrix as the weight parameters of the preset neural network model, and outputs a dense feature vector sequence.
[0061] The context encoding module (BiLSTM layer) takes the dense feature vector sequence as input and uses a bidirectional long short-term memory network to capture the contextual temporal features of the dense feature vector sequence from both forward and backward dimensions, generating a sequence H containing bidirectional hidden state features; in this embodiment, the hidden layer dimension of the bidirectional long short-term memory network is set to 256 dimensions.
[0062] The attention weighting and feature fusion module (Attention & Concatenation layer) takes the sequence H containing bidirectional hidden state features as input. Where, is the batch size. For sequence length, For feature dimensions;
[0063] Calculating the self-attention score matrix: Using a dot product attention mechanism, the sequence H containing bidirectional hidden state features is subjected to batch matrix multiplication with its own transpose matrix. The semantic association strength between any two positions in the sequence is calculated to obtain the original self-attention score matrix. The calculation formula is:
[0064] ;
[0065] This invention introduces a self-attention mechanism, enabling entity recognition models to dynamically focus on key information in sentences, effectively establishing long-distance semantic dependencies, and improving the contextual feature capture capability of entity recognition models.
[0066] Masking: Introduce a mask vector of the same length as the sequence H containing the bidirectional hidden state features, and fill the original self-attention score matrix with characters ( <pad>Fill the position with the minimum value (e.g.) The original self-attention score matrix is masked to obtain a masked score matrix, thereby eliminating the interference of invalid padding characters on the model's attention and ensuring that the model only focuses on the actual effective words in the power text.
[0067] Weight normalization: This involves normalizing the weights of the score matrix after masking along the row dimension. Normalization transforms the original score matrix into an attention weight matrix in probability distribution form. Each row represents the distribution of the current character's level of attention to the entire sentence;
[0068] Generate a global context vector: This involves adjusting the attention weight matrix. Perform matrix multiplication again with the sequence H containing bidirectional hidden state features to obtain a context vector sequence weighted by global features. ;
[0069] Multidimensional feature fusion: The sequence H containing bidirectional hidden state features is combined with the context vector sequence weighted by global features. In feature dimension The above steps are performed to splice and fuse the components, resulting in a fused feature sequence. ;
[0070] This invention integrates the original context features that retain the adjacent relationships with the weighted context features that establish long-distance dependencies, providing a more comprehensive and robust basis for the final tag decoding, and enhancing the boundary identification capability of power grid equipment name entities (such as "220kV II section bus 7 voltage transformer surge arrester") in long and difficult operation tickets.
[0071] Based on the fused feature sequence F, a preliminary label sequence is generated through a fully connected layer;
[0072] The tag decoding module (CRF layer) takes the preliminary tag sequence as input and uses the Viterbi algorithm to parse the preliminary tag sequence to obtain the entity tag sequence. This module adopts the Conditional Random Field algorithm to learn the transition constraint rules between entity tag sequences (e.g., the "I-DEV" tag cannot appear after the "O" tag) to ensure that the final output entity tag sequence is globally optimal and conforms to the syntax rules.
[0073] This invention injects rich domain prior knowledge into the entity recognition model by pre-training and loading domain-specific word vectors, enabling the entity recognition model to have a deeper semantic understanding of the equipment terminology in the operation ticket, thereby improving the accuracy of entity recognition.
[0074] S4. Train the entity recognition model by taking a text character ID sequence containing power entity labels as input and constructing a negative log-likelihood loss function based on a conditional random field (CRF).
[0075] Divide the text character ID sequence containing entity labels into training set and validation set according to the preset data division ratio; construct a mask vector of the same length as the training set; set the maximum number of iterations (e.g., 50).
[0076] In each iteration, the loss value between the predicted label sequence output by the entity recognition model and the preset standard label sequence is calculated based on the negative log-likelihood loss function. Based on the mask vector, invalid gradient information corresponding to the fill characters generated by the loss value is filtered out. The mask vector constrains the model to optimize parameters only for valid text features. The Adam optimizer is used to backpropagate and update the weight parameters of the word embedding module, context encoding module, attention weighting, and feature fusion module based on the valid gradient information generated by the loss value. After each training iteration, the validation loss value for the current iteration is calculated using the validation set, and the entity recognition model for the current iteration is saved. If the iteration reaches the maximum number of iterations or the entity recognition model has converged, training stops, and the entity recognition model with the optimal validation loss value for the current iteration is output as the trained entity recognition model.
[0077] S5. Obtain the text sequence to be recognized, wherein the text sequence to be recognized does not contain entity labels; obtain the text character ID sequence to be recognized by querying the dictionary mapping;
[0078] S6. Input the character ID sequence of the text to be recognized into the trained entity recognition model to obtain the entity label sequence of the text to be recognized. Then, decode the entity label sequence of the text to be recognized using the BIO decoding method to obtain the final entity text and its position in the text sequence to be recognized.
[0079] Specifically: The text sequence to be recognized and the entity label sequence of the text to be recognized are traversed bit by bit in a synchronous manner to identify the prefix and category information in the entity label sequence of the text to be recognized;
[0080] When a tag with the prefix B- is encountered, it is determined to be the start of a new entity. If an old entity that is being constructed already exists, the old entity and its type are first stored in the entity set. Then, the text content of the new entity is initialized with the character at the current traversal position in the text sequence to be identified, and the suffix of the tag is extracted as the type of the new entity.
[0081] When a tag with the prefix I- is encountered, its suffix category is extracted and matched with the type of the entity currently being built. If the types match, the character at the current traversal position in the text sequence to be identified is appended to the entity text content. If the types do not match or there is no entity currently being built, the entity building state is reset.
[0082] When a tag with the prefix 'O' is encountered, if an entity is currently being built, its text content and type are stored in the entity collection, and the current entity building state is cleared.
[0083] After the traversal is complete, check if there are any entities under construction that have not yet been stored in the collection. If so, store their complete text and type in the entity collection. Finally, output a structured list containing all entity text, category information and their start and end position indices in the text sequence to be identified.
[0084] The present invention discloses an electric entity recognition system that integrates domain knowledge and attention mechanisms, comprising:
[0085] The collection and pre-training module is used to collect plain text documents in a preset domain to form a domain corpus, and use the domain corpus as input to a word vector model to train the word vector model and obtain a dictionary and a word vector matrix.
[0086] The data annotation and mapping module is used to annotate the plain text document to obtain text containing entity tags; by querying the dictionary, the text containing entity tags is mapped to a sequence of text character IDs containing entity tags.
[0087] An entity recognition model building module is used to build an entity recognition model. It includes a word embedding module, a context encoding module, an attention weighting and feature fusion module, and a label decoding module. The word embedding module converts a text character ID sequence into a dense feature vector sequence. The context encoding module extracts features from the dense feature vector sequence. The attention weighting and feature fusion module uses an attention mechanism to process the extracted features and fuses the processed features with the extracted features to obtain fused features. The label decoding module parses the fully connected fused features to obtain an entity label sequence.
[0088] The model training module is used to input the text character ID sequence containing entity labels into the entity recognition model, train the entity recognition model, and obtain the trained entity recognition model.
[0089] The text mapping module is used to obtain the text sequence to be recognized and to obtain the text character ID sequence to be recognized by querying the dictionary mapping.
[0090] The model entity recognition module is used to input the text character ID sequence to be recognized into the trained entity recognition model, obtain the entity label sequence of the text to be recognized, and decode the final entity text and its position in the text sequence to be recognized.
[0091] In order to achieve the purpose of this invention, in addition to the above-described embodiments, in terms of domain knowledge injection, besides using Word2Vec, other word vector pre-training models such as GloVe and FastText can also be used, or more advanced pre-trained language models (such as BERT and ERNIE) can be used to perform secondary pre-training on domain corpora, which can also achieve the purpose of injecting domain knowledge.
[0092] In terms of model architecture, the BiLSTM layer in the context encoding module can be replaced by a GRU (Gated Recurrent Unit) or the Encoder layer of a Transformer. The self-attention mechanism can also be replaced with other variants, such as multi-head self-attention, to capture richer features from different subspaces.< / pad>
Claims
1. A method for recognizing electrical entities that integrates domain knowledge and attention mechanisms, characterized in that, Includes: A domain corpus is constructed by collecting plain text documents from a preset domain, and the domain corpus is used as input to a word vector model to train the word vector model and obtain a dictionary and a word vector matrix. The plain text document is annotated to obtain text containing entity tags; by querying the dictionary, the text containing entity tags is mapped to a sequence of text character IDs containing entity tags. An entity recognition model is constructed, comprising a word embedding module, a context encoding module, an attention weighting and feature fusion module, and a label decoding module. The word embedding module converts a text character ID sequence into a dense feature vector sequence; the context encoding module extracts features from the dense feature vector sequence; the attention weighting and feature fusion module uses an attention mechanism to process the extracted features and fuses the processed features with the extracted features to obtain fused features; the label decoding module parses the fused features after fully connected processing to obtain an entity label sequence. The text character ID sequence containing entity labels is input into the entity recognition model to train the entity recognition model and obtain the trained entity recognition model. Obtain the text sequence to be recognized, and obtain the text character ID sequence to be recognized by querying the dictionary mapping; Input the sequence of text character IDs to be recognized into the trained entity recognition model to obtain the entity label sequence of the text to be recognized, and decode the final entity text and its position in the text sequence to be recognized.
2. The method for recognizing power entities by fusing domain knowledge and attention mechanisms according to claim 1, characterized in that, The trained word vector model includes: The domain corpus is segmented and stop words are filtered out to obtain a structured domain preprocessed corpus. A unique text character ID is assigned to each word in the structured domain preprocessed corpus, establishing a mapping relationship between words and text character IDs to form a dictionary. An unsupervised learning approach is used to construct a Skip-gram-based Word2Vec neural network architecture. The maximum number of iterations and training hyperparameters are set, including word vector dimension, context sliding window size, and minimum word frequency filtering threshold, and the word vector matrix weights are initialized. The structured domain preprocessed corpus is input into the Skip-gram-based Word2Vec neural network architecture. The target characters in the structured domain preprocessed corpus are used to predict the feature characters within the context window range of the target characters, and the stochastic gradient descent algorithm is used to iteratively optimize the training hyperparameters. When the training loss function of the Skip-gram-based Word2Vec neural network architecture converges to the target range or reaches the maximum number of iterations, training stops, and the hidden layer weight matrix is extracted as the word vector matrix.
3. The method for recognizing power entities by fusing domain knowledge and attention mechanisms according to claim 1, characterized in that, The word embedding module takes the text character ID sequence as input, uses the word vector matrix as the weight parameters of the preset neural network model, and outputs a dense feature vector sequence. The context encoding module takes the dense feature vector sequence as input, uses a bidirectional long short-term memory network to capture the contextual temporal features of the dense feature vector sequence, and generates a sequence containing bidirectional hidden state features. The attention weighting and feature fusion module takes the sequence containing bidirectional hidden state features as input and obtains the fused feature sequence through dot product attention mechanism, masking, weight normalization and feature fusion. The tag decoding module uses the Viterbi algorithm to parse the fused feature sequence after fully connected processing to obtain the entity tag sequence.
4. The method for recognizing power entities by fusing domain knowledge and attention mechanisms according to claim 3, characterized in that, The attention weighting and feature fusion module specifically includes: The dot product attention mechanism is used to perform batch matrix multiplication on the sequence containing bidirectional hidden state features and its own transpose matrix to calculate the semantic association strength between any two positions in the sequence and obtain the original self-attention score matrix. Introduce a mask vector of the same length as the sequence containing bidirectional hidden state features. Fill the positions of the characters to be filled in the original self-attention score matrix with minimum values. Perform masking processing on the original self-attention score matrix to obtain the masked score matrix. Perform row-wise operations on the score matrix after masking. The normalization process transforms the original score matrix into an attention weight matrix in the form of a probability distribution. The attention weight matrix is multiplied again with the sequence containing bidirectional hidden state features to obtain the context vector sequence after global feature weighting. The sequence containing bidirectional hidden state features is concatenated and fused with the context vector sequence after global feature weighting on a preset feature dimension to obtain a fused feature sequence.
5. The method for recognizing power entities by fusing domain knowledge and attention mechanisms according to claim 3, characterized in that, The label decoding module uses a conditional random field algorithm to learn the transition constraint rules between entity label sequences, ensuring that the output entity label sequence is globally optimal and conforms to the syntax rules.
6. The method for recognizing power entities by fusing domain knowledge and attention mechanisms according to claim 1, characterized in that, The training of the entity recognition model includes: Constructing a negative log-likelihood loss function based on conditional random fields; The text character ID sequence containing entity labels is divided into training set and validation set according to the preset data partitioning ratio; a mask vector of the same length as the training set is constructed; and the maximum number of iterations is set. In each iteration, the loss value between the predicted label sequence output by the entity recognition model and the preset standard label sequence is calculated based on the negative log-likelihood loss function. Invalid gradient information generated by the loss value is filtered out based on the mask vector. The Adam optimizer is used to backpropagate and update the weight parameters of the word embedding module, context encoding module, attention weighting and feature fusion module based on the valid gradient information generated by the loss value. After each training iteration, the validation loss value of the current iteration is calculated using the validation set, and the entity recognition model of the current iteration is saved. If the number of iterations reaches the maximum number of iterations or the entity recognition model has converged, training is stopped, and the entity recognition model with the optimal validation loss value of the current iteration is output as the trained entity recognition model.
7. The method for recognizing power entities by fusing domain knowledge and attention mechanisms according to claim 1, characterized in that, Decoding the final entity text and its position in the text sequence to be identified includes: using the BIO decoding method to decode from the entity tag sequence of the text to be identified to obtain the final entity text and its position in the text sequence to be identified.
8. A power entity recognition system integrating domain knowledge and attention mechanisms, characterized in that, include: The collection and pre-training module is used to collect plain text documents in a preset domain to form a domain corpus, and use the domain corpus as input to a word vector model to train the word vector model and obtain a dictionary and a word vector matrix. The data annotation and mapping module is used to annotate the plain text document to obtain text containing entity tags; by querying the dictionary, the text containing entity tags is mapped to a sequence of text character IDs containing entity tags. An entity recognition model building module is used to build an entity recognition model. It includes a word embedding module, a context encoding module, an attention weighting and feature fusion module, and a label decoding module. The word embedding module converts a text character ID sequence into a dense feature vector sequence. The context encoding module extracts features from the dense feature vector sequence. The attention weighting and feature fusion module uses an attention mechanism to process the extracted features and fuses the processed features with the extracted features to obtain fused features. The tag decoding module parses the fused features after fully connected processing to obtain the entity tag sequence; The model training module is used to input the text character ID sequence containing entity labels into the entity recognition model, train the entity recognition model, and obtain the trained entity recognition model. The text mapping module is used to obtain the text sequence to be recognized and to obtain the text character ID sequence to be recognized by querying the dictionary mapping. The model entity recognition module is used to input the text character ID sequence to be recognized into the trained entity recognition model, obtain the entity label sequence of the text to be recognized, and decode the final entity text and its position in the text sequence to be recognized.
9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.