Power operation and maintenance text-oriented information enhancement named entity recognition system and equipment

Abstract

The invention provides an information enhancement named entity recognition system and equipment for a power operation and maintenance text. The system comprises a data set construction module used for carrying out error correction on characters in an operation and maintenance work order of the power equipment, determining entity categories and carrying out entity labeling; the word embedding module is used for converting a discrete symbol form of a word into a continuous vector representation; the position perception global attention mechanism module is used for carrying out position coding and attention score calculation and obtaining association information among different samples; the fine granularity enhancement module is used for capturing local information and identifying entities in the operation and maintenance work order more accurately; the conditional random field module is used for constructing a probabilistic framework and capturing a dependency relationship between tags; and the visual interaction and storage module is used for performing visual interaction of electric power operation and maintenance text named entity recognition. According to the invention, standardization of equipment management is facilitated, the operation and maintenance cost is reduced, and decision support is optimized.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of power operation and maintenance, and in particular, to an information-enhanced named entity recognition system and device for power operation and maintenance texts. Background Art

[0002] With the continuous increase in power demand, the workload of power equipment operation and maintenance is also increasing. The operation and maintenance data generated during the equipment operation and maintenance work is crucial for analyzing the operation status of the equipment and constructing a smart grid. The power equipment operation and maintenance work order is the main source of operation and maintenance data, which mainly records the specific information during the operation of the operation and maintenance staff, including the equipment name, line, damaged parts, and specific operation and maintenance content, etc. However, most of the existing power equipment operation and maintenance work orders are unstructured texts, and it is difficult and time-consuming to accurately extract the information therein. Based on this, how to efficiently extract information from a large number of unstructured power equipment operation and maintenance texts has become a key problem to be solved urgently. However, it is difficult to extract entity information from power equipment operation and maintenance work orders because the work orders often contain a large number of professional terms and industry-specific abbreviated expressions; secondly, the work orders are recorded by different operation and maintenance personnel, and their writing styles vary greatly and the texts often have problems such as non-standard expressions and incomplete descriptions. Traditional natural language processing methods in the general field often cannot achieve ideal results when dealing with these texts in professional fields, cannot better identify entity boundaries, and do not extract enough global information of the dataset. Therefore, developing an information-enhanced named entity recognition system and device for power operation and maintenance texts to effectively overcome the defects in the above-related technologies has become a technical problem urgently to be solved in the industry. Summary of the Invention

[0003] In view of the above problems existing in the prior art, the embodiments of the present invention provide an information-enhanced named entity recognition system and device for power operation and maintenance texts.

[0004] In a first aspect, an embodiment of the present invention provides an information-enhanced named entity recognition system for power operation and maintenance texts, including: a dataset construction module for correcting characters in power equipment operation and maintenance work orders, determining entity categories, and performing entity annotation; a word embedding module for converting words from a discrete symbol form into a continuous vector representation; a position-aware global attention mechanism module for performing position encoding and attention score calculation, and obtaining the association information between different samples; a fine-grained enhancement module for capturing local information to more accurately identify entities in the operation and maintenance work orders; a conditional random field module for constructing a probability framework and capturing the dependencies between labels; a visualization interaction and storage module for performing visualization interaction of named entity recognition of power operation and maintenance texts.

[0005] Based on the content of the above system embodiments, the information-enhanced named entity recognition system for power operation and maintenance texts provided in the embodiments of the present invention, the dataset construction module includes: a preprocessing module for removing duplicates from the acquired data and eliminating content containing useless characters, missing information, and incorrect statements; an entity category determination module for determining device names, voltage levels, affiliated lines, affiliated substations, damaged parts, specific operation and maintenance content, and time points; an entity annotation module for annotating text data, adopting the BMES annotation system, where "B" represents the start of an entity, "M" represents the middle part of the entity, "E" represents the end of the entity, and "S" represents a single-character entity.

[0006] Based on the content of the above system embodiments, the information-enhanced named entity recognition system for power operation and maintenance texts provided in the embodiments of the present invention, the conversion of words from a discrete symbol form to a continuous vector representation includes: using BERT-wwm-ext to perform character embedding on the input sequence, enhancing the word embedding of the Chinese corpus, which consists of 12 Transformer layers, each layer having 768 hidden units and 12 attention heads.

[0007] Based on the content of the above system embodiments, the information-enhanced named entity recognition system for power operation and maintenance texts provided in the embodiments of the present invention, the performing of position encoding and attention score calculation, and obtaining the correlation information between different samples includes:

[0008]

[0009]

[0010]

[0011]

[0012]

[0013]

[0014]

[0015] Among them, Attn is the position-aware global attention mechanism; softmax is the softmax function; is the position-aware attention score between positions t and j; V is the value matrix; is the global information; is the query Query vector of the token at position t; is the key Key vector of the token at position j; It is the relative position embedding vector between two tokens; u is the first learnable bias vector, which adjusts the attention according to the inherent attributes of the tokens; v is the second learnable bias vector, which adjusts the attention according to the direction and distance between two tokens, further enhancing the model's ability to capture spatial and sequential relationships; T is the matrix transpose symbol; t is the current position; j is the position to be attended to; i is the frequency index of the position encoding, and the value range is [0, d k / 2]; is the dimension of the key vector; L-DNorm is the learnable double normalization strategy; Q is the query matrix; K is the key matrix; , are two globally shared learnable memory units, where, helps calculate the similarity to generate the attention weights, updates the input features by combining with the attention weights, thus affecting the final output; X is the input sequence; , , is the parameter matrix used to map the input X to different spaces.

[0016] Based on the content of the above system embodiment, in the information-enhanced named entity recognition system for power operation and maintenance text provided in the embodiment of the present invention, the learnable double normalization strategy includes:

[0017]

[0018]

[0019]

[0020] Among them, is the intermediate matrix element obtained by matrix multiplication, representing the initial association value of the i-th row and j-th column; is the value after column normalization; is the learnable parameter for adjusting the column normalization intensity; exp is the exponential function; is the original association value of the n-th row and j-th column; n is the row and column dimension index; is the output value after the final double normalization; is the value of the i-th row and n-th column in the intermediate matrix after column normalization; is the learnable parameter for adjusting the row normalization intensity.

[0021] Based on the content of the above system embodiment, in the information-enhanced named entity recognition system for power operation and maintenance text provided in the embodiment of the present invention, the fine-grained enhancement module includes:

[0022]

[0023]

[0024]

[0025] Among them, is the result of fusing the non - linear features of the second convolutional layer with the first convolutional layer features as the leading, capturing the local details focused by different convolutional kernels in the input; Norm is the layer normalization operation; is the first one - dimensional convolutional layer; is the second one - dimensional convolutional layer; is the Hadamard product symbol; GELU is the activation function; is the result of fusing the non - linear features of the first convolutional layer with the second convolutional layer features as the leading, mining potential fine - grained semantic differences from another perspective; Out is the final output of the fine - grained enhancement module; is the third one - dimensional convolutional layer.

[0026] Based on the content of the above system embodiments, the information - enhanced named entity recognition system for power operation and maintenance texts provided in the embodiments of the present invention, the constructing a probability framework and capturing dependencies between tags includes:

[0027]

[0028] Among them, P(Y|X) is the conditional probability; is the quantization symbol of the compatibility between adjacent tags; is the evaluation symbol of the appropriateness of assigning the tag at position i; Y represents a complete tag sequence; is the set of all possible tagging sequences corresponding to the observation sequence X; is the quantization symbol of the compatibility between the tag at position i and the tag at position i + 1 among all possible tag sequences in the denominator; is the evaluation symbol of the appropriateness of assigning a specific tag at position i among all possible tag sequences in the denominator.

[0029] Based on the content of the above system embodiments, the information - enhanced named entity recognition system for power operation and maintenance texts provided in the embodiments of the present invention, the visual interaction for named entity recognition of power operation and maintenance texts includes: inputting operation and maintenance work orders, automatically extracting corresponding information, batch - processing multiple operation and maintenance work orders, and batch - exporting the recognition results.

[0030] In a second aspect, embodiments of the present invention provide an electronic device, including:

[0031] At least one processor, at least one memory, a communication interface, and a bus; wherein,

[0032] The processor, the memory, and the communication interface complete communication with each other through the bus;

[0033] The memory stores program instructions executable by the processor, and the processor calls the program instructions to be able to execute the information-enhanced named entity recognition system for power operation and maintenance texts provided by any one of the various implementation manners of the first aspect.

[0034] In a third aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium. The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions enable a computer to implement the information-enhanced named entity recognition system for power operation and maintenance texts provided by any one of the various implementation manners of the first aspect.

[0035] The information-enhanced named entity recognition system and device for power operation and maintenance texts provided by the embodiments of the present invention can, through system integration of a data set construction module, a word embedding module, a position-aware global attention mechanism module, a fine-grained enhancement module, a conditional random field module, and a visualization interaction and storage module, achieve key information extraction for power operation and maintenance texts, improve the efficiency and intelligent level of power equipment operation and maintenance management, and contribute to promoting equipment management standardization, reducing operation and maintenance costs, and optimizing decision support. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0037] Figure 1 It is a schematic structural diagram of the information-enhanced named entity recognition system for power operation and maintenance texts provided by an embodiment of the present invention;

[0038] Figure 2 It is a schematic entity structure diagram of the electronic device provided by an embodiment of the present invention;

[0039] Figure 3 It is a schematic entity category division structure diagram provided by an embodiment of the present invention;

[0040] Figure 4 It is a schematic structural diagram of the fine-grained information enhancement module provided by an embodiment of the present invention;

[0041] Figure 5Schematic diagram of the power equipment operation and maintenance management system provided by the embodiment of the present invention;

[0042] Figure 6 Schematic diagram of the display effect of the function page of the employee registration and management module provided by the embodiment of the present invention;

[0043] Figure 7 Schematic diagram of the display effect of the function page of the operation and maintenance work order information extraction module provided by the embodiment of the present invention;

[0044] Figure 8 Schematic diagram of the display effect of the function page of the operation and maintenance work order structured information management module provided by the embodiment of the present invention;

[0045] Figure 9 Schematic diagram of the dynamic effect of model training provided by the embodiment of the present invention;

[0046] Figure 10(a) is a schematic diagram of the heat map effect of the confusion matrix of the baseline model provided by the embodiment of the present invention;

[0047] Figure 10(b) is a schematic diagram of the heat map effect of the confusion matrix of the proposed model provided by the embodiment of the present invention. Detailed implementation manners

[0048] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention. In addition, the technical features in each embodiment or a single embodiment provided by the present invention can be combined with each other arbitrarily to form a feasible technical solution. This combination is not restricted by the order of steps and / or the mode of structural composition, but must be based on the fact that those of ordinary skill in the art can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the protection scope required by the present invention.

[0049] The embodiment of the present invention provides an information-enhanced named entity recognition system for power operation and maintenance texts. See Figure 1, the system includes: a dataset construction module for correcting characters in power equipment operation and maintenance work orders, determining entity categories, and performing entity annotation; a word embedding module for converting words from a discrete symbolic form into a continuous vector representation; a position-aware global attention mechanism module for performing position encoding and attention score calculation and obtaining correlation information between different samples; a fine-grained enhancement module for capturing local information to more accurately identify entities in operation and maintenance work orders; a conditional random field module for constructing a probability framework and capturing dependencies between tags; and a visualization interaction and storage module for performing visualization interaction of named entity recognition for power operation and maintenance texts.

[0050] Based on the content of the above system embodiment, as an alternative embodiment, in the information-enhanced named entity recognition system for power operation and maintenance texts provided in the embodiments of the present invention, the dataset construction module includes: a preprocessing module for de-duplicating the acquired data and removing content containing useless characters, missing information, and incorrect statements; an entity category determination module for determining the equipment name, voltage level, affiliated line, affiliated substation, damaged part, specific operation and maintenance content, and time point; and an entity annotation module for annotating the text data using the BMES annotation system, where "B" represents the start of an entity, "M" represents the middle part of an entity, "E" represents the end of an entity, and "S" represents a single-character entity.

[0051] Specifically, all data in the preprocessing module is sourced from power equipment operation and maintenance work orders recorded in the State Grid Production Management System (PMS). The acquired data is preprocessed by first de-duplicating and then removing records containing useless characters, a large amount of missing information, and incorrect statements. The main purpose of the NER task for power equipment operation and maintenance work orders in the entity category determination module is to extract key information on equipment operation and maintenance, facilitating further management and analysis of equipment by power enterprises, including equipment life prediction, defect correlation analysis, and construction of smart grids. Based on the above considerations, 7 entity categories with the most analysis value are determined: equipment name, voltage level, affiliated line, affiliated substation, damaged part, specific operation and maintenance content, and time, as Figure 3 shown.

[0052] Based on the content of the above system embodiment, as an alternative embodiment, in the information-enhanced named entity recognition system for power operation and maintenance texts provided in the embodiments of the present invention, the conversion of words from a discrete symbolic form into a continuous vector representation includes: using BERT-wwm-ext to perform character embedding on the input sequence, enhancing the word embedding of the Chinese corpus, which consists of 12 Transformer layers, each with 768 hidden units and 12 attention heads.

[0053] Specifically, character embedding is essential in the named entity recognition (NER) task because it converts words from a discrete symbolic form to a continuous vector representation. This process effectively reduces the dimensionality of the word space, creating a low-dimensional continuous mapping. By incorporating character-level information, the model can obtain a richer understanding of the semantics of words in the surrounding text, enhancing its ability to interpret context and accurately identify entities.

[0054] Based on the content of the above system embodiments, as an alternative embodiment, in the information-enhanced named entity recognition system for power operation and maintenance texts provided in the embodiments of the present invention, performing position encoding and attention score calculation, and obtaining the association information between different samples includes:

[0055]

[0056]

[0057]

[0058]

[0059]

[0060]

[0061]

[0062] Among them, Attn is the position-aware global attention mechanism; softmax is the softmax function; is the position-aware attention score between positions t and j; V is the value matrix; is the global information; is the query vector of the token at position t; is the key vector of the token at position j; is the relative position embedding vector between two tokens; u is the first learnable bias vector, which adjusts the attention according to the inherent attributes of the token; v is the second learnable bias vector, which adjusts the attention according to the direction and distance between two tokens, further enhancing the model's ability to capture spatial and sequential relationships; T is the matrix transpose symbol; t is the current position; j is the position to be attended to; i is the frequency index of the position encoding, with a value range of [0, d k / 2]; is the dimension of the key vector; L-DNorm is the learnable double normalization strategy; Q is the query matrix; K is the key matrix; , are two globally shared learnable memory units, where, Help calculate similarity to generate attention weights, Update the input features by combining with the attention weights, thereby affecting the final output; X is the input sequence; , , is a parameter matrix used to map the input X to different spaces.

[0063] Specifically, the Transformer is the most mainstream deep learning model, which benefits from its core attention mechanism, making the Transformer far superior to other models in obtaining long-range dependencies and achieving efficient parallel processing. In the Transformer model, the input sequence is processed simultaneously, lacking inherent sequence dependencies. Although this improves computational efficiency, it is difficult for the model to obtain the order of the input sequence. Therefore, positional embeddings are introduced in the Transformer model to explicitly represent the position information of each token, enabling the model to utilize this information to distinguish and understand the sequential structure of the data.

[0064] However, in the original Transformer encoder, the positional encoding is obtained through a linear combination of sine and cosine functions. This way of positional encoding enhances the model's distance perception ability, but it is difficult for the model to distinguish from which direction different tokens come. On the other hand, the original self-attention mechanism only focuses on the relationships between features at different positions within a single sample, ignoring the potential associations that may exist between different samples. To solve the above problems, a position-aware global attention mechanism is designed as shown in equations (1) to (6). This attention mechanism adopts a new way of positional encoding and calculation of attention scores, and introduces two globally learnable units to obtain the association information between different samples.

[0065] Based on the content of the above system embodiment, as an optional embodiment, in the information enhancement named entity recognition system for power operation and maintenance texts provided in the embodiments of the present invention, the learnable double normalization strategy includes:

[0066] (7)

[0067] (8)

[0068] (9)

[0069] Among them, is the intermediate matrix element obtained by matrix multiplication, representing the initial association value of the i-th row and j-th column; is the value after column normalization; is a learnable parameter for adjusting the intensity of column normalization; exp is the exponential function; is the original association value of the n-th row and the j-th column; n is the row-column dimension index; is the output value after final double normalization; is the value of the i-th row and the n-th column in the intermediate matrix after column normalization; is a learnable parameter for adjusting the row normalization intensity.

[0070] Specifically, an overly balanced attention distribution may affect the feature recognition ability of the model. Therefore, two learnable parameters α and β are set, and a learnable double normalization strategy as shown in equations (7) to (9) (L-DNorm) is constructed, enabling the model to dynamically adjust the normalization intensity during training.

[0071] Based on the content of the above system embodiment, as an optional embodiment, in the information enhancement named entity recognition system for power operation and maintenance texts provided in the embodiments of the present invention, the fine-grained enhancement module includes:

[0072] (10)

[0073] (11)

[0074] (12)

[0075] Among them, is the result of fusing the non-linear features of the second convolutional layer dominated by the features of the first convolutional layer, capturing the local details focused by different convolutional kernels in the input; Norm is the layer normalization operation; is the first one-dimensional convolutional layer; is the second one-dimensional convolutional layer; is the Hadamard product symbol; GELU is the activation function; is the result of fusing the non-linear features of the first convolutional layer dominated by the features of the second convolutional layer, mining potential fine-grained semantic differences from another perspective; Out is the final output of the fine-grained enhancement module; is the third one-dimensional convolutional layer.

[0076] Specifically, the position-aware global attention mechanism is good at processing global context information, but may not be accurate enough in detail capture. Especially in the named entity recognition task, the model needs to accurately identify specific entities with a small word count in a long sentence. Therefore, an additional fine-grained information enhancement module parallel to the position-aware global attention mechanism is designed to improve the model's ability to capture local information, enabling the model to more accurately identify entities in the text. The specific structure of this module is as Figure 4As shown in the figure, it is mainly composed of two parallel convolutional networks. CNN is good at extracting features from local windows and can capture phrase-level word group features. These local features are particularly helpful for named entity recognition. The fine-grained information enhancement module uses two convolutional layers with different convolutional kernel sizes in parallel. The small convolutional kernel is used to capture local fine-grained features, and the large convolutional kernel is used to capture relatively longer-range context dependencies. An interactive design is adopted between the two convolutional layers. The output of each layer will modulate the feature extraction of the other layer, and feature interaction is achieved through element-wise multiplication. The calculation method of this module is shown in equations (10) to (12). Conv1 and Conv2 represent two one-dimensional convolutional layers with convolutional kernel sizes of 1 and 5 respectively. Norm represents layer normalization operation to improve training stability, and GELU is the activation function used. Conv3 maps the features processed previously back to the original feature dimension. Finally, a residual mechanism is added to alleviate the problem of gradient disappearance.

[0077] Based on the content of the above system embodiment, as an alternative embodiment, in the information enhancement named entity recognition system for power operation and maintenance texts provided in the embodiments of the present invention, the constructing a probability framework and capturing dependencies between tags includes:

[0078] (13)

[0079] where P(Y|X) is the conditional probability; is the quantification symbol for the compatibility between adjacent tags; is the evaluation symbol for the appropriateness of assigning the tag at position i; Y represents a complete tag sequence; is the set of all possible tagging sequences corresponding to the observation sequence X; is the quantification symbol for the compatibility between the tag at position i and the tag at position i + 1 among all possible tag sequences in the denominator; is the evaluation symbol for the appropriateness of assigning a specific tag at position i among all possible tag sequences in the denominator.

[0080] Specifically, the conditional random field represents a probability framework designed to address the sequence tagging challenges in machine learning. This architecture is good at capturing dependencies between tags, which makes it particularly advantageous for tasks where traditional independent classification methods prove insufficient. Consider an input sequence X = [x1,..., xn] paired with its corresponding tag sequence Y = [y1,..., yn]. The CRF framework establishes a probability mapping between these sequences through equation (13). In equation (13), the denominator serves as a normalization term, summing over all possible tag sequences (YX) to ensure proper probability distribution properties. The optimization objective focuses on maximizing the conditional probability P(Y|X) during the training phase.

[0081] Based on the content of the above system embodiments, as an alternative embodiment, in the embodiment of the present invention, the information-enhanced named entity recognition system for power operation and maintenance texts, the visual interaction for named entity recognition of power operation and maintenance texts includes: inputting operation and maintenance work orders, automatically extracting corresponding information, batch processing multiple operation and maintenance work orders, and batch exporting the recognition results.

[0082] Specifically, this module also has a storage function, which can automatically save the sentences input by users and the named entity information recognized by the model. These stored data can not only be used for subsequent data analysis and mining to further optimize the model and improve the recognition accuracy, but also provide historical data query services for power operation and maintenance personnel. For example, operation and maintenance personnel can quickly find the named entity recognition results in past similar fault situations through keyword search, so as to provide a reference basis for the current fault handling and improve the efficiency of fault diagnosis and handling. At the same time, the stored data can also be used as part of the knowledge base and work together with other modules to provide rich data support for downstream applications such as constructing a power operation and maintenance knowledge graph.

[0083] The information-enhanced named entity recognition system and device for power operation and maintenance texts provided by the embodiments of the present invention can realize the extraction of key information for power operation and maintenance texts by integrating the dataset construction module, word embedding module, position-aware global attention mechanism module, fine-grained enhancement module, conditional random field module, visual interaction and storage module, improve the efficiency and intelligent level of power equipment operation and maintenance management, and help promote equipment management standardization, reduce operation and maintenance costs and optimize decision-making support.

[0084] Specifically, the information-enhanced named entity recognition system and device for power operation and maintenance texts provided by the embodiments of the present invention construct a large-scale standardized text dataset for the power operation and maintenance field, filling the gap in high-quality training data in this field. By annotating 7 types of core entities such as equipment names, voltage levels, and affiliated lines, it provides an important data basis for the intelligentization of power equipment operation and maintenance management. The construction of this dataset provides reliable training resources for subsequent intelligent applications such as operation and maintenance data analysis, fault diagnosis, and life prediction. The standardized processing of the dataset provides reusable benchmark data for other related research and applications in the industry, promoting the development of field technologies. The word embedding based on BERT-wwm-ext has strong context understanding ability and can accurately capture the semantic representations of words in different contexts. The rich semantic features provided by the pre-trained model provide a solid foundation for subsequent deep feature extraction, effectively improving the overall performance of the model. The dynamic characteristics of the word embedding enable the model to adaptively adjust the word representation according to the context, enhancing the flexibility of feature extraction. The improved position encoding method enhances the model's perception ability of sequence position information and improves the capture effect of long-distance dependency relationships. A globally shared learnable memory unit is designed to enable the model to obtain information from the entire dataset dimension, overcoming the limitation of traditional methods that only focus on a small number of samples. Through the decomposition design of attention scores, the effective fusion of content information and position information is achieved, improving the accuracy of feature extraction. It effectively avoids the problem that the abnormal feature values of a single token dominate the entire attention distribution, improving the robustness of the model. The dynamically adjustable normalization parameters enable the model to adaptively balance the attention distribution of different samples, optimizing the feature extraction effect. The application of the learnable double normalization strategy improves the training stability of the model and provides guarantee for high-quality feature learning. The parallel multi-scale convolutional network design realizes the accurate extraction of local detail features, improving the accuracy of entity recognition. The interactive feature fusion mechanism enhances the expression ability of features at different scales and optimizes the feature representation of the model. The application of residual connections effectively alleviates the gradient disappearance problem of deep networks and ensures the stability of model training. By establishing the dependency relationship between label sequences, the accuracy of entity boundary recognition is significantly improved. The application of the probability framework enables the model to learn the label transition rules, improving the rationality of sequence annotation. The final label prediction process is optimized, reducing the unreasonable label sequences in the prediction results. It supports batch processing and result export of single and multiple operation and maintenance work orders, improving the actual application efficiency. The automatic storage function realizes the continuous accumulation of recognition results and provides support for subsequent data analysis and model optimization. The fast retrieval function of historical data provides decision-making references for operation and maintenance personnel, improving the efficiency of fault handling. The stored structured data can be directly used to construct a power operation and maintenance knowledge graph, promoting the intelligent construction of the power system.

[0085] The specific application fields or related products of the present invention.

[0086] An information-enhanced named entity recognition system for power operation and maintenance texts proposed by the present invention can further construct a power equipment operation and maintenance management system, which can be transplanted to electronic device mobile terminals such as smartphones and tablets for use. The structure of this system is as Figure 5 shown.

[0087] The power equipment operation and maintenance management system consists of an employee registration and management module, an operation and maintenance work order information extraction module, and an operation and maintenance work order structured information management module.

[0088] The main function of the employee registration and management module is to add and manage the information of operation and maintenance employees. The employee information includes basic information such as employee number, name, gender, and phone number, supports fuzzy query according to a single piece of employee information, and supports modifying employee information and permissions. Figure 6 This is the function page display of this module.

[0089] The main function of the operation and maintenance work order information extraction module is to import and export the original equipment operation and maintenance work orders, perform named entity recognition on the operation and maintenance work orders, and convert unstructured operation and maintenance information into a structured form. This module also has batch management and batch structuring functions. Figure 7 This is the function page display of this module.

[0090] The operation and maintenance work order structured information management module is responsible for storing and querying the structured information after named entity recognition. The structured information is displayed according to the divided entity categories. Figure 8 This is the function page display of this module.

[0091] Evidence related to the technical effects obtained in the embodiments of the present invention.

[0092] A series of experiments were carried out on the power operation and maintenance data set constructed by the present invention for the named entity recognition model proposed by the present invention to verify the recognition performance of the model. In the experiment, three evaluation indicators, precision (P), recall (R), and F1 score, were used to verify the performance of the model. The calculation formula of precision (P) is as follows:

[0093]

[0094] Among them, TP represents the number of samples that the model correctly predicts as the positive class, and FP represents the number of samples that the model incorrectly predicts as the positive class. Precision (P) measures how many of the samples predicted as the positive class by the model are truly positive classes. The calculation formula of recall (R) is as follows:

[0095]

[0096] Among them, FN represents the number of samples that the model mispredicts as the negative class. The recall rate (R) measures how many of all the true positive class samples are correctly predicted as the positive class by the model. The formula for the F1 score is as follows:

[0097]

[0098] The F1 score is the harmonic mean of the precision and recall rates, aiming to balance these two metrics. It makes a trade-off between precision and recall and is a measurement standard that takes both into account.

[0099] 1. Ablation Experiment

[0100] To study the effectiveness of different modules in the named entity recognition model proposed in the present invention, an ablation experiment was conducted on the power operation and maintenance dataset constructed in the present invention. The experimental results are shown in Table 1.

[0101] Table 1 Ablation Experiment Results

[0102] Module P(%) R(%) F1(%) Baseline Model 76.8 73.9 75.3 ① + Position-Aware Global Attention Mechanism Module (without using global memory unit) 80.5 78.3 79.4 ② + Position-Aware Global Attention Mechanism Module 85.7 84.9 85.3 ③ + Fine-Grained Enhancement Module 85.4 83.1 84.2 ④ + Position-Aware Global Attention Mechanism Module + Fine-Grained Enhancement Module 88.3 88.9 88.6 ⑤ + BERT-wwm-ext + Position-Aware Global Attention Mechanism Module + Fine-Grained Enhancement Module 92.8 91.9 92.3

[0103] The baseline model in Table 1 is Transformer+CRF, and its F1 score is 75.3%. It can be seen from the experimental results in the table that after adding the position-aware global attention mechanism module to the baseline model, all metrics have a consistent improvement. Among them, the use of the global memory unit further improves the performance, reaching an F1 score of 85.3%, which is a 5.9% improvement compared to without the global memory unit. This proves the key role of global information modeling of the dataset scale in the named entity recognition task.

[0104] When adding the fine-grained enhancement module to the baseline model, the F1 score is 84.2%. Further, the combined use of the position-aware global attention mechanism module and the fine-grained enhancement module has a better effect, and its F1 score reaches 88.6%. The balanced improvement of the precision rate (88.3%) and the recall rate (88.9%) indicates that these two modules work together to enhance entity boundary detection and classification.

[0105] The complete model proposed in the present invention achieves the best performance, with an F1 score of 92.3%. Therefore, the addition of the pre-trained model BERT-wwm-ext is also crucial. Figure 9 Shows the training dynamics of the model in 50 iterations under the use of different modules.

[0106] As can be seen from the figure, the use of each module has been improved to varying degrees in terms of the three dimensions of the model convergence speed, stability, and final performance. In short, the superior prediction accuracy of the model proposed in the present invention can be attributed to its ability to accurately capture long-distance context dependencies and entity boundaries. In practical applications, the model can efficiently and accurately identify the professional terms in the maintenance work orders in the power field.

[0107] 2. Confusion Matrix Heatmap

[0108] To intuitively evaluate the performance of the model proposed in the present invention, a detailed comparison was made of the recognition results of this model and the baseline model for different entity categories. Figures 10(a) and 10(b) show the confusion matrix heatmaps of the two models, where the vertical axis represents the entity categories of the true annotations, the horizontal axis represents the entity categories predicted by the model, the color changes from light to dark indicating the change in the prediction ratio from low to high, and the value in each cell represents the corresponding prediction percentage. The diagonal cells represent the ratio values of the model prediction accuracy for each entity category, while the other cells represent the ratio of the samples predicted as one category but actually belonging to other entity categories.

[0109] From the comparison results of the confusion matrix heatmaps, it can be observed that the proposed model has significantly better recognition accuracy (the values on the main diagonal) for all entity categories than the baseline model. In addition, from the non-diagonal elements, it can be observed that the misrecognition rate of the improved model has been significantly reduced.

[0110] Specifically, the entity text lengths of "operation and maintenance content" vary greatly and the content is diverse, so this entity is relatively easy to be confused with other entity categories and is difficult to recognize. Through comparison, it can be seen that the proposed model has well improved this problem and greatly improved the recognition accuracy of the "operation and maintenance content" entity. In addition, two consecutive or textually similar entity categories are relatively easy to be confused. For example, "substation to which it belongs" and "line to which it belongs", "equipment name" and "damaged part", "voltage level" and "time". The proposed model has greatly alleviated this problem. These performance enhancements can be attributed to the model's precise feature extraction of the semantics of complex long texts and entity boundaries.

[0111] Comparative Analysis of Prediction Examples

[0112] To intuitively verify the entity recognition ability of the model proposed in the present invention in the text of power equipment maintenance work orders, especially its performance in dealing with domain-specific terms and complex entity structures. Table 2 gives three representative model prediction examples for comparative analysis, and evaluates the actual effect of the proposed model by comparing it with the baseline model.

[0113] Table 2

[0114] Example 1 Check the communication failure of the measurement and control device of the 110kV No. 1 main transformer of Tang. Correct Label: <![CDATA[[Drain] SUB [110 kV] VL [Main Transformer No. 1] EN [Monitoring and Control Device] DP [Check for Communication Failure] MS。 > Baseline Model: <![CDATA[[Drain] SUB [110 kV] VL [1# Main Transformer Monitoring and Control Device] EN [Communication [Failure Check]] MS .]]> Proposed Model: <![CDATA[[flow] SUB [110 kV] VL [Main transformer No. 1] EN [Monitoring and control device] DP [Check for communication failure] MS 。]]> Example 2 Defect handling for the damage of the closing stop contact of Phase C of the disconnector No. 04 of the 110kV Chang in the Yunzhang Second Circuit Line. Correct Label: <![CDATA[[Yunchang Second Circuit Line] LINE [110 kV] VL [Chang] SUB [04 Disconnector] EN Phase C [Closing Stop Contact] DP [Defect Handling for Damage] MS 。]]> Baseline Model: <![CDATA[[Yunchang Second Circuit Line] SUB [110 kV] VL [04 Disconnector at Chang] EN Phase C [Switching on] DP Stop [Treatment of Defect of Contact Damage] MS 。]]> Proposed Model: <![CDATA[[Yunchang Second Return Line] LINE [110 kV] VL [Chang] SUB [04 Disconnector] EN Phase C [Closing Stop Contact] DP [Defect Handling for Damage] MS 。]]> Example 3 Defect elimination task: Eliminate the damage of the energy storage motor of the operating mechanism of the 110kV bus coupler breaker No. 23 of Huashan Substation of QX23008. Correct Label: <![CDATA[Defect elimination task: Eliminate QX23008 [Huashan Substation] SUB [110kV] VL Bus coupler [Hua] SUB [23 Circuit breaker] EN [Operating mechanism energy storage motor] DP [Damage handling] MS ] 。]]> Baseline Model: <![CDATA[Defect elimination task: Eliminate QX

[23008] TIME [Huashan Substation] SUB [110 kV] VL [Bus coupler Hua] SUB [23 Circuit Breaker] EN Operating mechanism [Energy storage motor] DP [Damage handling] MS 。]]> Proposed Model: <![CDATA[Defect elimination task: Eliminate QX23008 [Huashan Substation] SUB [110kV] VL Bus coupler [Hua] SUB [23 Circuit breaker] EN [Operating mechanism energy storage motor] DP [Damage handling] MS 。]]>

[0115] The content in [] in Table 2 is an entity, and the subscript represents different entity categories. Among them, SUB represents the substation to which it belongs, VL represents the voltage level, EN represents the equipment name, DP represents the damaged part, MS represents the specific operation and maintenance content, LINE represents the line to which it belongs, and TIME represents the time. These texts contain different sentence lengths and writing styles. In Example 1, the baseline model incorrectly identified the damaged part as part of the equipment name, and the specific operation and maintenance content was not accurately recognized either. In Example 2, the baseline model failed to correctly identify the easily confused line to which it belongs and the substation to which it belongs, as well as the damaged part and the specific operation and maintenance content whose boundaries are not easily distinguishable. In Example 3, the baseline model incorrectly identified the irrelevant information in the same digital format as the time, and some other entity boundaries were not accurately recognized. However, the model proposed by the present invention well avoids the above errors and has a good ability to distinguish entity categories that are easily confused in terms of semantics and form. This indicates that the proposed model can accurately capture the location information, context information, and boundary information of entities and understand the dependency relationships between entities.

[0116] The system of the embodiment of the present invention is implemented relying on an electronic device. Therefore, it is necessary to introduce the relevant electronic device. For this purpose, an embodiment of the present invention provides an electronic device, as Figure 2 shown, the electronic device includes: at least one processor, a communication interface, at least one memory, and a communication bus. Among them, at least one processor, the communication interface, and at least one memory complete mutual communication through the communication bus. The at least one processor can call the logical instructions in the at least one memory to implement various systems provided in the system embodiments.

[0117] In addition, when the logical instructions in the above-mentioned at least one memory can be implemented in the form of a software functional unit and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that makes a contribution to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the systems described in various embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs, etc., all kinds of media that can store program codes.

[0118] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative work.

[0119] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to implement the methods or systems described in each embodiment or some parts of the embodiments.

[0120] The flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of systems, methods, and computer program products according to multiple embodiments of the present invention. Based on this understanding, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and sometimes in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0121] It should be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or apparatus comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or apparatus. Without further limitation, an element defined by the statement "comprising..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element. For any similar expressions such as "predetermined threshold" or "preset threshold", if no specific value is indicated, those of ordinary skill in the art can determine its specific value through simple experiments or corresponding debugging.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An information-enhanced named entity recognition system for power operation and maintenance texts, characterized in that, Including: A dataset construction module for correcting characters in power equipment operation and maintenance work orders, determining entity categories, and performing entity annotation; a word embedding module for converting words from discrete symbolic forms into continuous vector representations; a position-aware global attention mechanism module for performing position encoding and attention score calculation, and obtaining correlation information between different samples; a fine-grained enhancement module for capturing local information to more accurately identify entities in operation and maintenance work orders; a conditional random field module for constructing a probability framework and capturing dependencies between labels; a visualization interaction and storage module for performing visualization interaction of named entity recognition for power operation and maintenance texts.

2. The information-enhanced named entity recognition system for power operation and maintenance texts according to claim 1, characterized in that, The dataset construction module includes: a preprocessing module for removing duplicate content from the obtained data and eliminating content containing useless characters, missing information, and incorrect statements; an entity category determination module for determining equipment names, voltage levels, affiliated lines, affiliated substations, damaged parts, specific operation and maintenance content, and time points; an entity annotation module for annotating text data, adopting the BMES annotation system, where "B" represents the start of an entity, "M" represents the middle part of an entity, "E" represents the end of an entity, and "S" represents a single-character entity.

3. The information-enhanced named entity recognition system for power operation and maintenance texts according to claim 2, wherein The conversion of words from discrete symbolic forms into continuous vector representations includes: using BERT-wwm-ext to perform character embedding on the input sequence, enhancing word embedding for the Chinese corpus, consisting of 12 Transformer layers, with 768 hidden units and 12 attention heads in each layer.

4. The information-enhanced named entity recognition system for power operation and maintenance texts according to claim 3, wherein Performing position encoding and attention score calculation, and obtaining correlation information between different samples, includes: ； ； ； ； ； ； ； Among them, Q, K, and V are the query matrix, key matrix, and value matrix respectively; X is the input sequence; , , is the parameter matrix for mapping the input X to different spaces; is the relative position embedding vector between two tokens; t is the current position; j is the position to be attended to; T is the matrix transpose symbol; i is the frequency index of the position encoding, and the value range is [0, d k / 2]; is the dimension of the key vector; is the position-aware attention score between positions t and j; is the query vector of the token at position t; is the key vector of the token at position j; u is the first learnable bias vector, which adjusts the attention according to the inherent attributes of the token; v is the second learnable bias vector, which adjusts the attention according to the direction and distance between two tokens, further enhancing the model's ability to capture spatial and sequential relationships; is the global information; L-DNorm is the learnable double normalization strategy; , are two globally shared learnable memory units, where, helps calculate the similarity to generate the attention weights, updates the input features by combining with the attention weights, thereby affecting the final output; Attn is the position-aware global attention mechanism; softmax is the softmax function.

5. The information-enhanced named entity recognition system for power operation and maintenance texts according to claim 4, characterized in that, The learnable double normalization strategy includes: ； ； ； Among them, is the intermediate matrix element obtained by matrix multiplication, representing the initial association value of the i-th row and j-th column; is the value after column normalization; is a learnable parameter for adjusting the column normalization strength; exp is the exponential function; is the original association value of the n-th row and j-th column; n is the row and column dimension index; is the output value after final double normalization; is the value of the i-th row and n-th column in the intermediate matrix after column normalization; is a learnable parameter for adjusting the row normalization strength.

6. The information-enhanced named entity recognition system for power operation and maintenance texts according to claim 5, wherein The fine-grained enhancement module includes: ； ； ； Among them, is the result of fusing the non-linear features of the second convolutional layer dominated by the features of the first convolutional layer, capturing the local details focused by different convolutional kernels in the input; Norm is the layer normalization operation; is the first one-dimensional convolutional layer; is the second one-dimensional convolutional layer; is the symbol of Hadamard product; GELU is the activation function; is the result of fusing the non-linear features of the first convolutional layer dominated by the features of the second convolutional layer, mining potential fine-grained semantic differences from another perspective; Out is the final output of the fine-grained enhancement module; is the third one-dimensional convolutional layer.

7. The information-enhanced named entity recognition system for power operation and maintenance texts according to claim 6, characterized in that Constructing a probability framework and capturing dependencies between labels includes: ； Among them, P(Y|X) is the conditional probability; is the quantization symbol for the compatibility between adjacent labels; is the evaluation symbol for the appropriateness of assigning the label at position i; Y represents a complete label sequence; is the set of all possible labeled sequences corresponding to the observation sequence X; is the quantization symbol for the compatibility between the label at position i and the label at position i + 1 among all possible label sequences in the denominator; is the evaluation symbol for the appropriateness of assigning a specific label at position i among all possible label sequences in the denominator.

8. The information-enhanced named entity recognition system for power operation and maintenance texts according to claim 7, wherein Performing visualization interaction of named entity recognition for power operation and maintenance texts includes: inputting operation and maintenance work orders, automatically extracting corresponding information, batch-processing multiple operation and maintenance work orders, and batch-exporting recognition results.

9. An electronic device, characterized in that, Including: At least one processor, at least one memory, a communication interface, and a bus; wherein, The processor, memory, and communication interface complete communication with each other through the bus; The memory stores program instructions executable by the processor, and the processor invokes the program instructions to implement the system according to any one of claims 1 to 8.

10. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions cause the computer to implement the system according to any one of claims 1 to 8.

Images