Power equipment fault information extraction and intelligent diagnosis method based on BERT-BiLSTM-CRF model

Abstract

The invention discloses an electrical equipment fault information extraction and intelligent diagnosis method based on a BERT-BiLSTM-CRF model, and the method comprises the following steps: S1, collecting the operation data of electrical equipment, including fault logs, equipment states, operation records and other text information; s2, preprocessing the collected text data, removing noise, and performing word segmentation to prepare for subsequent processing; s3, performing context modeling on the preprocessed text by using a BERT model, and extracting semantic features; s4, inputting semantic information output by the BERT into a BiLSTM network, and capturing a long-distance dependency relationship in a text; s5, optimizing label distribution through a CRF layer, and accurately identifying key entities in the fault log; and S6, performing fault diagnosis and generating a processing suggestion in combination with the extracted information and the real-time data of the equipment. According to the method, the BERT, the BiLSTM and the CRF model are combined, the fault information of the power equipment is automatically extracted, the accuracy and the intelligent level of fault diagnosis are improved, and reliable decision support is provided for equipment maintenance.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing and information extraction, and specifically to a method for identifying named entities of power equipment based on the BERT-BiLSTM-CRF model, which is used to improve the accuracy and efficiency of text information processing in the power field. Background Art

[0002] Documents, reports, and equipment descriptions in the power system contain a large amount of professional terms and entity information, which are of crucial significance for power equipment management, maintenance, and fault diagnosis. However, currently, the processing of text information in the power industry mainly relies on manual access and analysis. This method not only consumes a large amount of human resources but also easily leads to understanding deviations and data omissions. In addition, with the continuous expansion of the scale of the power system and the sharp increase in the amount of data, traditional information extraction methods are difficult to meet the requirements of efficient and accurate information acquisition.

[0003] Currently, natural language processing technologies based on machine learning and deep learning have been gradually applied to the power field, with relatively good automation and processing capabilities. In particular, the BERT (Bidirectional Encoder Representations from Transformers) model has the ability of deep context understanding, and the structure combining BiLSTM (Bidirectional Long Short-Term Memory Network) and CRF (Conditional Random Field) is suitable for sequence labeling tasks, which can improve the accuracy of entity recognition. However, existing solutions may be affected by factors such as the complexity of text context, term diversity, and semantic ambiguity when facing complex power professional texts, resulting in insufficient accuracy of named entity recognition. Therefore, there is an urgent need for a more accurate and efficient method for identifying named entities of power equipment to meet the industry's needs. Summary of the Invention

[0004] The purpose of the present invention is to solve the problem that existing methods for identifying named entities of power equipment have insufficient accuracy in processing power equipment texts, especially when facing important entities such as equipment names, fault information, and operation instructions, and are unable to effectively extract or accurately identify them.

[0005] To solve the above problems, the present invention provides a method for identifying named entities of power equipment based on the BERT-BiLSTM-CRF model, including the following steps:

[0006] S1. Obtain text data containing power equipment information, and perform preprocessing operations such as data cleaning, noise removal, and word segmentation to prepare standardized text data for subsequent model input.

[0007] S2. Use the pre-trained BERT model to encode the context information of the input text. The BERT model can capture the deep semantic relationships between contexts and generate the word embedding representation of the text.

[0008] S3. Use the word embeddings generated by BERT as the input of the BiLSTM network. The BiLSTM model can capture the information of the context before and after in the text simultaneously, so as to better understand the grammar and semantics of words in different contexts.

[0009] S4. Based on the feature sequence output by BiLSTM, combine the conditional random field (CRF) for named entity recognition. The CRF layer enhances the accurate recognition of entity boundaries according to the sequence annotation rules, and finally labels the power equipment entities (such as equipment names, fault information, etc.) in the text.

[0010] S5. Post-process the entities output by the CRF layer to further optimize the accuracy of the entities, including operations such as removing duplicate entities and merging adjacent entities, to obtain the final named entities of power equipment.

[0011] S6. Apply the model to the actual power equipment text data for named entity recognition, output the recognized entities and their categories, and assist tasks such as power equipment management and fault diagnosis.

[0012] Preferably, the specific steps of S1 are as follows:

[0013] Step 1: Perform word segmentation and stop word removal on the text, and retain the keywords useful for power equipment information;

[0014] Step 2: Clean the text by methods such as regular expressions and keyword dictionaries, remove irrelevant noise data, and ensure the quality of the input data.

[0015] Preferably, the specific operation of the BERT model in S2 is as follows:

[0016] Step 1: Perform word segmentation on the input text, and use the Tokenizer tool of BERT to generate word embeddings;

[0017] Step 2: Use the BERT model for encoding to obtain the context representation of each word.

[0018] Preferably, the specific operation of BiLSTM in S3 is as follows:

[0019] Step 1: Input the context vectors generated by BERT into the BiLSTM model, and the model captures the information of the context before and after in the text through the bidirectional propagation mechanism;

[0020] Step 2: The feature sequence output by BiLSTM provides enhanced context information for each word.

[0021] Preferably, the specific steps of the CRF layer in S4 are as follows:

[0022] Step 1: Use CRF to label the sequence features output by BiLSTM, relying on the dependency relationship between the front and back tags to improve the accuracy of entity recognition;

[0023] Step 2: CRF determines whether each word belongs to a named entity and its category by learning the transition probability between tags.

[0024] Preferably, the post-processing steps in S5 are as follows:

[0025] Step 1: Screen the recognized entities and remove the entities that do not conform to the rules;

[0026] Step 2: Merge adjacent entities and adjust their boundaries to ensure the accuracy of the recognition results.

[0027] The beneficial effects of the present invention are as follows:

[0028] The present invention provides a method for identifying named entities of power equipment based on the BERT-BiLSTM-CRF model, which can make full use of the in-depth understanding of context information by the BERT model and combine the advantages of the BiLSTM and CRF models to improve the accuracy and robustness of the identification of named entities of power equipment; compared with the traditional rule-based method, the method of the present invention has stronger adaptability in dealing with complex texts and professional terms, can significantly improve the automation level of power equipment information extraction, reduce manual intervention, and improve the efficiency of power equipment management and maintenance. Description of the Drawings

[0029] Figure 1 is a flowchart of a method for identifying named entities of power equipment based on the BERT-BiLSTM-CRF model provided by an embodiment of the present invention;

[0030] Figure 2 is the semi-supervised learning training process in an embodiment of the present invention;

[0031] Figure 3 is the BERT-BiLSTM-CRF named entity recognition model framework in an embodiment of the present invention;

[0032] Figure 4 is the input structure of the Bert model in an embodiment of the present invention. Detailed Embodiments

[0033] In order to clearly and completely describe the technical solutions and technical effects of the present invention, the following is a detailed description through embodiments.

[0034] Embodiment 1

[0035] As Figure 1 shown, the present invention provides a method for identifying named entities of power equipment based on the BERT-BiLSTM-CRF model, which specifically includes the following steps:

[0036] S1. Obtain text data containing power equipment information and perform data preprocessing;

[0037] First, obtain the original text data containing power equipment information; this data can come from text resources such as power equipment maintenance records, operation manuals, inspection reports, etc.; since there may be irregular parts in the original data, it is necessary to perform preprocessing operations such as data cleaning, denoising, and word segmentation;

[0038] In this embodiment, the preprocessing method for the original text data includes the following steps;

[0039] Remove irrelevant information in the text, such as non-standard characters, duplicate information, etc., to ensure the cleanliness and effectiveness of the data;

[0040] Use a Chinese word segmentation tool or a deep learning-based word segmentation tool to cut the continuous character sequence into meaningful lexical units to prepare for the subsequent model input;

[0041] Standardize the vocabulary, for example, unify synonyms, standardize the names of power equipment, etc., to reduce interference between different vocabularies;

[0042] Through these preprocessing steps, finally obtain the normalized text data suitable for input into the model to prepare for the subsequent model training.

[0043] S2. Construct a named entity recognition label for power equipment;

[0044] In the preprocessed text data, mark the named entities related to power equipment (such as transformers, switches, cables, etc.); the marking of named entities can be carried out through manual marking or automated methods, and the specific steps are as follows;

[0045] According to domain knowledge or standard definitions, mark each vocabulary in the text, such as equipment name, part, defect type, defect cause, treatment measures, etc.;

[0046] In this embodiment, the BIO (Begin, Inside, Outside) marking scheme is adopted to perform fine-grained marking on the entities in the text to ensure that the boundaries of each entity are correctly delimited;

[0047] Construct a corresponding marking file for the training data set so that each text sample contains the actual entity and its corresponding label to ensure that the model can effectively learn;

[0048] Through the above steps, an annotated dataset containing power equipment named entities is constructed, providing an annotation basis for subsequent model training.

[0049] S3. BERT pre-trained model embedding;

[0050] Use the BERT (Bidirectional Encoder Representations from Transformers) model to extract text features; Figure 3 The framework of the BERT model in the embodiment of the present invention is shown. BERT can learn context information during the pre-training process and generate vectors representing the context semantics for each word;

[0051] As Figure 4 shown, the input structure of the BERT model includes [CLS], [SEP] tokens, and the vocabulary itself; each word in the input text is tokenized into lexical units and converted into a format acceptable to the model through the BERT Tokenizer;

[0052] BERT learns context information through a bidirectional Transformer architecture, and the generated word vectors will be used as inputs for downstream tasks (such as named entity recognition);

[0053] In this step, the BERT model is embedded as a pre-trained model into the entire named entity recognition process, which can provide rich context information to support subsequent sequence annotation tasks.

[0054] S4. Use the BiLSTM model for sequence modeling;

[0055] Based on the feature vectors generated by BERT, use a bidirectional long short-term memory network (BiLSTM) for sequence modeling, Figure 2 which shows the training process in semi-supervised learning, where BiLSTM is mainly used to capture long-distance dependencies in the text and improve the accuracy of entity recognition;

[0056] BiLSTM consists of two LSTM layers, one for modeling the text sequence from left to right and the other for modeling the text sequence from right to left; bidirectional modeling can comprehensively consider context information, especially in the named entity recognition of power equipment, which is crucial for capturing complex context relationships;

[0057] The input of BiLSTM is the word vectors generated by BERT, and the network learns and trains through these inputs to generate more semantic and context-rich feature representations;

[0058] BiLSTM can not only extract the dependencies between words, but also capture context information across a larger range, providing strong support for the subsequent CRF layer.

[0059] S5. The CRF layer performs entity label prediction;

[0060] The features output by BiLSTM will be passed to the conditional random field (CRF) layer for label prediction; the CRF layer takes into account the correlations between labels, optimizes the sequence prediction of named entities, and ensures that the entity recognition results conform to the actual semantics;

[0061] Figure 3 In the shown BERT-CRF model framework, the CRF layer can maximize the overall correctness of the label sequence according to the conditional probability; for example, if there are syntactic or semantic relationships between the front and back labels, the CRF can adjust the final label prediction according to these relationships;

[0062] The core of the CRF layer is to maximize the conditional probability of the given input sequence x and label sequence y,

[0063]

[0064] where f k is the feature function; ω k is the weight of the feature function; Z(x) is the normalization term.

[0065] S6. Output and evaluation of named entity recognition results;

[0066] Finally, the model outputs the recognition results and evaluates them; by comparing with the manually annotated ground truth data, evaluate the accuracy, precision, recall, F1 value and other metrics of the model; accuracy (Accuracy), precision (Precision), recall (Recall) and F1 value, etc. are commonly used evaluation metrics to measure the effect of named entity recognition;

[0067] Based on the feedback of experimental data, adjust the hyperparameters, training data volume or optimization algorithm of the model to further improve the recognition accuracy.

[0068] The above are only some embodiments of the present invention. For those of ordinary skill in the art, without departing from the inventive concept of the present invention, several modifications and improvements can still be made. Any modifications, equivalent replacements and improvements made within the inventive concept of the present invention all fall within the protection scope of the present invention.

Claims

1. A method for extracting and intelligently diagnosing fault information of power equipment based on the BERT-BiLSTM-CRF model, characterized in that: The following steps are involved: S1: Collect power equipment operation data, including fault logs, equipment status, operation records and other text information; S2: Preprocess the collected text data, remove noise and perform word segmentation to prepare for subsequent processing; S3: Use the BERT model to perform context modeling on the preprocessed text and extract semantic features; S4: Input the semantic information output by BERT into the BiLSTM network to capture the long-distance dependencies in the text; S5: Optimize label allocation through the CRF layer to accurately identify key entities in the fault log, such as faulty equipment, fault location, fault type, fault level, fault cause, and treatment measures; S6: Combine the extracted information with the real-time data of the equipment to perform intelligent fault diagnosis and generate processing suggestions.

2. According to claim 1, a method for extracting and intelligently diagnosing fault information of electric power equipment based on the BERT-BiLSTM-CRF model, characterized in that: The collection steps in S1 include: Step 1: Obtain real-time operation data through the power equipment monitoring system, including equipment status, sensor data, etc.; Step 2: Convert the equipment's fault logs and operation records into structured text data.

3. According to claim 1, a method for extracting and intelligently diagnosing fault information of electric power equipment based on the BERT-BiLSTM-CRF model, characterized in that: The preprocessing steps in S2 include: Step 1: De-noise the original text data to remove irrelevant information; Step 2: Use word segmentation algorithms to convert text data into processable vocabulary units and build a corpus.

4. According to claim 1, a method for extracting and intelligently diagnosing fault information of electric power equipment based on the BERT-BiLSTM-CRF model, characterized in that: The BERT model used in S3 further includes: Step 1: Input the preprocessed text data into the BERT model to embed contextual information; Step 2: Encode the text through a multi-layer Transformer structure to capture long-distance dependencies between words.

5. According to claim 1, a method for extracting and intelligently diagnosing fault information of electric power equipment based on BERT-BiLSTM-CRF model, characterized in that: The BiLSTM network in S4 further comprises: Step 1: Take the semantic features output by the BERT model as input and perform bidirectional LSTM encoding; Step 2: Capture the forward and reverse dependencies in the text through a bidirectional LSTM network.

6. According to claim 1, a method for extracting and intelligently diagnosing fault information of electric power equipment based on BERT-BiLSTM-CRF model, characterized in that: The CRF layer in S5 optimizes label assignment by the following steps: Step 1: Define the transfer matrix between labels and set the relationship between labels; Step 2: Use the conditional random field algorithm to accurately label the fault entities and eliminate label errors in the prediction.

7. According to claim 1, a method for extracting and intelligently diagnosing fault information of electric power equipment based on BERT-BiLSTM-CRF model, characterized in that: The intelligent fault diagnosis step in S6 includes: Step 1: Match the faulty equipment and parts identified by the model with the real-time sensor data of the equipment; Step 2: Generate an intelligent fault diagnosis report based on historical fault data and equipment status, and provide processing suggestions.

8. According to claim 1, a method for extracting and intelligently diagnosing fault information of electric power equipment based on BERT-BiLSTM-CRF model, characterized in that: The processing suggestion generation in S6 further includes: Step 1: Use failure modes and historical data to predict the possibility of equipment failure; Step 2: Based on equipment characteristics and failure prediction, make recommendations for preventive maintenance or emergency repairs.

Images