Entity identification method and device for mapping knowledge domain of metering electricity meter

Abstract

The invention is suitable for the technical field of metering ammeter fault diagnosis, and provides an entity identification method and device for a metering ammeter knowledge graph, and the method comprises the steps: obtaining the text data of a metering ammeter, and carrying out the preprocessing, and obtaining a marking unit of the text data; pre-training the marking units, generating a high-dimensional embedded vector of each marking unit by using an encoder, inputting the high-dimensional embedded vector into the MedBERT-GP hybrid model, and determining a prediction result of the entity; adjusting parameters of the model based on a loss function between a prediction result of the entity and a real label to obtain an optimized MedBERT-GP hybrid model; the optimized MedBERT-GP hybrid model is used for identifying and extracting entity information of text data of the metering electricity meter to form a mapping knowledge domain of the metering electricity meter. According to the invention, the key entity information in the ammeter fault data can be identified more accurately, the identification accuracy is improved, and the robustness of the model to various data noises and interferences is effectively improved.

Description

Technical Field

[0001] The present application belongs to the technical field of fault diagnosis of electric meter, and in particular, relates to an entity recognition method and device of a knowledge graph of electric meter. Background Art

[0002] Under normal operation of the power system, high-voltage meters can collect and record the transmission and distribution status of electricity in real time to ensure the safe and stable operation of the power system; if short circuit, overload or other faults occur, high-voltage meters will detect and provide relevant data support in real time, and the auxiliary equipment control system will issue a fault alarm or adjust the power flow to prevent the fault from expanding and ensure the safety of the power system and equipment. High-voltage meters include multiple operating status quantities such as current, voltage, power, frequency, and power factor, among which power and current are direct means of monitoring the operating status of meters.

[0003] In recent years, with the rapid development of machine learning and computer vision technology, mainstream models such as Convolutional Neural Network (CNN) and Support Vector Machine have been successfully applied to the field of meter status recognition. When faced with massive multi-type fault data, how to effectively integrate multi-type data, quickly and accurately extract knowledge, build knowledge graphs, apply knowledge graphs and interactively explore is still a problem to be solved.

[0004] At present, there are many technical difficulties in fault diagnosis in the field of metering electricity, especially in the process of accurately identifying and extracting key entity information from a large amount of electricity meter data. Traditional entity recognition methods often rely on rules and manual feature extraction, which cannot fully mine the deep semantic information in the data, resulting in low recognition accuracy, especially when faced with complex and changeable electricity meter fault data, recognition errors are more common. In addition, when faced with different data sources and formats, traditional methods have poor robustness and are easily affected by noise data, resulting in unstable recognition results and cannot meet the requirements of high reliability and stability in the industrial field. Summary of the invention

[0005] The embodiments of the present application provide an entity recognition method and device for a metering meter knowledge graph, which can more accurately identify key entity information in meter fault data, improve recognition accuracy, effectively improve the robustness of the model to various data noise and interference, and ensure stable operation under different data sources and environments.

[0006] This application is implemented through the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides an entity recognition method for a knowledge graph of an electric meter, comprising:

[0008] Get the text data of the meter;

[0009] Preprocessing the text data to obtain a tag unit of the text data;

[0010] Pre-train the tag units and use the encoder to generate a high-dimensional embedding vector for each tag unit;

[0011] The high-dimensional embedding vector of each token unit is fed into the MedBERT-GP hybrid model to determine the prediction result of the entity;

[0012] Based on the loss function between the predicted results of the entities and the true labels, the parameters of the MedBERT-GP hybrid model are adjusted to obtain the optimized MedBERT-GP hybrid model; the optimized MedBERT-GP hybrid model is used to identify and extract the entity information of the text data of the metering meter to form a knowledge graph of the metering meter.

[0013] In a possible implementation of the first aspect, pre-training the marking unit and using an encoder to generate a high-dimensional embedding vector for each marking unit includes:

[0014] Perform label unit embedding on each label unit to obtain a unit embedding vector;

[0015] Perform segment embedding on each tag unit to obtain a segment embedding vector;

[0016] Perform position embedding on each tag unit to obtain a position embedding vector;

[0017] Add the unit embedding vector, segment embedding vector, and position embedding vector to form the word vector representation of each token unit;

[0018] The encoder is used to process the word vector of each token unit to generate a high-dimensional embedding vector for each token unit.

[0019] In a possible implementation of the first aspect, the prediction result of the entity includes a boundary of the entity and a category of the entity;

[0020] The MedBERT-GP hybrid model is used to linearly transform the hidden representation of the high-dimensional embedding vector to generate a query vector and a key vector; and based on the query vector and the key vector, the dot product is used to calculate the span score of each entity; and the rotational position encoding is introduced to update the span score of each entity and determine the boundary of each entity; and the updated span scores of each entity are aggregated to generate a probability matrix representing the category of each entity and determine the category of each entity.

[0021] In a possible implementation manner of the first aspect, the span score is expressed as:

[0022] score(i,j)=q i ·k j

[0023] Among them, score(i,j) represents the score of the span from position i to position j being predicted as an entity category; q i is the query vector at position i; k j is the key vector for position j.

[0024] In a possible implementation of the first aspect, the MedBERT-GP hybrid model uses the MedBERT-wwm model as an encoder and the GlobalPointer model as a decoder; the MedBERT-wwm model is used to understand the input circuit breaker text and generate a hidden vector containing context information; the GlobalPointer model is used to convert the hidden vector containing context information output by the encoder into a probability matrix representing the entity span based on the span prediction strategy, and predict the entity span according to the global position information of the head and tail of the entity.

[0025] In a possible implementation of the first aspect, the loss function L is expressed as:

[0026]

[0027] Where N is the sum of all elements in the probability matrix; i and j are the locations of the entities; k is the type of the entity; is the probability prediction matrix from position i to position j of the k-th entity; is the actual label matrix from position i to position j of the k-th entity.

[0028] In a possible implementation of the first aspect, using an encoder to process a word vector of each tag unit to generate a high-dimensional embedding vector for each tag unit includes:

[0029] A bidirectional long short-term memory network is used to extract the deep semantic information of the context of the word vector of the tag unit. At the same time, the Transformer encoder is used to capture the global dependency between the word vectors of the tag unit through the self-attention mechanism to generate a high-dimensional embedding vector for each tag unit.

[0030] In a possible implementation of the first aspect, extracting deep semantic information of the context of the word vector of the tag unit using a bidirectional long short-term memory network includes:

[0031] The word vector of the tag unit is input into the forward LSTM of the bidirectional long short-term memory network, and the word vector of the input tag unit is processed sequentially to capture the sequential dependency information of the previous context;

[0032] The word vector of the tag unit is input into the backward LSTM of the bidirectional long short-term memory network, and the word vector of the input tag unit is processed in reverse order to capture the reverse dependency information of the previous context;

[0033] The sequential dependency information of the previous context and the reverse dependency information of the previous context are comprehensively represented to extract the deep semantic information of the context of the word vector of the tag unit.

[0034] In a possible implementation of the first aspect, preprocessing the text data to obtain a tag unit of the text data includes:

[0035] Perform word segmentation and stop word removal on the text data to obtain preprocessed text data;

[0036] The preprocessed text data is serialized to obtain the tag units of the text data.

[0037] In a second aspect, an embodiment of the present application provides an entity recognition device for a metering meter knowledge graph, and applies the entity recognition method for a metering meter knowledge graph as in the first aspect, including:

[0038] A data acquisition module, used to acquire text data of the meter;

[0039] A preprocessing module, used for preprocessing text data to obtain a marked unit of the text data;

[0040] A pre-training module is used to pre-train the embedding of each labeled unit and use the encoder to generate a high-dimensional embedding vector for each labeled unit;

[0041] The prediction module is used to input the high-dimensional embedding vector of each token unit into the MedBERT-GP hybrid model and determine the prediction result of the entity through linear transformation and dot product calculation;

[0042] The optimization module is used to adjust the parameters of the MedBERT-GP hybrid model based on the loss function between the predicted results of the entities and the true labels to obtain the optimized MedBERT-GP hybrid model; the optimized MedBERT-GP hybrid model is used to identify and extract the entity information of the text data of the metering meter to form a knowledge graph of the metering meter.

[0043] Compared with the related art, the embodiments of the present application have the following beneficial effects:

[0044] The entity recognition method and device of the knowledge graph of the metering meter in the embodiment of the present application automatically learns the semantic features in the field through deep learning technology in the form of pre-training, reduces manual intervention, improves efficiency, and generates a high-dimensional embedding vector representation of the tag unit (Token) through pre-training embedding, thereby improving the semantic understanding ability of the model; at the same time, combined with the improved MedBERT-GP hybrid model and the knowledge graph in the field of metering meters, the deep semantic features of the text are extracted, the key entity information in the meter fault data is accurately identified, and the efficient identification of special terms and relationships in the field is realized, the field adaptation ability of the model is enhanced, the limitations of a single method are compensated, the ability to parse complex query intentions is enhanced, the accuracy and robustness of named entity recognition in the field of metering meters are significantly improved, and a high-quality data foundation is provided for the construction and application of knowledge graphs.

[0045] The beneficial effects of the above-mentioned second aspect embodiment refer to the beneficial effects of the first aspect embodiment, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0047] Figure 1 This is an application scenario diagram of the entity recognition method and device of the metering meter knowledge graph provided in one embodiment of the present application;

[0048] Figure 2 This is the overall technical route of the MedBERT-GP hybrid model provided in one embodiment of the present application;

[0049] Figure 3 It is a flow chart of an entity recognition method of a knowledge graph of an electric meter provided in an embodiment of the present application;

[0050] Figure 4 This is a schematic diagram of word embedding provided by an embodiment of the present application;

[0051] Figure 5 This is a schematic diagram of scaling dot product attention score calculation provided by an embodiment of the present application;

[0052] Figure 6 It is a structural diagram of the attention mechanism of the Transformer model provided in one embodiment of the present application;

[0053] Figure 7 is a schematic diagram of the structure of a Transformer encoder provided in an embodiment of the present application;

[0054] Figure 8 It is a structural diagram of a GlobalPointer decoder provided in an embodiment of the present application;

[0055] Fig. 9 is a schematic diagram of the structure of the MedBERT-GP hybrid model provided in one embodiment of the present application;

[0056] Fig.10 It is a schematic diagram of the process of BiLSTM provided in one embodiment of the present application;

[0057] Fig.11 It is a structural diagram of an entity recognition device for a metering meter knowledge graph provided in one embodiment of the present application. DETAILED DESCRIPTION

[0058] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0059] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0060] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0061] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0062] References to "an embodiment", "one embodiment" or "some embodiments" described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0063] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0064] Related professional terms:

[0065] Since named entity recognition (NER) was proposed at the Sixth Information Understanding Workshop (MUC-6) in 1996, it has gone through three stages of development: based on dictionaries and rules, based on statistical learning, and based on deep learning. Early methods relied on manually constructed professional dictionaries (such as diseases, symptoms, drugs, etc.) and performed entity recognition through matching algorithms. Representative systems include IBM's MedKAT and Mayo Clinic's cTAKES. This type of method has the advantage of high accuracy, but the construction and maintenance costs are high, the quality and coverage of the dictionary are strict, and it is difficult to handle new words and complex contexts. Subsequently, statistical learning-based methods (such as conditional random fields, CRFs) were gradually introduced to capture contextual information through sequence annotation. The double-layer annotation model (DLAM) proposed by Gong Lejun et al. combines the domain dictionary with the CRF algorithm, taking into account the high accuracy of the dictionary method and the automation capability of the statistical model. However, the performance of the statistical method depends on high-quality corpus and has limited processing capabilities for complex language structures. In recent years, the rapid development of deep learning technology has made it the mainstream method of NER, using technologies such as convolutional neural network (CNN), recurrent neural network (RNN), long short-term memory network (LSTM) and Transformer-based pre-trained language models (such as BERT). Deep learning methods have powerful feature extraction and semantic modeling capabilities, which significantly improve the accuracy and generalization of entity recognition and become a hot topic in current research.

[0066] Since the 1990s, statistical learning methods have been widely used in named entity recognition (NER) tasks. Researchers have collected and manually annotated a large amount of medical text data to provide training samples for machine learning algorithms. These algorithms predict entity information in other texts by learning the relationship between text and entity labels. Common methods include hidden Markov models (HMM), conditional random fields (CRF), support vector machines (SVM), etc. Compared with dictionary- and rule-based methods, statistical learning methods reduce the cost of building dictionaries and rules, and improve the level of automation, generalization, and portability. In typical studies such as the electronic medical record information extraction task of I2B2 and the practice of the CCKS2018 task, the recognition effect was significantly improved by introducing distributed semantics, word vector generation, and model improvement (such as BioTrHMM for transfer learning). However, these methods rely on manual feature selection and have high requirements on the quality and quantity of labeled data; they are insufficient in complex semantic processing, and simple models are difficult to distinguish complex semantic categories, with limited generalization ability, and difficult to effectively migrate to other fields.

[0067] At present, the application of deep learning in named entity recognition (NER) has become mainstream, and the complex feature representation of text is learned through multi-layer neural networks. Traditional models such as convolutional neural networks (CNN) and recurrent neural networks (RNN / LSTM / GRU) are widely used to model local and long-distance dependencies of text sequences, especially the combination of BiLSTM and conditional random fields (CRF) The model performs well. In recent years, pre-trained language models based on Transformer structures (such as BERT, BioBERT, ALBERT, etc.) use self-attention mechanisms to model contextual relationships and long-distance dependencies, significantly improving training efficiency and model accuracy. Among them, BioBERT is optimized for the biomedical field and can more accurately identify medical information. Innovative methods include a joint model that combines convolutional networks, self-attention mechanisms, and CRFs, as well as Global Pointer and multi-feature embedding methods that introduce global boundary information to solve boundary ambiguity and nested entity problems. In addition, converting nested entity recognition into boundary and head-tail joint training further improves the effect of complex tasks. These advances have enabled NER to show higher accuracy and generalization capabilities when dealing with complex semantics, domain texts, and nested structures.

[0068] However, while the existing technology is developing, it still has some shortcomings. For example, feature selection relies on manual work. Traditional statistical learning methods require experts to manually design features based on experience, which leads to low efficiency and difficulty in capturing complex semantic relationships. For example, the ability to process complex semantics is insufficient. Linear models and shallow models are less effective and difficult to accurately identify when faced with complex semantics, nested entities or long-distance dependencies. For example, the cost of corpus annotation is high, and a large amount of high-quality annotated data is required as training samples. The high annotation cost and data scarcity limit the algorithm's ability to expand. For example, domain migration is difficult. When existing models migrate between domains, they show insufficient generalization capabilities and need to be redesigned or adjusted for different domains. For example, the boundaries of entities are fuzzy. When dealing with problems such as nested entities and unclear boundaries, existing technologies are difficult to ensure accurate identification, which can easily lead to missed or mislabeled labels.

[0069] Therefore, in response to the above shortcomings, this application proposes an entity recognition method for the knowledge graph of metering meters, which automatically learns the semantic features in the field through deep learning technology (such as pre-training models), reduces manual intervention, and improves efficiency. At the same time, it combines BiLSTM, Transformer and MedBERT-GP models, based on the knowledge graph in the field of metering meters, and embeds the model with domain knowledge to achieve efficient recognition of special terms and relationships in the field, enhance the model's domain adaptation ability, make up for the limitations of a single method, and enhance the ability to parse complex query intentions. By effectively narrowing the search scope, significantly improving the search accuracy and response speed, adapting to more fields and language scenarios, and improving the versatility of the retrieval system. This method aims to significantly improve the accuracy and robustness of named entity recognition in the field of metering meters, and provide a high-quality data foundation for the construction and application of knowledge graphs.

[0070] Reference Figure 1 , which is an application scenario diagram of the entity recognition method and device of the knowledge graph of the metering meter of the present invention. The prediction method and device run in the server 102. The server 102 obtains the relevant data of the entity recognition of the knowledge graph of the metering meter from the database 101, and the data covers a large amount of text data of the metering meter.

[0071] The data in the database 101 may be text data captured from the meter or manually uploaded text data, and uploaded to the cloud server 102. For example, the text data of the meter is manually or automatically uploaded at the terminal 103, and the model at the cloud 102 can identify and extract the entity information of the text data of the meter, form a knowledge graph of the meter, and return the knowledge graph of the meter to the terminal 103.

[0072] The model of the cloud server 102 is the entity recognition model of the knowledge graph of the metering meter proposed in the present invention. To address the data set noise and span hard boundary problems existing in related technologies, the present invention proposes a multi-strategy boundary smoothing regularization technology, and proposes an improved MedBERT-GP model, namely MedBERT-GP (MBS). In addition, to address the sample imbalance problem, the present invention proposes multiple sample imbalance loss functions, and proposes another improved MedBERT-GP model, namely MedBERTGP (MSIL), which mainly uses the fusion model of MedBERT-wwm and GlobalPointer, referred to as the MedBERT-GP hybrid model. The overall technical route of the hybrid model is as follows: Figure 2 shown. Figure 3 is a flow chart of an entity recognition method for a metering meter knowledge graph provided in an embodiment of the present application, with reference to Figure 3 , the entity recognition method of the metering meter knowledge graph includes:

[0073] Step 201, obtaining text data of the electric meter.

[0074] For example, the text data of the metering meter is generally the operation log data of the metering meter. Text data includes multiple types of data such as fault text data, procedure data and fixed value list, which provide valuable resources for the intelligent operation and maintenance of the metering meter. The text of power equipment is not only highly professional and semantically complex, but also contains a large number of nested entities and long entities. However, the existing model has weak medical semantic understanding ability and insufficient long entity recognition ability. The metering meter-related named entities in these text data are crucial for building a knowledge graph of metering meters and assisting maintenance personnel in inspections and troubleshooting. The MedBERT-GP hybrid model is studied and improved on this basis.

[0075] In addition, the text data of metering meters related to patents can also be collected, such as patent specifications, abstracts, technical background, etc., and the knowledge system of metering meters can be further enriched through the text data of metering meters related to patents. The text data of metering meters related to patents contains many unique technical innovations, design ideas, and potential application scenarios. For example, some patent specifications may elaborate on the implementation of special functions of a new type of metering meter, which helps to deeply understand the working principle of metering meters under different technical frameworks. At the same time, combining patent-related text data with operation log data can construct more comprehensive metering meter text data, and more potential relationships and laws about metering meters can be mined. For example, by analyzing the relationship between technical improvements in patents and the frequency of faults, it may be possible to predict which new technical applications may bring new fault types, or conversely, which technical improvements can effectively reduce the probability of specific faults. This is of great significance for formulating maintenance strategies in advance and optimizing the operating performance of metering meters. Then, the preprocessed text is subjected to tasks such as named entity recognition and relationship extraction. By accurately identifying various named entities in the metering meter text data, such as different component names, technical parameters, fault types, etc., and extracting the relationships between them, such as the association between components and faults, the impact of technical parameters on performance, etc., it is possible to construct a metering meter knowledge graph more accurately in the future.

[0076] Step 202: preprocess the text data to obtain marked units of the text data.

[0077] For example, the comprehensive data set is deeply processed using natural language processing (NLP) technology, such as preprocessing the text, including noise removal, lexical analysis, etc., converting the meter text data into a sequence form, and splitting the sentences into tokens. A token usually refers to a basic unit in a text, including a word, character, or other symbol.

[0078] Exemplarily, step 202 includes: performing word segmentation and stop word removal operations on the text data to obtain preprocessed text data; and performing serialization processing on the preprocessed text data to obtain tag units of the text data.

[0079] Converting the meter text data into sequence form and splitting the sentences into tokens is beneficial for subsequent data analysis and model building. After converting the text data into tokens, we can further perform statistical analysis on these tokens, such as calculating the frequency of occurrence of each token and analyzing the co-occurrence relationship between tokens. These statistical information can help us better understand the structure and characteristics of meter text data and provide a basis for subsequent model building.

[0080] At the same time, the form of the tag unit is also easier to be accepted by various natural language processing models. For example, when building a neural network model, the tag unit can be used as the neuron of the input layer, and the model can learn the semantic relationship between the tag units by adjusting the weights and thresholds of the neural network. In this way, when new metering meter text data is input, the model can accurately analyze and predict based on the previously learned knowledge.

[0081] In addition, converting the meter text data into tag units can also facilitate data storage and transmission. Compared with the original text data, the tag unit is more compact and standardized, occupies less storage space, and has a faster transmission speed. This is particularly important for processing large-scale meter text data, which can improve the efficiency of data processing and reduce the cost of data processing.

[0082] Step 203, pre-training the tag unit, using an encoder to generate a high-dimensional embedding vector for each tag unit.

[0083] For example, the Transformer model can be used to extract deep semantic features and output a high-dimensional embedding vector for each token. The Transformer encoder uses a multi-layer Transformer structure to extract deep semantic information of the context through a self-attention mechanism and a feedforward neural network.

[0084] The significance of the above pre-training is to provide a good initial state for subsequent training using the MedBERT-GP hybrid model. Pre-training enables the model to learn common language features and patterns on large-scale data, so that the high-dimensional embedding vector of the labeled unit contains rich semantic information. This helps the model converge to a better solution faster when using the MedBERT-GP hybrid model, and can improve the generalization ability of the model. For example, the semantic information in the high-dimensional embedding vector obtained by pre-training can be further utilized and refined in the MedBERT-GP hybrid model to better adapt to specific metering meter tasks. At the same time, pre-training also helps to reduce dependence on large-scale labeled data, because the pre-trained model has learned some basic semantic and structural information from a large amount of unlabeled data. In the subsequent training of specific tasks, this information can be used to optimize on relatively less labeled data to improve the performance and efficiency of the model.

[0085] In step 204, the high-dimensional embedding vector of each tag unit is input into the MedBERT-GP hybrid model to determine the prediction result of the entity.

[0086] The prediction result of the entity includes the boundary of the entity and the category of the entity.

[0087] Exemplarily, the MedBERT-GP hybrid model is used to linearly transform the hidden representation of the high-dimensional embedding vector to generate a query vector and a key vector; and based on the query vector and the key vector, the span score of each entity is calculated using the dot product; and the rotational position encoding is introduced to update the span score of each entity and determine the boundary of each entity; and the updated span scores of each entity are aggregated to generate a probability matrix representing the category of each entity to determine the category of each entity.

[0088] The hidden representation of the high-dimensional embedding vector is the output of the encoder in step 203.

[0089] Step 205: Based on the loss function between the predicted result of the entity and the true label, the parameters of the MedBERT-GP hybrid model are adjusted to obtain an optimized MedBERT-GP hybrid model.

[0090] Among them, the optimized MedBERT-GP hybrid model is used to identify and extract entity information of text data of metering meters to form a knowledge graph of metering meters.

[0091] This knowledge graph will become a powerful assistant for maintenance personnel to conduct inspections and troubleshoot. When encountering a fault, maintenance personnel can quickly locate the components that may have problems and their related technical parameters by querying the knowledge graph, and obtain solutions or preventive measures in related patent technologies. This not only improves the efficiency of troubleshooting, but also provides stronger support for the intelligent operation and maintenance of metering meters, and promotes the overall stable and efficient operation of the power system.

[0092] This embodiment uses deep learning technology in the form of pre-training to automatically learn semantic features in the field, reduce manual intervention, and improve efficiency, and generates high-dimensional embedding vector representations of tag units (Tokens) through pre-training embedding, thereby improving the semantic understanding ability of the model; at the same time, combined with the improved MedBERT-GP hybrid model and the knowledge graph in the field of metering meters, the deep semantic features of the text are extracted, and the key entity information in the meter fault data is accurately identified, and efficient identification of special terms and relationships in the field is achieved, the field adaptation ability of the model is enhanced, the limitations of a single method are compensated, and the ability to parse complex query intentions is enhanced, which significantly improves the accuracy and robustness of named entity recognition in the field of metering meters, and provides a high-quality data foundation for the construction and application of knowledge graphs.

[0093] In one embodiment, in order to better understand the MedBERT-GP hybrid model, the improvement strategy of the MedBERT-wwm model is introduced. BERT (Bidirectional Encoder Representations from Transformers, based on Transformer bidirectional encoder representation) is first pre-trained on a large-scale corpus in a general field to learn the general representation of language. Then it is fine-tuned on the target dataset to adapt to specific NLP tasks. The pre-training process of BERT has two main goals, including the Masked Language Model (MLM) task and the Next Sentence Prediction (NSP) task. In the MLM task, the model randomly selects some words in the input text sequence and masks them into special [M] tags. The specific masking strategy is as follows: 15% of the words are randomly selected in each sequence. For each selected word, there is an 80% probability of replacing the word with the [M] tag, a 10% probability of replacing the word with a random word in the vocabulary, and the remaining 10% probability of keeping the word unchanged. Then during the pre-training process, the model predicts the original words of these [M] tags. Through the MLM task, the model can capture bidirectional context information. Through the NSP task, the model can understand the logical relationship between sentences. These two tasks together provide the BERT model with powerful language understanding capabilities, enabling it to effectively adapt to various NLP tasks in the subsequent fine-tuning stage.

[0094] Compared with BERT, MedBERT-wwm has made improvements in corpus, pre-training tasks, etc. In view of the characteristics of the Chinese language, the model uses the Whole Word Masking (WWM) strategy on the MLM task, and uses the Sentence-Order Prediction (SOP) task instead of the NSP task. Whole word masking means that in the MLM task, instead of masking individual characters in the sentence, the entire vocabulary is masked. For Chinese text, whole word masking can better preserve the integrity and semantic information of the vocabulary. In the SOP task, the model is given a pair of sentences as input, which may be arranged in the correct order or reversed. The task of the model is to predict whether the two sentences maintain the correct order. Compared with the NSP task, SOP pays more attention to the understanding of the logical relationship and coherence between sentences, rather than just judging whether the sentences are adjacent. Through the two improved pre-training tasks of WWM and SOP, MedBERT-wwm can more effectively capture the characteristics of Chinese text and provide more refined and accurate language model support for NLP tasks related to metering.

[0095] After completing steps 201 and 202, the pre-training of the MedBERT-GP hybrid model for the tag unit is introduced, mainly for the input of the MedBERT-wwm model, and a high-dimensional embedding vector for each tag unit can be generated by using a multi-layer Transformer encoder to extract the deep semantic information of the text. Then step 203 includes:

[0096] Step 2031, perform labeling unit embedding on each labeling unit to obtain a unit embedding vector.

[0097] For example, each token (entity) is mapped to a high-dimensional vector, which is obtained by looking up from the pre-trained embedding matrix. These word embeddings (Token Embedding) are learned through a large-scale circuit breaker corpus and contain rich circuit breaker semantic information, enabling the model to deeply understand the deep semantics of the text, such as Figure 4 shown.

[0098] Step 2032: perform segment embedding on each tag unit to obtain a segment embedding vector.

[0099] For example, segment embedding is used to distinguish different sentences in the input. Each sentence corresponds to an embedding vector. For NER (named entity recognition) tasks, since each input usually contains only a single sentence, the role of segment embedding is simplified.

[0100] Step 2033, position embedding is performed on each tag unit to obtain a position embedding vector.

[0101] For example, position embeddings are used to record the position of each token in the sequence, ensuring that the Transformer encoder can understand the sequence information. These embeddings are generated by the position index of each token, providing a unique position representation.

[0102] The main component of the Transformer encoder is the Transformer model, and the core of the Transformer model is the self-attention mechanism. For each word in the input sequence, the Transformer model calculates the relevance weights of the word with other words, and then generates an output sequence based on these weights. Specifically, for each element of the input sequence, the Transformer model multiplies the embedding vector of the word with three different weight matrices to obtain the query vector Q, the key vector K, and the value vector V, as shown in formula (1):

[0103] Q=XW Q K=XW K V=XW V (1)

[0104] Among them, X represents the embedding vector of the input element, W Q , W K and W V is a learnable weight matrix. After obtaining the query vector, key vector, and value vector respectively, the Transformer model calculates Figure 5 The scaled dot product attention score shown. The Transformer model calculates the dot product of the query vector and the corresponding key vector to get the similarity between them, which is the attention score. These scores are then scaled and normalized by the Sofmax function.

[0105] There are no parameters to learn in the above process of calculating the scaled dot product attention. Therefore, the Transformer model uses a multi-head attention mechanism similar to the multi-channel in CNN, such as Figure 6 As shown in Figure 2, the Transformer model uses different attention heads to match different features in the input.

[0106] The calculation process of multi-head attention MultHead(Q,K,V) is shown as follows:

[0107]

[0108] Where h is the number of heads, W i K and W i V is the weight matrix of the ith head, W0 is the output weight matrix, which is used for linear transformation after vector concatenation. After that, the feedforward neural network performs two linear transformations to enhance the representation ability of the model. The first linear transformation maps the input to a higher-dimensional space, and then it is nonlinearly transformed through the ReLU activation function. Then, the second linear transformation FFN(x) maps the representation back to the original dimension, as shown in formula (3).

[0109] FFN(x)=max(0,xW1+b1)W2+b2 (3)

[0110] Step 2034, add the unit embedding vector, the segment embedding vector and the position embedding vector to form a word vector representation of each token unit.

[0111] Exemplarily, to generate the embedding representation, the three embeddings are added together to form the word embedding representation of each token, which is then input into the MedBERT-wwm encoder for processing.

[0112] Step 2035, use the encoder to process the word vector of each tag unit to generate a high-dimensional embedding vector for each tag unit.

[0113] MedBERT-wwm integrates token embedding, segment embedding, and position embedding, combined with a multi-layer Transformer structure. Figure 7 Trm stands for Transformer encoder. The powerful features of Transformer encoder can efficiently capture the semantic features and complex dependencies in text data, and provide powerful semantic representation capabilities for tasks such as named entity recognition. The self-attention mechanism is used to extract the global association between tokens in the sequence. The output hidden state of each layer is used as the input of the next layer by layer processing, and deeper semantic information is gradually extracted. After multiple layers of encoding, the final output generates a high-dimensional vector representation of the sentence, which contains the deep semantic features of the text.

[0114] For example, after obtaining the encoder output, the hidden representation of the high-dimensional embedding vector is sent to GlobalPointer for decoding. The GlobalPointer decoder is as follows: Figure 8 As shown in Figure 1, GlobalPointer implements the entity recognition task by calculating multiple upper triangular matrices. The elements in each matrix represent the relative score (logits) of a specific span being recognized as a certain type of entity. The MedBERT-GP hybrid model formed by the fusion of MedBERT-wwm and GlobalPointer is shown in Figure 1. Fig. 9 Step 204 includes:

[0115] Step 2041, linearly transform the hidden representation of the high-dimensional embedding vector to generate a query vector and a key vector.

[0116] For each hidden representation h i , vector q is generated by two linear transformations i and k j .

[0117] q i =W q h i (4)

[0118] k j =W k h j (5)

[0119] Among them, W q and W k is a learnable parameter matrix with dimension d×d. Calculate q i and k j The dot product q i ·k j , as the basis for score prediction of the current span.

[0120] Step 2042, based on the query vector and the key vector, the span score of each entity is calculated using the dot product.

[0121] The span score is expressed as:

[0122] score(i,j)=q i ·k j (6)

[0123] Among them, score(i,j) represents the score of the span from position i to position j being predicted as an entity category; q i is the query vector at position i; k j is the key vector for position j.

[0124] Step 2043, introduces rotational position encoding, updates the span score of each entity, and determines the boundary of each entity.

[0125] In order to make up for the lack of sequence head and tail position information in linear transformation, GlobalPointer introduces RoPE (Rotational Position Encoding). PoPE can be regarded as a transformation matrix P. After adding RoPE, P(q i ,k j )score′(i,j), thereby better capturing the location information of long entities.

[0126] For example, the updated span score can be expressed as:

[0127] score′(i,j)=q i ′·k j ′=q i k j +RoPE(i,j) (7)

[0128] PoPE is introduced to enhance the model's ability to handle long entities, especially in long entities and complex contexts, significantly improving the accuracy of boundary prediction.

[0129] Step 2044 , aggregate the updated span scores of each entity, generate a probability matrix representing the category of each entity, and determine the category of each entity.

[0130] Aggregate the dot product results q′ at all positions i k′ j , you can generate the logits matrix of the k-th entity, which represents the predicted scores of all spans, that is, each matrix element It represents the probability that span (i, j) belongs to the k-th entity.

[0131] During the training phase, the values ​​of the logits matrix are normalized by the Sigmoid function to obtain the probability prediction matrix of the k-th entity. According to the probability matrix Extract entity categories and their start and end positions to complete entity recognition tasks in text data.

[0132] In step 205, the MedBERT-GP hybrid model uses the MedBERT-wwm model as an encoder and the GlobalPointer model as a decoder; the MedBERT-wwm model is used to understand the input circuit breaker text and generate a hidden vector containing context information; the GlobalPointer model is used to convert the hidden vector containing context information output by the encoder into a probability matrix representing the entity span based on the span prediction strategy, and predict the entity span according to the global position information of the head and tail of the entity.

[0133] Exemplarily, the loss function L is expressed as:

[0134]

[0135] Where N is the sum of all elements in the probability matrix; i and j are the locations of the entities; k is the type of the entity; Represents the probability prediction matrix from position i to position j of the k-th entity, which is the prediction result of the entity; Represents the actual label matrix from position i to position j of the k-th entity, which is the true label.

[0136] GlobalPointer uses a binary cross entropy loss function model to predict the value of each matrix element With the true label Calculate the loss. GlobalPointer generates vectors through linear transformation, and combines RoPE position encoding to enhance the position information representation, and finally generates the logits matrix of the entity. After normalization using the Sigmoid function, the model is optimized through the binary cross entropy loss function to complete the accurate prediction of various entities.

[0137] This embodiment is based on deep learning technology, especially the Transformer structure and the MedBERT-GP model, and proposes a method for named entity recognition (NER) in the field of electric metering. This method mainly solves the problem that traditional methods cannot effectively handle nested entities, long entities and complex semantics in complex electric metering texts. Token embedding, segment embedding and position embedding are generated by pre-training the embedding matrix. These embedding information are combined to form a word vector representation of each Token, thereby improving the semantic understanding ability of the model. By generating the logits matrix and using the Sigmoid function for probability normalization, the entity category and its start and end positions are finally output to complete the entity recognition task in the patent text in the field of electric metering.

[0138] In order to make the application process of the solution of the present invention clearer, an example of a workflow is given.

[0139] The text data of the meter is: On December 15, 2024, the meter No. EM12345 failed, the displayed voltage value was 220V, the current value was 5A, and the user requested maintenance. The operator Zhang Wei checked the meter on December 16, 2024 and found that there might be a problem with the current sensor of the meter, and recommended replacing the current sensor.

[0140] Preprocess the text data (word segmentation): `meter number`->meter number; `EM12345`->meter number entity; `fault`->fault type; `voltage value`->voltage entity; `220V`->voltage value; `current value`->current entity; `5A`->current value; `operator`->operator entity; `Zhang Wei`->operator name; December 16, 2024->date; inspection->operation behavior; replacement of current sensor->operation behavior description

[0141] Pre-train the labeled units for feature extraction and generate a high-dimensional embedding vector for each labeled unit:

[0142] High-dimensional embedding vector example: `Meter number`: [0.34, -0.23, 0.56, ...]; `EM12345`: [0.88, -0.12, 0.32, ...]; `220V`: [0.72, -0.24, 0.54, ...]; `Zhang Wei`: [0.65, 0.34, -0.42, ...]; Input the high-dimensional embedding vector into the MedBERT-GP hybrid model for entity boundary prediction steps:

[0143] Through the multi-layer Transformer, deep semantic learning of the embedding vector is performed to calculate the association between tokens. The score of each potential entity is calculated: the "electricity meter number" entity goes from the "EM12345" boundary to the "electricity meter number".

[0144] "220V" is the voltage entity; "5A" is the current entity.

[0145] Output recognized entities: `Meter number`->"EM12345"; `Voltage value`->"220V"; `Current value`->"5A"; `Operator`->"Zhang Wei"; `Fault type`->"Current sensor fault"

[0146] Entity category prediction step:

[0147] Through cross-category matching, the category of the entity is further determined: `EM12345`->meter number; `220V`->voltage value; `5A`->current value; `Zhang Wei`->operator; `current sensor failure`->fault type

[0148] Output entity category: `Meter number`: "EM12345"; `Voltage value`: "220V"; `Current value`: "5A"; `Operator`: "Zhang Wei"; `Fault type`: "Current sensor fault"

[0149] Model optimization and entity extraction: The cross entropy loss function is used to calculate the accuracy of entity recognition, and parameter optimization is performed to improve the accuracy. The optimized model can accurately identify and extract key information in the text, such as meter number, voltage, current, etc.

[0150] Knowledge graph construction: Through the identified entity information, a knowledge graph of meter faults can be constructed. The relationship between nodes is as follows:

[0151] Node: Meter number: EM12345; Voltage value: 220V; Current value: 5A; Operator: Zhang Wei; Fault type: Current sensor fault

[0152] Spectrum relationship: "EM12345"—[Fault occurred]→"Current sensor fault"; "Zhang Wei"—[Perform inspection]→"EM12345"; "EM12345"—[Voltage]→"220V"; "EM12345"—[Current]→"5A"

[0153] The above embodiment uses specific workflow examples to illustrate the data preprocessing stage, preprocessing stage, training feature extraction stage, prediction and optimization stage, which can better reflect the role of the present invention in entity recognition of metering meters, and help to understand the text in a structured manner, so that complex meter fault-related text can be decomposed into meaningful elements, effectively improving the recognition accuracy of complex entities in the field of metering meters.

[0154] In one embodiment, in addition to the above embodiment, a specific implementation process of using an encoder to process the word vector of each tag unit in step 2035 to generate a high-dimensional embedding vector for each tag unit is introduced, and step 2035 includes:

[0155] A bidirectional long short-term memory network is used to extract the deep semantic information of the context of the word vector of the tag unit. At the same time, the Transformer encoder is used to capture the global dependency between the word vectors of the tag unit through the self-attention mechanism to generate a high-dimensional embedding vector for each tag unit.

[0156] BiLSTM (Bidirectional Long Short-Term Memory Network) is an upgraded version of the long short-term memory network and is widely used in tasks that process sequence data. Compared with the traditional unidirectional LSTM, BiLSTM consists of two LSTMs, the forward and backward LSTMs, which can more comprehensively capture the contextual information in the input sequence. In BiLSTM, the input sequence is first fed into two independent LSTM layers, namely the forward LSTM layer and the backward LSTM layer. Among them, the forward LSTM starts from the beginning of the input sequence and gradually processes the input information at each moment, and controls the flow and forgetting of information through the internal gating mechanism. Similarly, the backward LSTM starts from the end of the input sequence and reversely processes the input information at each moment. In this way, BiLSTM can simultaneously process the forward and backward information of the input sequence, thereby more comprehensively capturing the contextual semantic features in the text of the metering meter field, especially for complex dependencies and long-distance semantic connections in the text. By combining the hidden states of the forward and backward, BiLSTM can better understand the meaning of metering meter professional terms, especially the impact of context on terms, thereby improving the accuracy of entity recognition. BiLSTM performs well in capturing sequence dependency characteristics and accurately identifying entity boundaries, and can effectively solve entity nesting and boundary fuzziness problems, which are important requirements for named entity recognition tasks in the construction of knowledge graphs for electric metering. Fig.10 shown.

[0157] Exemplarily, the deep semantic information of the context of the word vector of the tag unit is extracted using a bidirectional long short-term memory network, including:

[0158] The word vector of the tag unit is input into the forward LSTM of the bidirectional long short-term memory network, and the word vector of the input tag unit is processed sequentially to capture the sequential dependency information of the previous context.

[0159] The word vector of the tag unit is input into the backward LSTM of the bidirectional long short-term memory network, and the word vector of the input tag unit is processed in reverse order to capture the reverse dependency information of the previous context.

[0160] The sequential dependency information of the previous context and the reverse dependency information of the previous context are comprehensively represented to extract the deep semantic information of the context of the word vector of the tag unit.

[0161] Exemplarily, start with input data, which is usually raw text data, but can also be feature embedding generated by a pre-trained model. Starting from the start position of the input sequence, process the input sequentially to capture the dependency information of the previous context. Starting from the end position of the input sequence, process the input in reverse order to capture the dependency information of the subsequent context. Combine the hidden states of the forward and backward LSTMs (e.g., concatenate or add) to generate a comprehensive representation of each sequence position, taking into account the bidirectional context information. Use the combined output to predict the optimal entity label sequence based on context relevance. Finally, generate a labeled sequence to identify entity information in the text.

[0162] BiLSTM is widely used in natural language processing tasks, such as named entity recognition, sentiment analysis, and machine translation. This embodiment uses BiLSTM to model bidirectional context, which greatly improves the model's processing effect on sequence data.

[0163] The entity recognition method of the knowledge graph of an electric meter provided in the embodiment of the present application is based on Transformer and BiLSTM. By combining the Transformer encoder and the BiLSTM model, and taking advantage of the self-attention mechanism and the bidirectional LSTM, the model's entity recognition ability in the text of the electric meter field is significantly improved. Secondly, using the PoPE (rotational position encoding) technology, PoPE is introduced to enhance the model's processing ability for long entities, especially in long entities and complex contexts, which significantly improves the accuracy of boundary prediction. In terms of multi-level feature fusion, by combining Token embedding, segment embedding and position embedding, a high-dimensional word vector representation is generated, and combined with deep semantic feature extraction, the semantic understanding ability of the model is improved, especially in the professional terminology and complex semantic understanding in the power field.

[0164] See also Fig.11 An embodiment of the present application provides an entity recognition device for a knowledge graph of a metering meter, which executes an entity recognition method for a knowledge graph of a metering meter as in the above embodiment, including a data acquisition module 301, a preprocessing module 302, a pretraining module 303, a prediction module 304 and an optimization module 305.

[0165] The data acquisition module 301 is used to acquire text data of the meter.

[0166] The preprocessing module 302 is used to preprocess the text data to obtain the marking units of the text data.

[0167] The pre-training module 303 is used to perform pre-training embedding on each tag unit and generate a high-dimensional embedding vector for each tag unit using an encoder.

[0168] The prediction module 304 is used to input the high-dimensional embedding vector of each tag unit into the MedBERT-GP hybrid model, and determine the prediction result of the entity through linear transformation and dot product calculation.

[0169] The optimization module 305 is used to adjust the parameters of the MedBERT-GP hybrid model based on the loss function between the predicted result of the entity and the true label to obtain the optimized MedBERT-GP hybrid model; the optimized MedBERT-GP hybrid model is used to identify and extract the entity information of the text data of the metering meter to form a knowledge graph of the metering meter.

[0170] Exemplarily, the pre-training module 303 is specifically used for:

[0171] Perform unit embedding on each tag unit to obtain a unit embedding vector; perform segment embedding on each tag unit to obtain a segment embedding vector; perform position embedding on each tag unit to obtain a position embedding vector; add the unit embedding vector, segment embedding vector and position embedding vector to form a word vector representation of each tag unit; use the encoder to process the word vector of each tag unit to generate a high-dimensional embedding vector for each tag unit.

[0172] Exemplarily, the prediction result of the entity includes the boundary of the entity and the category of the entity.

[0173] The MedBERT-GP hybrid model is used to linearly transform the hidden representation of the high-dimensional embedding vector to generate a query vector and a key vector; and based on the query vector and the key vector, the dot product is used to calculate the span score of each entity; and the rotational position encoding is introduced to update the span score of each entity and determine the boundary of each entity; and the updated span scores of each entity are aggregated to generate a probability matrix representing the category of each entity and determine the category of each entity.

[0174] Exemplarily, the span score is expressed as:

[0175] score(i,j)=q i ·k j (6)

[0176] Among them, score(i,j) represents the score of the span from position i to position j being predicted as an entity category; q i is the query vector for position i; k j is the key vector at position j.

[0177] Exemplarily, the MedBERT-GP hybrid model uses the MedBERT-wwm model as an encoder and the GlobalPointer model as a decoder; the MedBERT-wwm model is used to understand the input circuit breaker text and generate a hidden vector containing context information; the GlobalPointer model is used to convert the hidden vector containing context information output by the encoder into a probability matrix representing the entity span based on the span prediction strategy, and predict the entity span based on the global position information of the head and tail of the entity.

[0178] Exemplarily, the loss function L is expressed as:

[0179]

[0180] Where N is the sum of all elements in the probability matrix; i and j are the locations of the entities; k is the type of the entity; is the probability prediction matrix from position i to position j of the k-th entity; is the actual label matrix from position i to position j of the k-th entity.

[0181] Exemplarily, in the pre-training module 303, an encoder is used to process the word vector of each tag unit to generate a high-dimensional embedding vector for each tag unit, including:

[0182] A bidirectional long short-term memory network is used to extract the deep semantic information of the context of the word vector of the tag unit. At the same time, the Transformer encoder is used to capture the global dependency between the word vectors of the tag unit through the self-attention mechanism to generate a high-dimensional embedding vector for each tag unit.

[0183] Exemplarily, the deep semantic information of the context of the word vector of the tag unit is extracted using a bidirectional long short-term memory network, including:

[0184] The word vector of the tag unit is input into the forward LSTM of the bidirectional long short-term memory network, and the word vector of the input tag unit is processed sequentially to capture the sequential dependency information of the previous context; the word vector of the tag unit is input into the backward LSTM of the bidirectional long short-term memory network, and the word vector of the input tag unit is processed in reverse order to capture the reverse dependency information of the previous context; the sequential dependency information of the previous context and the reverse dependency information of the previous context are comprehensively represented to extract the deep semantic information of the context of the word vector of the tag unit.

[0185] Exemplarily, the data acquisition module 301 is specifically used for:

[0186] The text data is segmented and stop words are removed to obtain preprocessed text data; the preprocessed text data is serialized to obtain tag units of the text data.

[0187] The beneficial effects of the entity recognition device of the metering meter knowledge graph in the embodiment of the present application can be found in the beneficial effects of the entity recognition method of the metering meter knowledge graph in the above embodiment, which will not be repeated here.

[0188] It should be noted that although several units / modules or sub-units / modules of the entity recognition device of the knowledge graph of the metering meter are mentioned in the above detailed description, this division is only exemplary and not mandatory. In fact, according to the implementation of the present application, the features and functions of two or more units / modules described above can be concretized in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be concretized.

[0189] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0190] The present application also provides a computer-readable storage medium, in which computer-readable storage medium is stored computer execution instructions. When a processor executes the computer execution instructions, the entity recognition method of the metering meter knowledge graph provided in the above embodiment of the present application is implemented.

[0191] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0192] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for entity recognition of a knowledge graph of an electric meter, characterized in that: include: Get the text data of the meter; Preprocessing the text data to obtain a marking unit of the text data; Pre-train the tag units and use the encoder to generate a high-dimensional embedding vector for each tag unit; The high-dimensional embedding vector of each token unit is fed into the MedBERT-GP hybrid model to determine the prediction result of the entity; Based on the loss function between the predicted result of the entity and the true label, the parameters of the MedBERT-GP hybrid model are adjusted to obtain an optimized MedBERT-GP hybrid model; the optimized MedBERT-GP hybrid model is used to identify and extract entity information of the text data of the metering meter to form a knowledge graph of the metering meter.

2. The entity recognition method of the knowledge graph of the metering meter according to claim 1 is characterized in that: The pre-training of the labeling unit and using the encoder to generate a high-dimensional embedding vector for each labeling unit include: Perform label unit embedding on each label unit to obtain a unit embedding vector; Perform segment embedding on each tag unit to obtain a segment embedding vector; Perform position embedding on each tag unit to obtain a position embedding vector; Add the unit embedding vector, segment embedding vector, and position embedding vector to form the word vector representation of each token unit; The encoder is used to process the word vector of each token unit to generate a high-dimensional embedding vector for each token unit.

3. The entity recognition method of the knowledge graph of the metering meter according to claim 1 is characterized in that: The prediction result of the entity includes the boundary of the entity and the category of the entity; The MedBERT-GP hybrid model is used to linearly transform the hidden representation of the high-dimensional embedding vector to generate a query vector and a key vector; and based on the query vector and the key vector, a span score of each entity is calculated using a dot product; And introduce rotation position encoding, update the span score of each entity, and determine the boundary of each entity; And aggregate the updated span scores of each entity to generate a probability matrix representing the category of each entity and determine the category of each entity.

4. The entity recognition method of the knowledge graph of the metering meter according to claim 3 is characterized in that: The span fraction is expressed as: score(i,j)=q i ·k j Among them, score(i,j) represents the score of the span from position i to position j being predicted as an entity category; q i is the query vector at position i; k j is the key vector for position j.

5. The entity recognition method of the knowledge graph of the metering meter according to claim 1 is characterized in that: The MedBERT-GP hybrid model uses the MedBERT-wwm model as an encoder and the GlobalPointer model as a decoder; the MedBERT-wwm model is used to understand the input circuit breaker text and generate a hidden vector containing context information; the GlobalPointer model is used to convert the hidden vector containing context information output by the encoder into a probability matrix representing the entity span based on the span prediction strategy, and predict the entity span according to the global position information of the head and tail of the entity.

6. The entity recognition method of the knowledge graph of the metering meter according to claim 5 is characterized in that: The loss function L is expressed as: Where N is the sum of all elements in the probability matrix; i and j are the locations of the entities; k is the type of the entity; is the probability prediction matrix from position i to position j of the k-th entity; is the actual label matrix from position i to position j of the k-th entity.

7. The entity recognition method of the knowledge graph of the metering meter according to claim 2 is characterized in that: The encoder is used to process the word vector of each tag unit to generate a high-dimensional embedding vector for each tag unit, including: A bidirectional long short-term memory network is used to extract the deep semantic information of the context of the word vector of the tag unit. At the same time, the Transformer encoder is used to capture the global dependency between the word vectors of the tag unit through the self-attention mechanism to generate a high-dimensional embedding vector for each tag unit.

8. The entity recognition method of the knowledge graph of the metering meter according to claim 7 is characterized in that: The method of extracting deep semantic information of the context of the word vector of the tag unit by using the bidirectional long short-term memory network includes: Input the word vector of the tag unit into the forward LSTM of the bidirectional long short-term memory network, sequentially process the word vector of the input tag unit, and capture the sequential dependency information of the previous context; Input the word vector of the tag unit into the backward LSTM of the bidirectional long short-term memory network, process the word vector of the input tag unit in reverse order, and capture the reverse dependency information of the previous context; The sequential dependency information of the previous context and the reverse dependency information of the previous context are comprehensively represented to extract the deep semantic information of the context of the word vector of the marking unit.

9. The entity recognition method of the knowledge graph of the metering meter according to any one of claims 1 to 8, characterized in that: The preprocessing of the text data to obtain a marking unit of the text data includes: Perform word segmentation and stop word removal on the text data to obtain preprocessed text data; The preprocessed text data is serialized to obtain the marking units of the text data.

10. An entity recognition device for a knowledge graph of an electric meter, characterized in that: The entity recognition method using the knowledge graph of a metering meter as claimed in any one of claims 1 to 9 comprises: A data acquisition module, used to acquire text data of the meter; A preprocessing module, used for preprocessing the text data to obtain a marking unit of the text data; A pre-training module is used to pre-train the embedding of each labeled unit and use the encoder to generate a high-dimensional embedding vector for each labeled unit; The prediction module is used to input the high-dimensional embedding vector of each token unit into the MedBERT-GP hybrid model and determine the prediction result of the entity through linear transformation and dot product calculation; The optimization module is used to adjust the parameters of the MedBERT-GP hybrid model based on the loss function between the prediction result of the entity and the true label to obtain an optimized MedBERT-GP hybrid model; the optimized MedBERT-GP hybrid model is used to identify and extract entity information of the text data of the metering meter to form a knowledge graph of the metering meter.

Images