Transformer fault diagnosis method and system based on fusion knowledge graph and large language model

Abstract

The present invention discloses a transformer fault diagnosis method and system based on the fusion of knowledge graph and large language model, including obtaining unstructured text and semi-structured text; constructing a joint entity relationship extraction model and a question-answering module, the joint entity relationship extraction model including a coding layer, a head entity recognition layer and a tail entity and relationship joint recognition layer; the question-answering module including a LangChain model and a large language model, the output of LangChain serving as the input of the large language model; obtaining fault problem text, inputting the fault problem text into the LangChain model and outputting professional knowledge, inputting the professional knowledge and fault problem text into the large language model, outputting professional fault diagnosis answers; interactively combining the professional fault diagnosis answers with high-quality knowledge triple texts, and outputting the final fault diagnosis answers. The present invention can specifically solve the problems of weak information relevance and low decision-making efficiency in the transformer operation and maintenance process, and greatly improve the health management level of power transformers.

Description

Technical Field

[0001] The present invention relates to the technical field of transformer fault diagnosis, and in particular to a transformer fault diagnosis method and system based on a fusion knowledge graph and a large language model. Background Art

[0002] The trend of vigorously promoting the digital transformation and upgrading of power grids has put forward more stringent requirements for the operation and maintenance standards of traditional power equipment. Transformers are core equipment in the power system. Failures of varying degrees can pose challenges to the safe and stable operation of the power grid, resulting in significant losses for power companies.

[0003] Currently, transformer operation and maintenance (O&M) relies primarily on the traditional experience accumulated over time by O&M personnel. This approach has numerous limitations. First, empirical O&M measures lack scientific theoretical foundations and systematic explanations, making it difficult to provide clear fault analysis and handling logic for complex faults. Second, O&M personnel often react slowly to changing fault conditions. This is because accumulating experience takes time, and facing complex and changing fault conditions, it is difficult to quickly make accurate judgments and effective responses. Personnel using the traditional empirical O&M model may be unable to identify and address these changes in a timely manner, thus delaying the optimal time to address the fault. Furthermore, transformer failures are frequent, their structures are complex and varied, and different types of transformers have different O&M methods. Therefore, transformer O&M requires high professional expertise from technicians and presents significant challenges. Currently, power systems have accumulated a large number of fault handling cases, primarily in unstructured and semi-structured text formats. Traditional data analysis methods are insufficient for processing this unstructured and semi-structured data, failing to effectively uncover the underlying patterns and knowledge, making it difficult to implement data-driven intelligent O&M decisions.

[0004] In recent years, research on knowledge graphs has made considerable progress in the power system field. Knowledge graphs, characterized by strong data specialization and the integration of expert mechanisms, can integrate and represent information from diverse data sources, forming a knowledge network that provides more comprehensive information for power system decision support. However, existing technologies often use pipeline models to construct knowledge graphs. This traditional pipeline model suffers from problems such as error propagation, information redundancy, and the inability to identify overlapping entity pairs and relationships. Knowledge graphs are also easily limited by the knowledge content they construct. Furthermore, building and maintaining knowledge graphs requires significant resources and expertise, and the difficulty of updating and expanding them increases with the amount of knowledge, resulting in poor timeliness of information. Meanwhile, large language models (LLMs) have also rapidly developed, demonstrating excellent processing capabilities across a variety of natural language tasks. They use techniques such as pre-training and fine-tuning to accurately answer complex questions. However, LLMs are black-box models that implicitly encode knowledge information in model parameters, resulting in a lack of interpretability and susceptibility to hallucinations. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present invention provides a transformer fault diagnosis method and system based on the fusion of knowledge graph and large language model. By jointly extracting entity relationships to construct a knowledge graph and calling the large language model, it can specifically solve the problems of weak information correlation and low decision-making efficiency in the transformer operation and maintenance process, thereby greatly improving the health management level of power transformers.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In the first aspect, the present invention proposes a transformer fault diagnosis method based on the fusion of knowledge graph and large language model, including: obtaining unstructured text of transformer operation and maintenance and semi-structured text of transformer fault, and preprocessing the obtained unstructured text and semi-structured text;

[0008] Construct a joint entity relationship extraction model and a question-answering module. The joint entity relationship extraction model is used to extract knowledge from preprocessed unstructured text and semi-structured text, output original knowledge triple text, and perform knowledge fusion on the original knowledge triple text to obtain high-quality knowledge triple text. The joint entity relationship extraction model includes an encoding layer, a head entity recognition layer, and a tail entity and relationship joint recognition layer. The question-answering module includes a LangChain model and a large language model, and the output of the LangChain serves as the input of the large language model.

[0009] Obtain the fault problem text, input the fault problem text into the LangChain model and output professional knowledge, input the professional knowledge and fault problem text into the large language model, and output a professional fault diagnosis answer; interactively combine the professional fault diagnosis answer with the high-quality knowledge triple text to output the final fault diagnosis answer.

[0010] As a further technical solution, the preprocessing includes filtering stop words and outlier processing; wherein, the method for filtering stop words is: constructing a stop word dictionary based on the unstructured text and semi-structured text, and then using the stop word dictionary to filter the stop words of the unstructured text and semi-structured text; the outlier processing includes denoising, text normalization and missing value processing, wherein, the denoising is to remove errors and irrelevant text in the unstructured text and semi-structured text, the text normalization is to convert the unstructured text and semi-structured text into a consistent format, and the missing value processing is to use interpolation method for processing.

[0011] As a further technical solution, the encoding layer adopts the BERT model. The BERT model is a language representation model based on a multi-layer bidirectional Transformer, which is used to extract feature information and learn deep representations by jointly adjusting the context of each word. Specifically:

[0012] h0=SW S +W P

[0013] h α =Trans(h α-1 ),α∈[1,N]

[0014] Among them, S represents the one-hot encoding vector matrix of the words in the input sentence, W S Represents the word embedding matrix, W P represents the position embedding matrix, h α represents the hidden state vector, that is, the context representation of the input sentence at the αth layer, and N is the number of Transformer modules.

[0015] As a further technical solution, the head entity recognition layer uses two identical binary classifiers to detect the start and end positions of the head entity respectively, and assigns a binary mark to each token to mark whether the current mark corresponds to the start and end positions of the head entity. The specific operations for each token are as follows:

[0016]

[0017] in, and They represent the probability of marking the i-th token of the input text as the start position and the end position of the head entity respectively; x i is the encoded representation of the i-th token in the input sequence, i.e. x i =h N [i], where W(·) represents the weight matrix, b(·) represents the bias term, and σ represents the sigmoid activation function.

[0018] As a further technical solution, the specific operations for each token in the tail entity and relationship joint identification layer are as follows:

[0019]

[0020] in, and Respectively represent the probability of identifying the i-th token in the input sequence as the start and end position of the tail entity, Represents the encoded representation vector of the k-th entity detected by the previous layer.

[0021] As a further technical solution, the specific method for knowledge fusion of the original knowledge triple text is: fusion disambiguation and entity disambiguation; the fusion disambiguation adopts the semantic similarity method, that is, improving the word vector similarity between different synonyms to perform fusion disambiguation; the entity disambiguation adopts the deep learning method to perform entity linking on the identified entities.

[0022] As a further technical solution, the professional fault diagnosis answer is interactively combined with the high-quality knowledge triple text through extraction and transformation, specifically including: information extraction and structural text readability; the information extraction is to embed an attention key and value into the prefix at each layer of the large language model, namely: Where K∈R l×d , represents the given original key vector; V∈R l×d , represents the given original value vector; P k and P v Represents a trainable vector respectively; (i) represents the part of the vector corresponding to the i-th attention head;

[0023] The structured text readability is to convert the nodes in the high-quality knowledge triple text into prompt text and input it into the large language model to output the final fault diagnosis answer.

[0024] In the second aspect, the present invention proposes a transformer fault diagnosis system based on the fusion of knowledge graph and large language model, which includes the following modules:

[0025] A text acquisition module is configured to: acquire unstructured text of transformer operation and maintenance and semi-structured text of transformer faults, and pre-process the acquired unstructured text and semi-structured text;

[0026] The knowledge graph and question-answering model construction module is configured to: construct a joint entity relationship extraction model and a question-answering model for extracting knowledge from preprocessed unstructured and semi-structured texts, outputting original knowledge triple texts, and performing knowledge fusion on the original knowledge triple texts to obtain high-quality knowledge triple texts, wherein the joint entity relationship extraction model includes an encoding layer, a head entity recognition layer, and a tail entity and relationship joint recognition layer; the question-answering model includes a LangChain model and a large language model, and the output of the LangChain serves as the input of the large language model;

[0027] The fault diagnosis answer output module is configured to: obtain the fault problem text, input the fault problem text into the LangChain model and output professional knowledge, input the professional knowledge and fault problem text into the large language model, and output professional fault diagnosis answers; interactively combine the professional fault diagnosis answers with high-quality knowledge triple texts to output the final fault diagnosis answers.

[0028] One or more technical solutions of the present invention have the following beneficial effects:

[0029] (1) The present invention constructs a joint entity relationship extraction model. By converting entity recognition and relationship extraction tasks into joint extraction, it effectively solves the problems of error propagation, information redundancy, and overlapping triples in the traditional pipeline model, thereby improving the accuracy of entity relationship extraction.

[0030] (2) The present invention uses P-tuning v2 to fine-tune the large language model, which brings many improvements to the question-answering module. In terms of training efficiency, the training time and computing cost are significantly reduced. Traditional large language model training often requires a lot of computing resources and time, while P-tuning v2 technology can achieve better performance of the model in a shorter time through a specific fine-tuning strategy. At the same time, the generalization ability of the training model is retained, so that the model can still give reasonable answers when faced with unseen transformer fault-related questions. This avoids overfitting of the model to specific training data, thereby improving the robustness of the question-answering system in practical applications. By improving the information extraction capability and making the structured data easy to read, the professionalism and accuracy of the answers are further enhanced. Enhanced information extraction capabilities enable the model to more accurately extract key information from text and form triples, effectively matching them with the knowledge graph to obtain more relevant knowledge to enrich the answer content. The readability of structured data allows complex information in the knowledge graph to be better presented to users in natural language, improving their understanding and trust in the diagnostic results. Ultimately, this improves the overall accuracy of the question-answering system, providing more reliable and efficient intelligent assistance for transformer operation and maintenance. It also improves the model's interpretability, preventing it from being affected by hallucinations. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0032] Figure 1 Construct a flow chart for the knowledge graph question answering system based on a large language model in the present invention;

[0033] Figure 2 It is a framework diagram of the joint entity relationship extraction model in the present invention;

[0034] Figure 3 This is a flow chart of the knowledge question and answer in the present invention;

[0035] Figure 4 This is a flow chart of the method for generating question-answering data based on a large language model in the present invention; DETAILED DESCRIPTION

[0036] It should be noted that the following detailed description is illustrative and is intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0037] Example 1

[0038] The present invention provides a transformer fault diagnosis method based on the fusion of knowledge graph and large language model, such as Figure 1 As shown, the specific steps include:

[0039] S1: Obtain unstructured text of transformer operation and maintenance and semi-structured text of transformer faults, and preprocess the obtained unstructured text and semi-structured text;

[0040] S2: Construct a joint entity relationship extraction model and a question-answering module. The joint entity relationship extraction model is used to extract knowledge from preprocessed unstructured and semi-structured texts, output original knowledge triple texts, and perform knowledge fusion on the original knowledge triple texts to obtain high-quality knowledge triple texts. The joint entity relationship extraction model includes an encoding layer, a head entity recognition layer, and a tail entity and relationship joint recognition layer. The question-answering module includes a LangChain model and a large language model, and the output of the LangChain serves as the input of the large language model.

[0041] S3: Obtain the fault problem text, input the fault problem text into the LangChain model and output professional knowledge, input the professional knowledge and fault problem text into the large language model, and output a professional fault diagnosis answer; interactively combine the professional fault diagnosis answer with the high-quality knowledge triple text to output the final fault diagnosis answer.

[0042] In step S1, the accuracy of the transformer fault diagnosis method depends heavily on the selection and preprocessing of the data set. To ensure the quality of the text dataset, this embodiment uses crawler technology to crawl unstructured text, such as public text on web pages containing "power transformer operation and maintenance," as well as semi-structured text such as transformer field fault analysis reports and anomaly detection reports, as the original training dataset. Furthermore, power transformer defects are complex, and the recording formats of defect text vary. Therefore, it is necessary to analyze the recording characteristics of protection device defect text and preprocess the text data based on this analysis.

[0043] In this embodiment, the preprocessing includes filtering stop words and outlier processing; stop words in the text increase redundancy and noise in the data, and it is necessary to filter stop words to reduce their interference with the extraction of defective text entities. In order not to damage the entity information in the data, the method for filtering stop words is as follows: First, a stop word dictionary is constructed according to the characteristics of the text data (unstructured text and semi-structured text), and these characteristics include: 1. High frequency: The stop word dictionary usually contains words that appear very frequently in the text, such as 'de' (的), 'shi' (是), 'zai' (在), etc. 2. Functionality: Stop words often have no actual semantic content and are functional words, such as prepositions, conjunctions, and auxiliary words. 3. Redundancy: In text analysis, stop words may increase the dimension of the data but contribute little to the analysis results, which may lead to the problem of vector dimension explosion. 4. Context independence: The words in the stop word dictionary are usually independent of the context, and their meanings do not change with the context. Then, the Jieba tokenization package in Python is used to filter the stop words in the defective text in combination with the constructed stop word dictionary. During the outlier processing, it specifically includes denoising processing, text normalization processing, and missing value processing. Among them, denoising processing is to remove errors and irrelevant text in unstructured text and semi-structured text, text normalization is to convert unstructured text and semi-structured text into a consistent format. For example, 'pt' and 'PT' need to be uniformly replaced with capital letters to ensure the standardization of the data, and missing value processing is to use the interpolation method for processing.

[0044] To solve the problems of error propagation, information redundancy, and inability to recognize entity pair relationship overlap in the traditional pipeline model, in step S2, as Figure 1 shown, this embodiment constructs a joint entity relation extraction model (CasRel model). The key to improving the existing model lies in the design of the sublayer. The joint entity relation extraction model includes an encoding layer, a head entity recognition layer, and a tail entity and relation joint recognition layer. Specifically:

[0045] Encoding layer:

[0046] In step S2, the encoding layer in this embodiment uses a pre-trained BERT model. The BERT model is a language representation model based on multi-layer bidirectional Transformers, which is used to extract feature information. It learns deep representations by jointly adjusting the context of each word. Specifically:

[0047] h0 = SW S +W P

[0048] h α = Trans(h α-1 ), α ∈ [1, N]

[0049] Among them, S represents the one-hot encoding vector matrix of the words in the input sentence, W S Represents the word embedding matrix, W P represents the position embedding matrix, h α represents the hidden state vector, that is, the context representation of the input sentence at the αth layer, and N is the number of Transformer modules.

[0050] Head entity recognition layer:

[0051] The head entity recognition layer of the CasRel model directly encodes the output of the encoding layer. It uses two identical binary classifiers to detect the start and end positions of the head entity, respectively. Each token (word) is assigned a binary label to mark whether the current token corresponds to the start and end positions of the head entity. The specific operations for each token are as follows:

[0052]

[0053] in, and Respectively represent the probability of marking the i-th token of the input text as the start position and end position of the head entity. If the probability exceeds a certain threshold, the threshold for the head of the body is 0.6, and the threshold for the tail of the body is 0.5. In order to ensure a high recall rate while also improving the accuracy, the boundary of the head entity is effectively identified. The corresponding token will be assigned a label of 1, otherwise it will be assigned a label of 0; i is the encoded representation of the i-th token in the input sequence, i.e. x i =h N [i], where W(·) represents the weight matrix, b(·) represents the bias term, and σ represents the sigmoid activation function.

[0054] A likelihood function is also optimized in the head entity recognition layer to determine the scope of the head entity. Given a sentence, the maximum likelihood function of the head entity is:

[0055] In the above formula, p θ (s|x) represents the probability of entity s appearing given the parameter θ and the input sentence x, where L represents the length of the sentence and the indicator function I{Z} takes the value 1 if Z is true and 0 otherwise. is the binary mark of the starting position of the i-th token in x, Indicates the end position of the i-th token, parameter θ={W start b start W end b endThe process is to calculate the probability of a specific entity label sequence by multiplying the label probabilities at each position. During training, we hope to maximize this probability, that is, by adjusting the parameter θ to improve the model's prediction probability of the correct head entity label sequence.

[0056] Tail entity and relationship joint recognition layer:

[0057] The tail entity and relationship joint recognition layer and the head entity recognition layer directly decode the encoder output vector h N Different from the previous one, the joint entity and relationship recognition layer also considers the characteristics of the entity. The specific operations for each token are as follows:

[0058]

[0059] in, and Represent the probability of identifying the i-th token in the input text as the start and end position of the tail entity, respectively. Represents the encoding representation vector of the k-th entity detected by the previous layer. Similar to the head entity recognition layer, there will also be a probability threshold, and the thresholds for joint recognition of tail entities and relations are also 0.6 and 0.5. If the probability exceeds a certain threshold, the corresponding token will be assigned a label of 1, otherwise it will be assigned a label of 0. In order to make x i and The addition operation can be performed to take the average vector between the start and end tags of the k-th entity as

[0060] A likelihood function is also set in the joint recognition layer of tail entities and relations to determine the scope of the tail entity. By maximizing the likelihood function, the probability of the observed training data is maximized under these parameters. The set likelihood function is:

[0061]

[0062] Among them, p θ (o|s,x) represents the probability of the tail entity o appearing given the parameter θ, the input sentence x and the head entity s. is the binary mark of the starting position of the i-th token in x, Indicates the end position of the i-th token. For the case of a "null" empty object, all i tags and All are 0, parameters During the training process, we hope to maximize the likelihood probability, that is, to improve the model's prediction probability of the correct tail entity label sequence by adjusting the parameter θ.

[0063] In step S2, the optimization objective is also included. The final optimization objective function of the CasRel model is directly optimized at the triple level to maximize the likelihood function value. Specifically, the objective function can be expressed as:

[0064]

[0065] Where T j Represents a specific sentence x j The set of all possible triples associated with p θ (s|x) and Given in the above, for This means that for each input x j In, given the head entity s and not in T j In the case of relation r, the empty tail entity The predicted probability of . This objective function trains the model to accurately identify and distinguish entity relationships in text by maximizing the log-likelihood of these three parts. This approach allows the model to optimize directly at the triple level rather than relying on the traditional sequence labeling framework.

[0066] The joint entity relationship extraction model outputs the original knowledge triple text and performs knowledge fusion on the original knowledge triple text to obtain high-quality knowledge triple text. The original knowledge triple text generated in the knowledge extraction stage may be incomplete or lack logical relationships, which may cause knowledge conflicts in the retrieval process. Therefore, it is necessary to further integrate the extracted knowledge to ensure the accuracy and consistency of the knowledge, eliminate some uncertain information and contradictory knowledge in the original knowledge triple text, and improve the accuracy of the constructed knowledge graph. Specifically, it includes: fusion disambiguation and entity disambiguation, specifically:

[0067] Fusion digestion:

[0068] A common integration problem in the field of electric power knowledge is the co-reference problem, that is, the same physical object corresponds to multiple entity nodes. For example, two extracted words describe the same entity. In response to this phenomenon, the semantic similarity method is adopted in this embodiment. The cosine similarity of word vectors is calculated using the existing semantic similarity method. If it exceeds a certain threshold, it is a synonym. Then, for a group of synonyms, a clustering algorithm can be used to group them, and then the center point of each group is selected as the representative. That is, the similarity of word vectors between different synonyms is improved for fusion and resolution; entity disambiguation uses deep learning methods to perform entity linking on the identified entities for integration and resolution.

[0069] Entity Disambiguation:

[0070] To avoid information redundancy caused by storing entities in the database, entity disambiguation is used to resolve the ambiguity of homonymous entities. For example, "Beijing" is an address in the contract specification, but it is the name of a unit in the service bidding team specification. Therefore, a deep learning method is used to link the identified entities and merge the data. This deep learning method combines the existing long short-term memory network with dual attention: the first attention mechanism uses the entity embedding as an attention vector to highlight the information in the entity description; the second attention mechanism uses the entity context as an attention vector to highlight the information in the entity reference context. Finally, the correct entity is determined by combining similarity and prior probability.

[0071] In this example, high-quality knowledge triples are generated after knowledge fusion and stored in Neo4j. Neo4j is an open-source graph database management system developed in Java. It uses the Cypher declarative query language to operate and query graph databases, making it easy to operate. This example connects to Neo4j using the py2neo library to form a transformer fault diagnosis knowledge graph for storing structured data.

[0072] On the basis of the formation of the knowledge graph, the present invention calls the large language model to complete the construction of the question-answering module. Compared with the traditional knowledge graph question-answering system, the question-answering module in this embodiment adopts the method of generating logical forms first and then searching, which can directly utilize the powerful semantic understanding ability of the large language model. The construction of the question-answering module of the knowledge graph based on the large language model is completed through the following process: (1) The user asks a question to the module. After the question is filtered through information, it is combined with the relevant professional knowledge in the knowledge base to form a prompt, which is input into the professional question-answering module to obtain the answer; (2) The information extraction module extracts triples from the answer, matches it with the knowledge graph formed above, and obtains relevant node data; (3) After the user selects these node data, they are also input into the professional question-answering module in the form of prompts to obtain the answer enhanced by the knowledge graph. This two-way interaction realizes the deep integration of the large language model and the knowledge graph. It mainly includes three processes: information filtering, professional answering and extraction and transformation.

[0073] Information filtering:

[0074] Table 1 Prompt examples

[0075]

[0076] In the power transformer field, the prompt text P1 in Table 1 is input into the large language model to generate relevant questions. 70% of the question data regarding power transformer failures comes from existing question-answering datasets, and 30% of the relevant questions are generated using the large language model. Questions from non-professional fields are then filtered by adding a BERT-based text filter to limit the range of questions that the large model can answer. Using BERT's classification word vector H and a simple softmax-based classifier, the probability of predicting the class label L is:

[0077] P(L|H)=softmax(WH)

[0078] Where W is the parameter matrix for the classification task. By maximizing the log-probability of the correct label, all parameters in BERT and W are fine-tuned. This is modified to use a fully connected layer to obtain the probability of each label, ultimately selecting the label with the higher probability as the classification result. By filtering information, assuming that the set of all questions that can be input into the large language model is Q, the set of questions that the large language model can answer in a certain professional field is R, and the set of questions that can generate professional answers is D, it is clear that Q>R>D. Using fine-tuning to restrict will cause R to D, which will weaken the model's answering ability. Using a filter to force Q to R will ensure that the questions asked are within the range of R as much as possible. Although some data outside of R will enter the large model, the professional enhanced question-answering system designed in this article still retains a certain degree of general capabilities, and can also answer questions outside of R without professional verification.

[0079] Professional answer:

[0080] like Figure 3 As shown, this embodiment uses the LangChain+LLM (large language model) method to generate professional answers. First, the fault problem text is obtained, and the fault problem text is input into the LangChain model. Based on LangChain, professional knowledge related to the problem is retrieved in the knowledge base and output. Then, the professional knowledge and the fault problem text are combined to form a prompt text P3, which is input into the large language model, and finally a professional fault diagnosis answer is output.

[0081] Extraction conversion:

[0082] In order to interactively combine the professional fault diagnosis answers output by the large language model with the structured knowledge of the knowledge graph, this embodiment is completed through extraction and transformation. Extraction and transformation include information extraction and readability of structured text, where information extraction is to use P-tuning v2 fine-tuning to enhance the information extraction capability of the large language model, embed a trainable attention key and value into the prefix at each layer of the large language model, and give the original key vector K∈R l×dSum value vector V∈R l×d , the trainable vector P k , P v Will be connected with K and V respectively, and the calculation of the attention mechanism head is as follows:

[0083]

[0084] The superscript (i) represents the part of the vector corresponding to the i-th attention head. This method is used to fine-tune the large language model. The information extraction process forms new triples to facilitate entity matching with high-quality knowledge triple text in the knowledge graph, thereby enhancing the professionalism of the answer.

[0085] The readability of structured text is as follows: converting the nodes in the high-quality knowledge triple text into prompt text and inputting it into the large language model to output the final fault diagnosis answer. Specifically: Figure 4 As shown, to make structured data easier to read, the relevant nodes of the knowledge graph are converted into prompt text P2, which is then input into the large language model to obtain natural language answers enhanced by the knowledge graph, thus achieving a deep integration of the large language model and the knowledge graph. The specific method for generating question and answer data using the LLM (large language model) is as follows: input relevant user question data and the API_KEY of the large language model, create a link to the LLM interface through the API_KEY, select a user question data, generate a prompt based on the selected data, access the LLM interface by establishing a link, and input the prompt to generate a response. All question and answer data is extracted from the LLM generation results and aggregated into the final question and answer data, outputting the final fault diagnosis answer.

[0086] Example 2

[0087] This embodiment provides a transformer fault diagnosis system based on the fusion of knowledge graph and large language model, including the following modules:

[0088] A text acquisition module is configured to: acquire unstructured text of transformer operation and maintenance and semi-structured text of transformer faults, and pre-process the acquired unstructured text and semi-structured text;

[0089] The knowledge graph and question-answering model construction module is configured to: construct a joint entity relationship extraction model and a question-answering model for extracting knowledge from preprocessed unstructured text and semi-structured text, outputting original knowledge triple text, and performing knowledge fusion on the original knowledge triple text to obtain high-quality knowledge triple text, wherein the joint entity relationship extraction model includes an encoding layer, a head entity recognition layer, and a tail entity and relationship joint recognition layer; the question-answering model includes a LangChain model and a large language model, and the output of the LangChain serves as the input of the large language model;

[0090] The fault diagnosis answer output module is configured to: obtain the fault problem text, input the fault problem text into the LangChain model and output professional knowledge, input the professional knowledge and fault problem text into the large language model, and output professional fault diagnosis answers; interactively combine the professional fault diagnosis answers with high-quality knowledge triple texts to output the final fault diagnosis answers.

[0091] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A transformer fault diagnosis method based on the fusion of knowledge graph and large language model, characterized by: include: Obtain unstructured text of transformer operation and maintenance and semi-structured text of transformer faults, and preprocess the obtained unstructured text and semi-structured text; The preprocessing includes filtering stop words and outlier processing; wherein the method for filtering stop words is: constructing a stop word dictionary based on the unstructured text and semi-structured text, and then using the stop word dictionary to filter the stop words in the unstructured text and semi-structured text; the outlier processing includes denoising, text normalization and missing value processing, wherein the denoising is to remove errors and irrelevant text in the unstructured text and semi-structured text, the text normalization is to convert the unstructured text and semi-structured text into a consistent format, and the missing value processing is to use interpolation method for processing; Construct a joint entity relationship extraction model and a question-answering module. The joint entity relationship extraction model is used to extract knowledge from preprocessed unstructured text and semi-structured text, output original knowledge triple text, and perform knowledge fusion on the original knowledge triple text to obtain high-quality knowledge triple text. The joint entity relationship extraction model includes an encoding layer, a head entity recognition layer, and a tail entity and relationship joint recognition layer. The starting position threshold of the head entity and the tail entity is set to 0.6, and the ending position threshold is set to 0.

5. The question-answering module includes a LangChain model and a large language model. The output of the LangChain serves as the input of the large language model. The specific method for performing knowledge fusion on the original knowledge triple text is: fusion disambiguation and entity disambiguation; the fusion disambiguation adopts the semantic similarity method, and the semantic similarity method is used to calculate the cosine similarity of the word vector. If it exceeds a certain threshold, it is a synonym. Then, for a group of synonyms, a clustering algorithm is used to group them, and then the center point of each group is selected as the representative; the entity disambiguation adopts a deep learning method to perform entity linking on the identified entities, wherein the deep learning method selects the existing long short-term memory network and combines it with dual attention: the first attention mechanism uses the entity embedding as the attention vector to highlight the information part in the entity description; the second attention mechanism uses the entity context as the attention vector to highlight the information part in the entity reference context; finally, the correct entity is obtained by combining the similarity and the prior probability; Obtain the fault problem text, input the fault problem text into the LangChain model and output professional knowledge, input the professional knowledge and fault problem text into the large language model, and output a professional fault diagnosis answer; interactively combine the professional fault diagnosis answer with the high-quality knowledge triple text to output the final fault diagnosis answer.

2. The transformer fault diagnosis method based on the fusion of knowledge graph and large language model as claimed in claim 1 is characterized in that: The encoding layer uses the BERT model, which is a language representation model based on a multi-layer bidirectional Transformer. It is used to extract feature information and learn deep representations by jointly adjusting the context of each word. Specifically: Among them, S represents the one-hot encoding vector matrix of the words in the input sentence, represents the word embedding matrix, represents the position embedding matrix, Represents the hidden state vector, i.e., the input sentence in The contextual representation of the layer, is the number of Transformer modules.

3. The transformer fault diagnosis method based on the fusion of knowledge graph and large language model according to claim 1 is characterized in that: The head entity recognition layer uses two identical binary classifiers to detect the start and end positions of the head entity respectively, and assigns a binary mark to each token to mark whether the current mark corresponds to the start and end positions of the head entity. The specific operations for each token are as follows: in, and They represent the probability of marking the i-th token of the input text as the start position and end position of the head entity respectively; is the encoded representation of the i-th token in the input sequence, that is, , where W represents the weight matrix, b represents the bias term, Represents the sigmoid activation function.

4. The transformer fault diagnosis method based on the fusion of knowledge graph and large language model according to claim 1 is characterized in that: The specific operations for each token in the tail entity and relationship joint identification layer are as follows: in, and Respectively represent the first The probability of identifying a token as the start and end position of the tail entity, Indicates the first The encoding representation vector of each entity, is the encoded representation of the i-th token in the input sequence.

5. The transformer fault diagnosis method based on the fusion of knowledge graph and large language model according to claim 1 is characterized in that: The professional fault diagnosis answer and the high-quality knowledge triple text are interactively combined through extraction and transformation, specifically including: information extraction and structural text readability; the information extraction is to embed an attention key and value into the prefix at each layer of the large language model, namely: ;in, , represents the given original key vector; , represents the given original value vector; and Represents a trainable vector respectively; (i) represents the part of the vector corresponding to the i-th attention head; The structured text readability is to convert the nodes in the high-quality knowledge triple text into prompt text and input it into the large language model to output the final fault diagnosis answer.

6. A transformer fault diagnosis system based on a fusion knowledge graph and a large language model, adopting a transformer fault diagnosis method based on a fusion knowledge graph and a large language model as described in any one of claims 1 to 5, characterized in that: include: A text acquisition module is configured to: acquire unstructured text of transformer operation and maintenance and semi-structured text of transformer faults, and pre-process the acquired unstructured text and semi-structured text; The knowledge graph and question-answering model construction module is configured to: construct a joint entity relationship extraction model and a question-answering model, wherein the joint entity relationship extraction model is used to extract knowledge from preprocessed unstructured text and semi-structured text, output original knowledge triple text, and perform knowledge fusion on the original knowledge triple text to obtain high-quality knowledge triple text, wherein the joint entity relationship extraction model includes an encoding layer, a head entity recognition layer, and a tail entity and relationship joint recognition layer; the question-answering model includes a LangChain model and a large language model, and the output of the LangChain serves as the input of the large language model; The fault diagnosis answer output module is configured to: obtain the fault problem text, input the fault problem text into the LangChain model and output professional knowledge, input the professional knowledge and fault problem text into the large language model, and output a professional fault diagnosis answer; Interactively combine professional fault diagnosis answers with high-quality knowledge triple text to output the final fault diagnosis answer.

7. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps of the transformer fault diagnosis method based on the fusion of knowledge graph and large language model as described in any one of claims 1 to 5 are implemented.

8. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the transformer fault diagnosis method based on the fusion of knowledge graph and large language model as described in any one of claims 1 to 5 are implemented.

Images