Transformer fault diagnosis method and system based on the integration of knowledge graphs and large language models

Abstract

This invention provides a transformer fault diagnosis method and system based on the integration of knowledge graphs and large language models. [Solution] The knowledge graph question-and-answer system includes acquiring unstructured and semi-structured text; constructing a question-and-answer module including an encoding layer, a head entity recognition layer, a relationship extraction model including a tail entity and relationship recognition layer, a LangChain model, and a large language model that takes the output of the LangChain as input; acquiring fault problem text; inputting the fault problem text into the LangChain model; outputting expert knowledge; inputting the expert knowledge and fault problem text into the large language model; outputting an expert fault diagnosis answer; alternately combining the expert fault diagnosis answer with high-quality knowledge triple text; and outputting a final fault diagnosis answer. The present invention can significantly improve the health management level of power transformers.

Description

[Technical Field]

[0001] (Cross-reference of related applications) This invention claims priority to a Chinese patent application filed with the National Intellectual Property of China on December 23, 2024, with application number 202411905040.0, and titled "Transformer Fault Diagnosis Method and System Based on the Fusion of Knowledge Graphs and Large Language Models," the entirety of which is incorporated into this invention by reference, constitutes a part of this invention, and is used for all purposes.

[0002] The present invention relates to the field of transformer fault diagnosis technology, and more particularly to a transformer fault diagnosis method and system based on the fusion of knowledge graphs and large language models. [Background technology]

[0003] The trend towards the digital transformation and upgrade of power grids has led to stricter requirements for the operational maintenance standards of conventional power equipment. Transformers, as core components of the power system, can experience failures of varying degrees that threaten the safe and stable operation of the power grid and potentially result in significant losses for power companies.

[0004] Currently, transformer operation and maintenance work relies primarily on the traditional experience accumulated by operators and maintenance personnel over long periods. This approach has many limitations. On the one hand, empirical operation and maintenance measures lack scientific theoretical basis and system interpretation, making it difficult to provide clear fault analysis and processing logic when faced with complex failures. On the other hand, operators and maintenance personnel are often slow to react when faced with changes in failures. This is because it takes time for human experience to accumulate, and it is difficult to make quick, accurate judgments and effective responses when faced with complex and diverse failure situations that have not been experienced before. In the conventional empirical operation and maintenance mode, these changes cannot be recognized and processed immediately, which can delay the optimal timing of fault processing. At the same time, transformer failures are frequent, and their structures are complex and diverse, and different types of transformers have different operation and maintenance methods. Therefore, transformer operation and maintenance work requires a high level of expertise from engineers and is relatively difficult. Currently, many fault processing cases have accumulated in power systems, and these cases exist mainly in unstructured and semi-structured text format. Traditional data analysis methods are ineffective when processing such unstructured and semi-structured data, failing to effectively uncover the underlying disciplines and knowledge, and making it difficult to achieve data-driven intelligent operation and maintenance meaning determination.

[0005] In recent years, research on knowledge graphs has made considerable progress in the field of power systems. Knowledge graphs are characterized by their high level of data specialization and the integration of expert mechanisms, enabling them to unify and display information from different data sources, form knowledge networks, and provide more comprehensive information to support decision-making in power systems. However, conventional techniques often employ the main production line model to construct knowledge graphs. Such conventional main production line models have problems such as mispropagation, information redundancy, and the inability to recognize overlapping relationships between entity pairs. Furthermore, knowledge graphs are easily limited by the knowledge content they contain, and their construction and maintenance require significant resources and expertise. The difficulty of updating and expanding knowledge graphs increases with the amount of knowledge, leading to poor information timeliness. At the same time, large language models (LLMs) have also rapidly developed, demonstrating excellent processing capabilities in multiple natural language tasks. Techniques such as pre-training and fine-tuning enable them to accurately answer complex problems. However, LLMs, as black-box models, implicitly encode knowledge information into model parameters, resulting in a lack of interpretability and susceptibility to hallucinatory problems. [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] In response to the shortcomings of conventional technologies, the present invention provides a transformer fault diagnosis method and system based on the fusion of a knowledge graph and a large language model. By extracting and constructing a knowledge graph based on linked entity relationships and calling a large language model, the invention clearly solves problems such as weak information relevance and low decision-making efficiency in the operation and maintenance process of transformers, and can significantly improve the level of health management of power transformers. [Means for solving the problem]

[0007] To achieve the above objective, the present invention employs the following technical approach: According to a first aspect, the present invention provides a transformer fault diagnosis method based on the fusion of a knowledge graph and a large language model, the transformer fault diagnosis method comprising obtaining unstructured text on transformer operation and maintenance and semi-structured text on transformer faults, and pre-processing the obtained unstructured text and semi-structured text, The method involves constructing a collaborative entity relationship extraction model and a question-and-answer module, wherein the collaborative entity relationship extraction model performs knowledge extraction on pre-processed unstructured and semi-structured texts, outputs the original knowledge triple text, performs knowledge fusion on the original knowledge triple text, and is used to obtain high-quality knowledge triple text, and the collaborative entity relationship extraction model includes an encoding layer, a head entity recognition layer, and a tail entity and relationship recognition layer, and the question-and-answer module includes a LangChain model and a large language model, with the output of the LangChain being used as input to the large language model. This includes obtaining failure problem text, inputting the failure problem text into a LangChain model, outputting expertise, inputting the expertise and failure problem text into a large language model, outputting expert failure diagnosis answers, and alternately combining the expert failure diagnosis answers with high-quality knowledge triple text to output a final failure diagnosis answer.

[0008] As a further technical proposal, the preprocessing includes stop word filtering and outlier processing, wherein the stop word filtering method involves constructing a stop word dictionary based on the unstructured text and semi-structured text, and filtering stop words in the unstructured text and semi-structured text using the stop word dictionary; the outlier processing includes denoising, text normalization, and missing value processing, wherein the denoising involves removing text unrelated to errors in the unstructured text and semi-structured text; the text normalization involves converting the unstructured text and semi-structured text into a consistent format; and the missing value processing is an interpolation method.

[0009] As a further technical solution, the encoding layer adopts a BERT (Bidirectional Encoder Representations from Transformers) model, which is a language representation model based on a multi-layer bidirectional Transformer and is used to learn deep representations by extracting feature information and jointly adjusting the context of each word. Specifically, h0 = SW , i , , , , , N , , , , , i , , , , , , +W p h α = Trans(h α-1 ), where α ∈ [1, N], Here, S represents the one-hot encoded vector matrix of words in the input sentence, W s represents the word embedding matrix, W p represents the position embedding matrix, h α represents the hidden state vector, that is, the context representation of the α-th layer of the input sentence, and N is the number of Transformer modules.

[0010] As a further technical solution, the head entity recognition layer adopts two identical binary classifiers to detect the start and end positions of the head entity respectively, assigns a binary mark to each token, and is used to mark whether the current mark corresponds to the start and end positions of the head entity. The specific operations for each token are

Number

Number

Number

number

[0011] As a further technical proposal, the specific operations on each token in the tail entity and relationship recognition layer are as follows:

number

number

number

[0012] As a further technical proposal, the specific method for performing knowledge fusion on the original knowledge triple text is fusion elimination and entity ambiguity resolution. Fusion elimination employs a semantic similarity method, that is, it improves the word vector similarity between different synonyms to perform fusion elimination, and entity ambiguity resolution employs a deep learning method to perform entity linking on recognized entities.

[0013] As a further technical proposal, the aforementioned expert fault diagnosis answers and high-quality knowledge triple text are alternately combined by extraction and transformation, specifically including information extraction and structural text readability, wherein the information extraction involves embedding attention keys and values ​​as prefixes in each layer of the large language model, i.e.,

number

[0014] According to a second aspect, the present invention proposes a transformer fault diagnosis system based on the fusion of a knowledge graph and a large language model. A text acquisition module configured to acquire unstructured text for transformer operation and maintenance and semi-structured text for transformer failures, and to preprocess the acquired unstructured and semi-structured text, A knowledge graph and question-and-answer model construction module configured to construct a collaborative entity relationship extraction model and a question-and-answer model, which performs knowledge extraction on pre-processed unstructured and semi-structured text, outputs the original knowledge triple text, performs knowledge fusion on the original knowledge triple text, and is used to obtain high-quality knowledge triple text, wherein the collaborative entity relationship extraction model includes an encoding layer, a head entity recognition layer, and a tail entity and relationship recognition layer, and the question-and-answer model includes a LangChain model and a large language model, and the knowledge graph and question-and-answer model construction module takes the output of the LangChain as input to the large language model. It includes a fault diagnosis answer output module configured to acquire fault problem text, input the fault problem text into a LangChain model, output expertise, input the expertise and fault problem text into a large language model, output expert fault diagnosis answers, alternately combine the expert fault diagnosis answers and high-quality knowledge triple text, and output a final fault diagnosis answer. [Effects of the Invention]

[0015] One or more of the technical proposals of this invention have the following beneficial effects.

[0016] (1) The present invention improves the accuracy of entity relationship extraction by constructing a linked entity relationship extraction model and converting entity recognition and relationship extraction tasks into linked extraction, thereby effectively solving problems such as erroneous propagation, information redundancy, and triple duplication in the conventional main production line model.

[0017] (2) The present invention uses P-tuning v2 to fine-tune large language models, resulting in multifaceted improvements to the question-and-answer module. In terms of training efficiency, training time and computational cost have been significantly reduced. While conventional large language model training often requires enormous computational resources and time, P-tuning v2 technology, through specific fine-tuning policies, can achieve a relatively superior performance state for the model in a relatively short time. At the same time, by retaining the generalization ability with the trained model, the model can still provide reasonable answers even when faced with problems related to transformer failures that it has never seen before. The practical robustness of the question-and-answer system has been improved by avoiding the model overfitting to specific training data. The expertise and accuracy of the answers have been further enhanced by improved information extraction capabilities and the readability of structured data. By enhancing information extraction capabilities, the model can more accurately extract key information from text to form triples, effectively matching them with the knowledge graph to acquire more relevant knowledge and enrich the responses. The readability of the structured data allows for better presentation of complex information in the knowledge graph to the user in natural language, improving the user's understanding and confidence in the diagnostic results, ultimately achieving an overall improvement in the accuracy of the question-and-answer system, and providing reliable and efficient intelligent assistance for transformer operation and maintenance. At the same time, the interpretability of the model is improved, avoiding the influence of hallucinatory problems.

[0018] The accompanying drawings of the specification, which constitute part of the present invention, are for the purpose of providing a further understanding of the present invention, and the exemplary embodiments and descriptions thereof are for illustrative purposes only and do not constitute an unreasonable limitation of the present invention. [Brief explanation of the drawing]

[0019] [Figure 1] This is a flowchart for constructing a knowledge graph question-and-answer system based on a large language model in the present invention. [Figure 2] This is a framework diagram of the collaborative entity relationship extraction model in the present invention. [Figure 3] This is a knowledge question and answer flowchart for the present invention. [Figure 4] This is a flowchart of the method for generating question-and-answer data based on a large language model according to the present invention. [Modes for carrying out the invention]

[0020] The following detailed descriptions are illustrative and intended to further illustrate the present invention. Unless otherwise specified, all technical and scientific terms used in this invention have the same meaning as those commonly understood by those skilled in the art.

[0021] Example 1 The present invention provides a transformer fault diagnosis method based on the fusion of a knowledge graph and a large language model, and specifically includes the following steps, as shown in Figure 1.

[0022] S1: Obtain unstructured text for transformer operation and maintenance and semi-structured text for transformer failures, and perform preprocessing on the obtained unstructured and semi-structured text. S2: Construct a collaborative entity relationship extraction model and a question-and-answer module. The collaborative entity relationship extraction model extracts knowledge from pre-processed unstructured and semi-structured texts, outputs the original knowledge triple text, and performs knowledge fusion on the original knowledge triple text to obtain high-quality knowledge triple text. Here, the collaborative entity relationship extraction model includes an encoding layer, a head entity recognition layer, and a tail entity and relationship recognition layer. The question-and-answer module includes a LangChain model and a large language model, with the output of the LangChain being the input to the large language model. S3: Obtain the failure problem text, input the failure problem text into the LangChain model, output the expertise, input the expertise and failure problem text into the big language model, output the expert failure diagnosis answer, alternately combine the expert failure diagnosis answer and the high-quality knowledge triple text, and output the final failure diagnosis answer.

[0023] In step S1, the accuracy of the transformer fault diagnosis method largely depends on the selection and preprocessing of the dataset. To ensure the quality of the text dataset, this embodiment employs crawler technology to crawl unstructured text, such as disclosure text including "Operation and Maintenance of Power Transformers," and semi-structured text, such as transformer field fault analysis reports and anomaly detection reports, from web pages as the original training dataset. Furthermore, because the defect situations occurring in power transformer equipment are complex and the recording format of defect text differs, it is necessary to analyze the recording characteristics of defect text of protective devices and preprocess the text data based on that analysis.

[0024] In this embodiment, preprocessing includes stop word filtering and outlier handling. Stop words in text increase redundancy and noise in the data, and it is necessary to filter them to reduce interference with the extraction of defective text entities. In order not to destroy entity information in the data, the stop word filtering method first constructs a stop word dictionary based on the features of the text data (unstructured text and semi-structured text), and these features include: 1. High frequency: In a stop word dictionary, the words are generally vocabulary that appear very frequently in the text, such as "of", "yes", "in". 2. Functionality: Stop words often do not have actual semantic content and are functional vocabulary such as prepositions, conjunctions, and particles. 3. Redundancy: In text analysis, stop words can increase the dimensionality of the data, but their contribution to the analysis results is very small and can lead to a vector dimensionality explosion problem. 4. Context irrelevance: Words placed in a stop word dictionary are generally context-irrelevant, and their meanings do not change with changes in context. Then, defective text stop words are filtered by linking them with a stop word dictionary constructed using the Jieba participle package in Python. The outlier handling process specifically includes denoising, text normalization, and missing value handling, where denoising removes text unrelated to errors in unstructured and semi-structured text, text normalization converts unstructured and semi-structured text into a consistent format (for example, "pt" and "PT" must be uniformly replaced with uppercase letters to ensure data normativity), and missing value handling is performed using interpolation.

[0025] To address problems such as erroneous propagation, information redundancy, and inability to recognize overlapping relationships between entity pairs in conventional main production line models, in step S2, as shown in Figure 1, this embodiment constructs a collaborative entity relationship extraction model (Cascade and Residual Learning for Relational Triple Extraction, CasRel model) and improves upon conventional models by focusing on the design of sublayers. The collaborative entity relationship extraction model includes an encoding layer, a head entity recognition layer, and a tail entity and relationship recognition layer. Specifically, Coding layer: In step S2, the encoding layer in this embodiment employs a pre-trained BERT model, which is a multi-layer bidirectional Transformer-based language representation model used to extract feature information and learn depth representation by coordinating the context of each word. Specifically, h0=SW s +W p h α =Trans(h α-1 ), α∈[1,N] Here, S represents the thermocoding vector matrix of words in the input sentence, and W s This represents a word embedding matrix, W p represents the position embedding matrix, h α represents the hidden state vector, i.e., the context representation of the αth layer of the input statement, and N is the number of Transformer modules. Head entity recognition layer: The head entity recognition layer of the CasRel model directly encodes the output of the encoding layer, employs two identical binary classifiers to detect the start and end positions of the head entity, assigns a binary mark to each token (word), and uses it to mark whether the current mark corresponds to the start and end positions of the head entity. The specific operations for each token are as follows:

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0026] In step S2, the optimization objective is further included, and the final optimization objective function of the CasRel model is optimized directly on the triple level to maximize the likelihood function value. Specifically, the objective function is:

number

[0027] The linked entity relationship extraction model outputs the source knowledge triple text, performs knowledge fusion on the source knowledge triple text to obtain high-quality knowledge triple text. However, the source knowledge triple text generated during the knowledge extraction stage may contain problems such as incompleteness or lack of logical relationships, which can cause knowledge collisions during retrieval. Therefore, it is necessary to further integrate this extracted knowledge to ensure the accuracy and consistency of the knowledge, remove some uncertain information and contradictory knowledge from the source knowledge triple text, and improve the accuracy of the constructed knowledge graph. Specifically, this includes fusion dissolution and entity ambiguity resolution, and more specifically, Fusion dissolution: A common eutectic problem in the field of power knowledge is a synonym problem, where the same entity corresponds to multiple entity nodes. For example, two extracted words describe the same entity. To address this phenomenon, this embodiment employs a semantic similarity method. Using a conventional semantic similarity method, the cosine similarity of word vectors is calculated, and if it exceeds a certain threshold, they are considered synonyms. For each pair of synonyms, a clustering algorithm is used to group them, and the center point of each group can be selected as a representative. In other words, the similarity of word vectors between different synonyms is improved, and fusion and dissolution are performed. Entity ambiguity resolution is performed by using a deep learning method to link entities to the recognized entities and then performing eutectic and dissolution. Entity ambiguation resolution: To avoid the problem of information redundancy caused by storing entities in a database, entity ambiguity resolution is employed to resolve the ambiguity of multiple meanings for entities with the same name. For example, "Beijing" is an address attribute in contract metrics, but a unit name in service bidding team metrics. Therefore, a deep learning method is used to perform entity linking on recognized entities and realize data merging. Here, the deep learning method selected combines a conventional long- and short-time memory network with dual attention. The first attention mechanism emphasizes the information portion of the entity description using entity embeddings as attention vectors, and the second attention mechanism emphasizes the information portion of the context in which the entity is described using entity context as an attention vector. Finally, similarity and prior probability are linked to obtain the accurate entity.

[0028] In this embodiment, high-quality knowledge triple text is obtained through knowledge fusion, and Neo4j is used to store the high-quality knowledge triple text. Neo4j is an open-source graphics database management system developed based on the Java language, and it uses the Cypher declarative query language to manipulate and query the diagram database, making it convenient and easy to use. In this embodiment, connection to Neo4j is achieved by the py2neo library, and a transformer fault diagnosis knowledge graph is formed to store structured data.

[0029] Based on the formation of a knowledge graph, the present invention calls upon a large language model to complete the construction of a question-and-answer module. Compared to conventional knowledge graph question-and-answer systems, the question-and-answer module in this embodiment directly utilizes the powerful semantic comprehension capabilities of the large language model by employing a method of first generating a logical form and then searching for it. The construction of a question-and-answer module based on the knowledge graph of the large language model can be completed through the following process: (1) The user submits a question to the module. After information filtering, the question constitutes relevant expertise and hints in the knowledge base, which are then input into the expert question-and-answer module to obtain an answer. (2) The information extraction module extracts triples from the answer, matches them with the formed knowledge graph, and obtains relevant node data. (3) After these node data are selected by the user, they are similarly input into the expert question-and-answer module in the form of hints to obtain answers that extend the knowledge graph. This bidirectional alternation realizes a deep coupling between the large language model and the knowledge graph. It mainly involves three processes, which are information filtering, expert answers, and extraction transformation. Information filtering: [Table 1] In the power transformer field, the hint text P1 in Table 1 is input into a large language model to generate related problems. 70% of the problem data regarding power transformer failures originates from a conventional question-and-answer dataset, while 30% of the related problems are constructed using a method generated with the large language model. Furthermore, a BERT-based text filter is added to filter out non-specialized problems, limiting the range of problems the large model can answer. Using BERT's classifier word vector H, a simple classifier is created based on softmax, and the probability of predicting the type tag L is: P(L|H)=softmax(WH), Here, W is the parameter matrix for performing the classification task, and all parameters in BERT and W are fine-tuned by maximizing the log-probability of the correct tag, which is then modified to the probability of obtaining each tag using all connected layers, and finally the tag with the highest probability is selected as the classification result. By filtering the information, if Q is the set of all problems that the large language model can input, R is the set of problems that the large language model can answer in a particular field, and D is the set of problems that it can generate expert answers for, then obviously Q > R > D. Restricting with the fine-tuning method makes R → D, weakening the model's ability to answer. However, by filtering in the form of Q → R, we can ensure as much as possible that the queried problems are within the range of R, and although some non-R data enters the large language model, the expert-extended question-and-answer system designed herein still retains a certain degree of general capability and can answer non-R problems without expert verification. Expert answer: As shown in Figure 3, this embodiment employs the LangChain + LLM (Large Language Model) method to generate expert answers. First, it obtains the failure problem text, inputs the failure problem text into the LangChain model, searches for expertise related to the problem from the knowledge base based on LangChain, outputs the expertise, then constructs hint text P3 using the expertise and failure problem text, inputs it into the large language model, and outputs the final expert failure diagnosis answer. Extraction and transformation: To alternately combine the expert fault diagnosis answers output from the large language model with the structured knowledge of the knowledge graph, this embodiment is completed by an extractive transformation. The extractive transformation includes information extraction and readability of the structured text, where information extraction enhances the information extraction capabilities of the large language model using the P-tuning v2 fine-tuning method, embedding trainable attention keys and values ​​as prefixes in each layer of the large language model, and the original key vector K∈R l×d and value vector V∈R l×d Trainable vector P k Give P v These are connected to K and V respectively, and the calculation of the attention mechanism head is,

number

[0030] The readability of the structured text is achieved by converting nodes in high-quality knowledge triple text into hint text and inputting it into the large language model to output the final fault diagnosis answer. Specifically, as shown in Figure 4, for the readability of the structured data, a deep coupling between the large language model and the knowledge graph is achieved by converting the relevant nodes of the knowledge graph into hint text P2, and then inputting P2 into the large language model to obtain an extended natural language answer from the knowledge graph. The specific method for generating question-and-answer data in the LLM (Large Language Model) involves inputting the relevant user problem data and the API_KEY of the large language model, creating a link with the LLM interface using the API_KEY, selecting the user problem data, generating hints based on the selected data, accessing the LLM interface by establishing the link, inputting hints, and generating a response. All question-and-answer data is extracted from the LLM generation results, compiled into the final question-and-answer data, and the final fault diagnosis answer is output.

[0031] Example 2 In this embodiment, a transformer fault diagnosis system based on the fusion of a knowledge graph and a large language model is provided. A text acquisition module configured to acquire unstructured text for transformer operation and maintenance and semi-structured text for transformer failures, and to preprocess the acquired unstructured and semi-structured text, A knowledge graph and question-and-answer model construction module configured to construct a collaborative entity relationship extraction model and a question-and-answer model, which performs knowledge extraction on pre-processed unstructured and semi-structured text, outputs the original knowledge triple text, performs knowledge fusion on the original knowledge triple text, and is used to obtain high-quality knowledge triple text, wherein the collaborative entity relationship extraction model includes an encoding layer, a head entity recognition layer, and a tail entity and relationship recognition layer, and the question-and-answer model includes a LangChain model and a large language model, and the knowledge graph and question-and-answer model construction module takes the output of the LangChain as input to the large language model. It includes a fault diagnosis answer output module configured to acquire fault problem text, input the fault problem text into a LangChain model, output expertise, input the expertise and fault problem text into a large language model, output expert fault diagnosis answers, alternately combine the expert fault diagnosis answers and high-quality knowledge triple text, and output a final fault diagnosis answer.

[0032] The foregoing describes only preferred embodiments of the present invention and is not intended to limit it. Those skilled in the art will know that various modifications and changes are possible to the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention must be within the scope of protection of the present invention.

Claims

1. A computer-executable transformer fault diagnosis method based on the fusion of knowledge graphs and large language models, The method involves obtaining unstructured text for transformer operation and maintenance and semi-structured text for transformer failures, and pre-processing the obtained unstructured and semi-structured texts, wherein the pre-processing includes stop word filtering and anomaly processing, where the stop word filtering method involves constructing a stop word dictionary based on the unstructured and semi-structured texts and filtering stop words in the unstructured and semi-structured texts using the stop word dictionary; the anomaly processing includes noise reduction, text normalization, and missing value processing, where noise reduction involves removing text unrelated to errors in the unstructured and semi-structured texts; text normalization involves converting the unstructured and semi-structured texts into a consistent format; and missing value processing is performed using an interpolation method. A collaborative entity relationship extraction model and a question-and-answer module are constructed. The collaborative entity relationship extraction model extracts knowledge from pre-processed unstructured and semi-structured text, outputs the original knowledge triple text, and performs knowledge fusion on the original knowledge triple text to obtain high-quality knowledge triple text. Here, the collaborative entity relationship extraction model includes an encoding layer, a head entity recognition layer, and a tail entity and relationship recognition layer, with the start position threshold of the head entity and tail entity set to 0.6 and the end position threshold set to 0.

5. The question-and-answer module includes a LanGChain model and a large language model, with the output of the LanGChain being used as input to the large language model. The specific method for performing knowledge fusion on the aforementioned original knowledge triple text is fusion elimination and entity ambiguity resolution. Fusion elimination employs a semantic similarity method to calculate the cosine similarity of word vectors; words exceeding a certain threshold are considered synonyms. For each pair of synonyms, a clustering algorithm is used to group them, and the center point of each group is selected as a representative. Entity ambiguity resolution employs a deep learning method to perform entity linking on the recognized entities. Here, the deep learning method selected combines a conventional long- and short-time memory network with dual attention. The first attention mechanism emphasizes the information portion of the entity description using entity embeddings as attention vectors, the second attention mechanism emphasizes the information portion of the context of the entity using entity context as attention vectors, and finally, similarity and prior probability are linked to obtain the accurate entity. A transformer fault diagnosis method characterized by including obtaining fault problem text, inputting the fault problem text into a LangChain model, outputting expert knowledge, inputting the expert knowledge and fault problem text into a large language model, outputting an expert fault diagnosis answer, alternately combining the expert fault diagnosis answer and a high-quality knowledge triple text, and outputting a final fault diagnosis answer.

2. The aforementioned coding layer employs the BERT model, which is a multi-layer bidirectional transformer-based language representation model used to learn depth representation by extracting feature information and coordinating the context of each word. Specifically, h ０ =SW ｓ +W ｐ h α =Trans(h α－１ ), α∈[1,N], Here, S represents the thermocoding vector matrix of words in the input sentence, and W Ｓ This represents a word embedding matrix, W Ｐ represents the position embedding matrix, h α The transformer fault diagnosis method based on the fusion of a knowledge graph and a large language model according to claim 1, characterized in that represents a hidden state vector, i.e., the α-th layer context representation of the input sentence, and N is the number of Transformer modules.

3. The head entity recognition layer employs two identical binary classifiers to detect the start and end positions of the head entity, assigns a binary mark to each token, and is used to mark whether the current mark corresponds to the start and end positions of the head entity. The specific operations performed on each token are as follows: [Number 28] As stated above, Here, [Number 29] respectively represent the probabilities of marking the i-th token of the input text as the start position and the end position of the head entity, x ｉ is the encoded representation of the i-th token in the input sequence, i.e., x ｉ = h Ｎ [i], where W represents the weight matrix, b represents the bias term, and σ represents the sigmoid activation function, The transformer fault diagnosis method based on the fusion of the knowledge graph and the large language model according to claim 1, characterized in that.

4. The specific operations performed on each token in the tail entity and relationship recognition layer are as follows: [Number 30] As stated above, Here, [Number 31] These represent the probabilities of recognizing the i-th token in the input sequence as the start and end positions of the tail entity, respectively. [Number 32] represents the coded representation vector of the k-th entity detected in the previous layer, and x ｉ The transformer fault diagnosis method based on the fusion of a knowledge graph and a large language model according to claim 1, characterized in that is an encoded representation of the i-th token in the input sequence.

5. The aforementioned expert fault diagnosis answers and high-quality knowledge triple text are alternately combined by extraction and transformation, which specifically includes information extraction and structural text readability, and the information extraction involves embedding attention keys and values ​​as prefixes in each layer of the large language model, i.e., [Number 33] And here, K∈R ｌ×ｄ V represents the given original key vector, where V∈R ｌ×ｄ P represents the given original value vector. ｋ and P ｖ (i) represents the portion of the vector corresponding to the i-th attention head, The transformer fault diagnosis method based on the fusion of a knowledge graph and a large language model according to claim 1, characterized in that the readability of the structured text is achieved by converting nodes in a high-quality knowledge triple text into hint text and inputting it into a large language model to output a final fault diagnosis answer.

6. A transformer fault diagnosis system based on the fusion of a knowledge graph and a large language model employs a transformer fault diagnosis method based on the fusion of a knowledge graph and a large language model as described in any one of claims 1 to 5. A text acquisition module configured to acquire unstructured text for transformer operation and maintenance and semi-structured text for transformer failures, and to preprocess the acquired unstructured and semi-structured text, A knowledge graph and question-and-answer model construction module configured to construct a collaborative entity relationship extraction model and a question-and-answer model, wherein the collaborative entity relationship extraction model performs knowledge extraction on pre-processed unstructured and semi-structured texts, outputs the original knowledge triple text, performs knowledge fusion on the original knowledge triple text, and is used to obtain high-quality knowledge triple text, wherein the collaborative entity relationship extraction model includes an encoding layer, a head entity recognition layer, and a tail entity and relationship recognition layer, and the question-and-answer model includes a LanGChain model and a large language model, and the output of the LanGChain is used as input to the large language model. A method comprising a fault diagnosis answer output module configured to acquire fault problem text, input the fault problem text into a LangChain model, output expert knowledge, input the expert knowledge and fault problem text into a large language model, output an expert fault diagnosis answer, alternately combine the expert fault diagnosis answer and high-quality knowledge triple text, and output a final fault diagnosis answer.

7. A computer-readable storage medium in which a program is stored, characterized in that, when the program is executed by a processor, it realizes the steps in the transformer fault diagnosis method based on the fusion of a knowledge graph and a large language model as described in any one of claims 1 to 5.

8. An electronic device comprising memory, a processor, and a program stored in memory and operable by the processor, wherein the processor, when executing the program, implements the steps in the transformer fault diagnosis method based on the fusion of a knowledge graph and a large language model as described in any one of claims 1 to 5.

Images