A big data-based power transformer fault intelligent diagnosis method
By using a big data-based intelligent diagnostic method, combining key parameters of power transformers and external data, and utilizing word embedding, attention mechanisms, and large language models, the problem of data quality and comprehensive consideration of factors in power transformer fault diagnosis is solved, achieving higher diagnostic accuracy and model learning capabilities.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHONGQING UNIV OF POSTS & TELECOMM
- Filing Date
- 2024-04-23
- Publication Date
- 2026-07-31
AI Technical Summary
Existing power transformer fault diagnosis technologies face issues with data quality and reliability, and struggle to comprehensively consider electrical, mechanical, and environmental factors for effective diagnosis.
We employ a big data-based intelligent diagnostic method. By acquiring key parameters and external data of power transformers, we utilize word embedding, positional encoding, attention mechanisms, and neural network processing to process the data. We combine large language models and SVM models for diagnosis, retrieve the most similar documents to the query documents, and perform information gain and keyword extraction to improve the accuracy and robustness of the diagnosis.
It significantly improves the accuracy of power transformer fault diagnosis and the learning and generalization capabilities of the model, enhances the relevance of data and the accuracy of reconstructed indexes, and improves the accuracy of diagnostic results.
Smart Images
Figure CN118410165B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of deep learning and large models, and in particular to an intelligent fault diagnosis method for power transformers based on big data. Background Technology
[0002] The current development of power transformer fault diagnosis exhibits several trends. First, continuous advancements in sensor technology have enabled more accurate and comprehensive monitoring of various transformer parameters. Second, the rapid development of large-scale modeling and artificial intelligence technologies has opened up new avenues for intelligent fault diagnosis of power transformers. Simultaneously, model-based diagnostic methods are widely used, combining physical models or machine learning models for fault diagnosis. Furthermore, online monitoring and predictive maintenance technologies for transformers are becoming increasingly sophisticated, enabling prediction and maintenance before faults occur. Finally, multimodal data fusion technology is widely adopted, comprehensively considering data acquired from different sensors to improve the comprehensiveness and accuracy of diagnosis. These trends collectively drive the continuous innovation and development of power transformer fault diagnosis technology.
[0003] While significant progress has been made in power transformer fault diagnosis technology, several key challenges remain. First, data quality and reliability remain a challenge, as sensor errors, communication failures, and other factors can affect data accuracy. Second, power transformer faults are influenced by a variety of factors, including electrical, mechanical, and environmental factors; effectively diagnosing these faults by comprehensively considering all these factors is a difficult task. Summary of the Invention
[0004] To improve the reliability, practicality, and accuracy of power transformer diagnostic technology, this invention proposes a big data-based intelligent fault diagnosis method for power transformers, which specifically includes the following steps:
[0005] Obtain keyword parameters for power transformers and corresponding external data, and construct a query document based on the obtained data;
[0006] The query data of the power transformer to be detected is converted into word embedding vectors, and the word embedding vectors are positionally encoded to obtain the corresponding vector representations;
[0007] The vector representation of words is processed based on the attention mechanism to obtain the attention vector of each word;
[0008] The attention vector is processed by a neural network to obtain the representation vector of each word in the query document through nonlinear transformation.
[0009] Based on the representation vector of each word in the query document, retrieve the K documents that are most similar to the query document;
[0010] The retrieved K documents and the query document are input into a large language model for diagnosis. The large language model outputs the diagnosis results, and the diagnosis results of the large language model are incorporated into the external data.
[0011] Furthermore, when retrieving the K most similar documents to the query document, an SVM model is pre-trained. The representation vector of each word in the query document is input into the pre-trained SVM model to select a retrieval strategy. The retrieval strategies include similarity retrieval and keyword retrieval. Similarity retrieval is to retrieve the K most similar documents to the query document by calculating the similarity between two documents. Keyword retrieval is to retrieve the K most similar documents to the query document based on whether the keywords appear and the number of times the keywords appear after obtaining the keywords in the document.
[0012] Furthermore, in the process of similarity retrieval, the calculation of the similarity between two documents includes:
[0013] Calculate the vector representation of the document using the representation vector of each word in the document. The vector representation of a document x with n words is: Vector representation of the i-th word in document x The calculation process is as follows:
[0014]
[0015] The similarity between two documents is calculated using their vector representations, as follows:
[0016]
[0017] in, This represents the i-th word in document x. Frequency in document x This represents the i-th word in document x. In the frequency of inverse documents; Similarity(Vector) x Vector y Vector represents the similarity between document x and document y. x Vector is a vector representation of document x. y Let y be the vector representation of document y, and ||·|| denotes finding the norm of the vector.
[0018] Furthermore, the keyword retrieval process includes the following steps in extracting document keywords:
[0019] Initialize the weight of each word in the document, and iteratively update the weights until the weights no longer change;
[0020] Words with a weight greater than a set threshold are selected as keywords. The weight update process is represented as follows:
[0021]
[0022] Among them, w i d represents the weight of word i in the document, d represents the adjustment coefficient; ||·|| represents the norm; n is the number of words in the document.
[0023] Furthermore, after obtaining the representation vector of each word in the query document, the information gain of each word vector is calculated, and the k words with the largest information gains are selected as the retrieval targets. The information gain of word vector representation A relative to document D is expressed as:
[0024] Gain(D,A)=H(D)-H(D|A)
[0025] Where Gain(D,A) is the information gain of word vector representation A relative to document D; H(D) represents the information entropy of document D; and H(D|A) represents the information entropy of document D under word vector representation A.
[0026] Furthermore, the process of converting the query data of the power transformer to be detected into word embedding vectors and performing positional encoding on the word embedding vectors to obtain the corresponding vector representation includes the following steps:
[0027] The query document is input into a pre-trained model to encode the words in the document, thus obtaining the hidden state of each word;
[0028] The embedding vector of a word is calculated based on its hidden state. The calculation process is as follows:
[0029] E(w) = w0 × [CLS] + w1 × h w
[0030] The positional information of words in the document is converted into a vector representation, and the positional information of words is concatenated with the embedding vector to form the vector representation of the words;
[0031] Where E(w) is the word embedding vector of word w; w0 and w1 are model parameters; [CLS] is the symbol marking the beginning of a sentence; h w Let w be the hidden state of word w in the pre-trained model.
[0032] Furthermore, the process of converting word position information into vector representation includes:
[0033]
[0034]
[0035] Among them, POSE (pos,i) The position p represents the positional encoding of the word in the i-th dimension throughout the document; d represents the word vector dimension.
[0036] Furthermore, processing the word vector representations based on the attention mechanism to obtain the attention vector for each word includes the following steps:
[0037] First, the linear transformation matrix W needs to be randomly initialized. q W k W v Then calculate the query vector Q, the key vector K, and the value vector V, that is:
[0038] Q = E × W q
[0039] K = E × W k
[0040] V = E × W v
[0041] The attention score corresponding to the vector representation E of a word is expressed as:
[0042]
[0043] Where, d k This represents the dimension of the key vector K.
[0044] Furthermore, the process of obtaining the representation vector of each word in the query document by performing nonlinear transformation on the attention vector through a neural network includes:
[0045] y = σ(Attention·W) f )
[0046] Where y is the final vector representation of the query result; W f σ is the linear transformation matrix; σ represents the activation function.
[0047] Compared with the prior art, the present invention has the following beneficial effects:
[0048] 1. This invention creatively adds relevant external data as a retrieval method based on the key parameters of the transformer, which is beneficial to further improve the robustness of the large language model and significantly improves the diagnostic accuracy;
[0049] 2. This invention uses a self-attention network for external data related to transformers and their faults. After vectorizing the external data, the correlation between the vectors is enhanced, which improves the accuracy of subsequent index reconstruction and helps to improve the accuracy of the final result.
[0050] 3. After the LLM provides the corresponding results, this invention feeds the results back to the Query Transformer's embedding, which helps to continuously improve the model's learning and generalization capabilities, and also helps to improve the accuracy of diagnostic results. Attached Figure Description
[0051] Figure 1 This is a flowchart of an intelligent fault diagnosis method for power transformers based on big data, according to an embodiment of the present invention.
[0052] Figure 2 This is a flowchart illustrating the specific implementation of a big data-based intelligent fault diagnosis method for power transformers according to an embodiment of the present invention. Detailed Implementation
[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0054] This invention proposes an intelligent fault diagnosis method for power transformers based on big data, which specifically includes the following steps:
[0055] Obtain keyword parameters for power transformers and corresponding external data, and construct a query document based on the obtained data;
[0056] The query data of the power transformer to be detected is converted into word embedding vectors, and the word embedding vectors are positionally encoded to obtain the corresponding vector representations;
[0057] The vector representation of words is processed based on the attention mechanism to obtain the attention vector of each word;
[0058] The attention vector is processed by a neural network to obtain the representation vector of each word in the query document through nonlinear transformation.
[0059] Based on the representation vector of each word in the query document, retrieve the K documents that are most similar to the query document;
[0060] The retrieved K documents and the query document are input into a large language model for diagnosis. The large language model outputs the diagnosis results, and the diagnosis results of the large language model are incorporated into the external data.
[0061] This embodiment proposes a specific implementation process for an intelligent fault diagnosis method for power transformers based on big data, such as... Figure 1 Specifically, it includes the following steps:
[0062] 101. High-precision sensors are used to monitor key transformer parameters in real time. The data is then preprocessed, including removing duplicate values and deleting NULL values. Finally, a query is constructed using the following format to obtain the result QR:
[0063] "You now have a large model for predicting transformer failures. The transformer parameters you have collected are as follows: Parameter 1 Name: Parameter Value;
[0064] Parameter 2 name: Parameter value;
[0065] ...
[0066] Parameter n name: Parameter value.
[0067] Based on the parameters and their values above, please determine whether the transformer is currently malfunctioning.
[0068] Initially, if no external data is collected, diagnosis can be performed without adding external data.
[0069] 102. Collect external data related to transformers and their faults, including but not limited to personalized data, domain-specific context, semantically similar documents, and non-content signals. Import the query corpus into the Query Transformer for processing. This processing converts the query into a vector representation for comparison with the vector representation of the document. The key steps are as follows:
[0070] Step 1: Word Embedding. Here, the words in the input sequence are converted into word embedding vectors. The specific word embedding process is as follows: First, the sentence is imported into a pre-trained model, which encodes the sentence; then, the hidden states h of words w in the sentence are obtained. w and h w The vector is concatenated with [CLS]; finally, the concatenated vector is input into a linear layer for non-linear transformation, and the result is the word embedding vector of word w. The formula is expressed as follows:
[0071] E(w) = w0 × [CLS] + w1 × h w
[0072] In the formula above, E(w) is the word embedding vector of word w; w0 and w1 are model parameters; [CLS] is a special symbol indicating the start of a sentence; h w It represents the hidden state of word w in the pre-trained model.
[0073] Step 2: Positional Encoding. Positional encoding converts the word's positional information into a vector representation, which is then added to the word embedding vector to form the final input vector. The positional encoding is expressed by the following formula:
[0074]
[0075]
[0076] In the formula above, POSE (pos,i) The position p represents the positional encoding of the word at position p in the i-th dimension of the entire sentence; p represents the relative position of the word in the entire sentence; i represents the index value of the entire dimension; and d represents the dimension of the model.
[0077] Through the above two steps, the vector representation of word w is: E(w) + POSE(p,2i) + POSE(p,2i+1).
[0078] Step 3: Self-Attention Mechanism. A self-attention mechanism is used to calculate the similarity and relevance between each word and other words, and then weighted according to the relevance to obtain the final output. First, the linear transformation matrix W needs to be randomly initialized. q W k W v Then calculate the three vectors Q, K, and V, using the following formula:
[0079] Q = E × W q
[0080] K = E × W k
[0081] V = E × W v
[0082] In the formula above, E represents the result obtained from steps one and two; Q represents the semantic information of the current word, which is used to calculate the relevance of each word to other words; K represents the semantic information of all words, which is used to represent the influence of other words on the current word; and V represents the semantic information of all words, which is used to represent the final output of each word.
[0083] Finally, the final attention vector is calculated using the following formula, where the values represent the relative importance of each word to other words:
[0084]
[0085] In the formula above, Q, K, and V are the results obtained from the calculation, and d k This represents the dimension of K.
[0086] Step 4: Feedforward Neural Network. Finally, the output weights of the self-attention mechanism are input into the neural network for nonlinear transformation to improve the model's expressive power. The calculation formula is as follows:
[0087] y = σ(Attention·W) f )
[0088] In the formula above, σ represents the activation function, Attention is the attention vector obtained in step three, and W... f It is a linear transformation matrix, which can usually be obtained by initializing with random numbers, and its parameters need to be updated during backpropagation.
[0089] Step 5: Output the result y.
[0090] 103. Import the results of Query Transformer into Query Routing and DataStore respectively. Query Routing is used to route queries to the corresponding retrieval modules. That is, this module's function is to select the appropriate retrieval method for the document based on the vector representation of the words output by Query Transformer. The calculation formula for Query Routing in this embodiment is as follows:
[0091] R = Q(q, S)
[0092] In the formula above, R is the routing strategy, Q is the query analysis function, q represents the corresponding query, and S represents the set of retrieval modules. This set of retrieval modules includes various types, such as text retrieval modules, image retrieval modules, and video retrieval modules. The query analysis function Q can be used to train a query routing model using an SVM. After training, the model can predict the query type based on the given query.
[0093] 104. Based on the Query Routing results, create Vector Index and Summary Index structures and import them into DataStore. The Vector Index and Summary Index structures represent the Query Routing results from different dimensions. The Vector Index calculates the similarity between documents, while the Summary Index quickly finds documents related to the query. The calculation process for the Vector Index is as follows:
[0094] Vectorized representation: Vector = TF(w i )×IDF(w i ), where Vector represents the vector of the document, TF(w i ) indicates the word w i Frequency in the document, IDF(w i ) indicates the word w i Frequency of reverse document.
[0095] Similarity calculation: Where V1 and V2 represent two vectors respectively.
[0096] In Summary Index, summaries can be built by extracting key sentences or words. First, the weight of each word or sentence is calculated, and the formula is as follows:
[0097]
[0098] In the formula above, w i This represents the weight of word i, and d represents the adjustment coefficient, which can usually be set to 0.85. ∑w represents the sum of the similarities between word i and all other words j in the sentence. j This represents the sum of the weights of all words in the sentence. The calculation process is as follows:
[0099] Step 1: Initialize the weights of all words to 1;
[0100] Step 2: Iteratively calculate the weight of each word using the above formula;
[0101] Step 3: Repeat step 2 until the weight of each word no longer changes.
[0102] After obtaining the weight of each word or sentence, the keywords of the corpus are extracted using the following extraction formula:
[0103] Ext(Document) = [Sent i |Sent i ∈Document,Score(Sent i )≥Threshold]
[0104] In the above formula, Ext(Document) represents extracting key sentences from the document Document, and Sent i Score(Sent) represents the i-th sentence. i () represents the score of the i-th sentence, and Threshold represents the score threshold, which can usually be set to 0.80. The above describes keyword extraction for paragraphs; the calculation method for keyword extraction for sentences is similar.
[0105] 105. Re-sort the data in the DataStore according to the feature reweighting method. The feature reweighting method is based on information gain and the calculation formula is as follows:
[0106] Gain(D,A)=H(D)-H(D|A)
[0107] In the formula above, Gain(D,A) represents the information gain of the current feature A with respect to the entire dataset D, H(D) represents the information entropy of the entire dataset D, and H(D|A) represents the conditional entropy of the dataset D under the condition that the current feature is A. After calculating the information gain of each feature with respect to the entire dataset, the data is reordered according to the magnitude of the information gain, ensuring that the feature with the largest information gain is listed first.
[0108] The calculation of the information entropy H(D) of document D includes:
[0109]
[0110] Where, x i P(x) represents a word in document D. i ) represents x i The probability of a word appearing in article D can be calculated using word x. i The number of occurrences divided by the total number of words in the article; n represents the total number of words in the article.
[0111] The calculation of the information entropy H(D|A) of document D under word vector representation A includes:
[0112] H(D|A)=P(A)·H(D)
[0113] Where P(A) represents the probability of word vector A appearing in the entire article, and H(D) represents the information entropy of article D. The sorted data is then re-searched to find the K data points with the highest similarity to the Query Transformer output. The value of K can be chosen based on the size of the external dataset, typically set to 1 / 20 of the dataset size.
[0114] 106. Merge the search results with the previous QR results and import them into the LLM. The merging format can be as follows:
[0115] "You now have a large model for predicting transformer failures. The remaining relevant reference materials collected are as follows:"
[0116] Insert search results here.
[0117] The transformer parameters collected are as follows:
[0118] Parameter 1 name: Parameter value;
[0119] Parameter 2 name: Parameter value;
[0120] ...
[0121] Parameter n name: Parameter value.
[0122] Based on the parameters and their values above, please determine whether the transformer is currently malfunctioning.
[0123] 107. LLM outputs the results and feeds them back to the Query Transformer module. The feedback can be constructed by combining the 7 fused results with the LLM's response as newly added external data to be imported into the corresponding module.
[0124] like Figure 2 This embodiment also provides a specific implementation process for a big data-based intelligent fault diagnosis method for power transformers, which includes the following steps:
[0125] External data related to the transformer and its faults are input into the Query Transformer module. The transformer parameter data includes personalized data, specific domain context, semantically similar documents, non-content signals, etc., as query data. In this embodiment, the result of the previous diagnosis is used as external data.
[0126] In the Query Transformer module, external data is supplemented into the query data through Feedback From LLM, then the embedded representation of the query data is obtained, position encoding is performed based on the embedded representation, and then self-attention operation is performed on the position-encoded query data to obtain the output of the Query Transformer module.
[0127] The output of the Query Transformer module serves as the input to Query Routing. Query Routing can select a pre-trained SVM model for construction. The function of Query Routing is to select a search method for the query data based on the output of the Query Transformer module. In this embodiment, two search methods are included: Vector Index and Summary Index. These two search methods have been introduced above and will not be repeated here.
[0128] After retrieving the K most similar texts based on the search results, the retrieved texts and query data are input into the DataStore. In the DataStore, the query data is weighted and the most similar texts are reranked.
[0129] The rearranged data is retrieved (Retrieved Context), then inserted into the corresponding position in the constructed query data. The entire processed query data is then input into a pre-trained Large Language Model (LLM). The pre-trained Large Language Model outputs diagnostic results based on the input query data and uses these diagnostic results as external data for the next diagnosis.
[0130] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A method for intelligent fault diagnosis of power transformers based on big data, characterized in that, Specifically, the following steps are included: Obtain keyword parameters for power transformers and corresponding external data, and construct a query document based on the obtained data; The query data of the power transformer to be detected is converted into word embedding vectors, and the word embedding vectors are positionally encoded to obtain the corresponding vector representations; The vector representation of words is processed based on the attention mechanism to obtain the attention vector of each word; The attention vector is processed by a neural network to obtain the representation vector of each word in the query document through nonlinear transformation. Based on the representation vector of each word in the query document, retrieve the K documents most similar to the query document. When retrieving the K documents most similar to the query document, a pre-trained SVM model is used. The representation vector of each word in the query document is input into the pre-trained SVM model to select a retrieval strategy. The retrieval strategy includes similarity retrieval and keyword retrieval. Similarity retrieval is to retrieve the K documents most similar to the query document by calculating the similarity between two documents. Keyword retrieval is to retrieve the K documents most similar to the query document after obtaining the keywords of the document and then retrieve the K documents most similar to the query document based on whether the keywords appear and the number of times the keywords appear. In the process of similarity retrieval, the calculation of the similarity between two documents includes: Calculate the vector representation of the document using the representation vector of each word in the document. The vector representation of a document x with n words is: The vector representation of the i-th word in document x The calculation process is as follows: The similarity between two documents is calculated using their vector representations, as follows: in, This represents the i-th word in document x. Frequency in document x This represents the i-th word in document x. At the frequency of reverse document; This represents the similarity between document x and document y. Let x be the vector representation of document x. Let y be the vector representation of document y. This indicates finding the norm of a vector; The process of extracting keywords from documents during keyword retrieval includes the following steps: Initialize the weight of each word in the document, and iteratively update the weights until the weights no longer change; Words with a weight greater than a set threshold are selected as keywords. The weight update process is represented as follows: in, Indicates words in the document The weight, Indicates the adjustment coefficient; This indicates the norm; n is the number of words in the document; The retrieved K documents and the query document are input into a large language model for diagnosis. The large language model outputs the diagnosis results, and the diagnosis results of the large language model are incorporated into the external data.
2. The intelligent fault diagnosis method for power transformers based on big data according to claim 1, characterized in that, The process of converting the query data of the power transformer to be detected into a word embedding vector and then performing positional encoding on the word embedding vector to obtain the corresponding vector representation includes the following steps: The query document is input into a pre-trained model to encode the words in the document, thus obtaining the hidden state of each word; The embedding vector of a word is calculated based on its hidden state. The calculation process is as follows: The positional information of words in the document is converted into a vector representation, and the positional information of words is concatenated with the embedding vector to form the vector representation of the words; in, For words Word embedding vectors; and These are model parameters; A symbol used to mark the beginning of a sentence; For words Hidden state in a pre-trained model.
3. The intelligent fault diagnosis method for power transformers based on big data according to claim 2, characterized in that, The process of converting word position information into vector representation includes: in, Indicates position The positional encoding of a word in the i-th dimension throughout the entire document; This represents the dimension of the word vector.
4. The intelligent fault diagnosis method for power transformers based on big data according to claim 1, characterized in that, Processing the vector representations of words based on the attention mechanism to obtain the attention vector for each word includes the following steps: First, the linear transformation matrix needs to be randomly initialized. , , Then calculate the query vector Q, the key vector K, and the value vector V, that is: The attention score corresponding to the vector representation E of a word is expressed as: in, This represents the dimension of the key vector K.
5. The intelligent fault diagnosis method for power transformers based on big data according to claim 4, characterized in that, The process of obtaining the representation vector of each word in the query document by performing non-linear transformation on the attention vector through a neural network includes: Where y is the final vector representation of the query result; It is a linear transformation matrix; This represents the activation function.