Transformer fault prediction method and system

Abstract

The invention discloses a transformer fault prediction method and system, and belongs to the field of industrial fault diagnosis, and the method comprises the steps: obtaining classification feature data based on pre-obtained data of dissolved gas in oil; constructing a fault diagnosis model comprising an input coding network, a Top-K sparse self-attention network and a classification output network; inputting the classification feature data into a fault diagnosis model, and performing data preprocessing on the classification feature data based on an input coding network to obtain preprocessed coding features; performing Top-K sparse self-attention mechanism processing on the preprocessed coding features based on a Top-K sparse self-attention network to obtain sparse coding features; and performing multi-head self-attention processing and full-connection classification processing on the sparse coding features based on a classification output network to obtain a transformer fault prediction result. According to the transformer fault prediction method and system disclosed by the invention, efficient and accurate transformer fault prediction is realized.

Description

Technical Field

[0001] This invention relates to the field of industrial fault diagnosis, and in particular to a method and system for predicting transformer faults. Background Technology

[0002] Power transformers are core equipment in power systems, and their safe and stable operation is crucial to the reliability of the power grid. Transformer fault types mainly include partial discharge (PD), low-energy discharge (D1), high-energy discharge (D2), low-temperature overheating (T1), medium-temperature overheating (T2), high-temperature overheating (T3), and normal conditions. Timely and accurate prediction of transformer fault types is of great significance for preventative maintenance and reducing the risk of power outages. Dissolved gas analysis (DGA) is one of the most commonly used methods for transformer fault diagnosis. Traditional DGA diagnostic methods include the three-ratio method, Rogers ratio method, and David's triangle method. These methods determine the fault type by analyzing the ratios of characteristic gases in the transformer oil. For example, the three-ratio method... , , The combination of three ratio codes to determine the fault type has the advantages of clear rules and ease of understanding.

[0003] Meanwhile, with the development of deep learning technology, fault diagnosis methods based on neural networks have gradually become a research hotspot. Models such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Long Short-Term Memory (LSTMs) are widely used in transformer fault diagnosis. In recent years, the Transformer model, due to its powerful sequence modeling and global feature capture capabilities, has achieved significant results in natural language processing, computer vision, and other fields, and has been gradually introduced into the field of industrial fault diagnosis. However, traditional rule-based diagnosis methods are limited by ratio data calculation, resulting in technical problems such as ambiguous boundary conditions and inability to handle boundary samples. Furthermore, the diagnostic accuracy is limited by fixed rule thresholds, making it difficult to adapt to complex and changing actual working conditions. While existing deep learning methods can automatically learn features, they suffer from the following problems: first, relying on purely data-driven methods ignores domain expert knowledge, leading to insufficient generalization ability in small sample situations; second, they lack interpretability, making it difficult to meet the power system's requirements for transparency in fault diagnosis decisions. In summary, existing methods struggle to effectively integrate the advantages of expert rule knowledge and data-driven models, failing to achieve knowledge-guided intelligent diagnosis. Summary of the Invention

[0004] This invention provides a transformer fault prediction method and system that can solve the technical problems of existing technologies that are limited by fixed rule thresholds and are difficult to adapt to complex and ever-changing actual working conditions. While maintaining model performance, it significantly improves computational efficiency and achieves efficient and accurate transformer fault prediction.

[0005] This invention provides a transformer fault prediction method, comprising: Construct a fault diagnosis model that includes an input encoding network, a Top-K sparse self-attention network, and a classification output network; Classification feature data is obtained based on pre-acquired dissolved gas data in oil; The classification feature data is input into the fault diagnosis model to perform data preprocessing on the classification feature data based on the input coding network to obtain preprocessed coding features; The preprocessed encoded features are processed using the Top-K sparse self-attention network to obtain sparse encoded features. Based on the classification output network, multi-head self-attention processing and fully connected classification processing are performed on the sparse coding features to obtain transformer fault prediction results. The Top-K sparse self-attention mechanism includes the following processing: Based on the preprocessed encoded features, linear projection is performed to obtain multidimensional query vector, multidimensional key vector, and multidimensional value vector; A sparse attention matrix is ​​constructed based on the multidimensional query vector, multidimensional key vector, and multidimensional value vector. Sparse coding features are obtained by weighted aggregation based on the sparse attention matrix and multidimensional value vector.

[0006] This invention provides a transformer fault prediction method, constructing a fault diagnosis model comprising an input encoding network, a Top-K sparse self-attention network, and a classification output network. The method first performs linear projection on the preprocessed encoded features using a Top-K sparse self-attention mechanism, constructs a sparse attention matrix, and performs weighted aggregation to obtain sparse encoded features. By introducing the Top-K sparse self-attention mechanism, only the K key-value pairs with the highest attention scores are retained for computation, eliminating redundant low-attention key-value pair association calculations. This effectively reduces the computational complexity of standard self-attention mechanisms while focusing on capturing key feature associations, ensuring that the fault diagnosis model improves computational efficiency without sacrificing core feature extraction capabilities. Through input encoding, sparse attention processing, and classification output, computational efficiency is significantly improved while maintaining model performance, achieving efficient and accurate transformer fault prediction, and adapting to resource-constrained application scenarios.

[0007] Furthermore, the acquisition of classification feature data based on pre-acquired dissolved gas data in oil includes: Feature augmentation is performed based on pre-acquired dissolved gas data in oil to obtain a multidimensional experimental dataset; Interpretability analysis was performed based on the multidimensional experimental dataset to obtain the feature contribution distribution; Based on the aforementioned feature contribution distribution, preset feature contribution threshold, and preset three-ratio retention strategy, the multidimensional experimental dataset is simplified to obtain classification feature data.

[0008] In the above scheme, the present invention first expands the features of dissolved gas data in oil to obtain a multi-dimensional experimental dataset, then obtains the feature contribution distribution through interpretability analysis, and finally simplifies the features by combining a preset threshold and a three-ratio retention strategy. Through the preprocessing of multiple data, a multi-level and multi-dimensional feature system is constructed, providing rich feature support for fault prediction. Furthermore, interpretability analysis enables accurate identification of high-contribution features and redundant features. At the same time, a preset three-ratio retention strategy is set to force the retention of three-ratio features, ensuring that core features related to subsequent prior knowledge fusion are not omitted, providing high-quality input for subsequent model prediction, and improving the accuracy and stability of fault prediction.

[0009] Furthermore, the data preprocessing includes: Based on the classification feature data, a model hidden dimension mapping is performed to obtain the embedded feature sequence; Position encoding is performed based on the embedded feature sequence to obtain the encoded feature sequence; Multi-head self-attention processing is performed based on the encoded feature sequence to obtain multi-dimensional encoded features; Based on the multidimensional coding features, summation and standardization are performed to obtain preprocessed coding features.

[0010] In the above scheme, the original classification features are transformed into embedded feature sequences that the model can process efficiently through hidden dimension mapping. Positional encoding supplements the sequence association information of the features. Multi-head self-attention processing can capture the complex relationship between features. Summation and standardization stabilize the data distribution during model training and avoid gradient anomalies. This achieves systematic optimization of classification feature data, enabling the preprocessed encoded features to accurately reflect the correlation between dissolved gas data in oil and fault types.

[0011] Furthermore, the construction of the sparse attention matrix based on the multidimensional query vector, multidimensional key vector, and multidimensional value vector includes: The attention score matrix is ​​obtained based on the multidimensional query vector, the multidimensional key vector, and the preset attention head dimension. Based on the attention score matrix, a row-by-row maximum value index is performed to obtain the sparse attention matrix.

[0012] In the above scheme, when constructing the sparse attention matrix, the attention score matrix is ​​calculated first, and then key associations are filtered by the maximum value index of each row. The maximum value index of each row is used to accurately lock the key-value pairs that have the most influence on each query vector, and irrelevant feature associations are eliminated, reducing computational redundancy and making the sparse encoded features obtained by subsequent weighted aggregation more targeted. While reducing computational complexity, the effectiveness of feature extraction is ensured, thereby ensuring the accuracy of fault prediction results.

[0013] Furthermore, the fault diagnosis model also includes a prior knowledge fusion network. After the step of performing multi-head self-attention processing and fully connected classification processing on the sparse encoded features based on the classification output network to obtain the transformer fault prediction result, the model further includes: Based on the prior knowledge fusion network and classification feature data, three-ratio features are obtained, and based on the three-ratio features and the preset three-ratio method fault diagnosis rules, the rule prior probability distribution is obtained. The transformer fault prediction results and rule-based prior probability distributions are weighted and fused based on the prior knowledge fusion network to obtain the target fault diagnosis probability.

[0014] In the above scheme, the three-ratio feature is extracted through the prior knowledge fusion network, and the rule prior probability distribution is obtained by combining the preset three-ratio method fault diagnosis rule. Then, it is weighted and fused with the model prediction result to achieve the complementary advantages of data-driven and knowledge-driven approaches. While leveraging the model's ability to learn complex data patterns, the reliability of the prediction result is improved by using expert rules, so that the target fault diagnosis probability has both the accuracy supported by data and the robustness guaranteed by knowledge.

[0015] Furthermore, after the steps of obtaining the three-ratio features based on the prior knowledge fusion network and classification feature data, and obtaining the rule prior probability distribution based on the three-ratio features and the preset three-ratio method fault diagnosis rules, the method further includes: The maximum probability distribution value of the transformer fault prediction result is determined to be the sample confidence level; The comparison is based on the sample confidence level and a preset confidence threshold: When the preset unfamiliar sample conditions are met, the prior probability distribution of the rule is used as the target fault diagnosis probability. When the preset conditions are met, the transformer fault prediction results and rule-based prior probability distributions are weighted and fused based on the prior knowledge fusion network to obtain the sample fault diagnosis probability, and the sample fault diagnosis probability is used as the target fault diagnosis probability.

[0016] In the above scheme, a confidence backoff strategy is introduced for out-of-distribution samples or boundary samples that the model has difficulty in accurately judging. By calculating the sample confidence and comparing it with a preset confidence threshold, the rule-based prior probability distribution is directly used for samples that meet the preset unfamiliar sample conditions, and the fusion result is used for samples that meet the preset determination conditions. This avoids prediction bias caused by the model's misjudgment of unfamiliar samples, effectively improves the adaptability of the fault diagnosis model to special samples, and enhances the accuracy, security and robustness of fault prediction.

[0017] Furthermore, it also includes: A training classification dataset is obtained based on a pre-acquired historical dissolved gas dataset; Input the training classification dataset into the fault diagnosis model to obtain the training fault prediction result set; Obtain the true one-hot encoded vector set corresponding to the training classification dataset; Obtain the training prior probability distribution set based on the training classification dataset; Loss is calculated based on the real one-hot encoded vector set, the training prior probability distribution set, and the preset prior weights to obtain the training target soft label set; A training loss function is constructed based on the training target soft label set and the training fault prediction result set; The fault diagnosis model is trained based on the training loss function to obtain the target fault diagnosis model, so as to realize transformer fault prediction based on the target fault diagnosis model.

[0018] In the above scheme, the present invention constructs training target soft labels based on real one-hot encoding vectors, training prior probability distributions, and preset prior weights during the training process. Then, the model is trained through a loss function. By fusing the data patterns of real sample labels and prior knowledge of expert rules through soft labels, the model not only fits the data distribution during training, but also learns decision logic that conforms to physical mechanisms. This plays an effective regularization role, avoiding abnormal diagnosis caused by overfitting noisy data in a purely data-driven model. It significantly improves the generalization ability of the model, making the trained target fault diagnosis model more stable and with higher diagnostic accuracy.

[0019] This invention provides a transformer fault prediction method that significantly reduces the computational complexity of the fault diagnosis model and improves operational efficiency through a Top-K sparse self-attention mechanism, making it suitable for resource-constrained edge computing scenarios. Through feature expansion and interpretability analysis, it obtains concise, efficient, and physically consistent classification feature data, laying the foundation for prediction accuracy. By employing soft-label distillation training and prior knowledge fusion, it achieves a deep integration of data-driven and knowledge-driven approaches, improving the model's generalization ability and diagnostic accuracy in small-sample scenarios. Furthermore, it sets up a confidence backoff strategy and an adaptive fusion mechanism to enhance the model's adaptability to boundary and unfamiliar samples, improving the robustness and security of the prediction results. This ensures timely and accurate prediction of various transformer faults such as partial discharge and overheating, providing reliable support for preventative maintenance of power transformers, effectively reducing the risk of power grid outages, and ensuring the safe and stable operation of the power system.

[0020] The present invention also provides a transformer fault prediction system for implementing the above-mentioned transformer fault prediction method, comprising: The model building module is used to build a fault diagnosis model that includes an input encoding network, a Top-K sparse self-attention network, and a classification output network. The feature processing module is used to obtain classification feature data based on pre-acquired dissolved gas data in oil; An input encoding module is used to input the classification feature data into the fault diagnosis model, so as to perform data preprocessing on the classification feature data based on the input encoding network and obtain preprocessed encoded features; A Top-K sparse self-attention module is used to process the preprocessed encoded features using the Top-K sparse self-attention network to obtain sparse encoded features. The Top-K sparse self-attention mechanism processing includes: performing linear projection on the preprocessed encoded features to obtain a multi-dimensional query vector, a multi-dimensional key vector, and a multi-dimensional value vector; constructing a sparse attention matrix based on the multi-dimensional query vector, multi-dimensional key vector, and multi-dimensional value vector; and performing weighted aggregation on the sparse attention matrix and the multi-dimensional value vector to obtain sparse encoded features. The classification output module is used to perform multi-head self-attention processing and fully connected classification processing on the sparse coding features based on the classification output network to obtain transformer fault prediction results.

[0021] The present invention provides a transformer fault prediction system, which provides high-quality input to the model through a feature processing module, builds the core architecture through a model construction module, ensures the efficiency and effectiveness of feature processing through an input encoding module and a Top-K sparse self-attention module, and accurately outputs prediction results through a classification output module. The collaborative work of these modules achieves efficient and accurate transformer fault prediction.

[0022] Furthermore, the fault diagnosis model also includes a prior knowledge fusion network; the system also includes a decision fusion module, wherein the decision fusion module is used for: Based on the prior knowledge fusion network and classification feature data, three-ratio features are obtained, and based on the three-ratio features and the preset three-ratio method fault diagnosis rules, the rule prior probability distribution is obtained. The transformer fault prediction results and rule-based prior probability distributions are weighted and fused based on the prior knowledge fusion network to obtain the target fault diagnosis probability.

[0023] Furthermore, it also includes a model training module, wherein the model training module is used for: A training classification dataset is obtained based on a pre-acquired historical dissolved gas dataset; Input the training classification dataset into the fault diagnosis model to obtain the training fault prediction result set; Obtain the true one-hot encoded vector set corresponding to the training classification dataset; Obtain the training prior probability distribution set based on the training classification dataset; Loss is calculated based on the real one-hot encoded vector set, the training prior probability distribution set, and the preset prior weights to obtain the training target soft label set; A training loss function is constructed based on the training target soft label set and the training fault prediction result set; The fault diagnosis model is trained based on the training loss function to obtain the target fault diagnosis model, so as to realize transformer fault prediction based on the target fault diagnosis model. Attached Figure Description

[0024] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of a transformer fault prediction method provided in this embodiment; Figure 2 This is a schematic diagram of the global average SHAP feature contribution distribution of the 51 features provided in this embodiment; Figure 3 This is a schematic diagram illustrating the contribution distribution of the 16-dimensional classification feature data provided in this embodiment; Figure 4 This is a schematic diagram of the specific structure of the Top-K sparse self-attention mechanism provided in this embodiment; Figure 5This is a schematic diagram of a fault diagnosis model structure provided in this embodiment. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0028] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0029] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0030] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0031] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0032] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0033] This embodiment provides a method for predicting transformer faults, such as... Figure 1 As shown, it includes: S1. Construct a fault diagnosis model that includes an input encoding network, a Top-K sparse self-attention network, and a classification output network; S2. Obtain classification feature data based on pre-acquired dissolved gas data in oil; S3. Input the classification feature data into the fault diagnosis model to perform data preprocessing on the classification feature data based on the input coding network to obtain preprocessed coding features; S4. Based on the Top-K sparse self-attention network, the preprocessed encoded features are processed by the Top-K sparse self-attention mechanism to obtain sparse encoded features; S5. Based on the classification output network, perform multi-head self-attention processing and fully connected classification processing on the sparse coding features to obtain transformer fault prediction results; The Top-K sparse self-attention mechanism includes the following processing: Based on the preprocessed encoded features, linear projection is performed to obtain multidimensional query vector, multidimensional key vector, and multidimensional value vector; A sparse attention matrix is ​​constructed based on the multidimensional query vector, multidimensional key vector, and multidimensional value vector. Sparse coding features are obtained by weighted aggregation based on the sparse attention matrix and multidimensional value vector.

[0034] This embodiment provides a transformer fault prediction method, constructing a fault diagnosis model comprising an input encoding network, a Top-K sparse self-attention network, and a classification output network. The Top-K sparse self-attention mechanism first performs linear projection on the preprocessed encoded features, constructs a sparse attention matrix, and performs weighted aggregation to obtain sparse encoded features. By introducing the Top-K sparse self-attention mechanism, only the K key-value pairs with the highest attention scores are retained for calculation, discarding redundant low-attention key-value pair association calculations. This effectively reduces the computational complexity of standard self-attention mechanisms while focusing on capturing key feature associations, ensuring that the fault diagnosis model improves computational efficiency without sacrificing core feature extraction capabilities. Through input encoding, sparse attention processing, and classification output, computational efficiency is significantly improved while maintaining model performance, achieving efficient and accurate transformer fault prediction, and adapting to resource-constrained application scenarios.

[0035] Optionally, step S2 includes: Feature augmentation is performed based on pre-acquired dissolved gas data in oil to obtain a multidimensional experimental dataset; Interpretability analysis was performed based on the multidimensional experimental dataset to obtain the feature contribution distribution; Based on the aforementioned feature contribution distribution, preset feature contribution threshold, and preset three-ratio retention strategy, the multidimensional experimental dataset is simplified to obtain classification feature data.

[0036] In the specific implementation process, this embodiment uses the dissolved gas data (DGA) of transformer oil in the Guangdong power grid as the basis for constructing the experimental dataset. The original monitoring attributes include the concentrations of five fault characteristic gases: hydrogen, methane, ethane, ethylene, and acetylene, as well as the total hydrocarbon content. Each data point consists of the gas content of the transformer oil sample and the corresponding fault category label for the equipment's operating condition. The fault types are classified according to the IEC 60599 standard into seven categories: Normal, Partial Discharge / Low Energy Discharge (D1), High Energy Discharge (D2), Low Temperature Overheating (T1), Medium Temperature Overheating (T2), and High Temperature Overheating (T3) (as well as normal operating conditions).

[0037] To enhance the characterization capability of various fault types, this embodiment expands and selects features based on the original gas concentrations, constructing a 51-dimensional DGA feature system, as shown in Table 1 below. Here, F1–F5 represent five original dissolved gas concentration features; F6 defines total hydrocarbons as a benchmark for characterizing the total amount of fault gas; F7–F17 represent the ratios of each gas or gas combination relative to total hydrocarbons, used to characterize the proportion of a single gas or gas combination in the total fault gas; and F18–F32 represent pairwise ratios between gases, used to characterize the relationships between multiple gases. The relative proportions are defined as follows: F33 defines the total gas quantity; F39 defines the gas combination quantity G2, which is the sum of the concentrations of four gases: hydrogen, ethylene, ethane, and acetylene, excluding methane; F40–F44 are the normalized ratios of each gas relative to G2; F45 defines the gas combination quantity G3, which is the sum of the concentrations of three hydrocarbon gases: ethylene, ethane, and acetylene, excluding hydrogen and methane; F46–F50 are the normalized ratios of each gas relative to G3; F51 is a higher-order interaction feature (G2+G3) / G1, used to comprehensively reflect the relative changes in the intensity of discharge and thermal faults. Here, G1, G2, and G3 represent the total amount of fault gases in different combinations. The resulting 51-dimensional multidimensional experimental dataset, through multi-level feature design including "concentration—ratio—normalization—interaction," forms a strong ability to distinguish between discharge and overheating faults.

[0038] In terms of feature preprocessing, this embodiment does not perform global normalization / standardization on the input features to preserve the original dimensions and facilitate the direct application of prior rules to the original values. To address the issue of significant differences in the magnitude of gas concentrations, the model automatically learns feature importance through subsequent self-attention mechanisms and internal normalization operations, without the need for forced scaling of the input. Furthermore, outliers and missing values ​​are cleaned and appropriately imputed to ensure the quality of the model training data.

[0039] Table 1 DGA Feature System Furthermore, to construct a feature subset that possesses both high discriminative power and conforms to physical mechanisms, this embodiment quantitatively analyzes and filters the multidimensional experimental dataset of the constructed 51-dimensional DGA features based on SHAP values. By calculating the feature contribution values ​​of the 51 features, according to... Figure 2The global average SHAP feature contribution distribution of all 51 features shown reveals a significant long-tail effect in feature importance. Only a few features play a dominant role in fault diagnosis, while the contribution values ​​of a large number of features at the tail are close to zero, indicating significant redundancy in the original feature space. Based on this, this embodiment employs a dual mechanism of data-driven filtering and expert knowledge protection to construct a simplified feature set. Specifically, a preset feature contribution threshold τ = 0.05 is set. Features with a global average SHAP absolute value less than τ are considered redundant features with weak discriminative effect and are removed to reduce model computational complexity and suppress instability caused by noisy features. Simultaneously, considering the core position of the IEC60599 three ratios in fault mechanism determination, this embodiment sets a preset three ratio retention strategy, forcibly retaining them regardless of whether their SHAP contribution is higher than the threshold, to ensure the integrity of the input information of the prior knowledge distillation module. After the above strategy, the multidimensional experimental dataset is simplified from 51 dimensions to 16 dimensions of classification feature data. The contribution distribution of the filtered 16-dimensional classification feature data is shown below. Figure 3 As shown, the 16 key features constitute a high-density subset of information, which not only retains the most sensitive discrimination factors for various faults, but also ensures the physical integrity of the input of the prior knowledge distillation module.

[0040] Optionally, the data preprocessing includes: Based on the classification feature data, a model hidden dimension mapping is performed to obtain the embedded feature sequence; Position encoding is performed based on the embedded feature sequence to obtain the encoded feature sequence; Multi-head self-attention processing is performed based on the encoded feature sequence to obtain multi-dimensional encoded features; Based on the multidimensional coding features, summation and standardization are performed to obtain preprocessed coding features.

[0041] In the specific implementation process, when constructing a fault diagnosis model including an input encoding network, a Top-K sparse self-attention network, and a classification output network, the input encoding network includes a feature embedding layer, a position encoding layer, a multi-head attention layer, and a summation and normalization layer. Specifically: the original classification feature data is input to the feature embedding layer for model hidden dimension mapping, resulting in an embedded feature sequence; the embedded feature sequence is input to the position encoding layer for position encoding to obtain an encoded feature sequence; the encoded feature sequence is input to the multi-head attention layer for multi-head self-attention processing using standard multi-head self-attention and a feedforward network to obtain multi-dimensional encoded features; the multi-dimensional encoded features are input to the summation and normalization layer for summation and normalization, and preprocessed encoded features are obtained through residual connections and layer normalization.

[0042] Optionally, constructing a sparse attention matrix based on the multidimensional query vector, multidimensional key vector, and multidimensional value vector includes: The attention score matrix is ​​obtained based on the multidimensional query vector, the multidimensional key vector, and the preset attention head dimension. Based on the attention score matrix, a row-by-row maximum value index is performed to obtain the sparse attention matrix.

[0043] In the specific implementation process, in order to reduce the computational complexity of standard self-attention, this invention proposes a Top-K sparse self-attention mechanism. Specifically, in constructing a fault diagnosis model including an input encoding network, a Top-K sparse self-attention network, and a classification output network, the Top-K sparse self-attention network includes a Top-K attention layer and a summation normalization layer, realizing the Top-K sparse self-attention mechanism, feedforward network, residual connection, and layer normalization processing to obtain sparse encoding features.

[0044] The specific structure of the Top-K sparse self-attention mechanism is as follows: Figure 4 As shown, the core idea of ​​this mechanism is that for each query vector, only the K key-value pairs with the highest attention scores are retained for weighted aggregation, while the attention weights for the remaining positions are set to negative infinity (approaching 0 after softmax). The specific calculation process is as follows: First, linear projection is performed on the preprocessed encoded features, and the query, key, and value matrices are constructed through the following calculation process to obtain the multidimensional query vector Q, the multidimensional key vector K, and the multidimensional value vector V: ; Where X represents the input preprocessed encoded features. These are learnable projection weights; The attention score is then calculated using the following procedure. To obtain the attention score matrix based on the attention score: ; in, Dimensions for each attention head; Perform a row-by-row maximum indexing: for each row of the attention score matrix, take the top-K maximum values ​​and their indices, i.e., retain the K largest values ​​in each row, and perform weighted aggregation. Set the attention weights for the remaining positions to negative infinity to construct a sparse attention matrix. The specific calculation process is shown in the following formula: ; Finally, softmax normalization is applied and multiplied by V to obtain the sparse coding features. The specific calculation process is shown in the following formula: ; in, This represents the sparse coding features of the output.

[0045] Preferably, in this embodiment, two (or more) connected Top-K sparse self-attention networks are set up to achieve better sparse attention effects and further reduce the computational complexity of standard self-attention.

[0046] Optionally, the fault diagnosis model further includes a prior knowledge fusion network, and after step S5, it further includes: Based on the prior knowledge fusion network and classification feature data, three-ratio features are obtained, and based on the three-ratio features and the preset three-ratio method fault diagnosis rules, the rule prior probability distribution is obtained. The transformer fault prediction results and rule-based prior probability distributions are weighted and fused based on the prior knowledge fusion network to obtain the target fault diagnosis probability.

[0047] In the specific implementation process, the classification feature data is input into the improved Transformer model. After feature embedding, a four-layer encoder, and classification head processing, the original transformer fault prediction result logits is obtained. Applying the Softmax normalization function to the logits yields the model's predicted probability distribution. ,in The output of the classification head is 7, representing the total number of fault categories. To effectively embed expert prior knowledge in the power sector into the data-driven model, this embodiment designs a knowledge distillation strategy based on soft labels. First, an expert rule base is constructed according to the DGA fault judgment guideline in the international standard IEC 60599:2022. Specifically, the three-ratio method from the IEC 60599 standard is used as the core criterion of the expert system, extracting three key feature ratios from the classification feature data to obtain the three-ratio features: , , The fault codes and ratio boundaries of the three ratio features obtained according to the preset three-ratio method fault diagnosis rules (i.e., the IEC 60599 three-ratio method fault diagnosis rules) are shown in Table 2 below. Based on Table 2 and the prior probability distribution of the three-ratio feature acquisition rules... In the prediction phase, to further improve system reliability, the final failure probability output is the target failure diagnosis probability. Using models to predict probabilities With regular probability The weighted fusion is calculated as follows: ; Where α is the fusion coefficient.

[0048] Table 2 Fault Diagnosis Rules for the IEC 60599 Three Ratio Method Optionally, after the steps of obtaining the three-ratio features based on the prior knowledge fusion network and classification feature data, and obtaining the rule prior probability distribution based on the three-ratio features and the preset three-ratio method fault diagnosis rules, the method further includes: The maximum probability distribution value of the transformer fault prediction result is determined to be the sample confidence level; The comparison is based on the sample confidence level and a preset confidence threshold: When the preset unfamiliar sample conditions are met, the prior probability distribution of the rule is used as the target fault diagnosis probability. When the preset conditions are met, the transformer fault prediction results and rule-based prior probability distributions are weighted and fused based on the prior knowledge fusion network to obtain the sample fault diagnosis probability, and the sample fault diagnosis probability is used as the target fault diagnosis probability.

[0049] In practical implementation, when the fault diagnosis model has low confidence in a certain unfamiliar sample, this mechanism allows the system to smoothly revert to rule-based judgment, ensuring the safety of the diagnosis. Specifically, to enhance the robustness of the system on boundary samples and anomalous samples, this embodiment introduces a confidence backoff strategy. The sample confidence level is defined. The maximum probability distribution value of the transformer fault prediction results: Set a preset reliability threshold α. The specific judgment logic is as follows: when... When the model's judgment on the sample is relatively certain and it is determined to meet the preset determination conditions, the fusion probability is used. Classify; when When the condition is met, it indicates that the sample is an "unfamiliar sample" and is determined to meet the preset unfamiliar sample conditions, namely, its feature pattern differs significantly from the training distribution or is on the boundary of the fault type. At this time, the system automatically reverts to the prior rule judgment and directly adopts the rule. The argmax result is used as the final diagnosis. This mechanism ensures that when the deep learning model faces out-of-distribution samples or noise interference, the system can smoothly fall back to the engineering-validated IEC expert rules, guaranteeing the safety and reliability of the diagnostic results.

[0050] Optional, also includes: A training classification dataset is obtained based on a pre-acquired historical dissolved gas dataset; Input the training classification dataset into the fault diagnosis model to obtain the training fault prediction result set; Obtain the true one-hot encoded vector set corresponding to the training classification dataset; Obtain the training prior probability distribution set based on the training classification dataset; Loss is calculated based on the real one-hot encoded vector set, the training prior probability distribution set, and the preset prior weights to obtain the training target soft label set; A training loss function is constructed based on the training target soft label set and the training fault prediction result set; The fault diagnosis model is trained based on the training loss function to obtain the target fault diagnosis model, so as to realize transformer fault prediction based on the target fault diagnosis model.

[0051] In the specific implementation process, during the training phase, in order to guide the fault diagnosis model to learn decision boundaries that conform to physical mechanisms, this embodiment explicitly introduces rule priors into the loss calculation. Let... This is the one-hot vector of the real label (the real one-hot encoded vector). The prior probability distribution calculated for the rules. Training objective. Defined as: ; Here, λ is the prior weight hyperparameter. This strategy enables the model to fit the data distribution while moderately mimicking the judgment logic of expert rules, thus playing a regularization role and effectively avoiding abnormal diagnosis caused by overfitting noise in purely data-driven models.

[0052] In the specific implementation process, the feedforward network using the GEGLU activation function in this embodiment has smoother gradient characteristics compared to the traditional ReLU: ; in, For learnable weights, ⊙ represents element-wise multiplication. Characteristics of the input feedforward network.

[0053] In the specific implementation process, the classification output network is implemented using a classification head, which includes a multi-head attention layer and a summation and normalization layer. After layer normalization, flattening and fully connected classification processing, the original transformer fault prediction results of the output model are output.

[0054] In summary, the transformer fault prediction method proposed in this embodiment aims to solve the following technical problems: It proposes an efficient sparse attention mechanism to maintain the model's feature capture capability; it proposes a prior knowledge enhancement strategy to transform the diagnostic rules of the traditional three-ratio method into soft labels for distillation training, and achieves adaptive fusion of model prediction and rule priors in the inference stage; it introduces the SHAP interpretability analysis method to provide a quantitative assessment of feature importance, thereby enhancing the interpretability and credibility of the model.

[0055] Preferably, this embodiment provides a transformer fault prediction method based on an improved Transformer and prior knowledge enhancement, using a TransformerEncoder structure as the main body of the fault diagnosis model. The constructed fault diagnosis model structure is as follows: Figure 5 As shown, the backbone of the fault diagnosis model in this embodiment is an improved Transformer encoder structure, including modules such as a feature embedding layer, a positional encoding layer, a multi-head self-attention layer, and a summation and normalization layer. In the middle, a Top-k attention layer is introduced in the second and third layers, replacing ordinary multi-head attention with a strong sparse attention that retains the Top-k connections head-by-head according to the query. Finally, the output is processed through the multi-head self-attention layer and the summation and normalization layer. The output is then input into a decision fusion layer formed based on a prior knowledge fusion network, which fuses the output probability of the fault diagnosis model with the prior rule probabilities calculated according to IEC rules to obtain the final classification output, achieving joint decision-making based on data-driven and rule-prior knowledge.

[0056] In this implementation, each Transformer block adopts a Pre-Norm structure, meaning layer normalization is applied before attention and FFN. RMSNorm is supported as an alternative to further improve computational efficiency. DropPath regularization is also employed to prevent overfitting.

[0057] This embodiment also provides a transformer fault prediction system for implementing the above-described transformer fault prediction method, including: The model building module is used to build a fault diagnosis model that includes an input encoding network, a Top-K sparse self-attention network, and a classification output network. The feature processing module is used to obtain classification feature data based on pre-acquired dissolved gas data in oil; An input encoding module is used to input the classification feature data into the fault diagnosis model, so as to perform data preprocessing on the classification feature data based on the input encoding network and obtain preprocessed encoded features; A Top-K sparse self-attention module is used to process the preprocessed encoded features using the Top-K sparse self-attention network to obtain sparse encoded features. The Top-K sparse self-attention mechanism processing includes: performing linear projection on the preprocessed encoded features to obtain a multi-dimensional query vector, a multi-dimensional key vector, and a multi-dimensional value vector; constructing a sparse attention matrix based on the multi-dimensional query vector, multi-dimensional key vector, and multi-dimensional value vector; and performing weighted aggregation on the sparse attention matrix and the multi-dimensional value vector to obtain sparse encoded features. The classification output module is used to perform multi-head self-attention processing and fully connected classification processing on the sparse coding features based on the classification output network to obtain transformer fault prediction results.

[0058] Optionally, the fault diagnosis model further includes a prior knowledge fusion network; the system also includes a decision fusion module, wherein the decision fusion module is used for: Based on the prior knowledge fusion network and classification feature data, three-ratio features are obtained, and based on the three-ratio features and the preset three-ratio method fault diagnosis rules, the rule prior probability distribution is obtained. The transformer fault prediction results and rule-based prior probability distributions are weighted and fused based on the prior knowledge fusion network to obtain the target fault diagnosis probability.

[0059] Optionally, it also includes a model training module, wherein the model training module is used for: A training classification dataset is obtained based on a pre-acquired historical dissolved gas dataset; Input the training classification dataset into the fault diagnosis model to obtain the training fault prediction result set; Obtain the true one-hot encoded vector set corresponding to the training classification dataset; Obtain the training prior probability distribution set based on the training classification dataset; Loss is calculated based on the real one-hot encoded vector set, the training prior probability distribution set, and the preset prior weights to obtain the training target soft label set; A training loss function is constructed based on the training target soft label set and the training fault prediction result set; The fault diagnosis model is trained based on the training loss function to obtain the target fault diagnosis model, so as to realize transformer fault prediction based on the target fault diagnosis model.

[0060] In summary, this embodiment provides a transformer fault prediction method and system that proposes a Top-K sparse self-attention mechanism, which significantly improves computational efficiency while maintaining the performance of the fault diagnosis model, making it suitable for resource-constrained edge computing scenarios. It innovatively transforms the domain expert knowledge of the traditional three-ratio method into soft labels for knowledge distillation, achieving an effective fusion of data-driven and knowledge-driven approaches, significantly improving the model's generalization ability and diagnostic accuracy in small-sample scenarios. The adaptive fusion mechanism in the inference stage dynamically adjusts the weights of the rule priors based on the model confidence level and sets a confidence backoff strategy, reverting to expert rules when the model is uncertain, thus enhancing the robustness and reliability of the prediction. The integration of SHAP interpretability analysis enables quantitative evaluation of the contribution of each DGA feature to fault diagnosis, generating intuitive visualization results that meet the power system's requirements for transparency and traceability in diagnostic decisions. Furthermore, the fault diagnosis model structure is flexible and configurable, supporting combinations of various attention mechanisms and FFN variants, facilitating optimization for different application scenarios.

[0061] Based on the above embodiment of a transformer fault prediction method, another embodiment of the present invention provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a transformer fault prediction method according to any embodiment of the present invention.

[0062] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the terminal device.

[0063] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0064] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0065] Based on the above-described method embodiments, another embodiment of the present invention provides a computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute a transformer fault prediction method as described in any of the above-described method embodiments of the present invention.

[0066] The modules / units integrated in the device / terminal equipment, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0067] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for predicting transformer faults, characterized in that, include: Construct a fault diagnosis model that includes an input encoding network, a Top-K sparse self-attention network, and a classification output network; Classification feature data is obtained based on pre-acquired dissolved gas data in oil; The classification feature data is input into the fault diagnosis model to perform data preprocessing on the classification feature data based on the input coding network to obtain preprocessed coding features; The preprocessed encoded features are processed using the Top-K sparse self-attention network to obtain sparse encoded features. Based on the classification output network, multi-head self-attention processing and fully connected classification processing are performed on the sparse coding features to obtain transformer fault prediction results. The Top-K sparse self-attention mechanism includes the following processing: Based on the preprocessed encoded features, linear projection is performed to obtain multidimensional query vector, multidimensional key vector, and multidimensional value vector; A sparse attention matrix is ​​constructed based on the multidimensional query vector, multidimensional key vector, and multidimensional value vector. Sparse coding features are obtained by weighted aggregation based on the sparse attention matrix and multidimensional value vector.

2. The transformer fault prediction method as described in claim 1, characterized in that, The process of obtaining classification feature data based on pre-acquired dissolved gas data in oil includes: Feature augmentation is performed based on pre-acquired dissolved gas data in oil to obtain a multidimensional experimental dataset; Interpretability analysis was performed based on the multidimensional experimental dataset to obtain the feature contribution distribution; Based on the aforementioned feature contribution distribution, preset feature contribution threshold, and preset three-ratio retention strategy, the multidimensional experimental dataset is simplified to obtain classification feature data.

3. The transformer fault prediction method as described in claim 1, characterized in that, The data preprocessing includes: Based on the classification feature data, a model hidden dimension mapping is performed to obtain the embedded feature sequence; Position encoding is performed based on the embedded feature sequence to obtain the encoded feature sequence; Multi-head self-attention processing is performed based on the encoded feature sequence to obtain multi-dimensional encoded features; Based on the multidimensional coding features, summation and standardization are performed to obtain preprocessed coding features.

4. The transformer fault prediction method as described in claim 1, characterized in that, The construction of the sparse attention matrix based on the multidimensional query vector, multidimensional key vector, and multidimensional value vector includes: The attention score matrix is ​​obtained based on the multidimensional query vector, the multidimensional key vector, and the preset attention head dimension. Based on the attention score matrix, a row-by-row maximum value index is performed to obtain the sparse attention matrix.

5. The transformer fault prediction method as described in claim 1, characterized in that, The fault diagnosis model further includes a prior knowledge fusion network. After the step of performing multi-head self-attention processing and fully connected classification processing on the sparse encoded features based on the classification output network to obtain the transformer fault prediction result, the model further includes: Based on the prior knowledge fusion network and classification feature data, three-ratio features are obtained, and based on the three-ratio features and the preset three-ratio method fault diagnosis rules, the rule prior probability distribution is obtained. The transformer fault prediction results and rule-based prior probability distributions are weighted and fused based on the prior knowledge fusion network to obtain the target fault diagnosis probability.

6. The transformer fault prediction method as described in claim 5, characterized in that, After the steps of obtaining the three-ratio features based on the prior knowledge fusion network and classification feature data, and obtaining the rule prior probability distribution based on the three-ratio features and the preset three-ratio method fault diagnosis rules, the method further includes: The maximum probability distribution value of the transformer fault prediction result is determined to be the sample confidence level; The comparison is based on the sample confidence level and a preset confidence threshold: When the preset unfamiliar sample conditions are met, the prior probability distribution of the rule is used as the target fault diagnosis probability. When the preset conditions are met, the transformer fault prediction results and rule-based prior probability distributions are weighted and fused based on the prior knowledge fusion network to obtain the sample fault diagnosis probability, and the sample fault diagnosis probability is used as the target fault diagnosis probability.

7. The transformer fault prediction method as described in claim 5, characterized in that, Also includes: A training classification dataset is obtained based on a pre-acquired historical dissolved gas dataset; Input the training classification dataset into the fault diagnosis model to obtain the training fault prediction result set; Obtain the true one-hot encoded vector set corresponding to the training classification dataset; Obtain the training prior probability distribution set based on the training classification dataset; Loss is calculated based on the real one-hot encoded vector set, the training prior probability distribution set, and the preset prior weights to obtain the training target soft label set; A training loss function is constructed based on the training target soft label set and the training fault prediction result set; The fault diagnosis model is trained based on the training loss function to obtain the target fault diagnosis model, so as to realize transformer fault prediction based on the target fault diagnosis model.

8. A transformer fault prediction system, characterized in that, A method for implementing a transformer fault prediction method as described in any one of claims 1 to 7 includes: The model building module is used to build a fault diagnosis model that includes an input encoding network, a Top-K sparse self-attention network, and a classification output network. The feature processing module is used to obtain classification feature data based on pre-acquired dissolved gas data in oil; An input encoding module is used to input the classification feature data into the fault diagnosis model, so as to perform data preprocessing on the classification feature data based on the input encoding network and obtain preprocessed encoded features; A Top-K sparse self-attention module is used to process the preprocessed encoded features using the Top-K sparse self-attention network to obtain sparse encoded features. The Top-K sparse self-attention mechanism processing includes: performing linear projection on the preprocessed encoded features to obtain a multi-dimensional query vector, a multi-dimensional key vector, and a multi-dimensional value vector; constructing a sparse attention matrix based on the multi-dimensional query vector, multi-dimensional key vector, and multi-dimensional value vector; and performing weighted aggregation on the sparse attention matrix and the multi-dimensional value vector to obtain sparse encoded features. The classification output module is used to perform multi-head self-attention processing and fully connected classification processing on the sparse coding features based on the classification output network to obtain transformer fault prediction results.

9. A transformer fault prediction system as described in claim 8, characterized in that, The fault diagnosis model further includes a prior knowledge fusion network; the system also includes a decision fusion module, wherein the decision fusion module is used for: Based on the prior knowledge fusion network and classification feature data, three-ratio features are obtained, and based on the three-ratio features and the preset three-ratio method fault diagnosis rules, the rule prior probability distribution is obtained. The transformer fault prediction results and rule-based prior probability distributions are weighted and fused based on the prior knowledge fusion network to obtain the target fault diagnosis probability.

10. A transformer fault prediction system as described in claim 9, characterized in that, It also includes a model training module, wherein the model training module is used for: A training classification dataset is obtained based on a pre-acquired historical dissolved gas dataset; Input the training classification dataset into the fault diagnosis model to obtain the training fault prediction result set; Obtain the true one-hot encoded vector set corresponding to the training classification dataset; Obtain the training prior probability distribution set based on the training classification dataset; Loss is calculated based on the real one-hot encoded vector set, the training prior probability distribution set, and the preset prior weights to obtain the training target soft label set; A training loss function is constructed based on the training target soft label set and the training fault prediction result set; The fault diagnosis model is trained based on the training loss function to obtain the target fault diagnosis model, so as to realize transformer fault prediction based on the target fault diagnosis model.

Images