An Explainable Transformer Fault Diagnosis Method Based on TabPFN and Causal Correlation

Abstract

An interpretable transformer fault diagnosis method based on TabPFN and causal association includes: collecting dissolved gas concentration data in oil-immersed transformer oil to construct an initial sample set of characteristic gases and fault type labels; constructing a fault diagnosis model by fitting the TabPFN network with tabular prior data based on the training set, and evaluating the diagnostic stability under various small-sample imbalanced validation scenarios; performing t-SNE dimensionality reduction on the characteristic gas data of the test set and analyzing the spatial distribution law of fault features using silhouette coefficients; introducing the KernelSHAP method to quantify the contribution of each characteristic gas to fault type determination; constructing a causal association network among gas variables under different fault types using NOTEARS, and verifying the dominant role of key causal nodes through feature removal experiments. This invention utilizes TabPFN to achieve high-stability diagnosis under small-sample conditions and constructs a multi-level interpretable framework from association interpretation to causal interpretation, overcoming the problems of existing methods' dependence on large-scale labeled data and interpretability limited to correlation analysis.

Description

Technical Field

[0001] This invention relates to the field of transformer fault diagnosis technology, and specifically to an interpretable transformer fault diagnosis method based on TabPFN and causal correlation. Background Technology

[0002] Dissolved gas analysis (DGA) is a crucial method for fault diagnosis in oil-immersed transformers. When thermal or electrical faults occur inside the transformer, the insulating oil and solid insulating materials decompose, producing characteristic gases such as hydrogen (H2), methane (CH4), ethane (C2H6), ethylene (C2H4), and acetylene (C2H2). By detecting the composition and content of these gases, latent faults within the equipment can be effectively identified, ensuring the safe and stable operation of the power system.

[0003] Currently, transformer fault diagnosis methods based on DGA are mainly divided into two categories: one is the traditional ratio method, such as the IEC three-ratio method, Rogers ratio method, and Duval triangle method. These methods calculate the ratio between characteristic gases and determine the fault type by comparing it with coding rules, which has the advantages of clear physical meaning and easy understanding. However, the traditional ratio method has problems such as ambiguous diagnostic boundaries and incomplete coding coverage, and is prone to misjudgment when the gas concentration is close to the threshold boundary.

[0004] Another category is machine learning-based intelligent diagnostic methods, such as support vector machines, random forests, and extreme gradient boosting. These methods, by constructing a mapping between features and fault types, can automatically learn complex patterns in the data, thus improving diagnostic accuracy to some extent. However, existing intelligent diagnostic methods still have the following shortcomings: (1) Insufficient adaptability of small samples and difficulty in hyperparameter tuning: Transformer faults are rare events, and it is difficult to obtain a large number of labeled fault samples in actual engineering. Traditional machine learning models not only rely on large-scale training data, but their diagnostic performance also depends heavily on the selection of hyperparameters. They often need to be repeatedly debugged to achieve the ideal effect, and their diagnostic stability is poor in scenarios with scarce samples.

[0005] (2) Interpretability is limited to the level of correlation analysis and lacks causal inference ability. Existing methods mostly use ex post-hoc interpretation techniques such as SHAP to analyze feature importance, which can effectively answer the question of which features contribute more to the diagnostic results and provide valuable local explanations for model decision-making. However, these methods mainly focus on statistical correlation analysis between features and fault types, and it is difficult to further reveal the deep causal evolution relationship between gas variables, and cannot fully explain the internal process of fault occurrence from the mechanistic level. Summary of the Invention

[0006] To address the issues of insufficient diagnostic stability under small sample conditions and limited interpretability due to correlation analysis in existing technologies, this invention provides an interpretable transformer fault diagnosis method that integrates Tabular Prior-Data Fitted Network (TabPFN) with causal association. This method leverages the stable diagnostic capability of TabPFN under small sample conditions to overcome the dependence of traditional methods on large-scale labeled data. Simultaneously, it constructs a multi-level interpretable framework from correlation interpretation to causal interpretation. KernelSHAP is used to quantify feature contributions and provide local decision explanations, while the NOTEARS algorithm further mines the causal evolution network among gas variables and reveals the intrinsic correlation of fault mechanisms, thus providing a complete and traceable basis for the diagnostic results, from feature contributions to the causal chain.

[0007] The technical solution adopted in this invention is as follows: An interpretable transformer fault diagnosis method based on TabPFN and causal correlation includes the following steps: S1: Collect dissolved gas concentration data in oil-immersed transformer oil, construct an initial sample set containing characteristic gases and corresponding fault type labels, and divide the initial sample set into a training set and a test set; S2: Based on the training set, a fault diagnosis model is constructed by fitting the TabPFN network with tabular prior data, and the diagnostic stability of the fault diagnosis model is evaluated under various small-sample imbalanced validation scenarios. S3: Perform t-SNE dimensionality reduction on the characteristic gas data of the test set, and combine the profile coefficient to analyze the distribution pattern and separability of different fault types in the feature space; S4: The KernelSHAP method is introduced to perform local interpretability analysis on the diagnostic results of the fault diagnosis model and to quantify the contribution of each characteristic gas to the fault type determination. S5: The NOTEARS algorithm is used to construct a causal relationship network among characteristic gas variables under different fault types, and the key causal nodes in the causal relationship network are used to verify the dominant role of fault diagnosis through feature removal experiments.

[0008] S1 includes: S1.1: Collect dissolved gas monitoring data in oil-immersed transformer oil, wherein the characteristic gases include hydrogen (H2), methane (CH4), ethane (C2H6), ethylene (C2H4), and acetylene (C2H2); S1.2: Clean the collected data to remove outliers and label it according to the preset fault types, which include six categories: medium and low temperature overheating fault (T1), high temperature overheating fault (T2), partial discharge (PD), low energy discharge (D1), high energy discharge (D2) and normal (N). S1.3: Divide the labeled dataset into training and test sets according to a preset ratio to construct the initial sample set.

[0009] In step S2, the construction and reasoning process of the fault diagnosis model includes: S2.1: Based on a prior distribution containing multiple data distribution forms, a tabular data training task is generated. The Transformer model is used for offline pre-training to learn the mapping relationship from the training dataset to the prediction results, thereby approximating the Bayesian posterior distribution. The Bayesian posterior distribution is represented as: ; In the formula, For the training dataset, For the sample to be predicted, For the corresponding tags, Represents the latent model parameters; S2.2: In actual diagnosis, the training set and the test samples are used together as context input to pre-train the Transformer model. Based on the self-attention mechanism, the conditional probability distribution is directly output, and its reasoning process is expressed as follows: ; In the formula, This represents the pre-trained Transformer model. These are the parameters of the Transformer model after pre-training.

[0010] The final prediction result is obtained based on the maximum a posteriori probability principle: .

[0011] In S2, the evaluation metrics used to assess the diagnostic stability of the fault diagnosis model include accuracy, precision, recall, and F1 score. The definitions of each metric are as follows: ; ; ; ; In the above formula, TP is the number of samples that are actually positive and are correctly predicted as positive, TN is the number of samples that are actually negative and are correctly predicted as negative, FP is the number of samples that are actually negative but are incorrectly predicted as positive, and FN is the number of samples that are actually positive but are incorrectly predicted as negative.

[0012] In S2: various small-sample imbalanced verification scenarios include mild imbalance, moderate imbalance and high imbalance scenarios, and the proportion of different fault types of samples in the training set is different in each scenario.

[0013] S3 includes: S3.1: The t-SNE method is used to perform nonlinear dimensionality reduction on the original feature gas data of the test set, by minimizing the sample similarity probability distribution in the high-dimensional space. Probability distribution corresponding to low-dimensional space The Kullback-Leibler divergence between samples is used to project the samples into a low-dimensional space, and its optimization objective is: ; In the formula, Represents samples in high-dimensional space With sample The similarity probability, Represents samples in a low-dimensional embedding space With sample The corresponding probability; S3.2: Introducing the silhouette coefficient to quantitatively evaluate the dimensionality-reduced category structure, the silhouette coefficient is defined as: ; In the formula, Indicates sample The average distance between it and other samples in the same category Indicates sample The silhouette coefficient is the average distance between the nearest neighbor samples of different classes. The value ranges from [-1, 1]. The larger the value, the higher the inter-class separation and the better the intra-class compactness of the classification result.

[0014] S4 includes: S4.1: Based on the prediction results of the fault diagnosis model, a local explanatory model is constructed with a single sample as the unit. The feature subset is weighted and sampled using the KernelSHAP method to establish a linear approximation relationship between the input features and the model output. The explanatory model is expressed as follows: ; In the formula, To explain the model, A binary vector indicating the presence or absence of a feature. The total number of characteristic gases. Baseline value, For the first The marginal contribution of each characteristic gas to the prediction results; S4.2: Marginal contribution of each characteristic gas Satisfying the Shapley value allocation principle, its calculation form is as follows: ; In the formula, It is the collection of all characteristic gases. , Not containing characteristic gases Feature subset, This is the prediction function of the fault diagnosis model under a given feature subset.

[0015] The S4 further includes: performing statistical analysis on the distribution of characteristic gas contribution values ​​of samples of different fault types, identifying key characteristic gases with positive and negative contributions under each fault mode, and identifying the threshold effect of key characteristic gases on model output through SHAP dependency graph.

[0016] S5 includes: S5.1: Based on the characteristic gas data obtained in S1, a variable set is constructed. The NOTEARS algorithm is used to learn the structural relationships between the variables, and a causal model in the form of a directed acyclic graph is established. Its structure consists of a weighted adjacency matrix. This is expressed as follows, and the solution is obtained by optimizing the following objective function: ; In the formula, The loss function is a data-based fitting function. For sparse constraint terms, is the regularization coefficient, used to control the sparsity of the network structure; S5.2: Introduce a continuously differentiable acyclic constraint function to ensure that the learned graph structure is a directed acyclic graph. The acyclic constraint function is: ; In the formula, Represents the trace of a matrix. For matrix exponentiation operations, Represents the Hadamard product. The number of variables.

[0017] The S5 also includes: S5.3: Analyze the weights and directions of edges in the causal relationship network to identify key causal relationship chains under different fault types; S5.4: Verify the dominant role of key causal nodes through feature removal experiments; The feature removal experiment includes: removing the feature gas data corresponding to the key causal nodes identified in the causal association network from the training set, retraining and evaluating the change in the diagnostic accuracy of the fault diagnosis model. The greater the decrease in diagnostic accuracy, the stronger the dominant role of the key causal node in fault diagnosis.

[0018] This invention provides an interpretable transformer fault diagnosis method based on TabPFN and causal correlation, with the following technical advantages: 1) Achieve high stability and high accuracy in transformer fault diagnosis under scenarios with small sample sizes and class imbalance: This invention, S2, employs TabPFN to construct a fault diagnosis model. This model acquires general inference capabilities through offline pre-training, eliminating the need for repeated training and complex hyperparameter tuning for specific datasets. In various imbalanced validation scenarios with small sample sizes, the model can fully utilize limited sample information, maintaining high diagnostic accuracy and stability. This overcomes the problems of overfitting and insufficient generalization ability of traditional machine learning methods with rare fault samples. In the examples, the overall accuracy reaches 89.29%, and the area under the ROC curve (AUC) reaches 0.978. Multiple metrics outperform the comparative models, validating the superior performance of this method in data-scarce scenarios.

[0019] 2) A multi-level interpretable framework, from correlational to causal explanations, was established to achieve complete traceability of diagnostic results: The organic combination of S4 and S5 in this invention forms a progressive explanatory system of "feature contribution - causal structure". S4, based on KernelSHAP, quantifies the marginal contribution of each characteristic gas to the diagnostic result of a single sample, answering the question "which gases affected this judgment", which belongs to local correlation explanation. S4 further constructs a directed acyclic causal network among gas variables based on the NOTEARS algorithm, revealing the deep causal relationship of "who causes whom" in the gas evolution process, and verifies the dominant role of key causal nodes through feature removal experiments. This framework elevates the explanatory power of the model from the correlation level to the causal inference level, providing maintenance personnel with a complete and traceable chain of evidence from decision-making basis to fault mechanism, significantly enhancing the credibility of diagnostic results.

[0020] 3) It provides visualization and quantitative verification methods for the spatial distribution of fault characteristics, enhancing the intuitive judgment of model reliability: In this invention, step S3 projects high-dimensional feature gas data into a low-dimensional space using t-SNE dimensionality reduction and combines it with silhouette coefficients to quantitatively evaluate the cluster compactness and separation of different fault categories. This step can intuitively display the decision boundaries and distribution clustering states of different fault types in the feature space, verifying the rationality of the feature structure learned by the model from the perspective of spatial separability. It provides both visual and quantitative support for the effectiveness of the diagnostic model, helping maintenance personnel to understand and trust the model's judgment logic.

[0021] 4) Threshold effect identification of key characteristic gases has been achieved, providing accurate decision-making basis for equipment condition-based maintenance: This invention, in S4, not only provides the contribution of each characteristic gas to the Shapley value, but also further identifies the threshold effect of key features on the model output through the SHAP dependency graph. For example, it can be clearly seen that when the concentration of ethylene (C2H4) exceeds a certain threshold, its positive contribution to the determination of medium- and low-temperature overheating faults increases significantly. This threshold discovery capability can help maintenance personnel accurately grasp the warning boundaries of characteristic gases, transform the "black box" model output into operable condition monitoring rules, and improve the accuracy and timeliness of early fault warnings.

[0022] 5) It reveals the causal topological structure of the gas evolution process, providing a structured basis for the analysis of transformer fault mechanisms: This invention, S5, utilizes the NOTEARS algorithm to learn a weighted adjacency matrix that satisfies the constraints of a directed acyclic graph, thereby constructing a causal relationship network between characteristic gases under different fault types. By analyzing the weights and directions of the edges in the network, causal evolution chains such as "changes in methane (CH4) and ethane (C2H6) are the cause of changes in ethylene (C2H4), thus driving the fault state" can be uncovered. This structured causal model explains the intrinsic generation and transformation relationships of dissolved gases in oil during fault occurrence from a physical process perspective, overcoming the limitation of traditional correlation analysis in identifying causal directions, and providing in-depth mechanistic support for transformer fault root cause analysis and operation and maintenance strategy formulation. Attached Figure Description

[0023] The present invention will be further described below with reference to the accompanying drawings and examples; Figure 1 This is the confusion matrix for the test set based on TabPFN.

[0024] Figure 2 A visualization of the decision boundary of the TabPFN model based on t-SNE dimensionality reduction.

[0025] Figure 3 ROC curves for different models.

[0026] Figure 4 The results show the comparison of various models in different scenarios.

[0027] Figure 5 This is a distribution diagram of the SHAP values ​​of various gas characteristics under different fault types.

[0028] Figure 6 This is a SHAP dependency plot of characteristic gases for medium- and low-temperature overheating faults.

[0029] Figure 7 This is a causal relationship diagram between different faults and normal states.

[0030] Figure 8 This is a quantitative evaluation result of feature removal.

[0031] Figure 9 Generate a comprehensive simulation graph of the prior distribution statistical characteristics of the data.

[0032] Figure 10 This is a distribution diagram of the attention weights of the test sample to the training sample in contextual reasoning.

[0033] Figure 11 A visualization of the decision boundary of the TabPFN model based on t-SNE dimensionality reduction. Detailed Implementation

[0034] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the scope of protection of the present invention.

[0035] This embodiment is based on 558 sets of DGA fault samples from oil-immersed transformers, covering six fault types: medium-low temperature overheating fault (T1), high temperature overheating fault (T2), partial discharge (PD), low-energy discharge (D1), high-energy discharge (D2), and normal state (N). The 558 sets of data are divided into training and test sets in an 8:2 ratio, and the specific sample distribution is shown in Table 1.

[0036]

[0037] To comprehensively evaluate the diagnostic stability of this invention under small sample imbalance conditions, this embodiment further designs three small sample validation scenarios with different degrees of imbalance: Scenario 1 is mildly imbalanced, Scenario 2 is moderately imbalanced, and Scenario 3 is highly imbalanced. The distribution of training and test set samples in each scenario is shown in Table 2.

[0038]

[0039] The interpretable transformer fault diagnosis method based on TabPFN and causal correlation provided by the present invention is implemented through the following steps S1 to S5.

[0040] Step S1: Collect dissolved gas concentration data in oil-immersed transformer oil, construct an initial sample set containing characteristic gases and corresponding fault type labels, and divide the initial sample set into a training set and a test set.

[0041] When a thermal or electrical fault occurs inside a transformer, the insulating oil and solid insulating materials decompose, producing characteristic gases such as hydrogen (H2), methane (CH4), ethane (C2H6), ethylene (C2H4), and acetylene (C2H2). By detecting the composition and content of these gases, latent faults inside the equipment can be effectively identified. Therefore, this step specifically involves: collecting online monitoring data of dissolved gases in the oil of oil-immersed transformers, using H2, CH4, C2H6, C2H4, and C2H2 as five core characteristic gases; cleaning the raw data to remove outliers caused by sensor malfunctions or operational errors; and labeling the samples according to preset fault types, including six categories: medium-low temperature overheating fault (T1), high temperature overheating fault (T2), partial discharge (PD), low-energy discharge (D1), high-energy discharge (D2), and normal (N). The labeled dataset is divided into training and test sets according to a preset ratio. The ratio can be flexibly set based on the total number of samples. This embodiment uses an 80% training set and a 20% test set ratio to ensure the model's generalization performance. After the division, an initial sample set is constructed for subsequent model training and evaluation.

[0042] Step S2: Based on the training set, a fault diagnosis model is constructed by fitting the TabPFN network with tabular prior data, and the diagnostic stability of the fault diagnosis model is evaluated under various small-sample imbalanced validation scenarios.

[0043] Transformer faults are rare events, making it difficult to obtain a large number of labeled fault samples in practical engineering. Traditional machine learning models not only rely on large-scale training data, but their diagnostic performance also heavily depends on the selection of hyperparameters, often requiring repeated adjustments to achieve ideal results. Furthermore, they exhibit poor diagnostic stability in scenarios with scarce samples. To address this issue, this invention selects TabPFN as the fault diagnosis model. TabPFN is a foundational model for tabular data. Its core innovation lies in transforming the traditional training paradigm from training a model separately for each dataset to pre-training a general algorithm and then adapting it to new tasks through context learning. This shift gives it superior performance and efficiency compared to traditional methods in tasks with small sample tabular data, making it highly suitable for scenarios with scarce and imbalanced samples in transformer DGA fault diagnosis.

[0044] The construction and reasoning process of the fault diagnosis model specifically includes the following two stages: (1) Pre-training process based on prior distribution: To enable the model to generalize to various data distributions, we first construct multiple tabular data generation tasks with different statistical properties, including variable distribution forms, correlations between features, and noise levels, thus forming a prior distribution for data generation. Specifically, this prior distribution simulates the diversity of real-world data from the following three dimensions: 1) Distribution form of variables: For each feature variable, feature values ​​are generated by randomly sampling from its corresponding distribution family. Commonly used distribution families include, but are not limited to: a: Normal distribution: (1); Among them, the mean from Randomly selected from the middle, standard deviation Uniform sampling is performed from [0.1, 2.0].

[0045] b: Uniform distribution: (2); Among them, the lower boundary and the Upper Realm Randomly generated, satisfying To cover different value ranges.

[0046] c: Log-normal distribution (3); in, , It is used to simulate gas concentration variables with skewed distribution characteristics, such as acetylene (C2H2) concentration, which is usually extremely low and right-skewed during non-discharge faults.

[0047] d: Exponential distribution: (4); Among them, the rate parameter It is used to simulate a few extreme value scenarios.

[0048] 2) Correlation between features: To simulate the coexistence and constraints between different gases in real-world scenarios (such as the co-occurrence of C2H4 and CH4 in thermal faults), a random covariance matrix is ​​generated. To characterize the linear correlation structure between features. Specifically, first, a... random orthogonal matrix and diagonal matrix Among them, each eigenvalue Then construct: (5); This yields covariance matrices with different correlation directions and strengths, and correlation coefficients between features. The range of values ​​covers .

[0049] 3) Noise level: Different levels of observation noise are added to the generated data to simulate sensor measurement errors and random disturbances in the transformer operating environment. The noise term is defined as: (6); Among them, the noise standard deviation The noise level is randomly selected based on the signal-to-noise ratio (SNR) level. This embodiment sets three noise levels, as shown in Table 3:

[0050] By randomly combining the above three dimensions, a large amount of synthetic tabular data with significantly different statistical characteristics is generated for training tasks. The resulting prior distribution of data generation can cover various data distribution patterns that may be encountered in transformer DGA fault diagnosis. During the pre-training phase, the Transformer model learns from these diverse tasks, implicitly encoding various data generation mechanisms into the model parameters. This allows TabPFN to achieve strong generalization ability on unseen small-sample DGA data. This mechanism is the fundamental reason why TabPFN can achieve stable diagnosis without iterative training under small-sample conditions.

[0051] To intuitively represent the diversity of prior distributions relied upon in the TabPFN pre-training phase, this embodiment uses MATLAB to generate a set of comprehensive simulation graphs covering three dimensions: variable distribution form, correlation between features, and noise level. Figure 9 Subplot (a) shows the probability density curves obtained by random sampling from four distribution families: normal, log-normal, uniform, and exponential. The normal distribution represents a symmetric, concentrated variable; the log-normal distribution exhibits a right-skewed tail, consistent with the skewed characteristic of acetylene (C2H2) concentration in transformer DGAs being extremely low but occasionally showing high values ​​during non-discharge faults; the uniform distribution simulates scenarios with equal probability within the sensor's range; and the exponential distribution reflects the distribution characteristics of fault gas approaching zero under normal conditions but potentially decaying exponentially. This multi-distribution hybrid generation strategy allows the model to internalize adaptability to various marginal distributions during pre-training.

[0052] Figure 9Subplot (b) is a heatmap of the correlation matrix of randomly generated 5-dimensional features (corresponding to H2, CH4, C2H6, C2H4, and C2H2), with correlation coefficients ranging from -0.8 to 0.9. Dark red blocks in the figure represent strong positive correlations, simulating the co-growth relationship between ethylene (C2H4) and methane (CH4) in thermal faults; dark blue blocks represent negative or weak correlations, simulating the differentiated characteristics of gas production rates under different fault types. By continuously varying this correlation structure during pre-training, TabPFN can learn to capture reliable dependencies from a limited sample, avoiding misjudging accidental correlations as diagnostic patterns.

[0053] Figure 9 Subgraph (c) uses two-dimensional features as an example to show the sample distribution after superimposing three levels of noise (low noise SNR=25dB, medium noise SNR=15dB, and high noise SNR=5dB) on the same underlying structure. It can be seen that under low noise, the clusters are clearly distinguishable; under medium noise, the cluster boundaries begin to blur but are still roughly separated; and under high noise, the clusters overlap severely. Injecting these three levels of noise into the synthetic data simulates various operating conditions, from high-precision laboratory monitoring to field-based strong electromagnetic interference aging sensors. This enables TabPFN to have automatic noise reduction and robust discrimination capabilities during inference, and it can stably output the correct fault type even when faced with measurement fluctuations common in DGA data.

[0054] In summary, Figure 9 The prior distribution construction mechanism presented is the theoretical basis for TabPFN to achieve high accuracy and strong robustness in small-sample transformer fault diagnosis without iterative training. By covering a wide range of distribution forms, correlation patterns, and noise levels, the model implicitly encodes this diversity into the network parameters after pre-training. Thus, even when facing DGA tasks with scarce data and class imbalance, it can still accurately capture the complex mapping relationship between dissolved gases in oil and fault types.

[0055] Based on this, the Transformer model is used for offline training, enabling the model to learn the mapping relationship from the training dataset to the prediction results. The goal is to approximate the Bayesian posterior prediction distribution, as shown in the following formula: (7); In equation (7), For the training dataset, For the sample to be predicted, For the corresponding tags, This represents the latent model parameters. Through this process, the model can implicitly encode multiple latent data generation mechanisms in the parameters, thereby avoiding explicit estimation of specific parameters.

[0056] (2) Context-based reasoning mechanism: In actual fault diagnosis, the training data obtained in step S1, along with the test samples, are input into the model as unified contextual information. The model models the relationship between the training samples and the test samples based on a self-attention mechanism, thereby directly outputting the conditional probability distribution.

[0057] Its reasoning process can be expressed as: (8); In equation (8), This represents the pre-trained Transformer model. Its parameters are then determined. Finally, the final prediction result is obtained based on the maximum a posteriori probability principle: (9).

[0058] Specifically: In the inference phase of step S2, TabPFN constructs unified contextual information using the training set obtained in step S1 and the test samples. It then utilizes the Transformer's self-attention mechanism to model the complex relationship between the training samples and the test samples, directly outputting the conditional probability distribution, thereby achieving a single forward diagnosis without parameter updates. The mathematical process is as follows: Set training set ,in: The concentrations of five characteristic gases, This is a fault label. For the sample under test... ,Will and spliced ​​into a length of The unified context sequence is input to the pre-trained Transformer. The model uses a self-attention mechanism to calculate the interaction weights between any two elements in the sequence, thereby modeling the similarity structure and class associations between training samples and between training samples and test samples, and finally outputting the conditional probability distribution under a given context: (10); The diagnostic result is given by the maximum a posteriori probability: (11); The entire inference process involves only one forward propagation, requiring no iterative training or hyperparameter tuning.

[0059] The specific manifestation of self-attention relationship modeling is as follows: In each layer of the model, the embedding representation of the test sample is similar to the key of the training sample through the query, and the normalized attention weight is obtained. : (12); Weight Reflects the first The reference value of training samples for diagnosing test samples. Figure 10 A typical attention distribution is presented: training samples with similar gas patterns to the test sample (both being medium-low temperature overheating faults T1) receive high weights, while samples with large pattern differences (such as discharge faults) have their weights suppressed. This soft information filtering based on a self-attention mechanism enables the model to adaptively extract the most valuable evidence from a small number of labeled samples, forming accurate conditional probability estimates. Table 4 compares the essential differences between this contextual reasoning paradigm and the traditional machine learning paradigm from multiple dimensions.

[0060]

[0061] Figure 10 The horizontal axis represents the training samples arranged in descending order of attention weight, and the vertical axis represents the normalized weight values; high-weight samples correspond to T1 class training samples that are similar to the gas patterns of the test samples, while low-weight samples are discharge samples or normal samples.

[0062] Combine Table 4 and Figure 10 As can be seen, TabPFN constructs unified contextual information, dynamically captures the relationships between samples using a self-attention mechanism, and directly outputs the conditional probability distribution, cleverly transforming the small sample diagnosis problem into a contextual analogy problem, thereby achieving high stability and strong generalization ability in transformer rare fault scenarios.

[0063] This process eliminates the need for iterative training or parameter updates for specific tasks, completing inference with a single forward propagation, thus significantly improving diagnostic efficiency. Compared to traditional fault diagnosis methods based on machine learning or deep learning, the TabPFN model does not require repeated training on specific datasets. Instead, it acquires general inference capabilities through pre-training, maintaining high diagnostic accuracy and stability even under conditions of small samples and class imbalance. Furthermore, its context-based inference mechanism makes fuller use of limited sample information, overcoming the problems of overfitting and insufficient generalization ability in data-scarce scenarios found in existing methods.

[0064] To verify the diagnostic stability of this invention under small sample imbalance conditions, accuracy, precision, recall, and F1 score were selected as evaluation metrics, and each metric was defined as follows: (13); (14); (15); (16); In the formula, TP is the number of samples that are actually positive and are correctly predicted as positive, TN is the number of samples that are actually negative and are correctly predicted as negative, FP is the number of samples that are actually negative but are incorrectly predicted as positive, and FN is the number of samples that are actually positive but are incorrectly predicted as negative.

[0065] In this embodiment, the TabPFN model was context-set using the training set under the three imbalanced validation scenarios shown in Table 2, and its diagnostic performance was evaluated on the corresponding test sets. Simultaneously, three mainstream ensemble learning models—Random Forest (RF), XGBoost, and LightGBM—were selected for comparison, and the same experiments were conducted. Figure 1 The confusion matrix of the TabPFN model on the test set is shown. It can be seen that the classification accuracy of each fault category is high. TabPFN achieved accurate classification in 112 test samples, with an overall accuracy of 89.29%, indicating that TabPFN can effectively capture the complex mapping relationship between dissolved gases in oil and fault types. Figure 3 The ROC curves of different models are shown. The TabPFN model has an AUC value of 0.978, indicating that its overall classification performance is better than the comparison models. Figure 4 A comprehensive comparison of the performance of various models in different scenarios shows that TabPFN maintains a leading position in terms of accuracy, recall, and F1 score, especially in the highly imbalanced scenario 3, which demonstrates its significant advantage. This fully verifies the diagnostic stability and superiority of the proposed method under small sample imbalance conditions.

[0066] Step S3: Perform t-SNE dimensionality reduction on the characteristic gas data in the test set, and combine the profile coefficient to analyze the distribution pattern and separability of different fault types in the feature space.

[0067] To further verify the effectiveness of the features learned by the TabPFN model, this invention employs a method combining t-SNE dimensionality reduction and silhouette coefficients to perform spatial distribution analysis on the classification results of the TabPFN model on the test set. The classification results of the TabPFN model on the test set samples from step S2, along with their corresponding original feature gas data, are selected, and nonlinear dimensionality reduction processing is performed on the multidimensional features. t-SNE establishes a probability distribution mapping relationship between high-dimensional and low-dimensional spaces, ensuring that similar samples maintain a proximity relationship in the low-dimensional space.

[0068] The core idea of ​​t-SNE is to establish a probability distribution mapping relationship between high-dimensional and low-dimensional spaces, so that similar samples maintain a proximity relationship in the low-dimensional space. Specifically, it consists of the following two steps: (1) High-dimensional space similarity modeling: For any two samples in a high-dimensional space and First, calculate the conditional probabilities between them. , indicating sample choose The probability of being its neighbor is modeled based on a Gaussian distribution: (17); in, Based on samples The Gaussian variance centered at the center is adaptively determined through the perplexity parameter. Furthermore, the joint probability is obtained through symmetry processing. : (18); In the formula, This represents the total number of samples in the test set.

[0069] (2) Low-dimensional space similarity modeling: In the low-dimensional embedding space, samples and The similarity between them is modeled using a t-distribution with 1 degree of freedom, i.e., a Cauchy distribution: (19); The t-distribution has a heavier tail than the Gaussian distribution, which allows for greater capacity for distant pairs of points in the low-dimensional space, thus effectively alleviating the "crowding problem" and making different types of clusters more dispersed in the low-dimensional space.

[0070] (3) t-SNE minimizes the high-dimensional distribution With low-dimensional distribution The Kullback-Leibler divergence between the two dimensions ensures that the sample distribution in the low-dimensional embedding space preserves the neighborhood structure in the high-dimensional space as faithfully as possible. (20); In the formula, Represents samples in high-dimensional space With sample The similarity probability, This represents the corresponding probability in the low-dimensional embedding space. The KL divergence imposes a significant penalty for incorrectly widening the gap between high-dimensional similar points in the low-dimensional space, when... large and When the gradient is small, the penalty for incorrectly bringing dissimilar points closer together is relatively small. This asymmetric penalty mechanism makes t-SNE excellent at preserving local neighborhood structure, making it suitable for revealing the clustering distribution of fault features in low-dimensional space.

[0071] This mapping projects the test set samples onto a two-dimensional plane and combines them with the fault category labels predicted by TabPFN for visualization. Figure 2 The dimensionality reduction visualization results show that the sample points of different fault types form clear clusters on the two-dimensional plane, and the decision boundaries between each category are obvious, revealing that the fault features have a good degree of separability.

[0072] Based on this, a silhouette coefficient is introduced to quantitatively evaluate the category structure formed by the TabPFN classification results, which is defined as: (twenty one); In the formula, Indicates sample The average distance between it and other samples of the same type Indicates sample The silhouette coefficient is the average distance between the nearest neighbor and all samples of the opposite class. The silhouette coefficient ranges from -1 to 1; a higher value indicates better inter-class separation and intra-class compactness. The calculation results in this embodiment show that the silhouette coefficients for all classes are at a high level, quantitatively verifying the excellent separability of fault features and providing an intuitive and objective basis for the reliability and effectiveness of the diagnostic model.

[0073] The average silhouette coefficients of all test set samples were taken to obtain the average silhouette coefficients of each category, as shown in Table 5.

[0074]

[0075] As shown in Table 5, the average silhouette coefficients for each category are all above 0.65, with an overall mean of 0.74. The normal state (N) and partial discharge (PD) have the highest silhouette coefficients, reaching 0.85 and 0.81 respectively, indicating that these two types of samples form extremely compact and clearly defined clusters in the feature space. The silhouette coefficient for low-energy discharge (D1) is relatively low (0.65), which is consistent with the physical characteristic that its fault characteristics overlap with those of high-energy discharge (D2) in terms of gas composition (low-energy discharge may further evolve into high-energy discharge). Overall, the quantitative results of the silhouette coefficients fully verify that the features learned by the TabPFN model have good class separability.

[0076] Figure 11 The two-dimensional spatial distribution of the test set samples after t-SNE dimensionality reduction is shown. Different colored scatter points represent different fault types, and each category exhibits a clear clustering structure and distinct decision boundaries. Combined with the quantitative results of the silhouette coefficient in Table 5, the following analysis can be derived: a: Intra-class compactness: Samples of the same class are clustered tightly in two-dimensional space, such as in classes PD and N, which is consistent with their high silhouette coefficients (0.81 and 0.85). This compactness indicates that the TabPFN model can extract discriminative features with consistent expressive power for the same type of fault from five gas features.

[0077] b: Inter-class separation: The boundaries between different categories are obvious, especially the discharge faults (PD, D1, D2) and the overheating faults (T1, T2) are naturally separated in space, which is consistent with the physical mechanism of transformer faults; discharge faults are characterized by C2H2 as the main gas, while thermal faults are characterized by C2H4 and CH4, and the two have natural differences in gas composition.

[0078] c: Transition region: There is a small number of samples in the transition region between T1 and T2, which is consistent with the gradual evolution process from medium and low temperature overheating to high temperature overheating, reflecting the physical nature of the continuous change in the severity of the fault.

[0079] Figure 11 The six colors represent six categories: T1, T2, PD, D1, D2, and N. Each category forms a clear cluster, and the decision boundaries between categories are obvious.

[0080] Step S4: Introduce the KernelSHAP method to perform local interpretability analysis on the diagnostic results of the fault diagnosis model, and quantify the contribution of each characteristic gas to the fault type determination.

[0081] To enhance the transparency and reliability of diagnostic results, this invention introduces the KernelSHAP method to perform local interpretability analysis on the classification results of the TabPFN model on the test set. Based on the prediction results of the TabPFN model for the test samples in step S2, a local interpretation model is constructed on a per-sample basis. KernelSHAP establishes a linear approximation relationship between input features and model output by weighted sampling of feature subsets. Its interpretation model can be expressed as: (twenty two); In the formula, This represents the output of the explanatory model, where $z'$ is a binary vector indicating the presence or absence of a feature. The total number of characteristic gases. Baseline value, For the first The marginal contribution of each characteristic gas to the prediction results. Contribution values ​​for each characteristic gas. Satisfying the Shapley value allocation principle, its calculation form is as follows: (twenty three); In the formula, It is the collection of all characteristic gases. Not containing characteristic gases Feature subset, This is the prediction function of the TabPFN model for a given subset of features.

[0082] By calculating the Shapley values ​​corresponding to each characteristic gas, the degree of positive or negative influence of different features on the prediction results can be obtained, forming a local interpretation result for a single sample. Based on this, statistical analysis of the feature contribution distribution of samples with different fault types can identify the dominant role of key gas features and the variation law of their influence intensity under different fault modes.

[0083] Taking the medium-low temperature overheating fault (T1) as an example, Figure 5 The distribution of Shapley values ​​for each characteristic gas under this fault type is shown. It can be seen that C2H4 and CH4 usually have high positive Shapley values ​​and are key positive features for identifying this fault, while C2H2 shows a significant negative contribution. Figure 6 The study further presents Shapley dependency plots for various characteristic gases in low- and medium-temperature overheating faults. These plots identify the threshold effects of key features on model output; for example, when the C2H4 content exceeds a certain threshold, its contribution to model predictions increases significantly. This type of analysis not only effectively answers the question of "which features contribute significantly to the diagnostic results," providing valuable local explanations for model decision-making, but also offers maintenance personnel intuitive decision-making support, helping to transform complex model outputs into actionable equipment condition monitoring rules.

[0084] Step S5: The Non-combinatorial Optimization via Trace Exponential and Augmented lagRangian for Structure learning (NOTEARS) algorithm is used to construct a causal relationship network among the characteristic gas variables under different fault types, and the key causal nodes in the causal relationship network are verified to play a dominant role in fault diagnosis through feature removal experiments.

[0085] Existing interpretability methods mostly focus on statistical correlation analysis between features and fault types, making it difficult to further reveal the deep causal evolutionary relationships between gas variables. To overcome this limitation, this invention further introduces the trace index and augmented Lagrange method's non-combinatorial optimization structure learning (NOTEARS) algorithm to construct a causal correlation network between gas variables under different fault types, in order to reveal the causal structural relationships in the gas evolution process.

[0086] This step constructs a variable set based on the characteristic gas data obtained in step S1. Each variable corresponds to a different characteristic gas. By learning the structural relationships between the variables, a causal model in the form of a Directed Acyclic Graph (DAG) is established, the structure of which can be represented by a weighted adjacency matrix. This is indicated. NOTEARS solves for this structure by optimizing the following objective function: (twenty four); In the formula, The loss function is a data-based fitting function. For sparse constraint terms, is the regularization coefficient, used to control the sparsity of the network structure.

[0087] To ensure that the learned graph structure satisfies the acyclic constraint, a continuously differentiable acyclic constraint function is introduced: (25); In the formula, Represents the trace of a matrix. For matrix exponentiation operations, Represents the Hadamard product. The number of variables is used to ensure that the resulting causal structure is strictly a directed acyclic graph.

[0088] Based on this, a causal relationship network among various gas variables is obtained, and by analyzing the weights and directions of edges in the adjacency matrix, key causal relationship chains under different fault types are identified. Figure 7 A causal relationship diagram between different faults and normal states is presented. Taking the medium-low temperature overheating fault as an example, a clear causal relationship exists in the causal network. For instance, changes in the concentrations of CH4 and C2H6 may be the cause of changes in the C2H4 concentration, and changes in C2H4 are directly related to the fault state. This step elevates the interpretability from the traditional feature contribution level to the causal reasoning level, explaining the intrinsic generation and transformation relationship of dissolved gases in the oil when a fault occurs from a mechanistic perspective, providing a complete and traceable basis for diagnostic results from feature contributions to the causal chain.

[0089] To verify the dominant role of key nodes in fault diagnosis in causal networks, this invention designed a feature removal experiment. Figure 8The quantitative evaluation results of feature removal are presented. Specifically, the process involves removing the feature gas data corresponding to key causal nodes identified in the causal network from the training set, retraining, and evaluating the change in diagnostic accuracy of the fault diagnosis model. The results show that removing key causal nodes (such as C2H4) significantly reduces the model's diagnostic accuracy, with a greater decrease compared to randomly removing the same number of non-key features. This quantitative experiment confirms the dominant role of core nodes in the causal network in fault determination, further strengthening the reliability of the causal explanation.

[0090] In summary, this invention, through the organic combination of steps S1 to S5, fully leverages the stable inference capability of TabPFN under small sample conditions and innovatively constructs a multi-level interpretable framework from KernelSHAP local contribution analysis to NOTEARS causal network inference. This method not only provides high-precision fault diagnosis results but also endows the diagnostic results with complete traceability, from the quantitative contribution and threshold effect of feature influences to the causal evolution chain between gases, providing a solid technical basis for intelligent operation and maintenance of transformers and in-depth analysis of fault mechanisms.

Claims

1. An interpretable transformer fault diagnosis method based on TabPFN and causal correlation, characterized in that Includes the following steps: S1: Collect dissolved gas concentration data in oil-immersed transformer oil, construct an initial sample set containing characteristic gases and corresponding fault type labels, and divide the initial sample set into a training set and a test set; S2: Based on the training set, a fault diagnosis model is constructed by fitting the TabPFN network with tabular prior data, and the diagnostic stability of the fault diagnosis model is evaluated under various small-sample imbalanced validation scenarios. S3: Perform t-SNE dimensionality reduction on the characteristic gas data of the test set, and combine the profile coefficient to analyze the distribution pattern and separability of different fault types in the feature space; S4: The KernelSHAP method is introduced to perform local interpretability analysis on the diagnostic results of the fault diagnosis model and to quantify the contribution of each characteristic gas to the fault type determination. S5: The NOTEARS algorithm is used to construct a causal relationship network among characteristic gas variables under different fault types, and the key causal nodes in the causal relationship network are used to verify the dominant role of fault diagnosis through feature removal experiments.

2. The transformer fault diagnosis method based on TabPFN and causal correlation according to claim 1, characterized in that: S1 includes: S1.1: Collect dissolved gas monitoring data in oil-immersed transformer oil, wherein the characteristic gases include hydrogen (H2), methane (CH4), ethane (C2H6), ethylene (C2H4), and acetylene (C2H2); S1.2: Clean the collected data to remove outliers and label it according to the preset fault types, which include six categories: medium and low temperature overheating fault (T1), high temperature overheating fault (T2), partial discharge (PD), low energy discharge (D1), high energy discharge (D2) and normal (N). S1.3: Divide the labeled dataset into training and test sets according to a preset ratio to construct the initial sample set.

3. The interpretable transformer fault diagnosis method based on TabPFN and causal correlation according to claim 2, characterized in that: In step S2, the construction and reasoning process of the fault diagnosis model includes: S2.1: Based on a prior distribution containing multiple data distribution forms, a tabular data training task is generated. The Transformer model is used for offline pre-training to learn the mapping relationship from the training dataset to the prediction results, thereby approximating the Bayesian posterior distribution. The Bayesian posterior distribution is represented as: ； In the formula, is a training data set, is a sample to be predicted, is a corresponding label, denotes a latent model parameter; S2.2: In actual diagnosis, the training set and the sample to be tested are jointly input as context into the pre-trained Transformer model , which directly outputs the conditional probability distribution based on the self-attention mechanism, and the inference process is represented as: ； In the formula, This represents the pre-trained Transformer model. These are the parameters of the Transformer model after pre-training; The final prediction result is obtained based on the maximum a posteriori probability principle: 。 4. The interpretable transformer fault diagnosis method based on TabPFN and causal correlation according to claim 3, characterized in that: In S2, the evaluation metrics used to assess the diagnostic stability of the fault diagnosis model include accuracy, precision, recall, and F1 score. The definitions of each metric are as follows: ； ； ； ； In the above formula, TP is the number of samples that are actually positive and are correctly predicted as positive, TN is the number of samples that are actually negative and are correctly predicted as negative, FP is the number of samples that are actually negative but are incorrectly predicted as positive, and FN is the number of samples that are actually positive but are incorrectly predicted as negative.

5. The interpretable transformer fault diagnosis method based on TabPFN and causal correlation according to claim 4, characterized in that: In S2: various small-sample imbalanced verification scenarios include mild imbalance, moderate imbalance and high imbalance scenarios, and the proportion of different fault types of samples in the training set is different in each scenario.

6. The interpretable transformer fault diagnosis method based on TabPFN and causal correlation according to claim 5, characterized in that: S3 includes: S3.1: The t-SNE method is used to perform nonlinear dimensionality reduction on the original feature gas data of the test set, by minimizing the sample similarity probability distribution in the high-dimensional space. Probability distribution corresponding to low-dimensional space The Kullback-Leibler divergence between samples is used to project the samples into a low-dimensional space, and its optimization objective is: ； In the formula, Represents samples in high-dimensional space With sample The similarity probability, Represents samples in a low-dimensional embedding space With sample The corresponding probability; S3.2: Introducing a silhouette coefficient to quantitatively evaluate the dimensionality-reduced category structure, the silhouette coefficient being defined as: ； In the formula, Indicates sample The average distance between it and other samples in the same category Indicates sample The silhouette coefficient is the average distance between the nearest neighbor samples of different classes. The value ranges from [-1, 1]. The larger the value, the higher the inter-class separation and the better the intra-class compactness of the classification result.

7. The interpretable transformer fault diagnosis method based on TabPFN and causal correlation according to claim 6, characterized in that: S4 includes: S4.1: Based on the prediction results of the fault diagnosis model, a local explanatory model is constructed with a single sample as the unit. The feature subset is weighted and sampled using the KernelSHAP method to establish a linear approximation relationship between the input features and the model output. The explanatory model is expressed as follows: ； In the formula, To explain the model, A binary vector indicating the presence or absence of a feature. The total number of characteristic gases. Baseline value, For the first The marginal contribution of each characteristic gas to the prediction results; S4.2: Marginal contribution of each characteristic gas Satisfying the Shapley value allocation principle, its calculation form is as follows: ； In the formula, It is the collection of all characteristic gases. , Not containing characteristic gases Feature subset, This is the prediction function of the fault diagnosis model under a given feature subset.

8. The interpretable transformer fault diagnosis method based on TabPFN and causal correlation according to claim 7, characterized in that: The S4 further includes: performing statistical analysis on the distribution of characteristic gas contribution values ​​of samples of different fault types, identifying key characteristic gases with positive and negative contributions under each fault mode, and identifying the threshold effect of key characteristic gases on model output through SHAP dependency graph.

9. The interpretable transformer fault diagnosis method based on TabPFN and causal correlation according to claim 8, characterized in that: S5 includes: S5.1: Based on the characteristic gas data obtained in S1, a variable set is constructed. The NOTEARS algorithm is used to learn the structural relationships between the variables, and a causal model in the form of a directed acyclic graph is established. Its structure consists of a weighted adjacency matrix. This is expressed as follows, and the solution is obtained by optimizing the following objective function: ； In the formula, The loss function is a data-based fitting function. For sparse constraint terms, is the regularization coefficient, used to control the sparsity of the network structure; S5.2: Introduce a continuously differentiable acyclic constraint function to ensure that the learned graph structure is a directed acyclic graph. The acyclic constraint function is: ； In the formula, Represents the trace of a matrix. For matrix exponentiation operations, Represents the Hadamard product. The number of variables.

10. The interpretable transformer fault diagnosis method based on TabPFN and causal correlation according to claim 9, characterized in that: The S5 also includes: S5.3: Analyze the weights and directions of edges in the causal relationship network to identify key causal relationship chains under different fault types; S5.4: Verify the dominant role of key causal nodes through feature removal experiments; The feature removal experiment includes: removing the feature gas data corresponding to the key causal nodes identified in the causal association network from the training set, retraining and evaluating the change in the diagnostic accuracy of the fault diagnosis model. The greater the decrease in diagnostic accuracy, the stronger the dominant role of the key causal node in fault diagnosis.

Images