A fault diagnosis method and system for oil-immersed transformers based on feature enhancement and deep heterogeneity

Abstract

The present application relates to the technical field of oil-immersed transformer fault diagnosis, in particular to an oil-immersed transformer fault diagnosis method and system based on feature enhancement and deep heterogeneity. The method comprises the following steps: using a minority class oversampling method to synthesize new samples conforming to physical constraints from minority fault sample data; constructing an enhanced input space based on deep features generated by a self-encoder VAE to fully extract deep features of fault gases in the new samples; based on the extracted deep features, using a heterogeneous ensemble model to simulate a light gradient boosting machine LightGBM to capture shallow relationships of deep features; using a convolutional neural network 1D-CNN to identify local features of fault gases; establishing global dependency relationships based on a Transformer model; and significantly enhancing the robustness of the model through a dynamic weighting method, thereby ensuring that the model continuously outputs reliable and stable diagnosis results in various complex and highly uncertain actual application scenarios.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of oil-immersed transformer fault diagnosis, and in particular to an oil-immersed transformer fault diagnosis method and system based on feature enhancement and deep heterogeneity. BACKGROUND

[0002] Oil-immersed transformers have the advantages of good heat dissipation, low load loss, and low initial cost, and are widely used in power grid transmission, steel, chemical industry and other industrial fields, especially in high-voltage and extra-high-voltage transmission fields. When the insulation oil filled in the oil-immersed transformer is cracked and aged, hydrogen (H2), acetylene (C2H2), ethylene (C2H4), ethane (C2H6), carbon monoxide (CO) and other gases will be dissolved in the oil. By analyzing the types and contents of the dissolved gases, the internal state of the transformer can be determined. At present, the relatively mature oil-immersed transformer diagnosis method is the dissolved gas analysis method (DGA), but it can only diagnose three types of faults and has a high misjudgment rate for mixed faults, which cannot meet the demand for rapid response to faults after the expansion of the power grid.

[0003] Artificial intelligence algorithms can be deployed on edge computing devices or cloud servers to analyze and diagnose real-time collected transformer operation data in real time, discover potential faults in time and issue warning signals, thereby significantly improving the reliability and safety of transformer operation. As a core component of modern artificial intelligence, machine learning has the ability of self-optimization and continuous performance improvement. Due to the scarcity of oil-immersed transformer fault samples, the generalization ability of deep learning is limited. First, (1) the existing methods have weak fault gas feature extraction ability, and are mostly limited to simple judgment or ratio calculation based on static concentration threshold, the feature processing method is relatively simple, and there is a lack of deep mining and fusion of multi-dimensional feature space. In complex fault scenarios, traditional methods cannot fully capture the time sequence dynamic characteristics hidden in fault gases, resulting in insufficient fault recognition sensitivity and high misjudgment rate.

[0004] (2) Existing researches mostly rely on single machine learning model, which limits the model generalization ability and makes it difficult to cope with complex and variable actual scenarios. Once the data distribution deviates or there is noise, the model performance will decrease sharply, and it lacks robustness and explainability, cannot effectively fuse multi-source information, is prone to overfitting, and ultimately limits the prediction accuracy and practical application value.

[0005] In view of the problems of uneven sample distribution, insufficient feature information utilization and limited model generalization performance in current oil-immersed transformer fault diagnosis, it is urgent to propose a fault diagnosis model based on mixed feature enhancement and deep heterogeneity integration. SUMMARY

[0006] In order to solve the problems of uneven sample distribution, insufficient feature information utilization and limited model generalization performance in current oil-immersed transformer fault diagnosis, the application provides an oil-immersed transformer fault diagnosis method and system based on feature enhancement and deep heterogeneity.

[0007] In the first aspect, the application provides an oil-immersed transformer fault diagnosis method based on feature enhancement and deep heterogeneity, which adopts the following technical scheme:

[0008] An oil-immersed transformer fault diagnosis method based on feature enhancement and deep heterogeneity comprises:

[0009] Obtain a few fault sample data of an oil-immersed transformer;

[0010] Construct an oil-immersed transformer fault diagnosis model based on mixed feature enhancement and deep heterogeneity integration, wherein a few fault sample data is synthesized into new samples conforming to physical constraints by using a few class oversampling method;

[0011] Construct an enhanced input space based on deep features generated by the autoencoder VAE, and fully extract deep features of fault gases in the new samples;

[0012] Based on the extracted deep features, use a heterogeneity integration model to simulate a light gradient boosting machine LightGBM to capture the shallow relationship of deep features, use a convolutional neural network 1D-CNN to identify the local features of fault gases, and establish global dependency relationship based on a Transformer model;

[0013] Adjust the model output weight based on a dynamic fusion mechanism;

[0014] Optimize the constructed oil-immersed transformer fault diagnosis model;

[0015] Output fault diagnosis results by using the optimized fault diagnosis model.

[0016] Furthermore, the construction of the oil-immersed transformer fault diagnosis model based on hybrid feature enhancement and deep heterogeneous integration includes adopting an adaptive sample generation method to dynamically adjust neighborhood parameters based on the distribution characteristics of minority class samples. By synthesizing new samples that conform to physical constraints, the gas concentration is ensured to be non-negative while maintaining the original data distribution pattern. Then, a ten-dimensional expert knowledge system is constructed in feature engineering, integrating five gas concentrations from existing data and introducing fault feature ratios, discharge indication ratios, thermal fault ratios, and total hydrocarbon concentrations. This is coupled with four-dimensional deep features generated by a variational autoencoder (VAE), ultimately forming a 14-dimensional enhanced input space. Subsequently, a heterogeneous integrated model architecture is used to fuse three types of learners: the first branch simulates LightGBM to capture shallow relationships between features; the second branch uses a one-dimensional convolutional neural network (1D-CNN) to identify local features of fault gases; and the third branch is based on a Transformer model to establish global dependencies. Finally, a dynamic fusion mechanism is used to achieve synergy among the three models through attention weighting.

[0017] Furthermore, the method of synthesizing new samples that conform to physical constraints from minority fault sample data using minority class oversampling includes introducing the SMOTE oversampling algorithm based on dynamic adjustment of k-nearest neighbors. This algorithm mitigates the class imbalance problem in the original data by adaptively generating synthetic samples. The SMOTE algorithm avoids overfitting risk by generating synthetic samples through interpolation, and is expressed as: Let the minority sample set S = {xi ∈ R} d |i=1,…,N s}, where d is the feature dimension; for Calculate its k-nearest neighbor set N k (x i )={x i (1) ,…,x i (k) The distance metric used is Mahalanobis distance. Σ is the covariance matrix of the minority class samples; finally, linear interpolation is performed: from N k Randomly select a nearest neighbor sample x nn Generate a new sample x new : ,in, , which is a random interpolation number that controls the position of the new sample in the feature space.

[0018] Furthermore, the deep feature construction based on the autoencoder VAE enhances the input space, fully extracting the deep features of the fault gas in the new samples. This includes supplementing the original feature gas with five-dimensional features to weaken the type confusion problem and enhance the representation ability of minority fault samples. To eliminate dimensional differences, the original gas concentration and construction ratio are Z-score standardized. , where μ is the mean of the x data set; σ is the standard deviation of the data set elements; x' is the result after x is standardized; then a variational autoencoder VAE is used to learn the data distribution while performing feature extraction and data augmentation, wherein the VAE is composed of an encoder, a decoder and a reparameterization process, the encoder is used to map the input data to the latent space to obtain the mean and variance; the decoder maps the sample in the latent space back to the data space to reconstruct the input data; the reparameterization process introduces randomness to enable the model to learn the feature representation, which is represented as: first, the encoder is used to map the input data x to the mean μ and variance logσ of the latent space 2 : , where represents the parameters of the encoder; is the backpropagation gradient in the training process, and the reparameterization technique is introduced, which is represented as: , where is a random variable sampled from the standard normal distribution N(0, 1), and represents element-level multiplication; then the decoder is used to map the sample z in the latent space back to the data space to reconstruct the input data x: , where θ represents the parameters of the decoder; finally, the loss function is calculated, and the loss function of the VAE is composed of two parts of reconstruction loss and KL divergence, which is represented as:

[0019] ,

[0020] where β is a hyperparameter used to balance the reconstruction loss and the KL divergence.

[0021] Further, the heterogeneous ensemble model is used to simulate a LightGBM to capture shallow relationships of deep features, including using a fully connected network to simulate a gradient boosting decision tree to effectively capture high-order nonlinear interaction relationships between features, wherein a LightGBM branch module is used to simulate the decision logic of the LightGBM, a multi-layer fully connected neural network structure is used, specifically including two hidden layers, each layer uses a ReLU activation function to introduce nonlinearity, and the input feature vector is x∈R d , and the output is o∈R c , where d is the input dimension, c is the number of categories, and the forward propagation process is represented as:

[0022] ,

[0023] ,

[0024] ,

[0025] , W1∈R 256×d , W2∈R 128×256, W3∈R c×128 is a weight matrix, b1∈R 256 , b2∈R 128 , b3∈R c is a bias vector.

[0026] Further, the local features of the fault gas are identified by using the convolutional neural network 1D-CNN, including extracting local features in the sequence data by using 1D-CNN, wherein in the oil-immersed transformer fault diagnosis, the DGA data is regarded as a time series signal, and the 1D-CNN is used to capture the change mode of the gas concentration over time, and the 1D-CNN branch contains two convolutional layers and one pooling layer, first, the input features are expanded into single-channel sequences, and then the features are extracted by a convolution kernel:

[0027] ,

[0028] ,

[0029] wherein W1∈R 128×1×3 , W2∈R 64×128×3 is a convolution kernel weight, b1∈R 128 , b2∈R 64 is a bias, after the convolution operation, an adaptive max-pooling layer is used to compress the feature map into a fixed length, which is represented as:

[0030] ,

[0031] Finally, the output is obtained through a fully connected layer:

[0032] ,

[0033] wherein W3∈R c×64 is a weight matrix, b3∈R c is a bias vector.

[0034] Further, the global dependency relationship is established based on the Transformer model, including using the Transformer to capture the long-term dependency relationship between the DGA data at different time points in the oil-immersed transformer fault diagnosis, and the Transformer branch contains multiple layers of Transformer encoders, and each layer of encoder includes a multi-head self-attention mechanism and a feedforward neural network, for an input sequence X={x1,x2,…,x n}, wherein x i ∈R d , the calculation of the multi-head self-attention mechanism is represented as: first, the input sequence is mapped to a query (Q), a key (K) and a value (V) space: wherein WQ ,W K ,W V ∈R d×dk is a weight matrix, d k is the dimension of attention; then the attention score is calculated and normalized: , ; then the value vector is weighted and summed: , and finally the features are further processed by a feedforward neural network, denoted as: where W1∈R d×dff , W2∈R d×dff is a weight matrix, b1∈R dff , b2∈R d is a bias, d ff is the hidden layer dimension of the feedforward network; after obtaining the output of the encoder, the final output is obtained by global average pooling and a fully connected layer:

[0035] ,

[0036] where W3∈R c×d is a weight matrix, b3∈R c is a bias vector.

[0037] Further, the model output weight is adjusted based on the dynamic fusion mechanism, including using an attention fusion mechanism to dynamically adjust the contribution to the final prediction result by calculating the weight of each model output, where the output of each model is denoted as o i ∈R c , where c is the number of classes, the outputs are first concatenated into a vector: ; then it is mapped to a low-dimensional space by a fully connected layer: where W1∈R 64×3c is a weight matrix, b1∈R 64 is a bias vector, and then the weight of each model is calculated: where W2∈R 3×64 is a weight matrix, b2∈R 3 is a bias vector, and finally the final output is obtained by weighted summation:

[0038] ,

[0039] where w i is the weight of the i-th model.

[0040] Further, the constructed oil-immersed transformer fault diagnosis model is optimized, including adopting a feature engineering method to construct four new ratio features, including C2H2 / C2H4, CH4 / H2, C2H2 / C2H6, and C2H4 / C2H6, expanding the original 8-dimensional feature to a 12-dimensional feature space, setting the sample quantity batch size of each model training to 64, and configuring the total iteration number epochs during training to 500 rounds; the latent dimension of the VAE feature extractor is set to 4, in the power grid fault diagnosis architecture based on multi-model hybrid driving, the input dimension after feature fusion is set to 16, including 12-dimensional original features + 4-dimensional VAE features, and the number of heads of the Transformer branch is set to 4, and the hidden layer dimension of the attention mechanism is 64, so as to enhance the feature fusion capability of the model; finally, the learning rate of the model is set to 0.001, and the AdamW optimizer is used to control the update speed of the model parameters, so as to make the model stably converge to the optimal solution in the training process.

[0041] In a second aspect, an oil-immersed transformer fault diagnosis system based on feature enhancement and deep heterogeneity includes:

[0042] A data acquisition module is configured to acquire a small number of fault sample data of an oil-immersed transformer.

[0043] A model construction module is configured to construct an oil-immersed transformer fault diagnosis model based on mixed feature enhancement and deep heterogeneity integration, wherein a small number of fault sample data is synthesized into new samples conforming to physical constraints by using a small class oversampling method.

[0044] A feature enhancement module is configured to construct an enhanced input space based on deep features generated by a self-encoder VAE, and fully extract deep features of fault gases in the new samples.

[0045] A feature extraction module is configured to simulate a lightweight gradient boosting machine LightGBM to capture shallow relationships of deep features based on extracted deep features using a heterogeneous integration model; a convolutional neural network 1D-CNN is used to identify local features of fault gases; and a global dependency relationship is established based on a Transformer model.

[0046] A fusion module is configured to adjust model output weights based on a dynamic fusion mechanism.

[0047] An optimization module is configured to optimize the constructed oil-immersed transformer fault diagnosis model.

[0048] A diagnosis module is configured to output fault diagnosis results using the optimized fault diagnosis model.

[0049] In a third aspect, the present application provides a computer readable storage medium, wherein a plurality of instructions are stored, the instructions being adapted to be loaded by a processor of a terminal device and to execute the feature enhancement and deep heterogeneous oil-immersed transformer fault diagnosis method.

[0050] In a fourth aspect, the present application provides a terminal device, comprising a processor and a computer readable storage medium, the processor being configured to implement instructions; and the computer readable storage medium being configured to store a plurality of instructions, the instructions being adapted to be loaded by the processor and to execute the feature enhancement and deep heterogeneous oil-immersed transformer fault diagnosis method.

[0051] In summary, the present application has the following beneficial technical effects:

[0052] (1) In order to fully extract the fault gas features, a hybrid feature enhancement method of increasing fault feature dimension, VAE feature extraction and splicing is adopted. First, the fault type ratio features with physical meaning are artificially constructed according to expert knowledge and mechanism model, and the interpretability of the original signal is preserved. Then, the VAE is used to learn the low-dimensional latent distribution while preserving the reconstruction ability, and the implicit patterns that are difficult to express explicitly by the original features are captured. Finally, the artificial features and VAE latent features are adaptively spliced and normalized to form an enhanced feature vector that has both physical interpretability and high discriminability, thereby significantly improving the robustness and classification accuracy of the subsequent model on dynamic fault data.

[0053] (2) In order to improve the model's ability to mine fault gas information, a diagnosis architecture based on deep heterogeneous integration is innovatively designed and implemented. The core of the architecture consists of three complementary branch models. The LightGBM branch fully utilizes its advantages in efficiently processing structured data and is specifically used to capture the complex nonlinear mapping relationships and interactions between key features in the original sensor data or feature engineering generated data. The one-dimensional convolutional neural network (1D-CNN) branch focuses on extracting diagnostic meaningful local feature patterns from the time series of gas sensor responses using its inherent convolution operation. The Transformer branch relies on its powerful self-attention mechanism to effectively model and understand the global long-range dependencies within the sensor signal sequence, which is crucial for capturing the complete evolution of potential fault patterns in the time scale. Through this heterogeneous integration strategy, the model integrates the core modeling capabilities of tree models, convolution operations and self-attention mechanisms, thereby significantly improving the all-around representation and mining ability of the hidden diagnostic signals in the fault gas.

[0054] (3) In view of the inherent defects of traditional fusion strategies such as simple average fusion or artificial setting of fixed weight, which are prone to cause insufficient information utilization or misjudgment when integrating heterogeneous diagnostic models, an attention mechanism is innovatively introduced and designed. The mechanism can adaptively adjust the weight distribution ratio of each component module in the heterogeneous model according to the specific characteristics and context of the input data. This dynamic weighting method significantly enhances the robustness of the model, thereby ensuring its continuous output of reliable and stable diagnostic results in various complex and high-uncertainty practical application scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 is a schematic diagram of the fault diagnosis model of embodiment 1 of the present application;

[0056] Figure 2 is a schematic diagram of the double-path feature enhancement module of embodiment 1 of the present application;

[0057] Figure 3 is a structure diagram of the Transformer model Encoder of embodiment 1 of the present application;

[0058] Figure 4 is a schematic diagram of the comparative experiment of embodiment 1 of the present application;

[0059] Figure 5 is a schematic diagram of the ablation experiment of embodiment 1 of the present application. DETAILED DESCRIPTION

[0060] The present application will be further described in detail below with reference to the accompanying drawings.

[0061] Embodiment 1

[0062] With reference to Figure 1 , a fault diagnosis method for oil-immersed transformers based on feature enhancement and deep heterogeneity according to the present embodiment comprises:

[0063] 1. Firstly, in view of the inherent imbalance characteristics of industrial data, the adaptive sample generation (smote algorithm) technology is adopted. This method dynamically adjusts the neighborhood parameters based on the distribution characteristics of the minority class samples, synthesizes new samples that meet the physical constraints, ensures that the gas concentration is non-negative and maintains the original data distribution rule, and effectively alleviates the class imbalance problem. Secondly, a ten-dimensional expert knowledge system is constructed on the feature engineering: five gas concentrations are fused with the existing data, and the fault feature ratio, discharge indication ratio, thermal fault ratio and total hydrocarbon concentration are innovatively introduced. The feature set is coupled with the 4-dimensional deep features generated by the variational autoencoder (VAE), and finally forms a 14-dimensional enhanced input space, achieving the purpose of fully extracting deep features of fault gas.

[0064] Subsequently, a heterogeneous ensemble model architecture is adopted to innovatively integrate three types of learners: the first branch simulates LightGBM to capture the shallow relationships of features; the second branch uses a one-dimensional convolutional neural network (1D-CNN) to focus on identifying local features of fault gases; and the third branch is based on the Transformer model to establish global dependencies. This multi-level learning strategy ensures comprehensive extraction of fault information. However, this complex multi-branch model can have a serious impact on feature gas fault diagnosis if the learning weights are not allocated well, so after using the heterogeneous model, a dynamic fusion mechanism is adopted to achieve the synergy of the three models through attention weighting. This mechanism automatically assesses the contribution of each learner in different fault scenarios and generates an adaptive weight allocation scheme. The model proposed in this paper breaks through the limitations of traditional diagnostic models: it solves the problem of sample scarcity through generative data augmentation, improves information utilization through multi-source feature fusion, and enhances model generalization through a heterogeneous ensemble architecture, ultimately achieving accurate differentiation of complex fault patterns. The overall structure of the model is shown in Figure 1 .

[0065] 2, data synthesis module,

[0066] Since training a machine learning model often requires a large amount of data as support, but in transformer fault diagnosis, transformer faults are low-probability events with very low occurrence frequency, so it is very difficult to collect a large amount of balanced fault data. In oil-immersed transformer fault diagnosis, the proportion of severe fault samples such as discharge and high-energy discharge is usually low, which leads to traditional classification models such as support vector machines (SVM)

[0067] Decision trees and other samples are biased towards the majority class, and traditional oversampling (such as random duplication) can easily cause overfitting, and undersampling can lose effective information, which seriously affects the accuracy of fault recognition. To solve this problem, this study introduces the synthetic minority over-sampling technique (SMOTE), which aims to improve the decision space representation of minority class samples. The SMOTE algorithm generates synthetic samples through interpolation, rather than simply duplicating, to avoid the risk of overfitting. The mathematical process is as follows:

[0068] (1) Let the minority sample set S = {x d ∣i=1,…,N s} where d is the feature dimension.

[0069] (2) For , calculate its k-nearest neighbor set N k (x i ) = {x i (1) ,…,x i (k)where the distance metric adopts Mahalanobis distance (eliminate the influence of feature correlation):

[0070]

[0071] Σ is the covariance matrix of minority class samples.

[0072] (3) Linear interpolation: from N k Randomly select a neighbor sample x nn , generate new sample x new :

[0073]

[0074] where, is a random interpolation number, which controls the position of the new sample in the feature space.

[0075] 3, dual-channel feature enhancement module,

[0076] In the fault diagnosis of oil-immersed transformers, the following challenges are faced: First, in the traditional DGA gas ratio, only the typical fault mode can be represented, and the sensitivity to complex faults is insufficient, and the physical characteristics are limited. Second, the nonlinear correlation in the fault evolution process cannot be fully expressed by artificially constructing features. To solve the above problems, this paper will expand the complementarity of physical feature and deep feature extraction, construct a more discriminative feature space, and the dual-channel feature enhancement module is as follows Figure 2 .

[0077] On the basis of the original characteristic gas, according to the "Guide to the Analysis of Gases Dissolved in Oil of Electrical Equipment Immersed in Mineral Oil" formulated by the International Electrotechnical Commission (IEC) and the principle of chemical thermodynamics, five-dimensional features are supplemented to weaken the type confusion problem in traditional methods and enhance the representation ability of minority class fault samples. The five supplementary physical features are: C2H2 / C2H4, CH4 / H2, total hydrocarbon, C2H2 / C2H6 and C2H4 / C2H6. In order to eliminate the dimensional difference, the original gas concentration and the constructed ratio are standardized by Z-score:

[0078]

[0079] In the formula: μ is the mean of the x data set; σ is the standard deviation of the data set elements; x' is the result after x is standardized.

[0080] In transformer fault diagnosis, dissolved gas analysis (DGA) data has the characteristics of multi-class fault, class imbalance, and limited data volume. In order to improve the accuracy of fault diagnosis, effective feature extraction and enhancement of data are needed. Traditional feature extraction methods often fail to capture the complex distribution and potential features of data, while variational autoencoder (VAE) as a generative model can learn the distribution of data while extracting features and enhancing data. VAE consists of an encoder, a decoder, and a reparameterization process. The encoder maps the input data to the mean and variance of the latent space; the decoder maps the samples in the latent space back to the data space to reconstruct the input data; the reparameterization process introduces randomness to make the model learn more robust feature representations. The mathematical process is as follows:

[0081] (1) Encoder: The encoder is a neural network that maps the input data x to the mean μ and variance logσ of the latent space 2 :

[0082]

[0083] where θe represents the parameters of the encoder.

[0084] (2) Reparameterization: To enable gradient backpropagation during training, the reparameterization trick is introduced:

[0085]

[0086] where z = μ + σ ⊙ ε, ε is a random variable sampled from the standard normal distribution N(0, 1), and ⊙ denotes element-wise multiplication.

[0087] (3) Decoder: The decoder is also a neural network that maps the samples z in the latent space back to the data space to reconstruct the input data x:

[0088]

[0089] where θd represents the parameters of the decoder.

[0090] (4) Loss function: The loss function of VAE consists of two parts: reconstruction loss and KL divergence.

[0091]

[0092] where the reconstruction loss measures the difference between the reconstructed data and the original data, the KL divergence measures the difference between the latent space distribution and the prior distribution (usually the standard normal distribution), and β is a hyperparameter that balances the reconstruction loss and the KL divergence.

[0093] ​​In this module, the data expression ability is increased by physically expanding the characteristic gas dimension, and then the standardized data is input into the VAE. The original data is mapped to the latent space through the encoder of the VAE to obtain low-dimensional feature representation, and the features are combined with the original features to enhance the understanding ability of the model to the data and the diagnostic performance.

[0094] 3.4 Oil-immersed transformer fault diagnosis module based on heterogeneous model,

[0095] The core challenge of oil-immersed transformer fault diagnosis lies in the diversity and complexity of fault modes, such as fuzzy decision boundary and serious feature space fragmentation. Traditional single model cannot well solve these problems, and its comprehensive ability in problem solving is lacking. Therefore, this paper proposes a heterogeneous integrated model to realize the complementary advantages by integrating multiple models. Specifically, the full connection network of the simulated LightGBM is used to learn the decision boundary; the 1D-CNN time series feature extractor is used to capture the time series features; and the Transformer context modeling module is used to build the global feature dependency.

[0096] The design inspiration of the heterogeneous integrated model comes from the "diversity" principle in "ensemble learning". Single model often performs well on specific types of data or tasks, but its generalization ability is limited when facing complex, multi-source, and nonlinear data. By introducing multiple sub-models with different structures and training methods, the complementarity can be formed in feature extraction, pattern recognition, and context modeling, thereby improving the robustness and accuracy of the overall model. This "divide and conquer" strategy has been widely used in image recognition and natural language processing, and this paper first systematically introduces it into the field of oil-immersed transformer fault diagnosis.

[0097] The fuzzy decision boundary is due to the fact that there is not a simple one-to-one correspondence between fault symptoms and fault types. For example, different faults may produce similar gas ratios, while the gas characteristics of the same fault at different development stages differ significantly. Feature space fragmentation means that normal and fault state data points are scattered in high-dimensional feature space, and there are many areas that cannot be divided by linear or simple nonlinear models. The core advantage of the heterogeneous integrated model is its "divide and conquer" strategy. Instead of trying to solve all problems with a complex all-purpose model, it divides the complex fault diagnosis task into three sub-tasks and assigns the most suitable sub-model to solve them: one model is responsible for learning static and structured feature combination rules; one model is responsible for capturing short-term and local temporal patterns; and one model is responsible for understanding long-term and global context associations. The final diagnosis result is the collaborative output of the three models.

[0098] In practical applications, the blurring of decision boundaries and the fragmentation of feature spaces often lead to models that perform well on the training set but poorly on the test set or actual running data, i.e., "overfitting" or "poor generalization ability". For example, some models may misclassify "partial discharge" and "arc discharge" as the same type of fault because they exhibit similar gas ratio values (such as C2H2 / H2) in DGA data. However, through heterogeneous ensemble models, the LightGBM module can learn the subtle differences between different fault types in structured features, the 1D-CNN can identify local trends in gas concentration changes, and the Transformer can model long-term dependencies between different time points, effectively distinguishing these " seemingly similar" fault patterns.

[0099] The blurring of decision boundaries can be quantified by the margin distribution:

[0100]

[0101] Experiments show that a single LightGBM has M(0.1)=18.4% on the DGA dataset, while the heterogeneous ensemble reduces it to 7.2%, intuitively indicating a clearer boundary.

[0102] The margin is an important indicator of measuring the confidence of the classifier. A larger margin usually means better generalization ability. The indicator function II(·) in the formula is used to count the proportion of samples with a classification margin less than the threshold ε. The smaller M(ε) is, the fewer confused samples there are near the classification boundary, and the clearer the classification boundary is. Experimental results show that the heterogeneous ensemble model reduces M(0.1) from 18.4% to 7.2%, indicating that it has a significant advantage in handling boundary fuzzy samples. This improvement is mainly due to the multi-model collaborative decision mechanism: when a model has low confidence in classifying a sample, other models can provide supplementary information through their unique feature extraction capabilities, thereby improving the overall classification confidence.

[0103] LightGBM is a highly efficient gradient boosting decision tree (GBDT) algorithm widely used in structured data classification and regression tasks. In oil-immersed transformer fault diagnosis, LightGBM can effectively handle high-dimensional features and learn complex data patterns through boosting methods. The LightGBM branch module in this paper simulates the decision logic of LightGBM, using a multi-layer fully connected neural network structure, specifically including two hidden layers with 256 and 128 neurons respectively, and using ReLU activation function after each layer to introduce nonlinearity. Mathematically, let the input feature vector be x∈R d , and the output be o∈R cwhere d is the input dimension and c is the number of classes. The forward propagation process can be represented as:

[0104]

[0105]

[0106]

[0107] where W1∈R 256×d , W2∈R 128×256 , W3∈R c×128 are weight matrices, and b1∈R 256 , b2∈R 128 , b3∈R c are bias vectors.

[0108] Although LightGBM is essentially a decision tree-based model, this paper simulates its "residual fitting" and "boosting" ideas through a fully connected neural network, which is a "neural networked" approximation. The advantage of this approach is that it can be seamlessly integrated with CNN, Transformer and other modules to form an end-to-end trainable system. The introduction of the ReLU activation function not only increases the non-linear expression ability, but also effectively alleviates the gradient vanishing problem. The hidden layer dimension is selected as 256 and 128, which is the result of the trade-off between model complexity and computational efficiency. Experiments show that this structure maintains a high accuracy while meeting the real-time requirements of online monitoring in terms of inference speed.

[0109] To explicitly simulate the residual fitting idea of GBDT, we append a residual connection (ResidualConnection) after equation (9):

[0110]

[0111] where W skip ∈R c×d .

[0112] This design allows the network to have an identity path when backpropagating, alleviating the gradient vanishing problem, while making the fully connected network closer to the additive model in the function space, consistent with the boosting mechanism of GBDT.

[0113] Residual Connection was first proposed by ResNet to address the vanishing gradient problem in deep networks. In this paper, the residual connection not only stabilizes the training process but also simulates the core idea of fitting residuals in GBDT. Specifically, GBDT learns the residual of the previous model at each tree, while the residual connection in this paper allows the network to directly map input features to the output space, making it easier to learn the residual between input and output. This design is structurally closer to the additive model, where the final output is the sum of multiple sub-model outputs, and is highly consistent with the idea of boosting. The selection of hidden layer dimensions is based on the conservation of Glorot initialization variance:

[0114]

[0115] Glorot initialization (also known as Xavier initialization) is a commonly used weight initialization method designed to maintain the stability of signal variance during forward and backward propagation. The formula β=0.5 is an empirical parameter used to adjust the ratio between the hidden layer dimension and the input dimension. By initializing in this way, we can effectively avoid the problem of gradient explosion or disappearance, which is particularly important in deep networks. In this paper, the selection of hidden layer dimension 256 not only considers the stability of initialization, but also takes into account the model capacity and computational efficiency.

[0116] 1D-CNN is good at extracting local features from sequence data and performs well in time series data analysis. In oil-immersed transformer fault diagnosis, DGA data can be regarded as a time series signal, and 1D-CNN can capture the pattern of gas concentration changes over time. The 1D-CNN branch designed in this paper contains two convolutional layers and one pooling layer. First, the input features are expanded into a single-channel sequence, and then the convolution kernel is used to extract features:

[0117]

[0118]

[0119] where W1∈R 128×1×3 、W2∈R 64×128×3 are the convolution kernel weights, and b1∈R 128 、b2∈R 64 are the biases.

[0120] After the convolution operation, an adaptive max-pooling layer is used to compress the feature map to a fixed length:

[0121]

[0122] Finally, the output is obtained through a fully connected layer:

[0123]

[0124] where W3∈R c×64 is a weight matrix, b3∈R c is a bias vector.

[0125] 1D-CNN has the advantages of translation invariance and local receptive field when processing time series data. The convolution kernel size is 3, which means that the output of each time step only depends on the input of the current step and the previous and next steps. This local connection method is very suitable for capturing short-term mutation features. Adaptive max pooling solves the problem of inconsistent input sequence length, allowing the model to process DGA data sequences of different lengths. Compared with traditional RNN or LSTM, 1D-CNN has obvious advantages in training speed and parallel computing, especially suitable for embedded device deployment.

[0126] The convolution kernel size is fixed at 3, which can capture gas mutations within a 7-day sliding window corresponding to the minimum wavelet support ψ(2t-l). The dilated convolution (Dilated Convolution) experiment shows that when dilation=2, the receptive field R=2×(3-1)×2+1=9 can cover two weeks of trends, but the parameter amount increases by 1.8×. Finally, dilation=1 is selected to balance efficiency. The adaptive pooling output length is set to L=16, which ensures that when the input sequence length T∈[30,180], a fixed-size feature vector can be obtained, facilitating subsequent concatenation.

[0127] Dilated Convolution is an effective method to expand the receptive field without increasing the parameter amount. In this paper, although dilation=2 can cover a longer time range, it also introduces more noise and computational overhead. The final choice of dilation=1 is a trade-off between model complexity, computational efficiency, and feature expression ability. The choice of adaptive pooling output length L=16 is based on experimental verification of different length sequences, which can maintain feature expression ability while ensuring consistent input dimensions for subsequent modules.

[0128] The Transformer architecture has an advantage in handling data that requires long-term dependencies due to its strong context modeling ability. In oil-immersed transformer fault diagnosis, there may be long-term dependencies between DGA data at different time points, and Transformer can capture these relationships to improve diagnostic accuracy. The decoder structure of Transformer is as follows Figure 3 .

[0129] Transformers were initially applied in the field of natural language processing. Its core advantage is to model the dependency between any two positions in the sequence through self-attention mechanism (Self-Attention), without relying on recursive structure. This makes the Transformer avoid the gradient vanishing problem in RNN when processing long sequences, and has higher parallelization capability. In DGA data, some fault types (such as partial discharge) may show weak gas characteristics in the early stage, and gradually intensify after several weeks or months. Transformer can identify these weak signal correlations across time through self-attention mechanism, thereby improving the accuracy of early fault diagnosis.

[0130] The long-term dependency strength can be measured by the maximum mutual information (MaxMI):

[0131]

[0132] The experimental measurement of MaxMI is 0.47 nats, which is significantly higher than the random sequence of 0.12 nats, proving that there is a nonlinear dependence between the gas concentrations across months, providing the necessary basis for the Transformer.

[0133] The Transformer branch designed in this embodiment includes multiple layers of Transformer encoders. Each layer of encoder includes a multi-head self-attention mechanism and a feed-forward neural network. For the input sequence X={x1,x2,…,x n} where x i ∈R d , the calculation process of the multi-head self-attention mechanism is as follows:

[0134] First, map the input sequence to the query (Q), key (K) and value (V) space:

[0135]

[0136] where W Q ,W K ,W V ∈R d×dk is the weight matrix, and d k is the dimension of attention. Then, calculate the attention score and normalize it:

[0137]

[0138]

[0139] Subsequently, the value vector is weighted and summed:

[0140]

[0141] Finally, the features are further processed by a feed-forward neural network:

[0142]

[0143] where W1∈R d×dff , W2∈R d×dff are weight matrices, b1∈R dff , b2∈R d are biases, and d ff is the hidden layer dimension of the feed-forward network. After obtaining the output of the encoder, we get the final output through global average pooling and a fully connected layer:

[0144]

[0145] where W3∈R c×d is a weight matrix, and b3∈R c is a bias vector.

[0146] The multi-head self-attention mechanism maps the input to multiple subspaces (heads), allowing the model to simultaneously focus on different types of information at different positions in the sequence. This mechanism greatly enhances the model's expressive power. The scaling factor in equation (15) is to prevent the dot product value from being too large, causing the softmax function to saturate. The feed-forward network (FFN) part uses a two-layer fully connected structure, with the first layer expanding the dimension to four times and the second layer compressing it back to the original dimension. This "expansion-compression" structure helps the model learn more complex nonlinear mappings.

[0147] The number of heads h = 8 satisfies d k = d / h = 32, so that the dot product scaling factor d k = 42 matches the gradient scale; the feed-forward dimension adopts the four-fold expansion rule d ff = 4d = 256, consistent with BERT-base, with a parameter size of 8d 2 = 1.05M, and an inference delay of 8.7ms on an embedded GPU (Jetson Xavier), meeting the online monitoring requirement of ≤10ms. The position encoding uses 1D-learned instead of sinusoidal, as the DGA sampling interval is non-uniform (7-30 days), and the learned position vector PE∈R T×d automatically adapts to the non-periodic time pattern through backpropagation.

[0148] In summary, the heterogeneous ensemble model proposed in this section combines the decision logic of LightGBM, the local feature extraction capability of 1D-CNN, and the context modeling capability of Transformer, aiming to comprehensively capture the complex patterns in oil-immersed transformer DGA data and improve the accuracy and robustness of fault diagnosis. The proposal of the heterogeneous ensemble model not only enhances the diagnostic ability of traditional single models but also explores the balance between "model interpretability" and "engineering practicality". The LightGBM module provides strong interpretability (such as feature importance), the 1D-CNN module provides sensitivity to local mutation patterns, and the Transformer module provides modeling capability for long-term evolution trends. The synergy of the three enables the model not only to "diagnose faults" but also to "understand the evolution process of faults", providing more forward-looking decision support for maintenance personnel. In the future, this framework can be extended to fault diagnosis tasks of other power equipment, with broad engineering application prospects.

[0149] 3.5 Attention Fusion Module,

[0150] To integrate the three heterogeneous models of LightGBM, 1D-CNN, and Transformer, this section designs an attention fusion module. This module dynamically adjusts the contribution of each model output to the final prediction result by calculating the weight of each model output, enabling the model to better leverage the strengths of each branch.

[0151] The core idea of the attention fusion module is to assign weights based on the confidence of each model output. Specifically, we learn these weights through an attention network. The input of the attention network is the output of each model, which is passed through a fully connected layer and an activation function to output the weight of each model. These weights are normalized by the Softmax function so that their sum is 1. In mathematical expression, assuming the output of each model is o i ∈R c (where c is the number of classes), we first concatenate these outputs into a vector:

[0152]

[0153] Then, map it to a low-dimensional space through a fully connected layer:

[0154]

[0155] where W1∈R 64×3c is the weight matrix, and b1∈R 64 is the bias vector. Next, calculate the weight of each model:

[0156]

[0157] where W2∈R 3×64 is a weight matrix, b2∈R 3 is a bias vector. Here 3 represents three models. Finally, the final output is obtained by weighted summation:

[0158]

[0159] where w i is the weight of the i-th model.

[0160] The outputs of the three heterogeneous models designed in Section 3.4 are integrated through the attention fusion module in this section. Specifically, the output of each branch is stacked into a tensor and then input into the attention fusion module to calculate the weight:

[0161]

[0162] Then, the weight is calculated by the attention fusion module:

[0163]

[0164] Finally, the final output is obtained by weighted summation:

[0165]

[0166] Through the attention fusion module, the three heterogeneous models of LightGBM, 1D-CNN and Transformer can be effectively integrated. This integration method fully utilizes the advantages of each model, improving the accuracy and robustness of oil-immersed transformer fault diagnosis. The introduction of attention mechanism enables the model to dynamically adjust the contribution of each branch, thus achieving good diagnostic results under different fault types and data distributions.

[0167] 4、Experiment,

[0168] In order to verify the performance of the oil-immersed transformer fault diagnosis model based on mixed feature enhancement and deep heterogeneous integration proposed in this paper, a large number of experimental verification is carried out. This paper uses 500KV oil-immersed transformer as the fault diagnosis data set. The experimental part of this paper mainly includes data preprocessing, parameter setting, baseline model, experimental results, comparative experiment and ablation experiment. In the following chapters, this paper will introduce each step of the experiment in detail.

[0169] 4.1 Data Preprocessing,

[0170] To verify the effectiveness of the model proposed in this paper, the fault data of a 500KV oil-immersed transformer is used as a data set for experiments. The data set has two tables. Table one contains the content indicators of different gas components in the transformer oil, such as H2, CH4, C2H4, C2H2, CO, CO2, THC, and C2H6. Table two indicates the fault types of the fault coding column in table one, which has a total of six categories: medium-low temperature overheating, high temperature overheating, partial discharge, low energy discharge, high energy discharge, and normal. Table one has 1094 samples, and some sample data is shown in Table 1.

[0171] Table 1 Partial sample data

[0172] H2 CH4 C2H4 C2H2 CO CO2 THC C2H6 Fault code 17.2 2.9 0.6 0 222 428.9 3.7 0.2 1 18.9 2.6 0.6 0 189.1 357.5 3.3 0.1 2 21.88 2.17 0.67 0 154.11 323.51 2.95 0.11 3 28.87 2.1 0.16 0 115.68 231.39 2.27 0 4

[0173] The number of fault samples is shown in Table 2.

[0174] Table 2 Number of fault sample statistics

[0175] Fault code Fault type Fault sample number 1 Low and medium temperature overheating 211 2 High temperature overheating 181 3 Partial discharge 186 4 Low energy discharge 258 5 High energy discharge 123 6 Normal 135 Total 1094

[0176] As can be seen from Table 2, the number of low-energy discharge and medium-low temperature overheating is large, and the number of normal and high-energy discharge is small, which fully reflects the sample imbalance problem in oil-immersed transformer fault diagnosis.

[0177] In this paper, the smote algorithm is used for data expansion and balancing operation, and the k-nearest neighbor algorithm parameters are dynamically adjusted to generate samples that meet the physical constraints. The original samples are spliced with the synthesized samples to make the number of each fault category reach 300, and the total data sample size after balancing is 1800. Each fault sample is divided into training set and test set in the ratio of 8:2.

[0178] 4.2 Parameter setting,

[0179] In this section, we will show some cases and results. The training and testing process of the example is carried out on a Windows PC equipped with an NVIDIA GeForce GTX 3090 GPU processor and an Intel Core i910900 3.70 GHz CPU processor. This paper uses Python programming language and Pycharm development environment, and builds a model based on Pytorch. The model parameters are shown in Table 3.

[0180] Table 3 Model parameters

[0181] Parameter Description Value Epoch Model training times 500 η Learning rate 0.001 d_latent VAE latent space dimension 4 d_fused Feature fusion dimension 16 β KL divergence coefficient 0.5 k CNN convolution kernel scale 3 h Number of attention heads 4

[0182] Specifically, in this experiment, we used feature engineering methods to construct four new ratio features (C2H2 / C2H4, CH4 / H2, C2H2 / C2H6, C2H4 / C2H6) during data processing, expanding the original 8-dimensional feature space to 12-dimensional. The number of samples passed in each model training (batch size) is set to 64, and the total number of iterations (epochs) during training is configured to 500 rounds. The latent dimension of the VAE feature extractor is set to 4, and in the multi-model hybrid-driven power grid fault diagnosis architecture, we set the input dimension after feature fusion to 16 (12-dimensional original features + 4-dimensional VAE features). In addition, the number of heads of the Transformer branch is set to 4, and the hidden layer dimension of the attention mechanism is 64 to enhance the feature fusion capability of the model. Finally, the learning rate of the model is set to 0.001, which will control the update speed of the model parameters through the AdamW optimizer, prompting the model to converge to the optimal solution stably during training. Through the above parameter configuration, it is expected to train a hybrid model with strong feature extraction capability and high classification accuracy, thereby providing reliable technical support for oil-immersed transformer fault diagnosis tasks.

[0183] 4.3 Experimental results,

[0184] To verify the performance of the oil-immersed transformer fault diagnosis model based on hybrid feature enhancement and deep heterogeneous integration proposed in this paper, a large number of experiments were conducted based on the fault data collected from 500kv transformers, and the experimental results are shown in Table 4.

[0185] Table 4 Experimental results

[0186]

[0187] From the overall performance, the model performs very well on the test set, with an accuracy of 98.89%, and the macro-average and weighted average F1 scores also reach 98.89%, indicating that the model has strong overall generalization ability. The model performs very evenly on the six fault types, with F1 scores for each category above 0.98. The main errors are concentrated in the misjudgment of high-energy discharge (predicting other categories as high-energy discharge). Each category has 60 samples, and the total number of samples is 360, and the model only misclassifies 4 samples, verifying the generalization ability and reliability of the model.

[0188] 4.4 Comparative experiment,

[0189] To prove the effectiveness of the model proposed in this paper, four models are selected as comparative experiments, and experimental comparisons are carried out with the model proposed in this paper. The descriptions of these 4 baseline models are as follows:

[0190] NRBO-XGBoost model

[37] : NRBO-XGBoost model based on rough set feature selection, edge deployment + SHAP explainable, accuracy 88.33%.

[0191] MRVM-MSVM-BPNN model

[38] : multi-probability output model fusion improved DS evidence theory, accuracy 72.20%.

[0192] CVAE-CNN model

[39] : VAE data augmentation + 1D-CNN classifier, solve small sample problem, accuracy 82.20%.

[0193] ResVAE-Bagging ensemble model

[34] : ResVAE self-enhancement + Bagging ensemble, pre-training processing multi-class imbalance, accuracy 81.10%.

[0194] The comparative experimental results are shown in Table 2. Figure 4

[0195] The experimental results show that the model proposed by us is superior to other comparative models in accuracy (98.89%) and F1 score (0.9889), among which the accuracy and F1 score of the second NRBO-XGBoost model are 88.33% and 0.8227 respectively. We believe that this advantage comes from the mixed feature enhancement and dynamic fusion of multiple models, so that the model weight can be adjusted according to the data at any time, so as to better capture the complex features of transformer fault. In summary, we believe that the model designed in this paper has excellent diagnostic effect in oil-immersed transformer fault diagnosis and has strong practical value.

[0196] 4.5 Ablation experiments,

[0197] The model designed in this paper is a combination of three modules: a simulated LightGBM fully connected network, a 1D-CNN time series feature extractor, and a Transformer context modeling module. Among them, the simulated LightGBM fully connected network learns the decision boundary; the 1D-CNN time series feature extractor captures the time series features; and the Transformer context modeling module constructs the global feature dependency relationship. In order to further explore the contribution of each module to the performance of the model and verify its effectiveness, this section designs 6 groups of ablation experiments. These experiments correspond to the following module combinations: only using the LightGBM module, only using the 1D-CNN module, only using the Transformer module, fusing the LightGBM module and the 1D-CNN module, fusing the LightGBM module and the Transformer module, and fusing the 1D-CNN module and the Transformer module.​

[0198] From the experimental results of Figure 5, the proposed complete model achieves an accuracy of 98.89%, which is significantly improved compared to the model combination using only a single module or two modules. Specifically, the model using only the LightGBM module has an accuracy of 97.78%, the model using only the 1D-CNN module has an accuracy of 97.50%, and the model using only the Transformer module has an accuracy of 95.83%. In the two-module combination, the accuracy of the LightGBM and 1D-CNN combination is 97.22%, the accuracy of the LightGBM and Transformer combination is 96.94%, and the accuracy of the 1D-CNN and Transformer combination is 96.67%. These results fully demonstrate that the integrated model designed in this paper can more comprehensively mine feature information in the data by combining the advantages of the three modules, thereby achieving higher classification accuracy, effectively verifying the synergistic effect of the modules and the superiority of the overall architecture.

[0199] With the high-quality development of society, the power grid structure is becoming increasingly complex, and the load of transformers is rapidly expanding. It is difficult to make accurate diagnoses when facing imbalanced sample data, and it is difficult to meet the requirements of actual complex large-scale power equipment fault diagnosis. This study proposes an innovative oil-immersed transformer fault diagnosis method based on mixed feature enhancement and heterogeneous model integration, aiming to address the complexity of transformer fault features and the challenge of diagnostic accuracy. By artificially constructing multi-dimensional features and combining the feature extraction capabilities of variational autoencoder (VAE), the fault features are fully extracted, and the collaborative modeling advantages of multi-branch neural networks (LightGBM, 1D-CNN, Transformer) are utilized. Finally, the attention fusion mechanism dynamically weights and integrates the outputs of the three branches to achieve optimal decision-making, significantly improving the accuracy and robustness of fault diagnosis. Experimental results show that the method proposed in this paper performs well in processing 500KV dissolved gas analysis (DGA) data and extracting key fault features. However, there is still room for improvement: first, the computational complexity of the Transformer branch is high, which poses a challenge for edge device deployment; second, for some rare fault types (such as arc discharge with overheating), the lack of samples limits the recognition accuracy; finally, feature engineering relies on domain knowledge, and the automation level needs to be improved. Overall, the diagnostic framework proposed in this paper, which integrates mixed feature enhancement and heterogeneous models, provides a new technical path for intelligent operation and maintenance of power transformer equipment, promoting the digital and intelligent operation and maintenance of power systems. In the future, we will further collect more rich data, including transformer runtime, transformer operating temperature and humidity, etc., to realize multi-modal transformer fault diagnosis and provide more solid technical support for the smooth operation of power transformers.

[0200] Embodiment 2

[0201] The embodiment provides an oil-immersed transformer fault diagnosis system based on feature enhancement and deep heterogeneity, comprising:

[0202] The data acquisition module is configured to acquire oil-immersed transformer minority fault sample data;

[0203] The model construction module is configured to construct an oil-immersed transformer fault diagnosis model based on mixed feature enhancement and deep heterogeneous integration, wherein the minority fault sample data is synthesized into new samples conforming to physical constraints by using a minority class oversampling method;

[0204] The feature enhancement module is configured to construct an enhanced input space based on deep features generated by the autoencoder VAE, and fully extract deep features of fault gas in the new samples;

[0205] The feature extraction module is configured to simulate a LightGBM (Light Gradient Boosting Machine) to capture shallow relationships of deep features based on the extracted deep features by using a heterogeneous integrated model; recognize local features of the fault gas by using a 1D-CNN (Convolutional Neural Network); and establish a global dependency relationship based on a Transformer model.

[0206] The fusion module is configured to adjust model output weights based on a dynamic fusion mechanism.

[0207] The optimization module is configured to optimize the constructed oil-immersed transformer fault diagnosis model.

[0208] The diagnosis module is configured to output a fault diagnosis result by using the optimized fault diagnosis model.

[0209] A computer-readable storage medium, wherein a plurality of instructions are stored in the computer-readable storage medium, and the instructions are adapted to be loaded and executed by a processor of a terminal device.

[0210] A terminal device, comprising a processor and a computer-readable storage medium, wherein the processor is used to implement instructions, and the computer-readable storage medium is used to store a plurality of instructions, and the instructions are adapted to be loaded and executed by the processor to implement a feature enhancement and deep heterogeneous oil-immersed transformer fault diagnosis method.

[0211] The above are preferred embodiments of the present application, and are not intended to limit the protection scope of the present application, and therefore: any equivalent changes made according to the structure, shape, principle of the present application should be covered within the protection scope of the present application.

Claims

1. A fault diagnosis method for oil-immersed transformers based on feature enhancement and deep heterogeneity, characterized by, The application relates to an oil-immersed transformer fault diagnosis method based on mixed feature enhancement and deep heterogeneous integration. Obtaining a few fault sample data of an oil-immersed transformer; Constructing an oil-immersed transformer fault diagnosis model based on mixed feature enhancement and deep heterogeneous integration, wherein a few fault sample data are synthesized into new samples meeting physical constraints by using a few-class oversampling method; An enhanced input space is constructed based on deep features generated by a self-encoder VAE to sufficiently extract deep features of fault gases in the new samples; Based on the extracted deep features, a heterogeneous integration model is used to simulate a light gradient boosting machine (LightGBM) to capture shallow relationships of the deep features, a convolutional neural network (1D-CNN) is used to identify local features of the fault gases, and a Transformer model is used to establish global dependency relationships; The output weight of the model is adjusted based on a dynamic fusion mechanism; The constructed oil-immersed transformer fault diagnosis model is optimized; A fault diagnosis result is output by using the optimized fault diagnosis model. The oil-immersed transformer fault diagnosis model based on mixed feature enhancement and deep heterogeneous integration comprises an adaptive sample generation method, a neighborhood parameter is dynamically adjusted based on the distribution characteristics of few-class samples, new samples meeting physical constraints are synthesized to ensure that the gas concentration is non-negative and the original data distribution rule is maintained, a ten-dimensional expert knowledge system is constructed on the feature engineering, five gas concentrations of existing data are fused, and fault feature ratios, discharge indicator ratios, thermal fault ratios and total hydrocarbon concentrations are introduced, 4-dimensional deep features generated by a variational autoencoder (VAE) are coupled to form a 14-dimensional enhanced input space, then a heterogeneous integration model architecture is used to fuse three types of learners: the first branch simulates a LightGBM to capture shallow relationships of the features, the second branch uses a one-dimensional convolutional neural network (1D-CNN) to identify local features of the fault gases, and the third branch is based on a Transformer model to establish global dependency relationships, and finally a dynamic fusion mechanism is used to realize the cooperation of the three models through attention weighting. The method of synthesizing new samples that conform to physical constraints from minority faulty sample data using minority class oversampling includes introducing the SMOTE oversampling algorithm based on dynamic adjustment of k-nearest neighbors. This algorithm mitigates the class imbalance problem in the original data by adaptively generating synthetic samples. The SMOTE algorithm avoids overfitting risk by generating synthetic samples through interpolation. This can be expressed as: Let the minority sample set S = {xi ∈ R} d |i=1,…,N s }, where d is the feature dimension; for ∀xi∈S, calculate its k nearest neighbor set N. k (x i )={x i (1) ,…,x i (k) The distance metric used is Mahalanobis distance. Σ is the covariance matrix of the minority class samples; finally, linear interpolation is performed: from N k Randomly select a nearest neighbor sample x nn Generate a new sample x new : ,in, , which is a random interpolation number that controls the position of the new sample in the feature space.

2. The oil-immersed transformer fault diagnosis method based on feature enhancement and depth heterogeneity according to claim 1, characterized in that, The deep features generated based on the autoencoder VAE construct an enhanced input space, fully extract deep features of fault gas in new samples, including supplementing five-dimensional features on the basis of original characteristic gas to weaken the type confusion problem and enhance the representation ability of minority class fault samples, and performing Z-score standardization on original gas concentration and constructed ratios to eliminate dimensional differences: , where μ is the mean of the x data set; σ is the standard deviation of the data set elements; x' is the result after x is standardized; then a variational autoencoder VAE is used to learn the data distribution while performing feature extraction and data enhancement, wherein the VAE is composed of an encoder, a decoder and a reparameterization process, the encoder is used to map input data to a latent space to obtain a mean and a variance; the decoder is used to map samples in the latent space back to the data space to reconstruct the input data; the reparameterization process introduces randomness to enable the model to learn feature representation, which is represented as: first, the encoder is used to map input data x to the mean μ and variance logσ of the latent space 2 : , where ϕ represents parameters of the encoder; is the backpropagation gradient in the training process, and a reparameterization technique is introduced, which is represented as: , where ϵ is a random variable sampled from a standard normal distribution N(0, 1), and ⊙ represents element-level multiplication; then the decoder is used to map samples z in the latent space back to the data space to reconstruct input data x: , where θ represents parameters of the decoder; finally, a loss function is calculated, and the loss function of the VAE is composed of a reconstruction loss and a KL divergence, which is represented as: ， The parameter beta is a hyperparameter used to balance the reconstruction loss and the KL divergence.

3. The oil-immersed transformer fault diagnosis method based on feature enhancement and depth heterogeneity according to claim 2, characterized in that, The heterogeneous integrated model simulates the shallow relationship of capturing deep features of the LightGBM, including using a fully connected network to simulate gradient boosting decision trees to effectively capture high-order nonlinear interaction relationships between features, wherein the LightGBM branch module is used to simulate the decision logic of LightGBM, and a multi-layer fully connected neural network structure is used, specifically including two hidden layers, each layer uses a ReLU activation function to introduce nonlinearity, and the input feature vector is x∈R d , and the output is o∈R c , wherein d is the input dimension, c is the number of categories, and the forward propagation process is represented as: ， ， ， where W1∈R 256×d , W2∈R 128×256 , W3∈R c×128 is a weight matrix, b1∈R 256 , b2∈R 128 , b3∈R c is a bias vector.

4. The oil-immersed transformer fault diagnosis method based on feature enhancement and depth heterogeneity according to claim 3, characterized in that, The 1D-CNN is used to extract local features in sequence data, wherein in the oil-immersed transformer fault diagnosis, the DGA data are regarded as time series signals, the 1D-CNN is used to capture the change mode of the gas concentration with time, the 1D-CNN branch comprises two convolutional layers and one pooling layer, the input features are first expanded into a single-channel sequence, and then the features are extracted by a convolution kernel: ， ， where W1∈R 128×1×3 , W2∈R 64×128×3 are convolution kernel weights, b1∈R 128 , b2∈R 64 are biases, and after the convolution operation, an adaptive max-pooling layer is used to compress the feature map to a fixed length, denoted as ， Finally, the output is obtained through a fully connected layer. ， where W3∈R c×64 is a weight matrix, and b3∈R c is a bias vector.

5. The oil-immersed transformer fault diagnosis method based on feature enhancement and depth heterogeneity according to claim 4, characterized in that, The establishment of global dependencies based on the Transformer model includes using Transformer to capture long-term dependencies between DGA data at different time points in oil-immersed transformer fault diagnosis. The Transformer branch contains multi-layer Transformer encoders, each layer including a multi-head self-attention mechanism and a feedforward neural network. For the input sequence X={x1,x2,…,x…} n }, where x i ∈R d The computational representation of the multi-head self-attention mechanism is as follows: First, the input sequence is mapped to the query (Q), key (K), and value (V) spaces: Among them, W Q W K W V ∈R d×dk It is the weight matrix, d k This is the dimension of attention; then the attention score is calculated and normalized: , Then, a weighted summation is performed on the value vector: Finally, the features are further processed through a feedforward neural network, and are represented as follows: , where W1∈R d×dff W2∈R d×dff It is a weight matrix, b1∈R dff b2∈R d It is a bias, d ff This refers to the hidden layer dimension of the feedforward network; after obtaining the encoder's output, the final output is obtained through global average pooling and fully connected layers. ， where W3∈R c×d is a weight matrix, and b3∈R c is a bias vector.

6. The oil-immersed transformer fault diagnosis method based on feature enhancement and depth heterogeneity according to claim 5, characterized in that, The model output weight is adjusted based on a dynamic fusion mechanism, including adopting an attention fusion mechanism to dynamically adjust the contribution to the final prediction result by calculating the weight of each model output, wherein the output of each model is denoted as o i ∈R c wherein c is the number of categories, the output is first spliced into a vector: ; and then mapped to a low-dimensional space through a fully connected layer: wherein W1∈R 64×3c is a weight matrix, b1∈R 64 is a bias vector, and then the weight of each model is calculated: wherein W2∈R 3×64 is a weight matrix, b2∈R 3 is a bias vector, and finally the final output is obtained through weighted summation: ， where w i is the weight of the ith model.

7. The oil-immersed transformer fault diagnosis method based on feature enhancement and depth heterogeneity according to claim 6, characterized in that, The oil-immersed transformer fault diagnosis model is optimized, including adopting a feature engineering method to construct four new ratio features, including C2H2 / C2H4, CH4 / H2, C2H2 / C2H6, and C2H4 / C2H6, expanding the original 8-dimensional feature to a 12-dimensional feature space, setting the sample size batch size to 64 for each model training, and configuring the total number of iterations epochs to 500 rounds during training; the latent dimension of the VAE feature extractor is set to 4, in the power grid fault diagnosis architecture based on multi-model hybrid driving, the input dimension after feature fusion is set to 16, including 12-dimensional original features + 4-dimensional VAE features, and the number of heads of the Transformer branch is set to 4, and the hidden layer dimension of the attention mechanism is 64, to enhance the feature fusion capability of the model; finally, the learning rate of the model is set to 0.001, and the AdamW optimizer is used to control the update speed of the model parameters, so that the model converges to the optimal solution stably during training.

8. A feature enhancement and deep heterogeneous based oil-immersed transformer fault diagnosis system, which executes a feature enhancement and deep heterogeneous based oil-immersed transformer fault diagnosis method according to claim 1, characterized in that, Comprise: A data acquisition module configured to acquire oil-immersed transformer minority fault sample data; A model construction module configured to construct an oil-immersed transformer fault diagnosis model based on mixed feature enhancement and deep heterogeneous integration, wherein a minority class oversampling method is used to synthesize new samples conforming to physical constraints from the minority fault sample data; A feature enhancement module configured to construct an enhanced input space based on deep features generated by a self-encoder VAE to fully extract deep features of fault gases in the new samples; A feature extraction module configured to extract deep features based on the extracted deep features, use a heterogeneous integration model to simulate a light gradient boosting machine LightGBM to capture shallow relationships of deep features, use a convolutional neural network 1D-CNN to identify local features of fault gases, and establish global dependency relationships based on a Transformer model; A fusion module configured to adjust model output weights based on a dynamic fusion mechanism; An optimization module configured to optimize the constructed oil-immersed transformer fault diagnosis model; A diagnosis module configured to output fault diagnosis results using the optimized fault diagnosis model.

Images