Transformer bushing fault identification method based on multi-feature fusion and attention mechanism

Abstract

The invention provides a transformer bushing fault identification method based on multi-feature fusion and an attention mechanism, and relates to the technical field of transformer bushing fault monitoring, and the method comprises the steps: obtaining and preprocessing key state parameters, and generating a standard time sequence sample; on the basis of a Pyraform model architecture, a feature fusion module is additionally arranged at the front end of a model encoder to realize adaptive fusion of features, and a time difference weight mechanism is introduced into an attention mechanism to enhance the time sensing ability of the model, so that an improved Pyraform model is constructed; automatically searching and configuring the key hyper-parameters by adopting a Bayesian optimization algorithm; in the training process, a variable learning rate strategy and an early stop mechanism are adopted for linkage control; and acquiring key state parameters of the transformer bushing in real time, preprocessing the key state parameters, inputting the preprocessed key state parameters into the trained improved Pyraformer model, outputting a prediction trend value and an abnormal state probability of the key state parameters in a future preset time period, triggering multi-stage alarm, and periodically performing incremental learning and self-updating.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of transformer bushing fault monitoring, and particularly relates to a transformer bushing fault identification method based on multi-feature fusion and attention mechanism. BACKGROUND

[0002] As a key equipment in the power system, the running state of the transformer has an important influence on the stability and safety of power supply. The internal state deterioration of the transformer bushing, as an important insulation structure of the transformer, can easily cause serious accidents such as insulation breakdown and oil and gas leakage, and thus it is necessary to realize continuous monitoring and intelligent early warning of the state thereof.

[0003] However, the current commonly used transformer bushing online monitoring means mainly performs state evaluation through traditional threshold diagnosis or experience rules. However, such a method relies on the human-set threshold, and it is difficult to effectively capture the dynamic relationship between complex variables in a long time sequence, and thus the modeling and prediction ability for potential abnormal trends is insufficient. SUMMARY

[0004] The present application aims to provide a transformer bushing fault identification method based on multi-feature fusion and attention mechanism to solve the problem that the current commonly used transformer bushing online monitoring means is difficult to effectively capture the dynamic relationship between complex variables in a long time sequence, and thus the modeling and prediction ability for potential abnormal trends is insufficient.

[0005] To achieve the above-mentioned purpose, the present application provides the following technical scheme: a transformer bushing fault identification method based on multi-feature fusion and attention mechanism, comprising the following steps: obtaining key state parameters from a transformer bushing fault database and preprocessing to generate standard time sequence samples, wherein the key state parameters include end screen current and bushing capacitance; based on a Pyraformer model architecture, a feature fusion module is added in front of the model encoder to realize adaptive fusion of features, and a time difference weight mechanism is introduced in the attention mechanism to enhance the model time perception ability, so as to construct an improved Pyraformer model; a Bayesian optimization algorithm is used to automatically search and configure the key hyperparameters of the improved Pyraformer model; the improved Pyraformer model is trained through the standard time sequence samples, and a variable learning rate strategy and an early stopping mechanism are used for linkage control during the training process; the key state parameters of the transformer bushing are collected in real time, and after preprocessing, they are input into the trained improved Pyraformer model, and the prediction trend value and the abnormal state probability of the key state parameters in the future preset time period are output, and after comparison with the preset threshold, multi-level alarm is triggered, and the abnormal data is backflowed to the training set for periodic incremental learning and self-updating of the model.

[0006] Optionally, the preprocessing step specifically includes: time stamp alignment and hour-level resampling of the screen current and the bushing capacity; constructing a periodic time covariant feature, dividing a fixed-length sliding window sample, and normalizing each sample to build a three-dimensional input tensor [B, T, D] as a standard time series sample; wherein B is the number of samples; T is the time step; D is the feature dimension, including screen current, bushing capacity, and periodic covariant constructed by time stamp.

[0007] Optionally, the step of adding a feature fusion module in front of the model encoder specifically includes: constructing an independent subspace embedding channel for the screen leakage current sequence and the bushing capacity sequence respectively, wherein each channel is composed of one convolutional mapping; concatenating the two feature vectors along the channel dimension, and performing global average pooling operation on the concatenated feature map to generate a channel descriptor vector; passing the channel descriptor vector through two fully connected layers and a sigmoid activation function in turn to generate a weight vector representing the importance of each channel; broadcasting the generated weight vector along the channel dimension and multiplying it with the concatenated feature map channel by channel to realize adaptive re-labeling of the features; using layer normalization to cross-channel normalize the re-labeled features to unify the statistical distribution of the channel features, and finally outputting a feature fusion tensor.

[0008] Optionally, the step of introducing a time difference weight mechanism in the attention mechanism specifically includes: calculating the actual physical time interval difference between the query vector and the key vector corresponding to the data points in the attention mechanism; mapping the time interval difference through a learnable position encoding vector or a periodic time function to obtain a scalar bias term for adjusting the attention weight; introducing the scalar bias term into the calculation of the original attention score in an additive form, so that the allocation of the attention weight is explicitly dependent on the time proximity relationship between the data points.

[0009] Optionally, the step of automatically searching and configuring the key hyperparameters of the improved Pyraformer model using the Bayesian optimization algorithm specifically includes: taking the joint loss function value of the model on the validation set as the objective function to jointly optimize and search multiple hyperparameters of the improved Pyraformer model; wherein the joint loss function is composed of a mean square error loss function and a cross-entropy loss function by weighted combination; the multiple hyperparameters include the number of encoder layers, the number of decoder layers, the number of attention heads, and the feature dimension; the Bayesian optimization process uses a Gaussian process-based surrogate model and a high-confidence upper bound-based acquisition function to guide the search of hyperparameters.

[0010] Optionally, the step of adopting the variable learning rate strategy in combination with the early stopping mechanism specifically comprises: if the validation set loss value does not appear a predetermined threshold decline in a plurality of continuous training periods, triggering a learning rate decay mechanism; if the validation set loss still has no improvement in the subsequent plurality of continuous training periods after the learning rate decay, triggering the early stopping mechanism to terminate the training.

[0011] Optionally, the step of periodic fine-tuning specifically comprises: after triggering the alarm, recording the detailed parameters of the abnormal event and generating an evaluation report; after verification, the results are backfilled as labels; the data with new labels are stored in the transformer bushing fault database to form training samples for driving model incremental learning; using the deployed model parameters as the initial weights, the model is periodically fine-tuned and updated using the new incremental training sample set.

[0012] In another aspect, the present application also provides a transformer bushing fault identification system based on multi-feature fusion and attention mechanism, characterized in that it comprises: an acquisition module for acquiring key state parameters from the transformer bushing fault database and preprocessing to generate standard time series samples, wherein the key state parameters include end screen current and bushing capacitance; an improved model construction module for adding a feature fusion module to the front end of the model encoder based on the Pyraformer model architecture to realize adaptive fusion of features, and introducing a time difference weight mechanism into the attention mechanism to enhance the model's time perception ability, thereby constructing an improved Pyraformer model; an optimization module for automatically searching and configuring key hyperparameters of the improved Pyraformer model using a Bayesian optimization algorithm; a model training module for training the improved Pyraformer model through the standard time series samples and adopting a variable learning rate strategy in combination with an early stopping mechanism for linkage control during training; an output module for real-time acquisition of key state parameters of the transformer bushing, inputting the preprocessed key state parameters into the trained improved Pyraformer model, outputting predicted trend values and abnormal state probabilities of the key state parameters in a future predetermined time period, triggering multi-level alarms after comparison with a predetermined threshold, and returning abnormal data to the training set for periodic incremental learning and self-updating of the model.

[0013] In another aspect, the present application also provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned transformer bushing fault identification method based on multi-feature fusion and attention mechanism when executing the computer program.

[0014] In another aspect, the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the transformer bushing fault identification method based on multi-feature fusion and attention mechanism.

[0015] Compared with the prior art, the present application has the following advantages: The present application selects the Pyraformer model to perform fault monitoring on key state parameters related to transformer bushing faults. The Pyraformer model adopts a pyramid sampling structure to efficiently model long sequences and is suitable for multi-dimensional, long-period and low sampling frequency data structures in power monitoring. However, when the traditional Pyraformer model is applied to transformer bushing data, there are still problems such as complex model parameter adjustment, insufficient sensitivity to time sequence changes, and high deployment cost. Therefore, the present application constructs an improved Pyraformer model integrating multi-feature fusion and time perception mechanism, and combines automatic hyperparameter optimization and intelligent training strategy to realize high-precision end-to-end prediction and abnormal identification of transformer bushing state, significantly improving the accuracy of fault identification, the reliability of time sequence prediction, and the deployment adaptability of the system in actual industrial scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 The figure is a schematic diagram of the method steps of the present application.

[0017] Figure 2 The figure is a flow chart of the method of the present application.

[0018] Figure 3 The figure is a flow chart of the Bayesian algorithm of the present application.

[0019] Figure 4 The figure is a schematic diagram of the system structure of the present application.

[0020] In the figure: 10 - acquisition module, 20 - improved model construction module, 30 - optimization module, 40 - model training module, 50 - output module. DETAILED DESCRIPTION

[0021] The present application will be described in detail below with reference to the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all embodiments.

[0022] It should be noted that the terms "first", "second", and the like in the description and in the claims of the present application and the above-described accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or apparatus including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or apparatuses.

[0023] Those skilled in the art can understand that, unless specifically stated, the singular forms "a", "an" and "the" used herein also include the plural forms. It should be further understood that the use of the phrase "comprise" in the specification of the present application means that a feature, integer, step, operation, element and / or component exists, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we say an element is "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or there can be intermediate elements. In addition, "connected" or "coupled" used herein can include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any single unit and all combinations of the associated listed items.

[0024] Those skilled in the art can understand that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as that generally understood by those skilled in the art to which the present application belongs. It should also be understood that terms such as those defined in a general dictionary should be understood to have meanings consistent with those in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as such.

[0025] It should be understood that the sequence numbers and sizes of the steps in the embodiments do not mean the order of execution, and the execution order of the processes is determined by their functions and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0026] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0027] Please refer to Figures 1-3The application discloses a transformer bushing fault identification method based on multi-feature fusion and an attention mechanism, and steps of the method comprise the following: key state parameters are acquired from a transformer bushing fault database and are preprocessed to generate standard time sequence samples, wherein the key state parameters comprise end screen current and bushing capacitance; a feature fusion module is additionally arranged at the front end of a model encoder based on a Pyraformer model architecture to realize adaptive fusion of features, and a time difference weight mechanism is introduced in the attention mechanism to enhance the time perception ability of the model, so that an improved Pyraformer model is constructed; a Bayesian optimization algorithm is used to automatically search and configure key hyperparameters of the improved Pyraformer model; the improved Pyraformer model is trained through the standard time sequence samples, and a variable learning rate strategy and an early stopping mechanism are used to realize linkage control during the training process; key state parameters of the transformer bushing are collected in real time, are preprocessed and are input into the improved Pyraformer model after training, prediction trend values and abnormal state probabilities of the key state parameters in a preset time period in the future are output, multi-level alarms are triggered after comparison with preset threshold values, and abnormal data are returned to the training set to be used for periodic incremental learning and self-updating of the model.

[0028] Specifically, key state parameters of the transformer bushing are collected in real time through an online monitoring system, mainly including end screen current and capacitance, to form original time sequence data. The online monitoring system comprises a through-hole current sensor device installed at the end screen part of the transformer bushing, which is used for collecting key operation state parameters such as end screen current and capacitance in real time. The end screen current signal is obtained through a high-precision current sensor, reflects the leakage path conduction between the end screen and the ground of the bushing, and can sensitively reflect the local deterioration trend of the internal insulation structure; the bushing capacitance is measured by a frequency domain scanning or resonance method, and reflects the health condition of the overall dielectric layer of the bushing.

[0029] The above signals are collected at a preset sampling frequency and are recorded synchronously in a standard time stamp format to form multivariate original time sequence data with clear time sequence relationship. The collection period can be set to 1 hour to match the sliding window length and the prediction period in the subsequent model. The collection system has data buffering and fault supplementing mechanisms, so that data can be recovered when communication is interrupted or the sensor fails, and the robustness of the system is enhanced.

[0030] The collected multivariate raw data will serve as the basis for subsequent feature construction, window slicing and model input, providing high-quality and continuous input data support for subsequent prediction models. After preprocessing, the improved Pyraformer model is used for fault monitoring of key state parameters related to transformer bushing faults. The model adopts a pyramid sampling structure to efficiently model long sequences and is suitable for multivariate, long-term and low sampling frequency data structures in power monitoring. However, when the traditional Pyraformer model is applied to transformer bushing data, there are still problems such as complex model parameter tuning, insufficient sensitivity to time series changes, and high deployment costs. Therefore, the improved Pyraformer model is constructed by integrating multi-feature fusion and time perception mechanism, and combined with automatic hyperparameter optimization and intelligent training strategy, high-precision end-to-end prediction and abnormal identification of transformer bushing state are realized, which significantly improves the accuracy of fault identification, the reliability of time series prediction and the deployment adaptability of the system in actual industrial scenarios.

[0031] In some embodiments, the preprocessing step specifically includes: time stamp alignment and hour-level resampling of the screen current and the bushing capacity; constructing periodic time covariant features, dividing fixed-length sliding window samples, and standardizing each sample to construct a three-dimensional input tensor [B, T, D] and use it as a standard time series sample; wherein B is the number of samples; T is the time step; D is the feature dimension, including screen current, bushing capacity and periodic covariant constructed by time stamp.

[0032] Specifically, the collected data is time-aligned and hour-level resampled, periodic time features such as day of the week, hour, and month are constructed, and sliding windows are sliced to finally generate a three-dimensional input tensor in the format of [B, T, D]. The original collected data may have problems such as inconsistent time stamps, uneven intervals, or missing values during storage. To ensure data continuity and uniformity, first, the original time series of the screen current channel and the capacity channel are time-aligned, that is, the missing data is interpolated or filled using a unified reference time axis, to ensure that all parameters have corresponding values at the same time point. After time alignment, the system performs hour-level resampling on the data. For example, data collected every 10 minutes is aggregated into mean, maximum, or other statistical indicators per hour to match the time granularity required by the model input, while reducing noise interference and improving modeling stability.

[0033] In constructing the model input features, further extract time class covariates, construct typical periodic time features including week number 0-6, hour number 0-23, month number 1-12, etc. These features reflect the time dependence of the device operation, which can significantly enhance the model's understanding of the behavior patterns such as weekdays / weekends, peak / valley, etc. Subsequently, the preprocessed time series data is sliced according to a fixed length sliding window, for example, set the window length to 60, representing 60 hours, and the sliding step is 6-12 hours, from the whole sequence to construct multiple overlapping training samples. Each sample segment contains multiple time steps and multi-dimensional features, and finally these samples are organized into a three-dimensional tensor format such as [B, T, D], where B is the number of samples, T is the time step, and D is the feature dimension, which includes the main variables of the screen current and the sleeve capacitance and the time covariates to construct the input tensor.

[0034] The samples are normalized and stacked into a unified tensor format to adapt to the input requirements of the deep model, where B is the number of samples, T is the time step, and D is the multi-dimensional feature. After completing the data preprocessing, the multiple sample sequences obtained by dividing the sliding window need to be further standardized and constructed into a three-dimensional input tensor format acceptable by the deep model. The data in each sample window is normalized. In this embodiment, the mean normalization method is used, that is, for the leakage current feature channel and the capacitance feature channel, the mean and standard deviation of the channel in the sample window are calculated, and all values of the channel are subtracted from the mean and divided by the standard deviation to ensure comparability between different features and eliminate the dimension effect. Normalization can also avoid model bias caused by different parameter orders of magnitude, thereby improving model convergence speed and stability. Then, all the processed samples are stacked in the general three-dimensional tensor format [B, T, D]. Where: B represents the number of samples; T represents the time step included in each sample, for example, 60 hours; D represents the feature dimension included at each time point, including the main monitoring variables and time covariates. The finally constructed tensor will be used as the input of the subsequent model and can be sent into the deep learning architecture in parallel batches for training or prediction. To improve training efficiency, the application also uniformly stores and manages the tensor sample file in binary format, which facilitates fast loading and cross-platform deployment.

[0035] The application strictly aligns and resamples the original monitoring data, constructs periodic covariate features, effectively eliminates the time inconsistency and dimension difference of multi-source heterogeneous data, and provides high-quality, standardized input samples for the model. The three-dimensional tensor structure generated by the sliding window slicing fully retains the local patterns and long-term dependencies of the time series, laying a solid foundation for the feature extraction and rule learning of the subsequent deep model.

[0036] In some embodiments, the step of adding a feature fusion module in front of the model encoder specifically comprises: constructing an independent subspace embedding channel for the terminal screen leakage current sequence and the bushing capacitance sequence respectively, wherein each channel is composed of a layer of convolution mapping; splicing the two feature vectors along the channel dimension, and performing a global average pooling operation on the spliced feature map to generate a channel descriptor vector; sequentially passing the channel descriptor vector through two fully connected layers and a Sigmoid activation function to generate a weight vector representing the importance of each channel; broadcasting the generated weight vector along the channel dimension and multiplying it with the spliced feature map channel by channel to achieve adaptive re-labeling of the features; using layer normalization to cross-channel normalize the re-labeled features to unify the statistical distribution of the features in each channel, and finally outputting a feature fusion tensor.

[0037] The present application realizes adaptive importance allocation of different physical significance features by independently embedding the leakage current and capacitance sequence in a subspace and weighting the channels through the feature fusion module. It can suppress single channel noise interference, strengthen the representation ability of key features, and enhance the consistency between multiple features, thereby improving the recognition sensitivity and robustness of the model to complex fault modes.

[0038] In some embodiments, the step of introducing a time difference weight mechanism in the attention mechanism specifically comprises: calculating the actual physical time interval difference between the query vector and the key vector corresponding to the data points in the attention mechanism; mapping the time interval difference through a learnable position encoding vector or a periodic time function to obtain a scalar bias term for adjusting the attention weight; introducing the scalar bias term into the calculation of the original attention score in an additive form, so that the allocation of the attention weight is explicitly dependent on the time proximity relationship between the data points.

[0039] The present application introduces a time difference weight mechanism in the attention mechanism, which enables the model to explicitly perceive the actual interval and sequence between different time points in the sequence. This mechanism enhances the modeling ability of non-uniformly sampled data, sudden failures and periodic changes, solves the limitation of the standard attention mechanism that is not sensitive to time series information, and improves the judgment accuracy of the model on the occurrence time and evolution trend of the fault.

[0040] In some embodiments, the step of automatically searching and configuring the key hyperparameters of the improved Pyraformer model by using the Bayesian optimization algorithm specifically comprises: taking the joint loss function value of the model on the validation set as the objective function, and jointly optimizing and searching the multiple hyperparameters of the improved Pyraformer model; wherein the joint loss function is composed of a mean square error loss function and a cross-entropy loss function by weighted combination; the multiple hyperparameters include the number of encoder layers, the number of decoder layers, the number of attention heads, and the feature dimension; and the Bayesian optimization process uses a Gaussian process-based surrogate model and a high-confidence upper bound-based acquisition function to guide the hyperparameter search.

[0041] Specifically, before training, a Bayesian optimization algorithm based on a probability model, preferably a UCB strategy, is used to search for key hyperparameters in the model structure and select the optimal combination for model construction. To avoid the low efficiency and uncontrollability of traditional manual parameter tuning, a Bayesian optimization algorithm is introduced to automatically search and configure the structural and training hyperparameters of the improved Pyraformer model. Let the hyperparameter vector be θ∈Θ, and the joint loss function value on the validation set be defined as: , wherein represents the mean square error of the trend prediction task, represents the cross-entropy loss of the anomaly discrimination task, and α is the weighting coefficient. The optimization objective is to minimize In the Bayesian optimization process, Gaussian process is used as a surrogate model. For any hyperparameter θ, the posterior mean and variance are: ; ; wherein is the kernel function, K is the kernel matrix, and σ is the noise variance. The acquisition function uses the lower confidence bound LCB form: ; wherein β controls the balance between exploration and utilization. In each iteration, the θ that minimizes is selected as the candidate point, and the model is trained and validated, until the preset number of rounds or the convergence condition is reached.

[0042] The hyperparameters covered by Bayesian optimization include, but are not limited to, the number of encoder layers, decoder layers, number of attention heads, and hidden dimension size. The optimization objective is the joint loss function value on the validation set, specifically the weighted sum of mean squared error and cross-entropy loss. During the search process, an initial number of sample points (e.g., 10 sets) and a maximum search epoch (e.g., 50 epochs) are set to control search overhead and ensure global optimum reachability. Finally, the set of hyperparameters determined through Bayesian optimization will be used to formally build the improved Pyraformer model and fed into the model training phase.

[0043] This application employs a Bayesian optimization algorithm to automatically search for key hyperparameters of the model, overcoming the subjectivity and inefficiency of manual hyperparameter tuning. The process targets the validation set loss and guides the search direction through a probabilistic surrogate model and an intelligent acquisition function. This enables the discovery of high-performance and stable hyperparameter configurations within a limited number of evaluation iterations, significantly improving model training efficiency and generalization performance.

[0044] In some embodiments, the step of using a variable learning rate strategy and an early stopping mechanism for coordinated control specifically includes: if the validation set loss value does not decrease to a predetermined threshold within multiple consecutive training cycles, a learning rate decay mechanism is triggered; if the validation set loss does not improve within subsequent multiple consecutive training cycles after the learning rate decays, an early stopping mechanism is triggered to terminate training.

[0045] Specifically, an improved Pyraformer model is constructed, and a joint loss function is used during the training phase: mean squared error (MSE) is used to fit trend prediction, and cross-entropy loss is used for anomaly probability classification. At the same time, cosine annealing scheduling is used to control the dynamic decay of the learning rate, and an early stopping mechanism is introduced to avoid overfitting.

[0046] After completing hyperparameter optimization and constructing the improved Pyraformer model, the supervised training phase begins. The training data comes from the preprocessed and tensor-constructed standard sample set. The model aims to simultaneously perform trend prediction and anomaly identification tasks for transformer bushing operating parameters.

[0047] To enhance the model's ability to perceive unequal sampling intervals and periodic patterns, this invention introduces a time difference weighting mechanism into the attention calculation for the time difference weighting mechanism design. Let the timestamps of samples i and j be ti and tj, respectively, and their time difference be... Learnable positional encoding vector: As input, a vector is obtained through linear transformation or a small MLP. This is then superimposed on the input features, thus providing the model with a learnable representation of the temporal order. Periodic time function: ; Wherein, P is a period constant, such as 24 hours, 168 hours, and a scalar weight is obtained through linear mapping .

[0048] The combination mode with the attention score is an additive bias form ; In the formula: is the normalized attention score, is a softmax normalization function along the j dimension, ensuring that the sum of all attention weights for j is 1; is the query vector corresponding to the i-th sample; is the key vector corresponding to the j-th sample; d is the feature dimension of the query vector and the key vector; is an introduced additive bias, representing the time difference weight.

[0049] In the above manner, the model can explicitly perceive the time difference and periodicity at the attention mechanism level, thereby improving the recognition accuracy of sudden failures and periodic anomalies.

[0050] For the joint loss function design, the present embodiment adopts a multi-objective joint loss function to balance the performance of the model in numerical prediction and anomaly detection: the mean square error loss is used to fit the trend prediction value and the true value of the model output, the time series regression error is measured, and the cross-entropy loss is used to measure the accuracy of the model in classifying abnormal states; the two losses are weighted and fused by setting the weight to form the final training optimization objective function. This design can enable the model to learn trend modeling and classification discrimination under the same network architecture, thereby improving its recognition ability for complex anomaly patterns.

[0051] For the variable learning rate strategy, a variable learning rate scheduling strategy is introduced during the training process, preferably a cosine annealing scheduling. A higher learning rate is set in the initial stage to enable the model to quickly enter the convergence region, and as the training round number advances, the learning rate is smoothly reduced according to the cosine function formula, gradually reducing the step size, thereby stabilizing the optimization in the later period, and the minimum learning rate is set as a fixed lower limit to avoid slow convergence speed. This strategy avoids the oscillation or stagnation problem caused by fixed learning rate, effectively improving the smoothness of the training process and the final accuracy of the model.

[0052] For the early stopping mechanism, to prevent overfitting of the model in the later training period, the present invention introduces an early stopping mechanism. That is, if the validation set loss does not decrease significantly within a certain number of consecutive training periods, the training is terminated in advance. The tolerance round number is set, such as 10 consecutive rounds of error without decrease; the learning rate can be adjusted in advance in combination with the scheduler; if there is still no improvement, the training process is terminated. This mechanism not only saves training time, but also improves the generalization ability of the model.

[0053] The application adopts a variable learning rate strategy combined with an early stop mechanism to dynamically adjust the learning rhythm according to the verification error. The strategy maintains a faster convergence speed in the early stage of training, automatically reduces the learning rate in the later fine optimization stage to suppress oscillation, and decisively terminates the training when the performance no longer improves, effectively preventing overfitting, saving computing resources, and ensuring the stability and convergence reliability of model training.

[0054] In some embodiments, the step of periodic fine-tuning specifically includes: after triggering the alarm, recording the detailed parameters of the abnormal event and generating an evaluation report; after verification, the results are backfilled as labels; the data with new labels are stored in the transformer bushing fault database to form training samples for driving model incremental learning; using the newly incremental training sample set to periodically fine-tune and update the model with the deployed model parameters as the initial weights.

[0055] Specifically, after completing the model training, the best model parameter file is deployed to the running platform, which can select an edge computing node, such as a substation on-site server or a cloud platform server, to receive real-time collected data and perform online prediction tasks. The deployed system has stream inference capability and can receive real-time preprocessed latest state parameters, input the improved Pyraformer model, and output the key parameter value trend prediction results in the future several time steps, such as the final screen current and capacitance in the future 6 hours; abnormal state occurrence probability based on model internal classification output. The model inference result is updated at a set frequency for subsequent early warning judgment module.

[0056] The system compares the model output result with the set early warning threshold, triggers multi-level alarms such as early warning level, high risk level and serious level when the prediction value or probability exceeds the threshold, and pushes the alarm information through the platform, which can be linked with manual review and dispatching process. For time periods with multiple triggered abnormalities, the system supports backflow of confirmed data, periodically updates the model using fine-tuning or incremental learning, realizes closed-loop self-learning and performance continuous improvement.

[0057] The application constructs a complete closed-loop self-learning mechanism through abnormal data backflow and periodic fine-tuning update. The system can continuously optimize model parameters using newly generated field data, making the model continuously adapt to changes in device state and new fault modes, realizing the upgrade from static prediction to dynamic evolution, and ensuring the performance and continuous effectiveness in long-term operation.

[0058] In some embodiments, the prediction model employed in the present application is an improved Pyraformer, which is composed of multi-feature embedding, pyramid structure modeling module PAM and double-path prediction decoder, and has the ability to efficiently model long sequence dependencies. The Pyraformer model first embeds the observed principal variables and time class covariates respectively, and adds position encoding, then merges the inputs into the backbone module CSCM. CSCM is composed of several layers of pyramid attention modules (PAM) and feed forward networks (Feed Forward), each layer contains residual connection and normalization operation to improve stability.

[0059] The pyramid attention module adopts a layer-by-layer coarse scale sampling method to construct a pyramid type context structure, constructs sub-sequences of different granularities from top to bottom by controlling the sampling step, and calculates local attention within each layer. This structure significantly reduces the impact of sequence length on computational complexity, while retaining the ability of global information flow transmission, especially suitable for low-frequency, strong periodicity power time series modeling tasks. In the prediction phase, the model supports two decoding strategies: one is direct feature aggregation, which outputs after feature extraction followed by a linear layer; the second is a decoder with attention mechanism, which can enhance the model's ability to model future states. The second strategy is preferred in the specific implementation of the present application to improve the response ability to trend changes and abnormal fluctuations.

[0060] The Pyraformer model employed in the present application introduces a CSCM module in the input layer, which is used for coarse sampling context modeling. The module first performs dimension transformation on the input time series through linear mapping, then sequentially passes through multiple layers of convolution operation with step C to construct coarse-grained sub-sequence feature representation layer by layer. The convolution results of each scale are aggregated through jump connection, and then mapped back to the original feature dimension, providing enhanced representation of local and global features for the attention layer, and improving the performance of the model in multi-period long sequence modeling tasks.

[0061] In a preferred embodiment of the present application, the hyperparameter optimization algorithm adopts an improved Bayesian optimization framework. First, randomly select some hyperparameter combinations in the hyperparameter space to train and evaluate the model, obtain the corresponding objective function values as initial data; build a surrogate model: according to the initial data, use the selected model such as Gaussian process or random forest to build a surrogate model of the objective function, which can predict the objective function value and uncertainty of any hyperparameter combination; calculate the expected improvement on the posterior distribution of the surrogate model through the acquisition function UCB, thereby determining the next hyperparameter combination to be evaluated; input the combination into the actual model and calculate its performance indicator on the validation set as the objective function value; add the newly obtained data to the training set and refit the surrogate model to update the posterior distribution; judge whether the current iteration number reaches the preset upper limit or the performance improvement of continuous iterations is less than the threshold, if any condition is met, end the optimization and output the current optimal hyperparameter combination, otherwise return to continue iteration. Through this closed-loop process, the global optimal hyperparameter can be efficiently approximated within a limited number of evaluations, thereby significantly improving the final performance of the model.

[0062] Please refer to Figure 4 In another aspect, the present application also provides a transformer bushing fault identification system based on multi-feature fusion and attention mechanism, characterized in that it comprises: an acquisition module for acquiring key state parameters from a transformer bushing fault database and preprocessing to generate standard time series samples, wherein the key state parameters include end screen current and bushing capacitance; an improved model construction module for adding a feature fusion module to the front end of the model encoder based on the Pyraformer model architecture to realize adaptive fusion of features, and introducing a time difference weight mechanism in the attention mechanism to enhance the model's time perception ability, thereby constructing an improved Pyraformer model; an optimization module for automatically searching and configuring the key hyperparameters of the improved Pyraformer model using a Bayesian optimization algorithm; a model training module for training the improved Pyraformer model through the standard time series samples, and using a variable learning rate strategy and early stopping mechanism for joint control during training; an output module for real-time acquisition of key state parameters of the transformer bushing, inputting the preprocessed parameters into the trained improved Pyraformer model, outputting the predicted trend value and abnormal state probability of the key state parameters in the future preset time period, triggering multi-level alarms after comparison with the preset threshold, and returning the abnormal data to the training set for periodic incremental learning and self-updating of the model.

[0063] In another aspect, the present application also provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the transformer bushing fault identification method based on multi-feature fusion and attention mechanism when executing the computer program.

[0064] In another aspect, the present application also provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the transformer bushing fault identification method based on multi-feature fusion and attention mechanism when executed by a processor.

[0065] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or all or part of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0066] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing relevant hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, database or other medium used in the embodiments of the present application can include non-volatile and / or volatile memory. The non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. The volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0067] The above is only an embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent transformation or direct or indirect application in the related technical field using the content of the present application specification and drawings is also included in the patent protection scope of the present application.

Claims

1. A method for identifying transformer bushing faults based on multi-feature fusion and attention mechanism, characterized in that the steps include... include: Key state parameters are obtained from the transformer bushing fault database and preprocessed to generate standard time series samples. The key state parameters include the end screen current and bushing capacitance. Based on the Pyraformer model architecture, a feature fusion module is added to the front end of the model encoder to achieve adaptive feature fusion, and a time difference weight mechanism is introduced into the attention mechanism to enhance the model's time awareness, thereby constructing an improved Pyraformer model. The key hyperparameters of the improved Pyraformer model are automatically searched and configured using a Bayesian optimization algorithm. The improved Pyraformer model is trained using the standard time series samples, and a variable learning rate strategy and an early stopping mechanism are used for coordinated control during the training process. The system collects key state parameters of transformer bushings in real time, preprocesses them, and inputs them into the improved Pyraformer model after training. It outputs the predicted trend values ​​of key state parameters and the probability of abnormal states within a preset time period. After comparison with preset thresholds, it triggers multi-level alarms and feeds abnormal data back to the training set for the model's periodic incremental learning and self-updating.

2. The transformer bushing fault identification method based on multi-feature fusion and attention mechanism according to claim 1, characterized in that, The preprocessing steps specifically include: The end-screen current and the bushing capacitance are timestamped and resampled at the hour level. A periodic time covariate feature is constructed, a fixed-length sliding window sample is divided, and each sample is standardized to construct a three-dimensional input tensor [B, T, D] as a standard time series sample; where B is the number of samples; T is the time step; and D is the feature dimension, including the end screen current, bushing capacitance, and periodic covariates constructed from timestamps.

3. The transformer bushing fault identification method based on multi-feature fusion and attention mechanism according to claim 1, characterized in that, The specific steps of adding a feature fusion module to the front end of the model encoder include: Independent subspace embedding paths are constructed for the leakage current sequence of the last screen and the bushing capacitance sequence, respectively, wherein each path consists of a convolutional mapping layer; The two feature vectors are concatenated along the channel dimension, and a global average pooling operation is performed on the concatenated feature map to generate a channel descriptor vector. The channel descriptor vector is passed sequentially through two fully connected layers and a Sigmoid activation function to generate a weight vector representing the importance of each channel; The generated weight vector is broadcast along the channel dimension and multiplied channel by channel with the concatenated feature map to achieve adaptive recalibration of features; Layer normalization is used to normalize the recalibrated features across channels to unify the statistical distribution of features in each channel, and finally outputs the feature fusion tensor.

4. The transformer bushing fault identification method based on multi-feature fusion and attention mechanism according to claim 1, characterized in that, The steps for introducing a time difference weighting mechanism into the attention mechanism specifically include: Calculate the difference in the actual physical time interval between the corresponding data points of the query vector and the key vector in the attention mechanism; The time interval difference is mapped through a learnable location encoding vector or a periodic time function to obtain a scalar bias term for adjusting attention weights; The scalar bias term is introduced additively into the calculation of the original attention score, making the allocation of attention weights explicitly dependent on the temporal proximity between data points.

5. The transformer bushing fault identification method based on multi-feature fusion and attention mechanism according to claim 1, characterized in that, The steps for automatically searching and configuring the key hyperparameters of the improved Pyraformer model using the Bayesian optimization algorithm specifically include: Using the joint loss function value of the model on the validation set as the objective function, we jointly optimize and search for multiple hyperparameters of the improved Pyraformer model. The joint loss function is composed of a weighted combination of the mean squared error loss function and the cross-entropy loss function. The hyperparameters include the number of encoder layers, the number of decoder layers, the number of attention heads, and the feature dimension; The Bayesian optimization process employs a surrogate model based on Gaussian processes and a collection function based on a high confidence upper bound to guide the hyperparameter search.

6. The transformer bushing fault identification method based on multi-feature fusion and attention mechanism according to claim 1, characterized in that, The steps for coordinated control using a variable learning rate strategy and an early stopping mechanism specifically include: If the validation set loss value does not decrease to the predetermined threshold within multiple consecutive training cycles, the learning rate decay mechanism is triggered. If the validation set loss does not improve in subsequent training cycles after the learning rate decays, the early stopping mechanism is triggered to terminate training.

7. The transformer bushing fault identification method based on multi-feature fusion and attention mechanism according to claim 1, characterized in that, The specific steps of the periodic fine-tuning include: After an alarm is triggered, record the detailed parameters of the abnormal event and generate an evaluation report; After verification, the results will be used as labels for backfilling; The data with new labels is stored in the transformer bushing fault database to form training samples for driving incremental learning of the model. Using the deployed model parameters as initial weights, the model is periodically fine-tuned and updated using a new incremental training sample set.

8. A transformer bushing fault identification system based on multi-feature fusion and attention mechanism, characterized in that, include: The acquisition module is used to acquire key state parameters from the transformer bushing fault database and perform preprocessing to generate standard time series samples, wherein the key state parameters include the end screen current and bushing capacitance. An improved model building module is used to build an improved Pyraformer model based on the Pyraformer model architecture. A feature fusion module is added to the front end of the model encoder to achieve adaptive feature fusion, and a time difference weight mechanism is introduced into the attention mechanism to enhance the model’s time awareness. The optimization module is used to automatically search and configure the key hyperparameters of the improved Pyraformer model using a Bayesian optimization algorithm. The model training module is used to train the improved Pyraformer model using the standard time series samples, and employs a variable learning rate strategy and an early stopping mechanism for coordinated control during the training process. The output module is used to collect key state parameters of transformer bushings in real time. After preprocessing, the parameters are input into the improved Pyraformer model after training. The output module outputs the predicted trend values ​​of key state parameters and the probability of abnormal states within a preset time period. After comparison with preset thresholds, multi-level alarms are triggered, and abnormal data is fed back to the training set for the periodic incremental learning and self-updating of the model.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the transformer bushing fault identification method based on multi-feature fusion and attention mechanism as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the transformer bushing fault identification method based on multi-feature fusion and attention mechanism as described in any one of claims 1 to 7.

Images