A method for long-term prediction of dissolved gas concentration in transformer oil

By using a dual-view Transformer model, combined with an adaptive multi-granularity detector and a kernel-corrected attention module, the complex dependency problem of long-term prediction of dissolved gas concentration in transformer oil is solved, achieving high-precision long-term prediction and supporting efficient equipment maintenance and reliability assurance.

CN119848774BActive Publication Date: 2026-11-10HENAN UNIV OF SCI & TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411952824.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-12-27
Publication Date
2026-11-10
Estimated Expiration
2044-12-27

AI Technical Summary

Technical Problem

Existing models struggle to effectively capture the complex dependencies between multiple variables in long-term predictions of dissolved gas concentrations in transformer oil, especially when dealing with highly nonlinear and multi-granular time information, making it difficult to achieve accurate long-term predictions.

Method used

A dual-view Transformer model, including seasonal and trend component channels, is used. Through an adaptive multi-granularity detector and a kernel-corrected attention module, multi-granularity features are extracted and information from the trend and seasonal components is fused to generate the final prediction result.

Benefits of technology

It significantly improves the long-term prediction accuracy of dissolved gas concentration, supports efficient equipment maintenance, extends equipment lifespan, and reduces power supply interruptions, thus having significant practical value.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119848774B_ABST
    Figure CN119848774B_ABST
Patent Text Reader

Abstract

This invention discloses a long-term prediction method for dissolved gas concentration in transformer oil, belonging to the field of power equipment health status monitoring. The method includes: collecting historical dissolved gas data; constructing a dual-view Transformer model for long-term dissolved gas prediction; the dual-view Transformer model includes an adaptive multi-granularity detector, several kernel-corrected attention modules, and a dual-view correlation estimation module; training the dual-view Transformer model based on the historical dissolved gas data to obtain a trained dual-view Transformer model; acquiring the data to be tested, and using the trained dual-view Transformer model to obtain the prediction result for the data to be tested. This invention solves the problem that current methods are unable to comprehensively capture the complex dependencies between multiple variables, especially when dealing with highly nonlinear and multi-granularity time information, which presents significant limitations.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of power equipment health status monitoring, and particularly relates to a long-term prediction method for dissolved gas concentration in transformer oil. Background Technology

[0002] Dissolved gas analysis (DGA) in transformer oil is a widely accepted diagnostic technique for detecting potential faults in oil-immersed transformers. By analyzing the concentration of gases dissolved in the insulating oil, abnormal transformer conditions can be effectively identified. However, to achieve predictive maintenance and early fault warning, relying solely on historical dissolved gas data is insufficient; accurate prediction of future gas concentration trends is equally crucial. Currently, although various time-series forecasting models have been developed for dissolved gas concentration prediction, most models focus on short-term forecasts (such as changes within a single day). In contrast, long-term forecasting is of great significance for improving cost-effective transformer management and optimizing maintenance strategies. Therefore, achieving long-term forecasting while ensuring accuracy has become a core challenge in the field of dissolved gas concentration prediction.

[0003] Existing dissolved gas prediction models typically employ statistical and machine learning methods to process time-series data of dissolved gases in transformer oil. These methods include grey models (GM), artificial neural networks (ANN), and autoregressive integrated moving average (ARIMA) models. However, these methods have limitations when processing long-series data. For example, GM relies on manual feature engineering to extract relevant information from the time series; ANN and ARIMA suffer from low prediction accuracy in long-term predictions due to the accumulation of recursive errors and insufficient ability to handle nonlinear data. The rise of deep learning has provided new solutions to these problems. For instance, the temporal fusion Transformer (TFT) model combines self-attention mechanisms and long short-term memory (LSTM) networks, enabling it to simultaneously capture long- and short-term dependencies between different time steps. However, LSTM may face gradient vanishing or exploding problems when processing long series, which limits its extrapolation capabilities. Furthermore, most existing methods rely on single-granularity pattern representations, making it difficult to simultaneously capture high-frequency local patterns and long-term trend information. High-frequency local patterns can enhance the ability to refine modeling, while long-term trends are crucial for understanding the global state of the system. Nevertheless, existing models still fall short in learning multi-granularity time dependencies and complex couplings between variables, making it difficult to effectively support long-term predictions.

[0004] While existing models have achieved some success in short-term predictions, long-term predictions still face significant challenges. Current methods struggle to fully capture the complex dependencies between multiple variables, exhibiting significant limitations, particularly when dealing with highly nonlinear and multi-granular temporal information. Therefore, effectively capturing multi-granular temporal dependencies and complex couplings between variables remains a crucial problem to be solved in the field of dissolved gas prediction for long-term forecasting tasks. Summary of the Invention

[0005] To address the aforementioned shortcomings in existing technologies, this invention provides a long-term prediction method for dissolved gas concentration in transformer oil. This method solves the problem that current methods struggle to fully capture the complex dependencies between multiple variables, and have significant limitations, especially when dealing with highly nonlinear and multi-granular time information.

[0006] To achieve the above-mentioned objectives, the technical solution adopted by this invention is: a method for long-term prediction of dissolved gas concentration in transformer oil, comprising:

[0007] Historical data on dissolved gases in transformer oil were collected, and trend and seasonal components of the historical dissolved gas data were extracted. The trend and seasonal components were then used to construct matrices to obtain the trend matrix and the seasonal matrix.

[0008] A two-view Transformer model for long-term prediction of dissolved gases is constructed. The two-view Transformer model includes a seasonal component channel, a trend component channel, and a two-view correlation estimation module. Both the seasonal component channel and the trend component channel include an adaptive multi-granularity detector and several stacked kernel-corrected attention modules.

[0009] The seasonal component channel is used to extract multi-granular information from the input matrix using an adaptive multi-granularity detector, thereby obtaining a multi-granularity feature representation of the seasonal component's introduction location information; and based on the multi-granularity feature representation of the seasonal component's introduction location information, a stacked kernel correction attention module is used to obtain the seasonal component representation.

[0010] The trend component channel is used to extract multi-granular information from the input matrix using an adaptive multi-granularity detector, thereby obtaining a multi-granularity feature representation of the introduction location information of the trend component; and based on the multi-granularity feature representation of the introduction location information of the trend component, a stacked kernel correction attention module is used to obtain the trend component representation.

[0011] The dual-view correlation estimation module is used to adaptively fuse the trend component representation and the seasonal component representation to generate the final prediction result.

[0012] Using historical data on dissolved gases in transformer oil, a two-view Transformer model is trained to obtain the trained two-view Transformer model.

[0013] Obtain the test data and use the trained dual-view Transformer model to obtain the prediction results for the test data.

[0014] Furthermore, the expression for the input matrix is:

[0015]

[0016] Among them, X in The input matrix; Let be the concentration of gas C at time point T; T is the number of time steps; and C is the number of gas types.

[0017] Furthermore, the step of extracting multi-granularity information from the input matrix using an adaptive multi-granularity detector to obtain a multi-granularity feature representation of the seasonal component's introduced location information involves: reversibly standardizing each sequence in the input matrix to obtain standardized data; segmenting the standardized data in a channel-independent manner to generate several patches; performing multi-scale convolution on each patch to extract features at different scales and obtain convolution output; performing deep mapping on the convolution output through a multilayer perceptron to obtain embedded features; weighting and fusing the embedded features to obtain a multi-granularity feature representation; and using position encoding to incorporate location information into the multi-granularity feature representation to obtain a multi-granularity feature representation of the seasonal component's introduced location information.

[0018] Furthermore, the expression for the standardized data is:

[0019]

[0020]

[0021]

[0022] in, Let x be the gas sequence of the i-th gas. (i) Standardized data; γ and β are both learnable affine transformation parameters; To prevent division by zero of constants; E[x] (i) [x] is a gas sequence (i) The mean of Var[x] (i) [x] is a gas sequence (i) The variance; T is the number of time steps; j is the time step index; Let be the concentration value of the i-th gas at time point j.

[0023] Furthermore, the expression for the multi-granularity feature representation of the location information introduced by the seasonal component is as follows:

[0024]

[0025]

[0026]

[0027]

[0028]

[0029]

[0030]

[0031]

[0032] in, Multi-granularity feature representation that introduces location information for seasonal components; Multi-granularity feature representation of seasonal components; E pos is the position encoding matrix; D is the embedding dimension; pos is the position; i' is the dimension; K is the total number of scales; k is the index of the scale; W k Let be the fusion weight matrix for the k-th scale; It is the bias vector; For the embedding features at the k-th scale; Here are the weight parameters of the MLP at the k-th scale; MLP k For the k-th scale multilayer perceptron; GELU is the activation function; For the convolution output at the k-th scale; Conv1D (k) For the convolution operation at the k-th scale; The parameters of the convolution kernel at the k-th scale; for The Nth patch after splitting; Let x be the gas sequence of the i-th gas in the input matrix. (i) Standardized data; MGPatchify is used for patch segmentation; P is the patch length; S is the step size; T is the number of time steps; This is for rounding down.

[0033] Furthermore, the step of obtaining the seasonal component representation by utilizing a stacked kernel correction attention module based on the multi-granularity feature representation of the seasonal component's location information specifically involves: representing the seasonal component using the multi-granularity feature representation of the seasonal component's location information. In the input stacked kernel-corrected attention module, for each attention head, The projection is a query matrix, a key matrix, and a value matrix. Based on the query matrix, key matrix, and value matrix, a kernel function and a Kronecker Delta function are introduced to dynamically adjust the attention weights. The dynamically adjusted attention weights are input into a multi-head self-attention mechanism, and the outputs of each attention head are merged to obtain a merged output. The merged output is then input into the next kernel-corrected attention module to obtain the seasonal component representation.

[0034] Furthermore, the expression for the merged output is:

[0035]

[0036]

[0037]

[0038]

[0039] Where KD-MSA(H) represents the merged output; Concat represents the concatenation on the last dimension; H represents the number of attention heads; W O To output the projection matrix; represents the dynamically adjusted attention weights corresponding to the i1th attention head; ⊙ represents element-wise multiplication; Softmax is the activation function; This is the query matrix corresponding to the i1th attention head; This is the transpose of the key matrix corresponding to the i1th attention head; For kernel functions; The Kronecker Delta function; d k The dimension of the key; σ is the value matrix corresponding to the i1th attention head; σ is the bandwidth parameter of the kernel function; The vector representation of the query matrix; Let i1 be the vector representation of the key matrix; i1 and j1 are both attention head indices; ||·|| is the L2 norm.

[0040] Furthermore, residual connections and layer normalization are used to stabilize the training process in each stacked kernel-corrected attention module:

[0041]

[0042] Among them, Z l The final output features of the attention module are corrected for the kernel of layer l; FFN is a feedforward neural network; LN is a layer normalization operation; To correct the attention enhancement features of the attention module after multi-head self-attention in the l-th kernel; KD-MSA(LN(H) l-1 For core correction of multi-head self-attention; H l-1 The input features are used to correct the attention module of the (l-1)th layer kernel.

[0043] Furthermore, the adaptive fusion of trend component representation and seasonal component representation to generate the final prediction result specifically involves projecting the trend component representation and seasonal component representation onto the prediction space through a linear mapping to obtain... and calculate and The mean μ and standard deviation σ are calculated, and a fusion factor λ is learned through a multilayer perceptron; the fusion factor λ is then used to... and Weighted fusion yields the predicted value. Predicted values Perform inverse standardization to obtain the final prediction result.

[0044]

[0045]

[0046] logλ=MLP(μ,σ)

[0047]

[0048]

[0049]

[0050]

[0051] in, For gas sequence x (i) The final prediction result; x (i) Let Var[x] be the gas sequence of the i-th gas. (i) [x] is a gas sequence (i) The variance; ò is a constant to prevent division by zero; For gas sequence x (i) The predicted value; E[x (i) [x] is a gas sequence (i) The mean of ; γ and β are both learnable affine transformation parameters; The seasonal component represents the result projected onto the prediction space; The trend component represents the result projected onto the prediction space; L is the number of time steps for prediction; j' is the index of the time step for prediction. Let the seasonal component represent the result of the time step of the j'th prediction projected onto the prediction space; The trend component represents the result of the time step of the j'th prediction projected onto the prediction space; Represented as trend components; Represented by seasonal components; The projection matrix represents the seasonal components; This is the projection matrix of the trend component.

[0052] Furthermore, the expression for the loss function of the dual-view Transformer model is as follows:

[0053] L = L MSE +L MAE

[0054]

[0055]

[0056] Where L is the loss function; L MSE L represents the mean square error. MAE C is the mean absolute error; C is the number of gas types; i is the gas type index; For gas sequence x (i) The final prediction result; x (i) Let y be the gas sequence of the i-th gas; (i) For gas sequence x (i) The true result; ||·||2 is the L2 norm; ||·||1 is the L1 norm.

[0057] The beneficial effects of this invention are as follows: It effectively solves the limitations of multi-granularity trend and seasonal component cointegration modeling, overcomes the bias problem in traditional attention mechanisms, fully explores the dual-view correlation of time-series data, significantly improves prediction accuracy, and demonstrates broad application prospects. By accurately modeling the long-term trend and high-frequency changes of dissolved gas concentration, this invention can achieve long-term prediction of dissolved gas concentration, support efficient equipment maintenance, extend equipment lifespan, and reduce power supply interruptions caused by equipment failures. It has significant practical value in the intelligent operation and maintenance and reliability assurance of power systems. Attached Figure Description

[0058] Figure 1 This is a flowchart of the method of the present invention.

[0059] Figure 2 This is a schematic diagram of the dual-view Transformer architecture.

[0060] Figure 3 This is a schematic diagram of the structure of an adaptive multi-granularity detector.

[0061] Figure 4 A schematic diagram of the core correction attention module. Detailed Implementation

[0062] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0063] like Figure 1 As shown, in one embodiment of the present invention, a method for long-term prediction of dissolved gas concentration in transformer oil includes:

[0064] Historical data on dissolved gases in transformer oil were collected, and trend and seasonal components of the historical dissolved gas data were extracted. The trend and seasonal components were then used to construct matrices to obtain the trend matrix and the seasonal matrix.

[0065] A two-view Transformer model for long-term prediction of dissolved gases is constructed. The two-view Transformer model includes a seasonal component channel, a trend component channel, and a two-view correlation estimation module. Both the seasonal component channel and the trend component channel include an adaptive multi-granularity detector and several stacked kernel-corrected attention modules.

[0066] The seasonal component channel is used to extract multi-granular information from the input matrix using an adaptive multi-granularity detector, thereby obtaining a multi-granularity feature representation of the seasonal component's introduction location information; and based on the multi-granularity feature representation of the seasonal component's introduction location information, a stacked kernel correction attention module is used to obtain the seasonal component representation.

[0067] The trend component channel is used to extract multi-granular information from the input matrix using an adaptive multi-granularity detector, thereby obtaining a multi-granularity feature representation of the introduction location information of the trend component; and based on the multi-granularity feature representation of the introduction location information of the trend component, a stacked kernel correction attention module is used to obtain the trend component representation.

[0068] The dual-view correlation estimation module is used to adaptively fuse the trend component representation and the seasonal component representation to generate the final prediction result.

[0069] Using historical data on dissolved gases in transformer oil, a two-view Transformer model is trained to obtain the trained two-view Transformer model.

[0070] Obtain the test data and use the trained dual-view Transformer model to obtain the prediction results for the test data.

[0071] In this embodiment, by introducing an adaptive multi-granularity detector and a kernel-corrected attention module, the dual-view Transformer model can learn from multi-granularity temporal information and simultaneously capture long-term and short-term dependencies as well as coupling relationships between multiple variables. Furthermore, the dual-view Transformer model utilizes a dual-view correlation estimation module to adaptively refine the output of the kernel-corrected attention module, hierarchically modeling the trend and detailed information of future curves from trend and seasonal components, thereby maximizing information gain. The specific steps are as follows:

[0072] Step S1: Collect historical data of dissolved gases in transformer oil to construct the input matrix X in The data is then input into a two-view Transformer model. The input data includes the concentrations of C dissolved gases (such as H2, CH4, and C2H6), and long-term predictions are made based on T historical time points, with the goal of predicting the dissolved gas concentrations at L future time points.

[0073] Step S2: Construct an adaptive multi-granularity detector to extract multi-granularity temporal information. The adaptive multi-granularity detector module first segments the time series data into multi-granularity sequence blocks, which are used to capture local high-frequency patterns and long-term seasonal trends, respectively. Furthermore, by performing multi-granularity segmentation and convolution operations on the input data, the adaptive multi-granularity detector can generate multi-granularity embedding features for capturing multi-scale information.

[0074] Step S3: Employ a kernel-corrected attention module to learn long-term dependencies among multiple variables. The kernel-corrected attention module avoids the additional bias introduced by traditional attention mechanisms by combining a kernel function and the Kronecker Delta function to calculate a dynamic scaling factor. Furthermore, through a multi-head self-attention mechanism, the kernel-corrected attention module can effectively extract long-term dependencies in trend and seasonal components.

[0075] Step S4: The dual-view correlation estimation module performs hierarchical information fusion on the output of the kernel-corrected attention module. Based on the statistical characteristics of the trend and seasonal components, the dual-view correlation estimation module learns the fusion factor λ to achieve hierarchical modeling of future curve trends and details. Furthermore, the dual-view representation vector is projected onto the prediction space, and a multilayer perceptron is used to learn the weighted relationship between the trend and seasonal components, ultimately generating a prediction of future gas concentrations.

[0076] Step S5: Make predictions using the trained optimal network and compare the predicted results with the actual results using an error evaluation metric. The loss function includes mean squared error (MSE) and mean absolute error (MAE) in a 1:1 ratio to balance the model's performance in terms of accuracy and error sensitivity, maximizing the model's predictive effectiveness.

[0077] In this embodiment, as Figure 2 The diagram shows the structure of a two-view Transformer model. It illustrates how the two-view Transformer model extracts multi-granularity features through an adaptive multi-granularity detector, models long-term cross-variable dependencies using a kernel-corrected attention module, and finally generates prediction results through two-view correlation estimation.

[0078] The expression for the input matrix is:

[0079]

[0080] Among them, X in The input matrix; Let be the concentration of gas C at time point T; T is the number of time steps; and C is the number of gas types.

[0081] In this embodiment, historical data is represented as

[0082] The method of extracting multi-granular information from the input matrix using an adaptive multi-granularity detector to obtain a multi-granularity feature representation of the seasonal component's location information involves: reversibly standardizing each sequence in the input matrix to obtain standardized data; segmenting the standardized data in a channel-independent manner to generate several patches; performing multi-scale convolution on each patch to extract features at different scales and obtain convolution output; performing deep mapping on the convolution output through a multilayer perceptron to obtain embedded features; weighting and fusing the embedded features to obtain a multi-granularity feature representation; and using position encoding to incorporate location information into the multi-granularity feature representation to obtain a multi-granularity feature representation of the seasonal component's location information.

[0083] In this embodiment, the process of obtaining the multi-granularity feature representation of the introduction location information of the trend component is the same as that of the multi-granularity feature representation of the introduction location information of the seasonal component.

[0084] The expression for the standardized data is:

[0085]

[0086]

[0087]

[0088] in, Let x be the gas sequence of the i-th gas. (i) Standardized data; γ and β are both learnable affine transformation parameters; To prevent division by zero of constants; E[x] (i) [x] is a gas sequence (i) The mean of Var[x] (i) [x] is a gas sequence (i) The variance; T is the number of time steps; j is the time step index; Let be the concentration value of the i-th gas at time point j.

[0089] In this embodiment, the adaptive multi-granularity detector aims to extract multi-granularity features from time-series data, including local high-frequency patterns and long-term seasonal trends. First, to reduce the impact of data non-stationarity, for each input gas sequence x... (i) Perform Reversible Instance Normalization (RevIN) operation.

[0090] The expression for the multi-granularity feature representation of the location information of the seasonal component is as follows:

[0091]

[0092]

[0093]

[0094]

[0095]

[0096]

[0097]

[0098]

[0099] in, Multi-granularity feature representation that introduces location information for seasonal components; Multi-granularity feature representation of seasonal components; E pos is the position encoding matrix; D is the embedding dimension; pos is the position; i' is the dimension; K is the total number of scales; k is the index of the scale; W k Let be the fusion weight matrix for the k-th scale; It is the bias vector; For the embedding features at the k-th scale; Here are the weight parameters of the MLP at the k-th scale; MLP k For the k-th scale multilayer perceptron; GELU is the activation function; For the convolution output at the k-th scale; Conv1D (k) For the convolution operation at the k-th scale; The parameters of the convolution kernel at the k-th scale; for The Nth patch after splitting; Let x be the gas sequence of the i-th gas in the input matrix. (i) Standardized data; MGPatchify is used for patch segmentation; P is the patch length; S is the step size; T is the number of time steps; This is for rounding down.

[0100] In this embodiment, Figure 3 This is a schematic diagram of an adaptive multi-granularity detector. It shows the temporal block structure of the input data after multi-granularity segmentation and demonstrates how channel-independent processing models local and global dependencies.

[0101] The multi-granularity feature representation based on the location information of the seasonal component is used to obtain the seasonal component representation using a stacked kernel correction attention module. Specifically, this involves: representing the multi-granularity feature representation of the location information of the seasonal component. In the input stacked kernel-corrected attention module, for each attention head, The projection is a query matrix, a key matrix, and a value matrix. Based on the query matrix, key matrix, and value matrix, a kernel function and a Kronecker Delta function are introduced to dynamically adjust the attention weights. The dynamically adjusted attention weights are input into a multi-head self-attention mechanism, and the outputs of each attention head are merged to obtain a merged output. The merged output is then input into the next kernel-corrected attention module to obtain the seasonal component representation.

[0102] In this embodiment, the number of stacked kernel attention modules in the trend component channel is less than that in the seasonal component channel.

[0103] The expression for the merged output is:

[0104]

[0105]

[0106]

[0107]

[0108] Where KD-MSA(H) represents the merged output; Concat represents the concatenation on the last dimension; H represents the number of attention heads; W O To output the projection matrix; represents the dynamically adjusted attention weights corresponding to the i1th attention head; ⊙ represents element-wise multiplication; Softmax is the activation function; This is the query matrix corresponding to the i1th attention head; This is the transpose of the key matrix corresponding to the i1th attention head; For kernel functions; The Kronecker Delta function; d k The dimension of the key; σ is the value matrix corresponding to the i1th attention head; σ is the bandwidth parameter of the kernel function; The vector representation of the query matrix; Let i1 be the vector representation of the key matrix; i1 and j1 are both attention head indices; ||·|| is the L2 norm.

[0109] In each stacked kernel-corrected attention module, residual connections and layer normalization are used to stabilize the training process:

[0110]

[0111]

[0112] Among them, Z l The final output features of the attention module are corrected for the kernel of layer l; FFN is a feedforward neural network; LN is a layer normalization operation; To correct the attention enhancement features of the attention module after multi-head self-attention in the l-th kernel; KD-MSA(LN(H) l-1 For core correction of multi-head self-attention; H l-1 The input features are used to correct the attention module of the (l-1)th layer kernel.

[0113] In this embodiment, Figure 4This is a schematic diagram of the kernel-corrected attention module. It details how the kernel-corrected attention module calculates dynamic attention scores using kernel functions and the Kronecker Delta function, and captures long-term dependencies through a multi-head self-attention mechanism.

[0114] In this embodiment, the kernel-corrected attention module is used to capture long-term dependencies between multiple variables. For each attention head i1, the input features are... (Seasonal component) projection into a query, key, and value matrix:

[0115]

[0116]

[0117]

[0118] in, Let be the projection matrix of the i1th attention head.

[0119] To address the potential bias in attention scores in traditional attention mechanisms, a kernel function and the Kronecker Delta function are introduced to dynamically adjust attention weights.

[0120] In this embodiment, the feedforward neural network typically consists of two fully connected layers and an activation function.

[0121] The adaptive fusion of trend component representation and seasonal component representation to generate the final prediction result specifically involves projecting the trend component representation and seasonal component representation onto the prediction space through a linear mapping to obtain... and calculate and The mean μ and standard deviation σ are calculated, and a fusion factor λ is learned through a multilayer perceptron; the fusion factor λ is then used to... and Weighted fusion yields the predicted value. Predicted values Perform inverse standardization to obtain the final prediction result.

[0122]

[0123]

[0124] logλ=MLP(μ,σ)

[0125]

[0126]

[0127]

[0128]

[0129] in, For gas sequence x (i) The final prediction result; x (i) Let Var[x] be the gas sequence of the i-th gas. (i) [x] is a gas sequence (i) The variance; To prevent division by zero of constants; For gas sequence x (i) The predicted value; E[x (i) [x] is a gas sequence (i) The mean of ; γ and β are both learnable affine transformation parameters; The seasonal component represents the result projected onto the prediction space; The trend component represents the result projected onto the prediction space; L is the number of time steps for prediction; j' is the index of the time step for prediction. Let the seasonal component represent the result of the time step of the j'th prediction projected onto the prediction space; The trend component represents the result of the time step of the j'th prediction projected onto the prediction space; Represented as trend components; Represented by seasonal components; The projection matrix represents the seasonal components; This is the projection matrix of the trend component.

[0130] In this embodiment, the dual-view correlation estimation module is used to adaptively fuse information from the trend component and the seasonal component to generate the final prediction result.

[0131] The expression for the loss function of the dual-view Transformer model is as follows:

[0132] L = L MSE +L MAE

[0133]

[0134]

[0135] Where L is the loss function; L MSE L represents the mean square error. MAE C is the mean absolute error; C is the number of gas types; i is the gas type index; For gas sequence x (i) The final prediction result; x(i) Let y be the gas sequence of the i-th gas; (i) For gas sequence x (i) The true result; ||·||2 is the L2 norm; ||·||1 is the L1 norm.

Claims

1. A method for long-term prediction of dissolved gas concentration in transformer oil, characterized in that, include: Historical data on dissolved gases in transformer oil were collected, and trend and seasonal components of the historical dissolved gas data were extracted. The trend and seasonal components were then used to construct matrices to obtain the trend matrix and the seasonal matrix. A two-view Transformer model for long-term prediction of dissolved gases is constructed. The two-view Transformer model includes a seasonal component channel, a trend component channel, and a two-view correlation estimation module. Both the seasonal component channel and the trend component channel include an adaptive multi-granularity detector and several stacked kernel-corrected attention modules. The seasonal component channel is used to extract multi-granular information of the input matrix using the seasonal component matrix as the input matrix, and obtain a multi-granular feature representation of the introduction location information of the seasonal component. Based on the multi-granularity feature representation of the location information of the seasonal component, a stacked kernel correction attention module is used to obtain the representation of the seasonal component; The trend component channel is used to extract multi-granular information of the input matrix using the trend component matrix as the input matrix, and obtain a multi-granular feature representation of the introduction location information of the trend component. Based on the multi-granularity feature representation of the location information introduced by the trend component, the stacked kernel correction attention module is used to obtain the trend component representation; The dual-view correlation estimation module is used to adaptively fuse the trend component representation and the seasonal component representation to generate the final prediction result. Using historical data on dissolved gases in transformer oil, a two-view Transformer model is trained to obtain the trained two-view Transformer model. Obtain the test data and use the trained dual-view Transformer model to obtain the prediction results for the test data.

2. The method for long-term prediction of dissolved gas concentration in transformer oil according to claim 1, characterized in that, The expression for the input matrix is: Among them, X in The input matrix; Let be the concentration of gas C at time point T; T is the number of time steps; and C is the number of gas types.

3. The method for long-term prediction of dissolved gas concentration in transformer oil according to claim 1, characterized in that, The method of extracting multi-granularity information from the input matrix using an adaptive multi-granularity detector to obtain multi-granularity feature representations of the location information of the seasonal components involves: reversibly standardizing each sequence in the input matrix to obtain standardized data; segmenting the standardized data in a channel-independent manner to generate several patches; performing multi-scale convolution on each patch to extract features at different scales and obtain convolution output; and performing deep mapping on the convolution output through a multilayer perceptron to obtain embedded features. The embedded features are weighted and fused to obtain multi-granularity feature representations; Location encoding is used to incorporate location information into the multi-granularity feature representation, resulting in a multi-granularity feature representation of the seasonal component that incorporates location information.

4. The method for long-term prediction of dissolved gas concentration in transformer oil according to claim 3, characterized in that, The expression for the standardized data is: in, Let x be the gas sequence of the i-th gas. (i) Standardized data; γ and β are learnable affine transformation parameters; ò is a constant to prevent division by zero; E[x (i) [x] is a gas sequence (i) The mean of Var[x] (i) [x] is a gas sequence (i) The variance; T is the number of time steps; j is the time step index; Let be the concentration value of the i-th gas at time point j.

5. The method for long-term prediction of dissolved gas concentration in transformer oil according to claim 3, characterized in that, The expression for the multi-granularity feature representation of the location information of the seasonal component is as follows: in, Multi-granularity feature representation that introduces location information for seasonal components; Multi-granularity feature representation of seasonal components; E pos is the position encoding matrix; D is the embedding dimension; pos is the position; i' is the dimension; K is the total number of scales; k is the index of the scale; W k Let be the fusion weight matrix for the k-th scale; It is the bias vector; For the embedding features at the k-th scale; Here are the weight parameters of the MLP at the k-th scale; MLP k For the k-th scale multilayer perceptron; GELU is the activation function; For the convolution output at the k-th scale; Conv1D (k) For the convolution operation at the k-th scale; The parameters of the convolution kernel at the k-th scale; for The Nth patch after splitting; Let x be the gas sequence of the i-th gas in the input matrix. (i) Standardized data; MGPatchify is used for patch segmentation; P is the patch length; S is the step size; T is the number of time steps; This is for rounding down.

6. The method for long-term prediction of dissolved gas concentration in transformer oil according to claim 1, characterized in that, The multi-granularity feature representation based on the location information of the seasonal component is used to obtain the seasonal component representation using a stacked kernel correction attention module. Specifically, this involves: representing the multi-granularity feature representation of the location information of the seasonal component. In the input stacked kernel-corrected attention module, for each attention head, The projection consists of a query matrix, a key matrix, and a value matrix; based on the query matrix, key matrix, and value matrix, a kernel function and a Kronecker Delta function are introduced to dynamically adjust the attention weights; The dynamically adjusted attention weights are input into the multi-head self-attention mechanism, and the outputs of each attention head are merged to obtain the merged output. The merged output is then input into the next kernel-corrected attention module to obtain the seasonal component representation.

7. The method for long-term prediction of dissolved gas concentration in transformer oil according to claim 6, characterized in that, The expression for the merged output is: Where KD-MSA(H) represents the merged output; Concat represents the concatenation on the last dimension; H represents the number of attention heads; W O To output the projection matrix; represents the dynamically adjusted attention weights corresponding to the i1th attention head; ⊙ represents element-wise multiplication; Softmax is the activation function; This is the query matrix corresponding to the i1th attention head; This is the transpose of the key matrix corresponding to the i1th attention head; For kernel functions; The Kronecker Delta function; d k The dimension of the key; σ is the value matrix corresponding to the i1th attention head; σ is the bandwidth parameter of the kernel function; The vector representation of the query matrix; Let i1 be the vector representation of the key matrix; i1 and j1 are both attention head indices; ||·|| is the L2 norm.

8. The method for long-term prediction of dissolved gas concentration in transformer oil according to claim 6, characterized in that, In each stacked kernel-corrected attention module, residual connections and layer normalization are used to stabilize the training process: Among them, Z l The final output features of the attention module are corrected for the kernel of layer l; FFN is a feedforward neural network; LN is a layer normalization operation; To correct the attention enhancement features of the attention module after multi-head self-attention in the l-th kernel; KD-MSA(LN(H) l-1 For core correction of multi-head self-attention; H l-1 The input features are used to correct the attention module of the (l-1)th layer kernel.

9. The method for long-term prediction of dissolved gas concentration in transformer oil according to claim 1, characterized in that, The adaptive fusion of trend component representation and seasonal component representation to generate the final prediction result specifically involves projecting the trend component representation and seasonal component representation onto the prediction space through a linear mapping to obtain... and calculate and The mean μ and standard deviation σ are obtained, and the fusion factor λ is learned through a multilayer perceptron. Using the fusion factor λ and Weighted fusion yields the predicted value. Predicted values Perform inverse standardization to obtain the final prediction result. logλ=MLP(μ,σ) in, For gas sequence x (i) The final prediction result; x (i) Let Var[x] be the gas sequence of the i-th gas. (i) [x] is a gas sequence (i) The variance; ò is a constant to prevent division by zero; For gas sequence x (i) The predicted value; E[x (i) [x] is a gas sequence (i) The mean of ; γ and β are both learnable affine transformation parameters; The seasonal component represents the result projected onto the prediction space; The trend component represents the result projected onto the prediction space; L is the number of time steps for prediction; j' is the index of the time step for prediction. Let the seasonal component represent the result of the time step of the j'th prediction projected onto the prediction space; The trend component represents the result of the time step of the j'th prediction projected onto the prediction space; Represented as trend components; Represented by seasonal components; The projection matrix represents the seasonal components; This is the projection matrix of the trend component.

10. The method for long-term prediction of dissolved gas concentration in transformer oil according to claim 1, characterized in that, The expression for the loss function of the dual-view Transformer model is as follows: L=L MSE +L MAE Where L is the loss function; L MSE L represents the mean square error. MAE C is the mean absolute error; C is the number of gas types; i is the gas type index; For gas sequence x (i) The final prediction result; x (i) Let y be the gas sequence of the i-th gas; (i) For gas sequence x (i) The true result; ||·||2 is the L2 norm; ||·||1 is the L1 norm.