Transformer operation data prediction method and system
By employing noise reduction processing and frequency attention modulation, combined with multinomial regression and Fourier synthesis, the problems of long-range dependence and noise interference in transformer operation data were solved, achieving stable and high-precision prediction.
Patent Information
- Application Number
- CN202511816249.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-04
- Publication Date
- 2026-03-06
AI Technical Summary
Existing time series prediction models struggle to effectively capture long-range dependencies across multiple time scales in transformer operation data, and are susceptible to noise and abnormal fluctuations, resulting in unstable prediction results and poor generalization ability.
By employing noise reduction and frequency attention weight modulation, and selecting the optimal query vector and conditional features for denoising, combined with multinomial regression and Fourier synthesis, a predicted sequence of transformer operation data is generated.
It achieves accurate capture and reconstruction of multiple periodic patterns in transformer data, improves the stability and accuracy of prediction, effectively resists noise interference, preserves key details, and reduces computational complexity.
Smart Images
Figure QLYQS_1 
Figure QLYQS_10 
Figure QLYQS_23
Abstract
Description
Technical Field
[0001] This invention relates to the field of transformer operation monitoring technology, specifically to a method and system for predicting transformer operation data. Background Technology
[0002] In power systems, power transformers are critical assets, and their operational status directly affects the stability and security of the power grid. Accurate time-series analysis and forecasting of transformer operating data are core technologies for achieving condition-based maintenance, fault early warning, and lifespan assessment. This data is typical of industrial time series data, exhibiting complex periodicity, trends, and high noise characteristics.
[0003] However, existing time series prediction models have shortcomings in this scenario: transformer load and temperature changes are not only affected by intraday electricity consumption patterns, but also exhibit significant periodic characteristics. Existing technologies cannot effectively grasp these long-range dependencies spanning multiple time scales. Industrial field data inevitably contains a large amount of measurement noise, instantaneous interference, and abnormal fluctuations caused by equipment start-up and shutdown. Existing technology prediction results are unstable and have poor generalization ability. Summary of the Invention
[0004] This invention provides a method and system for predicting transformer operation data.
[0005] The technical solution of this invention is as follows: A method for predicting transformer operating data, comprising: S1. Select a noise level of a certain noise level and add noise to the known time series of transformer operation data to obtain the noisy time series; S2. Obtain the dot product of the key vector and query vector of the noisy time series. Based on the difference between the maximum value and the average value, select the optimal query vector for each key vector, extract the features of all optimal query vectors, and obtain the first feature. S3. Divide the first feature into several groups of sub-features along the channel dimension, obtain multiple frequency components of each group of sub-features, concatenate the multiple frequency components of all sub-features, and obtain the frequency attention weights through activation processing. Use the frequency attention weights to modulate the first feature to obtain the conditional feature. S4. Based on the conditional features, the noisy time series is denoised to obtain a coarse series; S5. Concatenate the noisy time series with the coarse series to form the input series. Obtain the dot product of the key vector and the query vector of the input series. Based on the difference between the maximum value and its average value, select the optimal query vector for each key vector. Use the projection of the conditional features as the key vector and value vector. Perform cross-attention processing based on the key vector, value vector, and the selected optimal query vector to obtain a new feature representation. S6. Obtain the trend series through multinomial regression based on the new feature representation. Obtain the periodic series based on the amplitude, phase, and frequency values of multiple frequency components. Add the trend series and the periodic series to obtain the predicted series of transformer operation data.
[0006] In S2, the dot product of the key vector and query vector of the noisy time series is obtained. Based on the difference between the maximum value and the average value, the optimal query vector is selected for each key vector. The specific calculation method is as follows: , in, For measurement, For the query vector OK, The first key vector List, Representing dimension, It is the number of key vectors; choose The query vector with the largest value is selected as the optimal query vector.
[0007] The multiple frequency components of each group of sub-features in S3 are calculated using the following method: , in, For frequency components, DCT is the Discrete Cosine Transform calculation. Let l be the feature of the i-th group, and l be the feature of the i-th group. Each time step The length of the sub-feature. For the first Sub-features in the first Feature values at each time step Indicates sub-features The corresponding specified frequency index, For length is of The basis functions.
[0008] The basis functions are calculated as follows: , in, Indicates length is One-dimensional signal The basis functions of the i-th frequency component, l is the i-th frequency component. Each time step .
[0009] In S4, based on conditional features, the noisy time series is denoised to obtain a coarse sequence, specifically: Starting with standard Gaussian noise of the noise level and using conditional features, the noise at the current time step is predicted. After multiple iterations, the predicted noise is subtracted from the current noisy time series to obtain a coarse sequence.
[0010] The multiple iterations are reverse diffusion.
[0011] S6 also includes obtaining residual terms by removing noise from the trend sequence and periodic sequence based on the noisy time series, and adding the residual terms to the trend sequence and periodic sequence to obtain the prediction sequence.
[0012] In S6, Fourier synthesis is used to obtain a periodic sequence based on the amplitude, phase, and frequency values of multiple frequency components.
[0013] A transformer operation data prediction system, used to implement the above-mentioned transformer operation data prediction method, includes: Given a sequence noise addition module, a noise level of a certain level is selected, and noise is added to a known time series of transformer operating data to obtain a noisy time series; The first feature extraction module obtains the dot product of the key vector and the query vector of the noisy time series. Based on the difference between the maximum value and the average value, it selects the optimal query vector for each key vector and extracts the features of all optimal query vectors to obtain the first feature. The conditional feature acquisition module divides the first feature into several groups of sub-features along the channel dimension, acquires multiple frequency components of each group of sub-features, concatenates the multiple frequency components of all sub-features, and obtains frequency attention weights through activation processing. The frequency attention weights are then used to modulate the first feature to obtain the conditional feature. The denoising module, based on conditional features, denoises the noisy time series to obtain a coarse sequence; The new feature representation acquisition module concatenates the noisy time series with the coarse series to form the input sequence. It obtains the dot product of the key vector and the query vector of the input sequence. Based on the difference between the maximum value and the average value, it selects the optimal query vector for each key vector. The projection of the conditional features is used as the key vector and value vector. Based on the key vector, value vector, and the selected optimal query vector, cross-attention processing is performed to obtain the new feature representation. The prediction sequence acquisition module obtains the trend sequence through multinomial regression based on the new feature representation. It obtains the periodic sequence based on the amplitude, phase, and frequency values of multiple frequency components. The trend sequence and the periodic sequence are added together to obtain the prediction sequence of the transformer operation data.
[0014] Periodic sequences are obtained by Fourier synthesis of the amplitude, phase, and frequency values of multiple frequency components.
[0015] This invention, applied to power transformer data monitoring, brings the following significant benefits: 1. This invention can dynamically analyze and focus on the spectral components of different frequencies in transformer data. This differentiated weight allocation of full-spectrum information can accurately capture and reconstruct the multiple periodic patterns contained in the data. The generated sequence exhibits a high degree of realism in periodicity, effectively solving the problem of single or vague periodic capture in existing technologies.
[0016] 2. This invention uses standard Gaussian distribution noise as a starting point for stepwise denoising, separating the true operating rules from noisy observation data. This gives it a natural resistance to common noise and outliers in the data. The final generated sequence can smooth out noise interference and restore and retain key details, thus improving the stability of prediction.
[0017] 3. This invention selects the optimal query vector, reducing computational complexity and enabling more efficient processing of long-sequence data generated by power transformers. Detailed Implementation
[0018] The technical solution of this invention is as follows: A method for predicting transformer operating data, comprising: S1. Select a noise level of a certain level and add noise to the known time series of transformer operation data to obtain the noisy time series.
[0019] The main experimental scenario of this application is to predict the temperature of transformer operating data. Therefore, it is first necessary to obtain a time series composed of known operating data over a period of time. The known operating data includes one of the transformer's temperature, current, power, and load parameters, which are time series generated as they change over time.
[0020] A noise level t is selected, where t is an integer between 1 and the total number of diffusion steps T. Based on preset forward diffusion noise scheduling parameters, noise is added to a known time series. This noise-adding process simulates one step of the forward diffusion process.
[0021] This step aims to perturb the known time series to an intermediate state with a specific noise level t, providing input for subsequent feature extraction and denoising processes.
[0022] It is important to note that the noise level remains consistent throughout the diffusion process, and it is a key conditional variable connecting different steps.
[0023] S2. Obtain the dot product of the key vector and query vector of the noisy time series. Based on the difference between the maximum value and the average value, select the optimal query vector for each key vector, extract the features of all optimal query vectors, and obtain the first feature.
[0024] When the dot product between a query vector and a key vector contributes significantly, its corresponding attention probability tends to deviate from a uniform distribution. In this case, the optimal query vector with the largest contribution is the most representative feature of the transformer. Conversely, when the probability approaches a uniform distribution, it means that the importance of the query vector in the attention mechanism weakens. Therefore, we can substitute these two probability terms into the following formula and set upper and lower bounds for the metric to approximately estimate the... The metric for each query vector. This formula is the difference between the maximum value of the dot product of the query vector and the key vector and its average value, i.e.: , in, For measurement, For the query vector OK, The first key vector List, Representing dimension, It is the number of key vectors; choose The query vector with the largest value is selected as the optimal query vector.
[0025] Therefore, it can be inferred that the larger the metric value of the query vector, the more significant its dot product contribution to the attention weights. Based on this relationship, the optimal query vector can be selected for each key vector. Subsequently, local self-attention processing can be used to extract features. This method can effectively reduce computational costs while highlighting high-value query vectors, improving the selectivity and expressiveness of attention processing.
[0026] S3. Divide the first feature into several groups of sub-features along the channel dimension, obtain multiple frequency components of each group of sub-features, concatenate the multiple frequency components of all sub-features, and obtain frequency attention weights through activation processing. Use the frequency attention weights to modulate the first feature to obtain the conditional feature.
[0027] The significance of frequency attention weight here is the weight of high-frequency and low-frequency signals of transformer temperature change. Therefore, the conditional features obtained by frequency attention weight modulation can be closer to the known sequence, that is, closer to the historical real data.
[0028] Channel frequency attention weights can help assess the importance and interrelationships between channels, thereby improving accuracy by focusing on task-relevant features and suppressing irrelevant noise.
[0029] This application proposes a multi-channel attention processing method to enhance the expressive power of fine-grained features. This step draws on the concept of channel attention mechanisms in image processing, but is specifically designed for the characteristics of time-series data.
[0030] Traditional channel attention mechanisms in the image domain typically use global average pooling to generate channel statistical descriptors. However, when this concept is transferred to time series analysis, it is found that global average pooling is equivalent to the lowest frequency component of the discrete cosine transform, which leads to the neglect of most frequency information. To construct a richer channel descriptor that includes multiple frequency components, a set of DCT basis functions is used instead of global average pooling. By calculating the DCT coefficients of each channel's time series, multiple frequency components are selected from them to form a new channel descriptor.
[0031] The basis functions are calculated as follows: , in, Indicates length is One-dimensional signal The basis functions of the i-th frequency component, l is the i-th frequency component. Each time step .
[0032] The specific method for calculating the multiple frequency components of each set of sub-features is as follows: , in, For frequency components, DCT is the Discrete Cosine Transform calculation. Let l be the feature of the i-th group, and l be the feature of the i-th group. Each time step The length of the sub-feature. For the first Sub-features in the first Feature values at each time step Indicates sub-features The corresponding specified frequency index, For length is of The basis functions.
[0033] This design introduces more frequency-dimensional information into attention processing, which helps to preserve local details and edge variations, further improving the expressiveness and prediction accuracy of time series modeling.
[0034] By concatenating the multiple frequency components of all sub-features, the overall compressed vector can be obtained. : , in This represents the final channel compression vector. Activation processing is applied to the final channel compression vector to obtain the frequency attention weights: .
[0035] This application extends and adapts channel attention processing in the image domain to a multi-frequency modeling step suitable for time series prediction. It improves upon the traditional global average pooling method by replacing it with a frequency domain processing method more suitable for time series analysis. For the time series data characteristics of transformers, it introduces temporal attention, which can dynamically weight features in the time dimension, further enhancing the ability to capture key temporal information.
[0036] S4. Based on the conditional features, the noisy time series is denoised to obtain a coarse sequence.
[0037] Guided by conditional features, noise is gradually removed to obtain a coarse sequence. For example, conditional features can guide the time point when the next high-frequency temperature signal appears, thereby obtaining the frequency curve of the temperature signal change and predicting the trend and period.
[0038] Specifically, starting with standard Gaussian noise of the noise level and using conditional features as conditions, the noise at the current time step is predicted. After multiple iterations, the predicted noise is subtracted from the current noisy time series to obtain a rough sequence.
[0039] This step can also be seen as a reverse diffusion process. First, the noisy time series is converted into a high-dimensional vector representation, which is then injected into various parts of the denoising process, such as as part of the key-value pairs in the attention process, or directly added to or multiplied by the features.
[0040] The purpose of this is to be able to distinguish between different noise reduction stages and know whether the current noise level is high or low.
[0041] Based on the current noise level and conditional features, the noise contained in the noisy time series is predicted. The role of the conditional features here is to provide semantic guidance, ensuring that the predicted noise is directed towards generating a sequence that conforms to the conditional features. Without conditional features, only an average noise can be predicted, resulting in a vague or nonspecific output.
[0042] After obtaining the predicted noise, a sampler is used to compute the cleaner data from the previous step, which is to perform denoising. This process is repeated until the time step is 0, resulting in a coarse sequence.
[0043] It is through this method that this application achieves a powerful predictive capability to gradually construct a coarse sequence of future time for a transformer from noise, guided by conditional features.
[0044] S5. Concatenate the noisy time series with the coarse sequence to form the input sequence. Obtain the dot product of the key vector and the query vector of the input sequence. Based on the difference between the maximum value and its average value, select the optimal query vector for each key vector. Use the projection of the conditional features as the key vector and value vector. Based on the key vector, value vector and the selected optimal query vector, perform cross-attention processing to obtain a new feature representation.
[0045] The noisy time series is used as conditional information to provide historical context and is concatenated with the coarse sequence in the sequence length dimension. The entire time series is continuously transformed in the time dimension to form the input sequence for this step.
[0046] Next, using the input sequence as the source, the self-attention processing weights after masking are calculated in the same way as in S2. This step aims to efficiently integrate information between the known context and the new feature representation, and to preliminarily model the dependencies within the prediction sequence.
[0047] Next, cross-attention calculation is performed. Specifically, the output of the self-attention layer obtained earlier is used as the query vector; simultaneously, the conditional features obtained in S3 are projected through two independent linear layers to obtain matrices serving as key and value vectors, which are then subjected to cross-attention processing. This step proactively utilizes conditional features rich in spectral information to guide and refine their own feature representation. Finally, through cross-attention processing, a new feature representation that deeply integrates historical context and global spectral conditional information is obtained.
[0048] Preferably, a causal mask is applied to the input sequence to ensure that during self-attention computation, any position in the input sequence can only focus on itself and the positions before it, thus preventing information leakage and maintaining the autoregressive characteristics of the generation process.
[0049] The application of a causal mask is crucial to ensuring that this step can be used for autoregressive prediction, as it guarantees the causality of the generation process.
[0050] The query vector in cross-attention comes from concatenated features rather than the original coarse sequence, which reflects the layer-by-layer transmission and refinement of information.
[0051] S6. Based on the new feature representation, a trend sequence is obtained through multinomial regression, and a periodic sequence is obtained based on the amplitude, phase, and frequency values of multiple frequency components. The trend sequence and the periodic sequence are added together to obtain the predicted sequence of transformer operation data.
[0052] The trend sequence is the predicted trend of transformer operating data over future time. For example, the trend sequence might be a slowly rising curve, indicating a slight upward trend in the overall operating temperature of the transformer. The trend sequence is generated through a polynomial regression branch, which maps the new feature representation to a set of polynomial coefficients. Simultaneously, a slowly varying polynomial basis vector is defined. The trend sequence is obtained by a weighted sum of the coefficients and the basis vector, used to capture long-term, slowly changing, non-periodic patterns in the sequence.
[0053] The periodic sequence represents the changes in data within the transformer's future operating cycle. For example, a fluctuation curve with a 24-hour period predicts the temperature changes during the day and night. It is generated through a Fourier synthesis branch. This branch first performs a discrete Fourier transform on the new feature representation, selecting the top y frequency components with the largest amplitudes and recording their amplitude, phase, and frequency. Subsequently, the periodic sequence is reconstructed by performing an inverse Fourier transform on these principal components. This sequence aims to accurately capture the recurring, fixed-period fluctuation patterns within the sequence.
[0054] Furthermore, to optimize the accuracy of the predicted sequence, a residual term is added. The residual term is obtained by removing noise from the trend sequence and the periodic sequence after adding noise to the time series.
[0055] This is achieved through one or more fully connected branches. The branches directly map the new feature representations to an output with the same dimension as the predicted sequence, as a residual term. The residual term is also used to capture irregular fluctuations and noise that the model cannot explain, which are neither part of the long-term trend nor the fixed period.
[0056] The existence of residual terms serves as an optimization mechanism, capturing some small, non-periodic fluctuations in the transformer, ensuring that this application maintains high prediction accuracy even when faced with strong irregular fluctuations.
[0057] Finally, by adding the above three items together, a complete prediction sequence is obtained. This explicit decomposition output not only provides the final prediction sequence, but also gives this application strong rationality, and allows for a clear understanding of the respective contributions of trend, periodicity and random fluctuations in the prediction results in transformer power system applications.
[0058] A transformer operation data prediction system, used to implement a transformer operation data prediction method, includes: Given a sequence noise addition module, a noise level of a certain level is selected, and noise is added to a known time series of transformer operating data to obtain a noisy time series; The first feature extraction module obtains the dot product of the key vector and the query vector of the noisy time series. Based on the difference between the maximum value and the average value, it selects the optimal query vector for each key vector and extracts the features of all optimal query vectors to obtain the first feature. The conditional feature acquisition module divides the first feature into several groups of sub-features along the channel dimension, acquires multiple frequency components of each group of sub-features, concatenates the multiple frequency components of all sub-features, and obtains frequency attention weights through activation processing. The frequency attention weights are then used to modulate the first feature to obtain the conditional feature. The denoising module, based on conditional features, denoises the noisy time series to obtain a coarse sequence; The new feature representation acquisition module concatenates the noisy time series with the coarse series to form the input sequence. It obtains the dot product of the key vector and the query vector of the input sequence. Based on the difference between the maximum value and the average value, it selects the optimal query vector for each key vector. The projection of the conditional features is used as the key vector and value vector. Based on the key vector, value vector, and the selected optimal query vector, cross-attention processing is performed to obtain the new feature representation. The prediction sequence acquisition module obtains the trend sequence through multinomial regression based on the new feature representation. It obtains the periodic sequence based on the amplitude, phase, and frequency values of multiple frequency components. The trend sequence and the periodic sequence are added together to obtain the prediction sequence of the transformer operation data.
[0059] Periodic sequences are obtained by Fourier synthesis of the amplitude, phase, and frequency values of multiple frequency components.
[0060] This invention can dynamically analyze and focus on the spectral components of different frequencies in transformer data. This differentiated weighting of the full spectrum information can accurately capture and reconstruct the multiple periodic patterns contained in the data. The generated sequence exhibits a high degree of realism in periodicity, effectively solving the problem of single or vague periodic capture in existing technologies.
[0061] This invention uses standard Gaussian noise as a starting point for stepwise denoising, separating the true operating rules from noisy observation data. This provides natural resistance to common noise and outliers in the data. The final generated sequence can smooth out noise interference, restore and retain key details, and improve the stability of prediction.
[0062] This invention selects the optimal query vector, reducing computational complexity and enabling more efficient processing of long-sequence data generated by power transformers.
Claims
1. A transformer operating data prediction method characterized by, Comprise: S1, select a certain noise level noise, the known time series of transformer operation data is added to the noise after, get the time series after adding noise; S2, obtain the dot product of the key vector and the query vector of the time series after adding noise, select the optimal query vector for each key vector based on the difference between the maximum value and the average value, extract the features of all optimal query vectors to obtain the first feature; S3, the first feature is divided into several groups of sub-features along the channel dimension, a plurality of frequency components of each group of sub-features are obtained, the plurality of frequency components of all sub-features are spliced, and the frequency attention weight is obtained through activation processing, the first feature is modulated by using the frequency attention weight to obtain the conditional feature; S4, based on the conditional feature, the time series after adding noise is denoised to obtain a rough sequence; S5, the time series after adding noise and the rough sequence are spliced to form an input sequence, the dot product of the key vector and the query vector of the input sequence is obtained, the optimal query vector is selected for each key vector based on the difference between the maximum value and the average value, the projection of the conditional feature is taken as the key vector and the value vector, and the cross attention processing is performed based on the key vector, the value vector and the selected optimal query vector to obtain a new feature representation; S6, based on the new feature representation, a trend sequence is obtained through polynomial regression, a periodic sequence is obtained based on the amplitude, phase and frequency value of a plurality of frequency components, and the trend sequence and the periodic sequence are added to obtain a prediction sequence of the transformer operation data.
2. The transformer operating data prediction method of claim 1, wherein, In S2, the dot product of the key vector and the query vector of the time series after adding noise is obtained, the optimal query vector is selected for each key vector based on the difference between the maximum value and the average value, and the specific calculation method is: ， wherein, is a metric, is the i-th column of the query vector, row, is the j-th column of the key vector, is the i-th column of the key vector, denotes a dimension, is the number of key vectors; selecting the query vector with the largest value as the optimal query vector.
3. The transformer operating data prediction method of claim 1, wherein In S3, the plurality of frequency components of each group of sub-features are calculated, and the specific method is: ， wherein, is a frequency component, DCT is a discrete cosine transform calculation, is the i-th group of sub-features, l is the j-th sub-feature of the i-th group, is the k-th time step, is the length of the sub-features, is the i-th group of sub-features, is the j-th sub-feature of the i-th group, is the k-th time step, denotes the specified frequency index corresponding to the sub-feature is the k-th time step, is the basis function of length is the k-th time step, is the k-th time step.
4. The transformer operating data prediction method of claim 3, wherein, The calculation method of the basis function is as follows: ， wherein denotes a basis function of the one-dimensional signal of length at the th frequency component, l at the th time step, .
5. The transformer operating data prediction method of claim 1, wherein, In S4, based on the conditional feature, the time series after adding noise is denoised to obtain a rough sequence, which is specifically: Starting from the standard Gaussian distribution noise of the noise level, predicting the noise of the current time step under the condition of the conditional feature, and obtaining the rough sequence by subtracting the predicted noise from the current time series after adding noise through multiple iterations.
6. The transformer operational data prediction method of claim 5, wherein, The multiple iterations are reverse diffusion.
7. The transformer operating data prediction method of claim 1, wherein, In S6, the noise is removed from the time series after adding noise, the trend sequence and the periodic sequence to obtain a residual term, and the residual term is added to the trend sequence and the periodic sequence to obtain a prediction sequence.
8. The transformer operating data prediction method of claim 1, wherein, In S6, Fourier synthesis is used to realize the periodic sequence based on the amplitude, phase and frequency value of a plurality of frequency components.
9. A transformer operating data prediction system for implementing a transformer operating data prediction method according to any one of claims 1 to 8, characterized by Comprise: A known sequence noise adding module selects a certain noise level noise, and adds noise to the known time series of transformer operation data to obtain a time series after adding noise; A first feature extraction module obtains the dot product of the key vector and the query vector of the time series after adding noise, selects the optimal query vector for each key vector based on the difference between the maximum value and the average value, and extracts the features of all optimal query vectors to obtain the first feature; The condition feature acquisition module divides the first feature into a plurality of groups of sub-features along the channel dimension, acquires a plurality of frequency components of each group of sub-features, splices the plurality of frequency components of all the sub-features, and obtains frequency attention weights through activation processing. The frequency attention weights are used to modulate the first feature to obtain a condition feature. The denoising module denoises the noisy time sequence based on the condition feature to obtain a rough sequence. The new feature representation acquisition module splices the noisy time sequence and the rough sequence to form an input sequence, obtains a key vector and a query vector dot product of the input sequence, selects an optimal query vector for each key vector based on a difference between a maximum value and an average value, takes a projection of the condition feature as the key vector and a value vector, and performs cross-attention processing based on the key vector, the value vector, and the selected optimal query vector to obtain a new feature representation. The prediction sequence acquisition module obtains a trend sequence based on the new feature representation through polynomial regression, obtains a periodic sequence based on amplitude, phase, and frequency values of the plurality of frequency components, and adds the trend sequence and the periodic sequence to obtain a prediction sequence of the transformer operation data.
10. The transformer operational data prediction system of claim 9, wherein, The periodic sequence is obtained by Fourier synthesis of the amplitude, phase, and frequency values of the plurality of frequency components.