Oil-immersed transformer hotspot temperature prediction method and system based on transformer model

Abstract

The application provides an oil-immersed transformer hotspot temperature prediction method and system based on a Transformer model, belongs to the technical field of transformer hotspot temperature prediction, and comprises the following steps: obtaining key operation parameter data of an oil-immersed transformer and performing data preprocessing; and inputting the key operation parameter data of the oil-immersed transformer after data preprocessing into a transformer hotspot temperature prediction model for prediction. Based on a mask reconstruction self-supervised pre-training mechanism, time sequence characteristics are learned by using unlabeled data, and the feature extraction capability and generalization performance of the model are improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of transformer hotspot temperature prediction, and particularly relates to an oil-immersed transformer hotspot temperature prediction method and system based on a Transformer model. BACKGROUND

[0002] The statements in this section merely provide background information related to the application and do not necessarily constitute prior art.

[0003] Under long-term high-load operation and complex working conditions, the internal insulation system of a transformer is prone to gradual degradation under the action of thermal stress. The winding hotspot temperature, as a key indicator reflecting the state of the winding thermal field, is directly related to the evaluation of the insulation life of the transformer, the judgment of the overload capacity, and the development of operation and maintenance strategies. Therefore, accurate prediction of the hotspot temperature is of great significance for ensuring the safe and stable operation of the power grid and improving the level of equipment health management.

[0004] With the rapid development of artificial intelligence technology, deep learning has gradually become an important tool in the field of time series prediction due to its strong feature extraction and nonlinear modeling capabilities, and has been widely applied in the state assessment and operation parameter prediction of power equipment.

[0005] In the aspect of hotspot temperature prediction, traditional neural network models such as Back Propagation Neural Network (BP) and Long Short-Term Memory Network (LSTM) model the historical operation data to depict the complex nonlinear relationship between input features and hotspot temperature, thereby achieving hotspot temperature prediction. To further weaken the noise interference in the original signal and improve the adaptability of the model to temperature dynamic changes, some studies introduce signal processing methods such as empirical mode decomposition combined with deep learning models to form a composite prediction method, which shows better engineering applicability in terms of prediction accuracy and robustness. Overall, the deep learning-based hotspot temperature prediction method has to some extent broken through the limitations of traditional thermodynamic models. At the same time, the Transformer model provides a new solution to the long sequence modeling problem in hotspot temperature prediction with its unique self-attention mechanism.

[0006] Compared with BP, LSTM and other models, the Transformer can directly mine the global dependence relationship of each time in the sequence, effectively alleviate the gradient dissipation problem of recurrent neural network in long sequence modeling, and the highly parallel architecture can also significantly improve the training efficiency, which is more suitable for the modeling needs of large-scale power time series data. At present, some researches have applied Transformer to power load forecasting, electric vehicle regulation capacity prediction and other power fields, and proposed XGBoost-Transformer, Bayesian-Transformer and other improved methods, which lay a technical foundation for its expansion to hotspot temperature prediction.

[0007] In addition, as a new training method in the field of deep learning, self-supervised learning (SSL) has been widely used in image processing, natural language processing and time series analysis. Its core advantage is that it can make full use of a large amount of unlabeled data for training, and guide the model to learn the internal laws and dynamic characteristics of the data through the design of mask reconstruction and other self-supervised tasks, which is especially suitable for industrial scenarios in power systems where data labeling is difficult and costly, providing a new idea for improving the generalization ability of hotspot temperature prediction models.

[0008] Although the existing winding hotspot temperature prediction method has made certain progress, combined with the actual application needs and technical characteristics, there are still problems such as insufficient long sequence dependence modeling, insensitive response to local features, and excessive dependence on limited labeled data. The traditional deep learning models such as BP, LSTM are limited by network structure, and it is difficult to effectively capture the global dependence relationship in long sequence data, and in the multi-step prediction scene, error accumulation is easy to occur, which leads to the rapid decline of prediction accuracy with the increase of step length, making it difficult to meet the actual needs of transformer long-term thermal state evaluation. In contrast, the Transformer model can effectively alleviate the gradient dissipation problem of recurrent neural network in long sequence modeling, but the existing Transformer model mostly uses a single global attention mechanism in hotspot temperature prediction, ignoring the frequent local fluctuations in hotspot temperature data, resulting in insufficient ability to capture local dynamic features, and it is difficult to adapt to the actual engineering needs of hotspot temperature prediction. And most of the existing models only use supervised learning, which relies too much on limited labeled data, and fails to fully utilize the running data, combined with the influence of noise and working condition disturbance on actual monitoring data, further restricting the generalization ability and prediction accuracy of the model. SUMMARY

[0009] In order to overcome the shortcomings of the prior art, the present application provides a winding hotspot temperature prediction method and system based on a Transformer model for oil-immersed transformer, which is used to realize short-term multi-step prediction of transformer hotspot temperature.

[0010] To achieve the above object, one or more embodiments of the present application provide the following technical solutions:

[0011] In a first aspect, a method for predicting hot-spot temperature of an oil-immersed transformer based on a Transformer model is disclosed, comprising:

[0012] Obtaining key operating parameter data of the oil-immersed transformer and performing data preprocessing;

[0013] Inputting the data-preprocessed key operating parameter data of the oil-immersed transformer into a transformer hot-spot temperature prediction model for prediction to obtain a hot-spot temperature prediction result in a target period;

[0014] The transformer hot-spot temperature prediction model adopts a Transformer architecture;

[0015] The Transformer architecture comprises an encoder and a decoder;

[0016] The encoder is formed by stacking a plurality of independent encoder layers with the same structure; each encoder layer comprises a locally enhanced multi-head attention sublayer and a feedforward network sublayer, and is connected in series through residual connection and layer normalization;

[0017] The decoder is formed by stacking a plurality of independent decoder layers with the same structure; each decoder layer comprises a masked local multi-head attention sublayer, an encoding-decoding cross-attention sublayer and a feedforward network sublayer, and is connected in series through residual connection and layer normalization.

[0018] As a further technical solution, a local attention window is introduced in the locally enhanced multi-head attention sublayer of the encoder and the masked local multi-head attention sublayer of the decoder;

[0019] The global attention mode is retained in the encoding-decoding cross-attention sublayer of the decoder to fully utilize the complete time sequence context information output by the encoder.

[0020] As a further technical solution, the encoder is used for deep representation learning of input time sequence features to capture global dependencies and local correlations between key operating parameters;

[0021] The masked attention in the decoder is used to mask future time step information, and the cross-attention is used to fuse the encoder output features to realize step-by-step decoding of the target prediction sequence.

[0022] As a further technical solution, the Transformer architecture further comprises an input embedding module and a prediction output module:

[0023] The input embedding module comprises a first linear embedding layer and a second linear embedding layer, which respectively perform dimension mapping on each time step input feature and superimpose position encoding to represent time sequence position information, thereby constructing an input sequence of an encoder and a decoder respectively;

[0024] The prediction output module is arranged after the decoder, maps the hidden features output by the decoder into the hotspot temperature prediction value of the target time step through a fully connected layer, and realizes the prediction output of the hotspot temperature of the transformer at multiple future time points.

[0025] As a further technical solution, when pre-training the transformer hotspot temperature prediction model, a mask reconstruction self-supervised pre-training task is performed on unannotated operation data, so that the transformer hotspot temperature prediction model learns the time sequence dependency between the key operation parameters of the oil-immersed transformer.

[0026] The model parameters after pre-training are used as initial weights, and supervised fine-tuning is performed on sample data containing real hotspot temperature labels, so that the model can realize short-term multi-step prediction of the hotspot temperature.

[0027] As a further technical solution, the key operation parameter data of the oil-immersed transformer includes load current, ambient temperature, high-voltage side active power, top oil temperature and hotspot temperature.

[0028] As a further technical solution, the key operation parameter data of the oil-immersed transformer is preprocessed, including:

[0029] The key operation parameter data of the oil-immersed transformer is filled with missing values and processed for abnormal values;

[0030] The processed data is selected for features to determine the optimal input variable set of the transformer hotspot temperature prediction model;

[0031] The data in the optimal input variable set is normalized and processed to obtain preprocessed data.

[0032] As a further technical solution, the processed data is selected for features, specifically including:

[0033] On the basis of the feature variables of load current, ambient temperature, high-voltage side active power and top oil temperature, the historical hotspot temperatures of the previous two time points of the current time point of the transformer hotspot temperature prediction model are introduced as supplementary input features, and candidate variables are screened based on the combination of electric-thermal coupling mechanism and correlation analysis to finally determine the optimal input variable set of the transformer hotspot temperature prediction model.

[0034] In a second aspect, a hot spot temperature prediction system for oil-immersed transformers based on a Transformer model is disclosed, comprising:

[0035] a data acquisition module configured to acquire key operating parameter data of the oil-immersed transformer and perform data preprocessing;

[0036] a prediction module configured to input the key operating parameter data of the oil-immersed transformer after data preprocessing into a transformer hot spot temperature prediction model for prediction to obtain a hot spot temperature prediction result in a target period;

[0037] wherein the transformer hot spot temperature prediction model adopts a Transformer architecture;

[0038] the Transformer architecture comprises an encoder and a decoder;

[0039] the encoder is formed by stacking a plurality of independent encoder layers with the same structure; each encoder layer comprises a locally enhanced multi-head attention sublayer and a feedforward network sublayer, and is connected in series through a residual connection and layer normalization;

[0040] the decoder is formed by stacking a plurality of independent decoder layers with the same structure; each decoder layer comprises a masked local multi-head attention sublayer, an encoding-decoding cross-attention sublayer and a feedforward network sublayer, and is connected in series through a residual connection and layer normalization.

[0041] The above one or more technical solutions have the following beneficial effects:

[0042] The technical solution of the present application is based on a transformer hot spot temperature prediction model, which can directly mine the nonlinear temperature evolution law between key variables from historical operating data without relying on the setting of traditional thermal model parameters. By introducing a local-global collaborative multi-head attention mechanism, the model can not only depict the long-term dependence of hot spot temperature data, but also effectively perceive the local temperature fluctuations caused by sudden working conditions, thereby improving the precision of feature modeling and the stability of prediction.

[0043] The technical solution of the present application is based on a masked reconstruction self-supervised pre-training strategy, which can complete feature representation learning on a large amount of unlabeled operating data, alleviate the model training restrictions caused by insufficient labeled samples, and improve the generalization ability of the model. Through the combination of supervised fine-tuning and non-autoregressive decoding, the model can realize the rapid output of hot spot temperature at multiple time steps in the future, avoid the problem of error accumulation in the autoregressive prediction framework, and help to improve the efficiency and accuracy of short-term multi-step prediction.

[0044] The advantages of the additional aspects of the present application will be partially given in the following description, partially become obvious from the following description, or be understood through the practice of the present application. Attached Figure Description

[0045] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0046] Figure 1 This is an improved Transformer architecture according to an embodiment of the present invention;

[0047] Figure 2 This is a schematic diagram of the multi-head attention mechanism in an embodiment of the present invention;

[0048] Figure 3 This is a schematic diagram of mask self-supervised pre-training;

[0049] Figure 4 A schematic diagram of the correlation coefficient matrix for various characteristic parameters;

[0050] Figure 5 For different d_model and mask_ratio A schematic diagram of the prediction error below;

[0051] Figure 6 For different seq_len A schematic diagram of the prediction error below;

[0052] Figure 7 For different pred_len The following is a schematic diagram of the model's prediction loss;

[0053] Figure 8 This is a schematic diagram of the hotspot temperature prediction results. Detailed Implementation

[0054] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0055] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0056] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0057] Example 1

[0058] This embodiment discloses a method for predicting hot spot temperature of oil-immersed transformers based on the Transformer model, including:

[0059] Step one: Obtain the key operating parameter data of the oil-immersed transformer and perform data preprocessing;

[0060] Step two: Based on the preprocessed key operating parameter data, an input feature set of the transformer hot spot temperature prediction model is constructed;

[0061] Step three: Construct a mask self-supervised pre-training task to pre-train the model on unlabeled operation data;

[0062] Step four: Use the model parameters obtained in the pre-training stage as the initial parameters for fine-tuning, and perform supervised fine-tuning on the dataset containing real hot spot temperature labels;

[0063] Step five: Use a non-autoregressive decoding method to output the hot spot temperature prediction values for multiple time steps in the future.

[0064] In this embodiment, in the above step one, the key operating parameter data of the oil-immersed transformer obtained includes load current, ambient temperature, high-voltage side active power, top layer oil temperature, and hot spot temperature. Among them, the key operating parameter data of the oil-immersed transformer is synchronously collected by the transformer field online monitoring system and the sensor, and the sampling period is 5 minutes, ensuring the correspondence of each parameter at the same time point. The collected data is the historical monitoring data of the actual operation of the transformer, which can truly reflect its thermal-electric operating characteristics.

[0065] In order to ensure the stability of model training and the reliability of prediction results, the data needs to be preprocessed according to the data characteristics to reduce the redundant information in the data model. Data processing includes data cleaning, feature selection and data normalization.

[0066] 1-1) Data cleaning: Due to occasional communication interruption or sensor failure of the field collection system, there are missing values or obvious outliers at some time points. In this embodiment, the linear interpolation method is used to fill in the single-point missing of the key operating parameter data of the oil-immersed transformer, and the time period with continuous missing or continuous exceeding threshold is deleted. The outliers are detected by the 3σ principle and judged combined with engineering practical experience. Values exceeding the range are considered abnormal and are deleted or replaced.

[0067] 1-2) Feature selection: In the hot spot temperature prediction task, since different input variables have different contributions to the performance of the model, there may be strong redundancy between some variables, which may even introduce noise and affect the prediction performance of the model. Therefore, it is necessary to screen and optimize the feature set.

[0068] Based on the electro-thermal coupling mechanism revealed by the IEC thermal model, the transformer hot spot temperature is mainly determined by both internal heating and external heat dissipation conditions. According to this mechanism and relevant literature, load current, top oil temperature, and ambient temperature are identified as the main influencing factors on the transformer hot spot temperature. Furthermore, considering the historical dependence of the thermal dynamics process, the model's prediction... t When determining the hotspot temperature at any given moment, introduce... t Hotspot temperature at the two sampling times before time step T hst ( t -1) and T hst ( t -2) is used as model input.

[0069] To further explore the potential value of other monitoring parameters, active power in the dataset was included as a candidate feature.

[0070] Meanwhile, to avoid feature redundancy, the Pearson Correlation Coefficient (PCC) method is used to quantitatively analyze the correlation between the feature variables. The Pearson correlation coefficients between each feature variable are calculated. r xy Its mathematical definition is as follows:

[0071] (1)

[0072] in, x i and y i They represent the first i The relationship between a certain feature variable and the hotspot temperature in a sample , Its mean, n This represents the number of samples.

[0073] Through calculation r xy This method can initially assess the correlation between various feature variables and hotspot temperatures, eliminating inputs with insignificant correlation to the target variable. It also quantifies the degree of correlation between different feature variables, avoiding information redundancy.

[0074] Feature selection effectively compresses the model input dimension, improves data representation capabilities, reduces noise interference, and lays a solid foundation for subsequent model training and prediction.

[0075] 1-3) Data Normalization: The data set after feature selection is divided into training set, validation set and test set in chronological order. The training set is used to optimize the structure and internal parameters of the prediction model, the validation set is used to monitor whether overfitting occurs and adjust the hyperparameters, and the test set is used to evaluate the final performance of the prediction model. In order to avoid the problem of gradient imbalance and slow convergence during model training caused by the difference in physical dimension of each input variable, only based on the training set data to calculate the normalization parameter, all input data is Min-Max normalized, and all input data is mapped to the interval [0, 1], as shown in equation (2).

[0076] (2)

[0077] wherein, X input is the original input data, X' input is the normalized data, X min and X max The minimum and maximum values are calculated from the training set. After obtaining the normalization parameters from the training set, they are also applied to the validation set and test set to ensure the consistency of data processing and avoid data leakage.

[0078] In an embodiment, the purpose of step two is to convert the key operating parameter data after feature selection and normalization processing in step one into a feature set suitable for input of the Transformer model, completing the data conversion from data preprocessing to model input. This step ensures that the subsequent model pre-training and fine-tuning can proceed smoothly, and specifically includes feature integration and input matrix construction.

[0079] 2-1) Feature Integration: The core features selected in step one are spliced in chronological order to form a unified time series feature set.

[0080] 2-2) Input matrix construction: The integrated features are generated into an input feature matrix with dimensions of L x N according to the requirements of the Transformer. X input wherein L is the time series length of the feature variable, and N is the feature dimension. The generated feature matrix is checked for completeness and dimension to ensure that there are no missing values and all features are normalized.

[0081] In an embodiment, the purpose of step three is to use a large amount of unlabeled historical operation data to build a masked self-supervised learning task, forcing the Transformer model to learn the coupling relationship between the time series features of the transformer hotspot temperature and the variables, and providing high-quality initial parameters for the subsequent fine-tuning stage.

[0082] In this embodiment, the transformer hotspot temperature prediction model adopts an improved Transformer architecture.

[0083] Hotspot temperature data is typical multivariate time series data, and its inherent time correlation and multidimensional feature structure are highly consistent with the core architecture of Transformer. Based on the reference to the original Transformer architecture, the sub technical solution of this embodiment combines the specific requirements of the hotspot temperature short-term prediction task to make structural adjustments to the model to better adapt to the task requirements.

[0084] 3-1) Basic architecture: Traditional Transformer excels in capturing long-term dependencies with its global self-attention mechanism. However, in the task of oil-immersed power transformer hotspot temperature prediction, time series not only presents long-term trends and periodicity, but also is frequently affected by factors such as power load mutations, cooling system switching, and other factors, showing significant local fluctuations and mutation characteristics. A single global attention mechanism may lack sufficient local sensitivity when dealing with dramatic mutations.

[0085] To solve the above problems, the sub technical solution of this embodiment improves the Transformer architecture in a modular way, and the overall structure is as shown in Figure 1 The core idea is to configure attention patterns differently in different functional modules to achieve the collaborative work of local perception and global modeling. Specifically, local attention windows are introduced in the encoder self-attention layer and the decoder self-attention layer to enhance the ability to capture mutation features at adjacent time points; and global attention is retained in the encoder-decoder cross-attention layer of the decoder to ensure complete use of the global context of the input sequence.

[0086] The improved Transformer still maintains the overall structure of the encoder-decoder. Among them, the encoder is stacked by L enc independent encoding layers, each of which contains a locally enhanced multi-head attention layer, a feedforward connection layer, and a residual connection and layer normalization. The decoder is stacked by L dec decoding layers, which is different from the encoding layer in that its attention mechanism is divided into two parts: one is the masked local multi-head attention layer, and the other is the global encoder-decoder cross-attention layer, to ensure that the prediction process only relies on historical information and avoids future data leakage.

[0087] 3-2) Attention mechanism and improvement: Multi-Head Attention (MHA) is the core computing mechanism of each attention sub-layer in the Transformer model. Its main function is to mine the mapping relationship between each time point of the transformer hotspot temperature data, improve the trend capturing ability of the model in long-term prediction, and enhance the interpretability of the model. Its basic structure is shown in Figure 2 .

[0088] Figure 2 In the middle, X input For the input feature matrix constructed in step two, the matrix size is L x N ; L represents the length of the time series; N is the feature dimension. First, the input matrix is mapped to a high-dimensional space through three independent linear fully connected layers, respectively, to obtain the query matrix Q , the key matrix K , and the value matrix V , the expression is:

[0089] (3)

[0090] In the formula, W q and b q , W k and b k , W v and b v are the weight and bias matrices of the three fully connected layers, respectively. Their initial values are generated by the standard random initialization method, and are automatically updated by the back propagation algorithm and gradient optimization during the model training process to learn the mapping relationship between the time steps in the input feature sequence. In the following, the weights and bias matrices of the linear fully connected layers are generated in the above manner and will not be described again.

[0091] After linear transformation, the obtained Q , K , V matrix dimensions are all L × D , where L represents the length of the time series; D represents the encoding dimension of the intermediate layer. Then, to realize multi-head parallel computing, the Q , K , V are divided into h sub-matrices in the column dimension, where hTo ensure the number of heads in a temporal multi-head attention mechanism, it is necessary to guarantee... D / h The integer is . Taking the query matrix Q as an example, it can be expressed as equation (4).

[0092] (4)

[0093] in, Q i The first part after matrix partitioning i There are submatrices, with dimensions of . L x D / h , recorded as L × d k Here, i For attention head index, i =1,2,…, h -1, h .

[0094] Transformer employs a multi-head parallel mechanism to divide the data into multiple parts. h Each submatrix performs attention calculations in each attention head, yielding the output of each attention head. In the traditional global attention mechanism, the... i The output of each attention head is:

[0095] (5)

[0096] in, Q i , K i , V i They represent Q , K , V After performing matrix partitioning along its column dimensions, the first i There are submatrices, where T represents the matrix transpose.

[0097] To enhance the model's sensitivity to local fluctuations, a configurable local attention window mechanism is introduced, and hyperparameters are used to... w Controls whether it is enabled with the local window radius. This is done after calculating the scaled dot product attention score. When setting the radius of the local attention window w If the value is greater than 0, then construct a local attention mask matrix. M Its element definition is as follows:

[0098] (6)

[0099] In the formula, This is a local attention mask matrix, specifically... Lrow L a real matrix of rows, denotes the element in the i th row and j th column of the matrix; i and j are time step indices, satisfying 1≤ i , j ≤ L , L where w denotes the length of the input sequence; i is a non-negative integer hyper-parameter; when j - w ≤ , =1, otherwise R =0, is a real set.

[0100] S The mask matrix and the attention score matrix i are obtained as follows: i the attention score of the th time point under the local window constraint:

[0101] (7)

[0102] where denotes the attention score of the i th time point under the local window constraint. This operation sets the attention score outside the window i - w , i + w to negative infinity, and after Softmax normalization, its weight tends to 0, so that the i th time point only pays attention to its adjacent 2 w +1 time points, achieving fine modeling of short-term fluctuations and sudden changes. If w =0, this mechanism degenerates into global attention, which is used to capture long-term dependencies and overall trends.

[0103] Finally, the outputs of all attention heads are concatenated along the feature dimension and mapped back to the original dimension through linear transformation to obtain the final output of multi-head attention:

[0104] (8)

[0105] where W O is the output projection matrix, head 1,…, head hThis represents the output of all attention heads. Concat(·) represents the concatenation operation of the outputs of each attention head according to the feature dimension. MHA represents the multi-head attention mechanism composed of the above attention heads in parallel.

[0106] By configuring different local attention window radii for different modules w This enables the synergy between local perception and global modeling, thereby better adapting to the multi-scale temporal dependence characteristics in hotspot temperature prediction.

[0107] 3-3) Overall Prediction Process: To achieve short-term, multi-step prediction of transformer hotspot temperatures, a Transformer model based on mask-based self-supervised pre-training was constructed, employing an integrated prediction framework of pre-training and fine-tuning. During the pre-training phase, a large amount of unlabeled running data was fully utilized for feature learning to improve the model's ability to extract general features. In the fine-tuning phase, the parameters obtained during pre-training were used as initial weights for supervised learning of hotspot temperature prediction in downstream tasks, ultimately achieving stable prediction of hotspot temperatures.

[0108] Training of the transformer hotspot temperature prediction model: pre-training stage based on self-supervised learning.

[0109] This paper introduces a mask-based self-supervised pre-training method to fine-grained short-term prediction of transformer hotspot temperatures. During the pre-training phase, the model focuses on mask reconstruction. By randomly masking a portion of the time steps in the input feature sequence according to a predetermined strategy, and then reconstructing this masked data using a Transformer network, the model is forced to delve deeper into the intrinsic patterns of temporal features. This phase not only improves the model's feature extraction and generalization capabilities but also provides better parameter initialization for subsequent prediction tasks. The specific training process for the mask reconstruction task is as follows: Figure 3 As shown.

[0110] In the specific implementation, the preprocessed input feature sequence X' input ∈R L×N Perform masking. Use a single-time-point-level random masking method, setting the mask ratio to [value missing]. H That is, in L Randomly selected from each time step H · L Masking is performed at specific positions. This involves setting all features of the input feature sequence to zero at that time step, marking the valid bits as invalid, and keeping the rest as their original values. After masking, the masked input is obtained. X masked ∈R L×N , L Indicates the length of the time series; NR is the real number set. By completely removing the original information of the masked positions, the model is forced to rely only on the temporal correlation and inter-feature coupling relationship of the non-masked area to infer the feature values of the masked area, thus completing the reconstruction task.

[0111] Masked data X masked The original feature dimension N is mapped to the encoding dimension D by a linear embedding layer, obtaining X emb ∈R L×D To further enhance the temporal relationship of time series data in the time dimension, position encoding based on the sine and cosine functions is introduced, as shown in equations (9) and (10):

[0112] (9)

[0113] (10)

[0114] wherein, p is the position of the time step in the sequence, p= 1, …, L ; i is the encoding dimension index, i =1, …, D .

[0115] Position encoding adds unique position information to each input token, enabling the model to explicitly determine the position of the data point in the time series sequence, thus effectively capturing the temporal dependence of the context. Further superimpose the position information on the masked data to obtain the final input of the encoder X enc ∈R L×D , as shown in equation (11):

[0116] (11)

[0117] wherein, X enc is the position encoded input, X emb is the encoding output, and PE is the position encoding function.

[0118] Position encoded input X enc is sent to the encoder. This module is composed of L encTransformer encoder layers are concatenated, each of which includes a multi-head self-attention layer and a feed-forward network layer, and is connected in series through a residual connection and layer normalization. In the multi-head self-attention layer, the model calculates the correlation weight between each position in the sequence based on the scaled dot-product attention, and dynamically aggregates the context information to realize the modeling of the temporal dependence. Subsequently, the feed-forward network utilizes two layers of linear mapping and the Relu activation function to perform nonlinear transformation on the attention output, thereby improving the feature expression capability of the model. The residual structure and layer normalization further alleviate the gradient vanishing problem in the deep network and promote the efficient transmission of features. After the processing of the layer encoder, the model finally outputs the encoded representation, which contains the context feature tensor of complete temporal dependence, denoted as L enc After the processing of the layer encoder, the model finally outputs the encoded representation, which contains the context feature tensor of complete temporal dependence, denoted as Y enc ∈R L×D .

[0119] (12)

[0120] wherein denotes the set of learnable parameters of the encoder in the pre-training stage. At this time Y enc The temporal and feature information of all non-masked areas has been fused, and can be directly used for the reconstruction of the masked area.

[0121] In the pre-training stage, the encoder output is mapped back to the original feature dimension N by the reconstruction head, realizing the reconstruction of the masked time step data, as shown in equation (13).

[0122] (13)

[0123] wherein is the reconstructed data matrix, ∈R L×N , W rec is the weight matrix of the reconstruction head, b rec is the bias vector of the reconstruction head.

[0124] The reconstruction loss only calculates the error between the predicted result and the true value at the masked time step. Mean square error (MSE) is used as the objective function of pre-training, as shown in equation (14):

[0125] (14)

[0126] wherein X ( i ) represents the true value at the ith masked time step; denotes the reconstruction value of the model at the i-th masked time step; N mask denotes the total number of masked time steps involved in the reconstruction; i is the index of the masked time step; L loss is the reconstruction loss function at the masked position.

[0127] The model parameters are updated by minimizing the loss function and other parameters. After the pre-training is completed, the encoder and embedding layer parameters are retained as the initialization parameters of the fine-tuning stage.

[0128] In an embodiment, the purpose of step four is to migrate the model parameters obtained in step three to the downstream prediction task, and to achieve high-precision multi-step prediction under limited labeled data.

[0129] Based on the initialization parameters obtained by pre-training, supervised fine-tuning is performed to achieve multi-step prediction of hotspot temperature. In order to suppress error accumulation and improve parallel computing efficiency, a non-autoregressive decoding strategy is adopted, and a masked local self-attention is introduced in the decoder to avoid future information leakage.

[0130] To adapt to the downstream prediction task, the model structure in the pre-training stage is adjusted. Among them, the reconstruction head is only used for masked reconstruction and is irrelevant to the prediction of hotspot temperature, so it is removed in the fine-tuning stage; while the encoder and embedding layer parameters are migrated to the fine-tuning stage. Consistent with the pre-training stage, the encoder models the input historical feature sequence through the local enhanced multi-head attention mechanism, and outputs the context feature tensor containing complete temporal dependencies Y enc .

[0131] In order to predict the hotspot temperature value of the future T time steps, the decoder input is initialized as a zero tensor with dimension R T×1 , that is, Y (0) =0∈R T×1 . Among them, T is the prediction step length, and 1 indicates that only the univariate hotspot temperature is predicted.

[0132] Y (0) First, the feature dimension is converted through the linear embedding layer, and embedded into the same feature space as the encoder, as shown in equation (15):

[0133] (15)

[0134] Among them, W dec dimension is 1× D , which is the learnable projection matrix of the decoder embedding layer, bdec is a bias vector added to the output of each time step by broadcasting mechanism.

[0135] Add position encoding for each predicted time step to introduce time position information, get decoder input sequence Y dec-in .

[0136] The input sequence after position encoding is sent to the decoder module, which is composed of L dec a cascade of Transformer decoder layers, each of which includes a masked local self-attention sublayer and an encode-decode cross-attention sublayer. Among them, the masked local self-attention sublayer restricts the decoder to only focus on the w time steps in front of it through a local window mask, both enhancing the modeling ability of short-term fluctuations in hot spot temperature and ensuring that only historical and current visible feature information is used to avoid future information leakage; the encode-decode cross-attention sublayer takes the output features of the decoder mask self-attention as the query matrix Q , the Y enc key matrix K and the value matrix V , and calculates the association weight between historical features and future hot spot temperature through scaled dot-product attention, establishing the mapping relationship between historical features and future evolution, thereby making full use of long-term sequential dependence. Through step four, the model realizes the multi-step prediction ability of hot spot temperature under limited labeled data, while suppressing error accumulation and improving prediction stability.

[0137] In an embodiment, the purpose of step five is to use a non-autoregressive decoding strategy to achieve efficient and stable prediction of future multi-step hot spot temperature.

[0138] The decoder uses a non-autoregressive generation strategy to predict the hot spot temperature of the future T steps in parallel, avoiding error accumulation in step-by-step prediction. After L dec layer decoder processing, the decoder output Y dec ∈R T×D .

[0139] Finally, Y dec is reduced to a single output dimension by a fully connected layer to obtain the hot spot temperature prediction sequence of the future T time steps Y pre ∈R T×1 .

[0140] (16)

[0141] wherein,W o 、 b o are the weight matrix and bias vector of the output layer, Y dec is the decoder output.

[0142] Fine-tuning stage with hotspot temperature prediction sequence Y pre The mean square error with the real hotspot temperature sequence is the loss function, and the whole model parameters are iteratively optimized by backpropagation. Among them, the encoder parameters are further updated based on the pre-trained weights, and the decoder and output layer parameters are trained from random initialization. Through the pre-training-fine-tuning integrated strategy, the model can realize high-precision multi-step prediction of hotspot temperature under limited labeled data.

[0143] Engineering verification example:

[0144] The data set used is the actual operation data of a 220 kV oil-immersed transformer in a substation of Jibei Power Co., Ltd. of State Grid from June to August 2023, with a sampling period of 5 minutes, a total of 26496 groups of data. The data set covers actual historical monitoring data such as load current I , ambient temperature T amb , high-voltage side active power P L , top oil temperature T tot , hotspot temperature T hst , etc. The data set is divided into training set, validation set and test set in the ratio of 8:1:1. In order to verify the superiority of the proposed MSSP-Transformer model in the task of hotspot temperature prediction, the Long Short-Term Memory (LSTM), the Transformer model without pre-training by self-supervised learning, and the Convolutional Neural Network-Long Short Term Memory (CNN-LSTM) are selected as comparison models. The construction and implementation of each prediction model are based on the PyTorch framework in the Python platform. Table I shows the main parameters of the comparison models, which are set according to the reference literature and optimized for the data set used to ensure the effectiveness of the comparison experiment.

[0145] Table I Main parameter settings of each comparison model

[0146]

[0147] Root Mean Square Error (RMSE), Mean Absolute Error (MAE) and R-square (R 2 ) are selected as the evaluation indexes of the model, and the specific calculation formulas of the evaluation indexes are shown in equations (17), (18) and (19).

[0148] (17)

[0149] (18)

[0150] (19)

[0151] In the formula, is the predicted hotspot temperature; is the real hotspot temperature; N is the number of hotspot temperature samples.

[0152] Feature selection: According to the 5 kinds of data features contained in the historical monitoring data, the correlation analysis of the hotspot temperature T hst and the top oil temperature T tot , ambient temperature T amb , load current I , active power on the high voltage side P L , the correlation coefficients between each feature parameter are calculated. The correlation coefficient r xy The value range of the correlation coefficient is [-1, 1], and the closer the absolute value is to 1, the stronger the correlation between the two variables. The correlation matrix between the features is shown in Figure 4

[0153] As can be seen from Figure 4 , the top oil temperature, ambient temperature and hotspot temperature all show strong linear correlation, which is consistent with the physical mechanism that the two respectively dominate the internal heat transfer and external heat dissipation boundary conditions in the thermal dynamic process of the transformer, so they are determined as the core input variables of the model. The correlation coefficient between the load current and the hotspot temperature is 0.57, which is not as significant as the first two, but as a direct driving factor of winding heating, its physical meaning is clear, so it is retained as an input feature.

[0154] ​In contrast, the correlation coefficient between active power and hotspot temperature is only 0.43, the lowest among all features, indicating its limited direct contribution to the prediction target. Meanwhile, its correlation coefficient with load current is as high as 0.91, suggesting significant redundancy. Combining the correlation analysis results with physical mechanism considerations, the top-level oil temperature, ambient temperature, and load current, together with the historical values of hotspot temperature, are finally selected as the input variables of the prediction model.

[0155] Design comparative experiments to verify the elimination of input features P L whether it will adversely affect the model prediction performance. Under the premise of keeping the model structure, hyperparameters, and training conditions completely consistent, two input schemes are constructed respectively: input set A is composed of T tot , T amb , I and T hst historical values, without P L ; input set B adds P L to the above features to form the complete feature set. Both sets of data are preprocessed according to the method described in Section 3.1, and multiple independent training and testing are performed under the prediction step of 3. The RMSE, MAE, and are used as model performance evaluation indicators, and the results are shown in Table II.

[0156] Table II P L Whether as input, the performance comparison of the model

[0157]

[0158] From Table II, the differences between the two schemes in various performance indicators are not significant, and the model without P L still maintains better performance in prediction accuracy and stability. This indicates that the introduction of P L does not substantially improve the model performance, further verifying the rationality of eliminating this variable in the feature selection stage.

[0159] Model key parameter settings: In the actual debugging process, it is found that the model's prediction performance is sensitive to the setting of the input sequence length seq_len (which can also be understood as the historical window size), the mask ratio mask_ratio , and the model hidden dimension d_model . Among them, seq_len is closely related to the first two parameters, so d_model is analyzed first.mask_ ratio The comprehensive influence on the model performance is determined according to the analysis result. seq_len .

[0160] Model hidden dimension d_model determines the feature representation ability of the Transformer, and a too small dimension is easy to cause the model to fail to fully learn the data variation rule on the training set, and a too large dimension will increase the calculation overhead and bring the risk of overfitting in the case of limited sample size. Masking ratio mask_ratio is a key control parameter in the self-supervised pre-training stage, and a too low ratio will make the reconstruction task too simple and difficult to force the model to learn the deep dependence between features, while a too high ratio will result in too little effective information and too difficult reconstruction. Based on the above analysis, different hidden layer dimensions are set in the interval of 64 to 256 with an interval of 32, and different pre-training masking ratios are set in the interval of 0.3 to 0.6 with an interval of 0.05, and the d_ model hot spot temperature prediction experiment with a prediction step of 3 is set. Through multiple experiments and comparison of the average prediction error (MAE), the prediction performance difference of the model under different parameter values is quantitatively evaluated, so as to determine the optimal parameter configuration of the model. The relationship between the prediction error and the hidden dimension and the masking ratio is shown in Figure 5 .

[0161] As can be seen from Figure 5 , with the increase of d_model , the prediction error shows a trend of first decreasing and then increasing, which is consistent with the analysis result in the previous article. In comparison, mask_ratio has a more complex effect on the prediction error and is affected by d_model . When d_model is small, mask_ratio change has no significant effect on the prediction error; but as d_model increases, the optimal mask_ratio of the model gradually increases. Comprehensive analysis shows that when d_model is 128, mask_ratio is 0.45, the average prediction error of the model is the lowest, thereby determining the optimal values of the two important parameters d_model and mask_ratio .

[0162] Under the premise that the two remain at the optimal value, the value of seq_len is changed to analyze its effect on the prediction effect of the model. seq_lendetermines the range of historical information available to the model when making predictions. When the value is too small, the model is difficult to capture the long-term dependence characteristics of hotspot temperature affected by load and environmental changes, leading to increased prediction error; while the value is too large, it may introduce redundant noise, increase the computational complexity and reduce the generalization ability of the model. Considering the data sampling interval is 5 min, set different values (corresponding to 3~12h historical window) in the interval of 36 to 144, carry out hotspot temperature prediction experiments with a prediction step of 3, and calculate the average prediction error MAE of each value under multiple experiments, as shown in seq_len . Figure 6 .

[0163] As can be seen from Figure 6 , with the increase from 36 to 72, the average prediction error of the model decreases significantly, reaching the lowest at 72; in the interval of 72 to 144, the prediction error of the model shows a fluctuating upward trend, the prediction effect is unstable, and the training time of the model increases significantly. Therefore, the input sequence length is finally set to 72. seq_len

[0164] Comparison of prediction results of each model: based on the above hyperparameter settings, the input sequence length seq_len is 72, and the prediction sequence length pred_len is set to 1, 3, 6, 12 and 24 respectively, i.e. using the past 6 hours of hotspot temperature related data to predict the hotspot temperature in the next 2h with a granularity of 5 minutes, to evaluate the performance of the model under different prediction steps. Considering that deep learning algorithms have certain randomness, in order to improve the credibility of the results, the above four models are tested independently for 10 times, and the average value is taken as the final result. The specific values of the performance indicators of each model in different prediction tasks are shown in Table III. As can be seen from Table III, in the prediction tasks of 1~24 prediction steps, the MSSP-Transformer model proposed in this paper achieves the optimal prediction effect, and its advantage becomes more and more obvious with the increase of prediction time.

[0165] Table III Average prediction indicators of different models

[0166]

[0167] In order to more intuitively present the performance difference of each model under different prediction steps, this paper takes MAE as the index, visualizes the prediction error of each model (MSSP-Transformer, CNN-LSTM, LSTM, Transformer), as shown in Figure 7 .

[0168] From Table III and Figure 7 ​It can be seen that LSTM and CNN-LSTM models have good performance in ultra-short-term prediction tasks with prediction steps of 1 and 3. However, as the prediction step increases, especially in medium and long-term prediction scenarios with steps of 12 and above, the prediction error of these two types of models increases significantly, reflecting the serious error accumulation problem in long-term modeling. The Transformer model without self-supervised learning pre-training performs more stably in medium and long-term prediction tasks, but its performance fluctuates slightly in short-step tasks, indicating that its modeling ability for local feature changes is still insufficient.

[0169] In comparison, the MSSP-Transformer model shows better prediction accuracy at each prediction step, especially in long-term prediction tasks with steps of 12 and 24, with the smallest error increase and the most stable prediction trend. This indicates that the self-supervised pre-training mechanism effectively enhances the model's perception and modeling ability of long-term dependencies in time series, effectively suppressing the accumulation of errors. Overall, the MSSP-Transformer model has better prediction performance in both short-term and long-term prediction tasks of oil-immersed transformer hotspot temperature, can more accurately depict the evolution trend of hotspot temperature, and has high engineering application value.

[0170] At the same time, in order to more intuitively show the prediction performance of each model, the hotspot temperature prediction task with a prediction step of 3 is selected, and part of the prediction results of the four models (MSSP-Transformer, CNN-LSTM, LSTM, Transformer) on the test set are visualized and compared as shown in Figure 8 .

[0171] From the real change curve of the hotspot temperature, it can be known that the hotspot temperature prediction task with a granularity of minutes is relatively complex, because the real hotspot temperature curve contains a large number of small mutations and fluctuations, but the overall trend of the hotspot temperature in the next few days can be predicted. From Figure 8 it can be seen that the MSSP-Transformer model accurately predicts the rising trend, fluctuating rising trend, stable fluctuation trend, and falling trend of the hotspot temperature, and has the best trend fitting degree compared with other models.

[0172] In view of the problems of insufficient long-term sequence dependence modeling, lack of local dynamic response ability, and low utilization rate of limited labeled data in oil-immersed transformer hotspot temperature prediction, an improved Transformer prediction model based on mask self-supervised pre-training is proposed.

[0173] The technical scheme of the embodiment effectively mines the potential time sequence rules of the unlabeled operation data by designing a mask reconstruction task, significantly improves the feature extraction capability of the model, and provides a more optimal parameter initialization for hot spot temperature prediction.

[0174] The technical scheme of the embodiment combines a local attention window with a global cross-attention mechanism in the Transformer architecture, takes into account the long-term evolution trend of the hot spot temperature and local dynamic features such as load mutation, and effectively enhances the adaptability of the model to non-stationary scenes.

[0175] On the actual operation data set of a 220kV transformer, the MSSP-Transformer achieves the best performance in the 1~24 step prediction task, with a determination coefficient The mean reaches 0.9354, and the mean absolute error mean is 0.6489°C, and the error accumulation rate is the lowest as the prediction step increases, showing high prediction performance.

[0176] In summary, the MSSP-Transformer performs well in both short-term and medium-term and long-term prediction tasks of transformer hot spot temperature, and can provide effective support for device thermal state evaluation, life management and risk warning, and has good engineering application value. Future research will further explore cross-device transfer learning and uncertainty quantification methods to improve the generalization ability and practicality of the model in multiple scenarios.

[0177] Therefore, the technical scheme of the embodiment is based on the self-supervised pre-training mechanism of mask reconstruction, learns time sequence features using unlabeled data, and improves the feature extraction capability and generalization performance of the model. Based on the local-global collaborative attention mechanism, a local attention window is introduced to enhance the sensitivity of the model to local features such as load mutation, while maintaining the ability to model global dependencies. A feature optimization and multi-step prediction framework is constructed, core features are selected based on the electric-thermal coupling mechanism and correlation analysis, and non-autoregressive decoding is used to realize multi-step prediction, effectively alleviating the error accumulation problem.

[0178] Embodiment Two

[0179] The purpose of the embodiment is to provide a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above method.

[0180] Embodiment Three

[0181] The purpose of the embodiment is to provide a computer readable storage medium.

[0182] A computer readable storage medium has a computer program stored thereon, which is executed by a processor to perform the steps of the above method.

[0183] Embodiment Four

[0184] The purpose of the present embodiment is to provide a Transformer model-based hot spot temperature prediction system for oil-immersed transformers, comprising:

[0185] A data acquisition module configured to acquire key operating parameter data of the oil-immersed transformer and perform data preprocessing;

[0186] A prediction module configured to input the key operating parameter data of the oil-immersed transformer after data preprocessing into a transformer hot spot temperature prediction model for prediction to obtain a hot spot temperature prediction result in a target period;

[0187] The transformer hot spot temperature prediction model adopts a Transformer architecture;

[0188] The Transformer architecture comprises an encoder and a decoder;

[0189] The encoder is formed by stacking a plurality of independent encoder layers with the same structure; each encoder layer comprises a locally enhanced multi-head attention sublayer and a feedforward network sublayer, and is connected in series through residual connection and layer normalization;

[0190] The decoder is formed by stacking a plurality of independent decoder layers with the same structure; each decoder layer comprises a masked local multi-head attention sublayer, an encoding-decoding cross-attention sublayer, and a feedforward network sublayer, and is connected in series through residual connection and layer normalization.

[0191] Embodiment Five

[0192] The purpose of the present embodiment is to provide a computer program product comprising instructions which, when run on a computer, cause the computer to perform the method and functions involved in any of the above embodiments.

[0193] The steps and method embodiments involved in the above embodiments correspond to Embodiment One, and the specific embodiments can refer to the relevant description part of Embodiment One. The term "computer readable storage medium" should be understood to include a single medium or multiple media of one or more instruction sets; it should also be understood to include any medium capable of storing, encoding, or carrying instruction sets for execution by a processor and causing the processor to perform any of the methods in the present application.

[0194] Those skilled in the art should understand that the modules or steps of the present application described above can be realized by general computer devices, or alternatively, they can be realized by program codes executable by the computer devices, so that they can be stored in the storage devices and executed by the computer devices, or they can be respectively manufactured into individual integrated circuit modules, or a plurality of modules or steps among them can be manufactured into a single integrated circuit module. The present application is not limited to any specific combination of hardware and software.

[0195] The specific embodiments of the present application described above in conjunction with the accompanying drawings are not intended to limit the protection scope of the present application. Those skilled in the art should understand that various modifications or changes made on the basis of the technical solutions of the present application without creative labor are still within the protection scope of the present application.

Claims

1. A method for predicting hot-spot temperature of oil-immersed transformer based on Transformer model, characterized in that, The method comprises the following steps: obtaining key operating parameter data of an oil-immersed transformer and performing data preprocessing; inputting the data of the key operating parameter data of the oil-immersed transformer after data preprocessing into a transformer hotspot temperature prediction model for prediction to obtain a hotspot temperature prediction result in a target period; wherein the transformer hotspot temperature prediction model adopts a Transformer architecture; the Transformer architecture comprises an encoder and a decoder; the encoder is formed by stacking a plurality of independent encoder layers with the same structure; each encoder layer comprises a locally enhanced multi-head attention sublayer and a feedforward network sublayer, and is connected in series through a residual connection and layer normalization; the decoder is formed by stacking a plurality of independent decoder layers with the same structure; each decoder layer comprises a masked local multi-head attention sublayer, an encoding-decoding cross-attention sublayer and a feedforward network sublayer, and is connected in series through a residual connection and layer normalization; a local attention window is introduced in the locally enhanced multi-head attention sublayer of the encoder and the masked local multi-head attention sublayer of the decoder; the global attention mode is retained in the encoding-decoding cross-attention sublayer of the decoder to make full use of the complete time context information output by the encoder; the encoder is used for deep representation learning of input time features to capture global dependencies and local correlations between key operating parameters; the masked attention in the decoder is used to shield future time step information, and the cross-attention is used to fuse the encoder output features to realize step-by-step decoding of the target prediction sequence.

2. The Transformer model-based hot-spot temperature prediction method of oil-immersed transformers according to claim 1, characterized in that, The Transformer architecture further comprises an input embedding module and a prediction output module: the input embedding module comprises a first linear embedding layer and a second linear embedding layer, which respectively perform dimension mapping on the input features of each time step and superimpose position encoding to represent time position information, thereby constructing the input sequences of the encoder and the decoder, respectively; the prediction output module is arranged after the decoder, maps the hidden features output by the decoder to the hotspot temperature prediction value of the target time step through a fully connected layer, and realizes the prediction output of the hotspot temperature of the transformer at multiple future time points.

3. The Transformer model-based hot-spot temperature prediction method of oil-immersed transformers according to claim 1, characterized in that, When pre-training the transformer hotspot temperature prediction model, a masked reconstruction self-supervised pre-training task is performed on unlabeled operating data to enable the transformer hotspot temperature prediction model to learn the time dependencies between the key operating parameters of the oil-immersed transformer; the model parameters after pre-training are used as initial weights, and supervised fine-tuning is performed on sample data containing real hotspot temperature labels to enable the model to perform short-term multi-step prediction.

4. The Transformer model-based hot-spot temperature prediction method of oil-immersed transformers according to claim 1, characterized in that, The key operating parameter data of the oil-immersed transformer comprises load current, ambient temperature, high-voltage side active power, top oil temperature and hotspot temperature; the data preprocessing of the key operating parameter data of the oil-immersed transformer comprises: filling missing values and processing outliers of the key operating parameter data of the oil-immersed transformer; performing feature selection on the processed data to determine the optimal input variable set of the transformer hotspot temperature prediction model; performing normalization data processing on the data in the optimal input variable set to obtain the preprocessed data.

5. The oil-immersed transformer hotspot temperature prediction system based on the Transformer model, characterized in that, The method comprises the following steps: a data acquisition module configured to acquire key operating parameter data of an oil-immersed transformer and perform data preprocessing; a prediction module configured to input the key operating parameter data of the oil-immersed transformer after data preprocessing into a transformer hotspot temperature prediction model for prediction to obtain a hotspot temperature prediction result in a target period; wherein the transformer hotspot temperature prediction model adopts a Transformer architecture; the Transformer architecture comprises an encoder and a decoder; the encoder is formed by stacking a plurality of independent encoder layers with the same structure; each encoder layer comprises a locally enhanced multi-head attention sublayer and a feedforward network sublayer, and is connected in series through residual connection and layer normalization; the decoder is formed by stacking a plurality of independent decoder layers with the same structure; each decoder layer comprises a masked local multi-head attention sublayer, an encoder-decoder cross-attention sublayer and a feedforward network sublayer, and is connected in series through residual connection and layer normalization; a local attention window is introduced in the locally enhanced multi-head attention sublayer of the encoder and the masked local multi-head attention sublayer of the decoder; the global attention mode is retained in the encoder-decoder cross-attention sublayer of the decoder to make full use of the complete time sequence context information output by the encoder; the encoder is used to perform deep representation learning on input time sequence features to capture global dependencies and local correlations between key operating parameters; the masked attention in the decoder is used to mask future time step information, and the cross-attention is used to fuse the encoder output features to realize step-by-step decoding of the target prediction sequence.

6. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to realize the method of any one of claims 1-4.

7. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to realize the steps of the method of any one of claims 1-4.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to perform the steps of the method of any one of claims 1-4.

Images