Method and system for predicting hot-spot temperature of oil-immersed transformer based on Transform model

Abstract

The invention provides an oil-immersed transformer hot-spot temperature prediction method and system based on a Transform model, and belongs to the technical field of transformer hot-spot temperature prediction, and the method comprises the steps: obtaining key operation parameter data of an oil-immersed transformer, and carrying out the data preprocessing; and inputting the key operation parameter data of the oil-immersed transformer after data preprocessing into the transformer hot-spot temperature prediction model for prediction. On the basis of a self-supervised pre-training mechanism of mask reconstruction, time sequence features are learned by using label-free data, so that the feature extraction capability and generalization performance of the model are improved.

Description

Technical Field

[0001] This invention belongs to the field of transformer hot spot temperature prediction technology, and particularly relates to a method and system for predicting hot spot temperature of oil-immersed transformers based on the Transformer model. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Under long-term high-load operation and complex working conditions, the internal insulation system of a transformer is susceptible to thermal stress and gradual deterioration. The winding hot spot temperature, as a key indicator reflecting the thermal state of the winding, directly affects the assessment of transformer insulation life, overload capacity judgment, and the formulation of operation and maintenance strategies. Therefore, achieving accurate prediction of hot spot temperatures is of great significance for ensuring the safe and stable operation of the power grid and improving equipment health management.

[0004] With the rapid development of artificial intelligence technology, deep learning, with its powerful feature extraction and nonlinear modeling capabilities, has gradually become an important tool in the field of time series forecasting and has been widely used in power equipment condition assessment and operating parameter prediction.

[0005] In hotspot temperature prediction, traditional neural network models such as Back Propagation Neural Network (BP) and Long Short-Term Memory Network (LSTM) model historical operating data to depict the complex nonlinear relationship between input features and hotspot temperatures, thus achieving hotspot temperature prediction. To further reduce noise interference in the original signal and improve the model's adaptability to dynamic temperature changes, some studies have introduced signal processing methods such as Empirical Mode Decomposition (EMD) combined with deep learning models to form composite prediction methods, which show better engineering applicability in terms of prediction accuracy and robustness. Overall, deep learning-based hotspot temperature prediction methods have, to some extent, overcome the limitations of traditional thermodynamic models. Meanwhile, the Transformer model, with its unique self-attention mechanism, provides a new solution to the long-sequence modeling challenge in hotspot temperature prediction.

[0006] Compared to models like BP and LSTM, Transformer can directly uncover global dependencies at different time points in a sequence, effectively mitigating the gradient vanishing problem in long-sequence modeling of recurrent neural networks. Its highly parallel architecture also significantly improves training efficiency, making it more suitable for modeling large-scale power time-series data. Currently, research has applied Transformer to power fields such as power load forecasting and electric vehicle regulation capacity forecasting, and proposed improved methods such as XGBoost-Transformer and Bayesian-Transformer, laying the technical foundation for its extension to hotspot temperature forecasting.

[0007] Furthermore, self-supervised learning (SSL), as an emerging training method in the field of deep learning, has been widely applied in image processing, natural language processing, and time series analysis. Its core advantage lies in its ability to fully utilize large amounts of unlabeled data for training. By designing self-supervised tasks such as mask reconstruction, it guides the model to autonomously learn the inherent patterns and dynamic characteristics of the data. It is particularly suitable for industrial scenarios in power systems where data labeling is difficult and costly, providing a new approach to improving the generalization ability of hotspot temperature prediction models.

[0008] While existing methods for predicting winding hotspot temperatures have made some progress, in-depth analysis considering practical application needs and technical characteristics reveals several remaining issues, including insufficient modeling of long-term time-series dependencies, insensitivity to local features, and excessive reliance on limited labeled data. Traditional deep learning models such as BP and LSTM, limited by their network structure, struggle to effectively capture global dependencies in long-term data. In multi-step prediction scenarios, error accumulation easily occurs, leading to a rapid decline in prediction accuracy with increasing step size, failing to meet the practical needs of long-term transformer thermal state assessment. In contrast, Transformer models can effectively alleviate the gradient vanishing problem in long-sequence modeling of recurrent neural networks. However, existing Transformer models often employ a single global attention mechanism in hotspot temperature prediction, ignoring frequent local fluctuations in hotspot temperature data. This results in insufficient ability to capture local dynamic features, making it difficult to adapt to the actual engineering requirements of hotspot temperature prediction. Furthermore, most existing models rely solely on supervised learning, excessively depending on limited labeled data and failing to fully utilize operational data. The influence of noise and operating condition disturbances on actual monitoring data further restricts the model's generalization ability and prediction accuracy. Summary of the Invention

[0009] To overcome the shortcomings of the prior art, this invention provides a method and system for predicting hot spot temperature of oil-immersed transformers based on the Transformer model, which can be used to achieve short-term multi-step prediction of transformer hot spot temperature.

[0010] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: Firstly, a method for predicting hotspot temperatures in oil-immersed transformers based on the Transformer model is disclosed, including: Acquire key operating parameter data of oil-immersed transformers and perform data preprocessing; The key operating parameters of the oil-immersed transformer after data preprocessing are input into the transformer hot spot temperature prediction model for prediction, and the hot spot temperature prediction results within the target time period are obtained. The transformer hotspot temperature prediction model adopts the Transformer architecture. The Transformer architecture includes an encoder and a decoder; The encoder is formed by stacking multiple independent encoder layers with the same structure; each encoder layer includes a locally enhanced multi-head attention sublayer and a feedforward network sublayer, which are connected in series with layer normalization through residual connections. The decoder is composed of multiple independent decoder layers with the same structure stacked together; each decoder layer includes a masked local multi-head attention sublayer, an encoding-decoding cross-attention sublayer and a feedforward network sublayer, and is connected in series with layer normalization through residual connections.

[0011] As a further technical solution, a local attention window is introduced into the locally enhanced multi-head attention sublayer of the encoder and the masked local multi-head attention sublayer of the decoder. Global attention patterns are preserved in the encoder-decoder cross-attention sublayer of the decoder to fully utilize the complete temporal context information of the encoder output.

[0012] As a further technical solution, the encoder is used to perform deep representation learning on the input temporal features in order to capture the global dependencies and local correlations between key operating parameters; The mask attention in the decoder is used to mask information from future time steps, and the cross attention is used to fuse the encoder output features to achieve stepwise decoding of the target prediction sequence.

[0013] As a further technical solution, the Transformer architecture also includes an input embedding module and a prediction output module: The input embedding module includes a first linear embedding layer and a second linear embedding layer. The first linear embedding layer and the second linear embedding layer respectively perform dimensional mapping on the input features at each time step and superimpose position encoding to represent temporal position information, thereby constructing the input sequences of the encoder and decoder respectively. The prediction output module is set after the decoder. It maps the hidden features output by the decoder to the hotspot temperature prediction value at the target time step through a fully connected layer, thereby realizing the prediction output of the hotspot temperature of the transformer at multiple future times.

[0014] As a further technical solution, when pre-training the transformer hot spot temperature prediction model, a mask reconstruction self-supervised pre-training task is performed on unlabeled operating data to enable the transformer hot spot temperature prediction model to learn the temporal dependencies between key operating parameters of oil-immersed transformers. The pre-trained model parameters are used as initial weights, and supervised fine-tuning is performed on sample data containing real hotspot temperature labels to enable the model to achieve short-term multi-step prediction of hotspot temperatures.

[0015] As a further technical solution, the key operating parameters of the oil-immersed transformer include load current, ambient temperature, active power on the high-voltage side, top oil temperature, and hot spot temperature.

[0016] As a further technical solution, the key operating parameter data of the oil-immersed transformer undergoes data preprocessing, including: Missing values ​​were filled and outliers were handled when acquiring key operating parameter data of oil-immersed transformers; Feature selection is performed on the processed data to determine the optimal set of input variables for the transformer hotspot temperature prediction model; Normalize the data in the optimal input variable set to obtain preprocessed data.

[0017] As a further technical solution, feature selection is performed on the processed data, specifically including: Based on the characteristic variables load current, ambient temperature, high-voltage side active power, and top oil temperature, the historical hot spot temperatures of the two previous times of the current moment in the transformer hot spot temperature prediction model are introduced as supplementary input features. Candidate variables are screened using a combination of electro-thermal coupling mechanism and correlation analysis to finally determine the optimal set of input variables for the transformer hot spot temperature prediction model.

[0018] Secondly, a hotspot temperature prediction system for oil-immersed transformers based on the Transformer model is disclosed, including: The data acquisition module is configured to acquire key operating parameter data of oil-immersed transformers and perform data preprocessing. The prediction module is configured to input the key operating parameter data of the oil-immersed transformer after data preprocessing into the transformer hot spot temperature prediction model for prediction, and obtain the hot spot temperature prediction results within the target time period. The transformer hotspot temperature prediction model adopts the Transformer architecture. The Transformer architecture includes an encoder and a decoder; The encoder is formed by stacking multiple independent encoder layers with the same structure; each encoder layer includes a locally enhanced multi-head attention sublayer and a feedforward network sublayer, which are connected in series with layer normalization through residual connections. The decoder is composed of multiple independent decoder layers with the same structure stacked together; each decoder layer includes a masked local multi-head attention sublayer, an encoding-decoding cross-attention sublayer and a feedforward network sublayer, and is connected in series with layer normalization through residual connections.

[0019] The above one or more technical solutions have the following beneficial effects: This invention's technical solution is based on a transformer hotspot temperature prediction model, which can directly extract the nonlinear temperature evolution patterns between key variables from historical operating data without relying on traditional thermal model parameter settings. By introducing a multi-head attention mechanism that coordinates local and global operations, the model can not only depict the long-term dependencies of hotspot temperature data, but also effectively perceive drastic local temperature fluctuations caused by sudden operating conditions, thereby improving the precision of feature modeling and the stability of prediction.

[0020] The technical solution of this invention is based on a mask reconstruction-based self-supervised pre-training strategy, which can complete feature representation learning on a large amount of unlabeled running data, alleviate the limitations of model training caused by insufficient labeled samples, and improve the model's generalization ability. By combining supervised fine-tuning with non-autoregressive decoding, the model can quickly output hotspot temperatures for multiple future time steps, avoiding the problem of gradual error accumulation in the autoregressive prediction framework, and helping to improve the efficiency and accuracy of short-term multi-step prediction.

[0021] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0022] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0023] Figure 1 This is an improved Transformer architecture according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the multi-head attention mechanism in an embodiment of the present invention; Figure 3 This is a schematic diagram of mask self-supervised pre-training; Figure 4 A schematic diagram of the correlation coefficient matrix for various characteristic parameters; Figure 5 For different d_model and mask_ratio A schematic diagram of the prediction error below; Figure 6 For different seq_len A schematic diagram of the prediction error below; Figure 7 For different pred_len The following is a schematic diagram of the model's prediction loss; Figure 8 This is a schematic diagram of the hotspot temperature prediction results. Detailed Implementation

[0024] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0025] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0026] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0027] Example 1 This embodiment discloses a method for predicting hot spot temperature of oil-immersed transformers based on the Transformer model, including: Step 1: Obtain key operating parameter data of the oil-immersed transformer and perform data preprocessing; Step 2: Construct the input feature set for the transformer hotspot temperature prediction model based on the preprocessed key operating parameter data; Step 3: Construct a masked self-supervised pre-training task to pre-train the model on unlabeled running data; Step 4: Use the model parameters obtained in the pre-training stage as the initial parameters for fine-tuning, and perform supervised fine-tuning on a dataset containing real hotspot temperature labels; Step 5: Output the predicted hotspot temperatures for multiple future time steps using a non-autoregressive decoding method.

[0028] In this embodiment, the key operating parameter data of the oil-immersed transformer acquired in step one above includes load current, ambient temperature, high-voltage side active power, top oil temperature, and hot spot temperature. The key operating parameter data of the oil-immersed transformer are collected synchronously with sensors through the transformer's on-site online monitoring system, with a sampling period of 5 minutes to ensure the correspondence of each parameter at the same point in time. The collected data is historical monitoring data from the actual operation of the transformer, which can accurately reflect its thermal-electrical operating characteristics.

[0029] To ensure the stability of model training and the reliability of prediction results, preprocessing is required based on data characteristics to reduce redundant information in the data model. Data processing includes data cleaning, feature selection, and data normalization.

[0030] 1-1) Data Cleaning: Due to occasional communication interruptions or sensor malfunctions in the field acquisition system, some time points contain missing values ​​or obvious outliers. This embodiment's sub-solution uses linear interpolation to fill in single-point missing data in the acquired key operating parameters of the oil-immersed transformer, and removes consecutively missing data or time periods exceeding the threshold. Outliers are detected using the 3σ principle and combined with practical engineering experience; values ​​outside the acceptable range are considered outliers and are removed or replaced.

[0031] 1-2) Feature Selection: In hotspot temperature prediction tasks, different input variables contribute differently to model performance, and some variables may have strong redundancy or even introduce noise, affecting model prediction performance. Therefore, it is necessary to screen and optimize the feature set.

[0032] Based on the electro-thermal coupling mechanism revealed by the IEC thermal model, the transformer hot spot temperature is mainly determined by both internal heating and external heat dissipation conditions. According to this mechanism and relevant literature, load current, top oil temperature, and ambient temperature are identified as the main influencing factors on the transformer hot spot temperature. Furthermore, considering the historical dependence of the thermal dynamics process, the model's prediction... t When determining the hotspot temperature at any given moment, introduce... t Hotspot temperature at the two sampling times before time step T hst ( t -1) and T hst ( t -2) is used as model input.

[0033] To further explore the potential value of other monitoring parameters, active power in the dataset was included as a candidate feature.

[0034] Meanwhile, to avoid feature redundancy, the Pearson Correlation Coefficient (PCC) method is used to quantitatively analyze the correlation between the feature variables. The Pearson correlation coefficients between each feature variable are calculated. r xy Its mathematical definition is as follows: (1) in, x i and y i They represent the firsti The relationship between a certain feature variable and the hotspot temperature in a sample , Its mean, n This represents the number of samples.

[0035] Through calculation r xy This method can initially assess the correlation between various feature variables and hotspot temperatures, eliminating inputs with insignificant correlation to the target variable. It also quantifies the degree of correlation between different feature variables, avoiding information redundancy.

[0036] Feature selection effectively compresses the model input dimension, improves data representation capabilities, reduces noise interference, and lays a solid foundation for subsequent model training and prediction.

[0037] 1-3) Data Normalization: The feature-selected dataset is divided into training, validation, and test sets in chronological order. The training set is used to optimize the prediction model structure and internal parameters, the validation set is used to monitor overfitting and adjust hyperparameters, and the test set is used to evaluate the final performance of the prediction model. To avoid problems such as gradient imbalance and slow convergence during model training due to different physical dimensions of the input variables, normalization parameters are calculated only based on the training set data. Min-Max normalization is performed on all input data to uniformly map all input data to the [0,1] interval, as shown in Equation (2).

[0038] (2) in, X input It is the original input data. X' input It is normalized data. X min and X max The minimum and maximum values ​​are calculated based on the training set. Normalization parameters, obtained from the training set, are also applied to the validation and test sets to ensure consistency in data processing and prevent data leakage.

[0039] In one implementation example, the purpose of step two is to transform the key operating parameter data, which has undergone feature selection and normalization in step one, into a feature set adapted to the input of the Transformer model, thus completing the data transformation from data preprocessing to model input. This step ensures that subsequent model pre-training and fine-tuning can proceed smoothly, and specifically includes feature integration and constructing the input matrix.

[0040] 2-1) Feature integration: The core features selected in step one are spliced ​​together in chronological order to form a unified time series feature set.

[0041] 2-2) Constructing the input matrix: Generate an input feature matrix of dimension K×k from the integrated features according to the requirements of Transformer. X input , where 𝐿 is the time series length of the feature variable and 𝑁 is the feature dimension. The generated feature matrix undergoes integrity and dimensionality checks to ensure no missing values ​​and that all features have been normalized.

[0042] In one implementation example, the purpose of step three is to use a large amount of unlabeled historical running data to construct a masked self-supervised learning task, forcing the Transformer model to learn the time series characteristics of transformer hotspot temperatures and the coupling relationships between variables, providing high-quality initial parameters for the subsequent fine-tuning stage.

[0043] In this embodiment, the transformer hotspot temperature prediction model adopts an improved Transformer architecture.

[0044] Hotspot temperature data is a typical example of multivariate time series data, and its inherent temporal correlation and multidimensional feature structure are highly compatible with the core architecture of Transformer. This embodiment's sub-solution, while referencing the original Transformer architecture, has structurally adjusted the model to better meet the specific requirements of short-term hotspot temperature prediction tasks.

[0045] 3-1) Infrastructure: Traditional Transformers excel at capturing long-term dependencies thanks to their global self-attention mechanism. However, in the task of predicting hotspot temperatures of oil-immersed power transformers, the time series not only exhibits long-term trends and periodicity but is also frequently affected by factors such as sudden changes in power load and switching of cooling systems, showing significant local fluctuations and abrupt changes. A single global attention mechanism may lack sufficient local sensitivity when dealing with drastic changes.

[0046] To address the aforementioned issues, this embodiment's sub-solution modifies the Transformer architecture, with the overall structure as follows: Figure 1 As shown. The core idea is to achieve collaborative work between local perception and global modeling by configuring attention modes differently in different functional modules. Specifically, local attention windows are introduced into the encoder's self-attention layer and the decoder's self-attention layer to enhance the ability to capture abrupt changes in features in the vicinity of time steps; while global attention is retained in the encoder-decoder cross-attention layer of the decoder to ensure the complete utilization of the global context of the input sequence.

[0047] The improved Transformer still maintains the overall encoder-decoder structure. The encoder consists of... L encThe decoder consists of stacked independent coding layers, each containing a multi-head attention layer with local enhancements, a feedforward connection layer, residual connections, and layer normalization. L dec It consists of stacked decoding layers. Unlike the encoding layers, its attention mechanism is divided into two parts: a masked local multi-head attention layer and a global encoding-decoding cross-attention layer, to ensure that the prediction process relies only on historical information and avoid future data leakage.

[0048] 3-2) Attention Mechanism and Improvement: Multi-Head Attention (MHA) is the core computational mechanism of each attention sublayer in the Transformer model. Its main function is to mine the mapping relationship between different time points of transformer hotspot temperature data, improve the model's trend capture ability in long-term prediction, and enhance the model's interpretability. Its basic structure is as follows: Figure 2 As shown.

[0049] Figure 2 middle, X input The input feature matrix constructed in step two has a size of [missing information]. L×N ; L Indicates the length of the time series; N The feature dimension is defined as follows: First, the input matrix is ​​mapped to a high-dimensional space through three independent fully connected linear layers to obtain the query matrix. Q Key matrix K Value matrix V The expression is: (3) In the formula, W q and b q , W k and b k , W v and b v These are the weights and bias matrices for the three fully connected layers. Their initial values ​​are generated using a standard random initialization method and are automatically updated during model training via backpropagation and gradient optimization to learn the mapping relationship between time steps in the input feature sequence. The weights and bias matrices of the linear fully connected layers in the following text are all generated in the same way and will not be described again.

[0050] After linear transformation, the result is Q , K , V The matrix dimensions are all L × D,in, L Indicates the length of the time series; D This represents the encoding dimension of the intermediate layer. Subsequently, to achieve multi-head parallel computation, [the following is used]... Q , K , V Partition the matrix along its column dimensions, dividing it into equal parts. h Submatrices, of which h To ensure the number of heads in a temporal multi-head attention mechanism, it is necessary to guarantee... D / h The integer is . Taking the query matrix Q as an example, it can be expressed as equation (4).

[0051] (4) in, Q i The first part after matrix partitioning i There are submatrices, with dimensions of . L×D / h , recorded as L × d k Here, i For attention head index, i =1,2,…, h -1, h .

[0052] Transformer employs a multi-head parallel mechanism to divide the data into multiple parts. h Each submatrix performs attention calculations in each attention head, yielding the output of each attention head. In the traditional global attention mechanism, the... i The output of each attention head is: (5) in, Q i , K i , V i They represent Q , K , V After performing matrix partitioning along its column dimensions, the first i There are submatrices, where T represents the matrix transpose.

[0053] To enhance the model's sensitivity to local fluctuations, a configurable local attention window mechanism is introduced, and hyperparameters are used to... w Controls whether it is enabled with the local window radius. This is done after calculating the scaled dot product attention score. When setting the radius of the local attention window w If the value is greater than 0, then construct a local attention mask matrix. MIts element definition is as follows: (6) In the formula, This is a local attention mask matrix, specifically... L OK L A real matrix of columns, Represents the first... i Line number j Column elements; i and j These are time step indices, with values ​​ranging from 1 to... i , j ≤ L ,in L Indicates the time step length of the input sequence; w Let be the radius of the local attention window, and be a non-negative integer hyperparameter; when | i - j |≤ w season =1, otherwise let =0, R It is the set of real numbers.

[0054] The mask matrix and the attention score matrix S i Perform the following operations to obtain the first local window under the constraints. i Attention scores at each time point : (7) In the formula, Represents the first under local window constraints i Attention scores at each time point. This operation will be located in the window [ i - w , i + w The attention score outside of [the previous value] is set to negative infinity. After Softmax normalization, its weights approach 0, thus making the [other value] [the next value]. i At any given time point, only its nearest 2 w +1 time point enables fine-grained modeling of short-term fluctuations and abrupt changes. If w If the value is 0, the mechanism degenerates into global attention, which is used to capture long-term dependencies and overall trends.

[0055] Finally, the outputs of all attention heads are concatenated along their feature dimensions and mapped back to the original dimensions using a linear transformation to obtain the final output of the multi-head attention: (8) in, W O To output the projection matrix, head1,…, head h This represents the output of all attention heads. Concat(·) represents the concatenation operation of the outputs of each attention head according to the feature dimension. MHA represents the multi-head attention mechanism composed of the above attention heads in parallel.

[0056] By configuring different local attention window radii for different modules w This enables the synergy between local perception and global modeling, thereby better adapting to the multi-scale temporal dependence characteristics in hotspot temperature prediction.

[0057] 3-3) Overall Prediction Process: To achieve short-term, multi-step prediction of transformer hotspot temperatures, a Transformer model based on mask-based self-supervised pre-training was constructed, employing an integrated prediction framework of pre-training and fine-tuning. During the pre-training phase, a large amount of unlabeled running data was fully utilized for feature learning to improve the model's ability to extract general features. In the fine-tuning phase, the parameters obtained during pre-training were used as initial weights for supervised learning of hotspot temperature prediction in downstream tasks, ultimately achieving stable prediction of hotspot temperatures.

[0058] Training of the transformer hotspot temperature prediction model: pre-training stage based on self-supervised learning.

[0059] This paper introduces a mask-based self-supervised pre-training method to fine-grained short-term prediction of transformer hotspot temperatures. During the pre-training phase, the model focuses on mask reconstruction. By randomly masking a portion of the time steps in the input feature sequence according to a predetermined strategy, and then reconstructing this masked data using a Transformer network, the model is forced to delve deeper into the intrinsic patterns of temporal features. This phase not only improves the model's feature extraction and generalization capabilities but also provides better parameter initialization for subsequent prediction tasks. The specific training process for the mask reconstruction task is as follows: Figure 3 As shown.

[0060] In the specific implementation, the preprocessed input feature sequence X' input ∈R L×N Perform masking. Use a single-time-point-level random masking method, setting the mask ratio to [value missing]. H That is, in L Randomly selected from each time step H · L Masking is performed at specific positions. This involves setting all features of the input feature sequence to zero at that time step, marking the valid bits as invalid, and keeping the rest as their original values. After masking, the masked input is obtained. X masked ∈R L×N , L Indicates the length of the time series; Nis the feature dimension, and is the set of real numbers. By completely removing the original information of the masked location, the model is forced to rely solely on the temporal correlation and feature coupling relationship between the non-masked regions to infer the feature values ​​of the masked region, thus completing the reconstruction task.

[0061] Data after masking X masked The original feature dimensions are reduced through a linear embedding layer. N Mapping to encoding dimension D ,get X emb ∈R L×D To further enhance the sequential relationship of time-series data in the time dimension, a positional encoding based on sine and cosine functions is introduced, as shown in equations (9) and (10): (9) (10) in, p This represents the position of the time step in the sequence. p= 1, …, L ; i For encoding dimension index, i =1, …, D .

[0062] Positional encoding adds unique location information to each input marker, enabling the model to clearly define the position of data points within the temporal sequence, thus effectively capturing the temporal dependencies of the context. This location information is then further superimposed onto the masked data to obtain the final input to the encoder. X enc ∈R L×D As shown in equation (11): (11) in, X enc The input is the position-encoded input. X emb For the encoded output, PE is the positional encoding function.

[0063] Position-encoded input X enc It is fed into the encoder. This module consists of... L encThe model consists of cascaded Transformer encoder layers, each including a multi-head self-attention layer and a feedforward network layer, connected in series with residual connections and layer normalization. In the multi-head self-attention layer, the model calculates the correlation weights between positions in the sequence based on scaled dot product attention, dynamically aggregating contextual information to model temporal dependencies. Subsequently, the feedforward network uses two layers of linear mapping and a ReLU activation function to perform a non-linear transformation on the attention output, thereby improving the model's feature representation capability. The residual structure and layer normalization further alleviate the gradient vanishing problem in deep networks and promote efficient feature transfer. After continuous... L enc After processing by the layer encoder, the model finally outputs an encoded representation, which contains a context feature tensor with complete temporal dependencies, denoted as . Y enc ∈R L×D .

[0064] (12) in, This represents the set of learnable parameters of the encoder during the pre-training phase. Y enc It has integrated the temporal and feature information of all non-masked regions and can be directly used for the reconstruction of masked regions.

[0065] During the pre-training phase, the encoder output is mapped back to the original feature dimensions using a reconstruction head. N This enables the reconstruction of the mask time step data, as shown in equation (13).

[0066] (13) In the formula, The reconstructed data matrix, ∈R L×N , W rec For the weight matrix of the reconstructed head, b rec This is the bias vector for the reconstructed head.

[0067] The reconstruction loss is calculated only for the error between the prediction and the true value at the mask time step. The mean squared error (MSE) is used as the objective function for pre-training, as shown in Equation (14): (14) in, X ( i ) represents the true value at the nth masked time step; Represents the reconstructed value of the model at the nth masked time step; 𝑁 maskRepresents the total number of mask time steps involved in the reconstruction; 𝑖 is the index of the mask time step; 𝐿 loss Let be the reconstruction loss function at the mask location.

[0068] Update by minimizing this loss function Parameters such as encoder and embedding layer parameters are retained after pre-training as initialization parameters for the fine-tuning stage.

[0069] In one implementation example, the purpose of step four is to transfer the model parameters obtained from the pre-training in step three to the downstream prediction task, so as to achieve high-precision multi-step prediction with limited labeled data.

[0070] Supervised fine-tuning is performed based on the initialization parameters obtained from pre-training to achieve multi-step prediction of hotspot temperatures. To suppress error accumulation and improve parallel computing efficiency, a non-autoregressive decoding strategy is adopted, and masked local self-attention is introduced into the decoder to avoid future information leakage.

[0071] To adapt to downstream prediction tasks, the model structure in the pre-training stage is adjusted. The reconstruction head, used only for mask reconstruction and unrelated to hotspot temperature prediction, is removed in the fine-tuning stage; while the encoder and embedding layer parameters are migrated to the fine-tuning stage. Consistent with the pre-training stage, the encoder models the historical feature sequence of the input through a locally enhanced multi-head attention mechanism, outputting a contextual feature tensor containing complete temporal dependencies. Y enc .

[0072] In order to predict the future T The hotspot temperature values ​​at each time step are initialized as the decoder input to a single dimension R. T×1 The all-zero tensor, i.e. Y (0) =0∈R T×1 .in, T For the prediction step size, 1 indicates that only the temperature of a single hotspot is predicted.

[0073] Y (0) First, the feature dimensions are transformed through a linear embedding layer and embedded into a feature space consistent with the encoder, as shown in Equation (15): (15) in, W dec Dimension 1× D , is the learnable projection matrix of the decoder embedding layer. b dec The bias vector, which has a dimension of 1×D, is added to the output of each time step through a broadcast mechanism.

[0074] Position encoding is added to each prediction time step to introduce temporal and positional information, resulting in the decoder input sequence. Y dec-in .

[0075] The position-encoded input sequence is fed into the decoder module, which consists of... L dec The Transformer decoder consists of cascaded layers, each including a masked local self-attention sublayer and an encoder-decoder cross-attention sublayer. The masked local self-attention sublayer uses a local window mask to restrict the decoder to focus only on the area in front of it at each time step. w Each time step enhances the modeling ability for short-term fluctuations in hotspot temperatures while ensuring that only historical and currently visible features are used, avoiding future information leakage; the encoder-decoder cross-attention sublayer uses the output features of the decoder's masked self-attention as the query matrix. Q encoder output Y enc Key matrix K Sum matrix V By scaling the dot product attention, the association weights between historical features and future hotspot temperatures are calculated, establishing a mapping relationship between historical features and future evolution, thus fully utilizing long-term temporal dependencies. Through step four, the model achieves multi-step prediction capability for hotspot temperatures with limited labeled data, while suppressing error accumulation and improving prediction stability.

[0076] In one implementation example, the purpose of step five is to use a non-autoregressive decoding strategy to achieve efficient and stable prediction of future multi-step hotspot temperatures.

[0077] The decoder employs a non-autoregressive generation strategy to predict the future in parallel. T Step-by-step hotspot temperature measurement avoids the accumulation of errors in step-by-step prediction. After L dec After layer-by-layer processing by the layer decoder, the decoder output Y is obtained. dec ∈R T×D .

[0078] Ultimately, Y dec The future dimension is reduced to a single output dimension by a fully connected layer. T Hotspot temperature prediction sequence at each time step Y pre ∈R T×1 .

[0079] (16) in, W o , b o Y represents the weight matrix and bias vector of the output layer.dec This is the decoder output.

[0080] Fine-tuning phase using hotspot temperature prediction sequences Y pre The mean squared error between the model and the actual hotspot temperature sequence is used as the loss function, and the parameters of the entire model are iteratively optimized through backpropagation. Specifically, the encoder parameters are further updated based on pre-trained weights, while the decoder and output layer parameters are trained from random initialization. Through this integrated pre-training and fine-tuning strategy, the model can achieve high-precision multi-step prediction of hotspot temperatures with limited labeled data.

[0081] Engineering verification example: The dataset used consists of actual operating data from a 220kV oil-immersed transformer at a substation of State Grid Hebei Electric Power Co., Ltd., collected over three months from June to August 2023. The sampling period was 5 minutes, resulting in 26,496 data sets. The dataset includes load current. I Ambient temperature T amb Active power on the high-voltage side P L Top oil temperature T tot Hotspot temperature T hst Actual historical monitoring data was used and divided into training, validation, and test sets in an 8:1:1 ratio. To verify the superiority of the proposed MSSP-Transformer model in hotspot temperature prediction, Long Short-Term Memory (LSTM), an unsupervised Transformer model, and a Convolutional Neural Network-Long Short-Term Memory (CNN-LSTM) were selected as comparison models. All prediction models were built and implemented using the PyTorch framework in Python. Table I shows the main parameters of the comparison models. Parameter settings were referenced from the cited literature and optimized for the dataset used to ensure the validity of the comparison experiments.

[0082] Table I. Main parameter settings for each comparison model

[0083] Choose the root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination (r-square, R²). 2As the model evaluation index, the specific calculation formulas of the evaluation index are shown in equations (17), (18), and (19).

[0084] (17) (18) (19) In the formula, To predict hotspot temperatures; Real hotspot temperatures; N This represents the number of hotspot temperature samples.

[0085] Feature selection: Based on five data features contained in historical monitoring data, hotspot temperature is selected. T hst With top oil temperature T tot Ambient temperature T amb Load current I Active power on the high-voltage side P L Correlation analysis was performed, and the correlation coefficients between each feature parameter were calculated. r xy The value range of is [-1, 1], and the closer its absolute value is to 1, the stronger the correlation between the two variables. The correlation matrix between the features is as follows: Figure 4 As shown.

[0086] Depend on Figure 4 It can be seen that the top oil temperature, ambient temperature, and hot spot temperature all exhibit a strong linear correlation. This is consistent with the physical mechanism that they respectively dominate the internal heat transfer and external heat dissipation boundary conditions in the transformer's thermal dynamics process, and therefore they were determined as the core input variables of the model. The correlation coefficient between load current and hot spot temperature is 0.57. Although it is not as significant as the former two, as a direct driving factor for winding heating, its physical significance is clear, and therefore it was retained as an input feature.

[0087] In contrast, the correlation coefficient between active power and hotspot temperature was only 0.43, the lowest among all features, indicating its limited direct contribution to the prediction target. Meanwhile, its correlation coefficient with load current was as high as 0.91, demonstrating significant redundancy. Considering both the correlation analysis results and the physical mechanism, top-layer oil temperature, ambient temperature, and historical values ​​of load current and hotspot temperature were ultimately selected as the input variables for the prediction model.

[0088] Design a comparative experiment to verify the removal of input features. P LWill this negatively impact the model's predictive performance? While maintaining identical model structure, hyperparameters, and training conditions, two input schemes are constructed: Input set A is... T tot , T amb , I and T hst Historical values ​​are composed of, but do not include, P L Input set B then adds the features mentioned above. P L This forms a complete feature set. Both sets of data were preprocessed according to the method described in Section 3.1, and then independently trained and tested multiple times on a task with a prediction step size of 3. RMSE, MAE, and [other metrics were used]. The results, used as performance evaluation metrics for the model, are shown in Table II.

[0089] Table II P L Performance comparison of models whether or not they are used as input

[0090] As can be seen from Table II, the differences between the two schemes in all performance indicators are not significant. (Removing...) P L The subsequent model maintained superior performance in terms of prediction accuracy and stability. This indicates that... P L The introduction of this variable did not substantially improve the model performance, further validating the rationale for removing it during the feature selection stage.

[0091] Key model parameter settings: During actual debugging, it was found that the model's predictive performance was affected by the length of the input sequence. seq_len (This can also be understood as the size of the history window) and the mask ratio. mask_ratio and model hidden dimensions d_model The settings are quite sensitive. Among them, seq_len The settings are closely related to the first two parameters, so we will first analyze them. d_model and mask ratio The overall impact on model performance will be determined based on the analysis results. seq_len The value of .

[0092] Model Hidden Dimensions d_model The dimensionality determines the feature representation capability of the Transformer. Too small a dimensionality can prevent the model from fully learning the patterns of data variation on the training set, while too large a dimensionality increases computational cost and carries the risk of overfitting with limited sample size. Mask ratio mask_ratioThis is a key control parameter in the self-supervised pre-training stage. Too low a ratio makes the reconstruction task too simple, making it difficult to force the model to learn deep dependencies between features; while too high a ratio results in insufficient effective information and excessive reconstruction difficulty. Based on the above analysis, different hidden layer dimensions are set in the range of 64 to 256 with an interval of 32; simultaneously, different pre-training mask ratios are set in the range of 0.3 to 0.6 with an interval of 0.05, and combined with... d_ model A hotspot temperature prediction experiment with a prediction step size of 3 was conducted. Through multiple experiments and comparison of the mean prediction error (MAE), the differences in model prediction performance under different parameter values ​​were quantitatively evaluated, thereby determining the optimal parameter configuration for the model. The relationship between prediction error and changes in hidden dimension and mask ratio is shown below. Figure 5 As shown.

[0093] Depend on Figure 5 It can be seen that, with d_model With the increase of [variable name], the prediction error generally shows a trend of first decreasing and then increasing, which is consistent with the results of the previous analysis. In contrast, mask_ratio The effect of prediction error is more complex and is influenced by... d_model The impact. In d_model When smaller, mask_ratio The change had no significant impact on the prediction error; however, as d_model With the increase of , the optimal model corresponds to mask_ratio Gradually increasing. Comprehensive analysis shows that when... d_model It is 128. mask_ratio When the value is 0.45, the model's average prediction error is the lowest, thus determining the model's optimal performance. d_model and mask_ratio The optimal values ​​for two important parameters.

[0094] While maintaining the optimal values ​​for both, change seq_len The value of is selected to analyze its impact on the model's prediction performance. seq_len This determines the range of historical information the model can utilize for prediction. If the value is too small, the model struggles to capture the long-term dependence of hotspot temperatures on load and environmental changes, leading to increased prediction errors. Conversely, if the value is too large, redundant noise may be introduced, increasing computational complexity and reducing the model's generalization ability. Considering a data sampling interval of 5 minutes, different values ​​were set within the range of 36 to 144. seq_len For each value (corresponding to a historical window of 3-12 hours), a hotspot temperature prediction experiment with a prediction step size of 3 was conducted. Multiple experiments were performed, and the average prediction error (MAE) for each value was calculated. The results are as follows: Figure 6 As shown.

[0095] Depend on Figure 6 It can be seen that, with seq_lenAs the input sequence length increased from 36 to 72, the average prediction error of the model decreased significantly, reaching its lowest point at 72. Between 72 and 144, the prediction error showed a fluctuating upward trend, indicating unstable prediction performance and a significant increase in training time. Therefore, the final input sequence length was set to 72.

[0096] Comparative analysis of prediction performance of various models: Based on the above hyperparameter settings, input sequence length seq_len The predicted sequence length is 72. pred_len The prediction steps were set to 1, 3, 6, 12, and 24, respectively, representing a granularity of 5 minutes. Hotspot temperature data from the past 6 hours was used to predict the hotspot temperature for the next 2 hours, thus evaluating the model's performance at different prediction step sizes. Considering the inherent randomness of deep learning algorithms, to improve the reliability of the results, each of the four models underwent 10 independent prediction trials, and the average value was taken as the final result. The specific performance metrics of each model in different prediction tasks are shown in Table III. As shown in Table III, the proposed MSSP-Transformer model achieved the best prediction results for prediction tasks with 1 to 24 prediction steps, and its advantage became increasingly apparent as the prediction time increased.

[0097] Table III. Mean values ​​of prediction indicators for different models

[0098] To more intuitively present the performance differences of each model under different prediction step sizes, this paper uses MAE as the metric to visualize the prediction error of each model (MSSP-Transformer, CNN-LSTM, LSTM, Transformer). The results are as follows: Figure 7 As shown.

[0099] From Table III and Figure 7 It can be seen that LSTM and CNN-LSTM models perform well in ultra-short-term prediction tasks with prediction strides of 1 and 3, respectively. However, as the prediction stride increases, especially in medium- to long-term prediction scenarios with strides of 12 and above, the prediction errors of these two types of models rise significantly, reflecting a serious error accumulation problem in long-term modeling. The Transformer model, which is not pre-trained with self-supervised learning, shows more stable overall performance in medium- to long-term prediction tasks, but its performance fluctuates slightly in short-stride tasks, indicating that its ability to model local feature changes is still insufficient.

[0100] In comparison, the MSSP-Transformer model exhibits superior prediction accuracy across all prediction step sizes, particularly in long-term time-series prediction tasks with step sizes of 12 and 24, where it shows the smallest error increase and the most stable prediction trend. This indicates that the self-supervised pre-training mechanism effectively enhances the model's ability to perceive and model long-term dependencies in time series, effectively suppressing the gradual accumulation of errors. Overall, the MSSP-Transformer model demonstrates superior prediction performance in both short-term and long-term hotspot temperature prediction tasks for oil-immersed transformers, more accurately characterizing the evolution trend of hotspot temperature and possessing high engineering application value.

[0101] Meanwhile, to more intuitively demonstrate the prediction performance of each model, a hotspot temperature prediction task with a prediction step size of 3 was selected. The prediction results of the four models (MSSP-Transformer, CNN-LSTM, LSTM, and Transformer) on the test set were visualized and compared. Figure 8 As shown.

[0102] As can be seen from the actual hotspot temperature change curves, predicting hotspot temperatures at the minute-by-minute granularity is quite complex because the actual hotspot temperature curves contain numerous small abrupt changes and fluctuations. However, the overall trend of hotspot temperature changes over the next few days can be predicted. Figure 8 As can be seen, the MSSP-Transformer model accurately predicted the upward trend, fluctuating upward trend, stable fluctuating trend, and downward trend of hotspot temperature, and had the best trend fit compared with other models.

[0103] To address the problems of insufficient long-term time-series dependency modeling, lack of local dynamic response capability, and low utilization of limited labeled data in the prediction of hot spot temperature of oil-immersed transformers, an improved Transformer prediction model with mask self-supervised pre-training is proposed.

[0104] This embodiment's sub-solution effectively mines the potential temporal patterns of unlabeled running data by designing a mask reconstruction task, significantly improving the model's feature extraction capability and providing better parameter initialization for hotspot temperature prediction.

[0105] This embodiment introduces a mechanism that combines local attention windows with global cross-attention in the Transformer architecture, taking into account both the long-term evolution trend of hotspot temperatures and local dynamic features such as load mutations, effectively enhancing the model's adaptability to non-stationary scenarios.

[0106] On the actual operation dataset of 220kV transformers, MSSP-Transformer achieved the best performance in prediction tasks from step 1 to step 24, with a determination coefficient of [missing value]. The mean value reached 0.9354, the mean absolute error was 0.6489°C, and the error accumulation rate was the lowest as the prediction step size increased, demonstrating high prediction performance.

[0107] In summary, the MSSP-Transformer demonstrates superior performance in both short-term and medium-to-long-term prediction of transformer hotspot temperatures, providing effective support for equipment thermal condition assessment, life management, and risk warning, and possesses significant engineering application value. Future research will further explore cross-equipment transfer learning and uncertainty quantification methods to enhance the model's generalization ability and practicality across multiple scenarios.

[0108] Therefore, this embodiment's sub-solution utilizes a self-supervised pre-training mechanism based on mask reconstruction to learn temporal features from unlabeled data, thereby improving the model's feature extraction capability and generalization performance. Based on a local-global collaborative attention mechanism, a local attention window is introduced to enhance the model's sensitivity to local features such as load abrupt changes, while maintaining global dependency modeling capabilities. A feature optimization and multi-step prediction framework is constructed, combining electro-thermal coupling mechanisms and correlation analysis to select core features, and employing non-autoregressive decoding to achieve multi-step prediction, effectively mitigating the error accumulation problem.

[0109] Example 2 The purpose of this embodiment is to provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described method.

[0110] Example 3 The purpose of this embodiment is to provide a computer-readable storage medium.

[0111] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the above method.

[0112] Example 4 The purpose of this embodiment is to provide a hot spot temperature prediction system for oil-immersed transformers based on the Transformer model, including: The data acquisition module is configured to acquire key operating parameter data of oil-immersed transformers and perform data preprocessing. The prediction module is configured to input the key operating parameter data of the oil-immersed transformer after data preprocessing into the transformer hot spot temperature prediction model for prediction, and obtain the hot spot temperature prediction results within the target time period. The transformer hotspot temperature prediction model adopts the Transformer architecture. The Transformer architecture includes an encoder and a decoder; The encoder is formed by stacking multiple independent encoder layers with the same structure; each encoder layer includes a locally enhanced multi-head attention sublayer and a feedforward network sublayer, which are connected in series with layer normalization through residual connections. The decoder is composed of multiple independent decoder layers with the same structure stacked together; each decoder layer includes a masked local multi-head attention sublayer, an encoding-decoding cross-attention sublayer and a feedforward network sublayer, and is connected in series with layer normalization through residual connections.

[0113] Example 5 The purpose of this embodiment is to provide a computer program product containing instructions that, when run on a computer, causes the computer to perform the methods and functions involved in any of the embodiments described above. The steps and methods involved in the apparatus of the above embodiments correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0114] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0115] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for predicting hot spot temperature of oil-immersed transformers based on the Transformer model, characterized by: include: Acquire key operating parameter data of oil-immersed transformers and perform data preprocessing; The key operating parameters of the oil-immersed transformer after data preprocessing are input into the transformer hot spot temperature prediction model for prediction, and the hot spot temperature prediction results within the target time period are obtained. The transformer hotspot temperature prediction model adopts the Transformer architecture. The Transformer architecture includes an encoder and a decoder; The encoder is formed by stacking multiple independent encoder layers with the same structure; each encoder layer includes a locally enhanced multi-head attention sublayer and a feedforward network sublayer, which are connected in series with layer normalization through residual connections. The decoder is composed of multiple independent decoder layers with the same structure stacked together; each decoder layer includes a masked local multi-head attention sublayer, an encoding-decoding cross-attention sublayer and a feedforward network sublayer, and is connected in series with layer normalization through residual connections.

2. The method for predicting hot spot temperature of oil-immersed transformers based on the Transformer model as described in claim 1, characterized in that... The encoder’s locally enhanced multi-head attention sublayer and the decoder’s masked local multi-head attention sublayer introduce local attention windows; Global attention patterns are preserved in the encoder-decoder cross-attention sublayer of the decoder to fully utilize the complete temporal context information of the encoder output.

3. The method for predicting hot spot temperature of oil-immersed transformers based on the Transformer model as described in claim 1, characterized in that, The encoder is used to perform deep representation learning on the input temporal features in order to capture the global dependencies and local correlations between key operating parameters; The mask attention in the decoder is used to mask information from future time steps, and the cross attention is used to fuse the encoder output features to achieve stepwise decoding of the target prediction sequence.

4. The method for predicting hot spot temperature of oil-immersed transformers based on the Transformer model as described in claim 1, characterized in that, The Transformer architecture also includes an input embedding module and a prediction output module: The input embedding module includes a first linear embedding layer and a second linear embedding layer. The first linear embedding layer and the second linear embedding layer respectively perform dimensional mapping on the input features at each time step and superimpose position encoding to represent temporal position information, thereby constructing the input sequences of the encoder and decoder respectively. The prediction output module is set after the decoder. It maps the hidden features output by the decoder to the hot spot temperature prediction value at the target time step through a fully connected layer, thereby realizing the prediction output of the hot spot temperature of the transformer at multiple future times.

5. The method for predicting hot spot temperature of oil-immersed transformers based on the Transformer model as described in claim 1, characterized in that, During the pre-training of the transformer hot spot temperature prediction model, a mask reconstruction self-supervised pre-training task is performed on unlabeled operating data to enable the transformer hot spot temperature prediction model to learn the temporal dependencies between key operating parameters of oil-immersed transformers. The pre-trained model parameters are used as initial weights, and supervised fine-tuning is performed on sample data containing real hotspot temperature labels to enable the model to make multi-step predictions in a short period of time.

6. The method for predicting hot spot temperature of oil-immersed transformers based on the Transformer model as described in claim 1, characterized in that, The key operating parameters of the oil-immersed transformer include load current, ambient temperature, active power on the high-voltage side, top oil temperature, and hot spot temperature. The key operating parameter data of the oil-immersed transformer undergoes data preprocessing, including: Missing values ​​were filled and outliers were handled when acquiring key operating parameter data of oil-immersed transformers; Feature selection is performed on the processed data to determine the optimal set of input variables for the transformer hotspot temperature prediction model; Normalize the data in the optimal input variable set to obtain preprocessed data.

7. A hotspot temperature prediction system for oil-immersed transformers based on the Transformer model, characterized in that... include: The data acquisition module is configured to acquire key operating parameter data of oil-immersed transformers and perform data preprocessing. The prediction module is configured to input the key operating parameter data of the oil-immersed transformer after data preprocessing into the transformer hot spot temperature prediction model for prediction, and obtain the hot spot temperature prediction results within the target time period. The transformer hotspot temperature prediction model adopts the Transformer architecture. The Transformer architecture includes an encoder and a decoder; The encoder is formed by stacking multiple independent encoder layers with the same structure; each encoder layer includes a locally enhanced multi-head attention sublayer and a feedforward network sublayer, which are connected in series with layer normalization through residual connections. The decoder is composed of multiple independent decoder layers with the same structure stacked together; each decoder layer includes a masked local multi-head attention sublayer, an encoding-decoding cross-attention sublayer and a feedforward network sublayer, and is connected in series with layer normalization through residual connections.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method described in any one of claims 1-6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it performs the steps of the method described in any one of claims 1-6 above.

Images