A short-term load prediction method of an LSTM-Transformer hybrid model

Abstract

This invention discloses a short-term load forecasting method based on an LSTM-Transformer hybrid model. The method includes acquiring historical load sequences and temperature sequences from multiple meteorological stations in the target area, performing time alignment and windowing processing to construct input-output sample pairs. Through a multi-station temperature attention mechanism, the temperatures from multiple meteorological stations at each time step are weighted and fused. The fused feature sequence is then input into a hybrid time-series model composed of an LSTM network and a Transformer encoder, outputting future multi-step load forecast results. For continuous missing segments in the historical load sequence, the trained hybrid time-series model is used to perform autoregressive recursive reconstruction using a sliding window approach to generate a complete load sequence. The model adaptively learns and weights the influence of meteorological stations at different geographical locations on the load of the target area. Combining the short-term local dynamic capture capabilities of LSTM with the long-term periodic dependency modeling capabilities of Transformer, it achieves deep fusion and accurate prediction of multi-scale features, improving the accuracy of short-term load forecasting.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to a short-term load forecasting method using an LSTM-Transformer hybrid model. Background Technology

[0002] Short-term load forecasting (STLF) is a crucial foundational task in power grid operation and dispatch. Its forecasting results directly serve aspects such as unit allocation, economic dispatch, grid security constraint verification, and distributed energy consumption. With the development of smart grids and multiple energy sources, load curves exhibit stronger nonlinearity, multi-scale periodicity, and high sensitivity to meteorological conditions, significantly increasing the modeling difficulty of load forecasting.

[0003] Existing load forecasting methods can be broadly classified into three categories: First, traditional statistical models, such as ARIMA and exponential smoothing, mainly rely on linear correlation structures and have limited ability to characterize nonlinear and long-term dependencies; second, machine learning methods, such as support vector regression, random forest, and gradient boosting tree, can uncover nonlinear relationships between features to a certain extent, but their ability to represent time-series dependencies and complex periodic structures is still insufficient; and third, deep learning models, such as RNN, LSTM, GRU, CNN-LSTM, and Transformer, can learn more complex time dependencies and nonlinear patterns, and have made significant progress in the field of load forecasting.

[0004] Existing forecasting methods suffer from the following problems in practical applications: Electricity load is influenced by multiple factors, including weekday / weekend cycles, seasonal variations, and patterns of social activities, exhibiting multi-scale periodic characteristics such as daily, weekly, and seasonal trends, accompanied by random disturbances and noise fluctuations. Traditional statistical models (such as autoregressive models and exponential smoothing) and shallow machine learning methods in existing technologies struggle to simultaneously and effectively capture both short-term local fluctuations and long-term dependency structures, thus limiting the improvement of forecast accuracy and robustness. There is a significant correlation between load and meteorological variables (especially temperature). In practical engineering, multiple meteorological observation stations are typically deployed in the same area, and the temporal distribution of temperature data at each station differs spatially. Existing methods often use single-station temperature data or perform simple averaging of multiple station temperatures, lacking an adaptive modeling mechanism to account for the differences in contributions from different stations, and failing to fully exploit the potential of multi-source meteorological information to improve load forecast accuracy. During the actual load data collection process, due to factors such as communication failures, metering equipment malfunctions, and data management defects, continuous data gaps often occur for several days or even a week. Although traditional linear interpolation or spline interpolation methods can fill the numerical gaps, they are prone to introducing over-smoothing effects, which significantly weaken the inherent daily periodicity and peak-valley characteristics of the load sequence, affecting the training quality and generalization performance of subsequent prediction models. Summary of the Invention

[0005] The purpose of this invention is to provide a short-term load forecasting method based on an LSTM-Transformer hybrid model, in order to solve the problems mentioned in the background art, such as the difficulty in achieving adaptive fusion of multi-site temperature information, joint modeling of short-term and long-term time-series dependencies, and effective model-driven reconstruction of data missing over long periods within a unified framework.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: According to one aspect of the present invention, a short-term load forecasting method using an LSTM-Transformer hybrid model is provided, the method comprising: Historical load sequences and temperature sequences from multiple meteorological stations in the target area are obtained, time-aligned and windowed, and input-output sample pairs are constructed. By using a multi-site temperature attention mechanism, the temperatures of multiple meteorological stations at each time step are weighted and fused to obtain an equivalent temperature, which is then concatenated with the corresponding load value to form a fused feature sequence. The fused feature sequence is input into a hybrid time series model consisting of an LSTM network and a Transformer encoder, wherein the LSTM network extracts short-term dynamic features and the Transformer encoder models long-term dependencies to output future multi-step load prediction results. For the continuous missing segments in the historical load sequence, the trained hybrid time series model is used to perform autoregressive recursive reconstruction in a sliding window manner to generate a complete load sequence. During the training phase, the hybrid time series model is optimized using a joint loss function that includes prediction loss, reconstruction loss, and collaborative regularization loss.

[0007] Based on the aforementioned scheme, the construction of input-output sample pairs includes: defining the historical window length as H and the prediction step size as K; For a time step t that meets the conditions, an input-output sample pair is constructed, where the input includes: a historical load subsequence consisting of H consecutive load values ​​from time tH to time t-1, and a historical temperature subsequence consisting of temperature observations from M meteorological stations aligned within the same time window; The output consists of K consecutive true load values ​​starting from time t.

[0008] Based on the aforementioned scheme, obtaining the equivalent temperature includes: For the current time step, acquire the temperature observation values ​​of the multiple meteorological stations to form a temperature observation vector; The temperature observations at each station are mapped to a latent representation space using trainable parameters, and the raw attention score for each station is calculated. The original attention scores of all stations are normalized to obtain the attention weight of each station at the current time step. The equivalent temperature at the current time step is obtained by weighting and summing the temperature observations of the multiple meteorological stations according to the attention weights.

[0009] Based on the aforementioned scheme, the formula for calculating the original attention score is as follows: ; in, For the m-th meteorological station at time step t The original attention score, Let b be the corresponding temperature observation value, W be the trainable weight matrix, b be the trainable bias vector, and w be the trainable weight vector.

[0010] Based on the aforementioned scheme, the LSTM network and the Transformer encoder work together in a series manner; the LSTM network, as a lightweight small model, outputs a hidden state sequence as the input of the Transformer encoder; the Transformer encoder, as a large model with a larger parameter scale, further encodes the hidden state sequence.

[0011] Based on the aforementioned scheme, the collaborative regularization loss is used to constrain the consistency between the features output by the LSTM network and the features of the intermediate layer of the Transformer encoder.

[0012] Based on the aforementioned scheme, the autoregressive recursive reconstruction using a sliding window method includes: Identify consecutive missing intervals in the historical load sequence; For the current time point t to be reconstructed within the missing interval, perform the following iterative steps: Construct a historical input window ending at time t-1. The load sequence in the historical input window includes known true load values, and for missing time points, if they have been reconstructed, use the reconstructed values; otherwise, use the initial filler values. The load sequence from the historical input window and the corresponding multi-site temperature sequence are input into the trained hybrid time series model to obtain the load prediction sequence starting from time t. The first value of the load prediction sequence is taken as the reconstructed load value at the current time point t; The iterative steps are executed sequentially by sliding the window until a reconstruction load value is obtained for all time points within the missing interval.

[0013] Based on the aforementioned scheme, after the autoregressive recursive reconstruction, the reconstructed load sequence is further smoothed and periodic consistency is verified.

[0014] Based on the aforementioned scheme, the joint loss function is expressed as: ; in, To predict losses, To reconstruct the loss, To coordinate regularization loss, for The weighting coefficients of the reconstruction loss. The weighting coefficients for the collaborative regularization loss.

[0015] Based on the aforementioned scheme, during the training phase, regularization constraints are applied to the attention weights calculated by the multi-site temperature attention mechanism.

[0016] As can be seen from the above technical solutions, this invention has at least the following advantages and positive effects compared with existing technologies: By introducing a multi-site temperature attention mechanism, the model can adaptively learn and weightedly fuse the influence of meteorological stations in different geographical locations on the load of the target area, extracting more accurate meteorological driving features; combining the precise capture of short-term local dynamics by LSTM and the powerful modeling capability of Transformer for long-range periodic dependence, it achieves deep fusion and accurate prediction of multi-scale features of load sequences, significantly improving the overall accuracy of short-term load prediction. A data reconstruction strategy based on autoregressive recursion using a trained deep model is proposed. For long missing segments in historical data, the model can utilize the learned load evolution patterns and meteorological correlations to generate reconstructed sequences that maintain temporal continuity and conform to load cycle characteristics. This greatly reduces the dependence on incomplete historical data and improves the applicability and prediction stability of the model in real, incomplete data scenarios. By designing a collaborative hybrid architecture of a small LSTM model and a large Transformer model, and introducing collaborative regularization during training, knowledge transfer and functional layering within the model are achieved. In practical applications, a lightweight small model can be flexibly selected for fast online prediction, or a high-capacity large model can be called for offline fine analysis, thus effectively balancing computational overhead and deployment flexibility while ensuring core prediction performance.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings: Figure 1 This is a schematic diagram of a short-term load forecasting method using an LSTM-Transformer hybrid model according to the present invention. Figure 2 This is a flowchart of a short-term load forecasting method using an LSTM-Transformer hybrid model according to the present invention. Detailed Implementation

[0019] To more clearly illustrate the purpose, technical solutions, and advantages of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein. On the contrary, these embodiments are provided so that the present invention will be more comprehensive and complete, and fully convey the concept of the exemplary embodiments to those skilled in the art.

[0020] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a full understanding of embodiments of the invention. However, those skilled in the art will recognize that the technical solutions of the invention can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of the invention.

[0021] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0022] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0023] The present invention will now be described in detail with reference to specific embodiments.

[0024] Example 1

[0025] like Figure 1 , 2 As shown in the figure, this embodiment provides a short-term load forecasting method based on an LSTM-Transformer hybrid model. The specific steps of this method are as follows: S1: Obtain the historical load sequence of the target area and the temperature sequence of multiple meteorological stations, perform time alignment and windowing processing, and construct input-output sample pairs.

[0026] The target power system acquires load sequence data over a historical period, typically collected at fixed time intervals (e.g., 15 minutes, 1 hour) to form a historical power load sequence for the target area. Simultaneously, acquire temperature observation sequences from M meteorological stations related to the target area, denoted as... To ensure time consistency, the load series and all temperature series need to be time-aligned. For data with timestamp discrepancies or inconsistent sampling frequencies, interpolation or resampling methods are used to unify them onto the same time grid, forming a strictly aligned multi-source time series.

[0027] A sliding window method is used to construct input-output sample pairs; the historical window length is defined as H, and the prediction step size as K. For a time step t that meets the conditions... The training samples are constructed, and the input of the samples consists of two parts: a historical load subsequence consisting of H consecutive load values ​​from time tH to time t-1. And a historical temperature subsequence consisting of temperature observations from M meteorological stations aligned within the same window. ,in T τ ( τ ∈[ tH , t-1 ])for M A dimensional vector containing all stations at time [time]. t The temperature; the output of this sample (i.e., the predicted target) is the true load value for K consecutive future time steps starting from time t. Samples are generated using a sliding time starting point t to form the dataset for model training and evaluation.

[0028] Furthermore, this embodiment formalizes the short-term load forecasting task as a regression problem from multivariate time series to univariate time series. Specifically, a forecasting function is defined. f θ Its parameter is θ; mapped through the prediction function: ; This function uses load data from H historical steps (i.e., tH to t-1) prior to the target prediction start time t. With corresponding multi-site temperature data As input, the output is a load forecast sequence for the next K time points starting at time t.

[0029] S2: Through a multi-site temperature attention mechanism, the temperatures of multiple meteorological stations at each time step are weighted and fused to obtain the equivalent temperature, which is then concatenated with the corresponding load value to form a feature vector sequence.

[0030] For any time step aligned by step S1 t Extract the temperature observation values ​​from all M meteorological stations at that moment and construct a temperature observation vector: .

[0031] To evaluate the importance of each station's temperature to load forecasting at a specific time step, a learnable attention scoring network is introduced. Each station's temperature is mapped to a high-dimensional representation space through a shared linear transformation layer to capture its latent features. Specifically, for the m-th meteorological station at time step... t Temperature observations ,calculate: ; in, Let W represent the temperature of the site in the latent feature space learned by the model, and let W and b be trainable weight matrices and bias vectors. A nonlinear activation function (such as the hyperbolic tangent function tanh) is applied to the transformed features, and the original attention score for the site is calculated using the weight vectors. ; That The original score is a trainable weight vector. This reflects the initial importance of station m at time t. The original attention scores of all M stations are normalized using the softmax function to obtain the final attention weight of station m at time t: ; in, The weight distribution is dynamic and adaptively adjusts as time t changes. This is based on the attention weights. The equivalent temperature at that moment is generated by weighted summation of the original multi-site temperature observations. : ; The original load value and the equivalent temperature are concatenated to form a fused feature vector for each time step: ; By iterating through each time step t within the historical window, a fused feature sequence can be constructed. As input to subsequent time series models, it provides multi-source information fusion results for subsequent time series modeling.

[0032] S3: Input the fused feature sequence into a hybrid time series model consisting of an LSTM network and a Transformer encoder, wherein the LSTM network extracts short-term dynamic features and the Transformer encoder models long-term dependencies to output future multi-step load prediction results.

[0033] The fusion feature sequence obtained in step S2 The data is input sequentially into an LSTM network. This LSTM network, as a small model, has a small parameter size and captures the local temporal dependencies and short-term dynamics of the sequence. For each time step τ (τ=tH,t-H+1,…,t-1) in the sequence, the LSTM unit calculates the current input feature x. τ And the hidden state h from the previous time step τ-1 The current hidden state is updated by calculation through a gating mechanism (input gate, forget gate, output gate). The process is represented as: ; After traversing the entire input sequence, the hidden state sequence is obtained. This sequence encodes the local temporal patterns and short-term fluctuations of load and equivalent temperature within a sliding window.

[0034] Stack the hidden state sequences output by the LSTM into a matrix form. The input sequence is then fed into the Transformer encoder, a larger model with a larger parameter scale, used to characterize long-range dependencies and multi-scale periodicity across time steps. Internally, the Transformer encoder calculates the correlation between different time steps using a multi-head self-attention mechanism. First, the input sequence is projected through h different linear projections to obtain h sets of query, key, and value matrices. Scaling dot product attention is then performed in parallel on these h sets of matrices, i.e., for each set of matrices, the following is calculated: ; The query matrix Q, key matrix K, and value matrix V are all obtained by linear projection of the hidden state matrix H. k Let be the dimension of the key vector; the outputs of h attention heads are obtained, and the outputs are concatenated to capture dependencies from different subspaces. After several layers of encoding, a high-order temporal feature representation that integrates short-term dynamics and long-range dependencies is obtained. .

[0035] Contextual information for prediction is extracted from the encoded temporal features U. In this embodiment, the representation vector u of the last time step is selected. t As a comprehensive summary of historical information, this vector is input into a linear projection layer and mapped to load forecasts for the next K time steps: ; in, and Given a trainable weight matrix and bias vector; the output is... This is the model's load prediction sequence for the next K steps.

[0036] By combining LSTM, which captures local dynamics, with Transformer, which models global dependencies, a hierarchical temporal feature extractor is constructed. This hybrid architecture can simultaneously capture complex short-term fluctuations and long-term patterns in load sequences, thereby improving the accuracy and robustness of multi-step forward prediction.

[0037] S4: For the continuous missing segments in the historical load sequence, the trained hybrid time series model is used to perform autoregressive recursive reconstruction in a sliding window manner to generate a complete load sequence.

[0038] For new historical data that needs to be predicted but contains long missing segments, the trained model is invoked. f θ The missing portions are reconstructed using an autoregressive approach to generate a complete, high-quality historical workload sequence. The original historical workload sequence is scanned, and all consecutive missing data intervals are identified based on timestamp continuity and preset missing data markers. Specifically, the timestamp difference between adjacent data points in the sequence is calculated; if the difference exceeds a given threshold for the normal sampling interval, data is considered missing between these two timestamps. Based on preset missing data markers, specific values ​​(such as NaN, NULL, -999, etc.) or flags are often used to mark invalid or missing data during data acquisition or preprocessing; all data points with such markers are detected in the sequence. All identified missing points are sorted and merged in chronological order, and consecutive missing points are grouped into missing intervals. For each identified missing interval, record the start time t. s End time t e And the length, used for boundary control during subsequent rolling reconstruction. To initiate subsequent autoregressive reconstruction, an initial load value is provided for the beginning portion of each missing interval to form a complete historical input window; for the starting time t of the missing interval... s The required history window [t] s -H,t sThe value within [-1] may contain both true and missing values. For missing values, simple interpolation, mean imputation, or estimation based on similar historical curves can be used for initial imputation (temporary imputation). This initial imputation value is only used to ensure that there is a complete input sequence when the model is first called and will be overwritten in subsequent steps.

[0039] Furthermore, a pre-trained hybrid LSTM-Transformer model is used. The missing intervals are reconstructed using an autoregressive method with a sliding window approach. For the missing intervals... From the start time t s Begin by reconstructing each missing point sequentially in chronological order. For the current point in time to be reconstructed... : Construct the data within the historical input window [tH, t-1], including the load sequence and temperature sequence. The load sequence includes the originally known true load values ​​within the window, and for missing time points within the window, if they have been reconstructed, the reconstructed values ​​are used; otherwise, the initial filler values ​​are used. The temperature sequence is the corresponding multi-site temperature sequence within this window. The constructed mixed historical load series and temperature series are input into the model. Perform one forward propagation computation: ; Obtain the load forecast sequence for the next K steps starting from time t. Extract the first predicted value from the predicted sequence. This is used as the final reconstructed load value at the current time point t, and updated in the historical load sequence, overwriting the initial fill value. The process is repeated sequentially by sliding the window (to the next missing time point t+1), and the historical window will then contain the reconstructed value just obtained. The above process is repeated until all time points within the missing interval are filled with reconstructed values. Through this recursive approach, the model utilizes its learned temporal dynamics to gradually propagate reconstructed information from the known data boundaries into the missing interval, generating a coherent load sequence that conforms to the learning pattern.

[0040] The initially reconstructed load sequence undergoes smoothing and consistency verification. Lightweight time smoothing methods, such as moving average filtering or median filtering, are applied to the reconstructed sequence to suppress unnatural fluctuations or isolated outliers that may be caused by accumulated recursive errors, resulting in a smoother overall curve. Periodic consistency verification is performed on the reconstructed sequence at daily or weekly scales, such as comparing the shape of the daily load curves of adjacent workdays to see if they are reasonable and whether the peak and trough positions are significantly misaligned. If significant local discrepancies are found, fine-tuning can be made, such as by locally aligning it with curves of similar historical days or using weighted adjustment methods.

[0041] S5: During the training phase, the hybrid time series model is optimized using a joint loss function that includes prediction loss, reconstruction loss, and collaborative regularization loss.

[0042] During the training phase, this embodiment uses a joint loss function to optimize the model parameters. This loss function consists of prediction loss, reconstruction loss, and size-model collaborative regularization loss. The prediction loss measures the accuracy of the model's prediction of future load sequences and is calculated using the mean absolute error (L1 loss) between the actual load values ​​and the corresponding predicted values ​​over the next K time steps. ; in, This represents the actual load value at the i-th step in the future. This is the corresponding predicted value output by the model.

[0043] For samples involving missing segment reconstruction, an auxiliary reconstruction loss is introduced. This term, used to constrain the difference between the model's output and the weak label (such as an initial estimate based on simple interpolation) during the missing time period, can be expressed as: ; in, For the set of missing time points constructed from the training samples, This represents the weak label load value at the corresponding time point.

[0044] To achieve knowledge transfer and output consistency between the LSTM small model and the Transformer large model, a collaborative regularization loss is introduced. This constraint is used to constrain the consistency between the features output by the LSTM network and the features of the intermediate layers of the Transformer encoder. This constraint can be applied to the final output or intermediate feature layers of the model, such as using mean squared error (MSE) or cosine similarity to constrain the difference between the corresponding outputs / features of the large and small models; it can be expressed as: ; in, and These are the prediction outputs of the large Transformer model and the small LSTM model, respectively.

[0045] The total loss is expressed as: ; in and The non-negative weighting coefficients are used to balance the importance of the reconstruction task and model synergy. The model parameters θ are updated via backpropagation by minimizing the total loss using a gradient descent algorithm (such as the Adam optimizer). After training, the model can reconstruct high-quality historical load data with high accuracy while maintaining short-term load prediction accuracy.

[0046] Furthermore, adding regularization and normalization operations such as Dropout and LayerNorm between the Transformer encoder, LSTM layer, and output layer helps improve the model's robustness to load temporal noise and extreme weather conditions. Dropout prevents overfitting by randomly discarding some neuron outputs during training; LayerNorm stabilizes the training process and accelerates convergence by normalizing intra-layer features. Regarding the multi-site temperature attention weights in step S2... Additional distribution regularization can be applied, such as entropy regularization or L2 regularization, so that the attention distribution is neither overly concentrated on a single station nor overly uniform, which is beneficial to improving the efficiency of meteorological information utilization and model generalization performance.

[0047] It should be noted that this embodiment includes two logical stages: the model training stage and the data reconstruction and application stage. In the model training stage, steps S1 to S3 are first executed on a relatively complete dataset, and the joint loss from step S5 is used for optimization to obtain the trained hybrid time-series model f. θ Subsequently, during the data reconstruction and application phase, for any new data with long missing segments, the pre-trained model f can be invoked. θ Step S4 is executed to reconstruct the missing data, and then the reconstructed complete sequence is used again through model f. θ (i.e., the forward calculation process in step S3) yields the final load forecast result.

[0048] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the claims. It should be understood that the invention is not limited to the precise structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A short-term load forecasting method using an LSTM-Transformer hybrid model, characterized in that, The method includes: Historical load sequences and temperature sequences from multiple meteorological stations in the target area are obtained, time-aligned and windowed, and input-output sample pairs are constructed. By using a multi-site temperature attention mechanism, the temperatures of multiple meteorological stations at each time step are weighted and fused to obtain an equivalent temperature, which is then concatenated with the corresponding load value to form a fused feature sequence. The fused feature sequence is input into a hybrid time series model consisting of an LSTM network and a Transformer encoder, wherein the LSTM network extracts short-term dynamic features and the Transformer encoder models long-term dependencies to output future multi-step load prediction results. For the continuous missing segments in the historical load sequence, the trained hybrid time series model is used to perform autoregressive recursive reconstruction in a sliding window manner to generate a complete load sequence. During the training phase, the hybrid time series model is optimized using a joint loss function that includes prediction loss, reconstruction loss, and collaborative regularization loss.

2. The short-term load forecasting method using an LSTM-Transformer hybrid model according to claim 1, characterized in that, The construction of input-output sample pairs includes: defining the historical window length as H and the prediction step size as K; For a time step t that meets the conditions, an input-output sample pair is constructed, where the input includes: a historical load subsequence consisting of H consecutive load values ​​from time tH to time t-1, and a historical temperature subsequence consisting of temperature observations from M meteorological stations aligned within the same time window; The output consists of K consecutive true load values ​​starting from time t.

3. The short-term load forecasting method using an LSTM-Transformer hybrid model according to claim 1, characterized in that, The obtained equivalent temperature includes: For the current time step, acquire the temperature observation values ​​of the multiple meteorological stations to form a temperature observation vector; The temperature observations at each station are mapped to a latent representation space using trainable parameters, and the raw attention score for each station is calculated. The original attention scores of all stations are normalized to obtain the attention weight of each station at the current time step. The equivalent temperature at the current time step is obtained by weighting and summing the temperature observations of the multiple meteorological stations according to the attention weights.

4. The short-term load forecasting method using an LSTM-Transformer hybrid model according to claim 3, characterized in that, The formula for calculating the original attention score is as follows: ； in, For the m-th meteorological station at time step t The original attention score, Let b be the corresponding temperature observation value, W be the trainable weight matrix, b be the trainable bias vector, and w be the trainable weight vector.

5. The short-term load forecasting method using an LSTM-Transformer hybrid model according to claim 1, characterized in that, The LSTM network and the Transformer encoder work together in series; the LSTM network, as a lightweight small model, outputs a hidden state sequence as the input of the Transformer encoder; the Transformer encoder, as a large model with a larger parameter scale, further encodes the hidden state sequence.

6. The short-term load forecasting method of the LSTM-Transformer hybrid model according to claim 1, characterized in that, The collaborative regularization loss is used to constrain the consistency between the features output by the LSTM network and the features of the intermediate layers of the Transformer encoder.

7. The short-term load forecasting method of the LSTM-Transformer hybrid model according to claim 1, characterized in that, The autoregressive recursive reconstruction using a sliding window method includes: Identify consecutive missing intervals in the historical load sequence; For the current time point t to be reconstructed within the missing interval, perform the following iterative steps: Construct a historical input window ending at time t-1. The load sequence in the historical input window includes known true load values, and for missing time points, if they have been reconstructed, use the reconstructed values; otherwise, use the initial filler values. The load sequence from the historical input window and the corresponding multi-site temperature sequence are input into the trained hybrid time series model to obtain the load prediction sequence starting from time t. The first value of the load prediction sequence is taken as the reconstructed load value at the current time point t; The iterative steps are executed sequentially by sliding the window until a reconstruction load value is obtained for all time points within the missing interval.

8. The short-term load forecasting method of the LSTM-Transformer hybrid model according to claim 7, characterized in that, After the autoregressive recursive reconstruction, the reconstructed load sequence is also smoothed and periodic consistency is checked.

9. A short-term load forecasting method using an LSTM-Transformer hybrid model according to claim 1, characterized in that, The joint loss function is expressed as: ； in, To predict losses, To reconstruct the loss, To coordinate regularization loss, The weighting coefficients for reconstructing the loss. The weighting coefficients for the collaborative regularization loss.

10. A short-term load forecasting method using an LSTM-Transformer hybrid model according to claim 1, characterized in that, During the training phase, regularization constraints are applied to the attention weights calculated by the multi-site temperature attention mechanism.

Images