A power load prediction method and system fusing multi-modal decomposition and transformer-bilstm

Abstract

The application relates to a power load prediction method and system combining multi-modal decomposition and a Transformer-BiLSTM, and belongs to the technical field of load prediction. An improved adaptive noise complete ensemble empirical mode decomposition algorithm and a sample entropy combination data decomposition method are used. The method firstly uses the algorithm to preliminarily decompose a load sequence to generate a group of intrinsic mode functions, thereby extracting effective signals and removing irrelevant noises, effectively separating the stationarity and non-stationarity of the load sequence, and reducing the complexity of the load data; subsequently, sample entropy is introduced to quantitatively evaluate the time sequence complexity of each IMF component, and components with similar complexity are aggregated and reconstructed, so that the number of components is reasonably reduced, thereby reducing the prediction modeling complexity and the solution workload.

Description

Technical Field

[0001] This invention relates to a power load forecasting method and system that integrates multimodal decomposition and Transformer-BiLSTM, belonging to the field of load forecasting technology. Background Technology

[0002] As my country's power market reform continues to deepen and the market-based trading mechanism is constantly improved, the electricity sales side is gradually developing towards diversification, refinement, and intelligence. Against this backdrop, user-level electricity load forecasting, as a key technical support connecting the supply and demand sides of electricity, is becoming increasingly important.

[0003] Compared to regional or system-level load forecasting, user-level load has stronger randomness, volatility, and individual heterogeneity: on the one hand, it is affected by multiple factors such as user behavior habits and weather conditions, resulting in complex and varied electricity consumption patterns; on the other hand, different types of users, such as industrial and commercial users, have significant differences in load patterns and response characteristics, which further increases the difficulty of modeling and forecasting.

[0004] However, accurate user-level load forecasting is crucial for enhancing the decision-making capabilities of electricity retailers in medium- and long-term contract formulation, day-ahead market bidding, and real-time balancing strategies. Therefore, constructing highly robust and adaptable accurate user-level load forecasting models to address complex and ever-changing user electricity consumption behaviors has become a key technological requirement for improving the core competitiveness of electricity retailers and ensuring the safe and economical operation of the power system in the current electricity market environment. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a power load forecasting method and system that integrates multimodal decomposition and Transformer-BiLSTM. By performing multimodal decomposition on the original load data to obtain each IMF component, and then inputting it into the Transformer-BiLSTM hybrid model to mine time-series features and long-range dependencies, the accuracy and stability of load forecasting are effectively improved, providing a reliable basis for electricity sales decisions.

[0006] The technical solution of the present invention is as follows: A power load forecasting method integrating multimodal decomposition and Transformer-BiLSTM, comprising the following steps: (1) Standardize and preprocess the raw power load and multi-dimensional meteorological data to eliminate the dimensional differences between the multi-source features; (2) An improved Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (ICEEMDAN) algorithm is used to perform adaptive mode decomposition on the preprocessed load sequence, generating a set of Intrinsic Mode Functions (IMFs), thereby extracting effective signals and removing irrelevant noise, effectively separating the stationarity and non-stationarity of the load sequence, and reducing the complexity of the load data; (3) Introduce sample entropy to evaluate the time series complexity of each intrinsic mode function component obtained by decomposition, and aggregate and reconstruct intrinsic mode function components with similar complexity to form several subsequences, thereby reasonably reducing the number of components, thereby reducing the complexity of prediction modeling and the workload of solving. (4) Construct a hybrid prediction architecture that integrates the Transformer model and the bidirectional long short-term memory network, model the aggregated subsequences separately, and generate the final load prediction result by superimposing the prediction results of each subsequence.

[0007] According to a preferred embodiment of the present invention, in step (1), the meteorological data includes perceived temperature, relative humidity, shortwave radiation, 2m temperature, and 10m wind speed, and the preprocessing process is as follows: The min-max normalization method is used to map the original power load and meteorological data into the interval [0,1] to improve the convergence stability of model training. At the same time, outliers are removed and missing data is filled by interpolation to ensure data continuity and consistency. After normalization, a one-dimensional time series containing only load values ​​is extracted separately as the load sequence required for mode decomposition. The normalized meteorological features are retained for use as external input for subsequent prediction models.

[0008] According to a preferred embodiment of the present invention, in step (2), the decomposition steps of the improved adaptive noise complete set empirical mode decomposition algorithm are as follows: (21) Introduce Gaussian white noise into the input signal to generate a noisy signal: (1) In the formula, x represents the normalized load sequence; This represents the noise standard deviation set during the first decomposition. It is the first eigenmode function component obtained through empirical mode decomposition; For the added first k Gaussian white noise; (22) Calculate the residuals obtained after the first decomposition. : (2) In the formula, The local mean function represents the signal and is used to characterize the average trend of the signal within a local region. The signal represents the first k One sampling point; n This represents the total number of sampling points; (23) Solve for the first eigenmode function component. : (3) (24) Calculate the residual of the second decomposition. and the second : (4) (5) (25) Perform iterative decomposition, when the... i The residual obtained after the second decomposition If the number of extreme points is less than 3, or its variance is lower than a preset threshold, the residual is determined to have no further physical meaning for decomposition, and the iteration automatically terminates. At this point, the 3rd iteration... i individual residuals and the i The formula for calculating each IMF component is as follows: (6) (7).

[0009] According to a preferred embodiment of the present invention, in step (3), the smaller the sample entropy value, the stronger the self-similarity of the sequence; the larger the entropy value, the higher the randomness and complexity of the sequence.

[0010] According to a preferred embodiment of the present invention, in step (3), for the product of N A time series composed of data The calculation process for sample entropy is as follows: (31) Reconstruct the time series sequentially into a set of vector sequences with an embedding dimension of m. ,in, These vectors represent the vectors from the first... i Starting from each sampling point, continuously m An embedding vector consisting of 10 data points; (32) Define two embedding vectors and The distance between them is the maximum absolute difference between them on their respective intrinsic mode function components, that is: (8) (33) For a given embedding vector Statistical satisfaction and vector The quantity (of which) ), denoted as ,definition: (9) Based on this, define : (10) (34) Increase the embedding dimension to m +1, construct a new sequence of embedding vectors. and For each i , Statistical satisfaction and The number of vectors, denoted as , Defined as: (11) Based on this, define : (12) therefore, Indicates similarity tolerance r Below, continuous in two embedding vector sequences m The probability of a point matching; and This corresponds to continuous under the same tolerance. m The probability of a +1 point match is given by the sample entropy, which is defined as: (13) when N When the value is finite, the following formula is used for estimation: (14) After completing the adaptive decomposition of the original load sequence and the aggregation of intrinsic mode function components based on sample entropy, the resulting reconstructed subsequence not only retains the multi-scale dynamic characteristics of the original signal, but also significantly reduces modeling redundancy.

[0011] According to a preferred embodiment of the present invention, in step (4), in order to fully explore the spatiotemporal dependencies in each subsequence and achieve high-precision prediction, a hybrid prediction model integrating bidirectional long short-term memory network and Transformer architecture is constructed. This model combines the ability of BiLSTM to finely characterize local time series patterns with the adaptive modeling advantage of Transformer for global long-term dependencies, thereby providing a robust and accurate prediction framework for complex power load sequences.

[0012] The Transformer model consists of an encoder and a decoder, both of which adopt a multi-layer stacked structure. Each layer contains core components such as a multi-head self-attention mechanism, a feedforward neural network, and residual connections. The multi-head self-attention mechanism maps the input sequence to multiple different subspaces through multiple sets of parallel learnable linear projections. Each set independently calculates attention weights, enabling the model to simultaneously focus on feature information at different positions and semantic levels in the sequence. Compared with single-head attention, the multi-head mechanism effectively enhances the model's ability to comprehensively perceive the global context of the time series from multiple perspectives, thereby improving the representation accuracy and generalization performance of the prediction task. The specific calculation process is as follows: Let the input sequence be X, and calculate the query matrix Q, the key matrix K, and the value matrix V using three sets of learnable weight matrices: (15) (16) (17) In the formula, , , These are the weight matrices corresponding to the query matrix, key matrix, and value matrix, respectively. The sample vector in the input sequence; By interactively matching the representation matrices of each variable with their corresponding matrices and those of other variables, different linear mappings are applied to the input features. Subsequently, the attention weights corresponding to each variable are calculated based on a scaled dot product multi-head self-attention mechanism, the mathematical expression of which is as follows: (18) In the formula, This represents the score for each sample in the vector; The dimension of the key vector; By introducing a multi-head self-attention mechanism, the model simultaneously learns different representation patterns of the input sequence in multiple parallel self-attention subspaces, thereby capturing diverse dependencies. The computational process is as follows: (19) (20) In the formula, The aggregation function representing each attention head; The parameter matrix represents the linear variation of multi-head attention. Each self-attention layer is followed by a feedforward neural network, whose structure consists of fully connected layers. This network performs a non-linear mapping on the feature representations of each time step or sequence position, calculated using the following formula: (twenty one) In the formula, , These are the weight matrices for the first and second fully connected layers, respectively. Both are automatically learned and updated during model training using backpropagation and gradient descent optimizers, requiring no manual setting. The difference lies in: Transform input features from dimensionality Mapped to a higher-dimensional intermediate space , Then the high-dimensional features are mapped back to the original dimension to ensure that the dimension is consistent when added with the residual connection; , All are bias terms.

[0013] Residual connections and normalization layers play a crucial role in model training. By introducing residual paths, they alleviate the gradient vanishing problem, while normalization operations stabilize the distribution of intermediate features, thereby improving the convergence and stability of model training. Their calculation formulas are as follows: (twenty two) (twenty three) In the formula, This represents the intermediate features after processing by a self-attention layer or a feedforward network. The final output of the Transformer model is generated after residual connection and normalization operations, and can be used as input for subsequent layers.

[0014] According to a preferred embodiment of the present invention, in step (4), the Bi-directional Long Short-Term Memory (BiLSTM) network is an optimization of the LSTM model. Its hidden layer structure integrates both forward LSTM and backward LSTM. Therefore, the Bi-directional Long Short-Term Memory network can process data in the normal order of the time series through the forward LSTM and in the reverse order of the time series through the backward LSTM. The outputs of both directions are connected to the same output layer, jointly determining the hidden layer values, thereby generating the final output. ; (30) (31) (32) In the formula, , The outputs are the forward LSTM and the backward LSTM, respectively. , These are the weight matrices from the forward LSTM and backward LSTM to the output layer, respectively. This represents the weight matrix of the output layer.

[0015] According to a preferred embodiment of the present invention, in step (4), the hybrid prediction architecture training process involves constructing a temporal morphology-aware composite loss function as the optimization objective for model training. This loss function has a mean squared error term. Based on this, slope difference penalty terms and curvature difference penalty terms are introduced, and the specific calculation formulas are as follows: (33) in: (34) Slope loss term Used to constrain the consistency of the rate of change between the predicted curve and the actual curve at adjacent time points: (35) Curvature loss term Used to constrain the trend of the predicted curve and the true curve on the second derivative: (36) In the formula, and These are the weighting coefficients for slope loss and curvature loss, respectively; The actual value; For the predicted values, during training, the model minimizes the temporal form of the composite loss function. Optimize parameters for the target.

[0016] A power load forecasting system integrating multimodal decomposition and Transformer-BiLSTM includes: The data preprocessing module is used to preprocess the raw power load and multi-dimensional meteorological data; The mode decomposition module uses an improved adaptive noise complete set empirical mode decomposition algorithm to adaptively decompose the preprocessed load sequence and generate a set of intrinsic mode functions. The component aggregation and reconstruction module is used to introduce sample entropy to perform time series complexity quantification evaluation on each intrinsic mode function component obtained from decomposition, and to aggregate and reconstruct intrinsic mode function components with similar complexity to form several subsequences; The load forecasting module constructs a hybrid forecasting architecture that integrates the Transformer model and the bidirectional long short-term memory network. It models the aggregated subsequences separately and generates the final load forecasting result by superimposing the forecasting results of each subsequence.

[0017] The beneficial effects of this invention are as follows: 1. This invention employs an improved adaptive noise complete set empirical mode decomposition algorithm to adaptively decompose the original load sequence, and introduces sample entropy to quantify the complexity of each component. Based on the entropy proximity, similar components are aggregated and reconstructed. This strategy effectively avoids the computational redundancy caused by repeatedly modeling highly similar components. While preserving the key multi-scale features of the load signal, it significantly reduces the model complexity, thereby improving modeling efficiency.

[0018] 2. This invention proposes a synergistic improvement strategy at the load forecasting model level: First, it constructs a hybrid architecture combining a Transformer and a bidirectional long short-term memory network, and designs a temporal morphology-aware composite loss function that incorporates slope and curvature constraints. This overcomes the shortcomings of single models in balancing global interaction and local morphological fitting, and the tendency of traditional point-by-point loss to lead to prediction phase lag and shape distortion. Second, it designs an automatic generation strategy for causal multi-scale lag features to avoid information leakage, and uses an independent calibration set to optimize the number of training epochs to enhance generalization ability. The synergistic effect of these improvements significantly enhances the model's morphological response and prediction stability during abrupt load changes such as morning and evening peak periods. Core error metrics such as RMSE and MAE on the validation set are significantly better than those of a single benchmark model. Attached Figure Description

[0019] Figure 1 This is a basic structural diagram of the Transformer model in an embodiment of the present invention; Figure 2 This is a cell structure diagram of the LSTM in an embodiment of the present invention; Figure 3 This is a unit structure diagram of BiLSTM in an embodiment of the present invention; Figure 4 This is a flowchart illustrating the overall prediction process in an embodiment of the present invention. Figure 5 This is the load data decomposition result of an industrial company in an embodiment of the present invention; Figure 6 This is the result of decomposing the load data of a hotel in an embodiment of the present invention; Figure 7 The entropy values ​​of each IMF component of the hotel load data in this embodiment of the invention; Figure 8 The entropy values ​​of each IMF component of the load data of an industrial company in this embodiment of the invention; Figure 9 This invention provides a comparison of the load forecast results for a hotel on any two days in an embodiment of the invention. Figure 9 (a) shows a comparison of load forecast results for a certain day in January. Figure 9 (b) shows a comparison of load forecast results for a certain day in February; Figure 10This invention provides a comparison of load forecast results for any two days for an industrial company. Figure 10 (a) shows a comparison of load forecast results for a certain day in January. Figure 10 (b) shows a comparison of load forecast results for a certain day in February; Figure 11 This is a radar chart of the evaluation indicators of various models for load forecasting of a hotel in an embodiment of the present invention, wherein, Figure 11 (a) is a radar chart of the mean absolute error (MAE) of each load forecasting model. Figure 11 (b) is a radar chart showing the mean square error (MSE) of each load forecasting model. Figure 11 Radar chart of the root mean square error (RMSE) index of each load forecasting model; Figure 12 This is a radar chart of various evaluation indicators for load forecasting models of an industrial company in an embodiment of the present invention. Figure 12 (a) is a radar chart of the mean absolute error (MAE) of each load forecasting model. Figure 12 (b) is a radar chart showing the mean square error (MSE) of each load forecasting model. Figure 12 Radar chart of the root mean square error (RMSE) index for each load forecasting model (c). Detailed Implementation

[0020] The present invention will be further described below with reference to the embodiments and accompanying drawings, but is not limited thereto.

[0021] Example 1: This embodiment provides a power load forecasting method that integrates multimodal decomposition and Transformer-BiLSTM, with the following steps: (1) Standardize and preprocess the raw power load and multi-dimensional meteorological data to eliminate the dimensional differences between the multi-source features; Meteorological data includes perceived temperature, relative humidity, shortwave radiation, 2m temperature, and 10m wind speed. The preprocessing procedure is as follows: The min-max normalization method is used to map the original power load and meteorological data into the interval [0,1] to improve the convergence stability of model training. At the same time, outliers are removed and missing data is filled by interpolation to ensure data continuity and consistency. After normalization, a one-dimensional time series containing only load values ​​is extracted separately as the load sequence required for mode decomposition. The normalized meteorological features are retained for use as external input for subsequent prediction models.

[0022] (2) An improved Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (ICEEMDAN) algorithm is used to perform adaptive mode decomposition on the preprocessed load sequence, generating a set of Intrinsic Mode Functions (IMFs) and a residual term, thereby extracting the effective signal and removing irrelevant noise, effectively separating the stationarity and non-stationarity of the load sequence, and reducing the complexity of the load data; The ICEEMDAN algorithm is an advanced method for signal processing, further optimized from CEEMDAN (Complete Ensemble Empirical Mode Decomposition with Adaptive Noise). Its key improvement lies in the fact that ICEEMDAN only incorporates the Kth-order IMF component obtained from Empirical Mode Decomposition (EMD) of Gaussian white noise as a perturbation term into the current residual, and directly constructs the target IMF through ensemble averaging. This avoids redundant decomposition and scale aliasing. This improvement effectively avoids the residual noise and pseudo-modes that are easily generated when CEEMDAN decomposes signals, thus significantly improving the stability and reconstruction accuracy of the decomposition, and further suppressing mode aliasing.

[0023] The decomposition steps of the improved adaptive noisy complete set empirical mode decomposition algorithm are as follows: (21) Introduce Gaussian white noise into the input signal to generate a noisy signal: (1) In the formula, x represents the normalized load sequence; This represents the noise standard deviation set during the first decomposition. It is the first eigenmode function component obtained through empirical mode decomposition; For the added first k Gaussian white noise; (22) Calculate the residuals obtained after the first decomposition. : (2) In the formula, The local mean function represents the signal and is used to characterize the average trend of the signal within a local region. The signal represents the first k One sampling point; n This represents the total number of sampling points; (23) Solve for the first eigenmode function component. : (3) (24) Calculate the residual of the second decomposition. and the second : (4) (5) (25) Perform iterative decomposition, when the... i The residual obtained after the second decomposition If the number of extreme points is less than 3, or its variance is lower than a preset threshold, the residual is determined to have no further physical meaning for decomposition, and the iteration automatically terminates. At this point, the 3rd iteration... i individual residuals and the i The formula for calculating each IMF component is as follows: (6) (7).

[0024] (3) Sample entropy is introduced to evaluate the time series complexity of each intrinsic mode function component obtained by decomposition, and intrinsic mode function components with similar complexity are aggregated and reconstructed. Based on the entropy value proximity, components with similar complexity are aggregated and reconstructed into three subsequences of high frequency, medium frequency and low frequency, so as to reasonably reduce the number of components, thereby reducing the complexity of prediction modeling and the workload of solution. While decomposing the original load sequence using the ICEEMDAN method can effectively improve the accuracy of the prediction model, the number of IMF components generated is often large, leading to a significant increase in the computational complexity of the overall prediction architecture. Furthermore, some IMF components are highly similar; modeling each component individually would result in redundant computations and reduce modeling speed. To alleviate these problems, sample entropy (SampEn) is introduced as an indicator to quantify the complexity of each IMF component. Components with similar complexity are aggregated and reconstructed, thereby reducing the workload.

[0025] Sample entropy is a metric used to measure the complexity of time series data. Its core idea is to assess the complexity of a time series by calculating the probability of new patterns appearing. Specifically, a smaller sample entropy value indicates stronger self-similarity of the series; a larger entropy value indicates higher randomness and complexity.

[0026] For by N A time series composed of data The calculation process for sample entropy is as follows: (31) Reconstruct the time series sequentially into a set of vector sequences with an embedding dimension of m. ,in, These vectors represent the vectors from the first... i Starting from each sampling point, continuously m An embedding vector consisting of 10 data points; (32) Define two embedding vectors and The distance between them is the maximum absolute difference between the two in their corresponding intrinsic mode function components, that is: (8) (33) For a given embedding vector Statistical satisfaction and vector The quantity (of which) ), denoted as ,definition: (9) Based on this, define : (10) (34) Increase the embedding dimension to m +1, construct a new sequence of embedding vectors. and For each i , Statistical satisfaction and The number of vectors, denoted as , Defined as: (11) Based on this, define : (12) therefore, Indicates similarity tolerance r Below, continuous in two embedding vector sequences m The probability of a point matching; and This corresponds to continuous under the same tolerance. m The probability of a +1 point match is given by the sample entropy, which is defined as: (13) when N When the value is finite, the following formula is used for estimation: (14) After completing the adaptive decomposition of the original load sequence and the aggregation of intrinsic mode function components based on sample entropy, the resulting reconstructed subsequence not only retains the multi-scale dynamic characteristics of the original signal, but also significantly reduces modeling redundancy.

[0027] (4) Construct a hybrid prediction architecture that integrates the Transformer model and the bidirectional long short-term memory network. Model the aggregated subsequences separately. The inputs include the historical load lag features of each subsequence (including long-period features with lags of 24 hours and 168 hours, and short-term lag features 1, 2, 6 and 12 hours before the prediction start point. All lag features strictly adopt historical data before the prediction time to avoid information leakage), meteorological features and calendar features. The output is the predicted value of the subsequence for the next 24 hours. The final load prediction result is generated by superimposing the prediction results of each subsequence. The prediction results of the high-frequency, medium-frequency and low-frequency subsequences are linearly superimposed and the dimensions are restored by inverse normalization to obtain the final 24-hour load prediction value.

[0028] To fully explore the spatiotemporal dependencies in each subsequence and achieve high-precision prediction, a hybrid prediction model integrating a bidirectional long short-term memory network and a Transformer architecture is constructed. This model combines the ability of BiLSTM to finely characterize local time series patterns with the adaptive modeling advantage of Transformer for global long-term dependencies, thus providing a robust and accurate prediction framework for complex power load sequences.

[0029] The Transformer model consists of an encoder and a decoder, both of which adopt a multi-layer stacked structure. Each layer contains core components such as a multi-head self-attention mechanism, a feedforward neural network, and residual connections. The multi-head self-attention mechanism maps the input sequence to multiple different subspaces through multiple sets of parallel learnable linear projections. Each set independently calculates attention weights, enabling the model to simultaneously focus on feature information at different positions and semantic levels in the sequence. Compared with single-head attention, the multi-head mechanism effectively enhances the model's ability to comprehensively perceive the global context of the time series from multiple perspectives, thereby improving the representation accuracy and generalization performance of the prediction task. The specific calculation process is as follows: Let the input sequence be X, and calculate the query matrix Q, the key matrix K, and the value matrix V using three sets of learnable weight matrices: (15) (16) (17) In the formula, , , These are the weight matrices corresponding to the query matrix, key matrix, and value matrix, respectively. The sample vector in the input sequence; By interactively matching the representation matrices of each variable with their corresponding matrices and those of other variables, different linear mappings are applied to the input features. Subsequently, the attention weights corresponding to each variable are calculated based on a scaled dot product multi-head self-attention mechanism, the mathematical expression of which is as follows: (18) In the formula, This represents the score for each sample in the vector; The dimension of the key vector; By introducing a multi-head self-attention mechanism, the model simultaneously learns different representation patterns of the input sequence in multiple parallel self-attention subspaces, thereby capturing diverse dependencies. The computational process is as follows: (19) (20) In the formula, The aggregation function representing each attention head; The parameter matrix represents the linear variation of multi-head attention. Each self-attention layer is followed by a feedforward neural network, whose structure consists of fully connected layers. This network performs a non-linear mapping on the feature representations of each time step or sequence position, calculated using the following formula: (twenty one) In the formula, , These are the weight matrices for the first and second fully connected layers, respectively. Both are automatically learned and updated during model training using backpropagation and gradient descent optimizers, requiring no manual setting. The difference lies in: Transform input features from dimensionality Mapped to a higher-dimensional intermediate space , Then the high-dimensional features are mapped back to the original dimension to ensure that the dimension is consistent when added with the residual connection; , All are bias terms.

[0030] Residual connections and normalization layers play a crucial role in model training. By introducing residual paths, they alleviate the gradient vanishing problem, while normalization operations stabilize the distribution of intermediate features, thereby improving the convergence and stability of model training. Their calculation formulas are as follows: (twenty two) (twenty three) In the formula, This represents the intermediate features after processing by a self-attention layer or a feedforward network. The final output of the Transformer model is generated after residual connection and normalization operations, and can be used as input for subsequent layers.

[0031] The basic structure of the Transformer model is as follows: Figure 1 As shown.

[0032] Long Short-Term Memory (LSTM) network models are a special type of recurrent neural network. By introducing three gate structures (forget gate, input gate, and output gate), LSTM models overcome the gradient vanishing or exploding problems that traditional neural networks are prone to when processing time series data. They can also achieve effective memory and forgetting of long-term dependent information.

[0033] The unit structure of LSTM is as follows Figure 2 As shown.

[0034] First, output based on the memory block from the previous time step. Input at the current moment To calculate the input gate Forgotten Gate Output gate and storage cells Immediately afterwards, the Gate of Oblivion... Memory unit of the previous moment It takes effect, and the input gate Then regulate the state of storage cells The two work together to determine which information to retain and which to discard, and accordingly adjust the current memory unit. Update. Finally, output the gate. Responsible for passing the updated memory cell state to the output section of the memory block. The following is a detailed explanation of the process:

[0035] (twenty four) (25) (26) (27) (28) (29) In the formula, , , These represent the weight matrices corresponding to the forget gate, input gate, and output gate, respectively: Subscript Corresponding forget gate, subscript Corresponding input gate, subscript Corresponding output gate; , , These represent the bias vectors for the forget gate, input gate, and output gate, respectively. It is the Sigmoid activation function.

[0036] Bidirectional Long Short-Term Memory (BiLSTM) is an optimization of the LSTM model. Its hidden layer structure combines forward LSTM and backward LSTM components. Therefore, BiLSTM can process data in both the normal time sequence using the forward LSTM and the reverse time sequence using the backward LSTM. The outputs from both directions are connected to the same output layer, jointly determining the hidden layer values ​​and generating the final output. It should be noted that the data generated here... This is only the result of time-series feature fusion, not the final load forecast result; (30) (31) (32) In the formula, , The outputs are the forward LSTM and the backward LSTM, respectively. , These are the weight matrices from the forward LSTM and backward LSTM to the output layer, respectively. This represents the weight matrix of the output layer.

[0037] The unit structure of BiLSTM is as follows Figure 3 As shown.

[0038] The training process of the hybrid prediction architecture involves constructing a temporal morphology-aware composite loss function as the optimization objective for model training. This loss function has a mean squared error term. Based on this, slope difference penalty terms and curvature difference penalty terms are introduced, and the specific calculation formulas are as follows: (33) in: (34) Slope loss term Used to constrain the consistency of the rate of change between the predicted curve and the actual curve at adjacent time points: (35) Curvature loss term Used to constrain the trend of the predicted curve and the actual curve on the second derivative: (36) In the formula, and These are the weighting coefficients for slope loss and curvature loss, respectively; The actual value; For the predicted values, during training, the model minimizes the temporal form of the composite loss function. Optimize parameters for the target.

[0039] Furthermore, to avoid the risk of overfitting caused by directly using the test set for model selection, this embodiment divides an independent calibration set (e.g., 31 days) from the end of the training set. This calibration set does not participate in model weight updates and is only used to monitor generalization error during training. After each training epoch, the composite loss is calculated on the calibration set. When the calibration set loss no longer decreases for several consecutive epochs, training is stopped, and the model weights at the optimal time of the calibration set loss are returned as the final prediction model. This ensures that the selected model has the best generalization performance on unknown data.

[0040] This example selects a hotel and an industrial company as research objects. The effectiveness of the proposed method is verified through numerical examples. The experimental data includes meteorological data and actual electricity consumption, with a sampling interval of 1 hour. Historical electricity consumption data, meteorological data, and date type are used as input features of the model. The experimental data covers the period from January 1, 2025 to March 10, 2026.

[0041] Based on the above data, this example divides the data into training and testing sets: the complete annual data from January 1, 2025 to December 31, 2025 is used as the training set, and the data from January 1, 2026 to March 10, 2026 is used as the validation set. The model input features mainly include three categories: meteorological features, time calendar features, and historical load features. Meteorological features include perceived temperature, relative humidity, shortwave radiation, 2m temperature, and 10m wind speed; time calendar features include hours, holidays, weekdays, non-working days, months, seasons, and weekday information; historical load features are adaptively constructed based on the temporal characteristics of each component and the forecast requirements to enhance the model's ability to characterize the periodicity and dependence of load. Finally, the model takes the above feature sequence for the 24 hours of the forecast day as input and outputs the load forecast results for that 24 hours.

[0042] Before data decomposition, the load data needs to be preprocessed to eliminate analytical biases caused by discontinuities. This includes removing outliers and filling in missing data to ensure the integrity and continuity of the dataset. Based on this, the ICEEMDAN algorithm is used to decompose the original load series. Ultimately, the original data of a hotel is decomposed into 10 IMF components and 1 residual component; the original data of an industrial company is decomposed into 9 IMF components and 1 residual component. The specific decomposition details are as follows: Figures 5-6 As shown.

[0043] After performing modal decomposition on the load data of an industrial company and a hotel, multiple feature components were obtained. Constructing a load forecasting model for each component individually suffers from computational redundancy and high computational cost. This example quantifies the data complexity and volatility randomness of each feature component by calculating the sample entropy corresponding to each modal component separately. The calculation results are as follows: Figures 7-8 As shown.

[0044] Looking at the sample entropy values ​​of each IMF component in the hotel load data, the entropy values ​​of IMF1 to IMF3 are generally high, and can be classified as high-frequency components; the sample entropy of IMF4 and IMF5 is at an intermediate level, and can be classified as mid-frequency components; while the sample entropy of IMF6 and all subsequent components, along with the Residual, are low, and can be classified as low-frequency components. Therefore, the superposition of each IMF component is shown in Table 1.

[0045] Table 1: Results of IMF component overlay of load data for a hotel

[0046] Based on the sample entropy values ​​of each IMF component in the load data of a certain industrial company, IMF1 and IMF2 have significantly higher sample entropy and can be classified as high-frequency components; the sample entropy of IMF3, IMF4, and IMF5 is at a medium level and is suitable for classification as mid-frequency components; while from IMF6 to Residual, the sample entropy values ​​have significantly decreased and can be classified as low-frequency components. Therefore, the superposition of each IMF component is shown in Table 2.

[0047] Table 2: Results of IMF component overlay on load data of an industrial company

[0048] Transformer-BiLSTM forecasting models were established for the IMF components of a hotel and an industrial company after overlay and reconstruction to achieve high-precision load forecasting. To visually demonstrate the model's forecasting performance, the forecasting results for any two days of load data from both locations were selected for visualization. The forecasting results are shown below. Figures 9-10As shown. Meanwhile, to verify the superiority of the proposed model, BiLSTM (M1), Transformer (M2), BiLSTM-Transformer (M3), ICEEMDAN-Transformer (M4), and ICEEMDAN-BiLSTM (M5) were introduced as comparative models for prediction accuracy comparison analysis. Among them, M6 is the model proposed in this example.

[0049] Depend on Figure 9 As can be seen, the hotel's load exhibits a typical bimodal distribution, with significant fluctuations during the morning peak (8:00-10:00) and evening peak (17:00-19:00), while the load remains stable in the early morning and midday. The M6 ​​model proposed in this embodiment shows the highest degree of overlap between its predicted curve and the actual load, accurately capturing the trend throughout the entire period. Its prediction errors for peak position and amplitude during peak hours are significantly lower than other models. Single models (M1, M2) and hybrid models (M3) generally suffer from underestimation of amplitude or insufficient capture of abrupt changes during peak hours. While the ICEEMDAN decomposition models (M4, M5) show improved accuracy, they still exhibit underfitting defects in local fluctuations.

[0050] Depend on Figure 10 As can be seen, the load of a certain industrial enterprise exhibits a continuous production-type dual-peak characteristic, with high load periods from 08:00–12:00 in the morning and 14:00–18:00 in the afternoon, and 09:00 being the main peak point. Load fluctuations are concentrated during production periods, and the load difference between weekdays and weekends is relatively small. The M6 ​​model proposed in this example has the best prediction curve that matches the actual load, accurately tracking the rising and falling trends of the morning peak, midday load fluctuations, and afternoon peak changes, without significant phase lag or amplitude deviation. The prediction errors at peak, trough, and fluctuation segments are the smallest. Single models (M1, M2) have poor adaptability to the strong continuity and temporal fluctuations of the industrial load, with overly smoothed prediction curves that fail to accurately reflect the abrupt changes and dual-peak characteristics of the actual production load. The hybrid model (M3) has a large prediction deviation in the peak and fluctuation segments. Although M4 and M5 improve the trend fitting ability, they are insufficient in capturing the temporal correlation characteristics between load components, and still have local errors such as overestimation of peaks and underestimation of troughs.

[0051] In the error assessment, three error metrics—mean squared error, root mean square error, and mean absolute error (MAE)—are selected to evaluate the prediction accuracy of each model. The formulas for the three error metrics are as follows:

[0052] (37) (38) (39) In the formula: The actual value; This is a predicted value; For the first The error between the true value and the predicted value of a sample.

[0053] To reveal the accuracy of each model in predicting electricity consumption, three error metrics—MSE, RMSE, and MAE—were used for evaluation. The error results are as follows: Figures 11-12 As shown.

[0054] from Figures 11-12 Looking at the load forecast error radar charts for the two locations: the M6 ​​model proposed in this embodiment is located at the innermost part of the radar chart for MAE, MSE, and RMSE in both hotel and industrial scenarios, with significantly lower error values ​​than all the comparison models; while the errors of the single model and the simple hybrid model are generally higher. The accuracy of the ICEEMDAN scheme combined with the single model has improved, but there is still a gap. This verifies the universality and optimal prediction performance of the model proposed in this example in different power consumption scenarios.

[0055] Example 2: This embodiment provides a power load forecasting system that integrates multimodal decomposition and Transformer-BiLSTM, including: The data preprocessing module is used to preprocess the raw power load and multi-dimensional meteorological data; The mode decomposition module uses an improved adaptive noise complete set empirical mode decomposition algorithm to adaptively decompose the preprocessed load sequence and generate a set of intrinsic mode functions. The component aggregation and reconstruction module is used to introduce sample entropy to perform time series complexity quantification evaluation on each intrinsic mode function component obtained from decomposition, and to aggregate and reconstruct intrinsic mode function components with similar complexity. The load forecasting module constructs a hybrid forecasting architecture that integrates the Transformer model and the bidirectional long short-term memory network. It models the aggregated subsequences separately and generates the final load forecasting result by superimposing the forecasting results of each subsequence.

[0056] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A power load forecasting method integrating multimodal decomposition and Transformer-BiLSTM, characterized in that, The steps are as follows: (1) Preprocess the raw power load and multi-dimensional meteorological data; (2) An improved adaptive noise complete set empirical mode decomposition algorithm is used to perform adaptive mode decomposition on the preprocessed load sequence to generate a set of intrinsic mode functions; (3) Introduce sample entropy to evaluate the time series complexity of each intrinsic mode function component obtained by decomposition, and aggregate and reconstruct intrinsic mode function components with similar complexity to form several subsequences; (4) Construct a hybrid prediction architecture that integrates the Transformer model and the bidirectional long short-term memory network, model the aggregated subsequences separately, and generate the final load prediction result by superimposing the prediction results of each subsequence.

2. The power load forecasting method integrating multimodal decomposition and Transformer-BiLSTM as described in claim 1, characterized in that, In step (1), the meteorological data include perceived temperature, relative humidity, shortwave radiation, 2m temperature and 10m wind speed.

3. The power load forecasting method integrating multimodal decomposition and Transformer-BiLSTM as described in claim 1, characterized in that, The preprocessing process is as follows: The minimum-maximum normalization method is used to map the original power load and meteorological data into the interval [0,1]. After normalization, a one-dimensional time series containing only load values ​​is extracted as the load sequence required for mode decomposition. The normalized meteorological features are retained.

4. The power load forecasting method integrating multimodal decomposition and Transformer-BiLSTM as described in claim 3, characterized in that, In step (2), the decomposition steps of the improved adaptive noise complete set empirical mode decomposition algorithm are as follows: (21) Introduce Gaussian white noise into the input signal to generate a noisy signal: （1）； In the formula, x represents the normalized load sequence; This represents the noise standard deviation set during the first decomposition. It is the first eigenmode function component obtained through empirical mode decomposition; For the added first k Gaussian white noise; (22) Calculate the residuals obtained after the first decomposition. : （2）； In the formula, The local mean function represents the signal and is used to characterize the average trend of the signal within a local region. The signal represents the first k One sampling point; n This represents the total number of sampling points; (23) Solve for the first eigenmode function component. : （3）； (24) Calculate the residuals of the second decomposition. and the second : （4）； （5）； (25) Perform iterative decomposition, when the... i The residual obtained after the second decomposition If the number of extreme points is less than 3, or its variance is lower than a preset threshold, the residual is determined to have no further physical meaning for decomposition, and the iteration automatically terminates. At this point, the 3rd iteration... i individual residuals and the i The formula for calculating each IMF component is as follows: （6）； （7）。 5. The power load forecasting method integrating multimodal decomposition and Transformer-BiLSTM as described in claim 4, characterized in that, In step (3), the smaller the sample entropy value, the stronger the self-similarity of the sequence; the larger the entropy value, the higher the randomness and complexity of the sequence.

6. The power load forecasting method integrating multimodal decomposition and Transformer-BiLSTM as described in claim 5, characterized in that, In step (3), for those by N A time series composed of data The calculation process for sample entropy is as follows: (31) Reconstruct the time series sequentially into a set of vector sequences with an embedding dimension of m. ,in, These vectors represent the vectors from the first... i Starting from each sampling point, continuously m An embedding vector consisting of 10 data points; (32) Define two embedding vectors and The distance between them is the maximum absolute difference between them on their respective intrinsic mode function components, that is: （8）； (33) For a given embedding vector Statistical satisfaction and vector The quantity, denoted as ,definition: （9）； Based on this, define : （10）； (34) Increase the embedding dimension to m +1, construct a new sequence of embedding vectors. and For each i , Statistical satisfaction and The number of vectors, denoted as , Defined as: （11）； Based on this, define : （12）； therefore, Indicates similarity tolerance r Below, continuous in two embedding vector sequences m The probability of a point matching; and This corresponds to continuous under the same tolerance. m The probability of a +1 point match is given by the sample entropy, which is defined as: （13）； when N When the value is finite, the following formula is used for estimation: （14）。 7. The power load forecasting method integrating multimodal decomposition and Transformer-BiLSTM as described in claim 6, characterized in that, In step (4), the Transformer model consists of an encoder and a decoder, both of which adopt a multi-layer stacked structure. Each layer contains a multi-head self-attention mechanism, a feedforward neural network and residual connections. The multi-head self-attention mechanism maps the input sequence to multiple different subspaces through multiple sets of parallel learnable linear projections. Each set independently calculates attention weights, so that the model can simultaneously focus on feature information at different positions and different semantic levels in the sequence. The specific calculation process is as follows: Let the input sequence be X, and calculate the query matrix Q, the key matrix K, and the value matrix V using three sets of learnable weight matrices: （15）； （16）； （17）； In the formula, , , These are the weight matrices corresponding to the query matrix, key matrix, and value matrix, respectively. The sample vector in the input sequence; By interactively matching the representation matrices of each variable with their corresponding matrices and those of other variables, different linear mappings are applied to the input features. Subsequently, the attention weights corresponding to each variable are calculated based on a scaled dot product multi-head self-attention mechanism, the mathematical expression of which is as follows: （18）； In the formula, This represents the score for each sample in the vector; The dimension of the key vector; By introducing a multi-head self-attention mechanism, the model simultaneously learns different representation patterns of the input sequence in multiple parallel self-attention subspaces, thereby capturing diverse dependencies. The computational process is as follows: （19）； （20）； In the formula, The aggregation function representing each attention head; The parameter matrix represents the linear variation of multi-head attention. Each self-attention layer is followed by a feedforward neural network, whose structure consists of fully connected layers. This network performs a non-linear mapping on the feature representations of each time step or sequence position, calculated using the following formula: （21）； In the formula, , These are the weight matrices for the first and second fully connected layers, respectively, with the following difference: Transform input features from dimensionality Mapped to a higher-dimensional intermediate space , Then the high-dimensional features are mapped back to the original dimensions; , All are bias terms; Residual connections and normalization layers play a crucial role in model training. By introducing residual paths, they alleviate the gradient vanishing problem, while normalization operations stabilize the distribution of intermediate features, thereby improving the convergence and stability of model training. Their calculation formulas are as follows: （22）； （23）； In the formula, This represents the intermediate features after processing by a self-attention layer or a feedforward network. This is the final output of the Transformer model.

8. The power load forecasting method integrating multimodal decomposition and Transformer-BiLSTM as described in claim 7, characterized in that, In step (4), the bidirectional long short-term memory network is an optimization of the LSTM model. Its hidden layer structure integrates both forward LSTM and backward LSTM. Therefore, the bidirectional long short-term memory network can process data in the normal order of the time series through the forward LSTM and in the reverse order of the time series through the backward LSTM. The outputs of both directions are connected to the same output layer, which together determine the hidden layer values, thereby generating the final output. ; （30）； （31）； （32）； In the formula, , The outputs are the forward LSTM and the backward LSTM, respectively. , These are the weight matrices from the forward LSTM and backward LSTM to the output layer, respectively. This represents the weight matrix of the output layer.

9. The power load forecasting method integrating multimodal decomposition and Transformer-BiLSTM as described in claim 8, characterized in that, In step (4), the training process of the hybrid prediction architecture involves constructing a temporal morphology-aware composite loss function as the optimization objective for model training. This loss function has a mean squared error term. Based on this, slope difference penalty terms and curvature difference penalty terms are introduced, and the specific calculation formulas are as follows: （33）； in: （34）； Slope loss term Used to constrain the consistency of the rate of change between the predicted curve and the actual curve at adjacent time points: （35）； Curvature loss term Used to constrain the trend of the predicted curve and the actual curve on the second derivative: （36）； In the formula, and These are the weighting coefficients for slope loss and curvature loss, respectively; The actual value; For the predicted values, during training, the model minimizes the temporal form of the composite loss function. Optimize parameters for the target.

10. A power load forecasting system integrating multimodal decomposition and Transformer-BiLSTM, characterized in that, include: The data preprocessing module is used to preprocess the raw power load and multi-dimensional meteorological data; The mode decomposition module uses an improved adaptive noise complete set empirical mode decomposition algorithm to adaptively decompose the preprocessed load sequence and generate a set of intrinsic mode functions. The component aggregation and reconstruction module is used to introduce sample entropy to perform time series complexity quantification evaluation on each intrinsic mode function component obtained from decomposition, and to aggregate and reconstruct intrinsic mode function components with similar complexity to form several subsequences; The load forecasting module constructs a hybrid forecasting architecture that integrates the Transformer model and the bidirectional long short-term memory network. It models the aggregated subsequences separately and generates the final load forecasting result by superimposing the forecasting results of each subsequence.

Images