A Comprehensive Energy Multi-Step Load Forecasting Method Based on Bayesian Optimization and Overfitting Resistance
By employing Bayesian optimization and anti-overfitting strategies, a hybrid CNN-multi-head attention-LSTM model is constructed, which addresses the issues of low load forecasting accuracy and model overfitting in integrated energy systems. This model achieves high-precision and high-generalization multi-step load forecasting, supporting optimized scheduling of energy systems.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ANHUI UNIV OF SCI & TECH
- Filing Date
- 2026-01-26
- Publication Date
- 2026-06-02
AI Technical Summary
Existing technologies have limited accuracy in load forecasting for integrated energy systems, low efficiency in hyperparameter optimization, severe model overfitting, and fail to fully consider the individual differences of different energy loads, leading to a decrease in the accuracy of long-term, multi-step forecasts.
We employ a Bayesian optimization framework for automatic hyperparameter optimization, construct a CNN-multi-head attention-LSTM hybrid model, and integrate Dropout regularization, L2 regularization, and early stopping strategies. Combined with dynamic feature engineering and classification training, we improve the model's feature extraction and anti-overfitting performance.
It improves the accuracy and stability of multi-step load forecasting for the next 24 hours of the integrated energy system, enhances the generalization ability of the model, and supports the optimized scheduling and economic operation of the energy system.
Smart Images
Figure CN122133858A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of energy system optimization technology, and in particular to a comprehensive multi-step energy load prediction method based on Bayesian optimization and anti-overfitting. Background Technology
[0002] Integrated energy systems are key infrastructure for the coordinated production, conversion, storage, and consumption of multiple energy sources, including electricity, cooling, and heating. They are of great significance for improving the overall efficiency of energy utilization, promoting the consumption of renewable energy, and achieving dual-carbon goals. Load forecasting, especially accurate forecasting of multi-step loads for the next few hours to days, is a core prerequisite for integrated energy systems to achieve optimized scheduling, supply-demand balance, and economical operation.
[0003] Traditional load forecasting methods mainly rely on time series analysis (such as ARIMA models) or physical mechanism-based modeling. When faced with load data from diverse sources, complex couplings, and strong nonlinearity and time-series dependence in integrated energy systems, these methods often fail to fully capture the inherent dynamic patterns, resulting in limited forecast accuracy and making it difficult to meet the needs of real-time scheduling and refined management of smart energy systems.
[0004] In recent years, deep learning technology, especially Long Short-Term Memory (LSTM) networks, has been widely used in load forecasting due to its powerful temporal modeling capabilities. However, directly applying LSTM and other models to comprehensive multi-step energy load forecasting still faces a series of challenges: First, model performance is highly dependent on the configuration of hyperparameters (such as the number of network layers, number of neurons, learning rate, regularization coefficient, etc.). Traditional hyperparameter optimization methods (such as grid search and random search) are usually inefficient, requiring the traversal of a large number of invalid combinations, and heavily reliant on expert experience. They are difficult to find globally near-optimal hyperparameter combinations within an acceptable timeframe, thus limiting further improvements in model performance.
[0005] Secondly, the comprehensive energy load data sample is limited and has high feature dimensionality, while hybrid deep learning models such as CNN-LSTM-attention have complex structures and a large number of parameters, making them prone to overfitting under small sample sizes or extreme conditions. Overfitting of the model to noise and specific patterns in the training data leads to a significant decrease in its generalization ability on unseen data and insufficient predictive stability.
[0006] Furthermore, for multi-step forecasting tasks such as forecasting the next 24 hours, the forecasting error accumulates and amplifies as the forecasting step size increases. Existing methods typically use a uniform model structure to forecast loads with different characteristics such as electricity, cooling, and heating, failing to fully consider the individual differences in fluctuation patterns, periodicity, and influencing factors of various loads, leading to an overall decrease in the accuracy of long-term multi-step forecasts.
[0007] Therefore, how to efficiently optimize the hyperparameters of complex models, effectively suppress model overfitting, and perform differentiated modeling for different energy load characteristics has become the key to improving the accuracy and practicality of multi-step load forecasting for integrated energy systems. Summary of the Invention
[0008] To address the aforementioned technical problems, this invention provides a comprehensive multi-step energy load forecasting method based on Bayesian optimization and anti-overfitting, comprising the following steps: S1. Obtain historical time-series data of the integrated energy system, and perform missing value repair and normalization on the historical time-series data to obtain standardized time-series data; wherein, the historical time-series data includes at least electricity load data, cooling load data and heating load data; S2. Based on the standardized time-series data, an initial feature set is constructed. A random forest model is used to calculate the importance of each feature in the initial feature set, and a core feature subset is selected from the initial feature set according to a preset cumulative importance threshold. The initial feature set includes statistical features obtained by rolling statistics of the historical time-series data based on multiple preset time windows, and time period features obtained by cyclic encoding based on timestamp information. S3. Define the hyperparameter search space of the hybrid model; using a Bayesian optimization framework, with the objective of minimizing the comprehensive loss on the validation set, perform an iterative search in the hyperparameter search space to determine an optimal combination of hyperparameters; wherein, the comprehensive loss includes a prediction error term and a hybrid model weight regularization term; S4. Configure and construct the hybrid model according to the optimal hyperparameter combination; during the training process of the hybrid model, apply Dropout regularization, L2 weight regularization, and an early stopping strategy based on the validation set performance simultaneously. S5. For each of the power load, cooling load, and heating load, the hybrid model is trained independently using the corresponding core feature subsets to obtain their respective prediction models; the current time-series features are input into the prediction model to output multi-step prediction values for each type of load within a specified future time period.
[0009] Furthermore, the hybrid model sequentially includes a convolutional layer for extracting local temporal features, a multi-head attention layer for capturing long-range dependencies, and a long short-term memory network layer for temporal modeling.
[0010] Furthermore, in S1, the missing value repair is performed using a combination of forward padding and backward padding; the normalization process uses the RobustScaler method based on the median and interquartile range.
[0011] Furthermore, in S2, the statistical features include the rolling mean and rolling standard deviation calculated within multiple sliding windows of different time lengths; the time period features include sine and cosine codes generated based on the hour, week, and month dimensions, respectively.
[0012] Furthermore, the step of selecting a core feature subset from the initial feature set based on a preset cumulative importance threshold specifically includes: The features in the initial feature set are arranged in descending order of importance, and their importance is accumulated sequentially. The top k features whose cumulative importance reaches a preset threshold are selected as the core feature subset, and k is not less than the preset minimum number of features.
[0013] Furthermore, in S3, the hyperparameter search space includes: the number of filters in the convolutional layer, the number of attention heads and key dimension in the multi-head attention layer, the number of units in the long short-term memory network layer and whether to enable the two-layer structure, the Dropout inactivation rate and L2 regularization coefficient of each network layer, and the learning rate of the model optimizer.
[0014] Furthermore, in S3, the Bayesian optimization framework is Optuna; the formula for calculating the comprehensive loss is:
[0015] In the formula, For the predicted value and the actual value in the future Mean square error at each time step To predict the step size; For the first Layer network weights The corresponding L2 regularization coefficient.
[0016] Furthermore, in S4, the early stopping strategy is: During training, the overall loss on the validation set is continuously monitored. If the overall loss does not decrease to a new minimum value within N consecutive training cycles, training is terminated, and the model parameters that perform best on the validation set are used as the final model parameters, where N is a preset patience value.
[0017] Furthermore, in S4, the hybrid model is a CNN-multi-head attention-LSTM hybrid model; the LSTM layer structure of the CNN-multi-head attention-LSTM hybrid model is adaptively configured, and the Bayesian optimization method automatically selects whether to use a single-layer LSTM structure or a double-layer LSTM structure based on the Boolean hyperparameters in the hyperparameter search space.
[0018] Furthermore, it also includes: S6. Perform inverse normalization on the multi-step predicted values to obtain the load prediction results in actual dimensions; and calculate at least one evaluation index among the mean absolute error, root mean square error and coefficient of determination to quantify the prediction accuracy.
[0019] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: This invention utilizes Bayesian optimization to achieve efficient automatic hyperparameter optimization, overcoming the shortcomings of traditional parameter tuning methods, such as low efficiency and reliance on experience. It constructs a hybrid model integrating convolutional neural networks, multi-head attention mechanisms, and long short-term memory networks, and integrates multi-layer Dropout, L2 regularization, and early stopping strategies to enhance the model's feature extraction, long-range dependency capture capabilities, and anti-overfitting performance. Furthermore, it implements categorized independent training and prediction based on the differentiated characteristics of electrical, cooling, and heating loads, effectively improving the overall accuracy, stability, and generalization ability of multi-step load forecasting for the integrated energy system over the next 24 hours, providing reliable technical support for the optimized scheduling and safe and economical operation of energy systems. Attached Figure Description
[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0021] Figure 1 The flowchart shows the integrated energy multi-step load prediction method based on Bayesian optimization and anti-overfitting strategy disclosed in this invention. Figure 2 This is a flowchart of the architecture of the CNN-multi-head attention-LSTM hybrid model disclosed in this invention. Detailed Implementation
[0022] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0023] This invention provides a comprehensive energy multi-step load forecasting method based on Bayesian optimization and anti-overfitting, aiming to solve the problems of low efficiency in hyperparameter tuning, severe model overfitting, and accuracy degradation in long-term multi-step forecasts in existing technologies. This method achieves high-precision, high-generalization-capability multi-step forecasting of the comprehensive energy load for electricity, cooling, and heating over the next 24 hours by integrating dynamic feature engineering, Bayesian hyperparameter optimization, and a hybrid deep learning model incorporating multiple anti-overfitting strategies.
[0024] Please see Figure 1 The integrated energy multi-step load forecasting method provided in this embodiment specifically includes the following steps: S1. Data Acquisition and Preprocessing: Acquire historical time-series data of the integrated energy system, perform missing value repair and normalization on the historical time-series data to obtain standardized time-series data.
[0025] Specifically, historical time-series data is periodically acquired from the integrated energy system's data acquisition and monitoring system. This historical time-series data includes at least: electricity load data P (unit: kW), cooling load data C (unit: CHWTON (kW)), heat load data H (unit: HT mmBTU (kW)), and corresponding timestamp data t. stamp The original data may contain missing values due to transmission or equipment failure.
[0026] First, a strategy combining forward and backward padding is used to repair missing values. For the first... Missing values at each time point The repair formula is: In the formula, For the first The load or characteristic value at a given time, if If time-based data exists, then use... Fill (i.e., fill forward); otherwise, if If time-based data exists, then use... Padded (i.e., backfilling). After padding, the complete original time series data matrix is obtained. ∈R NXD Where N is the number of time points and D is the original feature dimension (3 in this example, i.e., electrical, cooling and heating loads).
[0027] Secondly, to eliminate the interference of differences in load dimensions and extreme outliers on model training, the RobustScaler method based on robust statistics is used for normalization. This normalization is performed independently for each column of data, as shown in the following formula:
[0028] in, This is the median of the data in this column. The interquartile range of the data in this column ( It is the upper quartile. (The lower quartiles are used). After processing, a standardized data matrix is obtained. ∈R NXD Its data distribution revolves around a median of 0 and is not sensitive to outliers.
[0029] S2. Dynamic Feature Construction and Selection: Based on standardized time-series data, an initial feature set is constructed. The importance of each feature in the initial feature set is calculated using a random forest model. Based on a preset cumulative importance threshold, a subset of core features is selected from the initial feature set.
[0030] The initial feature set includes statistical features obtained by rolling statistics of historical time series data based on multiple preset time windows, and time period features obtained by cyclic encoding based on timestamp information.
[0031] This step aims to extract more predictive features from standardized data and remove redundant information. Specifically, it includes: S2.1 Constructing the initial feature set First, for each type of load data (electricity, cooling, and heating), the rolling mean and rolling standard deviation (rollstd) are calculated for multiple time windows (12 hours, 24 hours, 48 hours, and 168 hours in this embodiment) to capture the trend and fluctuation characteristics of the load at different time scales. The calculation formulas are as follows:
[0032]
[0033] This generates the rolling feature matrix. ∈R NX（3X4X2） .
[0034] Secondly, the timestamp information is converted into features that can express periodicity. Sine and cosine encodings are applied to the hour, day of week, and month respectively, enabling the model to understand the cyclical nature of time. The encoding formulas are as follows:
[0035]
[0036]
[0037] This generates a time feature matrix. ∈R Nx6 .
[0038] Next, and Concatenate to obtain the initial feature matrix ∈R NxFin (Fint is the initial total number of features).
[0039] S2.2 Core Feature Filtering Evaluation using the random forest model Importance of each feature to the prediction target (e.g., load in the next 24 hours) Sort all features from highest to lowest importance and calculate cumulative importance. .
[0040] Set the filtering strategy: set the filtering threshold , Keep before The core features constitute a subset of the core features, and the corresponding feature matrix is denoted as . ∈R Nxk Complete the feature engineering and noise reduction of the load data.
[0041] S3. Hyperparameter Optimization: Define the hyperparameter search space for the CNN-multi-head attention-LSTM hybrid model; adopt a Bayesian optimization framework, aiming to minimize the comprehensive loss on the validation set, and perform an iterative search in the hyperparameter search space. The comprehensive loss includes a prediction error term and a model weight regularization term; determine an optimal combination of hyperparameters through iterative search. The Bayesian optimization framework is Optuna.
[0042] Those skilled in the art will understand that this step is for the CNN-multi-head attention-LSTM hybrid model to be built, which has numerous hyperparameters that interact with each other. To efficiently and automatically find the optimal configuration, Bayesian optimization based on the Optuna framework is introduced. Specifically, it includes the following steps: S3.1 Defining the Hyperparameter Search Space The hyperparameter search space includes: the number of filters in the convolutional layer, the number of attention heads and key dimension in the multi-head attention layer, the number of units in the long short-term memory network layer and whether to enable the two-layer structure, the Dropout inactivation rate and L2 regularization coefficient of each network layer, and the learning rate of the model optimizer.
[0043] In one specific embodiment, as shown in Table 1, a reasonable range of hyperparameter values is defined for each component of the model.
[0044] Table 1
[0045] S3.2 Constructing the Objective Function and Optimization The objective function of Bayesian optimization is defined as minimizing the overall loss on the validation set. The loss function takes into account both prediction accuracy and model complexity, and is expressed by the following formula:
[0046] In the formula, the first term is the predicted value. Compared with the true value In the future =Mean squared error (MSE) over 24 steps, the second term is the sum of L2 regularization penalty terms for all weights W in the model, This is the set of trainable weight matrices for the model. This represents the L2 coefficient of the corresponding layer.
[0047] Start Optuna optimization and set the optimization direction to minimize. The total number of iterations is set. In each iteration, Optuna samples a set of hyperparameters from the search space, builds a model based on these parameters, trains it on the training set using an early stopping strategy, and computes it on the validation set. Optuna utilizes existing hyperparameter combinations. The results construct a probabilistic surrogate model to guide subsequent sampling towards parameter regions with lower loss. After optimization, the output makes... The smallest optimal combination of hyperparameters, best_hp.
[0048] S4. Construction and Training of the Anti-Overfitting Hybrid Model: Configure and construct a CNN-Multi-Head Attention-LSTM hybrid model according to the optimal hyperparameter combination; The CNN-Multi-Head Attention-LSTM hybrid model includes convolutional layers for extracting local temporal features, multi-head attention layers for capturing long-range dependencies, and long short-term memory network layers for temporal modeling; During the training of the hybrid model, Dropout regularization, L2 weight regularization, and an early stopping strategy based on validation set performance are applied simultaneously.
[0049] Based on the best_hp configuration in step S32 above, construct as follows: Figure 2 The detailed structure and data flow of the CNN-multi-head attention-LSTM hybrid model shown are as follows: (1) Input layer: Receiver shape is ( Temporal feature sequence of F) The length of the input window = 24 hours, the number of features F is the number of core features obtained in step S22. .
[0050] (2) Convolutional Layer: One-dimensional convolution (Conv1D) operations are used to extract local (short-term) load fluctuation patterns. The kernel size is set to 3, and the padding method is 'same' to maintain the time step length. The number of filters is determined by best_hp['conv_filters']. After the convolution output is activated by the ReLU function, Dropout regularization is immediately applied with a deactivation rate of best_hp['conv_dropout'] to obtain the output. :
[0051]
[0052] Among them, the output matrix .
[0053] (3) Multi-head attention layer: The query (Q), key (K), and value (V) matrices are obtained through different linear transformations:
[0054] in, .
[0055] The number of attention heads (num_heads) and the dimension of each head (key_dim) are determined by best_hp. Attention weights are calculated independently for each head, and the weighted aggregated value vector is calculated using the following formula:
[0056] The outputs of all heads are concatenated and passed through a linear projection layer to obtain the multi-head attention output. Finally, it is passed through a residual connection (with the input) The output is obtained through addition and layer normalization (LayerNorm) steps. This layer enables the model to focus on dependencies between arbitrary time steps in a long sequence.
[0057] (4) Feedforward Network: The feedforward network enhances the nonlinear fitting capability. It consists of a fully connected layer (the hidden layer dimension is determined by best_hp['ff_dim']), ReLU activation, Dropout (best_hp['ff_dropout']), and another fully connected layer. L2 weight decay (with coefficient best_hp['ff_l2']) is also added during the forward computation. Similarly, it is connected via residual connections (and...). (Addition) and layer normalization, output :
[0058]
[0059]
[0060]
[0061] Among them, the output .
[0062] (5) Adaptive LSTM layer: The LSTM layer structure of the CNN-multi-head attention-LSTM hybrid model is adaptively configured. The Bayesian optimization method automatically selects whether to use a single-layer LSTM structure or a double-layer LSTM structure based on the Boolean hyperparameters in the hyperparameter search space, as shown in Table 2, which illustrates the mechanism of single / double-layer LSTM.
[0063] Specifically, the structure of this layer is automatically selected by best_hp['use_second_lstm']. If True, a two-layer LSTM is constructed: the number of units in the first LSTM is determined by best_hp['lstm_units1'], followed by Dropout(best_hp['lstm_dropout1']); its output is used as the input to the second LSTM (the number of units is also configurable), followed by another Dropout.
[0064] If False, then only a single-layer LSTM is constructed followed by Dropout. L2 regularization (coefficient best_hp['lstm_l2_1']) is applied to the kernel weights and recurrent weights of the LSTM layer.
[0065] Table 2
[0066] In one specific embodiment, the "whether to enable two-layer LSTM" is first incorporated into the search space as a hyperparameter `use_second_lstm`, with a value range of {True, False}. Simultaneously, corresponding sub-hyperparameters are defined for each of the two modes (two-layer mode corresponds to `lstm_units2`, `lstm_l2_2`, `lstm_dropout2`, etc., and single-layer mode corresponds to `lstm_units_single`, `lstm_l2_single`, `lstm_dropout_single`, etc.), forming a complete hyperparameter search system. In Optuna's iterative search, each iteration randomly samples a value of `use_second_lstm` and dynamically constructs the model structure based on that value using the corresponding sub-hyperparameter. If the sample is True, a two-layer structure of "first-layer LSTM + Dropout + second-layer LSTM + Dropout" is constructed, enhancing the ability to capture complex temporal dependencies through the stacking of two layers. If the sample is False, a structure of "first-layer LSTM + Dropout + second-layer LSTM + Dropout" is constructed. The simplified structure of "single-layer LSTM + Dropout" reduces the risk of overfitting with fewer parameters and is suitable for scenarios with relatively stable time-series patterns, such as thermal loads. Then, for each iteration, the model is trained using the same training set, and the "total loss" (MSE loss + L2 regularization loss) is calculated using the validation set. This loss serves as the core metric, directly reflecting the model's generalization ability on unseen data. Optuna records the validation loss corresponding to different values of `use_second_lstm` using a Bayesian probability model, continuously focusing the search towards the region with the "lower loss." After the iteration, the `use_second_lstm` value that minimizes the validation loss is selected as the optimal solution (i.e., `best_hp[use_second_lstm]`), thus automatically determining the number of LSTM layers suitable for this load type. This achieves a precise match between the model structure and the load's time-series characteristics (using two layers to enhance fitting ability for complex loads and a single layer to avoid computational redundancy for simple loads), all without manual intervention. This ensures both model performance and balances generalization ability with computational efficiency.
[0067] (6) Output layer: A linear fully connected layer that maps the final output of the LSTM layer to the next 24 hours. The normalized load forecast vector (=24) It satisfies the formula:
[0068] in, , For output layer bias, output (Normalized forecast values for the next 24 hours); By integrating multi-layer Dropout, L2 regularization, and early stopping strategies, an anti-overfitting prediction model is constructed.
[0069] Those skilled in the art will understand that a triple anti-overfitting strategy is integrated during the model training process: (a) L2 regularization: As mentioned above, weight penalties are explicitly added to the objective function and network layers.
[0070] (b) Dropout: Randomly drop some neurons at multiple locations such as convolutional layers, feedforward networks, and LSTM layers to prevent neuronal over-adaptation.
[0071] (c) Early Stopping Strategy: During training, the overall loss on the validation set is continuously monitored. If the overall loss does not decrease to a new minimum value within N consecutive training epochs, training is terminated, and the model parameters that perform best on the validation set are used as the final model parameters, where N is a preset patience value. The process is shown in Table 3. During training, the validation set loss is continuously monitored. Initialize the best loss (best_loss) and the patience counter (wait_count) to 0. Calculate the current... .like If the decrease in performance exceeds the minimum improvement threshold min_delta, then best_loss is updated, the current model parameters are saved, and wait_count is reset to 0; otherwise, wait_count is incremented by 1. When wait_count reaches a preset patience value (e.g., 10), training is immediately terminated, and the saved best model parameters are loaded. This strategy effectively prevents the degradation of validation performance caused by overfitting on the training set.
[0072] Table 3
[0073] S5. Categorized Multi-Step Load Forecasting: For power load, cooling load and heating load, use their corresponding core feature subsets to independently train CNN-multi-head attention-LSTM hybrid models to obtain their respective prediction models; input the current time series features into the prediction models and output the multi-step prediction values of each type of load in the future specified time period.
[0074] Those skilled in the art will understand that, since electrical, cooling, and heating loads have different physical characteristics and variation patterns, this embodiment adopts a strategy of independent training and prediction based on different types.
[0075] First, for electricity, cooling, and heating loads, the corresponding core feature sequences are extracted, and training, validation, and test sets are divided.
[0076] Then, for each load type, steps S3 (hyperparameter optimization) and S4 (model building and training) are repeated independently to obtain three proprietary optimal prediction models. This allows each model to be deeply adapted to the unique temporal patterns of its target load.
[0077] Next, the feature sequence of the test set (with a time window of 24 hours) is input into the trained corresponding model to obtain the normalized prediction value Y_pred_scaled for the next 24 hours. Using median(X) and IQR(X) saved in step S1, inverse normalization is performed to obtain the actual load prediction value Y_pred.
[0078] S6. Post-processing and evaluation of results: Perform inverse normalization on the multi-step predicted values to obtain the load prediction results in actual dimensions; and calculate at least one of the following evaluation indicators: mean absolute error, root mean square error, and coefficient of determination, to quantify the prediction accuracy.
[0079] In one specific embodiment, an evaluation metric is calculated between the predicted value Y_pred and the actual value to quantify the prediction performance, mainly including: Mean absolute error:
[0080] Root mean square error:
[0081] Coefficient of determination:
[0082] in, For the number of test samples, This is the mean of the true load sequence.
[0083] The final output includes forecast curves and accuracy reports for the electricity, cooling, and heating loads for the next 24 hours.
[0084] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A comprehensive multi-step energy load forecasting method based on Bayesian optimization and anti-overfitting, characterized in that, Includes the following steps: S1. Obtain historical time-series data of the integrated energy system, and perform missing value repair and normalization on the historical time-series data to obtain standardized time-series data; wherein, the historical time-series data includes at least electricity load data, cooling load data and heating load data; S2. Based on the standardized time-series data, an initial feature set is constructed. A random forest model is used to calculate the importance of each feature in the initial feature set, and a core feature subset is selected from the initial feature set according to a preset cumulative importance threshold. The initial feature set includes statistical features obtained by rolling statistics of the historical time-series data based on multiple preset time windows, and time period features obtained by cyclic encoding based on timestamp information. S3. Define the hyperparameter search space of the hybrid model; using a Bayesian optimization framework, with the objective of minimizing the comprehensive loss on the validation set, perform an iterative search in the hyperparameter search space to determine an optimal combination of hyperparameters; wherein, the comprehensive loss includes a prediction error term and a hybrid model weight regularization term; S4. Configure and construct the hybrid model according to the optimal hyperparameter combination; during the training process of the hybrid model, apply Dropout regularization, L2 weight regularization, and an early stopping strategy based on the validation set performance simultaneously. S5. For each of the power load, cooling load, and heating load, the hybrid model is trained independently using the corresponding core feature subsets to obtain their respective prediction models; the current time-series features are input into the prediction model to output multi-step prediction values for each type of load within a specified future time period.
2. The integrated energy multi-step load forecasting method based on Bayesian optimization and anti-overfitting as described in claim 1, characterized in that, The hybrid model sequentially includes a convolutional layer for extracting local temporal features, a multi-head attention layer for capturing long-range dependencies, and a long short-term memory network layer for temporal modeling.
3. The integrated energy multi-step load forecasting method based on Bayesian optimization and anti-overfitting as described in claim 1, characterized in that, In S1, the missing value repair is performed using a combination of forward padding and backward padding; the normalization process uses the RobustScaler method based on the median and interquartile range.
4. The integrated energy multi-step load forecasting method based on Bayesian optimization and anti-overfitting as described in claim 1, characterized in that, In S2, the statistical features include the rolling mean and rolling standard deviation calculated within multiple sliding windows of different time lengths; the time period features include sine and cosine codes generated based on the hour, week, and month dimensions, respectively.
5. The integrated energy multi-step load forecasting method based on Bayesian optimization and anti-overfitting as described in claim 1, characterized in that, The step of selecting a core feature subset from the initial feature set based on a preset cumulative importance threshold specifically includes: The features in the initial feature set are arranged in descending order of importance, and their importance is accumulated sequentially. The top k features whose cumulative importance reaches a preset threshold are selected as the core feature subset, and k is not less than the preset minimum number of features.
6. The integrated energy multi-step load forecasting method based on Bayesian optimization and anti-overfitting as described in claim 2, characterized in that, In S3, the hyperparameter search space includes: the number of filters in the convolutional layer, the number of attention heads and key dimension in the multi-head attention layer, the number of units in the long short-term memory network layer and whether to enable the two-layer structure, the Dropout inactivation rate and L2 regularization coefficient of each network layer, and the learning rate of the model optimizer.
7. The integrated energy multi-step load forecasting method based on Bayesian optimization and anti-overfitting as described in claim 1, characterized in that, In S3, the Bayesian optimization framework is Optuna; the formula for calculating the comprehensive loss is: In the formula, For the predicted value and the actual value in the future Mean square error at each time step To predict the step size; For the first Layer network weights The corresponding L2 regularization coefficient.
8. The integrated energy multi-step load forecasting method based on Bayesian optimization and anti-overfitting as described in claim 1, characterized in that, In S4, the early stopping strategy is: During training, the overall loss on the validation set is continuously monitored. If the overall loss does not decrease to a new minimum value within N consecutive training cycles, training is terminated, and the model parameters that perform best on the validation set are used as the final model parameters, where N is a preset patience value.
9. The integrated energy multi-step load forecasting method based on Bayesian optimization and anti-overfitting as described in claim 1, characterized in that, In S4, the hybrid model is a CNN-multi-head attention-LSTM hybrid model; the LSTM layer structure of the CNN-multi-head attention-LSTM hybrid model is adaptively configured, and the Bayesian optimization method automatically selects whether to use a single-layer LSTM structure or a double-layer LSTM structure based on the Boolean hyperparameters in the hyperparameter search space.
10. The integrated energy multi-step load forecasting method based on Bayesian optimization and anti-overfitting as described in claim 1, characterized in that, Also includes: S6. Perform inverse normalization on the multi-step predicted values to obtain the load prediction results in actual dimensions. It also calculates at least one of the following evaluation metrics: mean absolute error, root mean square error, and coefficient of determination, to quantify prediction accuracy.