Power load prediction method based on heterogeneous ensemble learning and attention mechanism

Abstract

The invention discloses a power load prediction method based on heterogeneous ensemble learning and an attention mechanism. The method comprises the following steps: acquiring historical data of a power load, the historical data at least comprising time sequence data; preprocessing the historical data to obtain processed feature data; based on the feature data, an integrated learning model is constructed, and the integrated learning model at least comprises a combination of multiple base learners; predicting the feature data through the integrated learning model to obtain a preliminary prediction result; and introducing an attention mechanism to carry out weighted adjustment on the preliminary prediction result, and generating a final power load prediction value. According to the method, the nonlinear and complex modes of the power load are effectively captured, the prediction performance is remarkably improved, and accurate decision support is provided for intelligent operation of a power system.

Description

Technical Field

[0001] The present invention relates to the technical field of power load forecasting, and in particular to a power load forecasting method based on heterogeneous ensemble learning and attention mechanism. Background Art

[0002] Power load forecasting is a critical component of power grid dispatching decisions, but it faces complex and ever-changing influencing factors and data characteristics. Extracting effective features from massive amounts of historical data and accurately capturing load variation patterns are key challenges. Traditional single models struggle to fully account for time series characteristics, external factors, and their dynamic relationships, resulting in insufficient forecasting accuracy. Furthermore, load variation characteristics vary across time scales and scenarios, flexibly adapting to these variations presents a significant challenge. Furthermore, the complex and volatile power grid operating environment requires robust and adaptable forecasting models. In practical applications, balancing model complexity with computational efficiency to achieve high-precision, real-time forecasting is also a key issue. Addressing these challenges, power load forecasting methods require innovations in feature extraction, model fusion, and dynamic weight allocation to improve forecast accuracy, adaptability, and reliability. This not only impacts the safe and stable operation of the power grid but also directly impacts the optimal allocation of power resources and dispatch efficiency. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to propose a power load forecasting method based on heterogeneous ensemble learning and attention mechanism, which can solve at least one technical problem mentioned in the background technology.

[0004] According to one aspect of the present invention, a method for power load forecasting based on heterogeneous ensemble learning and attention mechanism is provided, the method comprising:

[0005] Acquiring historical data of power load, wherein the historical data at least includes time series data;

[0006] Preprocessing the historical data to obtain processed feature data;

[0007] Based on the feature data, construct an ensemble learning model, wherein the ensemble learning model includes at least a combination of multiple base learners;

[0008] Predicting the feature data using the ensemble learning model to obtain a preliminary prediction result;

[0009] An attention mechanism is introduced to perform weighted adjustment on the preliminary prediction results to generate the final power load prediction value.

[0010] In the above technical solution, to build a power load forecasting model, it is necessary to first obtain historical power load data containing time series data. This data type can characterize the dynamic change of power load over time, reflect the fluctuation trend and periodic characteristics on different time scales, and lay the foundation of time series information for subsequent model construction. Preprocessing of historical data is the key to ensuring the quality of model input data. The process includes data cleaning, normalization or standardization, feature engineering, etc. The feature data obtained in this way can more effectively characterize the factors affecting power load and provide reliable data support for accurate model prediction. Constructing an ensemble learning model based on the processed feature data is one of the core links. The ensemble learning model contains at least a combination of multiple base learners, such as decision trees, neural networks, support vector machines and other different types of learners. Different base learners have their own advantages and disadvantages, and they can complement each other after combination. Heterogeneous integration can improve the generalization ability and prediction accuracy of the model, making it more accurately fit the complex change pattern of power load. Using the constructed ensemble learning model to predict feature data, preliminary forecast results are obtained. This is the initial application of the model's learning effect. New input data is predicted based on the mapping relationship between power load and feature data learned during the model training process. While these preliminary forecast results reflect the model's basic forecasting capabilities, they do not consider the importance of the forecast results or the weight differences of key influencing factors in the time series, requiring further optimization. The innovation of this forecasting model lies in the introduction of an attention mechanism to weight the preliminary forecast results. The attention mechanism automatically learns to determine the importance weights of different time steps or different features for power load forecasting. In power load forecasting, load data at different time periods has different impacts. For example, load data during weekday morning and evening peak hours is more critical for predicting future peak loads, and different features contribute differently to load. The attention mechanism can focus more on key time points and features, reasonably weighting and adjusting preliminary forecast results, and generating final forecast values that better align with actual power load fluctuations, thereby improving the accuracy and reliability of the model's forecasts.

[0011] In some embodiments, preprocessing the historical data to obtain processed feature data includes:

[0012] Detecting abnormal values in the historical data; processing the abnormal values using a preset replacement strategy to generate corrected data; and normalizing the corrected data to obtain standardized data;

[0013] Based on the standardized data, time series features are extracted, where the time series features include at least lag features and rolling average features; the time series features are fused with external related features to generate processed feature data, where the external related features include at least weather data and holiday data.

[0014] In the above technical solution, acquiring and processing historical power load data is a key step in predictive modeling. First, outliers in the data must be detected and processed. This process uses techniques such as Z-scores, boxplots, K-means clustering, or deep learning autoencoders to identify abnormal fluctuations caused by factors such as sensor failure, recording errors, or extreme weather. After detecting anomalies, corrections are made using strategies such as local mean, median, time series interpolation (such as linear or spline interpolation), or weighted averaging to generate a standardized dataset that stably reflects actual load fluctuations.

[0015] During the data preprocessing stage, normalization is crucial. Minimum-maximum normalization or Z-score normalization can be used to unify features to the scale of [0,1] or the standard normal distribution. This eliminates the interference of dimensional differences on model training, improves numerical computation efficiency, and enhances model optimization stability. Normalized data lays the foundation for feature extraction, and time series features are subsequently extracted to capture the temporal patterns of power load. Time series feature design must include both lagged features (such as load values over the past 1-3 hours reflecting daily periodicity) and rolling average features (such as 24 / 48-hour windows smoothing short-term fluctuations). The two work together to reveal short-term correlations and long-term trends in load.

[0016] The final stage of feature construction requires the integration of external related variables to construct a comprehensive feature matrix. Weather data (meteorological parameters such as temperature, humidity, and wind speed) and holiday data (differences in working day / rest day patterns) have a significant regulatory effect on power load. For example, high temperatures trigger a surge in cooling load, low temperatures trigger heating demand, and the sudden drop in industrial load during holidays and the reversal of residential electricity consumption patterns all require precise modeling. By integrating external features with endogenous time series features, the final generated feature dataset will dynamically capture the spatiotemporal coupling laws of the load, providing multi-dimensional information support for high-precision predictions of complex power systems, significantly improving the model's generalization capabilities and business decision-making value.

[0017] In some embodiments, the feature data is used to construct an ensemble learning model, wherein the ensemble learning model includes at least a combination of multiple base learners, including:

[0018] Select at least two base learners, wherein the base learners include at least a random forest regression and a gradient boosting regression tree;

[0019] For each of the base learners, the feature data is used for training to generate a corresponding sub-model; the parameters of each of the sub-models are optimized by a cross-validation method, wherein the parameters include at least the number of trees and the maximum depth;

[0020] The optimized sub-models are combined to construct the integrated learning model.

[0021] In the above technical solution, random forest regression and gradient boosted regression trees are first selected as base learners when building an ensemble learning model. By constructing multiple decision trees and integrating the prediction results, random forests offer advantages in handling high-dimensional data, resisting overfitting, and enabling parallel computing, enabling them to capture nonlinear relationships in power load data. Gradient boosted regression trees, by sequentially constructing decision trees, correct for errors in the preceding order and excel at handling complex structures and highly correlated data, accurately fitting subtle load fluctuations and long-term trends. The combination of these two approaches aims to complement each other's strengths and enhance model accuracy.

[0022] The two base learners are trained separately using preprocessed feature data to generate sub-models. The random forest constructs a decision tree through random sampling and feature selection, integrating the predictions of each tree. The gradient boosted regression tree trains the decision tree fitting residuals sequentially using gradient descent to optimize the prediction results. The training goal is to ensure that each base learner fully absorbs the effective information in the feature data, forming a sub-model with predictive capabilities.

[0023] Cross-validation is used to optimize the parameters of each sub-model, such as the number of trees and maximum depth. Cross-validation divides the dataset into several subsets, alternating between training and validation, effectively utilizing the data and avoiding overfitting and underfitting. The number of trees in the random forest is optimized to determine the optimal model complexity, while the maximum depth controls the balance between fitting and generalization capabilities. Similarly, the number of trees and maximum depth in the gradient boosting regression tree are optimized to control fitting accuracy, computational cost, and the ability to capture data details. This helps to tap into the best performance of the base learners and improve prediction accuracy.

[0024] Finally, the optimized sub-models are combined to form an ensemble learning model. The prediction results of each sub-model can be aggregated through simple averaging, weighted averaging, or stacking. For example, weights can be assigned to sub-models based on their performance in cross-validation, allowing higher-performing sub-models to play a greater role. This combination can reduce model variance, improve generalization and adaptability, and enable the model to provide more accurate and stable predictions in complex power load fluctuation scenarios.

[0025] In some embodiments, the ensemble learning model is constructed based on the feature data, and the ensemble learning model includes at least a combination of multiple base learners and further includes:

[0026] Analyzing the nonlinear characteristics and periodic characteristics of the characteristic data;

[0027] Selecting a suitable combination of base learners according to the nonlinear characteristics and periodic characteristics;

[0028] Adjusting the hyperparameters of the base learner by a grid search method, wherein the hyperparameters include at least a learning rate and a minimum number of sample splits;

[0029] An integrated learning model suitable for the feature data is constructed based on the adjusted hyperparameters.

[0030] In the above technical solution, in-depth analysis of the nonlinear and cyclical characteristics of power load data is crucial for accurate model construction. Nonlinear feature analysis can be accomplished through observation of data distribution and calculation of nonlinear correlation indicators. Cyclic feature analysis can identify daily, weekly, and seasonal cyclical patterns using autocorrelation functions and Fourier transforms. A combination of base learners, such as random forest regression and gradient boosted regression trees, is selected based on the nonlinear and cyclical characteristics of the feature data. Random forest regression excels at capturing nonlinear relationships and fitting cyclical patterns by leveraging the segmented nature of decision trees. Gradient boosted regression trees learn cyclically varying residuals through iteration, offering complementary advantages.

[0031] Hyperparameter tuning is crucial for improving model performance. Grid search exhaustively searches for hyperparameter combinations, combined with cross-validation evaluation, to find the optimal hyperparameter configuration for the base learner. The learning rate controls the model update step size, requiring a balance between stability and training speed. The minimum number of sample splits constrains the growth of the decision tree, requiring consideration of overfitting risk and generalization ability.

[0032] When building an ensemble learning model, strategies such as simple average voting and weighted averaging can be employed to fuse the optimized base learners. For example, the prediction results of the two can be directly averaged, or weighted according to cross-validation performance and then weighted summed. This approach, which integrates feature analysis, base learner selection, hyperparameter adjustment, and model combination, enables the ensemble learning model to precisely adapt to the characteristics of power load data, improve prediction accuracy and reliability, and effectively support power system operation and scheduling and load forecasting decisions.

[0033] In some embodiments, an attention mechanism is introduced to perform weighted adjustment on the preliminary prediction results to generate a final power load prediction value, including:

[0034] For each base learner in the ensemble learning model, calculating the prediction contribution at different time steps;

[0035] Dynamically adjust the fusion weight of each base learner through the attention mechanism, wherein the fusion weight is determined according to time series characteristics and prediction error;

[0036] Performing weighted combination on the preliminary prediction results according to the fusion weight to generate adjusted prediction data;

[0037] The final power load forecast value is outputted based on the adjusted forecast data.

[0038] In the above technical solution, when introducing the attention mechanism into the ensemble learning model, the prediction contribution of each base learner at different time steps must first be calculated. This is achieved by analyzing the difference (such as the absolute value or square of the residual) between the base learner's prediction and the actual power load value, as well as the correlation. The contribution is high when the residual is small or the trend is consistent, and low when the residual is small.

[0039] Based on this, the attention mechanism dynamically adjusts the fusion weights of each base learner. Fusion weights are determined based on time series characteristics (such as weekday morning and evening peaks and holiday patterns) and prediction errors. Time series characteristics provide guidance on electricity usage characteristics, while prediction errors reflect the accuracy of the base learners. The attention mechanism constructs a weight calculation network or function that automatically learns and dynamically adjusts weights based on these factors, ensuring that the weights of the base learners adapt to their predictive value.

[0040] Based on this fusion weight, the initial prediction results are weighted and combined to generate the adjusted prediction data. Specifically, this is the weighted sum of the initial prediction results of each base learner multiplied by their weights, i.e., adjusted prediction data = Σ(base learner i prediction result × its fusion weight). This allows base learners with better prediction performance to have a greater impact on the final results, improving the accuracy and rationality of the prediction.

[0041] Finally, the power load forecast is output based on the adjusted forecast data. This is the result of optimizing factors such as the contribution of the base learner, time series characteristics, and forecast error. Compared with the initial forecast, the final forecast is more consistent with actual load trends and characteristics, providing a reliable decision-making basis for power system dispatch and operation management, helping to improve system efficiency, economy, and reliability.

[0042] In some embodiments, the historical data is preprocessed to obtain processed feature data, and then the following steps are further included:

[0043] For the feature dataset, an attention mechanism is designed to identify key moments in the time series. By calculating the load change rate, fluctuation amplitude, and periodic deviation of each time step, an attention weight is assigned to each time step. A high attention weight corresponds to a load mutation point, peak, or abnormal pattern.

[0044] According to the calculated attention weight, the feature data set is weighted enhanced, and the original feature vector is multiplied by the attention weight of the corresponding time step to generate attention-enhanced feature data;

[0045] Perform feature importance analysis on the generated attention enhancement feature data, use correlation analysis and feature importance ranking methods to determine the contribution of different features to the prediction, and generate a feature importance ranking table.

[0046] In this technical solution, after preprocessing feature data, an attention mechanism is designed to identify key moments in the time series. By calculating the load change rate, fluctuation amplitude, and periodic deviation at each time step, attention weights are assigned to each time step, with high weights corresponding to load mutation points or abnormal patterns. This allows the model to focus more on these key time steps, improving prediction accuracy.

[0047] The feature dataset is weighted and enhanced based on the attention weights to generate attention-enhanced feature data. This emphasizes key time-step features and mitigates the influence of non-critical features. In power load forecasting, this method enables the model to focus more on key time-step features, such as peak load periods, improving model learning and forecasting performance.

[0048] Finally, feature importance analysis is performed on the attention-enhanced feature data. Through correlation analysis and feature importance ranking, the relevance and importance of features to power load are quantitatively assessed. This helps identify features that are highly correlated with power load, providing a basis for feature selection in the model, improving model efficiency and accuracy, and providing valuable information to power system managers, helping them develop more reasonable scheduling plans and management strategies.

[0049] In some embodiments, an ensemble learning model is constructed based on the feature data, wherein the ensemble learning model includes at least a combination of multiple base learners, including:

[0050] Selecting random forest regression and gradient boosting regression tree as base learners, using the attention-enhanced feature data, and optimizing the parameters of each base learner through a cross-validation method to obtain an optimized base learner model;

[0051] According to the optimized base learner characteristics and feature importance ranking table, different feature subsets are assigned to each base learner to form a feature-model matching matrix, which assigns time series features to random forests and classification features to gradient boosting trees;

[0052] Using training data to evaluate the performance of each optimized base learner in different attention regions, dividing the data into three attention regions: high, medium, and low according to the attention weights, and constructing an attention-model responsibility mapping table;

[0053] The optimized base learners are trained using the attention-enhanced feature data, each base learner generates its own sub-model according to the feature-model matching matrix, and an integrated learning model is constructed.

[0054] In the above technical solution, random forest regression and gradient boosted regression trees are selected as base learners. Both perform well in handling complex data relationships and nonlinear problems. Using attention to enhance feature data training can improve the model's learning of key features. Using cross-validation to optimize the base learner parameters, such as adjusting the number and depth of trees in the random forest and the learning rate and number of iterations in the gradient boosted regression tree, can ensure that the optimized model better fits the training data while avoiding overfitting.

[0055] Based on the optimized base learner characteristics and the feature importance ranking table, different feature subsets are assigned to each base learner to form a feature-model matching matrix. Given the Random Forest's strength in processing time series features, time series features are assigned to it; Gradient Boosted Regression Tree's performance in processing categorical features is also assigned to it. This fully leverages the strengths of each base learner, improves model prediction performance, and lays the foundation for subsequent training and prediction.

[0056] The training data is used to evaluate the performance of each optimized base learner in different attention regions. The data is divided into three attention regions: high, medium, and low. An attention-model responsibility mapping table is constructed to clarify the responsibilities and roles of each base learner in different regions. In high-attention regions, base learners are required to focus on key features to improve prediction accuracy. In medium and low-attention regions, prediction accuracy requirements can be appropriately reduced. This balances the overall performance of the model with computational cost, achieves differentiated management of different base learners, and improves the model's ability to handle data of varying attention levels.

[0057] Using attention-enhanced feature data to train optimized base learners, each base learner generates its own sub-model based on the feature-model matching matrix, ultimately forming an ensemble learning model. During training, each base learner learns and optimizes to generate a sub-model with specific advantages based on its assigned feature subset. The ensemble learning model combines the prediction results of these sub-models, such as through weighted averaging and voting, to obtain the final prediction result. This fully utilizes the advantages of each base learner, compensates for the shortcomings of a single model, and improves the prediction capability of power load data.

[0058] In some embodiments, an attention mechanism is introduced to perform weighted adjustment on the preliminary prediction results to generate a final power load prediction value, including:

[0059] The test data is preprocessed to obtain test feature data, which is then input into each sub-model in the step ensemble learning model. Each sub-model generates its own prediction results to form a sub-prediction result set.

[0060] According to the attention-model responsibility mapping table, a differentiated fusion strategy is applied to the prediction results of each sub-model. For each time step in the test data, its attention weight is calculated, and the attention interval to which it belongs is determined. Then, a higher fusion weight is assigned based on the best model in that interval.

[0061] For each time step, the dynamic fusion weight is calculated based on the historical performance of each sub-model in the corresponding attention interval and the current attention weight. The historical accuracy of the sub-model in the attention interval is multiplied by the attention weight of the current time step, and then normalized to obtain the final weight to form a dynamic weight matrix.

[0062] The dynamic weight matrix is used to perform weighted averaging on the prediction results of each sub-model to obtain the fused power load prediction value; the fused power load prediction value is denormalized and the standardized prediction value is converted back to the original scale to obtain the actual power load prediction value.

[0063] In the above technical solution, test data is preprocessed to match the model input requirements, ensuring that the test data is preprocessed in the same manner as the training data. This results in test feature data, which lays the foundation for subsequent steps. This test feature data is then fed into the sub-models of the ensemble learning model. Each sub-model generates a set of sub-prediction results based on its learned features and patterns. These accurate sub-prediction results serve as reliable input for the fusion process.

[0064] Based on the attention-model responsibility mapping table, a differentiated fusion strategy is used to process the sub-model prediction results at each time step of the test data. Specifically, the attention weight of each time step is calculated to determine the attention interval to which it belongs, and a higher fusion weight is assigned based on the best model in that interval. The attention weight calculation is based on the time series characteristics of the test data (such as load change rate and fluctuation amplitude), thereby fully leveraging the advantages of each base learner in different attention areas, improving the accuracy and reliability of the fusion results.

[0065] For each time step, the dynamic fusion weights are calculated and normalized by combining the sub-model's historical accuracy over the corresponding attention interval with the attention weight at the current time step to form a dynamic weight matrix. The historical accuracy reflects the sub-model's predictive power within a specific attention interval, while the current attention weight highlights the importance of the current time step. This dynamic weight adjustment mechanism allows the fusion results to better adapt to data changes.

[0066] Finally, a dynamic weight matrix is used to perform a weighted average of the prediction results of each sub-model to obtain the fused power load forecast value. This weighted average process involves multiplying the prediction results of each sub-model by their corresponding dynamic fusion weights and then summing them to generate the final forecast result. To make the forecast result more practical, the fused power load forecast value needs to be denormalized, converting it from the standardized scale back to the original scale to obtain the actual power load forecast value. The denormalization method corresponds to the normalization method used in the data preprocessing stage. For example, when using minimum-maximum normalization, the denormalization formula is: actual forecast value = forecast value × (maximum value - minimum value) + minimum value.

[0067] In some embodiments, based on the calculated attention weights, the feature dataset is weighted enhanced, and the original feature vector is multiplied by the attention weights of the corresponding time steps to generate attention-enhanced feature data, including:

[0068] The query vector Q represents the power load forecast demand at the current moment, the key vector K represents the power load data at the historical moment, and the value vector V represents the feature information extracted from the historical data;

[0069] The attention mechanism is used to calculate the correlation between the query vector and the key vectors of all historical moments, dynamically adjust the attention weight of each time point, and multiply the calculated attention weight by the original feature vector to generate attention-enhanced feature data.

[0070] In the above technical solution, the QKV (query vector Q, key vector K, value vector V) representation is used in power load forecasting to effectively link the current forecast demand with historical data. Q represents the current forecast demand, K represents the historical power load data, and V represents the feature information extracted from the historical data. This representation is a common method used in the attention mechanism, capturing the relationship between the current forecast demand and historical data, allowing the attention mechanism to make targeted adjustments to the current forecast based on historical feature information.

[0071] The attention mechanism dynamically adjusts the attention weight for each time point by calculating the correlation between Q and all historical moments K. Specifically, the correlation is measured by calculating the similarity between Q and K (e.g., dot product similarity). The higher the correlation, the greater the attention weight for the corresponding time point. This dynamic weighting allows the model to automatically focus on the historical data points most relevant to the current forecast, improving forecast accuracy and relevance.

[0072] Next, the calculated attention weight is multiplied by the original feature vector V to generate attention-enhanced feature data. After multiplication by the attention weight, the portions of the original feature vector that are highly relevant to the current forecast demand are amplified, while those with less relevance are weakened. This weighted operation makes the feature data more targeted, highlighting historical features that have a significant impact on the current forecast while suppressing the influence of irrelevant or interfering features, providing higher-quality input data for subsequent model training and prediction. In summary, the QKV representation and attention mechanism can dynamically adjust the attention weight of historical data and generate attention-enhanced feature data, thereby improving the performance and prediction accuracy of the power load forecasting model.

[0073] According to another aspect of the present invention, a power load forecasting system based on heterogeneous ensemble learning and attention mechanism is provided, based on the above method; the system includes:

[0074] An acquisition module, configured to acquire historical data of power load, wherein the historical data at least includes time series data;

[0075] A preprocessing module, used to preprocess the historical data to obtain processed feature data;

[0076] A construction module, configured to construct an ensemble learning model based on the feature data, wherein the ensemble learning model comprises at least a combination of multiple base learners;

[0077] A prediction module, configured to predict the feature data using the ensemble learning model to obtain a preliminary prediction result;

[0078] The weighting module is used to introduce an attention mechanism to perform weighted adjustment on the preliminary prediction results to generate a final power load prediction value.

[0079] In the above technical solution, in order to better use the above method, this application proposes a power load forecasting system based on heterogeneous ensemble learning and attention mechanism. Each module corresponds to each step of the above method. The specific principles have been described above and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0081] Figure 1 This is a flow chart of an embodiment of a method for power load forecasting based on heterogeneous ensemble learning and attention mechanism according to the present invention;

[0082] Figure 2 It is a structural diagram of another embodiment of the power load forecasting system based on heterogeneous ensemble learning and attention mechanism of the present invention. DETAILED DESCRIPTION

[0083] The present invention will be described in further detail below with reference to the accompanying drawings and examples. It is particularly noted that the following examples are intended only to illustrate the present invention and are not intended to limit the scope of the present invention. Similarly, the following examples are only some embodiments of the present invention and are not intended to be exhaustive. All other embodiments obtained by those of ordinary skill in the art without creative effort are intended to fall within the scope of protection of the present invention.

[0084] Example 1

[0085] See also Figure 1 , a power load forecasting method based on heterogeneous ensemble learning and attention mechanism, the method comprising:

[0086] S1. Acquire historical data of power load, wherein the historical data at least includes time series data;

[0087] For example, historical load data is obtained from the power system database through the data acquisition interface, including load values, timestamps, and external features such as temperature and humidity recorded every 15 minutes. The timestamps are indexed using the DatetimeIndex of the Pandas library to form a structured data set.

[0088] S2. Preprocess the historical data to obtain processed feature data;

[0089] In this embodiment, outliers are detected in the historical data; the outliers are processed using a preset replacement strategy to generate corrected data; the corrected data are normalized to obtain standardized data; time series features are extracted based on the standardized data, and the time series features include at least lag features and rolling average features; the time series features are fused with external related features to generate the processed feature data, and the external related features include at least weather data and holiday data.

[0090] For example, the cleaned power load historical data set is time-indexed, and a unified time series format is established through timestamp standardization to facilitate subsequent feature extraction operations. Lag features are extracted from the time-indexed power load data, and the load values of the first n time steps are extracted as the prediction input features at the current moment according to the time window setting to form a feature matrix containing time dependencies. The rolling average features are calculated based on the sliding window algorithm. By calculating the mean of the historical load data in different time windows, the load change trend is captured to obtain a feature set reflecting the fluctuation characteristics of different time scales. The seasonal decomposition method is used to separate the trend term, seasonal term and residual term from the original load data, extract periodic features, and enhance the model's ability to recognize periodic load changes. External feature data, including meteorological data, date type, holiday information and other environmental factors, are integrated and fused with the extracted time series features to construct a comprehensive feature set.

[0091] S3. Constructing an ensemble learning model based on the feature data, wherein the ensemble learning model includes at least a combination of multiple base learners;

[0092] In this embodiment, S3, based on the feature data, an ensemble learning model is constructed, wherein the ensemble learning model includes at least a combination of multiple base learners, including:

[0093] S31, selecting at least two base learners, wherein the base learners include at least a random forest regression and a gradient boosting regression tree;

[0094] S32, for each base learner, respectively, using the feature data for training to generate a corresponding sub-model; optimizing the parameters of each sub-model by a cross-validation method, wherein the parameters include at least the number of trees and the maximum depth;

[0095] S33. Combining the optimized sub-models to construct the integrated learning model.

[0096] For example, historical power load data and related features are obtained from the SCADA system. Temporal features include hour, day of the week, and holiday markers, and meteorological features include temperature, humidity, and wind speed. Historical load values are sampled every 15 minutes, and a comprehensive feature set consisting of 100 features is constructed. Missing values are filled using linear interpolation, outliers are removed using the 3σ principle, and features are normalized using the Z-score method to maintain a mean of 0 and a variance of 1. A random forest regression model is constructed with 100 decision trees, each with a maximum depth of 10. The mean squared error (MSE) is used as the splitting criterion, and training subsets are generated through bootstrap sampling. A gradient boosted regression tree is constructed using the XGBoost framework, with a learning rate of 0.1, 200 iterations, and a maximum tree depth of 6. The Huber loss is used as the loss function to enhance robustness. Hyperparameter optimization uses 5-fold cross-validation, a grid search of the max_features parameter of the random forest in the range [0.3, 0.7], and Bayesian optimization of the min_child_weight parameter of the gradient boosted tree in the range [1, 5]. The trained random forest model achieved an RMSE of 45.3MW on the test set, while the gradient boosting tree model achieved an RMSE of 38.7MW. The attention mechanism employed a multi-head self-attention structure with four attention heads and a weight matrix dimension of 64. Attention scores were calculated for the time dimension of historical 24-hour load data, with the weight for peak periods increased to 0.15 (the baseline was 0.05). A dynamic weighting strategy was employed during the model fusion phase, with a weight coefficient of 0.4 for the random forest and 0.6 for the gradient boosting tree. The resulting RMSE for the integrated model was reduced to 32.1MW.

[0097] In this embodiment, S3, constructing an ensemble learning model based on the feature data, wherein the ensemble learning model includes at least a combination of multiple base learners and further includes:

[0098] S34, analyzing the nonlinear characteristics and periodic characteristics of the characteristic data;

[0099] S35. Selecting a suitable combination of base learners according to the nonlinear characteristics and periodic characteristics;

[0100] S36. Adjusting the hyperparameters of the base learner by a grid search method, wherein the hyperparameters include at least a learning rate and a minimum number of sample splits;

[0101] S37. Construct an integrated learning model suitable for the feature data based on the adjusted hyperparameters.

[0102] S4. Predicting the feature data using the ensemble learning model to obtain a preliminary prediction result;

[0103] S5. Introduce an attention mechanism to perform weighted adjustment on the preliminary prediction results to generate a final power load prediction value.

[0104] In this embodiment, S5, an attention mechanism is introduced to perform weighted adjustment on the preliminary prediction results to generate a final power load prediction value, including:

[0105] S51. Calculate the prediction contribution of each base learner in the ensemble learning model at different time steps;

[0106] S52. Dynamically adjust the fusion weight of each base learner through the attention mechanism, where the fusion weight is determined according to time series characteristics and prediction error;

[0107] S53, performing weighted combination on the preliminary prediction results according to the fusion weight to generate adjusted prediction data;

[0108] S54. Output the final power load forecast value based on the adjusted forecast data.

[0109] In this embodiment, historical power load data and related influencing factors, including external factors such as temperature, humidity, and holiday information, are collected. The data is preprocessed using time series analysis methods to remove outliers and fill in missing values. After standardization, a training dataset is obtained. A heterogeneous ensemble learning framework is constructed, combining two base models: random forest regression and gradient boosting regression tree. The preprocessed power load data is trained separately to capture the nonlinear and periodic features in the data. An attention mechanism is introduced to weight the input features at different time steps, enhancing the model's sensitivity to key time points (such as peaks and valleys of load fluctuations), thereby improving the ability to identify patterns in power load fluctuations.

[0110] For example, the prediction results of two base learner models, a random forest regression model and a gradient boosted regression tree, are obtained. For example, the prediction error of the random forest model during stable periods is 5%, while the error of the gradient boosted regression tree is reduced to 8% during peak periods. Feature weights are calculated using an attention mechanism, and the time-step features are normalized using the softmax function. If the load fluctuation at a certain time point exceeds 20% of the historical mean, an attention weight of 0.7 is assigned. When constructing the dynamic fusion weight matrix, if historical data shows that the gradient boosted regression tree's MAE for valley predictions is 3% lower than that of the random forest, it is assigned a weight of 60% during that period. Using time series analysis, an ARIMA model is used to detect the load characteristics of the current period. If the autocorrelation coefficient exceeds 0.5 and the partial autocorrelation coefficient is significant, it is determined to be a peak period. When dynamically adjusting the fusion weights, if the cosine similarity between the current load curve and the historical peak pattern reaches 0.9, the weight of the gradient boosted regression tree is increased to 65%. The weighted average algorithm uses a linear combination approach. For example, if the random forest prediction value is 100kW (weighted 40%) and the gradient boosting regression tree prediction value is 105kW (weighted 60%), the integrated result is 103kW. When applying the attention mechanism correction, an adjustment coefficient of 0.8 is applied to data points in the prediction sequence that exceed ±2 times the standard deviation. Based on the periodicity of power load, a Fourier transform is used to extract the daily cycle component. If a seasonal deviation exceeding 10% is detected, the predicted value is corrected by ±5%. The error feedback mechanism uses a sliding window method. If the RMSE of the random forest in the last five peak predictions decreases by 2%, its weight is increased by 3%.

[0111] Example 2

[0112] Compared with the first embodiment, the difference lies in: in step S2, after pre-processing the historical data to obtain the processed feature data, other steps are added. In step S3, based on the feature data, an ensemble learning model is constructed, and the ensemble learning model includes at least a combination of multiple base learners. In step S5, an attention mechanism is introduced to perform weighted adjustment on the preliminary prediction results to generate the final power load prediction value. The details are as follows;

[0113] In this embodiment, S2, pre-processing the historical data to obtain processed feature data, and then further comprising:

[0114] S21. Design an attention mechanism to identify key moments in the time series based on the feature dataset. By calculating the load change rate, fluctuation amplitude, and periodic deviation at each time step, an attention weight is assigned to each time step. A high attention weight corresponds to a load mutation point, peak, or abnormal pattern.

[0115] In this embodiment, the load change rate sequence is used to calculate the fluctuation amplitude of each time step to obtain the fluctuation amplitude sequence. According to the fluctuation amplitude sequence, the load mutation points and peak values are identified. Through periodic deviation analysis, the deviation value of each time step from the historical periodic data is calculated to obtain a periodic deviation sequence. Combining the fluctuation amplitude sequence and the periodic deviation sequence, the attention weight of each time step is calculated. If the fluctuation amplitude is greater than the threshold, the attention weight of the time step is increased. If the periodic deviation is greater than the threshold, the attention weight of the time step is increased. According to the attention weight, the time step corresponding to the high weight is identified and marked as the critical moment. The critical moment is matched with the load mutation point, peak value and abnormal pattern to determine the key features. A feature data set containing key moments and key features is generated for subsequent model training.

[0116] For example, in time series analysis, an attention mechanism is designed to identify critical moments in power load. First, feature extraction is performed on the original load data, and the load change rate, fluctuation amplitude and periodic deviation of each time step are calculated. For example, for the 24-hour load data of a certain day [3200, 3100, 3050, 3000, 3100, 3400, 3800, 4200, 4500, 4600, 4550, 4500, 4600, 4650, 4600, 4500, 4400, 4300, 4200, 4100, 4000, 3800, 3600, 3400], we calculate the load change rate of adjacent time steps and get [-3.13%, -1.61%, -1.64%, 3.33%, 9.68%, 11.76%, 10.53%, 7.14%, 2.22%, -1.09%, -1.10%, 2.22%, 1.09%, -1.08%, -2.17%, -2.22%, -2.27%, -2.33%, -2.38%, -2.44%, -5.00%, -5.26%, -5.56%]. For the fluctuation amplitude, we use the sliding window variance calculation with a window size of 3 hours and get [58.33, 41.67, 41.67, 208.33, 250.00, 266.67, 233.33, 166.67, 50.00, 33.33, 50.00, 75.00, 41.67, 33.33, 66.67, 66.67, 66.67, 66.67, 66.67, 100.00, 133.33, 100.00]. The periodic deviation is calculated by the difference between the load and the load of the same period of the previous day. Assume that the load of the same period of the previous day is [3150,3050,3000,2950,3050,3350,3750,4150,4450,4550,4500,4450,4550,4600,4550,4450,4350,4250,4150,4050,3950,3750,3550,335 0], the deviations are [1.59%, 1.64%, 1.67%, 1.69%, 1.64%, 1.49%, 1.33%, 1.20%, 1.12%, 1.10%, 1.11%, 1.12%, 1.10%, 1.09%, 1.10%, 1.12%, 1.15%, 1.18%, 1.20%, 1.23%, 1.27%, 1.33%, 1.41%, 1.49%]. Based on these features, we design the attention weight calculation formula: Attention(t) = α·|rate of change(t)|+β·fluctuation amplitude(t)+γ·|periodic deviation(t)|, where α=0.5, β=0.3, and γ=0.2 are weight coefficients.The calculated attention scores are [2.08, 1.53, 1.53, 6.46, 8.47, 9.71, 8.87, 6.79, 2.17, 1.18, 1.67, 2.48, 1.53, 1.18, 2.08, 2.08, 2.08, 2.08, 2.08, 3.03, 3.98, 3.03]. By setting a threshold of 4.0, we identify the critical time points as the 5th hour (3:00-4:00), 6th hour (5:00-6:00), 7th hour (6:00-7:00), 8th hour (7:00-8:00), and 22nd hour (21:00-22:00). These time points correspond to the period of rapid load increase in the morning and the period of significant load decrease in the evening, which are the periods that power grid dispatching needs to pay special attention to.

[0117] S22. Perform weighted enhancement on the feature data set according to the calculated attention weight, multiply the original feature vector by the attention weight of the corresponding time step, and generate attention-enhanced feature data;

[0118] In this embodiment, the query vector Q is used to represent the power load forecast demand at the current moment, the key vector K represents the power load data at the historical moment, and the value vector V represents the feature information extracted from the historical data. The attention mechanism is used to calculate the correlation between the query vector and the key vectors of all historical moments, and the attention weights for each time point are dynamically adjusted. The calculated attention weight is multiplied by the original feature vector to generate attention-enhanced feature data. The enhanced feature data is merged with the original feature data as the input of the ensemble learning model. In the ensemble learning stage, the fusion weights of the random forest regression and gradient boosting regression tree are dynamically adjusted, and the prediction results of the two are combined to generate the final load forecast value. The relevant hyperparameters of the attention mechanism are optimized through grid search and cross-validation methods to improve the model fusion effect. The weight distribution of the attention mechanism between different time steps and base learners is evaluated to further adjust and optimize the model.

[0119] For example, in the process of enhancing feature representation using the attention mechanism, the attention weights for each time step are first obtained, and then these weights need to be applied to the original features for enhancement. In specific implementation, assume that we have a time series dataset containing 100 samples, each sample has 24 time steps, and the feature dimension of each time step is 128. The shape of the attention weight matrix A obtained by the previous calculation is [100, 24], which represents the importance weight of each sample at each time step. We weight the original feature matrix X (shape [100, 24, 128]) with the attention weights. We need to first expand the attention weight matrix A to the shape [100, 24, 1], and then multiply it with the original features through the broadcast mechanism: X_enhanced = X*A_expanded. For example, for the feature vector X[42,15,:] with sample index i=42 and time step t=15, if its corresponding attention weight A[42,15]=0.83, the enhanced feature vector is X_enhanced[42,15,:]=X[42,15,:]*0.83. This amplifies the feature representations of high-attention regions (e.g., time steps with a weight of 0.83), while suppresses the feature representations of low-attention regions (e.g., time steps with a weight of 0.12). Experimental verification shows that this weighting scheme improves the model's F1 score from 0.76 to 0.89 and its accuracy from 0.82 to 0.91 in anomaly detection tasks. Particularly for time series containing sudden events, the enhanced features better capture pattern changes at critical moments. We also found that model performance varied when using different thresholds (e.g., 0.5, 0.7, and 0.9) to filter the attention weights. Ultimately, 0.7 was selected as the optimal threshold, effectively filtering out noise while preserving key information. This attention-enhanced feature representation method not only improves model performance but also enhances model interpretability, allowing us to intuitively understand which time points contribute more to the prediction results.

[0120] S23. Perform feature importance analysis on the generated attention enhancement feature data, use correlation analysis and feature importance ranking method to determine the contribution of different features to the prediction, and generate a feature importance ranking table.

[0121] In this embodiment, the feature_importance_ method of random forest regression is used to calculate the initial feature importance score. If nonlinear features exist, the SHAP value analysis is supplemented. The linear correlation between the attention enhancement feature and the actual load value is calculated through the Pearson correlation coefficient matrix, and features with a significance level of p<0.05 are screened. The feature importance scores of the random forest and gradient boosting regression trees are merged and weighted fused according to a 6:4 weight to generate a preliminary feature importance ranking table. According to the time step weight αt output by the attention mechanism, the importance score of the time series feature is dynamically weighted and adjusted. The recursive feature elimination method is used to verify the fused feature ranking, and redundant features with a contribution below the threshold of 0.01 are eliminated. A structured ranking table containing feature name, importance score, correlation coefficient, and significance mark is constructed. If it is detected that weather features interact with holiday features, a cross-feature importance index is added to the ranking table. The trend of feature importance over time is analyzed through a time series sliding window, and the dynamic importance index is supplemented. Finally, a three-dimensional feature importance ranking table containing static importance, dynamic weight, and correlation verification is output.

[0122] In this embodiment, S3, constructing an ensemble learning model based on the feature data, wherein the ensemble learning model includes at least a combination of multiple base learners, including:

[0123] S39, selecting a random forest regression and a gradient boosting regression tree as base learners, using the attention-enhanced feature data, and optimizing the parameters of each base learner by a cross-validation method to obtain an optimized base learner model;

[0124] In this embodiment, the power load data set is preprocessed, outliers are detected and processed, and data normalization or standardization is performed using the median substitution method or the mean substitution method to ensure that the dimensions of different features are consistent. Time series features are constructed, including lagged features and rolling averages, and other relevant features such as weather and holidays are selected. The random forest regression and gradient boosting regression tree models are initialized, and hyperparameters such as the number of trees, maximum depth, minimum number of samples required for node splitting, and number of randomly selected features are set. The hyperparameters of the random forest regression and gradient boosting regression trees are optimized using the cross-validation method to find the optimal combination. The optimized random forest regression and gradient boosting regression tree models are trained and the prediction results are obtained respectively. The attention mechanism is introduced to calculate the importance of each time step, generate time series features, merge them with the original feature data, and use them as the input of the ensemble learning model. The fusion weights of the random forest regression and gradient boosting regression trees are dynamically adjusted, and the weight coefficients are calculated through the attention mechanism to ensure that the sum of the weight coefficients is 1. The prediction results of random forest regression and gradient boosting regression tree are weighted averaged to obtain the final load forecast value to evaluate the model performance. The root mean square error and mean absolute error are used as evaluation indicators to analyze the deviation between the predicted value and the actual value and the average level of error.

[0125] For example, in the model building phase, we selected random forest regression and gradient boosting regression tree as base learners, and used attention-enhanced feature data for training. First, the parameters of the random forest regressor were optimized, and the performance of different parameter combinations was explored through grid search combined with 5-fold cross-validation. The specific parameter grid is set to: n_estimators (the number of trees) is in the range of [100, 200, 300], max_depth (maximum depth) is in the range of [10, 20, 30], and min_samples_split (the minimum number of samples required for internal node re-division) is in the range of [2, 5, 10]. After optimization, the best parameter combination of random forest is n_estimators = 200, max_depth = 20, min_samples_split = 5. At this time, the root mean square error (RMSE) on the validation set is 0.0823, R 2 The gradient boosting regression tree was also optimized with the following parameters: n_estimators in the range of [100, 200, 300], learning_rate in the range of [0.05, 0.1, 0.2], and max_depth in the range of [3, 5, 7]. The optimal parameters after optimization were n_estimators = 200, learning_rate = 0.1, and max_depth = 5. The RMSE on the validation set was 0.0765, and R 2The performance of the two base learners on the validation set was 0.9124. By comparing the performance of the two base learners on the validation set, the gradient boosting regression tree performed slightly better than the random forest regressor. To further improve the model performance, we integrated the two optimized base learners and used the simple averaging method to fuse the prediction results of the two models. Finally, we obtained an RMSE of 0.0742 and R 2 The performance is 0.9187, which is significantly improved compared to the single model. This shows that the combination of attention-enhanced features and ensemble learning methods can effectively capture complex patterns in the data and improve prediction accuracy.

[0126] S310, assigning different feature subsets to each base learner according to the optimized base learner characteristics and feature importance ranking table to form a feature-model matching matrix, assigning time series features to the random forest, and assigning classification features to the gradient boosting tree;

[0127] In this embodiment, a feature importance analysis method is used to score the importance of features in the power load dataset and obtain an importance ranking for each feature. Based on the results of the feature importance analysis, time series features and classification features are screened out. The time series features are assigned to the random forest regression to take advantage of its ability to process high-dimensional data and nonlinear features. The classification features are assigned to the gradient boosting regression tree to take advantage of its sensitivity to classification features. A feature-model matching matrix is constructed to clarify the feature subset corresponding to each base learner. The feature-model matching matrix ensures that the random forest regression processes the time series features and the gradient boosting regression tree processes the classification features. Based on the feature-model matching matrix, the training dataset is reorganized to generate independent training sets for the random forest regression and gradient boosting regression tree. The independent training sets are used to train the random forest regression and gradient boosting regression tree respectively to obtain optimized base learners. The prediction results of the optimized base learners are weighted and fused to generate the final power load forecast value.

[0128] For example, based on the optimized base learner properties and feature importance analysis, we constructed a feature-model matching matrix to improve ensemble model performance. First, feature importance analysis revealed that time series features contributed 65.3% to the random forest model, while categorical features contributed 58.7% to the gradient boosted tree. Based on this, time series features were assigned to the random forest model, while categorical features were assigned to the gradient boosted tree. During implementation, we used a feature selection algorithm to calculate mutual information scores, with an average score of 0.42 for time series features and 0.37 for categorical features. We then used grid search to optimize the random forest parameters (n_estimators = 120, max_depth = 15, min_samples_split = 8), achieving optimal performance when processing time series features, with the F1 score increasing from 0.78 to 0.83. We also set parameters for the gradient boosted tree that were more suitable for processing categorical features (learning_rate = 0.05, n_estimators = 150, max_depth = 6), improving accuracy by 7.2 percentage points. Through this feature-model matching strategy, the overall prediction accuracy of the integrated model increased from 82.5% to 89.3%, and the ROC-AUC value increased from 0.856 to 0.912.

[0129] S311, using the training data to evaluate the performance of each optimized base learner in different attention areas, dividing the data into three attention areas of high, medium, and low according to the attention weights, and constructing an attention-model responsibility mapping table;

[0130] In this embodiment, an attention mechanism is used to calculate the attention weight of each time step, and the data is divided into three attention areas: high, medium, and low according to the weights. The performance of random forest regression and gradient boosting regression tree in different attention areas is evaluated through training data. If the random forest performs better in the high volatility area, it is given a larger weight in this area; if the gradient boosting regression tree performs better in the stable area, it is given a larger weight in this area. An attention-model responsibility mapping table is constructed to record the optimal weight distribution of each base learner in different attention areas. The fusion weight of the base learner is dynamically adjusted according to the mapping table to generate the final load forecast result.

[0131] For example, to evaluate the performance of each optimized base learner in different attention regions, we first divided the training data into three attention regions: high, medium, and low, based on the calculated attention weights. Specifically, we calculated the volatility of a certain power load time series data, sorted the volatility, and took the top 30% as the high attention region (average volatility 0.023), the middle 40% as the medium attention region (average volatility 0.012), and the bottom 30% as the low attention region (average volatility 0.005). We then evaluated the performance of the random forest, gradient boosting tree, and LSTM models in each region. Evaluation results show that Random Forest achieves a mean squared error of 0.0018 in high-attention regions, significantly outperforming Gradient Boosted Trees (0.0027) and LSTMs (0.0023). Gradient Boosted Trees achieve a mean squared error of 0.0008 in low-attention regions, outperforming Random Forests (0.0013) and LSTMs (0.0011). LSTMs perform best in medium-attention regions, achieving a mean squared error of 0.0015, lower than Random Forests (0.0019) and Gradient Boosted Trees (0.0017). Based on these results, an attention-model responsibility mapping table was constructed: Random Forests are assigned a weight of 0.6 for high-attention regions, Gradient Boosted Trees (0.2), and LSTMs (0.2). Medium-attention regions are assigned a weight of 0.5 for LSTMs, a weight of 0.3 for Random Forests, and a weight of 0.2 for Gradient Boosted Trees. Low-attention regions are assigned a weight of 0.5 for Gradient Boosted Trees, a weight of 0.3 for LSTMs, and a weight of 0.2 for Random Forests. This dynamic weight allocation mechanism enables the ensemble model to adaptively adjust the contribution of each base learner based on market conditions, thereby maintaining high forecast accuracy under varying market conditions. Validated on test data from January to June 2022, the dynamic ensemble model using the attention mechanism reduced forecast error by 18.7% compared to the fixed-weight ensemble model. The improvement in forecast accuracy was particularly significant during periods of significant power load fluctuations, reaching 23.5%.

[0132] S312. Use the attention-enhanced feature data to train the optimized base learners, each base learner generates its own sub-model according to the feature-model matching matrix, and constructs an integrated learning model.

[0133] For example, a parameter optimization space is constructed, and the number of trees for random forest regression is defined as [100, 500], the maximum depth is defined as [5, 20], and the minimum number of samples for node splitting is defined as [2, 10]. The number of trees for gradient boosting regression is defined as [50, 300], the maximum depth is defined as [3, 15], and the learning rate is defined as [0.01, 0.2]. The number of attention mechanism heads is set to 4, the dimension is set to 64, and the dropout rate is set to 0.1. The k-fold cross-validation method is used to evaluate the performance of the parameter combination on the validation set, and the parameter combination with the smallest RMSE is selected as the final model parameter.

[0134] In this embodiment, S5, an attention mechanism is introduced to perform weighted adjustment on the preliminary prediction results to generate a final power load prediction value, including:

[0135] S55, preprocessing the test data to obtain test feature data, and inputting the data into each sub-model in the step ensemble learning model, so that each sub-model generates its own prediction result to form a sub-prediction result set;

[0136] S56. Apply a differentiated fusion strategy to the prediction results of each sub-model according to the attention-model responsibility mapping table. For each time step in the test data, calculate its attention weight, determine the attention interval to which it belongs, and then assign a higher fusion weight based on the best model in the interval.

[0137] For example, a sliding window method is used to generate 24-hour lagged features and 7-day rolling average features to form a time series feature vector. The ADF test is used to verify the stationarity of the features, and the non-stationary features are differentially processed. The head attention mechanism is initialized, a 64-dimensional attention space is set, and the correlation score between the key vector of the historical moment and the current query vector is calculated. Based on the correlation score, the prediction results of the random forest and gradient boosting regression tree are dynamically weighted to generate a preliminary fusion result. The attention weight interval is divided into [0-0.3)

[0138] [0.3-0.7) and [0.7-1], establish a mapping relationship between intervals and model responsibilities. If the current time step attention weight falls into the [0-0.3) interval, the gradient boosting regression tree weight is assigned 0.8 and the random forest weight is assigned 0.27. If the current time step attention weight falls into the [0.3-0.7) interval, the default 6:4 base model fusion ratio is maintained. If the current time step attention weight falls into the [0.7-1] interval, the random forest weight is assigned 0.75 and the gradient boosting regression tree weight is assigned 0.259. The weighted prediction value is calculated based on the final fusion weight, and the load forecast result matrix with time series labels is output.

[0139] S57. For each time step, calculate the dynamic fusion weight based on the historical performance of each sub-model in the corresponding attention interval and the current attention weight. Multiply the historical accuracy of the sub-model in the attention interval by the attention weight of the current time step, and then normalize to obtain the final weight to form a dynamic weight matrix.

[0140] Exemplarily, the attention mechanism is used to calculate the attention weight of the current time step, and the dynamic weight coefficient is generated by combining the time series features. The historical accuracy of the sub-model in the corresponding attention interval is multiplied by the attention weight of the current time step to obtain the preliminary weight value. The preliminary weight value is normalized to generate the dynamic fusion weight of each sub-model at the current time step. A dynamic weight matrix is constructed to store the fusion weight of each sub-model at different time steps. The dynamic weight matrix is weightedly fused with the prediction results of the base learner to generate the final prediction value. The attention mechanism parameters are adjusted according to the error distribution, and the weight allocation strategy is optimized. The feature enhancement input is combined with the dynamic weight matrix to improve the model's sensitivity to key time points. A model fusion strategy is adopted to combine the prediction results of random forest regression and gradient boosting regression tree to capture the long-term dependencies in the load data through time series feature extraction.

[0141] For example, in the dynamic fusion model, we first need to calculate the historical performance of each sub-model in different attention intervals. Assume there are three sub-models (M1, M2, M3) and five attention intervals (0-0.2, 0.2-0.4, 0.4-0.6, 0.6-0.8, 0.8-1.0). Through historical data analysis, we obtain the accuracy matrix H of each sub-model in each interval: M1's accuracy in the five intervals is [0.75, 0.82, 0.68, 0.59, 0.51], M2's is [0.62, 0.71, 0.85, 0.79, 0.66], and M3's is [0.58, 0.64, 0.72, 0.88, 0.93]. When the system receives new input data, the attention mechanism calculates the attention weight vector A_t = [0.05, 0.15, 0.35, 0.30, 0.15] for the current time step t, representing the importance of each attention interval. Next, we calculate the dynamic fusion weight W_t by weighting the historical accuracy matrix H with the attention weight vector A_t. For sub-model M1, its comprehensive score is 0.75 × 0.05 + 0.82 × 0.15 + 0.68 × 0.35 + 0.59 × 0.30 + 0.51 × 0.15 = 0.6545. Similarly, the comprehensive score for M2 is 0.7465, and the score for M3 is 0.7585. After normalizing these scores, we obtain the final dynamic fusion weight W_t = [0.3023, 0.3448, 0.3529]. During the prediction phase, the system weights and fuses the outputs of each sub-model according to W_t to produce the final prediction. This method adaptively adjusts the weights of each sub-model based on the attention distribution in different scenarios, leveraging the strengths of each model in different attention intervals and improving overall prediction accuracy. Experimental verification shows that compared to fixed-weight fusion, this dynamic fusion method improves accuracy on the test set from 83.2% to 87.6%, particularly in scenarios with large variations in attention distribution.

[0142] S58. Use the dynamic weight matrix to perform weighted averaging on the prediction results of each sub-model to obtain a fused power load prediction value; perform denormalization on the fused power load prediction value, convert the standardized prediction value back to the original scale, and obtain the actual power load prediction value.

[0143] In this embodiment, random forest regression and gradient boosted regression trees are used to predict the time series feature matrix, generating prediction results for each sub-model. The attention mechanism calculates the correlation between the query vector and the key vector to generate an attention mechanism weight matrix. Based on the attention mechanism weight matrix, the fusion weights of the random forest regression and gradient boosted regression trees are dynamically adjusted. The prediction results of the random forest regression and gradient boosted regression trees are weighted with the dynamic fusion weights to generate a fused power load forecast value. If the deviation between the fused prediction value and the actual value exceeds a preset threshold, the attention mechanism weight matrix is readjusted. The calculation parameters of the attention mechanism weight matrix are optimized through grid search and cross-validation methods. The optimized attention mechanism weight matrix is applied to the new time series feature matrix to generate new dynamic fusion weights. The prediction results of the random forest regression and gradient boosted regression trees are weighted using the new dynamic fusion weights to generate the final power load forecast value. Based on the final power load forecast value, the root mean square error and mean absolute error are calculated to evaluate model performance.

[0144] As an example, in practical applications, we employed a dynamic fusion method based on an attention mechanism for power load forecasting. First, we collected historical load data, meteorological data, and date feature data for a specific region from January to December 2022. After data cleaning and standardization, we constructed an input matrix containing features such as temperature, humidity, wind speed, and date type. Next, we extracted the temporal characteristics of the load series, calculating the average weekday load of 35.6 MW and the average weekend load of 28.3 MW. We also identified seasonal patterns, with peak load reaching 42.7 MW in the summer. We then constructed two sub-forecasting models: a random forest model and a gradient boosted regression tree model. For each sub-model, we trained it using the first 11 months of data and tested it on the last month's data to obtain the respective forecast results. The random forest model performed well for peak load forecasting, with an average error of 2.3%. The gradient boosted regression tree performed well for valley load forecasting, with an average error of 1.8%. Next, we designed an attention network consisting of two fully connected layers (64 and 32 neurons, respectively) and a softmax activation function. Its input is the feature vector at the current moment, and its output is the dynamic weights of the two sub-models. Through training, we obtained weight matrices tailored to different load conditions. For example, during weekday peak hours, the weights for the random forest model and gradient boosted regression tree model were 0.65 and 0.35, respectively; while during weekend off-peak hours, the weights were 0.32 and 0.68, respectively. Finally, we applied these dynamic weights to the predictions of each sub-model. For example, at 2:00 PM on a weekday, the random forest model predicted 38.5MW, while the gradient boosted regression tree model predicted 38.2MW. After fusion of the dynamic weights, the final prediction was 38.3MW, a difference of only 0.26% from the actual value of 38.4MW, significantly outperforming the predictions of a single model. This dynamic fusion method, based on the attention mechanism, adaptively adjusts the weights of each model based on the load characteristics of different time periods, effectively improving the accuracy and robustness of power load forecasting.

[0145] In this embodiment, S22 performs weighted enhancement on the feature data set according to the calculated attention weight, multiplies the original feature vector by the attention weight of the corresponding time step, and generates attention-enhanced feature data, including:

[0146] S221, using the query vector Q to represent the current power load forecast demand, the key vector K to represent the power load data at the historical moment, and the value vector V to represent the feature information extracted from the historical data;

[0147] S222. Calculate the correlation between the query vector and the key vectors of all historical moments through the attention mechanism, dynamically adjust the attention weight of each time point, multiply the calculated attention weight by the original feature vector, and generate attention-enhanced feature data.

[0148] In this embodiment, the query vector Q represents the current power load forecast demand, the key vector K represents the power load data at historical moments, and the value vector V represents the feature information extracted from the historical data. An attention mechanism is used to calculate the correlation between the query vector and the key vectors at all historical moments, dynamically adjusting the attention weights for each time point. The calculated attention weights are multiplied by the original feature vector to generate attention-enhanced feature data.

[0149] For example, in the process of enhancing feature representation using the attention mechanism, we first obtain the attention weights for each time step, and then need to apply these weights to the original features for enhancement. In specific implementation, assume that we have a time series dataset containing 100 samples, each sample has 24 time steps, and the feature dimension of each time step is 128. The shape of the attention weight matrix A obtained by the previous calculation is [100, 24], which represents the importance weight of each sample at each time step. We weight the original feature matrix X (shape [100, 24, 128]) with the attention weights. We need to first expand the attention weight matrix A to the shape [100, 24, 1], and then multiply it with the original features through the broadcast mechanism: X_enhanced = X*A_expanded.

[0150] For example, for the feature vector X[42,15,] with sample index i=42 and time step t=15, if its corresponding attention weight A[42,15]=0.83, the enhanced feature vector is X_enhanced[42,15,]=X[42,15,]*0.83. This amplifies the feature representations of high-attention regions (e.g., time steps with a weight of 0.83), while suppresses the feature representations of low-attention regions (e.g., time steps with a weight of 0.12). Experimental verification shows that this weighting scheme improves the model's F1 score from 0.76 to 0.89 and its accuracy from 0.82 to 0.91 in anomaly detection tasks. Particularly for time series containing sudden events, the enhanced features better capture pattern changes at critical moments. We also found that model performance varied when using different thresholds (e.g., 0.5, 0.7, and 0.9) to filter the attention weights. Ultimately, 0.7 was selected as the optimal threshold, effectively filtering out noise while preserving key information.

[0151] Embodiment 3

[0152] See also Figure 2 , a power load forecasting system based on heterogeneous ensemble learning and attention mechanism, based on the method described in Embodiment 1 and Embodiment 2; the system includes:

[0153] An acquisition module, configured to acquire historical data of power load, wherein the historical data at least includes time series data;

[0154] A preprocessing module, used to preprocess the historical data to obtain processed feature data;

[0155] A construction module, configured to construct an ensemble learning model based on the feature data, wherein the ensemble learning model comprises at least a combination of multiple base learners;

[0156] A prediction module, configured to predict the feature data using the ensemble learning model to obtain a preliminary prediction result;

[0157] The weighting module is used to introduce an attention mechanism to perform weighted adjustment on the preliminary prediction results to generate a final power load prediction value.

[0158] In the above technical solution, in order to better use the methods described in Example 1 and Example 2, this application proposes a power load forecasting system based on heterogeneous ensemble learning and attention mechanism. Each module corresponds to each step of the above method. The specific principles have been described above and will not be repeated here.

[0159] The above descriptions are only some embodiments of the present invention and do not limit the scope of protection of the present invention. Any equivalent device or equivalent process transformation made by using the contents of the description and drawings of the present invention, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A power load forecasting method based on heterogeneous ensemble learning and attention mechanism, characterized in that: The method comprises: Acquiring historical data of power load, wherein the historical data at least includes time series data; Preprocessing the historical data to obtain processed feature data; Based on the feature data, construct an ensemble learning model, wherein the ensemble learning model includes at least a combination of multiple base learners; Predicting the feature data using the ensemble learning model to obtain a preliminary prediction result; An attention mechanism is introduced to perform weighted adjustment on the preliminary prediction results to generate the final power load prediction value.

2. The power load forecasting method based on heterogeneous ensemble learning and attention mechanism according to claim 1, characterized in that: Preprocessing the historical data to obtain processed feature data includes: Detecting abnormal values in the historical data; processing the abnormal values using a preset replacement strategy to generate corrected data; and normalizing the corrected data to obtain standardized data; Based on the standardized data, time series features are extracted, where the time series features include at least lag features and rolling average features; the time series features are fused with external related features to generate processed feature data, where the external related features include at least weather data and holiday data.

3. The power load forecasting method based on heterogeneous ensemble learning and attention mechanism according to claim 1, characterized in that: The feature data is used to construct an integrated learning model, wherein the integrated learning model includes at least a combination of multiple base learners, including: Select at least two base learners, wherein the base learners include at least a random forest regression and a gradient boosted regression tree; For each of the base learners, the feature data is used for training to generate a corresponding sub-model; the parameters of each of the sub-models are optimized by a cross-validation method, wherein the parameters include at least the number of trees and the maximum depth; The optimized sub-models are combined to construct the integrated learning model.

4. The power load forecasting method based on heterogeneous ensemble learning and attention mechanism according to claim 3 is characterized in that: The integrated learning model is constructed based on the feature data, wherein the integrated learning model includes at least a combination of multiple base learners and further includes: Analyzing the nonlinear characteristics and periodic characteristics of the characteristic data; Selecting a suitable combination of base learners according to the nonlinear characteristics and periodic characteristics; Adjusting the hyperparameters of the base learner by a grid search method, wherein the hyperparameters include at least a learning rate and a minimum number of sample splits; An integrated learning model suitable for the feature data is constructed based on the adjusted hyperparameters.

5. The power load forecasting method based on heterogeneous ensemble learning and attention mechanism according to claim 1, characterized in that: The attention mechanism is introduced to perform weighted adjustment on the preliminary prediction results to generate the final power load forecast value, including: For each base learner in the ensemble learning model, calculating the prediction contribution at different time steps; Dynamically adjust the fusion weight of each base learner through the attention mechanism, wherein the fusion weight is determined according to time series characteristics and prediction error; Performing weighted combination on the preliminary prediction results according to the fusion weight to generate adjusted prediction data; The final power load forecast value is outputted based on the adjusted forecast data.

6. The power load forecasting method based on heterogeneous ensemble learning and attention mechanism according to claim 1, characterized in that: Preprocessing the historical data to obtain processed feature data, and then further comprising: For the feature dataset, an attention mechanism is designed to identify key moments in the time series. By calculating the load change rate, fluctuation amplitude, and periodic deviation of each time step, an attention weight is assigned to each time step. A high attention weight corresponds to a load mutation point, peak, or abnormal pattern. According to the calculated attention weight, the feature data set is weighted enhanced, and the original feature vector is multiplied by the attention weight of the corresponding time step to generate attention-enhanced feature data; Perform feature importance analysis on the generated attention enhancement feature data, use correlation analysis and feature importance ranking methods to determine the contribution of different features to the prediction, and generate a feature importance ranking table.

7. The method for power load forecasting based on heterogeneous ensemble learning and attention mechanism according to claim 6, characterized in that: Based on the feature data, an ensemble learning model is constructed, wherein the ensemble learning model includes at least a combination of multiple base learners, including: Selecting random forest regression and gradient boosting regression tree as base learners, using the attention-enhanced feature data, and optimizing the parameters of each base learner through a cross-validation method to obtain an optimized base learner model; According to the optimized base learner characteristics and feature importance ranking table, different feature subsets are assigned to each base learner to form a feature-model matching matrix, which assigns time series features to random forests and classification features to gradient boosting trees; Using training data to evaluate the performance of each optimized base learner in different attention regions, dividing the data into three attention regions: high, medium, and low according to the attention weights, and constructing an attention-model responsibility mapping table; The optimized base learners are trained using the attention-enhanced feature data, each base learner generates its own sub-model according to the feature-model matching matrix, and an integrated learning model is constructed.

8. The power load forecasting method based on heterogeneous ensemble learning and attention mechanism according to claim 7, characterized in that: The attention mechanism is introduced to perform weighted adjustment on the preliminary prediction results to generate the final power load forecast value, including: The test data is preprocessed to obtain test feature data, which is then input into each sub-model in the step ensemble learning model. Each sub-model generates its own prediction results to form a sub-prediction result set. According to the attention-model responsibility mapping table, a differentiated fusion strategy is applied to the prediction results of each sub-model. For each time step in the test data, its attention weight is calculated, and the attention interval to which it belongs is determined. Then, a higher fusion weight is assigned based on the best model in that interval. For each time step, the dynamic fusion weight is calculated based on the historical performance of each sub-model in the corresponding attention interval and the current attention weight. The historical accuracy of the sub-model in the attention interval is multiplied by the attention weight of the current time step, and then normalized to obtain the final weight to form a dynamic weight matrix. The dynamic weight matrix is used to perform weighted averaging on the prediction results of each sub-model to obtain the fused power load prediction value; the fused power load prediction value is denormalized and the standardized prediction value is converted back to the original scale to obtain the actual power load prediction value.

9. The power load forecasting method based on heterogeneous ensemble learning and attention mechanism according to claim 8, characterized in that: According to the calculated attention weights, the feature data set is weighted enhanced, and the original feature vector is multiplied by the attention weight of the corresponding time step to generate attention-enhanced feature data, including: The query vector Q represents the power load forecast demand at the current moment, the key vector K represents the power load data at the historical moment, and the value vector V represents the feature information extracted from the historical data; The attention mechanism is used to calculate the correlation between the query vector and the key vectors of all historical moments, dynamically adjust the attention weight of each time point, and multiply the calculated attention weight by the original feature vector to generate attention-enhanced feature data.

10. A power load forecasting system based on heterogeneous ensemble learning and attention mechanism, characterized in that: Based on the method according to any one of claims 1 to 9; the system comprises: An acquisition module, configured to acquire historical data of power load, wherein the historical data at least includes time series data; A preprocessing module, used to preprocess the historical data to obtain processed feature data; A construction module, configured to construct an ensemble learning model based on the feature data, wherein the ensemble learning model comprises at least a combination of multiple base learners; A prediction module, configured to predict the feature data using the ensemble learning model to obtain a preliminary prediction result; The weighting module is used to introduce an attention mechanism to perform weighted adjustment on the preliminary prediction results to generate a final power load prediction value.

Images