Integrated energy system short-term load prediction method based on improved multi-task model

Abstract

An integrated energy system short-term load prediction method based on an improved multi-task model includes the steps that in the input stage, the MIC is adopted to analyze the correlation degree of multi-element loads and weather factors, and STL is used for decomposing a load sequence to construct a component data set; in the modeling stage, the CNN is used as a feature extraction network, the spatial coupling relation of an input sequence is directly learned, the general representation of the multi-element load is learned by applying a hierarchical progressive routing strategy, and specific parameter information of different tasks is separated; in the output stage, modeling is conducted on time sequence characteristics through combination of LSTM, multi-element load prediction is achieved, aiming at the complex nonlinear relation between loads, a CPLE joint prediction model is established through combination of the characteristics of CNN and LSTM networks and a PLE multi-task learning method, and while multiple networks are adopted to learn the space-time coupling characteristics between the multi-element loads, the multi-element load prediction efficiency is improved. And different load specific knowledge is highlighted by using a progressive routing mode.

Description

Technical Field

[0001] This invention relates to the technical field of multi-dimensional load forecasting for integrated energy systems, specifically to a short-term load forecasting method for integrated energy systems based on an improved multi-task model. Background Technology

[0002] Integrated Energy Systems (IES), as an important component of new power systems, enable the coupling of different types of energy at various stages such as source, grid, and load, thereby increasing the penetration rate of renewable energy in the power system. Accurate and efficient forecasting of diverse loads within IESs greatly facilitates the development of IES planning schemes and the optimization of operational strategies. However, due to the diverse load types within IESs and the complex coupling relationships between energy subsystems, research on multi-load forecasting methods for IESs is necessary.

[0003] Traditional load forecasting primarily falls into two categories: statistical methods and artificial intelligence methods, mainly based on the forecasting object and focusing on single-energy systems. Statistical methods mainly include regression analysis, exponential smoothing, and Kalman filtering. These methods are simple in principle and computationally efficient, but they are weak in handling nonlinear systems and unsuitable for highly volatile load sequences. Artificial intelligence methods mainly include Support Vector Machines (SVM), Convolutional Neural Networks (CNN), and Long Short-Term Memory (LSTM). However, these methods suffer from drawbacks such as relatively simple models, loss of temporal information, and insufficient multi-dimensional feature extraction when faced with long-term series or multi-dimensional input data. To address these issues, some researchers have used combined models to learn complex variable changes. Combined models can combine the advantages of multiple models, improving the accuracy and generalization of forecasts. Although combined model methods have achieved good results in load forecasting of single power systems, direct application to IES (Integrated Energy Systems) is difficult to achieve accurate and reliable results. IES loads differ from traditional single-load patterns. Modeling and forecasting each type of load separately would ignore the coupling characteristics between energy sources, limiting the forecasting effectiveness. Therefore, multi-energy load forecasting under IES must fully consider the mutual conversion between energy sources.

[0004] Compared to single load forecasting, research on multi-energy load forecasting is still in its early stages. Currently, it can be broadly divided into two categories: one is independent forecasting of multiple loads. This approach considers the interaction characteristics between various load types, but the high complexity of the model makes parameter tuning difficult and training time long. Furthermore, it requires building a separate forecasting model for each type of load, leading to increased computational and maintenance costs. The other category is joint forecasting of multiple loads through a shared model. This type of model learns the complex coupling information between multiple loads by sharing parameters, improving forecasting efficiency and accuracy. However, the multi-task methods used are all based on a "hard-connected" approach, which forces all information to be shared without considering the conflicts caused by differences between sub-tasks. When the correlation between some tasks is weak, the model training results are difficult to achieve ideal levels. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes a short-term load forecasting method for integrated energy systems based on an improved multi-task model. It overcomes the complex and highly volatile nature of energy interactions within an integrated energy system (IES), as well as the shortcomings of hard sharing.

[0006] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0007] A short-term load forecasting method for integrated energy systems based on an improved multi-task model, characterized by the following steps:

[0008] First, outliers are identified using the quartile method, and missing or outlier data are replaced using linear interpolation.

[0009] Then, the seasonal and trend decomposition algorithm STL based on Local Weighted Regression LOESS is used to decompose the cold, heat, and electricity load data to obtain three components: trend, period, and residual.

[0010] Next, maximum information coefficient (MIC) analysis was performed on the load components and meteorological characteristics, and corresponding input sequences were constructed for each type of component in combination with time characteristics;

[0011] The input features are then normalized, and the Convolutional Progressive Hierarchical Extraction (CPLE) model is used to mine the differences and correlations among the multivariate loads, enabling the prediction of the three components.

[0012] Finally, the multi-energy load forecast is obtained by reconstructing the three components.

[0013] 2. The short-term load forecasting method for an integrated energy system based on an improved multi-task model according to claim 1, characterized in that:

[0014] (1) Input data processing

[0015] (a) Load data decomposition

[0016] STL is chosen for decomposition, and the decomposition formula of STL is expressed as equation (1):

[0017] Y t =T t +S t +R t (1)

[0018] Among them, Y t T represents the raw data at time t. t S t R t The tables represent the trend, seasonal, and residual terms at time t. The calculation process of the seasonality and trend decomposition algorithm STL based on LOESS consists of an inner loop and an outer loop. The inner loop decomposes the trend and seasonal terms through local weighted regression smoothing operations, while the outer loop adjusts the robust weights based on the results of the inner loop to reduce the impact of outliers on the next inner loop iteration.

[0019] The STL multiplication form can be implemented using the log function, as shown in equation (2):

[0020] log Y t =log T t +log S t +log R t (2)

[0021] STL is performed on the electrical, cooling, and heating load sequences according to equation (2).

[0022] STL decomposes the IES electricity, heat, and cooling load series into three subsequences: trend, seasonal, and residual. The trend sequence, a low-frequency component, represents the long-term trend and direction of load changes. The seasonal sequence, a high-frequency component, reflects the load changes with the seasons, i.e., the cyclical variation of the load. The residual sequence, the part remaining after subtracting the trend and cyclical sequences from the original sequence, represents the random fluctuations in the load data that are not explained by trend and seasonal effects.

[0023] (b) Feature sequence selection

[0024] This invention selects the Maximum Information Coefficient (MIC) method to analyze the correlation between multi-energy load components and meteorological factors.

[0025] The value of MIC ranges from [0,1]. The larger the value, the deeper the correlation. The specific calculation process is shown in equations (3) and (4).

[0026]

[0027]

[0028] In the formula, n is the number of samples, a and b represent the number of grids in the x and y directions of the variables in the two-dimensional coordinate system, respectively, and B(n) is a function of the sample size, usually set to n. 0.6 I(x,y) represents the mutual information between variables x and y, p(x) and p(y) are the probability density distributions of x and y, respectively, and p(x,y) is the joint probability density distribution of x and y.

[0029] Based on the above formula, the minimum interconversion ratio (MIC) values ​​between each component of cold, heat, and electricity load in both seasons and weather data were calculated, and the results were plotted into a heat map. MIC values ​​greater than 0.2 were considered strong correlations, and values ​​less than 0.1 were considered weak correlations.

[0030] Following the STL (Standardized Tolerance) index, the trend terms of electricity, heat, and cooling loads showed strong correlations, while the correlations among the remaining terms were weaker. The degree of correlation between load trend terms also varied across seasons. In summer, the correlation between electricity and cooling load trend terms was higher, while the correlation with heat load trend terms was lower, whereas the opposite was true in non-summer seasons. This aligns with actual conditions. Besides the mutual influence among multiple load components, loads are also sensitive to meteorological fluctuations. The impact of weather on load components differs across seasons. For example, multi-load trend terms showed strong correlations with dew point temperature and humidity in summer, while in non-summer seasons they were more strongly correlated with temperature, with MIC values ​​greater than 0.2. The above analysis indicates that different meteorological characteristics should be selected for input features for different load components in different seasons.

[0031] (c) Input / output feature set settings

[0032] Besides weather factors, input features also need to consider historical load and time encoding. The most recent historical load near the time to be predicted contains rich prior information, which is helpful for model training when used as input features. Time encoding is also an important feature for key discrimination of load sequences. Prediction models are constructed for the trend, periodicity, and residual components of multivariate loads. When predicting the multivariate load at time step t, the specific input / output feature sets of each model are shown in Table 1. Among them, the hourly feature corresponds to the encoding 0-23, and the weekday feature is set to 1 for weekdays and 0 for non-weekdays.

[0033] Table 1 Specific Input / Output Feature Set

[0034]

[0035] (2) Hybrid neural network model based on PLE multi-task learning framework

[0036] (a) Model Framework

[0037] This invention establishes a joint prediction model CPLE based on the Progressive Layered Extraction (PLE) model. This model mainly includes a feature extraction layer, a feature separation layer, and a sub-task layer. A CNN is used as the feature extraction layer. The task-specific expert module in this layer consists of two convolutional layers, while the shared expert module consists of four convolutional layers. The original input enters each expert network in the feature extraction layer and undergoes a 16-kernel convolution operation. The task-specific experts are used to abstract unique features relevant to the task itself from the input feature set, while the shared experts are used to extract coupling information between tasks. Each expert module has a gating unit, i.e., a single-layer fully connected network + softmax function. The gating unit uses the output of the expert sub-network as a selector to calculate the weighted sum of the selected vectors.

[0038] Given the need to learn more feature representations and the consumption of computational resources, a fully connected network is used as the expert for the feature separation layer. The task-specific expert consists of four fully connected networks, and the shared expert consists of eight. The feature separation layer is used to eliminate the connection between the task-specific tower network and other task-specific experts, and to enhance the learning of the differentiated features of each task. The output of the corresponding task expert is also weighed by a gating unit to balance the proportion of shared and specific experts. Finally, the corresponding output is fed to the corresponding sub-task layer, which consists of an LSTM and a fully connected network. The LSTM is used for time series information extraction, and the fully connected layer is responsible for outputting the predicted values ​​of cold, heat, and electricity loads.

[0039] (b) CNN network

[0040] One-dimensional convolutional networks are used as feature extraction units to mine information from input sequences. The principle of convolutional networks is to share the weight parameters in each convolutional kernel in local connections and combine different local receptive fields to obtain global information. One-dimensional convolutional neural networks are often used to process time series data. They can learn more new features based on the original data by setting multiple filters.

[0041] (c) PLE Multitask Learning

[0042] The Progressive Hierarchical Extraction Model (PLE) proposes a novel progressive routing scheme. Through gating units, it extracts public and private knowledge in a hierarchical manner, allowing lower-level experts to extract deep feature information and higher-level experts to gradually separate task-specific information. This significantly mitigates the negative transfer between shared and specific information. The basic structure of the Progressive Hierarchical Extraction Model (PLE) is as follows:

[0043] The PLE model consists of expert modules, gated units, and a tower network. The expert modules and gated units are located at the bottom of the model, forming a multi-layered feature extraction network. Expert modules learn knowledge in both shared and exclusive modes, and each expert module contains multiple expert subnetworks. The gated network is responsible for selectively fusing different expert modules, gradually separating public and private parameters at higher layers. Let the input vector be x, and the number of layers in the bottom layer be N. Then, the formula for calculating the output of each layer can be expressed as:

[0044] g k,j (x)=w k,j (x)S k,j (x) (5)

[0045]

[0046] Among them, g k,j (x) represents the output of the k-th subtask in the j-th layer of the network, i.e., the output of the gating unit. k,j (x) is the weight parameter of task k at layer j, expressed by a linear matrix. The calculation, based on the Softmax function, shows that its input is the output g of the j-1 layer network. k,j-1 (x),

[0047] S k,j (x) is the selection matrix of task k at level j, derived from the shared module E. s And the exclusive module E of task k k The composition, and the specific formula are shown in equation (7).

[0048]

[0049] In the formula m s and m k Let represent the number of shared experts and the number of exclusive experts for task k, respectively.

[0050] The tower network, located at the top of the model, is responsible for absorbing knowledge extracted from shared experts and task-specific experts and outputting task results. It consists of multiple layers of neural networks, where y in equation (8) k,N (x) represents the prediction result for task k, t k For the tower network of task k,

[0051] y k (x)=t k (g k,N (x)) (8)

[0052] (d) LSTM network

[0053] To better learn the temporal characteristics of multi-energy load data, LSTM is used as a sub-task layer of the proposed model.

[0054] The beneficial effects of this invention using the above-described technology to solve this problem are:

[0055] This invention addresses multi-energy load forecasting for integrated energy systems by proposing a joint load forecasting model based on STL and PLE multi-task learning. This model uses MIC and STL to construct feature sequences, and a convolutional progressive hierarchical extraction model (CPLE) to extract spatiotemporal coupling features. Compared with the prediction results of other models, this invention yields the following beneficial effects:

[0056] (1) STL separates the multiple patterns of IES load, reduces the time complexity of load sequence, and provides convenience for subsequent modeling and prediction;

[0057] (2) IES has close connections between various energy sources. Multi-task learning strategies are more conducive to mining the coupling relationship between multiple energy loads than single-task learning strategies, and capturing more key semantic information.

[0058] (3) The progressive sharing mechanism fully considers the differences in correlation between multi-energy loads, provides reasonable allocation of shared information for different tasks, and avoids interference of weak related information on sub-tasks;

[0059] (4) The hybrid network of CNN and LSTM can effectively extract the spatiotemporal information contained in the feature sequence, and the model performance is further improved when combined with PLE. Attached Figure Description

[0060] Figure 1 This is a flowchart of the IES load forecasting system in this invention;

[0061] Figure 2 This is a diagram of the annual multi-variable load curve in this invention;

[0062] Figure 3 This is a diagram showing the STL decomposition results of the multi-energy load in this invention;

[0063] Figure 4 This is a correlation analysis diagram of the multi-energy load components in this invention;

[0064] Figure 5 This is a diagram of the CPLE model framework in this invention;

[0065] Figure 6 This is a diagram of the hard parameter sharing model in this invention;

[0066] Figure 7 This is a diagram of the PLE model in this invention;

[0067] Figure 8 This is a comparison chart of the errors of the load components on different models in this invention;

[0068] Figure 9 A comparison of error radar charts for different models in this invention;

[0069] Figure 10 This is a comparison chart of the relative errors of the load on different models in this invention. Detailed Implementation

[0070] (1) The IES multivariate load forecasting process is as follows: Figure 1 As shown,

[0071] The prediction process of this invention is mainly divided into two parts: data processing, data modeling, and prediction.

[0072] First, outliers are identified using the quartile method, and missing or outlier data are replaced using linear interpolation.

[0073] Then, STL was used to decompose the cold, heat, and electrical load data to obtain three components: trend, period, and residual.

[0074] Next, MIC analysis was performed on the load components and meteorological characteristics, and corresponding input sequences were constructed for each type of component in combination with time characteristics;

[0075] The input features are then normalized, and the CPLE model is used to mine the differences and correlations among the multivariate loads, so as to achieve the prediction of the three components.

[0076] Finally, the multi-energy load forecast is obtained by reconstructing the three components.

[0077] (2) Input data processing

[0078] In IES, there are many types of energy-consuming devices, and the energy demand is highly random. The load sequence as a whole exhibits characteristics such as non-stationarity and large fluctuations. In order to more accurately characterize the load features and provide more refined data support, this invention needs to perform load data decomposition and feature screening.

[0079] (a) Load data decomposition

[0080] STL is a commonly used time series data decomposition algorithm that can decompose a time series into a trend term reflecting long-term changes, a seasonal term representing periodic changes, and a residual term containing noise. Each of these decomposition terms has a clear physical meaning, which can well understand the inherent laws and characteristics of the series. It is also highly robust to noise and can remove noise and outliers from the original data.

[0081] The load of IES is affected by a variety of factors, resulting in irregular load fluctuations and strong randomness. In order to improve the interpretability of the load sequence and reduce the influence of random factors, STL is selected for decomposition. The decomposition formula of STL is expressed as equation (1):

[0082] Yt =T t +S t +R t (1)

[0083] Among them, Y t T represents the raw data at time t. t S t R t The tables represent the trend, seasonal, and residual terms at time t. The STL calculation process consists of an inner loop and an outer loop. The inner loop decomposes the trend and seasonal terms through local weighted regression smoothing, while the outer loop adjusts the robust weights based on the results of the inner loop to reduce the impact of outliers on the next inner loop iteration.

[0084] STL is an additive model, which is suitable for sequences with relatively uniform time scales. However, in reality, many time series data exhibit a pattern where the period changes with the trend. In this case, a multiplicative model is more suitable. The multiplicative form of STL can be implemented using the log function, as shown in equation (2):

[0085] log Y t =log T t +log S t +log R t (2)

[0086] The appropriate decomposition method needs to be selected based on the specific circumstances of the data. The load data used in this invention comes from the Arizona State University Campus Metabolism project, which includes the electricity, heat, and cooling loads of the Tempe campus in 2020 and 2021. The sampling rate is 1 hour / sample, with a total of 17,544 sampling points. The relevant weather data comes from the Wunderground weather website, including temperature, humidity, dew point, air pressure, rainfall, gusts, and wind speed. Figure 2 This is the workload situation at the Tempe campus in 2020. For ease of observation, the data has been dimensionless. Figure 2 It can be seen that the magnitude of the periodic fluctuations of the three load sequences changes over time. The fluctuation amplitude of the heat load is much greater in winter than in summer, while the fluctuation amplitude of the cooling and electrical loads is the opposite. The degree of change is more drastic in summer than in winter. Therefore, this invention chooses the multiplicative model for further analysis.

[0087] STL was performed on the electrical, cooling, and heating load sequences according to equation (2), and the results are as follows: Figure 3 As shown,

[0088] Depend on Figure 3As can be seen, STL decomposes the IES electrical, heating, and cooling load sequences into three subsequences: trend, seasonal, and residual. The trend sequence is a low-frequency component representing the long-term trend and direction of load changes; the seasonal sequence is a high-frequency component reflecting the changes in load with seasonal variations, i.e., the cyclical variation of the load; and the residual sequence is the part remaining after subtracting the trend and cyclical sequences from the original sequence, representing random fluctuations in the load data not explained by trend and seasonal effects. In short, STL decomposes the internally coupled regular information of the load into different components, making the trend and cyclical terms more predictable and understandable than the original load. Furthermore, because STL decomposes the noise in the sequence into the residual term, it effectively reduces the impact of random factors on load forecasting, making load forecasting more stable and reliable. Therefore, in order to more effectively utilize the regular information in the load during forecasting and reduce the impact of load data non-stationarity on forecast accuracy, it is necessary to use STL to decompose the load data.

[0089] (b) Feature sequence selection

[0090] While IES contains data of various categories, a larger number of input features does not necessarily mean better training results. Excessive input features may introduce noisy data, causing the model's learning to deviate from its normal trajectory. By selecting features with high correlation to load from numerous feature sequences, the interference of noise and redundant information can be effectively reduced. Considering various linear and nonlinear relationships and the correlation between two variables, this invention chooses the MIC method to analyze the correlation between multi-energy load components and meteorological factors.

[0091] MIC is a mutual information-based metric for detecting the correlation between different variables. It can comprehensively cover various linear and nonlinear relationships. This method has high universality and low computational complexity, and is often used for feature selection in deep learning. The basic idea of ​​using MIC is to represent the relationship between two variables with a two-dimensional scatter plot, perform gridding on the scatter plot to estimate the probability density distribution of the variables, and finally normalize to obtain the MIC value. The value range of MIC is [0,1]. The larger the value, the deeper the correlation. The specific calculation process is shown in equations (3) and (4).

[0092]

[0093]

[0094] In the formula, n is the number of samples, a and b represent the number of grids in the x and y directions of the variables in the two-dimensional coordinate system, respectively, and B(n) is a function of the sample size, usually set to n. 0.6 I(x,y) represents the mutual information between variables x and y, p(x) and p(y) are the probability density distributions of x and y, respectively, and p(x,y) is the joint probability density distribution of x and y.

[0095] Considering that Arizona State University's Tempe campus is located in a subtropical climate with hot summers, the year is divided into summer and non-summer periods. Then, using the formula above, the minimum interconversion factors (MICs) between the cold, heat, and electrical load components and weather data are calculated for each season. The results are then used to create a heat map, as shown below. Figure 4 As shown, a MIC value greater than 0.2 is considered a strong correlation, and a MIC value less than 0.1 is considered a weak correlation.

[0096] Depend on Figure 4 It can be seen that after STL, the trend terms of electricity, heat, and cooling loads show strong correlations, while the correlations among the remaining terms are weak. The degree of correlation between load trend terms also varies across seasons. In summer, the correlation between electricity load trend terms and cooling load trend terms is high, while the correlation with heat load trend terms is low, whereas the opposite is true in non-summer seasons. This aligns with actual conditions. Besides the mutual influence among multiple load components, loads are also sensitive to meteorological fluctuations. The degree of influence of weather on load components varies across seasons. For example, the trend terms of multiple loads show strong correlations with dew point temperature and humidity in summer, while in non-summer seasons they are more strongly correlated with temperature, with MIC values ​​greater than 0.2. The above analysis indicates that different meteorological characteristics should be selected to constitute input features for different load components in different seasons.

[0097] (c) Input / output feature set settings

[0098] Besides weather factors, input features also need to consider historical load and time encoding. The most recent historical load near the time to be predicted contains rich prior information, which is helpful for model training when used as input features. Time encoding is also an important feature for key discrimination of load sequences. Prediction models are constructed for the trend, periodicity, and residual components of multivariate loads. When predicting the multivariate load at time step t, the specific input / output feature sets of each model are shown in Table 1. Among them, the hourly feature corresponds to the encoding 0-23, and the weekday feature is set to 1 for weekdays and 0 for non-weekdays.

[0099] Table 1 Specific Input / Output Feature Set

[0100]

[0101] (3) Hybrid neural network model based on PLE multi-task learning framework

[0102] (a) Model Framework

[0103] The joint prediction model CPLE based on PLE improvement established in this invention is as follows: Figure 5As shown, the model mainly includes a feature extraction layer, a feature separation layer, and a sub-task layer. A CNN is used as the feature extraction layer. The task-specific expert module in this layer consists of two convolutional layers, while the shared expert module consists of four convolutional layers. The original input enters each expert network in the feature extraction layer and undergoes a 16-kernel convolution operation. The specific experts are used to abstract unique features related to the task itself from the input feature set, while the shared experts are used to extract coupling information between tasks. Each expert module has a gating unit, which is a single-layer fully connected network + softmax function. The gating unit uses the output of the expert sub-network as a selector to calculate the weight of the selected vector. It is important to note that the task-specific expert output only fuses the shared experts and the corresponding task experts, while the shared expert output fuses all experts.

[0104] Given the need to learn more feature representations and the consumption of computational resources, a fully connected network is used as the expert for the feature separation layer. The task-specific expert consists of four fully connected networks, and the shared expert consists of eight. The feature separation layer is used to eliminate the connection between the task-specific tower network and other task-specific experts, and to enhance the learning of the differentiated features of each task. The output of the corresponding task expert is also weighed by a gating unit to balance the proportion of shared and specific experts. Finally, the corresponding output is fed to the corresponding sub-task layer, which consists of an LSTM and a fully connected network. The LSTM is used for time series information extraction, and the fully connected layer is responsible for outputting the predicted values ​​of cold, heat, and electricity loads.

[0105] (b) CNN network

[0106] CNNs possess powerful information mining capabilities, fully extracting spatial coupling information hidden in multi-energy loads and meteorological factors. Therefore, one-dimensional convolutional networks are used as feature extraction units to mine information from input sequences. The principle of convolutional networks is to share the weight parameters in each convolutional kernel in local connections, and to synthesize different local receptive fields to obtain global information. One-dimensional convolutional neural networks are often used to process time series data, and can learn more new features based on the original data by setting multiple filters.

[0107] (c) PLE Multitask Learning

[0108] Considering the rich and complex correlations among various IES workloads and the joint optimization of multiple objectives, a multi-task learning method is adopted to construct the prediction model. Multi-task learning can promote knowledge transfer and sharing among multiple related tasks, allowing tasks to complement each other's strengths. If the learning is split into individual tasks, the inherent connections between objectives are ignored, limiting the improvement of overall performance.

[0109] In deep learning, multi-task learning generally includes two modes: hard sharing and soft sharing. The hard sharing mechanism of hidden layer parameters is currently the most widely used, and its basic principle is as follows: Figure 6 As shown, hard parameter sharing can be divided into a sharing layer and a sub-task layer. The sharing layer uniformly shares the underlying parameters and extracts shared data features; the sub-task layer has independent parameters, learns relevant information for the task of interest from the shared features, and outputs prediction results.

[0110] However, hard sharing is too simplistic, and sharing information between tasks with poor correlation can lead to negative transfer problems. To address this, Multi-gate Mixture-of-Experts (MMoE) introduces gating and expert networks into the sharing layer. Different gates autonomously and dynamically learn different combinations of experts for their respective prediction objectives, thus better capturing task differences. Compared to hard sharing, MMoE achieves better prediction results. However, MMoE shares information indiscriminately across all tasks, treating all expert networks equally, which can introduce noise into some tasks. Furthermore, the poor interactivity between expert modules in MMoE limits the effectiveness of joint optimization. Therefore, to improve the efficiency of shared learning and reduce the impact of loosely correlated tasks, Tencent PCG proposes a novel learning framework, PLE, based on MMoE.

[0111] PLE explicitly separates the expert network in the parameter sharing layer into task-sharing experts and task-specific experts. Shared experts learn common data features, while task-specific experts extract features relevant to the task itself. The scope of task influence for different types of experts is defined. Based on this, PLE designs a novel progressive routing scheme that extracts public and private knowledge hierarchically through gating units. This allows lower-level experts to extract deep feature information, while higher-level experts gradually separate task-specific information. This significantly mitigates the negative transfer between shared and task-specific information. The basic structure of PLE is as follows: Figure 7 As shown,

[0112] The PLE model consists of expert modules, gated units, and a tower network. The expert modules and gated units are located at the bottom of the model, forming a multi-layered feature extraction network. Expert modules learn knowledge in both shared and exclusive modes, and each expert module contains multiple expert subnetworks. The gated network is responsible for selectively fusing different expert modules, gradually separating public and private parameters at higher layers. Let the input vector be x, and the number of layers in the bottom layer be N. Then, the formula for calculating the output of each layer can be expressed as:

[0113] g k,j (x)=w k,j (x)S k,j (x) (5)

[0114]

[0115] Among them, g k,j(x) represents the output of the k-th subtask in the j-th layer of the network, i.e., the output of the gating unit. k,j (x) is the weight parameter of task k at layer j, expressed by a linear matrix. The calculation, based on the Softmax function, shows that its input is the output g of the j-1 layer network. k,j-1 (x),

[0116] S k,j (x) is the selection matrix of task k at level j, derived from the shared module E. s And the exclusive module E of task k k The composition, and the specific formula are shown in equation (7).

[0117]

[0118] In the formula m s and m k These represent the number of shared experts and the number of exclusive experts for task k, respectively. Note that the selection matrix for shared modules consists of all experts at that layer.

[0119] The tower network, located at the top of the model, is responsible for absorbing knowledge extracted from shared experts and task-specific experts and outputting task results. It consists of multiple layers of neural networks, where y in equation (8) k,N (x) represents the prediction result for task k, t k For the tower network of task k,

[0120] y k (x)=t k (g k,N (x)) (8)

[0121] (c) LSTM network

[0122] To better learn the temporal characteristics of multi-energy load data, LSTM is used as a sub-task layer in the proposed model. LSTM is a network model based on an improved Recurrent Neural Network (RNN), which has excellent performance in modeling and predicting time series data. By introducing a gating mechanism to control feature flow and loss, LSTM enhances its ability to process long-term series, enabling it not only to model short-term dependencies well but also to overcome the gradient vanishing / exploding problem inherent in RNNs.

[0123] (1) Data sources and experimental environment

[0124] The data comes from the Arizona State University Campus Metabolism project, including the electricity, heat, and cooling loads of the Tempe campus in 2020 and 2021, with a sampling rate of 1 hour per sampling, totaling 17,544 sampling points. Related meteorological data comes from the Wunderground weather website, including temperature, humidity, dew point, air pressure, rainfall, gusts, and wind speed. The proposed model was built and trained using the Keras deep learning framework, with a hardware configuration of two Xeon 4215R CPUs and two NVIDIA RTX 3090 GPUs.

[0125] To verify the robustness and predictive performance of the proposed model in different seasons, joint multi-energy load prediction models were established for summer and non-summer periods. The summer data used August 2021 as the test set, and the non-summer data used December 2021 as the test set. The remaining data were divided in a 9:1 ratio, with 90% used as the training set and 10% as the validation set. Due to the inconsistent units used in IES load data, MW was used as the unit of measurement for each load for ease of expression. The conversion formulas for electricity, cooling, and heating load units are as follows:

[0126] 1MW=284Ton=3.4mmBtu / h (9)

[0127] In the forecasting process, the data cleaning stage uses the quartile method to identify outliers and replaces missing or outlier data with linear interpolation. Then, the STL multiplication mode is used to decompose the load data features, and highly correlated features are selected for different load components to form the input data. After determining the dataset, normalization is performed, and the processing formula is as shown in equation (10):

[0128]

[0129] In the formula, x max x is the maximum value of this input feature. min This is the minimum value of the input feature.

[0130] (2) Evaluation indicators

[0131] To comprehensively evaluate the performance of the proposed prediction model, the root mean square error (RMSE), mean absolute percentage error (MAPE), and weighted average precision (WMA) are used as evaluation metrics. The specific calculation formulas are shown in equations (11) and (12):

[0132]

[0133]

[0134] In the formula: y_act(i) is the actual load value at time i to be predicted, y_pred(i) is the predicted load value at time i, and n is the number of samples in the training / validation / test set. Considering the structure of the IES itself and the importance of each load, the weight values ​​of electricity, cooling, and heating loads in the WMA index are set to 0.4, 0.4, and 0.2, respectively. The relevant formulas are shown in equations (13) and (14).

[0135] MA = 1 - MAPE (13)

[0136] WMA = α elec MA elec +α cool MA cool +α heat MA heat (14)

[0137] (3) Hyperparameter settings

[0138] The choice of hyperparameters has a significant impact on the model's learning performance. Taking into account both training time and prediction accuracy, the network parameters of the CPLE model were determined after multiple optimizations, as shown in Table 2.

[0139] Table 2 Specific parameter settings

[0140]

[0141]

[0142] The effectiveness of the proposed STL and CPLE was verified through two sets of experiments. In the effectiveness analysis of CPLE, multiple comparative experiments were conducted to analyze the superiority of the multi-task framework over PLE, as well as the impact of CNN and LSTM on prediction accuracy. In the effectiveness analysis of STL, using CPLE as a baseline, the improvement of STL on prediction results was analyzed.

[0143] (4) CPLE effectiveness analysis (a) Comparison with single-task load prediction model

[0144] The energy consumption characteristics of energy systems (IES) determine that different types of energy have certain coupling relationships. To verify that utilizing these coupling relationships can improve load forecasting accuracy, the CPLE model was used for both joint and independent forecasting. The joint forecasting model considers coupling information, forecasting electrical, heating, and cooling loads simultaneously. The independent forecasting model forecasts each type of load separately. The forecasting errors of the two methods are shown in Table 5.

[0145] Table 3. Impact of Multi-Task Learning on Prediction Error

[0146]

[0147] Table 3 shows that the multi-task learning model provides more reliable predictions than the single-task learning model, with improvements in the errors for electrical, thermal, and cooling loads. Overall, the WMA of the multi-task model is 0.35% higher than that of the single-task model in summer, and the training time is reduced by 18.3%. In non-summer seasons, the WMA of the multi-task model is 0.9% higher than that of the single-task model, and the training time is reduced by 12.2%. Clearly, the single-task learning method cannot fully utilize the auxiliary coupling information provided by related sub-tasks. Even with the same network structure, the prediction performance is still limited by the prediction pattern of the single task, making it difficult for the shared layer to learn the auxiliary information contained in other tasks. Compared with the single-task prediction method, multi-task prediction only requires training one model, which significantly reduces training time and construction costs. Specifically, the electrical load accuracy shows the greatest improvement because electrical load has strong real-time characteristics and is influenced by more factors. Therefore, the introduction of a multi-task strategy results in the most significant improvement in electrical load accuracy.

[0148] Based on STL, we further verified the impact of multi-task learning on load forecasting results. Figure 8 Table 4 compares the evaluation results of the electrical, heating, and cooling load components under the single-task method and the multi-task method. Table 4 shows the error situation of the single-task model and the multi-task model after the component prediction results are fused.

[0149] Table 4. Impact of Post-STL Multi-Task Learning on Prediction Error

[0150]

[0151] Combining Tables 3 and 4, multi-task learning ability remains superior to single-task learning, and both methods show more accurate predictions after STL. Figure 8 As can be observed in (a), (b), and (c), in most cases, component prediction based on multiple tasks yields better results than that based on a single task. Furthermore, through overall evaluation... Figure 8(c) reveals that, regardless of the component prediction, CPLE based on a multi-task framework outperforms CPLE based on a single-task framework, especially in trend component prediction. Taking the WMA index as an example, the results show that in summer, the WMA improvement for the multi-task trend component is 26 times and 3.25 times that of the seasonal and residual components, respectively. In non-summer seasons, the improvement for the multi-task trend component WMA is 68 times and 32 times that of the seasonal and residual components, respectively. Furthermore, the trend components of electrical, heating, and cooling loads show a high degree of correlation, while the correlation between residual components and seasonal components is low. This verifies the necessity of multi-task learning, which can extract common features between strongly correlated tasks, helping the model better understand the correlation between tasks, thereby improving the model's predictive and generalization abilities. The seasonal and residual predictions under the multi-task framework also show a certain degree of improvement, indicating that the proposed progressive routing scheme can not only effectively uncover the connections between tasks and improve learning quality, but also minimize the negative impact of task differences on model performance.

[0152] (b) Comparison with multi-task load forecasting models

[0153] To highlight the superiority of the proposed method in IES multi-energy load forecasting, a horizontal comparison was conducted with traditional hard parameter sharing, MMOE, and PLE models. Simultaneously, a vertical comparison was also performed with PLE (with CNN) and PLE (with LSTM) models. Table 5 shows the comparison results of each model under different multi-task frameworks.

[0154] Table 5 Comparison of prediction errors of different models

[0155]

[0156] As shown in Table 5, PLE had the smallest multi-task learning error, followed by MMOE, while the hard-sharing method had the highest error. In summer and non-summer, PLE improved WMA by 0.07% and 0.22% compared to MMOE, respectively. This is because hard-sharing directly shares all parameters, making it difficult to accurately identify differences between tasks, thus limiting further improvement in model accuracy. While MMOE can theoretically reach the level of PLE, without prior knowledge network design, it is usually difficult to converge to the same effect as PLE in actual training. Looking at RMSE and MAPE metrics, in summer, PLE had higher accuracy in predicting electrical and cooling loads than MMOE, but lower accuracy in predicting heating loads. In non-summer, PLE had higher accuracy in predicting cooling and heating loads than MMOE, but lower accuracy in predicting electrical loads. This is because while PLE can alleviate the seesaw effect, it cannot avoid sacrificing the performance of other tasks while improving the performance of some tasks; however, overall, PLE is superior to MMOE.

[0157] The proposed model is an improvement on the PLE model. To verify the necessity of each module in the proposed method for prediction accuracy, ablation experiments were conducted. The PLE model was used as the baseline model, with its shared layer and sub-task layers both composed of fully connected networks. PLE (with CNN) and PLE (with LSTM) improved upon PLE by refining the shared layer and sub-task layer, respectively. The former uses a CNN module as the feature extraction unit, while the latter uses an LSTM as the subsequent modeling layer. The remaining training parameters remained consistent with the CPLE model. The MAPE, RMSE, and WMA indices of electrical, cooling, and heating loads on the test set were calculated, and the error distribution radar chart was obtained as shown below. Figure 9 As shown,

[0158] according to Figure 9 The results in (a), (b), and (c) show that the CPLE model performs best in predicting electricity, cooling, and heating loads, with the lowest RMSE and MAPE values. Figure 9 (d) The overall performance differences among the various models can be observed. Specifically, the PLE model has the lowest prediction accuracy, while other models show some improvement, especially the CPLE model, which shows the largest improvement.

[0159] Taking non-summer conditions as an example, PLE (with CNN), PLE (with LSTM), and CPLE improved WMA by 0.17%, 0.15%, and 0.51% respectively compared to PLE. This indicates that the proposed method is effective in improving both the shared layer and the sub-task layer, and combining the two can further improve prediction accuracy. The reasons for this are as follows: Compared to ordinary fully connected networks, using a one-dimensional convolutional network as the feature extraction layer can better uncover the hidden correlations between multiple tasks and provide coupling feature information that preserves temporal characteristics for the next module. The unique recurrent neural network structure of LSTM allows it to learn longer time-slice features, thus capturing more comprehensive temporal information. Using CNN alone lacks the ability to model long-term temporal dependencies, while using LSTM alone cannot effectively extract spatial coupling features; both methods have limitations. Therefore, the proposed model combines the characteristics of CNN and LSTM networks, adopting a PLE multi-task learning framework based on prior separation. This effectively enhances the model's expressive power by fully extracting the latent spatiotemporal relationships between different types of workloads while handling task conflicts.

[0160] (5) STL Validity Analysis

[0161] The proposed method requires predicting the three components obtained from STL separately, and then reconstructing the three components to obtain the final prediction result. In the experiments, two decomposition forms were used: STL based on multiplication (×) and STL based on addition (+), with CPLE (without STL) serving as a control to evaluate the prediction performance of the proposed method. Table 6 lists the evaluation results of the three methods. Figure 10 This shows the relative error of different models in two time periods: summer (August 2-8, 2021) and non-summer (December 1-7, 2021).

[0162] Table 6. Impact of STL on Prediction Error

[0163]

[0164] From Table 6, Figure 10 It can be seen that, compared with the model without STL, the model using STL has better prediction performance, smaller relative error, and the multiplicative form yields more accurate results than the additive form. From Figure 10 It can be observed that the relative error without STL varies significantly, while the relative error with STL is controlled within the range of [-5%, 5%], verifying the effectiveness of STL. STL can extract various patterns in the time series and assign them to different components. This method helps to better capture the regularity and trend in the sequence. Compared with directly predicting the original sequence, modeling and predicting different components significantly reduces the prediction difficulty. At the same time, the experimental results also verify that multiplicative decomposition is more suitable than additive decomposition for handling situations where the load cycle is constantly changing. In addition, the changes in heat load are more significant than those in electricity and cooling loads; therefore, the improvement in heat load is most significant when multiplicative decomposition is used.

[0165] To further improve the time series decomposition effect, the period window of STL was adjusted. This parameter represents the number of seasonal cycles occurring within a year; a smaller value allows for a faster rate of change. Considering a time step of 1 hour, four models were compared with period sizes of 3, 6, 12, and 24, while keeping other parameters unchanged. Table 7 shows the error results of STL under different periods.

[0166] Table 7 Comparison of model errors under different periods.

[0167]

[0168] As shown in Table 7, the prediction performance is best when period=12, regardless of whether it is summer or non-summer. Both excessively long and short period intervals will affect model performance. Too short an interval prevents the model from learning complete periodic information, while too long an interval makes the trend too smooth, leading to overfitting. From a practical standpoint, a period of 12 hours of samples is generally considered acceptable.

Claims

1. A short-term load forecasting method for integrated energy systems based on an improved multi-task model, characterized in that, The steps of this method are as follows: First, outliers are identified using the quartile method, and missing or outlier data are replaced using linear interpolation. Then, STL was used to decompose the cold, heat, and electrical load data to obtain three components: trend, period, and residual. Next, MIC analysis was performed on the load components and meteorological characteristics, and corresponding input sequences were constructed for each type of component in combination with time characteristics; The input features are then normalized, and the CPLE model is used to mine the differences and correlations among the multivariate loads, so as to achieve the prediction of the three components. Finally, the multi-energy load forecast is obtained by reconstructing the three components.

2. The short-term load forecasting method for integrated energy systems based on an improved multi-task model according to claim 1, characterized in that... The specific method is as follows: (1) Input data processing (a) Load data decomposition STL is chosen for decomposition, and the decomposition formula of STL is expressed as equation (1): Y t =T t +S t +R t (1) Among them, Y t T represents the raw data at time t. t S t R t The tables represent the trend, seasonal, and residual terms at time t. The STL calculation process consists of an inner loop and an outer loop. The inner loop decomposes the trend and seasonal terms through local weighted regression smoothing, while the outer loop adjusts the robust weights based on the results of the inner loop to reduce the impact of outliers on the next inner loop iteration. The STL multiplication form can be implemented using the log function, as shown in equation (2): log Y t =log T t +log S t +log R t (2) STL is performed on the electrical, cooling, and heating load sequences according to equation (2). STL decomposes the IES electricity, heat, and cooling load series into three subsequences: trend, seasonal, and residual. The trend sequence, a low-frequency component, represents the long-term trend and direction of load changes. The seasonal sequence, a high-frequency component, reflects the load changes with the seasons, i.e., the cyclical variation of the load. The residual sequence, the part remaining after subtracting the trend and cyclical sequences from the original sequence, represents the random fluctuations in the load data that are not explained by trend and seasonal effects. (b) Feature sequence selection This invention selects the MIC method to analyze the correlation between multi-energy load components and meteorological factors. The value of MIC ranges from [0,1]. The larger the value, the deeper the correlation. The specific calculation process is shown in equations (3) and (4). In the formula, n is the number of samples, a and b represent the number of grids in the x and y directions of the variables in the two-dimensional coordinate system, respectively, and B(n) is a function of the sample size, usually set to n. 0.6 I(x,y) represents the mutual information between variables x and y, p(x) and p(y) are the probability density distributions of x and y, respectively, and p(x,y) is the joint probability density distribution of x and y. Based on the above formula, the minimum interconversion ratio (MIC) values ​​between each component of cold, heat, and electricity load in both seasons and weather data were calculated, and the results were plotted into a heat map. MIC values ​​greater than 0.2 were considered strong correlations, and values ​​less than 0.1 were considered weak correlations. Following the STL (Standardized Tolerance) index, the trend terms of electricity, heat, and cooling loads showed strong correlations, while the correlations among the remaining terms were weaker. The degree of correlation between load trend terms also varied across seasons. In summer, the correlation between electricity and cooling load trend terms was higher, while the correlation with heat load trend terms was lower, whereas the opposite was true in non-summer seasons. This aligns with actual conditions. Besides the mutual influence among multiple load components, loads are also sensitive to meteorological fluctuations. The impact of weather on load components differs across seasons. For example, multi-load trend terms showed strong correlations with dew point temperature and humidity in summer, while in non-summer seasons they were more strongly correlated with temperature, with MIC values ​​greater than 0.

2. The above analysis indicates that different meteorological characteristics should be selected for input features for different load components in different seasons. (c) Input / output feature set settings Besides weather factors, input features also need to consider historical load and time encoding. The most recent historical load near the time to be predicted contains rich prior information, which is helpful for model training when used as input features. Time encoding is also an important feature for key discrimination of load sequences. Prediction models are constructed for the trend, periodicity, and residual components of multivariate loads. When predicting the multivariate load at time step t, the specific input / output feature sets of each model are shown in Table 1. Among them, the hourly feature corresponds to the encoding 0-23, and the weekday feature is set to 1 for weekdays and 0 for non-weekdays. Table 1 Specific Input / Output Feature Set (2) Hybrid neural network model based on PLE multi-task learning framework (a) Model Framework This invention establishes a Joint Prediction Model (CPLE) based on Improved Prediction Optimization (PLE). This model mainly includes a feature extraction layer, a feature separation layer, and a sub-task layer. A CNN is used as the feature extraction layer. The task-specific expert module in this layer consists of two convolutional layers, while the shared expert module consists of four convolutional layers. The original input enters each expert network in the feature extraction layer and undergoes a 16-kernel convolution operation. The task-specific experts are used to abstract unique features relevant to the task itself from the input feature set, while the shared experts are used to extract coupling information between tasks. Each expert module has a gating unit, i.e., a single-layer fully connected network + softmax function. The gating unit uses the output of the expert sub-network as a selector to calculate the weighted sum of the selected vectors. Given the need to learn more feature representations and the consumption of computational resources, a fully connected network is used as the expert for the feature separation layer. The task-specific expert consists of four fully connected networks, and the shared expert consists of eight. The feature separation layer is used to eliminate the connection between the task-specific tower network and other task-specific experts, and to enhance the learning of the differentiated features of each task. The output of the corresponding task expert is also weighed by a gating unit to balance the proportion of shared and specific experts. Finally, the corresponding output is fed to the corresponding sub-task layer, which consists of an LSTM and a fully connected network. The LSTM is used for time series information extraction, and the fully connected layer is responsible for outputting the predicted values ​​of cold, heat, and electricity loads. (b) CNN network One-dimensional convolutional networks are used as feature extraction units to mine information from input sequences. The principle of convolutional networks is to share the weight parameters in each convolutional kernel in local connections and combine different local receptive fields to obtain global information. One-dimensional convolutional neural networks are often used to process time series data. They can learn more new features based on the original data by setting multiple filters. (c) PLE Multitask Learning PLE employs a novel progressive routing scheme that uses gating units to extract public and private knowledge in a hierarchical manner. This allows lower-level experts to extract deep feature information, while higher-level experts gradually separate task-specific information. This significantly mitigates the negative transfer between shared and specific information. The basic structure of PLE... The PLE model consists of expert modules, gated units, and a tower network. The expert modules and gated units are located at the bottom of the model, forming a multi-layered feature extraction network. Expert modules learn knowledge in both shared and exclusive modes, and each expert module contains multiple expert subnetworks. The gated network is responsible for selectively fusing different expert modules, gradually separating public and private parameters at higher layers. Let the input vector be x, and the number of layers in the bottom layer be N. Then, the formula for calculating the output of each layer can be expressed as: g k,j (x)=w k,j (x)S k,j (x) (5) Among them, g k,j (x) represents the output of the k-th subtask in the j-th layer of the network, i.e., the output of the gating unit. k,j (x) is the weight parameter of task k at layer j, expressed by a linear matrix. The calculation, based on the Softmax function, shows that its input is the output g of the j-1 layer network. k,j-1 (x), S k,j (x) is the selection matrix of task k at level j, derived from the shared module E. s And the exclusive module E of task k k The composition, and the specific formula are shown in equation (7). In the formula m s and m k Let represent the number of shared experts and the number of exclusive experts for task k, respectively. The tower network, located at the top of the model, is responsible for absorbing knowledge extracted from shared experts and task-specific experts and outputting task results. It consists of multiple layers of neural networks, where y in equation (8) k,N (x) represents the prediction result for task k, t k For the tower network of task k, y k (x)=t k (g k,N (x)) (8) (d) LSTM network To better learn the temporal characteristics of multi-energy load data, LSTM is used as a sub-task layer of the proposed model.

Images