TCN-GRU-Adaboost-based load prediction method

Abstract

The invention provides a TCN-GRU-Adaboost-based load prediction method, which is applied to the field of load prediction of an integrated energy system. Firstly, a time sequence signal is simply decomposed and reconstructed; secondly, in combination with a time sequence convolutional network (TCN), a gating cycle unit (GRU) and an attention mechanism (Attention Mode), establishing a hybrid neural network; then combining an adaptive enhancement integration algorithm (Adaboost) with the hybrid neural network to form a load prediction model; and evaluating power, gas, heat and cold load data of a certain integrated energy system by using the constructed load prediction model. Compared with the prior art, the method is higher in precision, and better in robustness and generalization of prediction of different types of loads of the integrated energy system.

Description

Technical Field

[0001] The present invention relates to the technical field of load forecasting, and in particular to a load forecasting method based on TCN-GRU-Adaboost. Background Art

[0002] The separation design and independent operation mode of traditional energy systems weaken the interconnectivity of various energy sources and limit the reliability and flexibility of the system. To this end, many countries have proposed the necessity of building an integrated energy system (IES). IES promotes the coordination, complementarity, management and operation of energy systems by integrating multiple energy sources such as electricity, gas, heat, and cold, as well as energy conversion and storage equipment, and effectively improves the security of energy supply. Accurate load forecasting is crucial to the management strategy of IEs, but because IEs involve multiple forms of energy demand, there are coupling relationships and uncertainties, resulting in low prediction accuracy, slow response speed and poor stability of existing load forecasting methods. Summary of the invention

[0003] In view of the above problems, the present invention proposes a load forecasting method based on TCN-GRU-Adaboost. First, the time series signal is simply decomposed and reconstructed; secondly, a hybrid neural network is constructed by combining the time series convolutional network (TCN), the gated recurrent unit (GRU) and the attention mechanism (Attention Mechan is sm); then the adaptive boosting ensemble algorithm (Adaboost) and the hybrid neural network are combined into a load forecasting model; and then the constructed load forecasting model is used to evaluate the electricity, gas, heat and cooling load data of the integrated energy system.

[0004] The present invention is implemented by adopting the following technical solution: a load forecasting method based on TCN-GRU-Adaboost is implemented by the following steps:

[0005] S1: Collect the publicly available data on electricity, gas, heating and cooling loads of the integrated energy system;

[0006] S2: The collected data on electricity, gas, heat and cooling loads of the integrated energy system are simply decomposed and reconstructed and divided into a training set and a test set in a ratio of 7:3;

[0007] S3: Establish a hybrid neural network model that combines temporal convolutional networks, gated recurrent units, and attention mechanisms;

[0008] Furthermore, the step S3 comprises:

[0009] A hybrid neural network model (TGA) combining a temporal convolutional network, a gated recurrent unit, and an attention mechanism is established. The TCN model extracts past data through one-dimensional causal convolution to ensure temporal sequence, the residual connection accelerates the convergence speed, and the dilated convolution realizes temporal feature extraction. As a variant of the recurrent neural network, the GRU model has nonlinear fitting capabilities, can effectively extract data features, and achieves faster convergence speed while ensuring a prediction effect similar to that of LSTM. The steps are as follows.

[0010] Step 1. TCN is a special convolutional network. It is a new type of convolutional network through causal convolution and residual connection. Compared with traditional CNN, TCN can capture the dependency of time series through convolution layer, which is more suitable for time series data processing. Causal convolution is a strict time constraint model. The output at time t depends only on the value before time t, that is, future information will not interfere with past information, which makes TCN have strong time information processing capabilities. For a one-dimensional sequence x and a convolution kernel f, the dilated convolution F on the element s is as follows:

[0011]

[0012] Where d is the causal coefficient; K is the size of the convolution kernel; sd·i represents the sd·ith element of the previous layer. Residual connections ensure the accuracy of deep networks by coupling the input and output information of convolution operations. A residual block with input x and output y is defined as follows:

[0013] y=H(x,W)+x

[0014] Where H represents a series of convolution operation functions; W is the weight of the convolution kernel.

[0015] Each residual block contains two layers of causal convolution, using the Dropout layer to prevent overfitting, the rectified linear unit (ReLU) as the activation function to improve the nonlinearity of the network, and WeightNorm to normalize the convolution kernel weight values.

[0016] Step 2: GRU, as a variant of LSTM network, has a simpler structure and faster running speed, and can overcome the shortcomings of recurrent neural network (RNN) in processing long time series data. GRU combines the "forget gate" and "input gate" of LSTM into an "update gate", removes the "output gate" and adds a "reset gate". The "reset gate" is used to determine the information that needs to be ignored at the previous moment, and the "update gate" is used to determine whether the information at the current moment is passed to the next moment. The GRU calculation process is as follows:

[0017] zt=σ(W z x t +U z ht-1 )

[0018] rt=σ(W r x t +U r h t-1 )

[0019] h t =(1-z t )*h t-1 +z t *h t

[0020] ht=tanh(W h x t +U h (rt*ht-1))

[0021] Among them, z t and r t They are “update gate” and “reset gate” respectively; h t and h t are the state vector and candidate state vector at time t respectively; σ and tanh are activation functions; W z , W, W h , U z , U r , U h are model parameters.

[0022] Step 3. Add the attention mechanism. In traditional neural networks, all input information is treated equally, but in the actual prediction process, some information is more important than other information. The attention mechanism enables the model to focus on important information, thereby reducing the interference of redundant information, thereby significantly improving the performance of the model when processing long sequences. The core of the attention mechanism is to enable the model to automatically assign different importance weights to each part of the input sequence. The attention mechanism includes Query, Key, and Value vectors. Query is used to query relevant information, and Key is used to match Query to determine which information is most relevant to the current query. Finally, by matching Query and Key, the model extracts information from the corresponding Value. The summary is as follows:

[0023] Preferably, calculate the similarity between Query and Key:

[0024] sim(Query,Key i )=QueryKey i

[0025] Preferably, similar normalization:

[0026]

[0027] Preferably, an attention value is calculated.

[0028]

[0029] In the above formula, sim is a similarity function; α i It is expressed as Value weight factor; softmax represents activation function.

[0030] Furthermore, step S4 includes constructing a TGA-AdaBoost load forecasting model by embedding a hybrid neural network with an adaptive boosting integrated algorithm. In view of the fact that the TGA model may produce unstable forecasting results when applied to multivariate load forecasting of electricity, gas, heat energy and cold energy, the present invention introduces the Adaboost.RT algorithm into the model to enhance the forecasting ability of TGA.

[0031] The basic steps are:

[0032] S401: Input training samples, set the weak learner algorithm to TGA, the number of learners to T, and the threshold to

[0033] S402: Initialize sample weight D t =1 / m, error rate ε t =0;

[0034] S403: Train the TGA model and calculate the error E of each sample according to the following formula: t (i) and the network error ε t (i);

[0035] Et(i)=|(ft(xi)-yi)-yi

[0036]

[0037] S404: Update the weight distribution to:

[0038]

[0039] Where β t =ε t , Z is a normalization factor used to ensure that the sum of sample weights is 1;

[0040] S405: Output the final result after T rounds of training:

[0041]

[0042] Due to the performance of AdaBoost. RT algorithm has a threshold More sensitive, Too large or too small will significantly affect the prediction effect. Therefore, the present invention uses the root mean square error (RMSE) value to Replaced by a fixed value with an adaptive variable.

[0043]

[0044] Furthermore, the step S5 comprises:

[0045] S501: Time series reconstruction, signaling the historical load data of electricity, gas, heat and cooling, converting the original data into a sequence format suitable for model input to capture its time dependency and characteristics;

[0046] S502: Using a TGA hybrid neural network model consisting of a temporal convolutional network (TCN), a gated recurrent unit (GRU) and an attention mechanism (AttentionMechan i sm);

[0047] S503: Combine the TGA model with the Adaboost.RT algorithm to generate multiple weak learners (TGA models), and build a strong learner by repeated iterative training and updating sample weights to further improve the accuracy and robustness of the prediction;

[0048] S504: performing weighted summation on the prediction results of multiple TGA models according to their weights in AdaBoost to obtain a final prediction result;

[0049] Beneficial Effects

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] 1) The constructed hybrid neural network TGA model combines a temporal convolutional network, a gated recurrent unit, and an attention mechanism, and has an excellent ability to capture and process long-term dependencies. The temporal features of the input sequence are extracted through TCN and then input into the GRU to capture long-term dependencies. The Attention mechanism dynamically adjusts the importance of each time step in the input sequence, further improving the accuracy of the prediction.

[0052] 2) The adaptive boosting algorithm is further embedded in the TGA model, and Adaboost.RT is used to enhance the prediction ability of TGA. A series of TGA models are generated by combining TGA and Adaboost.RT, and the training samples are repeatedly trained to find samples with low prediction accuracy and focus on them. This method enhances the robustness and generalization of the model for different types of load forecasting. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is a flow chart of the method of the present invention;

[0054] Figure 2 It is a schematic diagram of the TCN structure;

[0055] Figure 3 It is a schematic diagram of the GRU structure;

[0056] Figure 4 Schematic diagram of the attention mechanism steps; DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0058] The present invention is achieved by adopting the following technical solutions:

[0059] Figure 1 A flow chart of a load prediction method based on TCN-GRU-Adaboost is shown in the present application, refer to Figure 1 , which is achieved by the following steps:

[0060] S1: Collect the publicly available data on electricity, gas, heating and cooling loads of the integrated energy system;

[0061] S2: The collected data on electricity, gas, heat and cooling loads of the integrated energy system are simply decomposed and reconstructed and divided into a training set and a test set in a ratio of 7:3;

[0062] S3: Establish a hybrid neural network model that combines temporal convolutional networks, gated recurrent units, and attention mechanisms;

[0063] S4: Using adaptive boosting ensemble algorithm to embed hybrid neural network to build load forecasting model;

[0064] S5: input the electricity, gas, heating and cooling load data into the load forecasting model for forecasting and evaluation;

[0065] In the S3:

[0066] A hybrid neural network model (TGA) combining a temporal convolutional network, a gated recurrent unit, and an attention mechanism is established. The TCN model extracts past data through one-dimensional causal convolution to ensure temporal sequence, the residual connection accelerates the convergence speed, and the dilated convolution realizes temporal feature extraction. As a variant of the recurrent neural network, the GRU model has nonlinear fitting capabilities, can effectively extract data features, and achieves faster convergence speed while ensuring a prediction effect similar to that of LSTM. The steps are as follows.

[0067] Step 1. TCN is a special convolutional network. It is a new type of convolutional network through causal convolution and residual connection. Compared with the traditional CNN, TCN can capture the dependency of time series through the convolution layer and is more suitable for time series data processing. Its structure is as follows Figure 2 As shown. Causal convolution is a strict time-constrained model. The output at time t depends only on the value before time t, that is, future information will not interfere with past information, making TCN have a strong ability to process time information. For a one-dimensional sequence x and a convolution kernel f, the dilated convolution F on the element s is as follows:

[0068]

[0069] Where d is the causal coefficient; K is the size of the convolution kernel; sd·i represents the sd·ith element of the previous layer. Residual connections ensure the accuracy of deep networks by coupling the input and output information of convolution operations. A residual block with input x and output y is defined as follows:

[0070] y=H(x,W)+x

[0071] Where H represents a series of convolution operation functions; W is the weight of the convolution kernel.

[0072] Each residual block contains two layers of causal convolution, using the Dropout layer to prevent overfitting, the rectified linear unit (ReLU) as the activation function to improve the nonlinearity of the network, and WeightNorm to normalize the convolution kernel weight values.

[0073] Step 2: GRU, as a variant of LSTM network, has a simpler structure and faster running speed, which can overcome the shortcomings of recurrent neural network (RNN) in processing long time series data. GRU combines the "forget gate" and "input gate" of LSTM into an "update gate", removes the "output gate" and adds a "reset gate". The structure of GRU is as follows Figure 3 As shown. The "reset gate" is used to determine the information that needs to be ignored at the previous moment, and the "update gate" is used to determine whether the information at the current moment is passed to the next moment. The GRU calculation process is as follows:

[0074] zt=σ(W z x t +U z h t-1 )

[0075] rt=σ(W r x t +U r h t-1 )

[0076] h t =(1-zt )*h t-1 +z t *h t

[0077] ht=tanh(W h x t +U h (rt*ht-1))

[0078] Among them, z t and r t They are “update gate” and “reset gate” respectively; h t and h t are the state vector and candidate state vector at time t respectively; σ and tanh are activation functions; W z , W, W h , U z , U r , U h are model parameters.

[0079] Step 3. In traditional neural networks, all input information is treated equally, but in the actual prediction process, some information is more important than other information. The attention mechanism can enable the model to focus on important information, thereby reducing the interference of redundant information, thereby significantly improving the performance of the model when processing long sequences. The core of the attention mechanism is to enable the model to automatically assign different importance weights to each part of the input sequence. The attention mechanism includes Query, Key, and Value vectors. Query is used to query relevant information, and Key is used to match Query to determine which information is most relevant to the current query. Finally, by matching Query and Key, the model extracts information from the corresponding Value. The steps of the attention mechanism are as follows Figure 4 As shown, the summary is as follows:

[0080] Preferably, calculate the similarity between Query and Key:

[0081] sim(Query,Key i )=QueryKey i

[0082] Preferably, similar normalization:

[0083]

[0084] Preferably, an attention value is calculated.

[0085]

[0086] In the above formula, sim is a similarity function; α iIt is expressed as Value weight factor; softmax represents activation function.

[0087] S4: Use the adaptive boosting ensemble algorithm to embed the hybrid neural network to build a load forecasting model.

[0088] Furthermore, step S4 includes constructing a TGA-AdaBoost load forecasting model by embedding a hybrid neural network with an adaptive boosting integrated algorithm. In view of the fact that the TGA model may produce unstable forecasting results when applied to multivariate load forecasting of electricity, gas, heat energy and cold energy, the present invention introduces the Adaboost.RT algorithm into the model to enhance the forecasting ability of TGA.

[0089] The basic steps are:

[0090] S401: Input training samples, set the weak learner algorithm to TGA, the number of learners to T, and the threshold to

[0091] S402: Initialize sample weight D t =1 / m, error rate ε t =0;

[0092] S403: Train the TGA model and calculate the error E of each sample according to the following formula: t (i) and the network error ε t (i);

[0093] Et(i)=|(ft(xi)-yi)-yi

[0094]

[0095] S404: Update the weight distribution to:

[0096]

[0097] Where β t =ε t , Z is a normalization factor used to ensure that the sum of sample weights is 1;

[0098] S405: Output the final result after T rounds of training:

[0099]

[0100] Due to the performance of AdaBoost. RT algorithm has a threshold More sensitive, Too large or too small will significantly affect the prediction effect. Therefore, the present invention uses the root mean square error (RMSE) value to Replaced by a fixed value with an adaptive variable.

[0101]

[0102] S5: Input the electricity, gas, heating and cooling load data into the load forecasting model for forecasting and evaluation.

[0103] In S5: there are the following 4 steps:

[0104] S501: Time series reconstruction, signaling the historical load data of electricity, gas, heat and cooling, converting the original data into a sequence format suitable for model input to capture its time dependency and characteristics;

[0105] S502: Using a TGA hybrid neural network model consisting of a temporal convolutional network (TCN), a gated recurrent unit (GRU) and an attention mechanism (AttentionMechan i sm);

[0106] S503: Combine the TGA model with the Adaboost.RT algorithm to generate multiple weak learners (TGA models), and build a strong learner by repeated iterative training and updating sample weights to further improve the accuracy and robustness of the prediction;

[0107] S504: performing weighted summation on the prediction results of multiple TGA models according to their weights in AdaBoost to obtain a final prediction result.

Claims

1. A load forecasting method based on TCN-GRU-Adaboost, characterized by: This is accomplished by the following steps: S1: Collect the publicly available data on electricity, gas, heating and cooling loads of the integrated energy system; S2: The collected data on electricity, gas, heat and cooling loads of the integrated energy system are simply decomposed and reconstructed and divided into a training set and a test set in a ratio of 7:3; S3: Establish a hybrid neural network model that combines temporal convolutional networks, gated recurrent units, and attention mechanisms; S4: Using adaptive boosting ensemble algorithm to embed hybrid neural network to build load forecasting model; S5: Input the electricity, gas, heating and cooling load data into the load forecasting model for forecasting and evaluation.

2. The load forecasting method based on TCN-GRU-Adaboost according to claim 1, characterized in that: The proposed hybrid neural network model combining the temporal convolutional network, the gated recurrent unit and the attention mechanism in step S3 is established, and its specific steps are: Step 1: Construct a TCN model, which uses a new convolutional network with causal convolution and residual connection. Compared with traditional CNN, TCN can capture the dependency of time series through convolution layers and is more suitable for time series data processing. Causal convolution is a strict time constraint model. The output at time t only depends on the value before time t, that is, future information will not interfere with past information, making TCN have a strong ability to process time information. For a one-dimensional sequence x and convolution kernel f, the dilated convolution F on element s is as follows: Where d is the causal coefficient; K is the size of the convolution kernel; sd·i represents the sd·ith element of the previous layer. The residual connection ensures the accuracy of the deep network by coupling the input and output information of the convolution operation. The residual block with input x and output y is defined as follows: y=H(x,W)+x Where H represents a series of convolution operation functions, and W is the weight of the convolution kernel; Each residual block contains two layers of causal convolution, using the Dropout layer to prevent overfitting, the Rectified Linear Unit (ReLU) as the activation function to improve the nonlinearity of the network, and WeightNorm to normalize the convolution kernel weights; Step 2: Build a GRU model. GRU combines the "forget gate" and "input gate" of LSTM into an "update gate", removes the "output gate", and adds a "reset gate". The GRU calculation process is as follows: zt=σ(W z x t +U z h t-1 ) rt=σ(W r x t +U r h t-1 ) h t =(1-z t )*h t-1 +z t *h t ht=tanh(W h x t +U h (rt*ht-1)) Among them, z t and r t They are "update gate" and "reset gate" respectively; h t and h t are the state vector and candidate state vector at time t respectively; σ and tanh are activation functions; W z , W, W h , U z , U r , U h are model parameters; Step 3: Add an attention mechanism to the model. The core of the attention mechanism is to enable the model to automatically assign different importance weights to each part of the input sequence. The attention mechanism includes Query, Key, and Value vectors. Query is used to query relevant information, and Key is used to match Query to determine which information is most relevant to the current query. Finally, by matching Query and Key, the model extracts information from the corresponding Value, which can be summarized as follows: sim(Query,Key i )=QueryKey i In the above formula, sim is a similarity function; α i It is expressed as Value weight factor; softmax represents activation function.

3. The load forecasting method based on TCN-GRU-Adaboost according to claim 1, characterized in that: In step S4, the load forecasting model is constructed by embedding the hybrid neural network with the adaptive enhanced integration algorithm, and the specific method is as follows: Step 1: Input training samples, set the weak learner algorithm to TGA, the number of learners to T, and the threshold to Step 2: Initialize sample weight D t =1 / m, error rate ε t =0; Step 3: Train the TGA model and calculate the error E of each sample according to the following formula: t (i) and the network error ε t (i); Et(i)=|(ft(xi)-yi)-yi| Step 4: Update the weight distribution: Where β t =ε t , Z is a normalization factor used to ensure that the sum of sample weights is 1; Step 5: Output the final result after T rounds of training: Step 6: Due to the performance of AdaBoost, the RT algorithm uses its threshold More sensitive, Too large or too small will significantly affect the prediction effect, so the present invention uses the root mean square error to Replace fixed values ​​with adaptive variables: Where RMSE is the root mean square error.

4. The load forecasting method based on TCN-GRU-Adaboost according to claim 1, characterized in that: The specific steps in step S5 are: S501: Time series reconstruction, signaling the historical load data of electricity, gas, heat and cooling, converting the original data into a sequence format suitable for model input to capture its time dependency and characteristics; S502: Using a TGA hybrid neural network model consisting of a temporal convolutional network (TCN), a gated recurrent unit (GRU) and an attention mechanism (AttentionMechanism); S503: Combine the TGA model with the Adaboost.RT algorithm to generate multiple weak learners (TGA models), and build a strong learner by repeated iterative training and updating sample weights to further improve the accuracy and robustness of the prediction; S504: performing weighted summation on the prediction results of multiple TGA models according to their weights in AdaBoost to obtain a final prediction result.

Images