Multi-scale Power Load Prediction Method Based on Pyramid-shaped Recurrent Neural Network

Abstract

The present invention discloses a multi-scale power load prediction method based on a pyramidal recurrent neural network, comprising: preprocessing power load data to construct training samples; using a hybrid multi-scale construction module to construct sequences of different scales based on each training sample and stack them into a pyramidal structure; in the pyramidal structure, alternately using an intra-scale information fusion module and an inter-scale information transfer module to model the temporal dependence within each scale sequence and the influence between different scale sequences from large scale to small scale; using a fusion prediction module to predict sequences of different scales respectively and fuse the prediction results; training a power load prediction model based on all training samples; and using the trained power load prediction model for prediction. The method of the present invention can achieve a complete modeling of multi-scale temporal patterns in a power load sequence, and has broad application prospects in the fields of power system scheduling, urban planning, energy conservation management, etc.

Description

Technical Field

[0001] The present invention belongs to the technical field of power load forecasting, and particularly relates to a multi-scale power load forecasting method based on a pyramid-shaped recurrent neural network. Background Art

[0002] With the development of society and the progress of technology, all walks of life's dependence on electricity is increasing day by day. Electricity has become the foundation of modern social development, supporting all aspects of industrial production, commercial operation, transportation, medical and health care, education and scientific research, as well as people's daily lives. If the power loads in various fields can be accurately predicted, it can better support subsequent tasks such as optimization planning and decision-making support, and can also provide effective assistance for power companies to make decisions in aspects such as power generation planning and power dispatching. Therefore, power load forecasting has become a very important task in the field of data mining.

[0003] Time series data such as power loads is greatly affected by human activities and often exhibits very obvious multi-scale characteristics, that is, the time series will show different temporal patterns at different time scales. When the sampling frequency is high, the sequence largely reflects the instantaneous changes of the time series data, while after increasing the time scale (that is, aggregating the data in the time dimension), the overall pattern of the power load sequence can be seen more clearly. A good power load forecasting method should be able to model the time dependencies within sequences at each scale and the influence between sequences at different scales simultaneously.

[0004] Existing power load forecasting methods are mainly divided into three categories: First, early statistical methods. This type of method builds a model based on statistical principles and analyzes and forecasts the power load sequence using the statistical characteristics and patterns in historical data, but it cannot model the non-linear relationships in the sequence. Second, traditional machine learning methods. This type of method uses machine learning algorithms to model historical data and can make predictions by learning the complex patterns and non-linear relationships in the data. However, it is overly dependent on data feature engineering and has poor scalability. Third, deep learning methods. This type of method uses deep learning algorithms to model historical data and models the power load sequence through multiple levels of non-linear transformation and feature extraction. Some methods attempt to construct and model sequences at different scales from the original sequence, and some methods also introduce special structures such as pyramids, regarding the values within sequences at different scales as a node in the pyramid structure and modeling the influence between sequences at different scales by constructing edges between nodes.

[0005] Although the above power load prediction methods have achieved certain effects, however, these methods still have some problems. Most power load prediction methods do not consider the multi-scale time series patterns in the power load sequence data. Although some methods consider the influence between sequences of different scales, they are only limited to the influence between a certain node and its adjacent nodes, ignoring the influence of a larger range of nodes or even the entire sequence on this node. Therefore, the complete modeling of the multi-scale time series pattern of power load prediction has not been realized in the prior art. Summary of the Invention

[0006] In view of the above, the object of the present invention is to provide a multi-scale power load prediction method based on a pyramid-shaped recurrent neural network. When modeling the multi-scale time series pattern in the power load sequence, this method stacks sequences of different scales into a pyramid structure, and designs two modules: intra-scale information fusion and inter-scale information transfer to respectively model the time dependence within each scale sequence and the influence between different scale sequences, so as to realize the complete modeling of the multi-scale time series pattern in the power load sequence, and has a wide application prospect in the fields of power system scheduling, urban planning, energy conservation management, etc.

[0007] To achieve the above object of the invention, the technical solutions provided by the present invention are as follows:

[0008] A multi-scale power load prediction method based on a pyramid-shaped recurrent neural network provided by an embodiment of the present invention includes the following steps:

[0009] Step (1), perform data preprocessing on the given power load data to construct training samples;

[0010] Step (2), use the hybrid multi-scale construction module to construct sequences of different scales based on each training sample and stack them into a pyramid structure;

[0011] Step (3), in the pyramid structure, alternately use the intra-scale information fusion module and the inter-scale information transfer module in the multi-scale information interaction module, and process the sequences of different scales from large scale to small scale from top to bottom, respectively modeling the time dependence within each scale sequence and the influence between different scale sequences;

[0012] Step (4), use the fusion prediction module to send the sequences of different scales that have modeled the multi-scale time series pattern into their respective predictors for prediction, and then fuse the prediction results of each scale to obtain the predicted value of each training sample;

[0013] Step (5), train the power load prediction model based on all training samples, and the power load prediction model includes a hybrid multi-scale construction module, a multi-scale information interaction module, and a fusion prediction module;

[0014] Step (6): Input the power load sequence to be predicted into the trained power load prediction model to obtain the prediction result.

[0015] Preferably, step (1) includes: performing data preprocessing on the given power load data, including deleting outliers, filling missing values, and normalizing; dividing the processed data by means of a sliding window to obtain a training data set; batchifying the training data set according to a fixed batch size, and the total number of batches is N; sequentially selecting a batch of training samples with index i from the training data set, where i ∈ {1, …, N}.

[0016] Preferably, step (2) includes: inputting the training sample sequence X into the hybrid multi-scale construction module, which contains several large-scale information capture modules. The previous large-scale information capture module extracts several large-scale feature information combinations from the input sequence to form the corresponding large-scale sequence, and then sends this large-scale sequence into the next large-scale information capture module to further extract larger-scale feature information, constructing a multi-scale sequence from small scale to large scale from bottom to top to obtain a multi-scale sequence set where X is the input training sample sequence, X s is the sequence of the s-th scale, s takes values of 1 ≤ s ≤ C, X C is the maximum-scale sequence; stacking the sequences of all C + 1 scales together to form a pyramid structure.

[0017] Preferably, the large-scale information capture module includes parallel convolutional layers, max pooling layers, average pooling layers, and min pooling layers, and is followed by stacking and linear layers, and the construction method is as follows:

[0018]

[0019] MinPooling(X s-1 ),AvgPooling(X s-1 )]

[0020]

[0021] where Conv(·), MaxPooling(·), MinPooling(·), and AvgPooling(·) are convolutional, max pooling, min pooling, and average pooling operations respectively, Stack(·) and Linear(·) are stacking operations and linear transformations respectively, is the intermediate state of the s-th scale in the multi-scale sequence construction process, X s is the sequence of the s-th scale, X s-1 is the sequence of the s - 1-th scale.

[0022] Preferably, step (3) includes:

[0023] 3a), in the intra-scale information fusion module, use a recurrent neural network to process the maximum-scale sequence X C or the sequence of the s-th scale after inter-scale information transfer and update the sequence with the hidden state of the network to obtain the sequence Z of the s-th scale after modeling the time dependence s ;

[0024] 3b), introduce a gating mechanism in the intra-scale information fusion module, and use the sequence of the s-th scale after inter-scale information transfer to control the sequence Z of the s-th scale after modeling the time dependence s balance the error caused by historical information during the modeling process, and finally obtain the sequence of the s-th scale after intra-scale information fusion

[0025] 3c), in the inter-scale information transfer module, first compress the sequence of the s-th scale after intra-scale information fusion to obtain global information then interact with each dimension in this global information, and finally fuse this global information with the sequence X of the (s - 1)-th scale s-1 to form the sequence of the (s - 1)-th scale after inter-scale information transfer thus modeling the influence between sequences of different scales

[0026] Preferably, the input of the intra-scale information fusion module is the maximum-scale sequence X C or the sequence of the s-th scale after inter-scale information transfer model the intra-scale time dependence relationship through a recurrent neural network, and use the hidden state H of the recurrent neural network s as the output, then transform the hidden state H through the first linear layer s to a high-dimensional space pass through the Dropout layer to prevent network overfitting, and finally use the second linear layer to transform the sequence back to the original dimension to obtain the sequence Z of the s-th scale after modeling the time dependence s , and the calculation process is as follows:

[0027]

[0028]

[0029] wherein, Linear1(·) and Linear2(·) are respectively the transformation operations of the first linear layer and the second linear layer, Dropout(·) is the operation to prevent network overfitting, is the representation of the hidden state H s in the high-dimensional space, and Z sThe sequence at the s-th scale after modeling the temporal dependence of the output of the second linear layer.

[0030] Preferably, the introduced gating mechanism screens the information in the sequence based on the Sigmoid layer, and the sequence at the s-th scale after passing the information between scales is sent to the Sigmoid layer to generate an update gate, and then multiplied by the sequence Z at the s-th scale after modeling the temporal dependence to obtain s which is used as the output of the intra-scale information fusion module. The calculation process is as follows: As the output of the intra-scale information fusion module, the calculation process is as follows:

[0031]

[0032] where is the sequence at the s-th scale after intra-scale information fusion, is the element-wise multiplication, and Sigmoid(·) is the gating mechanism.

[0033] Preferably, the inter-scale information transfer module is composed of a third linear layer, a fourth linear layer, a fifth linear layer, and a Dropout layer; the third linear layer compresses the information in the large-scale sequence, the fourth linear layer interacts the global information between different dimensions, the fifth linear layer maps the global information to the length of the small-scale sequence, and finally, after passing through the Dropout layer, it is added to the small-scale sequence to model the influence between different-scale sequences, obtaining as the output of the inter-scale information transfer module. The calculation process is as follows:

[0034]

[0035]

[0036]

[0037] where Linear3(·), Linear4(·), and Linear5(·) are the transformation operations of the third linear layer, the fourth linear layer, and the fifth linear layer respectively, and Dropout(·) is an operation to prevent network overfitting, is the sequence at the s-th scale after intra-scale information fusion, is the global information, is the influence of the s-th scale on the sequence at the s-1 scale, X s-1 is the sequence at the s-1 scale, is the sequence at the s-1 scale after inter-scale information transfer.

[0038] Preferably, the fusion prediction module consists of predictors that are independent of each other at each scale and a fuser. Among them, the predictors at each scale are composed of linear layers. The sequences at different scales that model multi-scale time series patterns are decoded through the linear layers to obtain the prediction results at each scale. The fuser fuses the prediction results at each scale according to the contribution degree of each scale to the final result, and the obtained fused result is the predicted value of each training sample.

[0039] Compared with the prior art, the beneficial effects of the present invention at least include:

[0040] (1) In the pyramid structure, the global information of the large-scale sequence is extracted from top to bottom using the inter-scale information transfer module and fused with the smaller-scale sequence, modeling the influence between all nodes at different scales, thereby completely modeling the influence between sequences at different scales.

[0041] (2) In the intra-scale information fusion module, the recurrent neural network is used to model the sequence, and the information update degree is controlled through the gating mechanism, which not only models the time-dependent relationship within each scale sequence but also screens the important information in the sequence, improving the prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0043] Figure 1 is the overall flowchart of the multi-scale power load prediction method based on the pyramid-shaped recurrent neural network provided by the embodiment of the present invention;

[0044] Figure 2 is the overall framework diagram of the multi-scale power load prediction method based on the pyramid-shaped recurrent neural network;

[0045] Figure 3 is the detailed architecture of the large-scale information capture module provided by the embodiment of the present invention;

[0046] Figure 4 is the detailed architecture of the intra-scale information fusion module provided by the embodiment of the present invention;

[0047] Figure 5 is the detailed architecture of the inter-scale information transfer module provided by the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] To make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not limit the protection scope of the present invention.

[0049] The inventive concept of the present invention is: to solve the technical problem of realizing the complete modeling of multi-scale time series patterns in a power load prediction model, the embodiments of the present invention provide a multi-scale power load prediction method based on a pyramid-shaped recurrent neural network. By constructing sequences of different scales from the input power load data and forming a pyramid structure, and through a multi-scale information interaction module to model the multi-scale time series patterns in the data from top to bottom, the complete modeling of the multi-scale time series patterns in the power load sequence is realized.

[0050] Figure 1 is the overall flowchart of the multi-scale power load prediction method based on the pyramid-shaped recurrent neural network provided by the embodiments of the present invention. Figure 2 is the overall framework diagram of the multi-scale power load prediction method based on the pyramid-shaped recurrent neural network. As Figure 1 and Figure 2 shown, the embodiments provide a multi-scale power load prediction method based on a pyramid-shaped recurrent neural network, including the following steps:

[0051] Define the power load prediction task as: given a set of sequences X = {x1, x2, …, x t , …, x T}, the value at the t-th (1 ≤ t ≤ T) moment is x t , T represents the input window size, set the prediction step size to h, and the power load prediction task is to predict the values from the (T + 1)-th moment to the (T + h)-th moment, that is, to predict

[0052] Step (1), perform data preprocessing on the given power load data, construct a training data set, and batch the training data set.

[0053] 1a), perform data preprocessing on the given power load data, delete the outliers that significantly exceed the reasonable range in the data, and fill the missing values in the data. Subsequently, use the standardization method to normalize the data, so as to adjust each sequence to a state with a mean of 0 and a variance of 1. The calculation method is:

[0054]

[0055] where x t is the value at the t-th moment in the sequence, μ is the average value of the data of this sequence, σ is the standard deviation of this sequence, is the value at the t-th moment after normalization processing;

[0056] After normalizing the data, set the sliding step size to 1, and use a sliding window with a window size of T to construct training samples from the data to form a training dataset.

[0057] 1b), Batch the training dataset according to a fixed batch size M, and the total number of batches is N, and the calculation method is:

[0058]

[0059] where N Samples is the total number of training samples in the training dataset.

[0060] 1c), Sequentially select a batch of training samples with index i from the training dataset, where i ∈ {1, …, N}.

[0061] Step (2), Use the hybrid multi-scale construction module to construct sequences of different scales from the input sample sequence and stack them into a pyramid structure.

[0062] In the hybrid multi-scale construction module, use multiple large-scale information capture modules to construct sequences of different scales from the given sample sequence X. The detailed architecture of the large-scale information capture module is as Figure 3 shown, including parallel convolutional layers, max pooling layers, average pooling layers, and min pooling layers, and followed by stacking and linear layers. The construction method is as follows:

[0063]

[0064] MinPooling(X s-1 ), AvgPooling(X s-1 )]

[0065]

[0066] where Conv(·), MaxPooling(·), MinPooling(·), and AvgPooling(·) are convolutional, max pooling, min pooling, and average pooling operations respectively, and Stack(·) and Linear(·) are stacking operations and linear transformations respectively, is the intermediate state at the s-th scale during the multi-scale sequence construction process, X s is the sequence at the s-th scale, and X s-1 is the sequence at the s - 1-th scale.

[0067] Specifically, the training sample sequence X is input into the hybrid multi-scale construction module, which contains several large-scale information capture modules. In each large-scale information capture module, the parameters of the convolutional kernel are automatically updated by the convolutional operation relying on the backpropagation algorithm to dynamically capture the most valuable features in the input sequence. The average pooling, max pooling, and min pooling operations then take some statistical information that cannot be obtained by the convolutional operations of the sequence itself as important information and send it to the large-scale feature representation additionally. Finally, the obtained features are stacked and different weights are assigned to the four features through a linear layer; the previous large-scale information capture module extracts several large-scale feature information from the input sequence to form the corresponding large-scale sequence, and then sends this large-scale sequence into the next large-scale information capture module to further extract larger-scale feature information, constructing a multi-scale sequence from small scale to large scale from bottom to top, and obtaining a multi-scale sequence set x = {X, X 1 , …, X s , …, X C}, where X is the input training sample sequence, X s is the sequence of the s-th (1 ≤ s ≤ C) scale, and X C is the maximum-scale sequence; the sequences of all C + 1 scales are stacked together to form a pyramid structure.

[0068] Step (3), in the multi-scale information interaction module, the intra-scale information fusion module and the inter-scale information transfer module are alternately used to process the sequences of different scales from large scale to small scale in the pyramid structure, respectively modeling the temporal dependence within each scale sequence and the influence between different scale sequences.

[0069] 3a), in the intra-scale information fusion module, a recurrent neural network is used to process the maximum-scale sequence X C or the s-th scale sequence after inter-scale information transfer , and the hidden state of the network is used to update the sequence to obtain the s-th scale sequence Z s after modeling the temporal dependence.

[0070] The detailed architecture of the intra-scale information fusion module is as shown in Figure 4 , including a recurrent neural network, a first linear layer, a Dropout layer, a second linear layer, and a Sigmoid layer, where L s is the length of the s-th scale sequence, d is the dimension of the sequence, and d′ is the dimension after linear transformation. Since the intra-scale information fusion and the inter-scale information transfer alternate in the multi-scale information interaction module, the intra-scale information fusion module inputs the maximum-scale sequence X C obtained in the hybrid multi-scale construction module when processing the maximum scale, and inputs the s-th scale sequence after inter-scale information transfer when processing other scales

[0071] Specifically, the intra-scale information fusion module first takes the maximum-scale sequence X C or the sequence at the s-th scale after inter-scale information transfer and feeds it into a recurrent neural network. The recurrent neural network models the temporal dependencies within the scale and uses the hidden state of the recurrent neural network as the output. This process can also be regarded as feature encoding of the sequence. In the specific implementation, a long short-term memory network is used to model . Then, the hidden state H s is transformed to a high-dimensional space through a Dropout layer to prevent overfitting of the network. Finally, a second linear layer is used to transform the sequence back to its original dimension, obtaining the sequence at the s-th scale after modeling the temporal dependencies The calculation process is as follows:

[0072]

[0073]

[0074] where Linear1(·) and Linear2(·) are the transformation operations of the first and second linear layers respectively, and Dropout(·) is the operation to prevent overfitting of the network, is the representation of the hidden state H s in the high-dimensional space, and Z s is the sequence at the s-th scale after modeling the temporal dependencies output by the second linear layer.

[0075] 3b), A gating mechanism is introduced in the intra-scale information fusion module, and the sequence at the s-th scale after inter-scale information transfer is used to control the sequence Z s at the s-th scale after modeling the temporal dependencies, balance the errors brought by historical information during the modeling process, and finally obtain the sequence at the s-th scale after intra-scale information fusion

[0076] Specifically, after obtaining the sequence Z s modeling the temporal dependencies through the recurrent neural network and linear layer transformation, it is also necessary to screen the information in the sequence. The sequence at the s-th scale after inter-scale information transfer is fed into a Sigmoid layer to generate an update gate, which is then multiplied by the sequence Z s at the s-th scale after modeling the temporal dependencies to obtain as the output of the intra-scale information fusion module. The calculation process is as follows:

[0077]

[0078] Among them, is the s-scale sequence after intra-scale information fusion, is element-wise multiplication, and Sigmoid(·) is the gating mechanism.

[0079] The externally introduced gating mechanism, on the one hand, directly uses the current input data to control the network output result, balancing the errors that may be brought by considering historical data in the recurrent neural network. On the other hand, it provides a more direct path in the process of gradient backpropagation in a way similar to residual connection, making the network training more efficient.

[0080] 3c), in the inter-scale information transfer module, first compress the s-scale sequence after intra-scale information fusion to obtain global information Then, interact with each dimension in this global information. Finally, fuse this global information with the (s - 1)-scale sequence X s-1 to form the (s - 1)-scale sequence after inter-scale information transfer Thereby modeling the influence between different scale sequences.

[0081] The detailed architecture of the inter-scale information transfer module is as Figure 5 shown, including the third linear layer, the fourth linear layer, the fifth linear layer, and the Dropout layer, where L s is the length of the s-scale sequence, L G is the length of the global information, and d is the dimension of the sequence. Using the third linear layer, the fourth linear layer, and the fifth linear layer to form a bottleneck structure, global information can be extracted from the s-scale sequence after intra-scale information fusion, controlling the computational complexity in the process of inter-scale information transfer, and the weights in the linear layer can also control the degree of information transfer, passing the large-scale information to the small-scale sequence through appropriate weights, so that the small-scale sequence can consider the influence of the large-scale sequence on the small-scale sequence when modeling its own time-dependent relationship.

[0082] Specifically, the input The third linear layer compresses the information in the large-scale sequence to obtain global information Thereby ensuring that only important information is considered when the large-scale sequence transfers information to the small-scale sequence, filtering out unimportant information and noise; the fourth linear layer interacts with the global information among different dimensions, thereby enhancing the expression ability of this global information; the fifth linear layer maps the global information to the length of the small-scale sequence to obtain the influence of the s-scale on the (s - 1)-scale sequence Finally, after passing through the Dropout layer, it is combined with the small-scale sequence Add them up, model the influence between sequences of different scales, and obtain As the output of the information transfer module between scales, the calculation process is as follows:

[0083]

[0084]

[0085]

[0086] Among them, Linear3(·), Linear4(·), and Linear5(·) are the transformation operations of the third linear layer, the fourth linear layer, and the fifth linear layer respectively, and Dropout(·) is an operation to prevent network overfitting. is the sequence of the s-th scale after in-scale information fusion, is the global information, is the influence of the s-th scale on the sequence of the s - 1-th scale, X s-1 is the sequence of the s - 1-th scale, is the sequence of the s - 1-th scale after information transfer between scales.

[0087] Step (4), use the fusion prediction module to send the sequences of different scales that have modeled multi-scale time series patterns into their respective predictors for prediction, and then fuse the prediction results of each scale to obtain the predicted value of each training sample.

[0088] Specifically, the fusion prediction module consists of predictors that are independent of each other for each scale and a fuser. Among them, the predictors for each scale are composed of linear layers, and the sequences of each scale that have modeled multi-scale time series patterns are decoded through the linear layers to obtain the prediction results for each scale; the fuser fuses the prediction results for each scale through parametric learning, that is, learns from the training data the contribution degree of the sequences of each scale to the final result, and fuses the prediction results for each scale according to this contribution degree. The calculation process is as follows:

[0089]

[0090]

[0091] Among them, Linear s (·) is the predictor (linear layer) of the s-th scale, Linear P (·) is the fuser (linear layer without bias term), Stack(·) is the stacking operation, is the prediction result of the sequence of the s-th (1 ≤ s ≤ C) scale, is the prediction result of the initial input sample sequence X, is the prediction result of the maximum scale sequence, is the final predicted value.

[0092] Step (5), training the power load prediction model based on all training samples. The power load prediction model includes a hybrid multi-scale construction module, a multi-scale information interaction module, and a fusion prediction module.

[0093] 5a), calculating the prediction loss That is, the error between the predicted value corresponding to the training sample and the true value Y, and the calculation method is:

[0094]

[0095] 5b), adjusting the network parameters in the whole model according to the losses of all samples in the batch The calculation method is:

[0096]

[0097] Among them, is the loss of the m-th sample in the batch, and M is the number of samples in each batch.

[0098] Adjust the network parameters in the whole model according to the loss Set η as the learning rate and θ as all the parameters in the network. The update formula is as follows:

[0099]

[0100] Among them, η is the learning rate and θ is all the parameters in the network, is the partial derivative of θ.

[0101] 5c), repeating steps 1c) - 5b) until all batches of the training data set have participated in the model training.

[0102] 5d), repeating steps 1c) - 5c) until the specified number of iterations is reached.

[0103] Step (6), inputting the power load sequence to be predicted into the trained power load prediction model to obtain the prediction result

[0104] The above specific embodiments have elaborated on the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not used to limit the present invention. Any modifications, supplements, equivalent replacements, etc. made within the scope of the principles of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multi-scale power load prediction method based on a pyramidal recurrent neural network, characterized in that It includes the following steps: Step (1), perform data preprocessing on the given power load data to construct training samples; Step (2), use the hybrid multi-scale construction module to construct sequences of different scales based on each training sample and stack them into a pyramid structure; Step (3), in the pyramid structure, alternately use the intra-scale information fusion module and the inter-scale information transfer module in the multi-scale information interaction module to process the sequences of different scales from large scale to small scale top-down, respectively model the temporal dependencies within each scale sequence and the influences between different scale sequences, including: 3a), in the intra-scale information fusion module, use a recurrent neural network to process the maximum-scale sequence X C or the sequence of the s-th scale after inter-scale information transfer and update the sequence with the hidden state of the network to obtain the sequence Z of the s-th scale after modeling the temporal dependence s ; where the input of the intra-scale information fusion module is the maximum-scale sequence X C or the sequence of the s-th scale after inter-scale information transfer model the temporal dependence within the scale through a recurrent neural network, and use the hidden state H s of the recurrent neural network as the output, then transform the hidden state H s to a high-dimensional space pass through a Dropout layer to prevent overfitting of the network, and finally use a second linear layer to transform the sequence back to its original dimension to obtain the sequence Z of the s-th scale after modeling the temporal dependence s , and the calculation process is as follows: Among them, Linear1(·) and Linear2(·) are the transformation operations of the first linear layer and the second linear layer respectively, and Dropout(·) is an operation to prevent the network from overfitting. is the hidden state H s representation in the high-dimensional space, Z s is the sequence of the sth scale after modeling the time dependence of the output of the second linear layer; 3b), introduce a gating mechanism in the intra-scale information fusion module, and use the sequence of the s-th scale after inter-scale information transmission to control the sequence of the s-th scale Z after modeling the temporal dependence relationship, balance the errors brought by historical information during the modeling process, and finally obtain the sequence of the s-th scale after intra-scale information fusion s 3c), in the inter-scale information transfer module, first fuse the information within the scale and then compress the sequence at the s-th scale to obtain the global information Then, perform interactions on each dimension of this global information. Finally, fuse this global information with the sequence at the (s - 1)-th scale X s-1 to form the sequence at the (s - 1)-th scale after inter-scale information transfer Thereby modeling the influence between sequences of different scales; among them, the inter-scale information transfer module consists of a third linear layer, a fourth linear layer, a fifth linear layer, and a Dropout layer; the third linear layer compresses the information in the large-scale sequence, the fourth linear layer performs interactions on the global information between different dimensions, the fifth linear layer maps the global information to the length of the small-scale sequence, and finally, after passing through the Dropout layer, it is added to the small-scale sequence to model the influence between sequences of different scales, obtaining As the output of the inter-scale information transfer module, the calculation process is as follows: Among them, Linear3(·), Linear4(·), and Linear5(·) are the transformation operations of the third linear layer, the fourth linear layer, and the fifth linear layer respectively, and Dropout(·) is an operation to prevent network overfitting. is the sequence of the s-th scale after the in-scale information fusion. is the global information. is the influence of the s-th scale on the sequence of the (s - 1)-th scale, X s-1 is the sequence of the (s - 1)-th scale. is the sequence of the (s - 1)-th scale after the inter-scale information transfer. Step (4), use the fusion prediction module to send the sequences of different scales that model the multi-scale temporal patterns into their respective predictors for prediction, and then fuse the prediction results of each scale to obtain the predicted value of each training sample; Step (5), train the power load prediction model based on all training samples, and the power load prediction model includes a hybrid multi-scale construction module, a multi-scale information interaction module and a fusion prediction module; Step (6), input the power load sequence to be predicted into the trained power load prediction model to obtain the prediction result.

2. The multi-scale power load prediction method based on a pyramidal recurrent neural network according to claim 1, wherein The said step (1) includes: Perform data preprocessing on the given power load data, including deleting outliers, filling missing values and normalizing; divide the processed data by means of a sliding window to obtain a training data set; batch the training data set according to a fixed batch size, and the total number of batches is N; sequentially select a batch of training samples with index i from the training data set, where i ∈ {1, …, N}.

3. The multi-scale power load prediction method based on a pyramid-shaped recurrent neural network according to claim 1, wherein The said step (2) includes: Input the training sample sequence X into the hybrid multi-scale construction module, which contains several large-scale information capture modules. The previous large-scale information capture module extracts several large-scale feature information from the input sequence to form the corresponding large-scale sequence, and then sends the large-scale sequence into the next large-scale information capture module to further extract larger-scale feature information, constructing the multi-scale sequence from small scale to large scale bottom-up to obtain the multi-scale sequence set where X is the input training sample sequence, X s is the sequence of the s-th scale, s ranges from 1 ≤ s ≤ C, X C is the maximum scale sequence; Stack the sequences of all C + 1 scales together to form a pyramid structure.

4. The multi-scale power load prediction method based on a pyramidal recurrent neural network according to claim 3, wherein The large-scale information capture module includes parallel convolutional layers, max pooling layers, average pooling layers and min pooling layers, and is followed by stacking and linear layers, and the construction method is as follows: MinPooling(X s-1 ), AvgPooling(X s-1 )] Among them, Conv(·), MaxPooling(·), MinPooling(·), and AvgPooling(·) are convolution, max pooling, min pooling, and average pooling operations respectively, Stack(·) and Linear(·) are stacking operation and linear transformation respectively, is the intermediate state at the s-th scale during the construction of the multi-scale sequence, X s is the sequence at the s-th scale, X s-1 is the sequence at the (s - 1)-th scale.

5. The multi-scale power load prediction method based on a pyramidal recurrent neural network according to claim 1, wherein The introduced gating mechanism screens the information in the sequence based on the Sigmoid layer, and passes the sequence of the s-th scale after the information transfer between scales into the Sigmoid layer to generate an update gate, and then multiplies it with the sequence Z of the s-th scale after modeling the time dependence s to obtain as the output of the intra-scale information fusion module. The calculation process is as follows: Among them, is the sequence of the s-th scale after information fusion within the scale, is element-wise multiplication, and Sigmoid(·) is the gating mechanism.

6. The multi-scale power load prediction method based on a pyramid-shaped recurrent neural network according to claim 1, characterized in that The fusion prediction module consists of predictors independent of each scale and a fuser. Among them, the predictors of each scale are composed of linear layers. The sequences of different scales that model the multi-scale temporal patterns are decoded through the linear layers to obtain the prediction results of each scale. The fuser fuses the prediction results of each scale according to the contribution degree of each scale to the final result, and the fused result is the predicted value of each training sample.

Images