EMD-ATT-LSTM-based short-term load prediction method and system

Abstract

The invention discloses a short-term load prediction method and system based on EMD-ATT-LSTM, and relates to the technical field of low-carbon industrial park load prediction, and the method comprises the steps: constructing a fine-grained attention model on the basis of a long short-term memory network; decomposing the preprocessed load data into a plurality of intrinsic mode functions; and carrying out load prediction after testing stability. According to the invention, complex features of load data are effectively decomposed and captured, and the prediction accuracy is improved; through an attention mechanism, the sensitivity of the model to key load characteristics is enhanced; and the LSTM network is adopted, so that the long-term dependency relationship in the load data can be effectively processed.

Description

Technical Field

[0001] The present invention relates to the technical field of load forecasting in low-carbon industrial parks, and in particular to a short-term load forecasting method and system based on EMD-ATT-LSTM. Background Art

[0002] With the proposal of the "dual carbon" goal, the construction of low-carbon industrial parks has gradually become the focus of energy system development. In low-carbon industrial parks, accurate load prediction is crucial for optimal energy scheduling. However, due to the nonlinear and non-stationary characteristics of load data, traditional load forecasting methods have limited performance in complex load environments. Therefore, how to improve the accuracy of load forecasting through advanced modeling and algorithmic means has become a hot topic in current research. Empirical mode decomposition (EMD) is an effective tool for processing nonlinear and non-stationary signals. By decomposing complex signals into intrinsic mode functions (IMFs) of different scales, subsequent modeling is more accurate. The attention mechanism can dynamically adjust the model's attention to different input information, thereby improving the model's predictive ability. The long short-term memory network (LSTM) is good at capturing long-term dependencies in time series, so it has significant advantages in load forecasting. Summary of the invention

[0003] In view of the above-mentioned problems, the present invention is proposed.

[0004] Therefore, the technical problem solved by the present invention is: how to improve the load prediction accuracy in a complex load environment through modeling and algorithms.

[0005] To solve the above technical problems, the present invention provides the following technical solutions: a short-term load forecasting method based on EMD-ATT-LSTM, comprising: constructing a fine-grained attention model based on a long short-term memory network; decomposing the preprocessed load data into a number of intrinsic modal functions; and performing load forecasting after checking the stability.

[0006] As a preferred solution of the short-term load forecasting method based on EMD-ATT-LSTM described in the present invention, wherein: the long short-term memory network LSTM calculates h for each time step t for the input sequence t ∈R m , the loop function is expressed as:

[0007] (h t ,c t )=F(h t-1 ,c t-1 ,x t )

[0008] f t =σ(W f [ht-1 x t ]+b f )

[0009] i t =σ(W i [h t-1 x t ]+b i )

[0010]

[0011]

[0012] o t =σ(W o [h t-1 x t ]+b 0 )

[0013] h t =o t ☉tanh(c t )

[0014] Among them, f t ,i t c t , o t ∈R m ; W f , W i ,W c ,W o ∈R m×n Weight matrix; c t is the context vector; b f ,b i , b c and b 0 Bias vector; σ represents the activation function, ⊙ represents the Hadamard product, x t is the input sequence.

[0015] As a preferred solution of the short-term load forecasting method based on EMD-ATT-LSTM described in the present invention, the fine-grained attention model maintains a scalar for each dimension, so that the number of attention scalars increases from T to nT, which is expressed as:

[0016]

[0017] As a preferred solution of the short-term load forecasting method based on EMD-ATT-LSTM described in the present invention, wherein: the score function f is expanded in the fine-grained attention model att (·) to return the hidden state vector h tThe score set corresponding to the dimension is expressed as:

[0018]

[0019] in is assigned to the context vector n th score of the dimension; It is a fully connected neural network, which further normalizes the dimension-related scores in the dimension and is expressed as:

[0020]

[0021] The context vector is calculated as follows:

[0022]

[0023] As a preferred solution of the short-term load forecasting method based on EMD-ATT-LSTM described in the present invention, the load data decomposition includes: dividing the data into a training set and a test set according to the season; performing empirical mode decomposition on the training set to obtain the corresponding intrinsic mode function and residual sequence; adding the data in the test set one by one, performing empirical mode decomposition each time, until the entire test set is added in sequence.

[0024] As a preferred solution of the short-term load forecasting method based on EMD-ATT-LSTM described in the present invention, the intrinsic mode function is expressed as:

[0025]

[0026] Among them, X(t) is the original data, IMF i (t) is the i-th intrinsic mode function, and r(t) is the residual term.

[0027] As a preferred scheme of the short-term load forecasting method based on EMD-ATT-LSTM described in the present invention, wherein: the load forecasting after checking stability includes dividing into Group A and Group B after obtaining all decompositions, representing the decomposition with the largest number of intrinsic modal functions and the remaining decompositions respectively; using the Pearson correlation statistical method and the comparison algorithm to check the stability of each group of intrinsic modal functions; according to the results, dividing the entire decomposition set into three sets of stable intrinsic modal functions, unstable intrinsic modal functions and residuals; using data preparation, LSTM encoder, and the algorithms in attention and decoder LSTM, training the attention-based LSTM to obtain the prediction results of each set; all prediction results are combined by summing to form an integrated output of the time series.

[0028] In the second aspect, another object of the present invention is to provide a short-term load forecasting system based on EMD-ATT-LSTM, including a fine-grained attention model construction module, a data processing module, an empirical mode decomposition module and a load forecasting module; the fine-grained attention model construction module is used to construct a fine-grained attention model based on a long short-term memory network; the data processing module is used to preprocess the original load data; the empirical mode decomposition module is used to decompose the preprocessed load data into several intrinsic mode functions; the load forecasting module is used to perform load forecasting after verifying the stability.

[0029] In a third aspect, a computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the short-term load forecasting method based on EMD-ATT-LSTM as described above when executing the computer program.

[0030] In a fourth aspect, a computer-readable storage medium stores a computer program thereon, and when the computer program is executed by a processor, the steps of the short-term load forecasting method based on EMD-ATT-LSTM as described above are implemented.

[0031] Beneficial effects of the present invention: The present invention provides a short-term load forecasting method and system based on EMD-ATT-LSTM, which effectively decomposes and captures the complex characteristics of load data, thereby improving the accuracy of prediction; through the attention mechanism, the sensitivity of the model to key load characteristics is enhanced; and the LSTM network is used to effectively process long-term dependencies in load data. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0033] Figure 1 An overall flow chart of a short-term load forecasting method based on EMD-ATT-LSTM provided by an embodiment of the present invention;

[0034] Figure 2 A program flow chart of an EMD model of a short-term load forecasting method based on EMD-ATT-LSTM provided by an embodiment of the present invention;

[0035] Figure 3 An EMD-Att-LSTM model training flowchart of a short-term load forecasting method based on EMD-ATT-LSTM is provided for one embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0037] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0038] Example 1

[0039] Reference Figure 1-Figure 3 , as an embodiment of the present invention, provides a short-term load forecasting method based on EMD-ATT-LSTM, comprising:

[0040] S1: Building a fine-grained attention model based on long short-term memory networks;

[0041] Going a step further, Long Short-Term Memory Networks (LSTM): LSTM is a special class of RNN models that has a special ability to remember long-term temporal dependencies. The default property of these networks is to remember information for a long time. In addition, these models have a chain-like structure called a cell state with repeated cycles. The repeated cycles help the network retain relevant information for a short period of time. For an input sequence x 1 , x 2 , ..., x T (x t ∈R n ), the LSTM network calculates h for each time step t t ∈R m The loop function of the LSTM unit can be defined as

[0042] (h t ,c t )=F(h t-1 ,c t-1 ,x t ) (1)

[0043] f t =σ(W f [h t-1 x t ]+b f )

[0044] i t =σ(W i [h t-1 x t ]+b i )

[0045]

[0046]

[0047] o t =o(W o [h t-1 x t ]+b 0 )

[0048] h t =o t ☉tanh(c t ) (2)

[0049] where f t ,i t , ,c t ,o t ∈R m ; W f , W i , W c ,W o ∈R m×n Weight matrix; c t is the context vector; b f ,b i ,b c and b 0 Bias vector; σ represents the activation function, and ☉ represents the Hadamard product.

[0050] It should be noted that existing attention models use a single scalar score of the context vector at time t. It can be observed that instead of using the context vector c t It is better to calculate and use the hidden state h at time t The scalar score of each dimension of , because each dimension represents a different perspective of the internal structure being captured. In the encoder-decoder model calculation, c t share the same attention score, so h t The contribution of all dimensions is equal.

[0051] Furthermore, when different dimensions of the encoded information are considered differently, better model performance can be obtained by paying attention to each dimension. The present invention proposes a fine-grained attention model. The model maintains a scalar for each dimension in H, increasing the number of attention scalars from T to nT. The following equations (3) and (4) show the comparison between the basic attention model and the fine-grained attention model.

[0052] α 1 h 1 +α 2 h 2 +…+α T h T =c t (3)

[0053]

[0054] In the proposed model, we extend the score function f att (·) to return the hidden state vector h t The score set corresponding to the dimension is expressed as

[0055]

[0056] in is assigned to the context vector n th score of the dimension; is a fully connected neural network. These dimension-related scores are further normalized in dimension and expressed as

[0057]

[0058] The context vector is calculated as follows

[0059]

[0060] S2: Decompose the preprocessed load data into several intrinsic mode functions;

[0061] Furthermore, the empirical mode decomposition module (EMD) uses EMD to decompose the preprocessed load data into several intrinsic mode functions (IMFs), which represent different scales and frequency components of the load data.

[0062] EMD decomposition formula:

[0063]

[0064] Among them, X(t) is the original data, IMF i (t) is the i-th intrinsic mode function, and r(t) is the residual term.

[0065] It should be noted that the divide-and-conquer algorithm recursively divides the problem into subproblems for simplification and directly solves the simple subproblems. Subsequently, the subproblem solutions are combined to find the solution to the original problem. This method uses the empirical mode decomposition (EMD) method to decompose the load demand data into multiple IMFs and a residual. Although EMD decomposes complex series into subsequent simple series, it is still necessary to thoroughly check the stability of the decomposition. This information helps the training process as stable information and leads to more accurate estimates. For the analysis, we consider seasonal data. The main idea is to check whether the data remains stable after adding more samples. Our method can be explained as follows.

[0066] Step 1: Split the data into training and testing sets by season.

[0067] Step 2: Perform EMD on the training set to obtain the corresponding IMF and residual sequence.

[0068] Step 3: Add the data in the test set one by one, and perform EMD decomposition each time.

[0069] Step 4: Repeat step 3 until the entire test set is added sequentially.

[0070] S3: Load forecasting after stability verification;

[0071] Furthermore, step 5: After obtaining all decompositions, we divide them into two groups: Group A and Group B, representing the decomposition with the highest number of IMFs and the remaining decompositions, respectively.

[0072] Step 6: Use statistical methods such as Pearson correlation and comparison algorithms such as lower bound to test the stability of each group of IMFs.

[0073] Step 7: Based on the results, the entire decomposition set is divided into three sets: stable IMF, unstable IMF and residual.

[0074] Step 8: Using the algorithms used in data preparation, encoder LSTM, and attention and decoder LSTM, train the attention-based LSTM to obtain prediction results for each set.

[0075] Step 9: All forecast results are combined by summing to form the integrated output of the time series.

[0076] The load forecasting process is mainly divided into two stages. In the first stage, EMD is used to obtain the potential multi-scale dynamics of the load time series, such as trends and seasonality. In the second stage, the decomposed time series is predicted using an attention-based deep learning model. The processing flow of the EMD model is as follows Figure 2 As shown in Figure 2, the fine-grained attention model groups the data into stable and unstable IMF terms and residual terms, and then makes predictions for each group. Figure 3 Shows the training process of the proposed EMD-Att-LSTM model.

[0077] Example 2

[0078] An embodiment of the present invention provides a short-term load forecasting system based on EMD-ATT-LSTM, including: a fine-grained attention model construction module, a data processing module, an empirical mode decomposition module and a load forecasting module; the fine-grained attention model construction module is used to construct a fine-grained attention model based on a long short-term memory network; the data processing module is used to preprocess the original load data; the empirical mode decomposition module is used to decompose the preprocessed load data into a number of intrinsic mode functions; the load forecasting module is used to perform load forecasting after checking stability

[0079] Example 3

[0080] An embodiment of the present invention is different from the first two embodiments in that:

[0081] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0082] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute instructions), or used in conjunction with these instruction execution systems, apparatuses or devices. For the purpose of this specification,

[0083] A "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

[0084] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0085] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0086] Example 4

[0087] An embodiment of the present invention provides a short-term load forecasting method based on EMD-ATT-LSTM. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0088] The data is divided into a training set and a test set in a ratio of 8:2; the training set data is decomposed by EMD to generate several IMF components; each model is used to perform short-term load forecasting (1 hour to 24 hours) on the test set. The final data is shown in the following table:

[0089]

[0090] As can be seen from the above table, the improved model (EMD-ATT-LSTM) shows lower root mean square error RMSE and mean absolute percentage error MAPE in all time ranges, as well as higher determination coefficient R 2 , indicating that its prediction accuracy is better than the baseline model. As the prediction time increases, the effect of the improved model decreases slowly, proving that it has a strong ability to model long-term dependencies.

[0091] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A short-term load forecasting method based on EMD-ATT-LSTM, characterized in that: include: Building a fine-grained attention model based on long short-term memory networks; Decompose the preprocessed load data into several intrinsic mode functions; After checking the stability, load forecasting is performed.

2. The short-term load forecasting method based on EMD-ATT-LSTM according to claim 1, characterized in that: The long short-term memory network LSTM calculates h for each time step t for the input sequence. t ∈R m , the loop function is expressed as: (h t ,c t )=F(h t-1 ,c t-1 ,x t ) f t =σ(W f [h t-1 x t ]+b f ) i t =σ(W i [h t-1 x t ]+b i ) o t =σ(W o [h t-1 x t ]+b0) h t =o t ☉tanh(c t ) in, W f , W i , W c , W o ∈R m×n Weight matrix; c t is the context vector; b f , b i , b c and b0 bias vector; σ represents the activation function, ⊙ represents the Hadamard product, x t is the input sequence.

3. The short-term load forecasting method based on EMD-ATT-LSTM according to claim 1, characterized in that: The fine-grained attention model maintains a scalar for each dimension, increasing the number of attention scalars from nT to nT, expressed as:

4. The short-term load forecasting method based on EMD-ATT-LSTM as claimed in claim 3, characterized in that: Expand the score function f in the fine-grained attention model att (·) to return the hidden state vector h t The score set corresponding to the dimension is expressed as: in is assigned to the context vector n th score of the dimension; It is a fully connected neural network, which further normalizes the dimension-related scores in the dimension and is expressed as: The context vector is calculated as follows:

5. The short-term load forecasting method based on EMD-ATT-LSTM according to claim 4, characterized in that: The load data decomposition includes: dividing the data into a training set and a test set according to season; performing empirical mode decomposition on the training set to obtain the corresponding intrinsic mode function and residual sequence; adding the data in the test set one by one, performing empirical mode decomposition each time, until the entire test set is added in sequence.

6. The short-term load forecasting method based on EMD-ATT-LSTM according to claim 5, characterized in that: The intrinsic modal function is expressed as: Among them, X(t) is the original data, IMF i (t) is the i-th intrinsic mode function, and r(t) is the residual term.

7. The short-term load forecasting method based on EMD-ATT-LSTM according to claim 6, characterized in that: The load forecasting after stability inspection includes, after all decompositions are obtained, dividing them into group A and group B, representing the decomposition with the largest number of intrinsic modal functions and the remaining decompositions, respectively; using the Pearson correlation statistical method and the comparison algorithm to inspect the stability of each group of intrinsic modal functions; According to the results, the entire decomposition set is divided into three sets: stable intrinsic modal functions, unstable intrinsic modal functions, and residuals; the attention-based LSTM is trained using the algorithms in data preparation, LSTM encoder, and attention and decoder LSTM to obtain the prediction results for each set; all prediction results are combined by summing to form an integrated output of the time series.

8. A system using the EMD-ATT-LSTM-based short-term load forecasting method as claimed in any one of claims 1 to 7, characterized in that: include: Fine-grained attention model building module, data processing module, empirical mode decomposition module and load forecasting module; The fine-grained attention model building module is used to build a fine-grained attention model based on the long short-term memory network; The data processing module is used to pre-process the original load data; The empirical mode decomposition module is used to decompose the preprocessed load data into a number of intrinsic mode functions; The load forecasting module is used to perform load forecasting after checking stability.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the short-term load forecasting method based on EMD-ATT-LSTM described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the short-term load forecasting method based on EMD-ATT-LSTM described in any one of claims 1 to 7 are implemented.

Images