Industrial user electricity consumption short-term prediction method based on two-stage attention mechanism

Abstract

The application provides an industrial user electricity consumption short-term prediction method based on a two-stage attention mechanism, and relates to the technical field of electricity consumption short-term prediction, wherein the method comprises the following steps: obtaining original time series data of industrial user electricity consumption; performing data processing on the original time series data of industrial user electricity consumption to generate a sample data set; constructing an industrial user electricity consumption short-term prediction model based on a two-stage attention mechanism, training the model by using the sample data set, and generating a trained model; and realizing short-term prediction of industrial user electricity consumption by inputting real data into the trained model. The application adopting the above scheme solves the difficulties in accuracy and intelligence of the existing industrial user electricity consumption short-term prediction method, and realizes short-term prediction of industrial user electricity consumption in different industries.

Description

Technical Field

[0001] This application relates to the field of short-term electricity consumption forecasting technology, and in particular to a method and system for short-term electricity consumption forecasting of industrial users based on a two-stage attention mechanism. Background Technology

[0002] Electricity consumption forecasting is a crucial component of power management systems. Accurate and real-time analysis and forecasting not only ensure the safe operation of the power system but also provide a basis for dispatching, supplying electricity, and trading in a market environment. As the proportion of electricity consumption by industrial users in the total social electricity consumption continues to rise, their electricity consumption behavior has a significant impact on the power system's supply and demand balance. Industrial user electricity consumption time-series data exhibits nonlinear and non-stationary characteristics. Understanding the patterns of industrial user electricity consumption over time helps in regulating electricity demand at the city, regional, or national level. Because electricity is fundamental to social life, forecasting errors can lead to increased operating costs and losses and inconveniences for businesses and daily life. Therefore, with the marketization of electricity production and consumption, electricity consumption forecasting for industrial users faces higher standards and challenges in terms of accuracy, reliability, and intelligence.

[0003] Currently, electricity consumption forecasting can be categorized into long-term, medium-term, and short-term forecasts based on time scale. Medium- and long-term forecasts are used to determine demand response strategies and formulate plans for power equipment installation and maintenance. Short-term forecasts, typically measured in minutes or hours, are used for short-term dispatching and energy storage planning. The real-time information obtained can assist in responding to emergencies, enabling rapid and efficient dispatching, and alleviating the supply-demand imbalance in modern power systems. Industrial user electricity consumption time-series data is characterized by large fluctuations, high frequency, and periodic changes, making forecasting more challenging. Extensive research has explored and studied short-term forecasting models using actual data.

[0004] Initially, statistical methods were used to predict industrial electricity consumption. These methods typically assume linear characteristics in the input data and fail to capture the nonlinear and non-stationary features of short-term electricity consumption data, leading to significant errors when dealing with sudden changes. Machine learning methods can better handle the non-stationary and nonlinear characteristics of electricity consumption time-series data, but face complex feature extraction challenges in the early stages. With the development of deep learning methods, deep neural networks have significantly improved their intelligence in predicting important power system parameters. They extract data features layer by layer through deep structures, exhibiting strong adaptive capabilities related to time series data. Summary of the Invention

[0005] This application aims to at least partially address one of the technical problems in the related art.

[0006] Therefore, the first objective of this application is to propose a short-term prediction method for industrial user electricity consumption based on a two-stage attention mechanism. This method addresses the difficulties in accuracy and intelligence of existing short-term prediction methods for industrial user electricity consumption. By decomposing and reconstructing the time-series data of industrial user electricity consumption, high-frequency, low-frequency, and trend components representing its different frequency domain characteristics are obtained. A short-term prediction dataset is constructed based on the sliding window technique. Based on an encoder-decoder framework, a two-stage attention mechanism is introduced to extract the spatial and temporal dependencies between input variables. A two-layer convolutional neural network is used to extract features from the original sequence to further capture the instantaneous mutation features of the user electricity consumption time-series data. The final prediction result is obtained, enabling short-term prediction of industrial user electricity consumption in different industries.

[0007] The second objective of this application is to propose a short-term forecasting system for industrial user electricity consumption based on a two-stage attention mechanism.

[0008] The third objective of this application is to provide a non-transitory computer-readable storage medium.

[0009] To achieve the above objectives, the first aspect of this application proposes a short-term prediction method for industrial user electricity consumption based on a two-stage attention mechanism, comprising: acquiring raw time-series data of industrial user electricity consumption; processing the raw time-series data of industrial user electricity consumption to generate a sample dataset; constructing a short-term prediction model for industrial user electricity consumption based on a two-stage attention mechanism, and training the model using the sample dataset to generate a trained model; and realizing short-term prediction of industrial user electricity consumption by inputting real data into the trained model.

[0010] Optionally, in one embodiment of this application, the raw time-series data of industrial user electricity consumption is processed to generate a dataset, including the following steps:

[0011] Data cleaning is performed on the raw time-series data of electricity consumption by industrial users.

[0012] By integrating empirical mode decomposition algorithms, the cleaned industrial user electricity consumption time series data is transformed into a set of subsequences arranged from high to low frequency, where each subsequence reflects the characteristics of the original sequence in a specific frequency domain.

[0013] The Fine-to-Coarse reconstruction algorithm is used to calculate the high-frequency, low-frequency, and trend components of the original sequence from a set of subsequences.

[0014] The prediction indicators of the original data are standardized, and the standardized results are used to construct a sample dataset using the sliding window technique. The sample dataset is then divided into training and testing sets according to the proportion. The prediction indicators include four time series data: the cleaned industrial user electricity consumption time series data, the high-frequency, low-frequency and trend components obtained by decomposition.

[0015] Optionally, in one embodiment of this application, an encoder-decoder structure is used as the model framework to construct a short-term prediction model for industrial user electricity consumption with a two-stage attention mechanism, wherein...

[0016] In the encoder stage, a recurrent neural network is combined with an attention mechanism to extract the spatial correlation between various variables and the target sequence at the same time.

[0017] In the decoder stage, a recurrent neural network is combined with an attention mechanism, using the encoder's computation results as input data to extract the time-related features of the same variable at different times;

[0018] In addition to the encoder-decoder structure, in order to further capture the mutation features of the original sequence, a deep neural network structure with two layers of convolution is constructed using the time series data of industrial users' electricity consumption as the input variable. The deep neural network structure with two layers of convolution and pooling operations is used.

[0019] The outputs of the encoder-decoder stage and the two-layer convolution stage are used as input data for the output layer. The final prediction result is calculated by using the concatenated vector of the two-stage prediction results through a fully connected layer, thus completing the model construction.

[0020] Optionally, in one embodiment of this application, the model is trained using a dataset, specifically as follows:

[0021] The model is trained using the training set in the dataset, including: updating parameters using the backpropagation algorithm during model training; the objective function of model training is the mean squared error (MSE); mini-batch stochastic gradient descent and Adam optimizer are selected to minimize the difference between the predicted and the true values; where the batch size depends on the size of the sample dataset.

[0022] The model is validated using the validation set in the dataset to ensure that no overfitting occurs.

[0023] To achieve the above objectives, a second aspect of this application proposes a short-term industrial user electricity consumption forecasting system based on a two-stage attention mechanism, comprising: an acquisition module, a data processing module, an industrial user electricity consumption short-term forecasting model generation module, and a forecasting module, wherein...

[0024] The acquisition module is used to acquire raw time-series data of electricity consumption by industrial users;

[0025] The data processing module is used to process the raw time-series data of electricity consumption by industrial users and generate sample datasets.

[0026] The module for generating a short-term prediction model for industrial user electricity consumption is used to construct a short-term prediction model for industrial user electricity consumption with a two-stage attention mechanism, and to train the model using a dataset to generate a trained model.

[0027] The prediction module is used to make short-term predictions of electricity consumption for industrial users by inputting real data into a trained model.

[0028] Optionally, in one embodiment of this application, the data processing module is specifically used for:

[0029] Data cleaning is performed on the raw time-series data of electricity consumption by industrial users.

[0030] By integrating empirical mode decomposition algorithms, the cleaned industrial user electricity consumption time series data is transformed into a set of subsequences arranged from high to low frequency, where each subsequence reflects the characteristics of the original sequence in a specific frequency domain.

[0031] The Fine-to-Coarse reconstruction algorithm is used to calculate the high-frequency, low-frequency, and trend components of the original sequence from a set of subsequences.

[0032] The prediction indicators of the original data are standardized, and the standardized results are used to construct a sample dataset using the sliding window technique. The sample dataset is then divided into training and testing sets according to the proportion. The prediction indicators include four time series data: the cleaned industrial user electricity consumption time series data, the high-frequency, low-frequency and trend components obtained by decomposition.

[0033] To achieve the above objectives, a third aspect of this application proposes a non-transitory computer-readable storage medium that, when the instructions in the storage medium are executed by a processor, can perform a short-term prediction method for industrial user electricity consumption based on a two-stage attention mechanism.

[0034] This application presents a method and system for short-term industrial user electricity consumption prediction based on a two-stage attention mechanism, along with a non-transitory computer-readable storage medium. These methods address the limitations of existing short-term industrial user electricity consumption prediction methods in terms of accuracy and intelligence. By decomposing and reconstructing time-series industrial user electricity consumption data, high-frequency, low-frequency, and trend components representing different frequency domain characteristics are obtained. A short-term prediction dataset is constructed based on sliding window technology. Using an encoder-decoder framework, a two-stage attention mechanism is introduced to extract spatial and temporal dependencies between input variables. A two-layer convolutional neural network is employed to extract features from the original sequence, further capturing the instantaneous abrupt changes in the user electricity consumption time-series data. The final prediction result is obtained, enabling short-term prediction of industrial user electricity consumption in different industries.

[0035] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0036] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0037] Figure 1 A flowchart illustrating a short-term prediction method for industrial user electricity consumption based on a two-stage attention mechanism, provided in Embodiment 1 of this application;

[0038] Figure 2 This is a flowchart illustrating the overall process of the short-term electricity consumption forecasting method for industrial users based on a two-stage attention mechanism, as described in this application.

[0039] Figure 3 This is a schematic diagram illustrating the statistical data of electricity consumption of an industrial user according to an embodiment of this application;

[0040] Figure 4 This is a graph showing the EEMD decomposition results of statistical data on the electricity consumption of an industrial user in an embodiment of this application.

[0041] Figure 5 The image shows the result of reconstructing statistical data on the electricity consumption of an industrial user according to an embodiment of this application using the Fine-to-Coarse algorithm.

[0042] Figure 6 This is a schematic diagram of the sliding window mechanism of the short-term prediction method for industrial user electricity consumption based on a two-stage attention mechanism according to an embodiment of this application.

[0043] Figure 7 This is a schematic diagram of the input data format and prediction process of the short-term prediction method for industrial user electricity consumption based on a two-stage attention mechanism according to an embodiment of this application.

[0044] Figure 8 This is a schematic diagram of the ASFM model structure of the short-term prediction method for industrial user electricity consumption based on a two-stage attention mechanism according to an embodiment of this application.

[0045] Figure 9 This is a schematic diagram of the first-stage encoder + Attention structure of the ASFM model for the short-term prediction method of industrial user electricity consumption based on a two-stage attention mechanism, as described in an embodiment of this application.

[0046] Figure 10 This is a schematic diagram of the second-stage decoder + Attention mechanism of the ASFM model in the short-term prediction method for industrial user electricity consumption based on a two-stage attention mechanism according to an embodiment of this application.

[0047] Figure 11 This is a schematic diagram of the ASFM convolutional structure of the short-term prediction method for industrial user electricity consumption based on a two-stage attention mechanism according to an embodiment of this application.

[0048] Figure 12 This is a schematic diagram of the structure of a short-term forecasting system for industrial user electricity consumption based on a two-stage attention mechanism, provided in Embodiment 2 of this application. Detailed Implementation

[0049] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0050] The following describes, with reference to the accompanying drawings, a method and system for short-term prediction of industrial user electricity consumption based on a two-stage attention mechanism, according to embodiments of this application.

[0051] Figure 1 This is a flowchart of a short-term prediction method for industrial user electricity consumption based on a two-stage attention mechanism, provided in Embodiment 1 of this application.

[0052] like Figure 1 As shown, the short-term electricity consumption forecasting method for industrial users based on a two-stage attention mechanism includes the following steps:

[0053] Step 101: Obtain raw time-series data of electricity consumption by industrial users;

[0054] Step 102: Process the raw time-series data of industrial users' electricity consumption to generate a sample dataset;

[0055] Step 103: Construct a short-term prediction model for industrial user electricity consumption using a two-stage attention mechanism, and train the model using a sample dataset to generate a trained model.

[0056] Step 104: Short-term prediction of industrial users' electricity consumption is achieved by inputting real data into the trained model.

[0057] This application presents a method for short-term prediction of industrial user electricity consumption based on a two-stage attention mechanism. The method involves: acquiring raw time-series data of industrial user electricity consumption; processing this raw time-series data to generate a sample dataset; constructing a short-term prediction model for industrial user electricity consumption based on a two-stage attention mechanism; training the model using the sample dataset; and finally, inputting real data into the trained model to achieve short-term prediction of industrial user electricity consumption. This method addresses the limitations of existing short-term prediction methods in terms of accuracy and intelligence. By decomposing and reconstructing the time-series data of industrial user electricity consumption, high-frequency, low-frequency, and trend components representing different frequency domain characteristics are obtained. A short-term prediction dataset is constructed based on sliding window technology. A two-stage attention mechanism is introduced based on an encoder-decoder framework to extract spatial and temporal dependencies between input variables. A two-layer convolutional neural network is used to extract features from the original sequence, further capturing the instantaneous abrupt changes in the time-series data of user electricity consumption. Finally, the method yields the prediction result, enabling short-term prediction of industrial user electricity consumption in different industries.

[0058] The problem of short-term forecasting of electricity consumption by industrial users can be formally defined as:

[0059] For the original industrial user electricity consumption sequence O(t), where t = 1, 2, ..., T represents time and is the sequence length, the goal of the short-term forecasting method is to analyze the historical variation patterns of O(t) and, for any time point t in the sequence, predict the value at time τ in the future, with t as the baseline. The expression is as follows:

[0060]

[0061] in, F(·) represents the prediction model constructed in this application, which mainly focuses on the scenario of τ=1 for the purpose of predicting the value of the target (at time t+τ).

[0062] Furthermore, in this embodiment of the application, the raw time-series data of industrial user electricity consumption is processed to generate a dataset, including the following steps:

[0063] Data cleaning is performed on the raw time-series data of electricity consumption by industrial users.

[0064] By integrating empirical mode decomposition algorithms, the cleaned industrial user electricity consumption time series data is transformed into a set of subsequences arranged from high to low frequency, where each subsequence reflects the characteristics of the original sequence in a specific frequency domain.

[0065] The Fine-to-Coarse reconstruction algorithm is used to calculate the high-frequency, low-frequency, and trend components of the original sequence from a set of subsequences.

[0066] The prediction indicators of the original data are standardized, and the standardized results are used to construct a sample dataset using the sliding window technique. The sample dataset is then divided into training and testing sets according to the proportion. The prediction indicators include four time series data: the cleaned industrial user electricity consumption time series data, the high-frequency, low-frequency and trend components obtained by decomposition.

[0067] To address the high-frequency, unstable, and nonlinear characteristics of industrial user electricity consumption time series data, the original sequence of industrial user electricity consumption data after data cleaning is decomposed using the Integrated Empirical Mode Decomposition (EEMD) algorithm to extract detailed features of the original complex sequence. The high-frequency component, low-frequency component, and trend component of the original sequence are calculated using the Fine-to-Coarse reconstruction algorithm and used as data features of high-frequency fluctuations, low-frequency changes, and overall trends of the target sequence, respectively.

[0068] The time series of industrial user electricity consumption O(t) is decomposed using the Ensemble Empirical Mode Decomposition (EEMD) algorithm. The EEMD algorithm can decompose an initial complex time series into a finite number of local feature sequences representing different time scales of the original series—Intrinsic Mode Functions (IMFs)—and a residual sequence. The specific steps of the EEMD algorithm are as follows:

[0069] Step 201: Given a white noise sequence (e.g., a Gaussian white noise sequence z(t)∽N(0,σ),... 2 Adding this to the target sequence O(t) of industrial user electricity consumption yields a new sequence.

[0070] Step 202, according to the calculation method of Empirical Mode Decomposition (EMD), for The decomposition yields a series of IMFs and a residual term (containing...) and R n (t));

[0071] Step 203: Repeat steps 201 and 202 a total of m times, adding a different white noise sequence each time. Take the average value of each IMF after m decompositions as the final IMF sequence. Similarly, take the average value of the residual terms obtained in each calculation as the final residual term.

[0072] For the EMD algorithm process, it is geared towards each sequence. Assuming input When i=1, the specific implementation process is as follows:

[0073] Step 301, Identification All extreme points in the middle;

[0074] Step 302: Use cubic spline interpolation to interpolate the maxima and minima to obtain... The corresponding upper and lower envelopes are denoted as e. max (t), e min (t);

[0075] Step 303, calculate the local mean: a1(t)=[e max (t)+e min [(t)] / 2;

[0076] Step 304, from Subtracting a1(t) from the middle, we obtain the current oscillation pattern d1(t):

[0077] Step 305: Judge d1(t). If d1(t) meets the termination condition, then set the IMF... i (t) = d1(t), thus obtaining the first IMF, proceed to step 306; otherwise, let Proceed to step 301;

[0078] Step 306, from Subtract the current IMF component from the middle. i (t), like Monotonic or meeting the termination condition (termination condition: Where l is the sequence length, the iteration stops, and the trend term is calculated. like If the termination condition has not yet been met, then let i = i + 1, then proceed to step 301.

[0079] Through EEMD decomposition, the original nonlinear, non-stationary, and high-frequency fluctuating industrial user electricity consumption time-series data is transformed into a set of subsequences arranged from high to low frequency. Each subsequence reflects the characteristics of the original sequence in a specific frequency domain and can be used as derived variables to assist in target prediction. Considering that the number of subsequences generated by the EEMD algorithm increases with the length of the time series (the relationship between the number of subsequences n and the sequence length T is n = (log₂T) - 1), the Fine-to-Coarse algorithm is used to reconstruct them, obtaining the high-frequency, low-frequency, and trend components of the original sequence.

[0080] Applying the Fine-to-Coarse reconstruction algorithm to the EEMD decomposition results of industrial user electricity consumption data distinguishes components with higher oscillation frequencies from those with lower oscillation frequencies, allowing for the merging of components with similar characteristics and reducing the computational burden on the model.

[0081] Use c i This represents the subsequences in the decomposition results (IMFs are arranged in ascending order of their numbers, with the residual sequence Residue at the end), i = 1, 2, ..., n+1, where n is the number of IMFs. The Fine-to-Coarse process is as follows:

[0082] Step 401, calculate c1 to c in sequence. n The mean of the sum of each subsequence (excluding Residue), denoted as T;

[0083] Step 402: Use t-test to determine for what value of i T is significantly different from 0;

[0084] Step 403: When i is determined to make T significantly different from 0, the IMFs numbered 1 to i are defined as high-frequency oscillating subsequences, and the high-frequency components are obtained by adding them together, denoted as H; the Residue sequence is used as the trend component that reflects the long-term change trend of the original sequence, denoted as Tr; the remaining sequences are added together to obtain the low-frequency components, denoted as L.

[0085] The reconstructed components, along with the original sequence (O, H, L, Tr, a total of four items), serve as input variables for short-term electricity consumption forecasting. Each forecast indicator is standardized, and the calculation method is as follows:

[0086]

[0087] Where, x max x min x represents the maximum and minimum values ​​of the sequence, respectively. norm These are the standardized calculation results for the corresponding time points. The standardized sequences are denoted as O. s H s Ls Tr s A sample set is constructed using the sliding window technique.

[0088] When using indicators of different orders of magnitude to predict a target, standardization is performed on each predictive indicator to avoid amplifying the role of higher-value indicators in the comprehensive analysis. To further conduct model training and testing, the sample dataset is divided into training and testing sets in a ratio of 8:2.

[0089] Furthermore, in this embodiment, an encoder-decoder structure is used as the model framework to construct a short-term prediction model for industrial user electricity consumption based on a two-stage attention mechanism, wherein...

[0090] In the encoder stage, a recurrent neural network is combined with an attention mechanism to extract the spatial correlation between various variables and the target sequence at the same time.

[0091] In the decoder stage, a recurrent neural network is combined with an attention mechanism, using the encoder's computation results as input data to extract the time-related features of the same variable at different times;

[0092] In addition to the encoder-decoder structure, in order to further capture the mutation features of the original sequence, a deep neural network structure with two layers of convolution is constructed using the time series data of industrial users' electricity consumption as the input variable. The deep neural network structure with two layers of convolution and pooling operations is used.

[0093] The outputs of the encoder-decoder stage and the two-layer convolution stage are used as input data for the output layer. The final prediction result is calculated by using the concatenated vector of the two-stage prediction results through a fully connected layer, thus completing the model construction.

[0094] The encoder-decoder structure of the short-term electricity consumption prediction model ASFM for industrial users is built based on LSTM units. In the encoder stage, the input sequence is encoded into feature representations, and an attention mechanism is used to learn the spatial correlation between input variables, assigning appropriate importance weights to different variables.

[0095] For a given input data (in this application) The main task of ASFM at time t can be described as learning the input data. To encoder hidden state h t Nonlinear mapping function:

[0096]

[0097] Among them, h t ∈R m m represents the number of units contained in the hidden state, f e This represents an LSTM unit in the encoder stage. An LSTM unit contains cell states. t And three key Sigmoid gate structures—the forget gate. t Input gate t and output gate t The calculation process is as follows:

[0098]

[0099]

[0100]

[0101]

[0102] h t =o t ⊙tanh(s t )

[0103] [*;*] denotes the vector concatenation operation. Represents the hidden state h of the previous time step. t-1 With the present The connection, W f W i W o , and b f b i b o b s ∈R m These are the parameters to be learned. σ is the Sigmoid function, and ⊙ represents element-wise multiplication. This application introduces a feedforward network—Attention—that can be jointly trained with the encoder, for the k-th sequence in the input variable. The weight calculation process for Attention is as follows:

[0104]

[0105]

[0106] The weights are determined by the hidden state h of the encoder at the previous time step. t-1 and the input at the current moment Jointly determined, representing the influence of each input variable on the outcome. [ht-1 ;s t-1 [] represents the cascade of hidden layer states and cell states at time t-1, v e b e ∈R T W e ∈R T×2m U e ∈R T×T These are the parameters to be learned during the training process, and the result is obtained after tanh operation. The Softmax operation shown makes all Attention weights sum to 1.

[0107] At different times, each input variable has its own weight, which allows us to measure the magnitude of the influence of different variables on the final prediction result. This introduces variables...

[0108]

[0109] Therefore, the LSTM cell calculation method can be updated as follows:

[0110]

[0111] This application introduces an Attention mechanism in the decoder stage as a filter for the time dependence of each input variable, and calculates the hidden state h of the encoder. t Perform automatic selection and weighting across time steps.

[0112] The hidden state history value is d t′-1 ∈R p The cell state is s′ t′-1 ∈R p p is the number of units, t′ represents time, t′=1,2,......,T. For the encoder hidden state h t The Attention process for weight calculation is as follows:

[0113]

[0114]

[0115] Among them, [d t′-1 ;s′ t′-1 ]∈R 2p For vector concatenation operations, where v d ,b d ∈R p W d ∈R m×2p and U e ∈R m×mThese are the learning parameters. Attention weights. This represents the importance of the t-th hidden state in the encoder, for h within the time window T. t Vector c t′ The calculation method is denoted as:

[0116]

[0117] c t′ The sequence of electricity consumption values ​​for industrial users {o1,o2,...,o T Combining these, we get

[0118]

[0119] Among them, [o T c T ]∈R m+1 The parameters to be learned include and Similar to the computation process in the encoder stage, the hidden state in the decoder stage can be represented as:

[0120]

[0121] The nonlinear mapping function f in this stage d Also using LSTM units, the update process is as follows:

[0122]

[0123]

[0124]

[0125]

[0126] d t′ =o′ t′ ⊙tanh(s′ t′ )

[0127] in, It is the connection between the hidden state of the previous moment and the encoder input, W′ f ,W′ i ,W′ o ,W′ s ∈R p×(p+1) and b′ f ,b′ i ,b′ o ,b′ s ∈R p These are the parameters to be learned. σ represents the Sigmoid function, and ⊙ represents element-wise multiplication.

[0128] Therefore, by using an encoder-decoder structure with a joint attention mechanism, the prediction result of the target sequence of electricity consumption for industrial users can be obtained. Notation:

[0129]

[0130] This application employs a two-stage convolutional structure combined with an encoder-decoder prediction stage. The input variable for the convolutional stage is O. s Historical values, with a window size of T. Features are extracted from the original sequence through convolutional operations, compensating for the shortcomings of attention-based encoder-decoder models in capturing mutation information.

[0131] The input to a convolutional structure is a two-dimensional tensor of size M×N, and the weights of the convolutional kernel are tensors of size P×Q. Therefore, the convolution operation is represented as:

[0132]

[0133] O mn This is the output of the structure at position (m,n), w pq Let i be the weight at position (p,q) in the convolution kernel. (m+p)(n+q) Let be the corresponding point in the input tensor, b be the bias, and f(·) be the activation function. Assuming a convolution kernel of size z′ is used, the width of the output tensor is (Tz′+1). Therefore, the m-th value in the output vector is O. m It can be represented as:

[0134]

[0135] Where m = 1, 2, ..., Tz′+1. Based on this, one-dimensional Max-pooling is used to generate new features, taking the maximum value within a specified pooling window based on the previous feature mapping. Due to the high complexity of industrial user electricity consumption time-series data, the model designed in this application employs two layers of convolution and pooling operations. After two stages of convolution-pooling layer operations, the output result is calculated by a fully connected layer.

[0136] Combining the computation results from the encoder-decoder stage, and These represent the prediction results obtained through two different feature extraction algorithms. To calculate the final result, the final stage of the prediction model uses a fully connected layer to concatenate the prediction results from the two stages into a vector. The predicted value of the target sequence at time T+τ was calculated.

[0137] Furthermore, in this embodiment of the application, the model is trained using a dataset, specifically as follows:

[0138] The model is trained using the training set in the dataset, including: updating parameters using the backpropagation algorithm during model training; the objective function of model training is the mean squared error (MSE); mini-batch stochastic gradient descent and Adam optimizer are selected to minimize the difference between the predicted and the true values; where the batch size depends on the size of the sample dataset.

[0139] The model is validated using the validation set in the dataset to ensure that no overfitting occurs.

[0140] The batch size depends on the specific size of the data, and the learning rate is 0.001.

[0141] The parameters and corresponding selection ranges involved in ASFM include: time window: T∈[6,12,18,24]; encoder and decoder hidden layer size: m=p∈[16,32,64,128]; convolutional kernel size: Conv_kernal∈{[1×2],[1×3],[1×5]; pooling window size: Pooling-size; grid search is used during the process, and parameters are selected within the set range.

[0142] After model training, parameter selection, and overfitting testing, the ASFM model is applied to predict the electricity consumption of industrial users, and combined with relevant devices, it enables short-term prediction of electricity consumption of industrial users.

[0143] Figure 2 This is a flowchart illustrating the overall process of the short-term electricity consumption forecasting method for industrial users based on a two-stage attention mechanism, as described in this application.

[0144] like Figure 2 As shown, the short-term prediction method for industrial user electricity consumption based on a two-stage attention mechanism can be divided into two parts. The first part is the industrial user electricity consumption time-series data processing and dataset construction stage; the second part is the deep learning model construction and prediction implementation stage based on the two-stage attention mechanism. The input is the original historical time-series data of industrial user electricity consumption; the second part is the industrial user electricity consumption time-series data processing and dataset construction stage, which mainly includes data preprocessing, time-series data feature extraction, data standardization, and dataset construction; the third part is the deep learning model construction and prediction implementation stage based on the two-stage attention mechanism, which mainly includes the design and construction of a recurrent neural network model (ASFM) based on the two-stage attention mechanism, parameter setting and model training, and ASFM model testing; finally, the output is the industrial user electricity consumption prediction model and device.

[0145] Figure 3This is a schematic diagram of the electricity consumption statistics of an industrial user in an embodiment of this application.

[0146] like Figure 3 As shown in the figure, the statistical data of daily and hourly electricity consumption of an industrial user during a certain period is as follows.

[0147] Figure 4 This is a graph showing the EEMD decomposition results of statistical data on the electricity consumption of an industrial user in an embodiment of this application.

[0148] like Figure 4 As shown in the figure, in the EEMD decomposition result graph of the statistical data of an industrial user's electricity consumption, the top curve is the original sequence statistical data, and the curve below the original sequence statistical data is a series of subsequences arranged from high to low frequency. Through EEMD decomposition, the original nonlinear, non-stationary, high-frequency fluctuating industrial user electricity consumption time series data is transformed into a set of subsequences arranged from high to low frequency, where each subsequence reflects the characteristics of the original sequence in a specific frequency domain.

[0149] Figure 5 The image shows the result of reconstructing statistical data on the electricity consumption of an industrial user according to an embodiment of this application using the Fine-to-Coarse algorithm.

[0150] like Figure 5 As shown in the figure, after Fine-to-Coarse reconstruction, the high-frequency, low-frequency, and trend components of the electricity consumption statistics of an industrial user are respectively labeled. Applying the Fine-to-Coarse reconstruction algorithm to the EEMD decomposition results of the electricity consumption statistics of an industrial user distinguishes components with higher oscillation frequencies from those with lower oscillation frequencies, and can merge components with similar characteristics, reducing the computational burden on the model.

[0151] Figure 6 This is a schematic diagram of the sliding window mechanism of the short-term prediction method for industrial user electricity consumption based on a two-stage attention mechanism according to an embodiment of this application.

[0152] like Figure 6 As shown in the schematic diagram of the sliding window mechanism in the short-term prediction method for industrial user electricity consumption based on the two-stage attention mechanism, t represents the current time, T represents the window size, τ is the target prediction step size, and each sample contains historical data with a window size of T and future prediction target data with a distance of τ from the current time t.

[0153] Figure 7 This is a schematic diagram of the input data format and prediction process of the short-term industrial user electricity consumption prediction method based on a two-stage attention mechanism according to an embodiment of this application.

[0154] like Figure 7As shown, the input data contains 4 variables, including historical values ​​of the original sequence, high-frequency sequence, low-frequency sequence and trend component. The window size is represented by T. The input variables are processed by the prediction model ASFM to calculate the short-term forecast results of industrial user electricity consumption.

[0155] Figure 8 This is a schematic diagram of the ASFM model structure of the short-term prediction method for industrial user electricity consumption based on a two-stage attention mechanism, according to an embodiment of this application.

[0156] like Figure 8 As shown, the input variable of the model is the high-frequency sequence H obtained by decomposing the original sequence. s Low-frequency sequence L s Trend sequence Tr s and the original sequence history value O s After variables are input into the model, the spatial correlation between variables is first extracted by an encoder combined with an attention structure. The calculation results of this stage are then processed by a decoder combined with an attention structure to obtain the calculation results of the encoder-decoder stage. Furthermore, to enhance the model's ability to extract cathodic features from the original sequence, the historical values ​​O of the original sequence are... s The computational results of the convolutional stage are obtained through convolutional structure processing (including two layers of convolution-pooling operations). In the final prediction stage, the prediction results of the two stages are compared. and The final prediction result is obtained through calculation via a fully connected layer.

[0157] Figure 9 This is a schematic diagram of the first-stage encoder + Attention structure of the ASFM model for the short-term prediction method of industrial user electricity consumption based on a two-stage attention mechanism, as described in an embodiment of this application.

[0158] like Figure 9 As shown, time dependency and space dependency coexist in the input variable O. s H s L s Tr s In the model, after the variables enter the model, the spatial correlation between the variables at the same time is calculated by the first stage Attention. Different variables are assigned corresponding weights according to the degree of correlation between the variables and the prediction target. The results are obtained after calculation by the LSTM unit and output to the next stage.

[0159] Figure 10 This is a schematic diagram of the second-stage decoder + Attention mechanism of the ASFM model in the short-term prediction method for industrial user electricity consumption based on a two-stage attention mechanism, as described in an embodiment of this application.

[0160] like Figure 10 As shown, the LSTM computation unit in the decoder stage uses the computation results from the encoder stage to calculate the attention weights, extracts the temporal dependencies of the same variable at different times, and then outputs the computation results.

[0161] Figure 11 This is a schematic diagram of the ASFM convolutional structure of the short-term prediction method for industrial user electricity consumption based on a two-stage attention mechanism, as described in an embodiment of this application.

[0162] like Figure 11 As shown, the ASFM model of the short-term prediction method for industrial user electricity consumption based on a two-stage attention mechanism employs two layers of convolution and pooling operations. After the historical values ​​of the original sequence are input into the model as input variables, the network structure for feature extraction is as follows: one-dimensional convolutional layer --> pooling layer --> one-dimensional convolutional layer --> pooling layer --> fully connected layer. Finally, the result of this stage is calculated.

[0163] Figure 12 This is a schematic diagram of the structure of a short-term forecasting system for industrial user electricity consumption based on a two-stage attention mechanism, provided in Embodiment 2 of this application.

[0164] like Figure 12 As shown, the short-term industrial user electricity consumption forecasting system based on a two-stage attention mechanism includes: an acquisition module 10, a data processing module 20, an industrial user electricity consumption short-term forecasting model generation module 30, and a forecasting module 40, wherein...

[0165] Module 10 is used to acquire raw time-series data of electricity consumption by industrial users;

[0166] Data processing module 20 is used to process the raw time-series data of electricity consumption of industrial users and generate sample datasets;

[0167] The industrial user electricity consumption short-term prediction model generation module 30 is used to construct an industrial user electricity consumption short-term prediction model with a two-stage attention mechanism, and to train the model using a dataset to generate a trained model.

[0168] The prediction module 40 is used to make short-term predictions of electricity consumption by industrial users by inputting real data into the trained model.

[0169] Furthermore, in this embodiment of the application, the data processing module is specifically used for:

[0170] Data cleaning is performed on the raw time-series data of electricity consumption by industrial users.

[0171] By integrating empirical mode decomposition algorithms, the cleaned industrial user electricity consumption time series data is transformed into a set of subsequences arranged from high to low frequency, where each subsequence reflects the characteristics of the original sequence in a specific frequency domain.

[0172] The Fine-to-Coarse reconstruction algorithm is used to calculate the high-frequency, low-frequency, and trend components of the original sequence from a set of subsequences.

[0173] The prediction indicators of the original data are standardized, and the standardized results are used to construct a sample dataset using the sliding window technique. The sample dataset is then divided into training and testing sets according to the proportion. The prediction indicators include four time series data: the cleaned industrial user electricity consumption time series data, the high-frequency, low-frequency and trend components obtained by decomposition.

[0174] The short-term industrial user electricity consumption forecasting system based on a two-stage attention mechanism according to this application includes: an acquisition module, a data processing module, an industrial user electricity consumption short-term forecasting model generation module, and a forecasting module. The acquisition module is used to acquire raw time-series data of industrial user electricity consumption. The data processing module is used to process the raw time-series data of industrial user electricity consumption to generate a sample dataset. The industrial user electricity consumption short-term forecasting model generation module is used to construct a short-term industrial user electricity consumption forecasting model based on a two-stage attention mechanism and train the model using the dataset to generate a trained model. The forecasting module is used to achieve short-term forecasting of industrial user electricity consumption by inputting real data into the trained model. Therefore, this method can solve the difficulties in accuracy and intelligence of existing short-term forecasting methods for industrial users' electricity consumption. By decomposing and reconstructing the time-series data of industrial users' electricity consumption, high-frequency, low-frequency, and trend components representing its different frequency domain characteristics are obtained. A short-term forecasting dataset is constructed based on the sliding window technique. Based on the encoder-decoder framework, a two-stage attention mechanism is introduced to extract the spatial and temporal dependencies between input variables. A two-layer convolutional neural network is used to extract features from the original sequence to further capture the instantaneous mutation features of the user's electricity consumption time-series data. The final prediction result is obtained, realizing short-term forecasting of electricity consumption for industrial users in different industries.

[0175] To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the short-term prediction method for industrial user electricity consumption based on a two-stage attention mechanism as described in the above embodiments.

[0176] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0177] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0178] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0179] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0180] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0181] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0182] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0183] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A short-term forecasting method for industrial user electricity consumption based on a two-stage attention mechanism, characterized in that, Includes the following steps: Obtain the raw time-series data of industrial users' electricity consumption O(t), t=1, 2, ..., T, where T represents time and is the sequence length; The raw time-series data of electricity consumption of the industrial users are processed to generate a sample dataset; A short-term prediction model for industrial user electricity consumption based on a two-stage attention mechanism was constructed, and the model was trained using a sample dataset to generate a trained model. Short-term prediction of electricity consumption by industrial users can be achieved by inputting real data into a trained model. For any time point t in sequence O(t), the prediction model predicts the value at time τ in the future, with t as the baseline, as follows: in, F(·) represents the predicted value at time t+τ; An encoder-decoder structure is used as the model framework to construct a short-term prediction model for industrial user electricity consumption using the two-stage attention mechanism. In the encoder stage, a recurrent neural network is combined with an attention mechanism to extract the spatial correlation between various variables and the target sequence at the same time. In the decoder stage, a recurrent neural network is combined with an attention mechanism, using the encoder's computation results as input data to extract the time-related features of the same variable at different times; In addition to the encoder-decoder structure, in order to further capture the mutation features of the original sequence, a deep neural network structure with two layers of convolution is constructed using industrial user electricity consumption time series data as input variables. The deep neural network structure with two layers of convolution and pooling operations is used. The outputs of the encoder-decoder stage and the two-layer convolution stage are used as input data for the output layer. The final prediction result is calculated by using the concatenated vector of the two-stage prediction results through a fully connected layer, thus completing the model construction.

2. The method as described in claim 1, characterized in that, The process of processing the raw time-series data of the industrial user's electricity consumption to generate a dataset includes the following steps: Data cleaning is performed on the raw time-series data of the industrial users' electricity consumption; By integrating empirical mode decomposition algorithms, the cleaned industrial user electricity consumption time series data is transformed into a set of subsequences arranged from high to low frequency, where each subsequence reflects the characteristics of the original sequence in a specific frequency domain. The Fine-to-Coarse reconstruction algorithm is used to calculate the high-frequency, low-frequency, and trend components of the original sequence. The prediction indicators of the original data are standardized, and the standardized results are used to construct a sample dataset using the sliding window technique. The sample dataset is then divided into a training set and a test set according to the proportion. The prediction indicators include four time series data, namely the cleaned industrial user electricity consumption time series data, the high-frequency, low-frequency and trend components obtained by decomposition.

3. The method as described in claim 1, characterized in that, The process of training the model using a dataset specifically involves: The model is trained using the training set in the dataset, including updating parameters using the backpropagation algorithm during model training. The objective function of the model training is the mean squared error. We choose mini-batch stochastic gradient descent and Adam optimizer to minimize the difference between the predicted and the true values, where the batch size depends on the size of the sample dataset. The model is validated using the validation set in the dataset to ensure that no overfitting occurs.

4. A short-term forecasting system for industrial user electricity consumption based on a two-stage attention mechanism, characterized in that, It includes an acquisition module, a data processing module, a short-term forecasting model generation module for industrial user electricity consumption, and a forecasting module, among which... The acquisition module is used to acquire the original time-series data O(t) of industrial users' electricity consumption, t=1, 2, ..., T, where T represents time and is the sequence length; The data processing module is used to process the raw time-series data of the industrial user's electricity consumption to generate a sample dataset. The industrial user electricity consumption short-term prediction model generation module is used to construct an industrial user electricity consumption short-term prediction model with a two-stage attention mechanism, and to train the model using a dataset to generate a trained model. The prediction module is used to make short-term predictions of industrial users' electricity consumption by inputting real data into the trained model. For any time point t in sequence O(t), the prediction model predicts the value at time τ in the future, with t as the baseline, as follows: in, F(·) represents the predicted value at time t+τ; An encoder-decoder structure is used as the model framework to construct a short-term prediction model for industrial user electricity consumption using the two-stage attention mechanism. In the encoder stage, a recurrent neural network is combined with an attention mechanism to extract the spatial correlation between various variables and the target sequence at the same time. In the decoder stage, a recurrent neural network is combined with an attention mechanism, using the encoder's computation results as input data to extract the time-related features of the same variable at different times; In addition to the encoder-decoder structure, in order to further capture the mutation features of the original sequence, a deep neural network structure with two layers of convolution is constructed using industrial user electricity consumption time series data as input variables. The deep neural network structure with two layers of convolution and pooling operations is used. The outputs of the encoder-decoder stage and the two-layer convolution stage are used as input data for the output layer. The final prediction result is calculated by using the concatenated vector of the two-stage prediction results through a fully connected layer, thus completing the model construction.

5. The system as described in claim 4, characterized in that, The data processing module is specifically used for: Data cleaning is performed on the raw time-series data of the industrial users' electricity consumption; By integrating empirical mode decomposition algorithms, the cleaned industrial user electricity consumption time series data is transformed into a set of subsequences arranged from high to low frequency, where each subsequence reflects the characteristics of the original sequence in a specific frequency domain. The Fine-to-Coarse reconstruction algorithm is used to calculate the high-frequency, low-frequency, and trend components of the original sequence. The prediction indicators of the original data are standardized, and the standardized results are used to construct a sample dataset using the sliding window technique. The sample dataset is then divided into a training set and a test set according to the proportion. The prediction indicators include four time series data, namely the cleaned industrial user electricity consumption time series data, the high-frequency, low-frequency and trend components obtained by decomposition.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-3.

Images