Electrical load time sequence change point prediction method based on multistage attention network

Abstract

The invention provides an electrical load time sequence change point prediction method based on a multistage attention network, and the method comprises the steps: S1, collecting a plurality of pieces of historical electricity utilization information, extracting a plurality of feature values from the historical electricity utilization information, and generating a training vector corresponding to the historical electricity utilization information based on the plurality of feature values; s2, training an electrical load prediction model through the training vector, wherein the electrical load prediction model is used for estimating a future electrical load; s3, collecting real-time power consumption information, inputting the real-time power consumption information into the trained power consumption load prediction model, and judging whether power consumption is abnormal or not based on a prediction result of the power consumption load prediction model; if it is judged that the electricity consumption is abnormal, the prediction result serves as electricity consumption abnormal data; s4, judging whether power utilization change occurs or not based on the power utilization abnormal data; and if it is judged that the power utilization change occurs, determining a power utilization load change point based on the power utilization abnormal data. The method has the advantages that the change point of the electrical load can be accurately predicted, and important support is provided for optimization management of an electric power system.

Description

Technical Field

[0001] The present invention relates to the field of power load forecasting, and in particular to a method for predicting change points of power load time series based on a multi-level attention network. Background Art

[0002] With the rapid development of smart grids and renewable energy, the prediction and management of power load change points have become increasingly important. Accurate prediction of power load not only helps the stable operation of the power system, but also improves energy efficiency and reduces resource waste. However, due to the nonlinearity and time-varying nature of power load data, traditional prediction methods often have difficulty effectively capturing complex patterns and change points in load data.

[0003] In the power load time series data, power load changes refer to sudden changes in load patterns or shifts in trends, which may be caused by a variety of factors, such as seasonal changes, holidays, fluctuations in economic activities, and changes in user behavior. Therefore, accurately detecting change points in power load time series data is of great practical significance for power dispatching, demand response, and equipment maintenance.

[0004] Traditional power load change point prediction methods are mostly based on linear regression, time series analysis, and statistical models, but these methods have limited applicability when faced with complex time series data. For example, they cannot fully capture the nonlinear characteristics and potential long-term dependencies in the data. In particular, when sudden load changes occur, the prediction accuracy is insufficient. Summary of the invention

[0005] The purpose of the present invention is to provide a method for predicting the change points of power load time series based on a multi-level attention network, which can accurately predict the change points of power load and provide important support for the optimal management of the power system.

[0006] In order to achieve the above object, the present invention provides a method for predicting the change point of power load time series based on a multi-level attention network, comprising the steps of:

[0007] S1. Collect multiple historical electricity usage information, extract multiple feature values ​​from the historical electricity usage information, and generate a training vector corresponding to the historical electricity usage information based on the multiple feature values;

[0008] S2. Training a power load prediction model using the training vector, where the power load prediction model is used to predict future power load;

[0009] S3, collecting real-time power consumption information and inputting it into the trained power load prediction model, judging whether power consumption abnormality occurs based on the prediction result of the power load prediction model; if it is judged that power consumption abnormality occurs, using the prediction result as power consumption abnormality data;

[0010] S4. Determine whether a change in electricity consumption occurs based on the abnormal electricity consumption data; if a change in electricity consumption occurs, confirm a point of change in electricity load based on the abnormal electricity consumption data.

[0011] Optionally, the plurality of historical electricity usage information includes the first to Nth historical electricity usage information; let x k represents the training vector generated based on the k-th historical electricity consumption information, represents the i-th eigenvalue extracted from the k-th historical electricity consumption information, i∈[1,d], and d represents the number of eigenvalues ​​in the training vector.

[0012] Optionally, the characteristic values ​​include power load, temperature, and humidity.

[0013] Optionally, the power load prediction model includes: an input attention mechanism module, a multi-sequence fusion module, an encoder module, a time attention mechanism module and a decoder module connected in sequence; the encoder module is an LSTM module, and the decoder module is a Bi-LSTM module;

[0014] The training vector x 1 ~x T As the input of the input attention mechanism module from time step 1 to time step T;

[0015] The training vector x is calculated by cooperating with the input attention mechanism module and the encoder module. t The attention weights of each feature value in t∈[1,T];

[0016] The multi-sequence fusion module is based on the training vector x at time step t t The attention weights of each feature value in update the training vector x t is the vector x ′ t ; vector x ′ t as the input of the encoder module at time step t;

[0017] The temporal attention mechanism module cooperates with the encoder module to calculate h i Attention weights at time step t h i is the hidden state of the LSTM module at time step i; according to h1~h T The attention weights of each at time step t for h1~h T Calculate the weighted sum to get the context vector c t ;h1~h T represents the hidden state of the encoder module from time step 1 to time step t;

[0018] The decoder module is based on the context vector c t , the hidden state of the decoder module at time step t, predicts the power load at time step t.

[0019] Optionally, the input attention module is based on the hidden state h of the encoder module at time step t-1 t-1 and the cell state s t-1 , is the training vector x t The kth eigenvalue in Calculate the corresponding weight

[0020]

[0021] In formula (1), exp(·) represents the natural exponential function;

[0022]

[0023] In formula (2), ω e The transpose of e is the weight vector, tanh(·) represents the activation function, W e is the weight matrix, [h t-1 ;s t-1 ] means to hide the state h t-1 and the cell state s t-1 Concatenate into a new vector, U e is the training vector x k The weight matrix, b e is the bias term.

[0024] Optionally, the vector x updated by the multi-sequence fusion module ′ t for:

[0025]

[0026] The optional temporal attention mechanism module is based on the hidden state h of the encoder module at time step t-1. t-1 and the cell state s of the decoder module at time step t-1 ′ t-1 , calculate h i Attention weights at time step t h i is the hidden state of the encoder module at time step i, i∈[1,T];

[0027]

[0028]

[0029] In formula (5), ω v The transpose of v is the weight vector, W v is the weight matrix, [d t-1 ;s ′ t-1 ] means to put d t-1 and ′ t-1 Concatenate into a new vector, U v is the weight matrix, b v is the bias vector;

[0030] The context vector is calculated by the temporal attention mechanism module

[0031] Optionally, let d t-1 d t d t+1 They represent the hidden states of the Bi-LSTM module at time steps t-1, t, and t+1 respectively. The Bi-LSTM module is based on d t-1 and d t+1 Updated t ;

[0032] After the update,

[0033] In formula (7), is a temporary variable;

[0034] The Bi-LSTM module is based on the context vector c t and the updated hidden state d t , generate the forecast result of the power load at time step t

[0035]

[0036] In formula (8), ω y The transpose of y is the weight vector, [d t ;c t ] means to set d t and c t Concatenate into a new vector, W y is the weight matrix, b ω 、b y is the bias term.

[0037] Optionally, in step S3, if the difference between the predicted power load and the power load threshold exceeds a set difference threshold, the predicted power load is used as abnormal power load data.

[0038] Optionally, step S4 includes:

[0039] The time point when the power consumption abnormality is first determined is taken as the focus point; if the number of consecutive predicted power consumption changes since the focus point reaches a preset threshold, it is determined that a power consumption change has occurred, and the focus point is taken as the power load change point.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] The power load time series change point prediction method based on the multi-level attention network of the present invention collects historical power consumption information and realizes feature value extraction to train the power load prediction model, thereby accurately and timely detecting the power load change point. The power load prediction model of the present invention greatly improves the performance of the power load prediction model in long time series data by introducing input attention mechanism and time attention mechanism. Through the two-layer attention mechanism, it is possible to pay attention to different features of the input data at different levels, such as the weights of various feature values ​​in the training vector and the weights of each time step in the encoder module, thereby effectively capturing the key patterns and change points in the time series. The present invention not only enhances the expressive power of the power load prediction model, but also improves the adaptability to data complexity, so that the accuracy of the power load prediction results and change point detection can be significantly improved.

[0042] The decoder of the present invention adopts a Bi-LSTM module, and observes the backward (past) and forward (future) dynamic characteristics at each time step, thereby reducing error accumulation in long time series detection and improving the accuracy of power load prediction detection.

[0043] The present invention improves the judgment efficiency and interpretability by identifying the time point of abnormal power consumption and automatically judging the change point of power load. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for the description. Obviously, the drawings described below are an embodiment of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work:

[0045] Figure 1 It is a flow chart of the method for predicting the change point of power load time series based on the multi-level attention network of the present invention;

[0046] Figure 2 Schematic diagram of the structure of the power load prediction model in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0049] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0050] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0051] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0052] In addition, in the description of the present application, the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0053] This embodiment provides a method for predicting the change point of power load time series based on a multi-level attention network. Figure 1 As shown, it includes the steps:

[0054] S1. Collect multiple historical electricity consumption information, extract multiple feature values ​​from the historical electricity consumption information, and generate training vectors corresponding to the historical electricity consumption information based on the multiple electricity consumption feature values;

[0055] In this embodiment, the plurality of historical electricity usage information includes the first to the Nth historical electricity usage information; let xk represents the training vector generated based on the k-th historical electricity consumption information, represents the i-th eigenvalue extracted from the k-th historical electricity consumption information, i∈[1,d], and d represents the number of eigenvalues ​​in the training vector.

[0056] The characteristic values ​​in this embodiment include power load, temperature, and humidity.

[0057] S2. training a power load prediction model by using the power training vector, where the power load prediction model is used to predict future power load;

[0058] Figure 2 The schematic diagram of the structure of the power load prediction model in this embodiment is shown in FIG. Figure 2 As shown, the power load prediction model includes: an input attention mechanism module 10, a multi-sequence fusion module 20, an encoder module 30, a time attention mechanism module 40 and a decoder module 50 connected in sequence. In this embodiment, the encoder module 30 is an LSTM (Directional Long Short-Term Memory) module, and the decoder module 50 is a Bi-LSTM (Bi-directional Long Short-Term Memory) module.

[0059] like Figure 2 As shown, the training vector x 1 ~x T As the input of the attention mechanism module 10 from time step 1 to time step T.

[0060] In this embodiment, the training vector x is calculated by inputting the attention mechanism module 10 and cooperating with the encoder module 30. t The attention weights of each feature value in; t∈[1,T].

[0061] like Figure 2 As shown, the input attention module is based on the hidden state h of the encoder module 30 at time step t-1 t-1 and the cell state s t-1 , is the training vector x t The kth eigenvalue in Calculate the corresponding weight Use the electric load forecasting model to focus on the features that are most relevant for the forecast.

[0062]

[0063] In formula (1), exp(·) represents the natural exponential function;

[0064]

[0065] In formula (2), tanh(·) represents the activation function, ω e The transpose of e is the weight vector (converting the output of tanh(·) into an attention score ω e We learn how to extract the most relevant information from the output of tanh(·), W e is the weight matrix, [h t-1 ;s t-1 ] means to hide the state h t-1 and the cell state s t-1 concatenate into a new vector (vertically stacked and merged into a longer vector whose dimension is usually equal to the number of eigenvalues ​​in the training vector), U e is the training vector x k The weight matrix, b e is the bias term.

[0066] Unit Status t-1 Refers to the cell state of the LSTM module in the encoding phase. It is the internal state of the encoder LSTM at time step t-1, which stores the long-range dependency information of the time series data until time step t-1. Figure 2 This state is reflected in the encoder module 30 to help calculate the attention weights for each feature value. The cell state is used together with the hidden state to calculate the input attention weights to capture the most important feature categories. For example, if the temperature has a greater impact on the power load, the temperature feature may receive a higher weight and thus dominate the decision at the current time step.

[0067] The multi-sequence fusion module 20 is based on the training vector x at time step t t The attention weights of each feature value in update the training vector x t is the vector x ′ t ; vector x ′ t As input to the encoder module 30 at time step t.

[0068] In this embodiment, Figure 2 As shown, the vector x updated by the multi-sequence fusion module 20 is ′ t for:

[0069]

[0070] In this embodiment, Figure 2As shown, the temporal attention mechanism module 40 cooperates with the encoder module 30 to calculate h i Attention weights at time step t h i is the hidden state of the LSTM module at time step i. According to h1~h T The attention weights of each at time step t for h1~h T Calculate the weighted sum to get the context vector c t Among them, h1~h T represents the hidden state of the encoder module 30 from time step 1 to time step t.

[0071] The temporal attention mechanism module 40 is based on the hidden state h of the encoder module 30 at time step t-1 t-1 and the cell state s of decoder module 50 at time step t-1 ′ t-1 , calculate h i Attention weights at time step t h i is the hidden state of the encoder module 30 at time step i, i∈[1,T];

[0072]

[0073]

[0074] In formula (5), ω v The transpose of v is the weight vector, ω v Convert the output of tanh(·) to an attention score ω v We learned how to extract the most relevant information from the output of tanh(·). v is the weight matrix, [d t-1 ;s ′ t-1 ] means to put d t-1 and ′ t-1 Concatenate into a new vector. v is the weight matrix, b v is the bias vector.

[0075] The context vector is calculated by the temporal attention mechanism module 40

[0076] T in formula (6) represents the total number of time steps processed in the encoder module 30, which determines the context vector c t How many encoder hidden states contribute to .

[0077] Unit Status ′ t-1 is the hidden information stored by the Bi-LSTM module at time step t-1. Unit state s ′ t-1 It helps the power load forecasting model calculate the temporal attention weight of the hidden state at the current time step t. Through the temporal attention mechanism, the power load forecasting model can dynamically adjust the degree of attention to past time steps, especially those time points that have a greater impact on the current forecast.

[0078] Unit Status ′ t-1 It is used in the temporal attention mechanism to calculate the attention weight of the current time step, so as to better understand the temporal dependencies in the time series. Figure 2 This can be reflected by the hidden state of the decoder module 50, which runs through the entire decoding stage, ensuring that the model takes historical information into account when making predictions.

[0079] The decoder module 50 is based on the context vector c t , the hidden state of the decoder module 50 at time step t, predicts the power load at time step t.

[0080] like Figure 2 As shown, in this embodiment, let d t-1 d t d t+1 Respectively represent the hidden states of the Bi-LSTM module at time steps t-1, t, and t+1. The Bi-LSTM module is based on d t-1 and d t+1 Updated t ; Through the Bi-LSTM module, both backward (past) and forward (future) dynamic features are observed at each time step, preventing error accumulation and improving accuracy.

[0081] After the update,

[0082] In formula (7), is a temporary variable used for internal calculations of the Bi-LSTM module to update the hidden state d of the decoder module 50 at time step t t .

[0083] The Bi-LSTM module is based on the context vector c t and the updated hidden state d t , generate the forecast result of the power load at time step t

[0084]

[0085] In formula (8), ω y The transpose of y is the weight vector, [d t ;c t ] means to set d t and c t Concatenate into a new vector, W y is the weight matrix, b ω 、b y is the bias term.

[0086] In the process of training the power load prediction model, the present invention uses the actual power load y at time step t as the basis. t And the prediction results The difference between and determines the accuracy of the prediction result. According to the anomaly score e t The prediction performance of the model is evaluated by using three indicators: root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE) (this is the existing technology and will not be described here).

[0087] S3. Collect real-time electricity consumption information and input it into a trained electricity load prediction model, and determine whether abnormal electricity consumption occurs based on the prediction results of the electricity load prediction model; if abnormal electricity consumption occurs, use the prediction results as abnormal electricity consumption data.

[0088] In step S3, if the difference between the predicted power load and the power load threshold exceeds the set difference threshold, the predicted power load is used as abnormal power load data.

[0089] In this embodiment, through experiments, the power load threshold is set according to the normal distribution results of the predicted power load. Since the anomaly score conforms to the Gaussian distribution, the power load threshold (also called the anomaly threshold) can be set according to the Laida criterion (3σ criterion). In a normal distribution, about 99.7% of the data will fall within 3 times the standard deviation of the mean. Therefore, it is statistically reasonable to regard values ​​outside this range as abnormal. When setting the power load threshold, the model will mark the time point when the anomaly score is greater than the power load threshold as an abnormal time point for detecting change points. The specific steps are: calculate the proportion of normal points in the data set; determine the normal range of most points in the data set according to the 3σ criterion; compare the performance through experiments, and select the best performance power load threshold with indicators such as F1 score.

[0090] S4. Determine whether a change in electricity consumption occurs based on the abnormal electricity consumption data; if a change in electricity consumption occurs, confirm the point of change in electricity load based on the abnormal electricity consumption data.

[0091] Step S4 includes:

[0092] The time point when the abnormal power consumption is first judged to occur is taken as the focus point; if the number of consecutive predicted power consumption changes since the power consumption focus point reaches the preset number threshold, it is judged that power consumption changes occur, and the power consumption focus point is taken as the power load change point.

[0093] The power load time series change point prediction method based on the multi-level attention network of the present invention greatly improves the performance of the power load prediction model in long time series data by introducing input attention mechanism and time attention mechanism. Through the two-layer attention mechanism, it is possible to pay attention to different features of the input data at different levels, thereby effectively capturing the key patterns and change points in the time series. The present invention not only enhances the expressive power of the power load prediction model, but also improves the adaptability to data complexity, so that the accuracy of the power load prediction results and change point detection can be significantly improved.

[0094] The decoder of the present invention adopts a Bi-LSTM module, and observes the backward (past) and forward (future) dynamic characteristics at each time step, thereby reducing error accumulation in long time series detection and improving the accuracy of power load prediction detection.

[0095] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0096] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for predicting change points in power load time series based on a multi-level attention network, characterized in that: Contains steps: S1. Collect multiple historical electricity usage information, extract multiple feature values ​​from the historical electricity usage information, and generate a training vector corresponding to the historical electricity usage information based on the multiple feature values; S2. Training a power load prediction model using the training vector, where the power load prediction model is used to predict future power load; S3, collecting real-time power consumption information and inputting it into the trained power load prediction model, and judging whether power consumption abnormality occurs based on the prediction result of the power load prediction model; If it is determined that abnormal power consumption occurs, the prediction result is used as abnormal power consumption data; S4. judging whether a change in electricity consumption occurs based on the abnormal electricity consumption data; If it is determined that a change in electricity consumption occurs, the electricity load change point is confirmed based on the abnormal electricity consumption data.

2. The method for predicting change points of power load time series based on a multi-level attention network as claimed in claim 1, characterized in that: The plurality of historical electricity usage information includes the first to the Nth historical electricity usage information; let x k represents the training vector generated based on the k-th historical electricity consumption information, represents the i-th eigenvalue extracted from the k-th historical electricity consumption information, i∈[1,d], and d represents the number of eigenvalues ​​in the training vector.

3. The method for predicting change points of power load time series based on a multi-level attention network as claimed in claim 1, characterized in that: The characteristic values ​​include power load, temperature, and humidity.

4. The method for predicting change points of power load time series based on a multi-level attention network as claimed in claim 1, characterized in that: The power load prediction model comprises: an input attention mechanism module, a multi-sequence fusion module, an encoder module, a time attention mechanism module and a decoder module connected in sequence; the encoder module is an LSTM module, and the decoder module is a Bi-LSTM module; The training vector x 1 ~x T As the input of the input attention mechanism module from time step 1 to time step T; The training vector x is calculated by cooperating with the input attention mechanism module and the encoder module. t The attention weights of each feature value in ; t∈[1,T]; The multi-sequence fusion module is based on the training vector x at time step t t The attention weights of each feature value in update the training vector x t is the vector x ′ t ; vector x ′ t as the input of the encoder module at time step t; The temporal attention mechanism module cooperates with the encoder module to calculate h i Attention weights at time step t h i is the hidden state of the LSTM module at time step i; according to h1~h T The attention weights of each at time step t for h1~h T Calculate the weighted sum to get the context vector c t ;h1~h T represents the hidden state of the encoder module from time step 1 to time step t; The decoder module is based on the context vector c t , the hidden state of the decoder module at time step t, predicts the power load at time step t.

5. The method for predicting change points of power load time series based on a multi-level attention network as claimed in claim 4, characterized in that: The input attention module is based on the hidden state h of the encoder module at time step t-1. t-1 and the cell state s t-1 , is the training vector x t The kth eigenvalue in Calculate the corresponding weight In formula (1), exp(·) represents the natural exponential function; In formula (2), ω e The transpose of e is the weight vector, tanh(·) represents the activation function, W e is the weight matrix, [h t-1 ;s t-1 ] means to hide the state h t-1 and the cell state s t-1 Concatenate into a new vector, U e is the training vector x k The weight matrix, b e is the bias term.

6. The method for predicting change points of power load time series based on a multi-level attention network as claimed in claim 5, characterized in that: The vector x updated by the multi-sequence fusion module ′ t for:

7. The method for predicting change points of power load time series based on a multi-level attention network as claimed in claim 6, characterized in that: The temporal attention mechanism module is based on the hidden state h of the encoder module at time step t-1. t-1 The cell state s of the decoder module at time step t-1 is ′ t-1 , calculate h i Attention weights at time step t h u is the hidden state of the encoder module at time step i, i∈[1,T]; In formula (5), ω v The transpose of v is the weight vector, W v is the weight matrix, [d t-1 ;s ′ t-1 ] means to put d t-1 and ′ t-1 Concatenate into a new vector, U v is the weight matrix, b v is the bias vector; The context vector is calculated by the temporal attention mechanism module 8. The method for predicting change points of power load time series based on a multi-level attention network as claimed in claim 7, characterized in that: Let d t-1 d t d t+1 They represent the hidden states of the Bi-LSTM module at time steps t-1, t, and t+1 respectively. The Bi-LSTM module is based on d t-1 and d t+1 Updated t ; After the update, In formula (7), is a temporary variable; The Bi-LSTM module is based on the context vector c t and the updated hidden state d t , generate the forecast result of the power load at time step t In formula (8), ω y The transpose of y is the weight vector, [d t ;c t ] means to set d t and c t Concatenate into a new vector, W y is the weight matrix, b ω 、b y is the bias term.

9. The method for predicting change points of power load time series based on a multi-level attention network as claimed in claim 8, characterized in that: In step S3, if the difference between the predicted power load and the power load threshold exceeds the set difference threshold, the predicted power load is used as abnormal power load data.

10. The method for predicting change points of power load time series based on a multi-level attention network as claimed in claim 9, characterized in that: Step S4 includes: The time point when the power consumption abnormality is first determined is taken as the focus point; if the number of consecutive predicted power consumption changes since the focus point reaches a preset threshold, it is determined that a power consumption change has occurred, and the focus point is taken as the power load change point.

Images