Multi-head attention mechanism-based long and short-term electricity consumption prediction method for charging station
By combining LSTM and multi-head attention mechanism in charging station power prediction, the charging station power consumption data is solved, and the problem of difficult to capture computing complexity and space-time dependence in the existing technology is achieved, and higher prediction accuracy and stability are achieved sex.
Patent Information
- Application Number
- CN202411387053.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-30
- Publication Date
- 2025-01-07
AI Technical Summary
The existing charging station power prediction technology has problems such as computational complexity, uncertainty of user behavior and temporal dependence, resulting in low prediction accuracy.
Using a charging station long short-term power prediction method based on a multi-head attention mechanism, combined with the LSTM model and a multi-head attention mechanism, the long-term and short-term dependencies and spatiotemporal characteristics in the data are captured, and the long-term and short-term dependencies and spatial-temporal characteristics are improved. Prediction accuracy.
Through the use of the multi-head attention mechanism, we can better understand the complex spatio-temporal relationships in the power consumption data of the charging station, improve the accuracy and stability of prediction, reduce information loss, reduce prediction errors, and improve prediction accuracy.
Smart Images
Figure CN119275826A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of energy consumption prediction, and in particular to a method for predicting long-term and short-term electricity consumption of a charging station based on a multi-head attention mechanism. Background Art
[0002] As the number of electric vehicles increases, the demand for charging infrastructure is also growing. The number and frequency of use of public charging stations have increased significantly around the world. The demand for charging electric vehicles increases the load on the power grid, especially during peak hours. If not managed, it can lead to overload of the power system and even power outages. Therefore, predicting the energy consumption of charging stations is essential to balance supply and demand, avoid overloading the grid, and ensure the stability of the power system.
[0003] By predicting the energy consumption of charging stations, it can help power companies and charging station operators to better perform energy scheduling and resource allocation. For example, according to the prediction results, electricity production can be arranged in advance, the layout of charging stations and the distribution of charging piles can be optimized, thereby improving energy efficiency and reducing energy waste. The current charging station electricity consumption prediction technology mainly has the following problems: 1) Computational complexity: For charging station electricity consumption prediction, the computational consumption of traditional statistical models is usually too high and inefficient. 2) Uncertainty of user behavior: The user's charging behavior has great uncertainty and is affected by many factors (such as weather, holidays, oil price fluctuations, etc.), making it difficult to accurately predict charging demand. 3) Spatiotemporal dependence of data: Electric vehicle charging behavior has obvious spatiotemporal dependence, and the charging demand varies greatly at different times and locations. Traditional prediction models often find it difficult to capture these spatiotemporal characteristics at the same time, resulting in low prediction accuracy.
[0004] Therefore, in order to solve the above problems, a charging station long-term and short-term electricity consumption prediction method based on a multi-head attention mechanism is needed, which can process the complex data patterns hidden in the data and give higher weights to the important parts of the data, so as to achieve more accurate predictions. Summary of the invention
[0005] In view of this, the purpose of the present invention is to overcome the defects in the prior art and provide a method for predicting long-term and short-term electricity consumption of charging stations based on a multi-head attention mechanism, which can process the complex data patterns hidden in the data and give higher weights to important parts of the data, thereby achieving more accurate predictions.
[0006] The method for predicting long-term and short-term power consumption of a charging station based on a multi-head attention mechanism of the present invention comprises:
[0007] Selecting a charging station electricity consumption data set; processing the data set to obtain a processed data set;
[0008] Loading the processed data set into the electricity consumption prediction model, performing calculations through the electricity consumption prediction model, and outputting calculation results;
[0009] Calculate the power consumption prediction model loss according to the operation results, train the power consumption prediction model, and obtain a trained prediction model;
[0010] The trained prediction model is used to predict the power consumption of the charging station and obtain the power consumption prediction result.
[0011] Further, the data set is processed, specifically including:
[0012] The electricity consumption sequence E∈R for the past k time steps k , and the mean-variance normalization is performed to obtain
[0013]
[0014] An increasing sequence of time day indices for the past k time steps I∈R k , a day is divided into 48 time points at 30-minute intervals, then each element in I is in the interval [1,48], and I is encoded by one-hot encoding to obtain
[0015]
[0016] The corresponding day of the week index D∈R in the past k time steps k , each index ranges from [1, 6], and D is encoded by one-hot encoding.
[0017] The corresponding binary holiday flag index H∈R k , each element can be 0 or 1, and H is encoded by one-hot encoding.
[0018] Will Merge into a matrix X:
[0019]
[0020] Formula (1) is the normalized historical charging station electricity consumption sequence, where e t represents the electricity consumption data corresponding to time step t; Formulas (2), (3) and (4) are the time day index, weekday index and holiday sign index after one-hot encoding, respectively, where i t represents the time point in a day corresponding to time step t, d t Indicates the day of the week corresponding to time step t, h t Indicates whether the day corresponding to time step t is a holiday;
[0021] The data set formed by the data matrix X is divided into training set, validation set and test set according to the set ratio.
[0022] Furthermore, when loading the data set, a sliding window is used to load, and the input and output lengths of the sliding window are defined, and the number of batches of processed data is defined at the same time. After loading is completed, the input and real data matching the sliding window are obtained. The input data is the processed charging station data set X; the real data is the charging station electricity consumption data e corresponding to the next time step of the last time step in the sliding window. t .
[0023] Furthermore, the electricity consumption prediction model adopts an LSTM model with multiple layers that can be stacked;
[0024] The LSTM model captures long-term and short-term dependencies in the data. The output of the top LSTM layer is fed into the attention layer, which uses a multi-head attention mechanism to simultaneously focus on different parts of the input sequence through multiple different attention heads. The output of the attention layer is fed into a feedforward neural network, which maps the features processed by the attention layer to a single value, that is, predicting the future power consumption of the charging station.
[0025] Among them, the LSTM model maintains the internal memory cell state s throughout its life cycle t And generate the intermediate output h t , the LSTM model structure also defines the input node g t , input gate i t 、Forget gate t and output gate o t , the expressions of these nodes are as follows:
[0026]
[0027] s t =g t ⊙i t +s t-1 ⊙f t
[0028]
[0029] in, They are the weight matrices b corresponding to each gate. f ,b i ,b g ,b o is the bias term; ⊙ represents element-by-element multiplication, σ represents the sigmoid activation function, Represents the tanh activation function.
[0030] Furthermore, the power consumption prediction model is used for calculation, including:
[0031] The terminal LSTM layer is connected to the attention layer, and the feature data processed by LSTM is passed into the attention layer; the attention layer adopts a multi-head attention mechanism, with n representing the number of attention heads. The feature data entering the attention layer is assigned to the query (Q), key (K), and value (V). Q, K, and V are first linearly transformed through different n linear layers, and then the scaled dot product attention is calculated. The calculation results of each attention head are merged, and finally the merged result is linearly transformed through the linear layer and output; the specific calculation process is as follows:
[0032] MultiHead(Q,K,V)=Concat(head1,head2,...,head n )(12)
[0033] head i =Attention(QW i Q ,KW i K ,VW i V )(13)
[0034]
[0035] Formula (12) represents the merging of the calculation results of different attention heads in the multi-head attention mechanism, where head i represents the calculation result of the i-th attention head; the calculation of the attention head is shown in formula (13), where Attention is the scaled dot product attention calculation function, W i Q represents the weight of the i-th linear layer that the query (Q) needs to pass through, W i K represents the weight of the i-th linear layer that the key (K) needs to pass through, W i V represents the weight of the i-th linear layer that the value (V) needs to pass through; Formula (14) represents the calculation process of the scaled dot product attention function, where d k It represents the dimension size of the key (K), and softmax is a normalized exponential function.
[0036] Further, the operation results are output, including:
[0037] The power consumption prediction model only outputs the power consumption of the charging station in the future time step each time. The predicted electricity consumption After processing and merging into the input data, the electricity consumption forecasting model will output Recursively predict the power consumption of a charging station for multiple future time steps.
[0038] Furthermore, the power consumption prediction model loss is calculated based on the operation results, and the power consumption prediction model is trained, specifically including:
[0039] The power consumption prediction model loss includes the power consumption loss of the charging station. loss :
[0040]
[0041] Among them, e t This is the real power consumption data of charging stations. is the electricity consumption data of the charging station predicted by the model;
[0042] After the loss calculation is completed, back propagation is used to update the parameters of the power consumption prediction model to make the loss of the power consumption prediction model smaller.
[0043] Furthermore, in the process of training the electricity consumption prediction model, the trained prediction model is obtained according to the following method:
[0044] After each round of neural network training, a validation set will be used for validation. The validation process will not affect the parameters of the neural network. Then the neural network model with the smallest loss is saved. The model with the smallest loss is the optimal model. If the loss no longer decreases, the optimal model is trained, and the optimal model is the trained prediction model.
[0045] The beneficial effects of the present invention are as follows: the present invention discloses a method for predicting long-term and short-term electricity consumption of charging stations based on a multi-head attention mechanism. By combining LSTM (Long Short-Term Memory Network) with a multi-head attention mechanism, LSTM is good at capturing long-term and short-term dependencies in time series data, while the multi-head attention mechanism can better capture important features and associations in input data, and can more comprehensively understand the complex spatiotemporal relationships in charging station electricity consumption data, thereby improving the accuracy and stability of predictions. Compared with the traditional LSTM model, LSTM may encounter the problem of information loss when processing long time series, that is, early information may be gradually lost during the transmission process. The multi-head attention mechanism can effectively reduce information loss by directly establishing dependencies between input data, improve the effect of long time series prediction, reduce prediction errors, and improve prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:
[0047] Figure 1 This is a schematic diagram of the power consumption prediction model architecture of the present invention;
[0048] Figure 2 It is a schematic diagram of the flow chart of the power consumption prediction method of the present invention;
[0049] Figure 3 This is a schematic diagram of the LSTM structure introduced in the present invention. DETAILED DESCRIPTION
[0050] The present invention is further described below in conjunction with the accompanying drawings, as shown in the drawings:
[0051] This embodiment discloses a method for predicting long-term and short-term power consumption of a charging station based on a multi-head attention mechanism, comprising the following steps:
[0052] Selecting a charging station electricity consumption data set; processing the data set to obtain a processed data set;
[0053] Loading the processed data set into the electricity consumption prediction model, performing calculations through the electricity consumption prediction model, and outputting calculation results;
[0054] Calculate the power consumption prediction model loss according to the operation results, train the power consumption prediction model, and obtain a trained prediction model;
[0055] The trained prediction model is used to predict the power consumption of the charging station and obtain the power consumption prediction result.
[0056] The present invention combines LSTM and multi-head attention mechanism to capture the long-term and short-term dependencies in the charging station electricity consumption data, so that the model can more comprehensively understand the timing characteristics of the input data, process the complex data patterns hidden in the data, and give higher weights to the important parts of the data, ignoring irrelevant or noisy information, thereby improving the accuracy of charging station electricity consumption prediction.
[0057] In this embodiment, a data set needs to be selected first. Specifically, a real-world data set can be selected, such as the real-world data set Electric Nation. Nearly 700 electric car owners of Electric Nation participated in an 18-month trial. After obtaining the data set, in order to enable the model to converge in a shorter time and prevent gradient explosion, the original electricity consumption sequence data needs to be preprocessed, including:
[0058] The electricity consumption sequence E∈R for the past k time steps k , we use the “mean variance normalization” method to get As shown in formula 1;
[0059] An increasing sequence of time day indices for the past k time steps I∈R k For example, if a day is divided into 48 time points at 30-minute intervals, then each element in I is in the interval [1,48]. Performing one-hot encoding on I yields As shown in formula 2;
[0060] The corresponding day of the week index D∈R in the past k time steps k , each index ranges from [1, 6], and D is encoded by one-hot encoding. As shown in formula 3;
[0061] The corresponding binary holiday flag index H∈R k , each element can be 0 or 1, and H is encoded by one-hot encoding. As shown in formula 4;
[0062] You can add or delete these features as needed. Merge into a matrix X, as shown in the following example:
[0063] As shown in formula 5;
[0064]
[0065]
[0066] As shown in formulas 1, 2, 3, 4, and 5, formula 1 is the normalized historical charging station power consumption sequence, where e t represents the electricity consumption data corresponding to time step t. Formula 2, Formula 3 and Formula 4 are the time day index, weekday index and holiday sign index after one-hot encoding respectively, where i t represents the time point in a day corresponding to time step t, d t Indicates the day of the week corresponding to time step t, h t Indicates whether the day corresponding to time step t is a holiday. Merge to get complete data X;
[0067] Finally, the dataset is divided into training set, validation set, and test set according to the proportion.
[0068] In this embodiment, the training set, validation set, and test set obtained in the above steps are loaded into the model. When loading the data set, a sliding window is used for loading. It is necessary to define the input and output lengths of the sliding window, and the number of batches of processed data. After loading is completed, the input and real data matching the sliding window are obtained. The input data is the processed charging station data set X, as shown in Formula 5, and the real data is the charging station electricity consumption data e corresponding to the next time step of the last time step in the sliding window t .
[0069] In this embodiment, the electricity consumption prediction model adopts an LSTM model with multiple layers of stacking; the increase in the number of layers makes it a deep learning network, and of course only one layer of LSTM can be set;
[0070] The LSTM model captures long-term and short-term dependencies in the data. The output of the top LSTM layer is fed into the attention layer, which uses a multi-head attention mechanism to simultaneously focus on different parts of the input sequence through multiple different attention heads. The output of the attention layer is fed into a feedforward neural network, which maps the features processed by the attention layer to a single value, that is, predicting the future power consumption of the charging station.
[0071] Among them, the LSTM model maintains the internal memory cell state s throughout its life cycle t And generate the intermediate output h t , the LSTM model structure also defines the input node g t , input gate i t 、Forget gate t and output gate o t , the expressions of these nodes are as follows:
[0072]
[0073] s t =g t ⊙i t +s t-1 ⊙f t
[0074]
[0075] in, They are the weight matrices b corresponding to each gate. f ,b i ,b g ,b o is the bias term; ⊙ represents element-by-element multiplication, σ represents the sigmoid activation function, Represents the tanh activation function.
[0076] In this embodiment, the calculation is performed through the power consumption prediction model, specifically including:
[0077] The terminal LSTM layer is connected to the attention layer, and the feature data processed by LSTM is passed into the attention layer; the attention layer adopts a multi-head attention mechanism, with n representing the number of attention heads. The feature data entering the attention layer is assigned to the query (Q), key (K), and value (V). Q, K, and V are first linearly transformed through different n linear layers, and then the scaled dot product attention is calculated. The calculation results of each attention head are merged, and finally the merged result is linearly transformed through the linear layer and output; the specific calculation process is as follows:
[0078] MultiHead(Q,K,V)=Concat(head1,head2,...,headn )(12)
[0079] head i =Attention(QW i Q ,KW i K ,VW i V )(13)
[0080]
[0081] Formula (12) represents the merging of the calculation results of different attention heads in the multi-head attention mechanism, where head i represents the calculation result of the i-th attention head; the calculation of the attention head is shown in formula (13), where Attention is the scaled dot product attention calculation function, W i Q represents the weight of the i-th linear layer that the query (Q) needs to pass through, W i K represents the weight of the i-th linear layer that the key (K) needs to pass through, W i V represents the weight of the i-th linear layer that the value (V) needs to pass through; Formula (14) represents the calculation process of the scaled dot product attention function, where d k It represents the dimension size of the key (K), and softmax is a normalized exponential function.
[0082] Among them, when using the electricity consumption prediction model, it is also necessary to set relevant parameters, such as the training iteration cycle epoch, random number seed seed, bias value patience (to speed up the time to obtain the optimal model), network model optimizer optimizer, learning rate lr, weight parameters, etc.
[0083] In this embodiment, outputting the operation result specifically includes:
[0084] The processed data set is loaded into the power consumption prediction model to obtain the output value of the model; the power consumption prediction model only outputs the power consumption of the charging station in the future time step each time The predicted electricity consumption After processing and merging into the input data, the electricity consumption forecasting model will output Thus, the power consumption of the charging station in multiple future time steps can be recursively predicted.
[0085] In this embodiment, the power consumption prediction model loss is calculated according to the operation result, and the power consumption prediction model is trained, which specifically includes:
[0086] The power consumption prediction model loss includes the power consumption loss of the charging station.loss :
[0087]
[0088] Among them, e t This is the real power consumption data of charging stations. is the electricity consumption data of the charging station predicted by the model;
[0089] After the loss calculation is completed, back propagation is used to update the parameters of the power consumption prediction model to make the loss of the power consumption prediction model smaller.
[0090] In this embodiment, during the training process of the electricity consumption prediction model, the trained prediction model is obtained according to the following method:
[0091] After each round of neural network training, a validation set will be used for validation. The validation process will not affect the parameters of the neural network. Then the neural network model with the smallest loss is saved. The model with the smallest loss is the optimal model. If the loss no longer decreases, the optimal model is trained, and the optimal model is the trained prediction model.
[0092] In addition, when testing, the model is tested using the divided test set, the optimal model is loaded, the input data of the test set is assigned to the optimal model, the model's predicted output power consumption is obtained, and the output power consumption is compared with the actual power consumption of the charging station. Performance evaluation is performed using multiple evaluation indicators, and the most commonly used performance indicators are MAE (mean absolute error) and SAE (sum of absolute errors).
[0093] The core goal of the short- and long-term power consumption prediction of charging stations provided by the present invention is to predict the power consumption of charging stations in the future, such as the power consumption in the next hour, day or 10 days, so as to help the control center or central integrator make intelligent decisions on power transmission, distribution and energy allocation to the charging stations, so as to take energy-saving measures.
[0094] The present invention uses stackable multiple layers of LSTM (Long Short-Term Memory Network) to process sequence data and capture long-term and short-term dependencies in the data. Through the memory and forgetting mechanism, LSTM can effectively process and store information with a long time span and avoid the gradient vanishing problem. The output of the top LSTM layer is sent to the attention layer. The attention layer uses a multi-head attention mechanism to simultaneously focus on different parts of the input sequence through a variety of different "attention heads", which allows the model to simultaneously focus on information at multiple different positions, improving the model's ability to capture complex patterns.
[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.
Claims
1. A method for predicting long-term and short-term power consumption of charging stations based on a multi-head attention mechanism, characterized by: include: Select the charging station electricity consumption dataset; Processing the data set to obtain a processed data set; Loading the processed data set into the electricity consumption prediction model, performing calculations through the electricity consumption prediction model, and outputting calculation results; Calculate the power consumption prediction model loss according to the operation results, train the power consumption prediction model, and obtain a trained prediction model; The trained prediction model is used to predict the power consumption of the charging station and obtain the power consumption prediction result.
2. The method for predicting long-term and short-term electricity consumption of charging stations based on a multi-head attention mechanism according to claim 1 is characterized in that: Processing the data set specifically includes: The electricity consumption sequence E∈R for the past k time steps k , and the mean-variance normalization is performed to obtain An increasing sequence of time day indices for the past k time steps I∈R k , a day is divided into 48 time points at 30-minute intervals, then each element in I is in the interval [1,48], and I is one-hot encoded to obtain The corresponding day of the week index D∈R in the past k time steps k , each index ranges from [1, 6], and D is encoded by one-hot encoding. The corresponding binary holiday flag index H∈R k , each element can be 0 or 1, and H is encoded by one-hot encoding. Will Merge into a matrix X: Formula (1) is the normalized historical charging station electricity consumption sequence, where e t represents the electricity consumption data corresponding to time step t; Formulas (2), (3) and (4) are the time day index, weekday index and holiday sign index after one-hot encoding, respectively, where i t represents the time point in a day corresponding to time step t, d t Indicates the day of the week corresponding to time step t, h t Indicates whether the day corresponding to time step t is a holiday; The data set formed by the data matrix X is divided into training set, validation set and test set according to the set ratio.
3. The method for predicting long-term and short-term electricity consumption of charging stations based on a multi-head attention mechanism according to claim 2 is characterized by: When loading the data set, use sliding window loading, define the input and output lengths of the sliding window, and define the number of batches of processed data; after loading is completed, obtain the input and real data that match the sliding window, and the input data is the processed charging station data set X; The real data is the charging station electricity consumption data corresponding to the next time step of the last time step in the sliding window. t .
4. The method for predicting long-term and short-term electricity consumption of charging stations based on a multi-head attention mechanism according to claim 1 is characterized in that: The electricity consumption prediction model adopts an LSTM model that can stack multiple layers; The LSTM model captures long-term and short-term dependencies in the data; The output of the top LSTM layer is fed into the attention layer, which uses a multi-head attention mechanism to simultaneously focus on different parts of the input sequence through multiple different attention heads; The output of the attention layer is fed into a feedforward neural network, which maps the features processed by the attention layer to a single value, i.e., predicting the future power consumption of the charging station. Among them, the LSTM model maintains the internal memory cell state s throughout its life cycle t And generate the intermediate output h t , the LSTM model structure also defines the input node g t , input gate i t 、Forget gate t and output gate o t , the expressions of these nodes are as follows: s t =g t ☉i t +s t-1 ☉f t in, They are the weight matrices b corresponding to each gate. f ,b i ,b g ,b o is the bias term; ⊙ represents element-by-element multiplication, σ represents the sigmoid activation function, Represents the tanh activation function.
5. The method for predicting long-term and short-term electricity consumption of charging stations based on a multi-head attention mechanism according to claim 4 is characterized in that: The calculation is performed through the electricity consumption prediction model, including: The terminal LSTM layer is connected to the attention layer, and the feature data processed by LSTM is passed into the attention layer; the attention layer adopts a multi-head attention mechanism, with n representing the number of attention heads. The feature data entering the attention layer is assigned to the query (Q), key (K), and value (V). Q, K, and V are first linearly transformed through different n linear layers, and then the scaled dot product attention is calculated. The calculation results of each attention head are merged, and finally the merged result is linearly transformed through the linear layer and output; the specific calculation process is as follows: MultiHead(Q,K,V)=Concat(head1,head2,...,head n ) (12) head i =Attention(QW i Q ,KW i K ,VW i V ) (13) Formula (12) represents the merging of the calculation results of different attention heads in the multi-head attention mechanism, where head i represents the calculation result of the i-th attention head; the calculation of the attention head is shown in formula (13), where Attention is the scaled dot product attention calculation function, W i Q represents the weight of the i-th linear layer that the query (Q) needs to pass through, W i K represents the weight of the i-th linear layer that the key (K) needs to pass through, W i V represents the weight of the i-th linear layer that the value (V) needs to pass through; Formula (14) represents the calculation process of the scaled dot product attention function, where d k It represents the dimension size of the key (K), and softmax is a normalized exponential function.
6. The method for predicting long-term and short-term electricity consumption of charging stations based on a multi-head attention mechanism according to claim 1 is characterized by: Output operation results, including: The power consumption prediction model only outputs the power consumption of the charging station in the future time step each time. The predicted electricity consumption After processing and merging into the input data, the electricity consumption forecasting model will output Recursively predict the power consumption of a charging station for multiple future time steps.
7. The method for predicting long-term and short-term electricity consumption of charging stations based on a multi-head attention mechanism according to claim 1 is characterized in that: The power consumption prediction model loss is calculated based on the operation results, and the power consumption prediction model is trained, including: The power consumption prediction model loss includes the power consumption loss of the charging station. loss : Among them, e t This is the real power consumption data of charging stations. is the electricity consumption data of the charging station predicted by the model; After the loss calculation is completed, back propagation is used to update the parameters of the power consumption prediction model to make the loss of the power consumption prediction model smaller.
8. The method for predicting long-term and short-term electricity consumption of charging stations based on a multi-head attention mechanism according to claim 1 is characterized by: During the training process of the electricity consumption prediction model, the trained prediction model is obtained according to the following method: After each round of neural network training, a validation set will be used for validation. The validation process will not affect the parameters of the neural network. Then the neural network model with the smallest loss is saved. The model with the smallest loss is the optimal model. If the loss no longer decreases, the optimal model is trained, and the optimal model is the trained prediction model.