A cell power consumption prediction method based on a graph multi-head attention mechanism

Abstract

The application relates to a cell power consumption prediction method based on a graph multi-head attention mechanism, which comprises the following steps: acquiring geographical position data, historical power consumption data and historical external factor data of a cell; pre-processing the historical power consumption data, the historical external factor data and the geographical position data; the historical external factor data comprises air temperature and electricity price; constructing a space-time graph neural network model based on a graph multi-head self-attention mechanism; obtaining a prediction result through the space-time graph neural network model based on the graph multi-head self-attention mechanism; comparing the prediction result with a real value to obtain a loss value; performing reverse error propagation through the loss value to train the model; and predicting future power consumption of the cell through the trained model. The application models historical power consumption data, models spatial relationships among cells and time characteristics of historical data, and fully mines space-time relationships in the cell power consumption prediction problem.

Description

Technical Field

[0001] This invention relates to a method for predicting electricity consumption in residential areas based on a graph multi-head attention mechanism, belonging to the field of power load prediction technology. Background Technology

[0002] Predicting electricity consumption in residential communities helps improve the level of electricity planning and decision-making. Targeting the electricity consumption characteristics of different communities allows power companies to provide tailored services. Using artificial intelligence to predict electricity consumption in communities can achieve higher prediction accuracy than traditional methods. Current electricity prediction methods can be divided into two categories based on the type of input data: (1) using only historical electricity consumption as input data, such as Chinese patent application number CN201710135997.5, which discloses a method for predicting electricity consumption in residential communities based on deep learning technology using a convolutional neural network model. (2) inputting temperature data along with historical electricity consumption. This type of method uses temperature to assist in electricity prediction by establishing a relationship between temperature and electricity consumption, such as Chinese patent application number CN201911031216.3, which discloses a method for predicting the sub-item load of fine-grained electricity consumption behavior of residential users. This prediction method is based on AdaBoost machine learning technology and solves the problems of small sub-item electricity load data and the complexity and variability of influencing factors in residential communities. Chinese patent application number CN201210439494.4 discloses a method and device for predicting electricity consumption in a residential community. This method is based on deep learning technology of deep neural networks. By using historical temperature data and historical electricity consumption data to construct an electricity consumption prediction model, it realizes the prediction of electricity consumption in the community based on temperature.

[0003] The methods described above only predict electricity consumption for a single cell, or only consider the time-series characteristics of cell electricity consumption. Furthermore, while existing technologies consider external factors such as temperature, they do not effectively extract the features of these external factors and integrate them with historical electricity consumption data. Specifically: For individual residential communities, existing methods only predict electricity consumption for that single community, resulting in a limited prediction range and poor capability.

[0004] Existing technologies do not consider spatial characteristics; they do not take into account the spatial characteristics between residential areas. In large-scale residential area electricity consumption forecasting, each residential area can be viewed as a node on a graph, forming a graph structure—this is the spatial characteristic between residential areas. Spatial characteristics play a crucial role in residential area electricity consumption forecasting. For example, two geographically adjacent residential areas may experience electricity consumption increases at different times due to high temperatures.

[0005] The exploration of external factors is too simplistic. Current technology uses temperature to assist in power prediction, treating temperature merely as one input data point, and lacks advanced deep learning techniques for feature mining of external factors. Summary of the Invention

[0006] To overcome the above problems, this invention provides a method for predicting electricity consumption in residential communities based on a graph multi-head attention mechanism. This method models historical electricity consumption data, as well as the spatial relationships between communities and the temporal characteristics of historical data, fully exploring the spatiotemporal relationships in the problem of predicting electricity consumption in residential communities.

[0007] The technical solution of the present invention is as follows: A method for predicting electricity consumption in residential communities based on a graph multi-head attention mechanism includes: Acquire the community's geographical location data, historical electricity consumption data, and historical external factor data; preprocess the historical electricity consumption data, historical external factor data, and geographical location data; the historical external factor data includes temperature and electricity price; Construct a spatiotemporal graph neural network model based on a graph multi-head self-attention mechanism; Using the historical electricity data, historical external factor data, and geographical location data as inputs to the spatiotemporal graph neural network model based on the graph multi-head self-attention mechanism, the prediction result of the spatiotemporal graph neural network model based on the graph multi-head self-attention mechanism is obtained. The predicted result is compared with the actual value to obtain the loss value; The spatiotemporal graph neural network model based on the graph multi-head self-attention mechanism is trained by backpropagation using the loss value. The trained spatiotemporal graph neural network model based on the graph multi-head self-attention mechanism is used to predict the future electricity consumption of the community.

[0008] Furthermore, the historical electricity data, historical external factor data, and geographical location data are preprocessed, including: The historical electricity data and the external factor data are extracted into data frame format, with time information as the row index and cell number as the column index, to obtain the electricity data frame and the external factor data frame; Clean the data frame, removing cells with all data being empty or containing data anomalies, and use linear interpolation to fill in the missing data for cells. Finally, the z-score method is used to standardize the input data so that the data is in the range of -1 to 1; The geographic location data is extracted into a data frame format, with the station number as the row index; Based on the latitude and longitude data of each station, the calculation is performed using the map API. i Other sites j Distance between ; Calculate the adjacency matrix : ; in, The standard deviation is denoted as .

[0009] Furthermore, a spatiotemporal graph neural network model based on a graph multi-head self-attention mechanism is constructed, specifically as follows: External factor feature extraction: The 2D external factor data at time t is transformed into a D-dimensional representation using two fully connected layers. ; Capturing spatial features of external factors using graph convolutional networks: ; in, and These are the learnable parameters in the network. For activation functions; Capture the effects of time propagation using the GRU module: ; in, , , , , , These are the learnable parameters in the network. This is a vector representation of external factors. This indicates a splicing operation. This indicates element-wise multiplication; Historical electricity data vector representation: The historical power consumption data is projected using a two-layer fully connected network to obtain a historical power consumption data vector. .

[0010] The input vector is obtained by fusing the historical electricity data vector with the external factor data vector: Subtracting the historical electricity data vector from the external factor data vector yields the encoder's input vector. : .

[0011] Furthermore, the construction of a spatiotemporal graph neural network model based on a graph multi-head self-attention mechanism also includes spatiotemporal feature representation, specifically: Each cell's node representation is vectorized using node embedding technology; The node vectors are projected into D-dimensional space through a two-layer fully connected network, represented as follows: ; The time features of historical electricity data are vectorized using one-hot encoding. The node vectors are projected into D-dimensional space through a two-layer fully connected network, represented as follows: ; Calculate each time step t Each node on i Dynamic spatiotemporal characteristics representation e i，t : .

[0012] Furthermore, the spatiotemporal graph neural network model based on the graph multi-head self-attention mechanism also includes a prediction module. The prediction module is an encoder-decoder structure, and both the encoder and decoder are composed of L multi-head self-attention modules stacked together. Each multi-head self-attention module consists of a spatial attention mechanism, a temporal attention mechanism, and a gating fusion unit. Furthermore, the spatial attention mechanism is specifically as follows: For each time step t ,node v and i Attention value between The calculation is as follows: ; ; in, and The first l -1 node in an attention block i and nodes v At time step t The hidden vector representation, and They are nodes i and nodes v At time step t The spatiotemporal characteristics are represented, and All are the first k A nonlinear mapping function for each attention head. d The dimension of the vector in a single attention head. Indicates the inner product. This indicates a splicing operation. Attention value The normalized value, The set of nodes; Update nodes using a multi-head attention mechanism. i At time step t Hidden vector representation: ; in, For the first k A nonlinear mapping function for each attention head. K For the number of attention heads, This indicates that after the spatial attention mechanism, the first... l Nodes in an attention block i At time step t Spatial attention features are represented.

[0013] Furthermore, the time attention mechanism is specifically as follows: For each node i Time step t and t Attention value between The calculation is as follows: ; ; in, and The first l -1 node in an attention block i At time step t and time step t The hidden vector representation, and They are nodes i At time step t and time step t The spatiotemporal characteristics are represented, and All are the first k A nonlinear mapping function for each attention head. Attention value The normalized value, The set of time steps. For historical time steps; Update nodes using a multi-head attention mechanism. i At time step t Hidden vector representation:

[0014] in, For the first k A nonlinear mapping function for each attention head. K For the number of attention heads, After passing through the time attention mechanism, the firstl Nodes in an attention block i At time step t Temporal attention features are represented.

[0015] Furthermore, the gating fusion unit specifically comprises: go through L The spatial attention features and temporal attention features after each attention block are respectively represented as follows: and ; Spatial attention features and temporal attention features are fused using a gated fusion unit: ; ; in, It is a non-linear function. Will Mapped to the range (0,1), This is a spatiotemporal attention feature that integrates spatial attention features and temporal attention features.

[0016] Furthermore, between the encoder and decoder, a relational attention mechanism is used to transform the input hidden vector after the decoder into an output hidden vector, which is then used as the input to the decoder. For each node i , No. k Predicting time steps in each attention head t and input time step t Attention value between The calculation is as follows: ; ; in, and All are the first k A nonlinear mapping function for each attention head. P To input the number of time steps, Attention value The normalized value; The nodes are obtained by weighted summation of the input hidden vector after the encoder using attention values. i At time step t Predicted hidden vector : ; in, For the first k A nonlinear mapping function of an attention head.

[0017] Furthermore, using the historical electricity data, historical external factor data, and geographical location data as inputs to the spatiotemporal graph neural network model based on the graph multi-head self-attention mechanism, the prediction results of the spatiotemporal graph neural network model based on the graph multi-head self-attention mechanism are obtained, including: The prediction time step obtained after decoding t The spatiotemporal eigenvector representation, To obtain the external factor vector representation, the spatiotemporal feature vector representation of the prediction time step is added to the external factor vector representation. t The output vector representation : ; Through nonlinear activation function Generate time step t The final output result : .

[0018] The present invention has the following beneficial effects: This invention not only models historical electricity consumption data, but also models the spatial relationships between communities and the temporal characteristics of historical data, fully exploring the spatiotemporal relationships in the problem of community electricity consumption prediction.

[0019] This invention vectorizes and extracts features from external influencing factors such as temperature and electricity price using graph convolutional networks and GRU modules, and then fuses them with the input vector representing historical electricity consumption, thereby improving the accuracy of prediction.

[0020] The model structure of this invention adopts an encoder-decoder structure. Both the encoder and the decoder use multi-head self-attention modules. Through spatial attention mechanism and temporal attention mechanism, the complex spatiotemporal dependencies between input time steps, between nodes, and between input time steps and output time steps are fully explored. Attached Figure Description

[0021] Figure 1 This is a spatiotemporal graph neural network model based on a graph multi-head self-attention mechanism, as described in an embodiment of the present invention. Detailed Implementation

[0022] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0023] A method for predicting electricity consumption in residential communities based on a graph multi-head attention mechanism includes: Acquire the community's geographical location data, historical electricity consumption data, and historical external factor data; preprocess the historical electricity consumption data, historical external factor data, and geographical location data; the historical external factor data includes temperature and electricity price; Construct a spatiotemporal graph neural network model based on a graph multi-head self-attention mechanism; Using the historical electricity data, historical external factor data, and geographical location data as inputs to the spatiotemporal graph neural network model based on the graph multi-head self-attention mechanism, the prediction result of the spatiotemporal graph neural network model based on the graph multi-head self-attention mechanism is obtained. The predicted result is compared with the actual value to obtain the loss value; The spatiotemporal graph neural network model based on the graph multi-head self-attention mechanism is trained by backpropagation using the loss value. The trained spatiotemporal graph neural network model based on the graph multi-head self-attention mechanism is used to predict the future electricity consumption of the community.

[0024] In one embodiment of the present invention, preprocessing of the historical electricity data, historical external factor data, and geographical location data includes: The historical electricity data and the external factor data are extracted into data frame format, with time information as the row index and cell number as the column index, to obtain the electricity data frame and the external factor data frame; Clean the data frame, removing cells with all data being empty or containing data anomalies, and use linear interpolation to fill in the missing data for cells. Finally, the z-score method is used to standardize the input data so that the data is in the range of -1 to 1; The geographic location data is extracted into a data frame format, with the station number as the row index; Based on the latitude and longitude data of each station, the calculation is performed using the map API. i Other sites j Distance between ; Calculate the adjacency matrix : ; in, The standard deviation is denoted as .

[0025] refer to Figure 1 In one embodiment of the present invention, a spatiotemporal graph neural network model based on a graph multi-head self-attention mechanism is constructed, specifically as follows: External factor feature extraction: The 2D external factor data at time t is transformed into a D-dimensional representation using two fully connected layers. ; Capturing spatial features of external factors using graph convolutional networks: ; in, and These are the learnable parameters in the network. For activation functions; Capture the effects of time propagation using the GRU module: ; in, , , , , , These are the learnable parameters in the network. This is a vector representation of external factors. This indicates a splicing operation. This indicates element-wise multiplication; Historical electricity data vector representation: The historical power consumption data is projected using a two-layer fully connected network to obtain a historical power consumption data vector. .

[0026] The input vector is obtained by fusing the historical electricity data vector with the external factor data vector: Subtracting the historical electricity data vector from the external factor data vector yields the encoder's input vector. : .

[0027] In one embodiment of the present invention, the spatiotemporal graph neural network model based on the graph multi-head self-attention mechanism is constructed, and the spatiotemporal feature representation is further included, specifically: The nodes representing each cell are vectorized using the Node2Vec technique; The node vectors are projected into D-dimensional space through a two-layer fully connected network, represented as follows: ; The time features of historical electricity data are vectorized using one-hot encoding. The node vectors are projected into D-dimensional space through a two-layer fully connected network, represented as follows: ; Calculate each time step t Each node on i Dynamic spatiotemporal characteristics representation e i，t : .

[0028] In one embodiment of the present invention, a spatiotemporal graph neural network model based on a graph multi-head self-attention mechanism is constructed, which further includes a prediction module. The prediction module is an encoder-decoder structure, and both the encoder and decoder are composed of L multi-head self-attention modules stacked together. Each multi-head self-attention module consists of a spatial attention mechanism, a temporal attention mechanism, and a gating fusion unit. In one embodiment of the present invention, the spatial attention mechanism is specifically as follows: For each time step t ,node v and i Attention value between The calculation is as follows: ; ; in, and The first l -1 node in an attention block i and nodes v At time step t The hidden vector representation, and They are nodes i and nodes v At time step t The spatiotemporal characteristics are represented, and The first k A nonlinear mapping function for each attention head. d The dimension of the vector in a single attention head. Indicates the inner product. This indicates a splicing operation. Attention value The normalized value, The set of nodes; Update nodes using a multi-head attention mechanism. i At time step t Hidden vector representation: ; in, This is the first k A nonlinear mapping function for each attention head. K For the number of attention heads, This indicates that after the spatial attention mechanism, the first... l Nodes in an attention block i At time step tSpatial attention features are represented.

[0029] , and Both are composed of linear transformation layers and nonlinear activation functions. Since the parameters of the linear transformation layers are learned during training, the nonlinear mapping functions learned in the end are different in numerical terms (model parameters), but are the same in structure.

[0030] In one embodiment of the present invention, the time attention mechanism is specifically as follows: For each node i Time step t and t Attention value between The calculation is as follows: ; ; in, and The first l -1 node in an attention block i At time step t and time step t The hidden vector representation, and They are nodes i At time step t and time step t The spatiotemporal characteristics are represented, and All are the first k A nonlinear mapping function for each attention head. Attention value The normalized value, The set of time steps. For historical time steps; Update nodes using a multi-head attention mechanism. i At time step t Hidden vector representation:

[0031] in, For the first k A nonlinear mapping function for each attention head. K For the number of attention heads, After passing through the time attention mechanism, the first l Nodes in an attention block i At time step t Temporal attention features are represented.

[0032] , and Both are composed of linear transformation layers and nonlinear activation functions. Since the parameters of the linear transformation layers are learned during training, the nonlinear mapping functions learned in the end are different in numerical terms (model parameters), but are the same in structure.

[0033] In one embodiment of the present invention, the gating fusion unit specifically comprises: go through L The spatial attention features and temporal attention features after each attention block are respectively represented as follows: and ; Spatial attention features and temporal attention features are fused using a gated fusion unit: ; ; in, It is a non-linear function. Will Mapped to the range (0,1), This is a spatiotemporal attention feature that integrates spatial attention features and temporal attention features.

[0034] In one embodiment of the present invention, between the encoder and the decoder, the input hidden vector after the decoder is converted into an output hidden vector through a relational attention mechanism, which is then used as the input of the decoder. For each node i , No. k Predicting time steps in each attention head t and input time step t Attention value between The calculation is as follows: ; ; in, and All are the first k A nonlinear mapping function for each attention head. P To input the number of time steps, Attention value The normalized value; The nodes are obtained by weighted summation of the input hidden vector after the encoder using attention values. i At time step t Predicted hidden vector : ; in, For the first k A nonlinear mapping function for attention heads.

[0035] , and Both are composed of linear transformation layers and nonlinear activation functions. Since the parameters of the linear transformation layers are learned during training, the nonlinear mapping functions learned in the end are different in numerical terms (model parameters), but are the same in structure.

[0036] In one embodiment of the present invention, the historical electricity data, historical external factor data, and geographical location data are used as inputs to the spatiotemporal graph neural network model based on the graph multi-head self-attention mechanism to obtain the prediction result of the spatiotemporal graph neural network model based on the graph multi-head self-attention mechanism, including: The prediction time step obtained after decoding t The spatiotemporal eigenvector representation, To obtain the external factor vector representation, the spatiotemporal feature vector representation of the prediction time step is added to the external factor vector representation. t The output vector representation : ; Through nonlinear activation function Generate time step t The final output result : .

[0037] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structure made using the contents of the present invention specification and drawings, or directly or indirectly applied to other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for predicting electricity consumption in residential communities based on a graph multi-head attention mechanism, characterized in that, include: Obtain the community's geographical location data, historical electricity consumption data, and historical external factor data; The historical electricity data, historical external factor data, and geographical location data are preprocessed; the historical external factor data includes temperature and electricity price. Construct a spatiotemporal graph neural network model based on a graph multi-head self-attention mechanism; Using the historical electricity data, historical external factor data, and geographical location data as inputs to the spatiotemporal graph neural network model based on the graph multi-head self-attention mechanism, the prediction result of the spatiotemporal graph neural network model based on the graph multi-head self-attention mechanism is obtained. The predicted result is compared with the actual value to obtain the loss value; The spatiotemporal graph neural network model based on the graph multi-head self-attention mechanism is trained by backpropagation using the loss value. The trained spatiotemporal graph neural network model based on the graph multi-head self-attention mechanism is used to predict the future electricity consumption of the community. Preprocessing of the historical electricity data, historical external factor data, and geographical location data includes: The historical electricity data and the external factor data are extracted into data frame format, with time information as the row index and cell number as the column index, to obtain the electricity data frame and the external factor data frame; Clean the data frame, removing cells with all data being empty or containing data anomalies, and use linear interpolation to fill in the missing data for cells. Finally, the z-score method is used to standardize the input data so that the data is in the range of -1 to 1; The geographic location data is extracted into a data frame format, with the station number as the row index; Based on the latitude and longitude data of each station, the calculation is performed using the map API. i Other sites j Distance between ; Calculate the adjacency matrix : ； in, Standard deviation; A spatiotemporal graph neural network model based on a graph multi-head self-attention mechanism is constructed, specifically as follows: External factor feature extraction: The 2D external factor data at time t is transformed into a D-dimensional representation using two fully connected layers. ; Capturing spatial features of external factors using graph convolutional networks: ； in, and These are the learnable parameters in the network. For activation functions; Capture the effects of time propagation using the GRU module: ； in, , , , , , These are the learnable parameters in the network. This is a vector representation of external factors. This indicates a splicing operation. This indicates element-wise multiplication; Historical electricity data vector representation: The historical power consumption data is projected using a two-layer fully connected network to obtain a historical power consumption data vector. ; The input vector is obtained by fusing the historical electricity data vector with the external factor data vector: Subtracting the historical electricity data vector from the external factor data vector yields the encoder's input vector. : 。 2. The method for predicting electricity consumption in residential areas based on graph multi-head attention mechanism according to claim 1, characterized in that, The construction of a spatiotemporal graph neural network model based on a graph multi-head self-attention mechanism also includes spatiotemporal feature representation, specifically: Each cell's node representation is vectorized using node embedding technology; The node vectors are projected into D-dimensional space through a two-layer fully connected network, represented as follows: ; The time features of historical electricity data are vectorized using one-hot encoding. The node vectors are projected into D-dimensional space through a two-layer fully connected network, represented as follows: ; Calculate each time step t Each node on i Dynamic spatiotemporal characteristics representation e i，t : 。 3. The method for predicting electricity consumption in residential areas based on graph multi-head attention mechanism according to claim 2, characterized in that, The spatiotemporal graph neural network model based on the graph multi-head self-attention mechanism also includes a prediction module. The prediction module is an encoder-decoder structure. Both the encoder and the decoder are composed of L multi-head self-attention modules stacked together. The multi-head self-attention module is composed of a spatial attention mechanism, a temporal attention mechanism, and a gating fusion unit.

4. The method for predicting electricity consumption in residential areas based on graph multi-head attention mechanism according to claim 3, characterized in that, The spatial attention mechanism is specifically as follows: For each time step t ,node v and i Attention value between The calculation is as follows: ； ； in, and The first l -1 node in an attention block i and nodes v At time step t The hidden vector representation, and They are nodes i and nodes v At time step t The spatiotemporal characteristics are represented, and The first k A nonlinear mapping function for each attention head. d The dimension of the vector in a single attention head. Indicates the inner product. This indicates a splicing operation. Attention value The normalized value, The set of nodes; Update nodes using a multi-head attention mechanism. i At time step t Hidden vector representation: ； in, It is a nonlinear mapping function. K For the number of attention heads, This indicates that after the spatial attention mechanism, the first... l Nodes in an attention block i At time step t Spatial attention features are represented.

5. The method for predicting electricity consumption in residential areas based on graph multi-head attention mechanism according to claim 4, characterized in that, The time attention mechanism is specifically as follows: For each node i Time step t and τ Attention value between The calculation is as follows: ； ； in, and The first l -1 node in an attention block i At time step t and time step τ The hidden vector representation, and They are nodes i At time step t and time step τ The spatiotemporal characteristics are represented, and For the first k A nonlinear mapping function for each attention head. Attention value The normalized value, The set of time steps. For historical time steps; Update nodes using a multi-head attention mechanism. i At time step t Hidden vector representation: in, It is a nonlinear mapping function. K For the number of attention heads, After passing through the time attention mechanism, the first l Nodes in an attention block i At time step t Temporal attention features are represented.

6. The method for predicting electricity consumption in residential areas based on graph multi-head attention mechanism according to claim 5, characterized in that, The gated fusion unit is specifically: go through L The spatial attention features and temporal attention features after each attention block are respectively represented as follows: and ; Spatial attention features and temporal attention features are fused using a gated fusion unit: ； ； in, It is a non-linear function. Will Mapped to the range (0,1), This is a spatiotemporal attention feature that integrates spatial attention features and temporal attention features.

7. The method for predicting electricity consumption in residential areas based on graph multi-head attention mechanism according to claim 6, characterized in that, Between the encoder and decoder, a relational attention mechanism is used to transform the input hidden vector after the decoder into an output hidden vector, which is then used as the input to the decoder. For each node i , No. k Predicting time steps in each attention head t and input time step τ Attention value between The calculation is as follows: ； ； in, and All are the first k A nonlinear mapping function for each attention head. P To input the number of time steps, Attention value The normalized value; The nodes are obtained by weighted summation of the input hidden vector after the encoder using attention values. i At time step t Predicted hidden vector : ； in, For the first k A nonlinear mapping function for attention heads.

8. The method for predicting electricity consumption in residential areas based on graph multi-head attention mechanism according to claim 7, characterized in that, Using the historical electricity data, historical external factor data, and geographical location data as inputs to the spatiotemporal graph neural network model based on the graph multi-head self-attention mechanism, the prediction results of the spatiotemporal graph neural network model based on the graph multi-head self-attention mechanism are obtained, including: The prediction time step obtained after decoding t The spatiotemporal eigenvector representation, To obtain the external factor vector representation, the spatiotemporal feature vector representation of the prediction time step is added to the external factor vector representation. t The output vector representation : ； Through nonlinear activation function Generate time step t The final output result : 。

Images