Cloud-edge collaborative multi-residential area load forecasting method based on federated learning

Abstract

The present invention relates to a short-term load forecasting method for a power system, and aims to provide a cloud-edge collaborative multi-residential area load forecasting method based on federated learning. The present invention adopts cloud-edge collaborative technology, and realizes the transmission of load forecasting model parameters between cloud servers and computing clients distributed in various residential areas through homomorphic encryption technology, so as to reduce communication costs and protect the privacy of residents' data; by adopting a unified global model rather than different local models to forecast the load of residential areas, the problem of poor load forecasting results in residential areas due to small amount of data or insufficient computing power is avoided. The present invention uses meteorological data with high correlation with load as model features to participate in model training, and reasonably uses the difference in data volume between different computing clients to improve training accuracy and efficiency. The user data of the residential area is kept locally, and only the model parameters are transmitted, which significantly reduces the communication time and the bandwidth required for communication, and improves the efficiency and economy of the model.

Description

Technical Field

[0001] The invention relates to a short-term load forecasting method for an electric power system, and belongs to a multi-feature residential area load forecasting technology based on federated learning. Background Art

[0002] With the access of various distributed energy sources to the power system, the original one-way power flow has been transformed into a two-way power flow, and the monitoring difficulty has increased. At the same time, with the development of my country's economy, the scale of electricity consumption in residential areas has shown a significant upward trend. Therefore, effective power load forecasting in residential areas is conducive to maintaining power stability in residential areas and improving the power quality in residential areas.

[0003] Electric power load forecasting is divided into medium- and long-term load forecasting, short-term load forecasting and ultra-short-term load forecasting. Currently, researchers are most concerned about short-term load forecasting, which is often carried out in hours, taking into account human activities and meteorological factors, and predicting the load of the research object. Commonly used methods include time series prediction, neural network prediction, etc. Among the neural network prediction methods, the commonly used models are recurrent neural networks with memory and long short-term memory networks.

[0004] In recent years, the power system has increasingly higher requirements for the accuracy of load forecasting, including feature accuracy and result accuracy. There have been many public documents in the industry discussing this, for example: "HGWOACOA-LSTMN method for short-term load forecasting of commercial and residential distribution networks" proposes a distribution network short-term power load forecasting method based on the gray wolf coyote hybrid optimization algorithm (and long short-term memory network, the forecast object is the commercial and residential distribution network load. "Household short-term power load forecasting based on state frequency memory network" further conducts short-term power load forecasting based on households, and proposes a household short-term power load forecasting model based on state frequency memory network. "Short-term Load Forecasting Based on VMD-PSO-Multiple Kernel Extreme Learning Machine Method") proposes a multi-core extreme learning machine model based on variational mode decomposition and particle swarm optimization. "Attention-GRU Short-term Load Forecasting Method Based on Sparrow Search Optimization" first applies the Attention mechanism to assign weights to the input sequence; then inputs the GRU combined network to learn the internal features and output the predicted time load value; finally, the sparrow search algorithm is used to perform combined optimization of the network hyperparameters, and the optimal network structure hyperparameters are obtained with the minimum validation set loss as the objective function.

[0005] However, in all the above studies, the research object is a unit (family, region, etc.); the calculation requirements for predicting the load of a single object are high, the amount of data is large, and it is difficult to perform large-scale migration prediction. In addition, existing load predictions often directly add climate factor characteristics to the prediction model, which has an adverse effect on the efficiency of model training. Summary of the invention

[0006] The technical problem to be solved by the present invention is to overcome the deficiencies in the prior art and provide a cloud-edge collaborative multi-residential area load forecasting method based on federated learning.

[0007] To solve the technical problem, the solution of the present invention is:

[0008] A method for cloud-edge collaborative multi-residential area load forecasting based on federated learning is provided, comprising the following steps:

[0009] (1) Using computing clients located in each residential area to obtain historical electricity load and historical meteorological data of the residential area;

[0010] (2) Build a long short-term memory network model based on the attention mechanism (attention-LSTM) and deploy it in the cloud server and each computing client respectively;

[0011] (3) The cloud server randomly selects several residential areas, and each computing client uses the power load and meteorological data of the residential area to locally train the model, and then encrypts the trained model parameters using the homomorphic encryption algorithm and transmits them to the cloud server;

[0012] (4) The cloud server receives and decrypts the model parameters from the computing client, aggregates and updates the global model, and sends the updated model parameters to all computing clients;

[0013] (5) After repeating steps (3)-(4) and reaching the preset number of training rounds, the cloud server uses the final global model to predict the electricity load of each residential area.

[0014] As a preferred embodiment of the present invention, in step (1), the Spearman correlation coefficient is used to calculate and analyze the correlation between historical meteorological data and historical electricity load, to obtain a correlation grade, and strongly correlated meteorological data that meets preset conditions is used as input to the model.

[0015] As a preferred embodiment of the present invention, the calculation formula of the Spearman correlation coefficient is:

[0016]

[0017] In the formula, r srepresents the Spearman correlation coefficient; n represents the number of each of the two sets of data. In principle, the number of the two sets of data to be analyzed must be the same; d i Represents the difference between any two data orders, that is:

[0018] d i = rg(X i )-rg(Y i )

[0019] In the formula, X i , Y i Represents any two data points in the data, rg(X i ) indicates X i The sorting position in the data; rg(Y i ) indicates Y i The sort position in the data;

[0020] After obtaining the correlation degree through calculation of Spearman correlation coefficient, the classification is carried out according to the following criteria:

[0021]

[0022] As a preferred embodiment of the present invention, the historical meteorological data includes: maximum temperature, minimum temperature, average temperature, relative humidity and rainfall.

[0023] As a preferred embodiment of the present invention, in the step (2), the long short-term memory network model based on the attention mechanism refers to an Autoencoder-LSTM-FNN model, which is composed of an Autoencoder layer, an LSTM layer, an FNN layer and an output layer; wherein the Autoencoder layer is used to extract implicit features of the original data as input to the LSTM layer, the LSTM analyzes the input data and outputs it to the FNN layer, the data is then processed by the FNN layer and input to the output layer, and finally the output layer outputs the final result.

[0024] As a preferred solution of the present invention, in step (3), the cloud server uses a virtual client method to perform random selection:

[0025] (1) Count the number of data entries of each computing client i that may participate in training as F i , calculate the importance P of client i in the global model i r It is expressed as:

[0026] (2) Based on all original computing clients, k replica objects are generated as virtual clients according to their importance. The number of replica objects generated by the i-th client is: M i =P ir k;

[0027] (3) Integrate the virtual client with the original client. When the cloud server randomly selects a client for model training, the probability of the i-th client being selected is: When the values ​​of k and N are determined, the higher the importance of a client in the global model, the greater the probability of being selected in each training, thereby ensuring that clients with high importance have more training times.

[0028] As a preferred solution of the present invention, the homomorphic encryption algorithm in step (3) is an RSA algorithm using multiplicative homomorphic encryption, including key generation, homomorphic encryption, homomorphic assignment and homomorphic decryption.

[0029] As a preferred solution of the present invention, in step (4), the FedAvg algorithm is used to aggregate and update the global model, and the following formula is specifically used:

[0030]

[0031] Among them, G t+1 represents the global model after the t+1th round of aggregation; G t represents the global model after the tth round of aggregation; λ represents the set update coefficient; L t+1 i It represents the model updated by the i-th computing client in the t+1th round of local training.

[0032] Brief description of the invention principle:

[0033] In view of the deficiencies in the prior art, the present invention adopts cloud-edge collaboration technology and realizes the transmission of load forecasting model parameters between cloud servers and computing clients distributed in various residential areas through homomorphic encryption technology, thereby reducing communication costs and protecting residents' data privacy. At the same time, by adopting a unified global model rather than different local models to forecast load in residential areas, the problem of poor load forecasting results in residential areas due to small amounts of data or insufficient computing power is avoided. In order to reduce the amount of data parameter transmission in cloud-edge communication, Spearman analysis is performed on the meteorological data of each residential area, and meteorological data with high correlation with load is used as model features to participate in model training.

[0034] Compared with the prior art, the advantages of the present invention are:

[0035] (1) Spearman correlation analysis is used to classify the correlation of meteorological data, which avoids the participation of low-correlation meteorological data in calculations and occupying communication bandwidth, thus affecting the efficiency of model training.

[0036] (2) The idea of ​​cloud-edge collaboration is used to train a joint load forecasting model for multiple residential areas, so that clients with insufficient data and poor computing power can use the global model for load forecasting, thereby maximizing the use of data from each residential area and achieving the best collaborative forecasting effect.

[0037] (3) The model structure adopts the Autoencoder-LSTM-FNN structure, which can enhance the influence of important information on the weights of the neural network. At the same time, LSTM has a strong memory capacity, which greatly improves the accuracy of load sequence prediction.

[0038] (4) In each round of training, the virtual client method is used to select computing clients, and the difference in data volume between different computing clients is rationally utilized to improve training accuracy and efficiency.

[0039] (5) Federated learning keeps user data in residential areas locally, so that there is no large-scale data transmission during the communication process; since only model parameters are transmitted, the communication time and bandwidth required for communication are significantly reduced, thereby improving the efficiency and economy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a schematic diagram of the AutoEncoder-LSTM-FNN model architecture;

[0041] Figure 2 It is a flowchart of the virtual client method;

[0042] Figure 3 It is the load forecasting model training method;

[0043] Figure 4 Training loss curve for different k value models;

[0044] Figure 5 This is the loss change curve for the k=8 model training;

[0045] Figure 6 The training loss change curves of the two models with k=6. DETAILED DESCRIPTION

[0046] The cloud-edge collaborative multi-residential area load forecasting method based on federated learning is described in detail below.

[0047] 1. Use the computing clients located in each residential area to obtain the historical electricity load and historical meteorological data of the residential area.

[0048] Neural networks, especially LSTM, can effectively predict load sequences. However, the input of irrelevant feature data often increases training time, reduces training accuracy, and affects model effects. Therefore, before inputting meteorological data as feature quantities into the local model, the Spearman correlation coefficient is used to conduct a specific analysis of the correlation between various meteorological data and power load. Compared with the traditional Pearson correlation coefficient analysis, the Spearman correlation coefficient does not require a linear correlation between the two sets of data, but solves it based on the sorting position of the original data to obtain the positive and negative correlation between the two sets of data. Since the impact of meteorological data on power load is often nonlinear, the Spearman correlation coefficient can be used for clearer analysis.

[0049] The calculation formula of Spearman's correlation coefficient is as follows:

[0050]

[0051] In the formula, r s represents the Spearman correlation coefficient; n represents the number of each of the two sets of data. In principle, the number of the two sets of data to be analyzed must be the same;

[0052] d i Represents the difference between any two data orders, that is: d i = rg(X i )-rg(Y i )

[0053] In the formula, X i , Y i Represents any two data points in the data, rg(X i ) indicates X i The sorting position in the data; rg(Y i ) indicates Y i The sort position in the data;

[0054] Since the Spearman coefficient mainly considers the order of data arrangement and is insensitive to outliers in the data, it is applicable to a wider range of conditions. s The closer the absolute value is to 1, the higher the correlation is. A negative number indicates a negative correlation, and a positive number indicates a positive correlation.

[0055] The present invention adopts the following criteria for classification:

[0056]

[0057] The strongly correlated meteorological data that meet the preset conditions are used as part of the input of the neural network for model training.

[0058] 2. Build a long short-term memory network model (attention-lstm) based on the attention mechanism and deploy it in the cloud server and each computing client respectively.

[0059] In order to avoid the problem of poor load forecasting results in some residential areas due to small amount of data or insufficient computing power, the present invention adopts a unified global model and deploys it in the cloud server and each computing client respectively, instead of deploying local models with different structures on the computing client of each residential area for training separately.

[0060] The present invention adopts the attention-based long short-term memory network model (attention-LSTM). The long short-term memory network (1STM) is a variant of the recurrent neural network (RNN). This neural network introduces memory units, overcomes the "forgetting" defect existing in traditional recurrent neural networks, and can analyze data over a long period of time. The memory unit of LSTM consists of three nonlinear gating units that regulate the unit state.

[0061] Similar to how the human brain focuses on important information and ignores irrelevant information when processing information, adding an attention mechanism (AM) to a neural network can enhance the impact of important information on the weights of the neural network. AM can use parameter learning and weight mapping to assign different weights to the implicit state of the previous LSTM network. AM is essentially a neural network structure designed by simulating the attention allocation mechanism of the human brain. It can calculate the correlation between input data and output data and the distribution of important weight features. LSTM application AM can focus on features that have a greater impact on the output variable, thereby achieving the goal of improving the accuracy of the neural network.

[0062] Specifically, the attention-lstm of the present invention adopts the Autoencoder-LSTM-FNN model, which is specifically composed of an Autoencoder layer, an LSTM layer, an FNN layer, and an output layer. The Autoencoder layer extracts the implicit features of the original data as the input of the LSTM layer, and the LSTM analyzes the input data and outputs it to the FNN layer. The data is then processed by the FNN layer and input to the output layer, and the final result is finally output by the output layer. The schematic diagram of the model structure is shown in FIG. Figure 1 As shown, it can be established using pytorch in python.

[0063] After the global model is built, it is deployed on the cloud server and computing clients in each residential area. Using the global model deployed on the edge side for local training can avoid centralizing massive power load data and meteorological data on the cloud server, solving the problems of insufficient computing power of the cloud server and insufficient data in local areas during the prediction process, as well as inefficient transmission of massive data.

[0064] 3. Use the cloud server to randomly select several residential areas, and each computing client uses the power load and meteorological data of the residential area to locally train the model. The trained model parameters are encrypted using the homomorphic encryption algorithm and then transmitted to the cloud server.

[0065] In order to enable the features of clients with many local data entries to be fully learned, and at the same time avoid the low accuracy of the generated global model due to the low accuracy of the local model trained by the clients with few local data entries, the present invention adopts a virtual client method to select residential area clients in each round of global model training.

[0066] Virtual client method flow chart Figure 2 As shown, the specific implementation is as follows:

[0067] (1) Count the number of data entries of each computing client i that may participate in training as F i , calculate the importance P of client i in the global model i r It is expressed as:

[0068] (2) Based on all original computing clients, k replica objects are generated as virtual clients according to their importance. The number of replica objects generated by the i-th client is: M i =P i r k: The number of objects copied is not required to be an integer, because it only affects the probability of the server selecting the client.

[0069] (3) Integrate the virtual client with the original client. When the cloud server randomly selects a client for model training, the probability of the i-th client being selected is: When the values ​​of k and N are determined, the higher the importance of a client in the global model, the greater the probability of being selected in each training, thereby ensuring that clients with high importance have more training times.

[0070] In order to ensure user privacy and data security between residential areas, the present invention adopts the virtual client method to select the client and train the local model. Then, the local model parameters will be transmitted using the homomorphic encryption algorithm for interpretation and aggregation by the cloud server.

[0071] The homomorphic encryption algorithm can use the most classic RSA algorithm in multiplicative homomorphic encryption, which mainly consists of four parts: key generation, homomorphic encryption, homomorphic assignment and homomorphic decryption.

[0072] 4. The cloud server receives and decrypts the model parameters from the computing client, aggregates and updates the global model using the FedAvg algorithm, and sends the updated model parameters to all computing clients;

[0073] After receiving the locally trained model parameters transmitted by each computing client, the cloud server uses the key to decrypt the ciphertext, which can ensure that the data will not be leaked during the entire transmission process. After the ciphertext is decrypted, the model parameters from different residential areas are obtained, and the FedAvg algorithm is used to aggregate and update the global model.

[0074] The FebAvg algorithm updates the global model using the following formula:

[0075]

[0076] Among them, G t+1 represents the global model after the t+1th round of aggregation; G t represents the global model after the tth round of aggregation; λ represents the set update coefficient; L t+1 i It represents the model updated by the i-th computing client in the t+1th round of local training.

[0077] The updated global model parameters are stored, and the model training is determined to be terminated. If the preset number of training rounds is not reached, the global model parameters are sent to all computing clients before the next training. The next randomly selected client uses the new model parameters to re-train the local global model. This cycle continues until the training termination condition is met.

[0078] The global model parameters also need to be encrypted using the homomorphic encryption algorithm before being sent to all computing clients. After decryption, the clients use their respective local data for a new round of training.

[0079] 5. After repeating steps 3 and 4 and reaching the preset number of training rounds, the cloud server uses the final global model to predict the power load of each residential area.

[0080] In each round of training, k clients are randomly selected from N residential area clients to perform local model training and global model update, which reduces the overall training time and does not significantly affect the model training effect. The above training process is repeated until the preset number of training rounds is reached. Finally, after aggregation and update, a global prediction model for load forecasting of N residential areas is generated. The entire training process is as follows: Figure 3 shown.

[0081] The cloud server uses the final global model to predict the electricity load of each residential area.

[0082] A specific application example:

[0083] 1. Select the hourly load and meteorological data of 8 communities in a certain area for one year, and record the load and meteorological data every 15 minutes. Among them, the meteorological data includes: maximum temperature, minimum temperature, average temperature, relative humidity and rainfall. Then the total number of clients in the residential area is N=8, and k=4 clients are selected for each training to update the local model, and the global model of the server is updated at the same time. The update method adopts the proportional selection method proposed by the present invention.

[0084] 2. The relationship between the five meteorological data and the load is calculated using the Spearman correlation coefficient formula. The meteorological data and the load data are converted into one-dimensional sequences respectively. According to the Spearman correlation coefficient formula, the correlation measurement table between the five meteorological data and the load of the eight cells is calculated as shown in the following table:

[0085]

[0086] According to the above table, each set of data meets the significance test. The meteorological data that are strongly correlated with the load results are: maximum temperature, minimum temperature and average temperature. The three sets of data are used as the characteristic input of the model. At the same time, the first three time points, that is, one hour as the time interval, are taken, and the load data of the first three hours are used as the characteristic input.

[0087] 3. In the model training stage, the model is set as the Autoencoder-LSTM-FNN model; the first three time points, that is, one hour as the time interval, the load and meteorological data features of the first three hours are used as the input data of the AutoEncoder, and the number of input nodes of the AutoEncoder layer is the same as the number of selected features, which is 6; the number of hidden layer nodes is set to 64 or 128, and the output of the hidden layer is used as the input of the LSTM. The activation function of this layer uses ReLU. When the AutoEncoder layer is trained to the maximum accuracy, triggering early stopping or reaching the maximum setting cycle, it will output data to the subsequent neural network. In general, the more LSTM layers there are, the better the fitting effect of the neural network, but the training time will increase significantly with the increase in the number of LSTM layers. Therefore, the number of LSTM layers is set to 2 layers. The number of input nodes of the first layer LSTM is the same as the number of hidden nodes of the AutoEncoder layer, and the number of output nodes is 64 or 128. The number of input nodes of the second layer LSTM is the same as the previous layer, and the number of output nodes is 32 or 64. Both LSTM layers use Sigmoid as the activation function. The number of FNN layer output nodes is set to 8, the number of output layer input nodes is set to 8, and the activation function uses ReLU. The number of output layer nodes is set to 1, so as to obtain the predicted load data at the next time point. The neural network model uses Adam as the optimizer of network parameters. The Adam optimizer is suitable for large-scale data. In order to prevent overfitting and improve the generalization ability of the model, the Dropout method and L2 regularization method are introduced in the neural network model.

[0088] 4. After 20 rounds of training, the loss function value of the global model tends to be stable. During the training process, the cross-validation method is used. The change of the loss value is shown in the following figure, where different curves are different training results obtained by selecting different clients during the model training process. The loss function is MSE (Mean Square Error). Figure 4 shown.

[0089] It can be seen that when k = 1, 3, and 5, the global model converges, and when k = 5, the global model converges the fastest. In the fourth round of training, the error reached 0.0026.

[0090] 5. If the load data of 8 cells and the selected meteorological data are directly trained, after 20 rounds of training, the loss value changes as follows: Figure 5 shown.

[0091] It can be seen that by adopting the model structure proposed in the present invention and using the data of the entire community for model training, the prediction loss reaches 0.0026 in the second round of training. However, the time spent on direct training is several times that of the method proposed in the present invention, and it occupies a larger computing space. Therefore, the multi-residential area load forecasting method based on federated learning and proportional selection proposed in the present invention can greatly reduce the computing time and computing power occupied by the central server without significantly affecting the prediction accuracy.

[0092] 6. To verify the virtual client method proposed in this invention, the load prediction results of federated learning using the virtual client method and not using the virtual client method are compared. Select k = 6, and the loss value changes as follows: Figure 6 shown.

[0093] As can be seen from the above figure, the use of the virtual client method can make the prediction model loss converge faster and achieve the expected effect.

Claims

1. A cloud-edge collaborative multi-residential area load forecasting method based on federated learning, characterized in that: The following steps are involved: (1) Using the computing client located in each residential area, the historical electricity load and historical meteorological data of the residential area are obtained; the Spearman correlation coefficient is used to calculate and analyze the correlation between the historical meteorological data and the historical electricity load, and the correlation classification is obtained, and the strongly correlated meteorological data that meets the preset conditions is used as the input of the model; The calculation formula of the Spearman correlation coefficient is: In the formula, r s represents the Spearman correlation coefficient; n represents the number of each of the two sets of data. In principle, the number of the two sets of data to be analyzed must be the same; d i Represents the difference between any two data orders, that is: d i =rg(X i )-rg(Y i ) Where, X i , Y i Represents any two data points in the data, rg(X i ) indicates X i The sorting position in the data; rg(Y i ) indicates Y i The sort position in the data; After obtaining the correlation degree through calculation of Spearman correlation coefficient, the classification is carried out according to the following criteria: (2) Build a long short-term memory network model based on the attention mechanism (attention-LSTM) and deploy it in the cloud server and each computing client respectively; (3) The cloud server randomly selects several residential areas, and each computing client uses the power load and meteorological data of the residential area to locally train the model, and then encrypts the trained model parameters using the homomorphic encryption algorithm and transmits them to the cloud server; The cloud server uses the virtual client method for random selection: Count the number of data entries of each computing client i that may participate in training as F i , calculate the importance P of client i in the global model i r It is expressed as: Based on all original computing clients, k replica objects are generated as virtual clients according to their importance. The number of replica objects generated by the i-th client is: M i =P i r k; When the virtual client is integrated with the original client and the cloud server randomly selects a client for model training, the probability of the i-th client being selected is: When the values ​​of k and N are determined, the higher the importance of a client in the global model, the greater the probability of being selected in each training, so as to ensure that clients with high importance have more training times; (4) The cloud server receives and decrypts the model parameters from the computing client, aggregates and updates the global model, and sends the updated model parameters to all computing clients; The step (4) is to aggregate and update the global model using the FedAvg algorithm, and specifically uses the following formula: Among them, G t+1 represents the global model after the t+lth round of aggregation; G t represents the global model after the tth round of aggregation; λ represents the set update coefficient; L t+1 i It represents the model updated by the i-th computing client in the t+1th round of local local training; (5) After repeating steps (3)-(4) and reaching the preset number of training rounds, the cloud server uses the final global model to predict the electricity load of each residential area.

2. The method according to claim 1, characterized in that The historical meteorological data include: maximum temperature, minimum temperature, average temperature, relative humidity and rainfall.

3. The method according to claim 1, characterized in that In the step (2), the long short-term memory network model based on the attention mechanism refers to an Autoencoder-LSTM-FNN model, which is composed of an Autoencoder layer, an LSTM layer, an FNN layer and an output layer; wherein the Autoencoder layer is used to extract implicit features of the original data as input to the LSTM layer, the LSTM analyzes the input data and outputs it to the FNN layer, the data is then processed by the FNN layer and input to the output layer, and finally the output layer outputs the final result.

4. The method according to claim 1, characterized in that: The homomorphic encryption algorithm in step (3) is an RSA algorithm using multiplicative homomorphic encryption, including key generation, homomorphic encryption, homomorphic assignment and homomorphic decryption.

Images