Federal learning-based power load prediction system and training method thereof

Abstract

The invention discloses a federated learning-based power load prediction system and a training method thereof, the system comprises a server, a client, an encryption and decryption module and a storage and processing module, and the method adopts a federated learning framework to realize distributed training and optimization of a power load prediction model. According to the method, a plurality of clients are allowed to locally train the power load prediction model, the model parameters are sent to the server for aggregation and updating, and the data set is distributed at each client locally, so that the risk that the data is illegally accessed or leaked is reduced, and the user experience is improved. According to the method, the LSTM neural network model is adopted to predict the power load, the characteristics of the time sequence data can be fully utilized, the prediction accuracy is improved, the regularization hyper-parameters are optimized by utilizing the Bayesian optimization algorithm in the prediction process, the performance of the model can be further improved, and the prediction result is more accurate.

Description

Technical Field

[0001] The present invention relates to the technical field of power load forecasting, and in particular to a power load forecasting system based on federated learning and a training method thereof. Background Art

[0002] Power load forecasting has become a key link in power system operation and planning. However, traditional power load forecasting methods often rely on a large amount of historical data and complex mathematical models, and the data is usually stored in data centers, which brings about problems such as data privacy leakage and communication bottlenecks. The existing technical solutions have the following defects:

[0003] 1. Traditional power load forecasting methods require a large amount of historical data to be stored in a data center, which increases the risk of illegal access or leakage of data. Once the data center is attacked by hackers or internal personnel leak data, it will pose a serious threat to user privacy and power grid security;

[0004] 2. Traditional power load forecasting methods are usually trained based on global data sets and fail to fully consider the local data characteristics and needs of each client. This may lead to deviations in the model when predicting power load in a specific area, reducing the accuracy and practicality of the prediction;

[0005] 3. Under the centralized machine learning framework, the client needs to frequently upload raw data or model parameters to the server for training and updating, which not only increases the network bandwidth usage, but also may cause communication delays and bottlenecks, affecting the overall training efficiency and prediction accuracy. Summary of the invention

[0006] The purpose of the present invention is to overcome the existing technical defects and provide a power load forecasting system based on federated learning and a training method thereof. The method adopts the framework of federated learning to realize the distributed training and optimization of the power load forecasting model. The method allows multiple clients to train the power load forecasting model locally and send the model parameters to the server for aggregation and updating. Unlike the traditional method, the present invention no longer stores a large amount of historical data in a data center, but distributes it locally on each client, thereby reducing the risk of illegal access or leakage of data. The server is responsible for receiving the model parameters sent by each client, updating the global model after aggregation, and then sending the updated model parameters back to each client to realize collaborative training and optimization of the model.

[0007] In order to solve the above technical problems, the technical solution provided by the present invention is: a power load forecasting system based on federated learning, comprising a server, a client, an encryption and decryption module and a storage and processing module;

[0008] The server is responsible for the deployment of the global model, the updating of global parameters, and the reception and aggregation of local model parameters from the client;

[0009] There are multiple clients, each of which is deployed with a local model, which is trained using local power load data, and the training results are encrypted and sent to the server;

[0010] The encryption and decryption module ensures data privacy during data transmission between the client and the server;

[0011] The storage and processing module is used to store global model parameters, local model parameters and historical power load data, and perform data processing and model training.

[0012] The power load forecasting training method based on federated learning provided by the present invention is applied to a power load forecasting system based on federated learning, and the method comprises the following steps:

[0013] Step S1: System initialization: The server deploys the global model and initializes the global parameters. Each client deploys the local model and collects local power load data to obtain a local data set.

[0014] Step S2: Client encrypted information transmission and verification: Each client sends an encrypted information packet containing its identification and timestamp to the server, and the server decrypts and verifies each encrypted information packet. The client that passes the verification is selected as the target client;

[0015] Step S3: multi-round iterative training: the server encrypts the global parameters in the global model and the current number of iterations of each local model, and sends them to each target client. The target client decrypts the encrypted global parameters and the current number of iterations of each local model, trains the local model with the local data set, and obtains the gradient of the local model. The target client encrypts the gradient to obtain the encrypted gradient and sends it to the server. The server decrypts the received multiple encrypted gradients, trains the global model based on the multiple encrypted gradients, and updates the global parameters to obtain the updated global model parameters.

[0016] Step S4: Model optimization and prediction: The server resends the updated global model parameters to all clients to update the global model. All clients use the updated global model to perform power load prediction and obtain prediction data.

[0017] Step S5: Prediction data transmission: distill the prediction data, and transmit the prediction data after distillation to the storage and processing module.

[0018] Furthermore, the step S1 specifically includes the following steps:

[0019] Step S11: The server deploys a global model: The server selects the deep learning framework TensorFlow to build a global model. The global model selects a neural network model with a general structure, the long short-term memory network LSTM, which is used to process time series data.

[0020] Step S12: Initialize global parameters: The server initializes the weight and bias parameters of the global model;

[0021] Step S13: Client deploys local model: Each client participating in federated learning deploys a local model with the same structure as the global model. These local models are updated using local power load data during training;

[0022] Step S14: Collect local power load data: The client collects power load data from the local data source, performs preprocessing, cleans and normalizes the power load data to obtain a local data set.

[0023] Furthermore, the step S2 specifically includes the following steps:

[0024] Step S21: Sending encrypted information: Each client generates an encrypted information package containing its unique identification and timestamp, and sends it to the server. The encrypted information package is used to verify the identity of the client and the timeliness of the data;

[0025] Step S22: Decryption and verification: After receiving the encrypted information package, the server uses the decryption key to decrypt it. The server verifies whether the client's identification is legal and whether the timestamp is within the set range. The client that passes the verification is selected as the target client and participates in the subsequent federated learning training process.

[0026] Furthermore, the step S3 specifically includes the following steps:

[0027] Step S31: Encrypting global parameters and iteration times: The server encrypts the global parameters of the global model, i.e., the current parameter weights and biases, and the current iteration times of each local model, and sends them to each target client;

[0028] Step S32: Decryption and training: After receiving the encrypted global parameters and iteration times, the target client uses the decryption key to decrypt them. The target client uses the local data set to train the local model to obtain the gradient of the local model.

[0029] Step S33: Encrypting and sending the gradient: The target client encrypts the gradient obtained by training to obtain an encrypted gradient, and sends it to the server to ensure the privacy protection of the gradient during transmission;

[0030] Step S34: Decrypt and update the global model: After receiving the encrypted gradient, the server uses the decryption key to decrypt it. The server trains the global model based on multiple encrypted gradients, updates the global parameters to obtain updated global model parameters, and calculates the updated global model parameters using the following formula:

[0031]

[0032] Among them, θk +1 is the updated global model parameter, k is the number of iterations, N is the total number of samples of all clients, ni is the number of samples of the i-th client, is the local model parameter of the i-th client, η is the learning rate, Updated local model parameters for the i-th client.

[0033] Furthermore, the step S4 specifically includes the following steps:

[0034] Step S41: Sending updated global model parameters: The server resends the updated global model parameters to all clients;

[0035] Step S42: Use the updated model to make predictions: The client loads the updated global model parameters, optimizes the updated global model parameters, inputs the local data set into the optimized global model, and performs power load prediction to obtain prediction data.

[0036] Furthermore, in step S42, optimizing the updated global model parameters specifically includes the following steps:

[0037] Step S421: Regularization optimization: Regularization optimization is performed on the global model parameters updated by each edge computing node client through the central node server. The formula for calculating the regularization term is as follows:

[0038]

[0039] Where R(ω) is the regularization term, ω is the updated global model parameter, λ is the regularization coefficient, and o is the number of updated global model parameters;

[0040] Step S422: Bayesian optimization: Use the Bayesian optimization algorithm to optimize the regularization optimization hyperparameters to further improve the performance of the model.

[0041] Furthermore, the step S5 specifically includes the following steps:

[0042] Step S51: Data distillation: Aggregate and average the predicted data. Through distillation, a more compact and easier-to-transmit data set can be generated while retaining enough information for subsequent analysis. The formula for calculating the predicted data after distillation is as follows:

[0043]

[0044] in, is the prediction data of all clients, m is the number of clients, is the prediction data of the i-th client;

[0045] Step S52: Data transmission: The distilled prediction data is transmitted to the storage and processing module.

[0046] The advantages of the present invention compared with the prior art are:

[0047] (1) During the entire training process, data transmission between the client and the server is encrypted to ensure privacy protection during data transmission and avoid the risk of data being intercepted or tampered with during transmission. Federated learning only sends model parameters without sharing original data, ensuring that the local data of each client is retained locally and will not be uploaded to the server or other clients, thereby greatly reducing the risk of privacy leakage. The present invention can reduce the number of communications between the client and the server by adopting an efficient load forecasting method based on single federated learning, further reducing the risk of privacy leakage.

[0048] (2) The use of the long short-term memory network (LSTM) neural network model for power load forecasting can make full use of the characteristics of time series data and improve the accuracy of forecasting. In the forecasting process, the Bayesian optimization algorithm is used to optimize the regularization hyperparameters, which can further improve the performance of the model and make the forecast results more accurate.

[0049] (3) After the server decrypts the multiple encrypted gradients received, it aggregates and updates the global model parameters. This process is completed in a short time, thereby improving the overall training efficiency.

[0050] (4) Under the framework of federated learning, personalized model training can be performed for each client's local data, which enables the model to better adapt to the power load characteristics of different regions and improve the practicality of the prediction;

[0051] (5) The present invention can be seamlessly integrated with the existing power load forecasting system, and is also easy to deploy and maintain, and can be easily expanded to more clients and scenarios. As the power load data continues to increase and change, the system can support continuous updating and iteration of the global model, which can ensure that the model is always in the optimal state and improve the accuracy and reliability of the forecast;

[0052] (6) In single-shot federated learning, the single communication between the client and the server is completed by transmitting distilled data. This method can significantly reduce the communication cost and improve the communication efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a schematic diagram of the overall process of the power load forecasting training method based on federated learning of the present invention;

[0054] Figure 2 It is a flow chart of step S1 proposed by the power load forecasting training method based on federated learning of the present invention;

[0055] Figure 3 It is a flow chart of step S2 proposed by the power load forecasting training method based on federated learning of the present invention;

[0056] Figure 4 It is a flow chart of step S3 proposed by the power load forecasting training method based on federated learning of the present invention;

[0057] Figure 5 It is a flow chart of step S4 proposed by the power load forecasting training method based on federated learning of the present invention;

[0058] Figure 6 It is a flow chart of step S5 proposed in the power load forecasting training method based on federated learning of the present invention. DETAILED DESCRIPTION

[0059] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0060] In the description of the present invention, it should be understood that terms such as “upper”, “lower”, “front”, “back”, “left”, “right”, “top”, “bottom”, “inside” and “outside” indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the present invention.

[0061] Embodiment 1: A power load forecasting system based on federated learning, comprising a server, a client, an encryption and decryption module, and a storage and processing module;

[0062] The server is responsible for the deployment of the global model, the update of global parameters, and the reception and aggregation of local model parameters from the client. In this embodiment, the server is a Dell PowerEdge R740 equipped with an Intel Xeon Scalable processor, a large-capacity memory 64GB DDR4 ECC and a high-speed storage device NVMe SSD;

[0063] There are three clients, each of which is deployed with a local model, trained using local power load data, and encrypted and sent the training results to the server. The client is an embedded device or a low-power PC, and the model is NVIDIA Jetson Nano.

[0064] The encryption and decryption module ensures data privacy during data transmission between the client and the server, and uses Python's cryptography library to implement encryption and decryption based on AES (Advanced Encryption Standard);

[0065] The storage and processing module is used to store global model parameters, local model parameters and historical power load data, and perform data processing and model training, using the distributed database Apache Cassandra.

[0066] Embodiment 2: This embodiment is based on embodiment 1. Figure 1 As shown, the power load forecasting training method based on federated learning provided by the present invention is applied to the power load forecasting system based on federated learning, and the method comprises the following steps:

[0067] Step S1: System initialization: The server deploys the global model and initializes the global parameters. Each client deploys the local model and collects local power load data to obtain a local data set.

[0068] Step S2: Client encrypted information transmission and verification: Each client sends an encrypted information packet containing its identification and timestamp to the server, and the server decrypts and verifies each encrypted information packet. The client that passes the verification is selected as the target client;

[0069] Step S3: multi-round iterative training: the server encrypts the global parameters in the global model and the current number of iterations of each local model, and sends them to each target client. The target client decrypts the encrypted global parameters and the current number of iterations of each local model, trains the local model with the local data set, and obtains the gradient of the local model. The target client encrypts the gradient to obtain the encrypted gradient and sends it to the server. The server decrypts the received multiple encrypted gradients, trains the global model based on the multiple encrypted gradients, and updates the global parameters to obtain the updated global model parameters.

[0070] Step S4: Model optimization and prediction: The server resends the updated global model parameters to all clients to update the global model. All clients use the updated global model to perform power load prediction and obtain prediction data.

[0071] Step S5: Prediction data transmission: distill the prediction data, and transmit the prediction data after distillation to the storage and processing module.

[0072] Embodiment 3: This embodiment is based on the above embodiment. Figure 2 As shown, the step S1 specifically includes the following steps:

[0073] Step S11: Server deploys global model: The server selects the deep learning framework TensorFlow to build a global model. The global model selects a neural network model with a general structure, the long short-term memory network LSTM, which is used to process time series data and build an LSTM global model using the TensorFlow framework.

[0074] Step S12: Initialize global parameters: The server initializes the weight and bias parameters of the global model;

[0075] Step S13: Client deploys local model: Each client participating in federated learning deploys a local model with the same structure as the global model. These local models are updated using local power load data during the training process. An LSTM local model with the same structure as the global model is deployed on each client.

[0076] Step S14: Collect local power load data: The client collects power load data from the local data source and performs preprocessing, cleans and normalizes the power load data to obtain a local data set, collects power load data from the local data source (smart grid sensor), cleans and normalizes the data to obtain a local data set.

[0077] Embodiment 4: This embodiment is based on the above embodiment. Figure 3 As shown, step S2 specifically includes the following steps:

[0078] Step S21: Sending encrypted information: Each client generates an encrypted information package containing its unique identification and timestamp, and sends it to the server. The encrypted information package is used to verify the identity of the client and the timeliness of the data. Each client generates an encrypted information package containing a unique identification and timestamp, encrypts it using AES, and sends it to the server;

[0079] Step S22: Decryption and verification: After receiving the encrypted information packet, the server uses the decryption key to decrypt it. The server verifies whether the client's identification is legal and whether the timestamp is within the set range. The client that passes the verification is selected as the target client to participate in the subsequent federated learning training process. The server uses the AES decryption key to decrypt the information packet, verify the legitimacy of the client identification and the validity of the timestamp, and selects the client that passes the verification as the target client.

[0080] Embodiment 5: This embodiment is based on the above embodiment. Figure 4 As shown, step S3 specifically includes the following steps:

[0081] Step S31: Encrypting global parameters and iteration times: The server encrypts the global parameters of the global model, i.e., the current parameter weights and biases, and the current iteration times of each local model, and sends them to each target client. The server encrypts the global parameters (weights and biases) and iteration times using AES and sends them to each target client.

[0082] Step S32: Decryption and training: After receiving the encrypted global parameters and iteration times, the target client uses the decryption key to decrypt them. The target client uses the local data set to train the local model to obtain the gradient of the local model. The target client decrypts the global parameters and iteration times, and uses the local data set to train the local model to obtain the gradient.

[0083] Step S33: Encrypting and sending the gradient: The target client encrypts the gradient obtained by training to obtain an encrypted gradient, and sends it to the server to ensure the privacy protection of the gradient during transmission. The target client encrypts the gradient using AES and sends it to the server;

[0084] Step S34: Decrypt and update the global model: After receiving the encrypted gradient, the server uses the decryption key to decrypt it. The server trains the global model based on multiple encrypted gradients, updates the global parameters to obtain updated global model parameters. Three clients participate in federated learning. The number of samples of each client is 1000, 1500, and 2000 respectively. N is the total number of samples of all clients, which is 4500. The learning rate η is 0.01. The formula used to calculate the updated global model parameters is as follows:

[0085]

[0086] Among them, θk +1 is the updated global model parameter, k is the number of iterations, N is the total number of samples of all clients, ni is the number of samples of the i-th client, is the local model parameter of the i-th client, η is the learning rate, Updated local model parameters for the i-th client.

[0087] Embodiment 6: This embodiment is based on the above embodiment. Figure 5 As shown, step S4 specifically includes the following steps:

[0088] Step S41: Sending updated global model parameters: The server resends the updated global model parameters to all clients. The server sends the updated global model parameters to all clients.

[0089] Step S42: Use the updated model for prediction: The client loads the updated global model parameters, optimizes the updated global model parameters, inputs the local data set into the optimized global model, performs power load prediction to obtain prediction data, and inputs the local data set into the optimized global model for power load prediction.

[0090] In step S42, optimizing the updated global model parameters specifically includes the following steps:

[0091] Step S421: Regularization optimization: Regularization optimization is performed on the global model parameters updated by each edge computing node client through the central node server. The updated global model parameters are a vector with a length of 100. The formula for calculating the regularization term is as follows:

[0092]

[0093] Where R(ω) is the regularization term, ω is the updated global model parameter, λ is the regularization coefficient, and o is the number of updated global model parameters;

[0094] Step S422: Bayesian optimization: Use the Bayesian optimization algorithm to optimize the regularization optimization hyperparameters to further improve the performance of the model.

[0095] Embodiment 7: This embodiment is based on the above embodiment. Figure 6 As shown, step S5 specifically includes the following steps:

[0096] Step S51: Data distillation: Aggregate and average the predicted data. Through distillation, a more compact and easier-to-transmit data set can be generated while retaining enough information for subsequent analysis. The number of clients is 3. The formula for calculating the predicted data after distillation is as follows:

[0097]

[0098] in, is the prediction data of all clients, m is the number of clients, is the prediction data of the i-th client;

[0099] Step S52: Data transmission: The distilled prediction data is transmitted to the storage and processing module.

[0100] The present invention and its embodiments are described above, which is not restrictive. The drawings show only one embodiment of the present invention, which is not limited thereto. In short, if ordinary technicians in the field are inspired by it and design embodiments similar to the technical solution without creativity without departing from the purpose of the invention, they should all fall within the protection scope of the present invention.

Claims

1. A power load forecasting training method based on federated learning, applied to a power load forecasting system based on federated learning, the system comprising a server, a client, an encryption and decryption module and a storage and processing module, characterized in that: The method comprises the following steps: Step S1: System initialization: The server deploys the global model and initializes the global parameters. Each client deploys the local model and collects local power load data to obtain a local data set. Step S2: Client encrypted information transmission and verification: Each client sends an encrypted information packet containing its identification and timestamp to the server, and the server decrypts and verifies each encrypted information packet. The client that passes the verification is selected as the target client; Step S3: multi-round iterative training: the server encrypts the global parameters in the global model and the current number of iterations of each local model, and sends them to each target client. The target client decrypts the encrypted global parameters and the current number of iterations of each local model, trains the local model using the local data set, and obtains the gradient of the local model. The target client encrypts the gradient to obtain the encrypted gradient and sends it to the server. The server decrypts the received multiple encrypted gradients, trains the global model based on the multiple encrypted gradients, and updates the global parameters to obtain the updated global model parameters. Step S4: Model optimization and prediction: The server resends the updated global model parameters to all clients to update the global model. All clients use the updated global model to perform power load prediction and obtain prediction data. Step S5: Prediction data transmission: distill the prediction data, and transmit the prediction data after distillation to the storage and processing module.

2. The power load forecasting training method based on federated learning according to claim 1 is characterized in that: The step S2 comprises the following steps: Step S21: Sending encrypted information: Each client generates an encrypted information package containing its unique identification and timestamp, and sends it to the server. The encrypted information package is used to verify the identity of the client and the timeliness of the data; Step S22: Decryption and verification: After receiving the encrypted information package, the server uses the decryption key to decrypt it. The server verifies whether the client's identification is legal and whether the timestamp is within the set range. The client that passes the verification is selected as the target client and participates in the subsequent federated learning training process.

3. The power load forecasting training method based on federated learning according to claim 1 is characterized in that: The step S3 comprises the following steps: Step S31: Encrypting global parameters and iteration times: The server encrypts the global parameters of the global model, i.e., the current parameter weights and biases, and the current iteration times of each local model, and sends them to each target client; Step S32: Decryption and training: After receiving the encrypted global parameters and number of iterations, the target client uses the decryption key to decrypt them. The target client uses the local data set to train the local model to obtain the gradient of the local model. Step S33: Encrypting and sending the gradient: The target client encrypts the gradient obtained by training to obtain an encrypted gradient, and sends it to the server to ensure the privacy protection of the gradient during transmission; Step S34: Decrypt and update the global model: After receiving the encrypted gradient, the server uses the decryption key to decrypt it. The server trains the global model based on multiple encrypted gradients, updates the global parameters to obtain updated global model parameters, and calculates the updated global model parameters using the following formula: Among them, θk+1 is the updated global model parameter, k is the number of iterations, N is the total number of samples of all clients, ni is the number of samples of the i-th client, is the local model parameter of the i-th client, η is the learning rate, Updated local model parameters for the i-th client.

4. The power load forecasting training method based on federated learning according to claim 1 is characterized in that: The step S4 comprises the following steps: Step S41: Sending updated global model parameters: The server resends the updated global model parameters to all clients; Step S42: Use the updated model to make predictions: The client loads the updated global model parameters, optimizes the updated global model parameters, inputs the local data set into the optimized global model, and performs power load prediction to obtain prediction data.

5. The power load forecasting training method based on federated learning according to claim 4 is characterized in that: The step S42 comprises the following steps: Step S421: Regularization optimization: Regularization optimization is performed on the global model parameters updated by each edge computing node client through the central node server. The formula for calculating the regularization term is as follows: Where R(ω) is the regularization term, ω is the updated global model parameter, λ is the regularization coefficient, and o is the number of updated global model parameters; Step S422: Bayesian optimization: Use the Bayesian optimization algorithm to optimize the regularization optimization hyperparameters to further improve the performance of the model.

6. The power load forecasting training method based on federated learning according to claim 1 is characterized in that: The step S5 comprises the following steps: Step S51: Data distillation: Aggregate and average the predicted data. The formula for calculating the predicted data after distillation is as follows: in, is the prediction data of all clients, m is the number of clients, is the prediction data of the i-th client; Step S52: Data transmission: The distilled prediction data is transmitted to the storage and processing module.

Images