Federal learning-based short-term load prediction method and device

Abstract

The invention provides a short-term load prediction method and device based on federated learning. The method comprises the following steps: acquiring an initial short-term load prediction model sent by a central server; training the short-term load prediction model by adopting a local training sample, and calculating the gradient sensitivity of a trained model parameter matrix; based on the gradient sensitivity of each model parameter in the model parameter matrix, dividing the model parameters into sensitive parameters and non-sensitive parameters; carrying out encryption processing on the sensitive parameters by adopting a homomorphic encryption algorithm, and sending a processed model parameter matrix to a central server; the central server carries out federal averaging operation on the model parameter matrixes sent by all the clients; and updating the short-term load prediction model based on the model parameter average value matrix, training again, and repeating the above process until a training completion condition is reached. According to the method, the accuracy and the calculation efficiency of short-term load prediction can be effectively improved on the premise of ensuring data privacy.

Description

Technical Field

[0001] The present invention relates to the technical field of power grid load forecasting, and in particular to a short-term load forecasting method and device based on federated learning. Background Art

[0002] With the rapid development of big data and intelligent technology, traditional power systems are gradually transforming into smart grids. As a key application in power systems, short-term load forecasting can effectively help power departments balance power generation and consumption, optimize resource allocation and improve energy efficiency. Short-term load forecasting predicts future power demand by analyzing historical load data and external influencing factors such as meteorological and socio-economic data.

[0003] However, with the introduction of multi-source data, privacy protection issues have gradually become the main challenge hindering cross-industry and cross-regional data sharing. The dispersion and sensitivity of data make it difficult for various departments and organizations to directly share data, leading to the intensification of data silos, which in turn limits the performance improvement of load forecasting models. Summary of the invention

[0004] The embodiment of the present invention provides a short-term load forecasting method and device based on federated learning to solve the problem of poor data privacy protection in the short-term load forecasting process.

[0005] In a first aspect, an embodiment of the present invention provides a short-term load forecasting method based on federated learning, which is applied to a client; comprising:

[0006] Step 1: Obtain an initial short-term load forecasting model sent by a central server, wherein the short-term load forecasting model is a neural network model constructed by the central server;

[0007] Step 2: Use local training samples to train the short-term load forecasting model, and calculate the gradient sensitivity of the trained model parameter matrix; the input data of the training samples include power data, meteorological data, and social financial data of the latest N cycles; the labels of the training samples include load data of the next cycle;

[0008] Step 3: Based on the gradient sensitivity of each model parameter in the model parameter matrix, the model parameters are divided into sensitive parameters and insensitive parameters;

[0009] Step 4: Use the Paillier homomorphic encryption algorithm to encrypt sensitive parameters, and send the processed model parameter matrix to the central server; so that the central server performs a federated average operation on the model parameter matrices sent by all clients to obtain a model parameter average value matrix;

[0010] Step 5: Update the short-term load forecasting model based on the model parameter average value matrix, return to step 2, and repeat the above process until the training completion condition is met to obtain the short-term load forecasting model.

[0011] In a second aspect, an embodiment of the present invention provides a short-term load forecasting method based on federated learning, which is applied to a central server, including:

[0012] Step 1: Construct a neural network model, and train the neural network model using a public training sample based on a federated learning algorithm to obtain an initial short-term load forecasting model; the input data of the training sample includes power data, meteorological data, and social financial data of the most recent N cycles; the label of the training sample includes the load data of the next cycle;

[0013] Step 2: Send the initial neural network model to each client; so that each client uses local training samples to train the short-term load forecasting model, and calculates the gradient sensitivity of the trained model parameter matrix; based on the gradient sensitivity of each model parameter in the model parameter matrix, the model parameters are divided into sensitive parameters and non-sensitive parameters; and the sensitive parameters are encrypted using the Paillier homomorphic encryption algorithm;

[0014] Step 3: Obtain the processed model parameter matrix sent by each client; and perform a federated average operation on the model parameter matrices sent by all clients to obtain a model parameter average value matrix;

[0015] Step 4: Send the model parameter average value matrix to each client so that each client uses the model parameter average value matrix to update the short-term load forecasting model, return to step 3, repeat the above process until the training completion condition is met, and obtain the short-term load forecasting model.

[0016] In a second aspect, an embodiment of the present invention provides a short-term load forecasting device based on federated learning, which is applied to a client; comprising:

[0017] A model acquisition module, used to acquire an initial short-term load forecasting model sent by a central server, wherein the short-term load forecasting model is a neural network model constructed by the central server;

[0018] A sensitivity calculation module, used to train the short-term load forecasting model using local training samples and calculate the gradient sensitivity of the trained model parameter matrix; the input data of the training samples include power data, meteorological data and social financial data of the latest N cycles; the labels of the training samples include load data of the next cycle;

[0019] A sensitive parameter determination module, used for dividing the model parameters into sensitive parameters and non-sensitive parameters based on the gradient sensitivity of each model parameter in the model parameter matrix;

[0020] An encryption sending module is used to encrypt sensitive parameters using the Paillier homomorphic encryption algorithm and send the processed model parameter matrix to the central server; so that the central server performs a federated average operation on the model parameter matrices sent by all clients to obtain a model parameter average value matrix;

[0021] The model parameter updating module is used to update the short-term load forecasting model based on the model parameter average value matrix, return to the sensitivity calculation module, and repeat the above process until the training completion condition is met to obtain the short-term load forecasting model.

[0022] The embodiment of the present invention provides a short-term load forecasting method and device based on federated learning. The method first obtains an initial short-term load forecasting model sent by a central server, and the short-term load forecasting model is a neural network model constructed by the central server; then the short-term load forecasting model is trained using local training samples, and the gradient sensitivity of the trained model parameter matrix is ​​calculated; the input data of the training samples includes power data, meteorological data and social financial data of the most recent N cycles; the label of the training sample includes the load data of the next cycle; based on the gradient sensitivity of each model parameter in the model parameter matrix, the model parameters are divided into sensitive parameters and non-sensitive parameters; the sensitive parameters are encrypted using the Paillier homomorphic encryption algorithm, and the processed model parameter matrix is ​​sent to the central server; so that the central server performs a federated averaging operation on the model parameter matrices sent by all clients to obtain a model parameter average value matrix; finally, the short-term load forecasting model is updated based on the model parameter average value matrix, and the training is repeated, and the above process is repeated until the training completion condition is met to obtain a short-term load forecasting model. In order to ensure the privacy of data, this embodiment performs sensitivity analysis on the model parameters and uses the Paillier homomorphic encryption algorithm to encrypt only sensitive parameters during the transmission process. This can effectively improve the accuracy and computational efficiency of short-term load forecasting while ensuring data privacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0024] Figure 1is a flow chart for implementing a short-term load forecasting method based on federated learning provided by an embodiment of the present invention;

[0025] Figure 2 is an interactive flow chart of a short-term load forecasting method based on federated learning provided by an embodiment of the present invention;

[0026] Figure 3 It is a schematic diagram of the structure of the LSTM layer provided in an embodiment of the present invention;

[0027] Figure 4 is a schematic diagram of the structure of a short-term load forecasting device based on federated learning provided in an embodiment of the present invention;

[0028] Figure 5 is a schematic diagram of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0030] In the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0031] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0032] In addition, the “plurality” mentioned in the embodiments of the present application should be interpreted as two or more.

[0033] In order to make the purpose, technical solutions and advantages of the present invention more clear, specific embodiments will be described below in conjunction with the accompanying drawings.

[0034] See also Figure 1, which shows an implementation flow chart of a short-term load forecasting method based on federated learning provided by an embodiment of the present invention. The execution subject of the method is a client, which is described in detail as follows:

[0035] Step 1: Obtain an initial short-term load forecasting model sent by a central server, where the short-term load forecasting model is a neural network model constructed by the central server.

[0036] In order to address data privacy issues, this embodiment uses a federated learning method to train the neural network model. Unlike the traditional centralized method, federated learning allows each participant to train the model locally without sharing the original data. It only needs to upload the model parameters to the central server for aggregation. This method not only reduces the communication overhead of data transmission, but also effectively protects data privacy.

[0037] In a possible implementation, the short-term load forecasting model is an LSTM network model;

[0038] The LSTM network model includes 2 LSTM layers and 1 fully connected layer; and uses the root mean square error as the loss function for training and the Adam algorithm for optimization.

[0039] Furthermore, the LSTM network model includes 2 LSTM layers and 1 fully connected layer, such as Figure 3 As shown, the LSTM layer includes: unit state (c t ), hidden state (h t ), forget gate (f t ), input gate (i t ), output gate (o t ).

[0040] Given the input x at time step t t , the previous hidden state h t-1 and the cell state c at the previous time t-1 , the LSTM layer computes the following:

[0041] (1) Forget gate (f t ):f t =σ(W f x t +U f h t-1 +b f )

[0042] (2) Input gate (i t ) and input modulation gate

[0043]

[0044] (3) Unit status update (ct ):

[0045] (4) Output gate (o t ) and hidden state (h t ): o t =σ(W o x t +U o h t-1 +b o ), h t =o t ⊙tanh(c t );

[0046] Among them, W f is the input x t The weight matrix U f is the hidden state h at the previous time t-1 The weight matrix of b. f is the bias term. σ is the sigmoid activation function. W i , U i , b i is the weight matrix of the input gate, the hidden state matrix of the previous time, and the bias term. c , U c , b c is the corresponding matrix of cell state and bias term. W o , U o , b o is the matrix and bias term of the output gate. ⊙ denotes element-wise multiplication. tanh is the hyperbolic tangent activation function.

[0047] After the short-term load forecasting model is built, the central server uses public training samples to train the short-term load forecasting model. Among them, the public training sample set uses the European Open Power System Data dataset, which covers many European countries and summarizes the power series and weather series at the national level. The power series data includes load, wind power, solar power generation, and electricity prices. The weather series data includes hourly geographically aggregated weather data, including temperature, radiation intensity, etc., as well as the time series of heat demand aggregated for each country. In addition, the exchange index (closing price of the corresponding index for each country) and discrete data (such as "working days and holidays") collected from Yahoo Finance and other sources are encoded as embedded features and merged to enrich the dataset as a social financial dataset corresponding to each country. During non-trading hours, the index is uniformly set to the final closing price before the closing, and all data are read once an hour. Then it is merged into the power dataset, meteorological dataset, and social dataset according to the timestamp, and the sampling interval of all time series is 1 hour.

[0048] The processed dataset contains data from multiple countries and can cover a full year of data. The dataset includes eight features: power load, photovoltaic power generation, direct radiation intensity, diffuse radiation intensity, temperature, heat demand, and stock market closing prices. Different clients are set up in different regions, and the public dataset is divided into two subsets: 70% for training and 30% for testing.

[0049] In this embodiment, after obtaining a public data set, it is necessary to preprocess the data set to obtain a public training sample. The specific process is as follows:

[0050] Collect public datasets;

[0051] Fill in missing data in public datasets;

[0052] Normalize the filled public data set to obtain a standardized data set;

[0053] The standardized data set is sorted according to time, and the sorted standardized data set is divided into multiple training samples using a sliding window method to obtain a common training sample set.

[0054] Before training, the public data set needs to be preprocessed. The specific process is as follows:

[0055] First, the missing data in the dataset is removed, and the regions and variable columns with high missing value rates are removed to ensure that the datasets in different regions maintain the same data structure and achieve a more balanced and reasonable distribution. In addition, the datasets in each region are analyzed through correlation to remove redundant and irrelevant features.

[0056] Secondly, the padded data set is normalized, and all input variables are scaled to [0,1] using the maximum and minimum normalization formula, which is shown in the following formula:

[0057]

[0058] Where x′ is the normalized input variable; max(x) and min(x) are the maximum and minimum values ​​of the variable to be normalized, respectively.

[0059] Finally, a sliding window is used to predict the next data with 24 data, the sliding step is 1, and the training sample contains the features of the 24 data and the load value corresponding to the next hour. The input dimension is M×L, where M is the number of feature columns in the feature table and L is the width of the sliding window.

[0060] In this embodiment, in order to ensure the data privacy of federated learning, the central server and each client need to negotiate key distribution for subsequent encryption of sensitive parameters. After training the initial short-term load forecasting model, the central server sends the short-term load forecasting model to each client.

[0061] Step 2: Training step: Use local training samples to train the short-term load forecasting model and calculate the gradient sensitivity of the trained model parameter matrix; the input data of the training samples include power data, meteorological data and social financial data of the last N cycles; the labels of the training samples include the load data of the next cycle.

[0062] In a possible implementation manner, before using local training samples to train the short-term load forecasting model in step 2, the method provided in this embodiment further includes:

[0063] Collect local data sets; the local data sets include local power data, meteorological data and social financial data; the power data include load data; the data types in the data sets are the same as the data types in the public data sets.

[0064] Filling in missing data in the local data set;

[0065] Normalize the filled local data set to obtain a standardized data set;

[0066] The standardized data set is sorted according to time, and the sorted standardized data set is divided into multiple training samples using the sliding window method.

[0067] In this embodiment, the local short load prediction model of each client consists of 2 LSTM layers and 1 fully connected layer, and the root mean square error is used as the loss function to train the short load prediction model. Specifically, the initial learning rate of the short load prediction model is set to 0.001, and the Adam algorithm is used for optimization. In addition, the learning rate decays by 0.1 times every 10 training cycles, and the batch size is set to 32.

[0068] Each client performs A rounds of local training for local personalization before performing federated averaging, and A can be 8 to 15.

[0069] In one possible implementation, calculating the gradient sensitivity of the trained model parameter matrix in step 2 includes:

[0070] Based on the formula Calculate the gradient sensitivity of the trained model parameter matrix;

[0071] Among them, S mRepresents the model parameter w at the mth position in the model parameter matrix m The gradient sensitivity of J m (y k ) is the loss function l(X,y,W) with respect to w m The partial derivative of ; l(X,y,W) represents the loss function given the input matrix X, the corresponding true label y and the model parameter matrix W; K represents the number of training samples.

[0072] Step 3: Based on the gradient sensitivity of each model parameter in the model parameter matrix, the model parameters are divided into sensitive parameters and insensitive parameters.

[0073] In a possible implementation, the step 3 divides the model parameters into sensitive parameters and non-sensitive parameters based on the gradient sensitivity of each model parameter in the model parameter matrix, including:

[0074] The model parameters in the model parameter matrix are sorted in descending order of gradient sensitivity to obtain a sensitivity sequence;

[0075] The first M model parameters in the sensitivity sequence are taken as sensitive parameters; and the model parameters other than the sensitive parameters in the sensitivity sequence are taken as non-sensitive parameters.

[0076] In one embodiment, a sensitive threshold may be set, and model parameters in the model parameter matrix that are greater than the sensitive threshold are used as sensitive parameters, and model parameters in the model parameter matrix that are not greater than the sensitive threshold are used as insensitive parameters.

[0077] Step 4: Use the Paillier homomorphic encryption algorithm to encrypt sensitive parameters, and send the processed model parameter matrix to the central server; so that the central server performs a federal averaging operation on the model parameter matrices sent by all clients to obtain a model parameter average value matrix.

[0078] In one possible implementation, the step 4 of encrypting sensitive parameters using the Paillier homomorphic encryption algorithm and sending the processed model parameter matrix to the central server includes:

[0079] The encryption function C=Enc(m,r,λ) of the Paillier homomorphic encryption algorithm is used to encrypt the sensitive parameters, and the processed model parameter matrix is ​​sent to the central server;

[0080] Among them, C represents the encrypted sensitive parameter; m represents the model parameter, r represents the random value used to ensure security during the encryption process, and λ represents the public key of the Paillier homomorphic encryption algorithm.

[0081] In the proposed federated learning framework, the specific encryption process is as follows:

[0082] Each client j uses the model parameters θ j Train its local model. In order to ensure privacy and improve encryption and decryption efficiency, only the sensitive parameter set Θ sens,j These parameters are encrypted using the Paillier homomorphic encryption algorithm before being sent to the central server, and the remaining non-sensitive parameters Θ non-sens,j The client sends the model parameters Θ to the central server without encryption. sens,j The encryption can be expressed as:

[0083] C sens,j =Enc(Θ sens,j ,k)

[0084] Among them C sens,j represents the encrypted model parameters, and k is the public key used for encryption.

[0085] In a possible implementation, in step 4, sending the processed model parameter matrix to the central server; so that the central server performs a federated average operation on the model parameter matrices sent by all clients to obtain a model parameter average value matrix, including:

[0086] The processed model parameter matrix is ​​sent to the central server; so that the central server calculates the average value of the model parameters at each position in the model parameter matrix based on the model parameter matrices sent by all clients; and obtains the model parameter average value matrix.

[0087] Specifically, the central server receives the encrypted sensitive parameter C sens,j and the unencrypted non-sensitive parameter Θ non-sens,j The model parameter matrix is ​​then aggregated using the FedAVG algorithm. The homomorphic property of the Paillier encryption scheme allows the central server to calculate the average value of the encrypted parameters without decryption. The specific formula is as follows:

[0088]

[0089] Among them, C sens-avg represents the encrypted average value of the sensitive parameter, Θ non-sens-avg represents the average value of the non-sensitive parameter, and N represents the number of clients.

[0090] Finally, the central server will include the average value of sensitive parameters C sens-avg and the average value of the insensitive parameter Θ non-sens-avg The model parameter matrix is ​​sent back to each client.

[0091] Step 5: Obtain the model parameter average value matrix sent by the central server, and update the short-term load forecasting model based on the model parameter average value matrix, return to step 2, and repeat the above process until the training completion condition is met to obtain the short-term load forecasting model.

[0092] After obtaining the model parameter average matrix sent by the central server, each client decrypts C sens-avg Get the average value of sensitive parameters Θ sens-new , combined with Θ non-sens-avg Construct new model parameters Θ new , the specific formula is as follows:

[0093] Θ sens-new = Dec(C sens-avg ,k -1 )

[0094] Θ new =Θ sens-new ∪Θ non-sens-avg

[0095] where Θ new represents the updated model parameters for the next round of training, k -1 is the private key used for decryption.

[0096] For each client, the trained neural network is used to predict the load for the next 24 hours, and the evaluation results are evaluated using the root mean square error (RMSE) and the mean absolute percentage error (MAPE). RMSE is used to measure the difference between the actual value and the predicted value, while MAPE is used to evaluate the accuracy of the prediction. The expressions of RMSE and MAPE are as follows:

[0097]

[0098] where y i Indicates the actual value, represents the predicted value, and n is the total number of data samples for each client.

[0099] It can be seen from the above embodiments that this embodiment provides a short-term load forecasting method based on a federated learning framework for cross-industry and cross-regional data sharing and short-term load forecasting. Secondly, this embodiment develops a short-term load forecasting model based on LSTM, which can effectively learn context information from historical data and make accurate predictions. Thirdly, this embodiment performs gradient sensitivity analysis on the local short-term load forecasting model, and selectively uses the Paillier homomorphic encryption algorithm in the federated aggregation process based on the results of privacy sensitivity analysis, which reduces the scale of homomorphic encryption operations, encrypts and transmits model parameters to enhance data privacy protection and reduce the amount of calculation.

[0100] Figure 2 FIG. 1 shows an interactive flow diagram of a short-term load forecasting method based on federated learning provided by an embodiment of the present invention, with reference to FIG. Figure 2 The short-term load forecasting method applied to the central server provided in this embodiment is described in detail as follows:

[0101] Step 1: Construct a neural network model, and train the neural network model using a public training sample based on a federated learning algorithm to obtain an initial short-term load forecasting model; the input data of the training sample includes power data, meteorological data, and social financial data of the most recent N cycles; the label of the training sample includes the load data of the next cycle;

[0102] Step 2: Send the initial neural network model to each client; so that each client uses local training samples to train the short-term load forecasting model, and calculates the gradient sensitivity of the trained model parameter matrix; based on the gradient sensitivity of each model parameter in the model parameter matrix, the model parameters are divided into sensitive parameters and non-sensitive parameters; and the sensitive parameters are encrypted using the Paillier homomorphic encryption algorithm;

[0103] Step 3: Obtain the processed model parameter matrix sent by each client; and perform a federated average operation on the model parameter matrices sent by all clients to obtain a model parameter average value matrix;

[0104] Step 4: Send the model parameter average value matrix to each client so that each client uses the model parameter average value matrix to update the short-term load forecasting model, return to step 3, repeat the above process until the training completion condition is met, and obtain the short-term load forecasting model.

[0105] In a possible implementation, the model parameter matrices sent by all clients in step 3 are federated averaged to obtain a model parameter average matrix, including:

[0106] Based on the formula Calculating the average value of the sensitive parameters at the same position in the model parameter matrix;

[0107] Based on the formula Calculating the average value of the insensitive parameters of the agreed positions in the model parameter matrix;

[0108] Among them, C sens-avg represents the encrypted average value of the sensitive parameter, Θ non-sens-avg represents the average value of the non-sensitive parameter, C sens,j Represents the encrypted value of the sensitive parameter; non-sens,j Indicates a non-sensitive parameter; N indicates the number of clients.

[0109] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0110] The following is an embodiment of the device of the present invention. For details not described in detail therein, reference may be made to the corresponding method embodiment described above.

[0111] Figure 4 The following is a schematic diagram of the structure of a short-term load forecasting device based on federated learning provided by an embodiment of the present invention, which is applied to a client. For ease of explanation, only the part related to the embodiment of the present invention is shown, which is described in detail as follows:

[0112] like Figure 4 As shown, the short-term load forecasting device 100 based on federated learning includes:

[0113] A model acquisition module 110, used to acquire an initial short-term load forecasting model sent by a central server, wherein the short-term load forecasting model is a neural network model constructed by the central server;

[0114] The sensitivity calculation module 120 is used to train the short-term load forecasting model using local training samples and calculate the gradient sensitivity of the trained model parameter matrix; the input data of the training samples include power data, meteorological data and social financial data of the latest N cycles; the labels of the training samples include load data of the next cycle;

[0115] A sensitive parameter determination module 130, for dividing the model parameters into sensitive parameters and non-sensitive parameters based on the gradient sensitivity of each model parameter in the model parameter matrix;

[0116] The encryption and sending module 140 is used to encrypt sensitive parameters using the Paillier homomorphic encryption algorithm and send the processed model parameter matrix to the central server; so that the central server performs a federated average operation on the model parameter matrices sent by all clients to obtain a model parameter average value matrix;

[0117] The model parameter updating module 150 is used to update the short-term load forecasting model based on the model parameter average value matrix, return to the sensitivity calculation module, and repeat the above process until the training completion condition is met to obtain the short-term load forecasting model.

[0118] In a possible implementation, the sensitivity calculation module 120 includes:

[0119] Based on the formula Calculate the gradient sensitivity of the trained model parameter matrix;

[0120] Among them, S m Represents the model parameter w at the mth position in the model parameter matrix m The gradient sensitivity of J m (y k ) is the loss function l(X,y,W) with respect to w m The partial derivative of ; l(X,y,W) represents the loss function given the input matrix X, the corresponding true label y and the model parameter matrix W; K represents the number of training samples.

[0121] In a possible implementation, the sensitive parameter determination module 130 includes:

[0122] The model parameters in the model parameter matrix are sorted in descending order of gradient sensitivity to obtain a sensitivity sequence;

[0123] The first M model parameters in the sensitivity sequence are taken as sensitive parameters; and the model parameters other than the sensitive parameters in the sensitivity sequence are taken as non-sensitive parameters.

[0124] In a possible implementation, the encryption sending module 140 includes:

[0125] The processed model parameter matrix is ​​sent to the central server; so that the central server calculates the average value of the model parameters at each position in the model parameter matrix based on the model parameter matrices sent by all clients; and obtains the model parameter average value matrix.

[0126] In a possible implementation, the encryption sending module 140 includes:

[0127] The encryption function C=Enc(m,r,λ) of the Paillier homomorphic encryption algorithm is used to encrypt the sensitive parameters, and the processed model parameter matrix is ​​sent to the central server;

[0128] Among them, C represents the encrypted sensitive parameter; m represents the model parameter, r represents the random value used to ensure security during the encryption process, and λ represents the public key of the Paillier homomorphic encryption algorithm.

[0129] In a possible implementation, the sensitivity calculation module 120 includes:

[0130] Collecting local data sets; the local data sets include local power data, meteorological data and social financial data; the power data include load data;

[0131] Filling in missing data in the local data set;

[0132] Normalize the filled local data set to obtain a standardized data set;

[0133] The standardized data set is sorted according to time, and the sorted standardized data set is divided into multiple training samples using the sliding window method.

[0134] In a possible implementation, the short-term load forecasting model is an LSTM network model;

[0135] The LSTM network model includes 2 LSTM layers and 1 fully connected layer; and uses the root mean square error as the loss function for training and the Adam algorithm for optimization.

[0136] The structure of the short-term load forecasting device based on federated learning provided in an embodiment of the present invention is applied to a central processing unit, and is described in detail as follows:

[0137] A model building module is used to build a neural network model, and train the neural network model using a public training sample based on a federated learning algorithm to obtain an initial short-term load forecasting model; the input data of the training sample includes power data, meteorological data, and social financial data of the most recent N cycles; the label of the training sample includes load data of the next cycle;

[0138] A model sending module is used to send the initial neural network model to each client; so that each client uses local training samples to train the short-term load forecasting model and calculate the gradient sensitivity of the trained model parameter matrix; based on the gradient sensitivity of each model parameter in the model parameter matrix, the model parameters are divided into sensitive parameters and non-sensitive parameters; and the sensitive parameters are encrypted using the Paillier homomorphic encryption algorithm;

[0139] The federated averaging module is used to obtain the processed model parameter matrices sent by each client; and perform a federated averaging operation on the model parameter matrices sent by all clients to obtain a model parameter average value matrix;

[0140] The parameter sending module is used to send the model parameter average value matrix to each client so that each client uses the model parameter average value matrix to update the short-term load forecasting model, return to step 3, repeat the above process until the training completion condition is met, and obtain the short-term load forecasting model.

[0141] In one possible implementation, the federated averaging module includes:

[0142] Based on the formula Calculating the average value of the sensitive parameters at the same position in the model parameter matrix;

[0143] Based on the formula Calculating the average value of the insensitive parameters of the agreed positions in the model parameter matrix;

[0144] Among them, C sens-avg represents the encrypted average value of the sensitive parameter, Θ non-sens-avg represents the average value of the non-sensitive parameter, C sens,j Represents the encrypted value of the sensitive parameter; non-sens,j Indicates a non-sensitive parameter; N indicates the number of clients.

[0145] Figure 5 is a schematic diagram of a terminal provided by an embodiment of the present invention. Figure 5 As shown, the terminal 5 of this embodiment includes: a processor 50 and a memory 51. The memory 51 is used to store a computer program 52, and the processor 50 is used to call and run the computer program 52 stored in the memory 51 to perform the steps in the above-mentioned embodiments of the short-term load forecasting method based on federated learning, such as Figure 1 Alternatively, the processor 50 is used to call and run the computer program 52 stored in the memory 51 to implement the functions of each module / unit in the above-mentioned device embodiments, for example Figure 4 The functions of the modules 110 to 150 are shown.

[0146] Exemplarily, the computer program 52 may be divided into one or more modules / units, which are stored in the memory 51 and executed by the processor 50 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program 52 in the terminal 5. For example, the computer program 52 may be divided into Figure 4 Modules 110 to 150 are shown.

[0147] The terminal 5 may be the client mentioned in this embodiment, or may be a central server. The terminal 5 may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art will appreciate that Figure 5 It is only an example of terminal 5 and does not constitute a limitation on terminal 5. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal may also include input and output devices, network access devices, buses, etc.

[0148] The processor 50 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0149] The memory 51 may be an internal storage unit of the terminal 5, such as a hard disk or memory of the terminal 5. The memory 51 may also be an external storage device of the terminal 5, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal 5. Further, the memory 51 may also include both an internal storage unit of the terminal 5 and an external storage device. The memory 51 is used to store the computer program and other programs and data required by the terminal. The memory 51 may also be used to temporarily store data that has been output or is to be output.

[0150] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0151] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0152] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0153] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0154] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0155] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0156] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned short-term load forecasting method embodiments based on federated learning can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practices in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practices, computer-readable media does not include electrical carrier signals and telecommunication signals.

[0157] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A short-term load forecasting method based on federated learning, characterized in that: Applied to the client; including: obtaining the initial short-term load forecasting model sent by the central server, the short-term load forecasting model is a neural network model constructed by the central server; training step: using local training samples to train the short-term load forecasting model, and calculating the gradient sensitivity of the trained model parameter matrix; the input data of the training samples include the power data, meteorological data and social financial data of the latest N cycles; the labels of the training samples include the load data of the next cycle; based on the gradient sensitivity of each model parameter in the model parameter matrix, the model parameters are divided into sensitive parameters and non-sensitive parameters; the Paillier homomorphic encryption algorithm is used to encrypt the sensitive parameters, and the processed model parameter matrix is ​​sent to the central server; so that the central server performs a federal averaging operation on the model parameter matrix sent by the client to obtain a model parameter average value matrix; based on the model parameter average value matrix, the short-term load forecasting model is updated, and the training step is returned to the training step, and the above process is repeated until the training completion condition is met to obtain the short-term load forecasting model.

2. The short-term load forecasting method based on federated learning according to claim 1, characterized in that: The step of calculating the gradient sensitivity of the trained model parameter matrix includes: Based on the formula Calculate the gradient sensitivity of the trained model parameter matrix; Among them, S m Represents the model parameter w at the mth position in the model parameter matrix m The gradient sensitivity of J m (y k ) is the loss function l(X,y,W) with respect to w m The partial derivative of ; l(X,y,W) represents the loss function given the input matrix X, the corresponding true label y and the model parameter matrix W; K represents the number of training samples.

3. The short-term load forecasting method based on federated learning according to claim 1 is characterized in that: The model parameters are divided into sensitive parameters and non-sensitive parameters based on the gradient sensitivity of each model parameter in the model parameter matrix, including: The model parameters in the model parameter matrix are sorted in descending order of gradient sensitivity to obtain a sensitivity sequence; The first M model parameters in the sensitivity sequence are taken as sensitive parameters; and the model parameters other than the sensitive parameters in the sensitivity sequence are taken as non-sensitive parameters.

4. The short-term load forecasting method based on federated learning according to claim 1 is characterized in that: The processed model parameter matrix is ​​sent to the central server; so that the central server performs a federated average operation on the model parameter matrices sent by all clients to obtain a model parameter average value matrix, including: The processed model parameter matrix is ​​sent to the central server; so that the central server calculates the average value of the model parameters at each position in the model parameter matrix based on the model parameter matrices sent by all clients; and obtains the model parameter average value matrix.

5. The short-term load forecasting method based on federated learning according to claim 1 is characterized in that: The method of encrypting sensitive parameters using the Paillier homomorphic encryption algorithm and sending the processed model parameter matrix to the central server includes: The encryption function C=Enc(m,r,λ) of the Paillier homomorphic encryption algorithm is used to encrypt the sensitive parameters, and the processed model parameter matrix is ​​sent to the central server; Among them, C represents the encrypted sensitive parameter; m represents the model parameter, r represents the random value used to ensure security during the encryption process, and λ represents the public key of the Paillier homomorphic encryption algorithm.

6. The short-term load forecasting method based on federated learning according to claim 1 is characterized in that: Before the short-term load forecasting model is trained using local training samples, the method provided in this embodiment further includes: Collecting local data sets; the local data sets include local power data, meteorological data and social financial data; the power data include load data; Filling in missing data in the local data set; Normalize the filled local data set to obtain a standardized data set; The standardized data set is sorted according to time, and the sorted standardized data set is divided into multiple training samples using the sliding window method.

7. The short-term load forecasting method based on federated learning according to claim 1 is characterized in that: The short-term load forecasting model is an LSTM network model; The LSTM network model includes 2 LSTM layers and 1 fully connected layer; and uses the root mean square error as the loss function for training and the Adam algorithm for optimization.

8. A short-term load forecasting method based on federated learning, characterized in that: Applied to the central server, including: Step 1: Construct a neural network model, and train the neural network model using a public training sample based on a federated learning algorithm to obtain an initial short-term load forecasting model; the input data of the training sample includes power data, meteorological data, and social financial data of the most recent N cycles; the label of the training sample includes the load data of the next cycle; Step 2: Send the initial neural network model to each client; so that each client uses local training samples to train the short-term load forecasting model, and calculates the gradient sensitivity of the trained model parameter matrix; based on the gradient sensitivity of each model parameter in the model parameter matrix, the model parameters are divided into sensitive parameters and non-sensitive parameters; and the sensitive parameters are encrypted using the Paillier homomorphic encryption algorithm; Step 3: Obtain the processed model parameter matrix sent by each client; and perform a federated average operation on the model parameter matrices sent by all clients to obtain a model parameter average value matrix; Step 4: Send the model parameter average value matrix to each client so that each client uses the model parameter average value matrix to update the short-term load forecasting model, return to step 3, repeat the above process until the training completion condition is met, and obtain the short-term load forecasting model.

9. The short-term load forecasting method based on federated learning according to claim 8, characterized in that: The model parameter matrices sent by all clients are federated averaged to obtain a model parameter average matrix, including: Based on the formula Calculating the average value of the sensitive parameters at the same position in the model parameter matrix; Based on the formula Calculating the average value of the insensitive parameters of the agreed positions in the model parameter matrix; Among them, C sens-avg represents the encrypted average value of the sensitive parameter, Θ non-sens-avg represents the average value of the non-sensitive parameter, C sens,j Represents the encrypted value of the sensitive parameter; non-sens,j Indicates a non-sensitive parameter; N indicates the number of clients.

10. A short-term load forecasting device based on federated learning, characterized in that: Applied to the client; includes: A model acquisition module, used to acquire an initial short-term load forecasting model sent by a central server, wherein the short-term load forecasting model is a neural network model constructed by the central server; A sensitivity calculation module, used to train the short-term load forecasting model using local training samples and calculate the gradient sensitivity of the trained model parameter matrix; the input data of the training samples include power data, meteorological data and social financial data of the latest N cycles; the labels of the training samples include load data of the next cycle; A sensitive parameter determination module, used for dividing the model parameters into sensitive parameters and non-sensitive parameters based on the gradient sensitivity of each model parameter in the model parameter matrix; An encryption sending module is used to encrypt sensitive parameters using the Paillier homomorphic encryption algorithm and send the processed model parameter matrix to the central server; so that the central server performs a federated average operation on the model parameter matrices sent by all clients to obtain a model parameter average value matrix; The model parameter updating module is used to update the short-term load forecasting model based on the model parameter average value matrix, return to the sensitivity calculation module, and repeat the above process until the training completion condition is met to obtain the short-term load forecasting model.

Images