A short-term power load prediction method based on KNN federated distillation learning and Seq2Seq

Abstract

This invention discloses a short-term power load forecasting method based on KNN federated distillation learning and Seq2Seq. The method involves acquiring historical power load data for a preset time period, inputting it into a trained local model on an edge computing device, and obtaining power load data for a future preset time period. This method trains local models from multiple edge computing devices together, employing federated learning and knowledge distillation during training. The local models use an encoder to extract spatiotemporal features and a decoder to convert these features into output for prediction. The KNN algorithm selects the parameters of the most similar local models from each edge computing device and aggregates them as the parameters of the teacher model. Knowledge distillation is achieved by transmitting knowledge between edge computing devices, rather than simply using a global model to update local models, thus reducing the impact of data heterogeneity. Experimental results show that the proposed S3TKFDL has better performance and robustness compared to the baseline model.

Description

Technical Field

[0001] This invention belongs to the field of power load forecasting technology, specifically relating to a short-term power load forecasting method based on KNN federated distillation learning and Seq2Seq. Background Technology

[0002] Accurate short-term power load forecasting is crucial for the safe and efficient operation of modern power systems. For power companies, short-term power load forecasting helps them rationally plan power generation, conduct transactions and dispatch in the electricity market in advance, and avoid over-investment in power generation facilities. For energy planners, short-term power load forecasting helps them scientifically plan power plant construction, rationally promote grid upgrades, and facilitate the integration of renewable energy, thereby ensuring the reliability and sustainability of energy supply. For large power users, short-term power load forecasting helps them predict future power consumption, adjust production plans, and increase electricity consumption when electricity prices are low to save costs.

[0003] Short-term power load forecasting methods can be categorized into three types: statistical methods, machine learning methods, and deep learning methods. Statistical methods, such as Autoregressive (AR) and Integrated Moving Average Autoregressive (ARIMA), are easy to understand and implement, but they face challenges in capturing the nonlinear relationships in complex load data. Machine learning methods, such as Support Vector Regression (SVR) and Feedforward Neural Networks (FNN), are more flexible and can adapt to time-varying patterns, but feature engineering is costly. Deep learning methods, such as Gated Recurrent Units (GRU) and Long Short-Term Memory Networks (LSTM), demonstrate powerful capabilities in handling large-scale data and can automatically discover the required feature representations.

[0004] However, most of the aforementioned short-term electricity load forecasting methods are centralized, meaning that load data is stored on a central server during model training, which can easily lead to privacy leaks. Specifically, load data often contains identifiable personal information, thus posing a high risk of privacy breaches. When load data obtained from smart meters contains sufficient information, managers can often identify individual consumers. Currently, federated learning has been widely adopted to avoid privacy leaks during model training. Federated learning allows models to be trained on multiple edge computing devices with local data, thus eliminating the need for centralized data storage. By transmitting the model instead of sending data to a central server, federated learning ensures that identifiable personal information does not leave the local device, thereby preventing privacy leaks.

[0005] However, model heterogeneity and data heterogeneity pose significant challenges to applying federated learning in short-term power load forecasting. On one hand, the diverse structures of local models make it difficult for the global model to efficiently train on all local data. During federated learning, the global model may fail to effectively capture the latent patterns of all nodes, resulting in suboptimal learning performance. On the other hand, power load data varies significantly across different regions (e.g., residential and industrial areas). Because the global model struggles to extract all local characteristics from data stored on different edge computing devices, it often performs well on data from some edge computing devices but poorly on data from others. Summary of the Invention

[0006] The purpose of this invention is to address the problems raised in the background art by proposing a short-term power load forecasting method based on KNN federated distillation learning and Seq2Seq.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] The present invention proposes a short-term power load forecasting method based on KNN federated distillation learning and Seq2Seq, which includes acquiring power load data within a historical preset time period, inputting it into a local model of a trained edge computing device, and obtaining power load data within a future preset time period.

[0009] The local model of the edge device includes an encoder and a decoder connected in sequence. The encoder includes a convolutional neural network, an activation layer, a batch normalization layer, an LSTM network, a temporal attention mechanism, a graph convolutional neural network, a spatial attention mechanism, and a first fully connected layer connected in sequence from data input to output. The decoder includes a preset number of hidden layers and a second fully connected layer connected in sequence.

[0010] Preferably, in the local model of the trained edge device, the power load data within a preset historical time period is first converted into a graph representation through spatiotemporal embedding, and then the converted graph is used as the input of the encoder.

[0011] Preferably, in the encoder, the transformed image is sequentially passed through a convolutional neural network, an activation layer, and a batch normalization layer to obtain the first feature, and the first feature is fused with the transformed image to obtain the second feature;

[0012] The second feature is then passed through an LSTM network and a temporal attention mechanism to obtain the third feature. The output of the LSTM network is then fused with the third feature to obtain the fourth feature.

[0013] The fourth feature is then passed through a graph convolutional neural network and a spatial attention mechanism to obtain the fifth feature. The output of the graph convolutional neural network is then fused with the fifth feature to obtain the sixth feature.

[0014] The sixth feature is then passed through the first fully connected layer to obtain the encoder output.

[0015] Preferably, in the decoder, the encoder output passes through a preset number of hidden layers in sequence, and the outputs of each hidden layer are spliced ​​together to obtain a seventh feature. The seventh feature then passes through a second fully connected layer to obtain power load data for a preset time period in the future.

[0016] Preferably, the local model of the edge device adopts [a specific method / approach] during training. Local models from multiple edge computing devices are trained together, including The first training phase of the round and The second training phase of the round:

[0017] First, the dataset is collected and preprocessed;

[0018] The preprocessed dataset is partitioned using a density-based spatial clustering algorithm, resulting in the electricity load data in the preprocessed dataset being divided into... Training data for local models on an edge computing device;

[0019] first The local model of an edge computing device was processed The first training phase of the round, and the first training phase uses federated learning, yields... The parameters of the local model of each edge computing device;

[0020] The local model of each edge computing device is then processed The second training phase of the round, those who have been trained Local models of edge computing devices, and from trained Choose one of the local models of each edge computing device to input the power load data for a preset historical time period.

[0021] Preferably, The local model of an edge computing device in In the rounds of federated learning, the first round of federated learning begins with initialization. The local model parameters of each edge computing device (used in the first round of federated learning) Local model parameters of each edge computing device), and The local model parameters of each edge computing device are aggregated to obtain the global parameters for the current federated learning round. Then, the global parameters for the current federated learning round are used to update the parameters. The local model parameters of each edge computing device, and the updated parameters are the same as those after the first round of federated learning. The local model parameters of each edge computing device are used, and the updated parameters are used as the basis for the second round of federated learning. The local model parameters of each edge computing device are cyclically processed. Rounds, get The parameters of the local model of each edge computing device;

[0022] The calculation formula for the first aggregation is as follows:

[0023] ;

[0024] This involves updating the global parameters using the current federated learning round. The formula for calculating the local model parameters of each edge computing device is as follows: (The formula shows the process of updating the local model parameters of each edge computing device)

[0025] ;

[0026] in, The value can be [0, ], Indicates the first After the first round of federated learning The parameters of the local model of an edge computing device, when middle When the value is 0, it indicates that during the first round of federated learning... The local model parameters (i.e., initial parameters) of each edge computing device. Indicates the learning rate. express gradient, Indicates the first Global parameters in round-fed learning Indicates the first In the first round of federated learning The amount of local model training data for each edge computing device Indicates the first The sum of local model training data across all edge computing devices in each round of federated learning.

[0027] Preferably, The local model of an edge computing device was processed During the second training phase of the first round, The local model parameters of the edge computing device are as follows: After rounds of federated learning Parameters of the local model of an edge computing device;

[0028] For each edge computing device's local model, parameters are selected based on the KNN algorithm for the current edge computing device's local model. The local model parameters of the closest other edge computing device;

[0029] For the local model of current edge computing devices The local model parameters of the most similar other edge computing devices are aggregated in the second aggregation to obtain the aggregated parameters, and the aggregated parameters are used as the teacher model parameters of the current edge computing device.

[0030] The local model of the current edge computing device learns from the teacher model of the current edge computing device through knowledge distillation. At the same time, the local model of the current edge computing device is trained independently based on the training data and updated with the local model parameters of the current edge computing device based on the gradient descent algorithm. The updated parameters are used as the local model parameters of the current edge computing device in the second training stage of the next round.

[0031] This cycle repeats. Rounds, to get trained A local model of an edge computing device.

[0032] Preferably, in the second training phase of each round, The local model parameters of each edge computing device are represented as follows: ;in, Indicates the first The local model parameters of the th edge computing device in the second training phase of each round are as follows: The range of values ​​for is [1, ... ];

[0033] The KNN algorithm is used for selecting local model parameters for the current edge computing device. The formula for calculating the local model parameters of the closest other edge computing devices is as follows:

[0034] ;

[0035] in, Indicates the first In the second training phase of the round The local model of the first edge computing device The local model parameters of the most similar other edge computing devices, and ;

[0036] The calculation formula for the second aggregation is as follows:

[0037] ;

[0038] in, Indicates the first In the second training phase of the round Aggregated parameters of the local model of an edge computing device Indicates the first The sum of local model training data from all edge computing devices in the second training phase of the round. Indicates the first In the second training phase of the round The number of local model training data for the nearest edge computing device.

[0039] Preferably, during the process of the local model of the current edge computing device learning from the teacher model, the total loss The calculation formula is as follows:

[0040] The formula for calculating the total loss function in the second training phase is as follows:

[0041] ;

[0042] in,

[0043] ;

[0044] in, Indicates the first After the second training phase of the round The total loss of the local model of each edge computing device Indicates the first After the second training phase of the round Knowledge distillation loss of the local model of an edge computing device Indicates the first After the second training phase of the round Supervised learning loss of the local model on an edge computing device To balance and The coefficient.

[0045] Preferably, the fusion operation is an addition operation.

[0046] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0047] This short-term power load forecasting method, based on KNN federated distillation learning and Seq2Seq, adopts... The local models of each edge computing device are trained together, employing federated learning and knowledge distillation during training. The local models use an encoder to extract spatiotemporal features, and a decoder to convert these features into output for prediction. The KNN algorithm is used to select the parameters of the most similar local models from each edge computing device, and these parameters are aggregated to serve as the parameters of the teacher model. Knowledge distillation is achieved by transmitting knowledge across edge computing devices, rather than simply using a global model to update local models, thus reducing the impact of data heterogeneity. Experimental results also show that the proposed S3TKFDL exhibits better performance and robustness compared to the baseline model. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of the local model structure of the edge computing device in the short-term power load forecasting method based on KNN federated distillation learning and Seq2Seq of this invention;

[0049] Figure 2 This is a block diagram of the modules for the first and second phases of training. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0051] In one embodiment, such as Figure 1 As shown, a short-term power load forecasting method based on KNN federated distillation learning and Seq2Seq is provided, including:

[0052] Step 1: Obtain historical power load data within a preset time period and input it into the local model of the trained edge computing device (since training uses...). The local models of each edge computing device are trained together, meaning that after training, the result will be... A trained local model for an edge computing device, The local models of each trained edge computing device are independent of each other in the actual application stage. Therefore, one trained local model of an edge computing device can be selected for power load data prediction to obtain power load data within a preset time period in the future.

[0053] Step 2: The local model of the edge device (the structure of the local model is based on the spatiotemporal module of Seq2Seq) includes an encoder (for spatiotemporal feature extraction) and a decoder (for prediction) connected in sequence. The encoder includes a convolutional neural network (CNN), an activation layer (using the ReLU activation function), a batch normalization layer (BN), an LSTM network, a temporal attention mechanism, a graph convolutional neural network (GCN), a spatial attention mechanism, and a first fully connected layer connected in sequence from data input to output. The decoder includes a preset number of hidden layers and a second fully connected layer connected in sequence.

[0054] Step 2.1: In the local model of the trained edge device, the power load data within the historical preset time period is first converted into a graph representation through spatiotemporal embedding, and then the converted graph is used as the input of the encoder.

[0055] Step 2.2: In the encoder, the transformed image is sequentially passed through a convolutional neural network (CNN), an activation layer (using the ReLU activation function), and a batch normalization layer (BN) to obtain the first feature (first feature). The first feature (which is a hidden state containing local temporal information) is fused (added) with the transformed graph to obtain the second feature; where the first feature... The calculation process can be simply represented as follows:

[0056] ;

[0057] The second feature is then passed through an LSTM network and a temporal attention mechanism to obtain the third feature. The output of the LSTM network is then fused (added) with the third feature to obtain the fourth feature. To capture global temporal information, an LSTM network and a temporal attention mechanism are used, with a gating mechanism controlling the information flow. This process is represented by the following formula:

[0058] ;

[0059] ;

[0060] ;

[0061] ;

[0062] ;

[0063] ;

[0064] ;

[0065] in, It is the time step. It's an input gate. It is the Gate of Oblivion. It's an output gate. It is a candidate memory unit. It is a memory unit. It is a hidden state that contains global time information. , , and It's weight. , , and It's a bias. It is the Sigmoid function. It's the Tanh function. It is an elemental-level Hadamard product. It is a hidden state that allows you to focus on important time information. It is an attention mechanism;

[0066] The fourth feature is then processed sequentially through a graph convolutional neural network and a spatial attention mechanism to obtain the fifth feature. The output of the graph convolutional neural network... The sixth feature is obtained by fusing (adding) the fifth feature; in order to capture spatial information, a GCN capable of capturing non-Euclidean spatial features is used based on a learnable adjacency matrix (located in the graph convolutional neural network) and a spatial attention mechanism. This process is represented by the following formula:

[0067] ;

[0068] ;

[0069] ;

[0070] in, It is a learnable adjacency matrix. It is a diagonal node degree matrix. It is the identity matrix. It's weight. It is a node embedding matrix. It's the SoftMax function. It is a hidden state that contains spatial information. It is a hidden state that allows attention to important spatial information;

[0071] The sixth feature is then passed through the first fully connected layer to obtain the encoder's output (the fully connected layer maps the hidden state to the same feature dimension).

[0072] Step 2.3: In the decoder, the encoder output passes through a preset number of hidden layers in sequence, and the outputs of each hidden layer are spliced ​​together to obtain the seventh feature. The seventh feature then passes through the second fully connected layer to obtain the power load data for a preset time period in the future.

[0073] In another embodiment, such as Figure 2 As shown, the local model of the edge device adopts the following during the training process: Local models from multiple edge computing devices are trained together, including The first training phase of the round and The second training phase of the round (first to conduct) The first training phase of the round will then proceed (Second training phase of the round)

[0074] First, a dataset was collected (in this embodiment, two open-source power load datasets, GEFCom2012 and GEFCom2017, were used. The GEFCom2012 dataset contains 32,944 load data points from 20 regions in the United States from January 2004 to June 2008; the GEFCom2017 dataset contains 397,464 load data points from 500 regions in the United States from January 2005 to December 2011), and preprocessed. In this embodiment, the preprocessing adopts normalization, which normalizes the power load data of the dataset to the range of 0 to 1.

[0075] The preprocessed dataset is partitioned using a density-based spatial clustering algorithm, resulting in the electricity load data in the preprocessed dataset being divided into... The training data for local models on each edge computing device (implemented by creating local data for local models on different edge computing devices), expressed by the formula: ,in Indicates the first Training data for local models on edge computing devices. It is DBSCAN (Density-Based Spatial Clustering of Applications with Noise), a density-based spatial clustering algorithm.

[0076] first The local model of an edge computing device was processed The first training phase of the round, and the first training phase uses federated learning, yields... The parameters of the local model of each edge computing device;

[0077] in, The local model of an edge computing device in In the rounds of federated learning, the first round of federated learning begins with initialization. The local model parameters of each edge computing device (used in the first round of federated learning) Local model parameters of each edge computing device), and The local model parameters of each edge computing device are first aggregated to obtain the global parameters for the current federated learning round (in this embodiment, the central processing unit performs the first aggregation), and then the global parameters for the current federated learning round are updated using the global parameters of the current federated learning round. The local model parameters of each edge computing device, and the updated parameters are the same as those after the first round of federated learning. The local model parameters of each edge computing device are used, and the updated parameters are used as the basis for the second round of federated learning. The local model parameters of each edge computing device are cyclically processed. Rounds, get The parameters of the local model of each edge computing device;

[0078] The calculation formula for the first aggregation is as follows:

[0079] ;

[0080] This involves updating the global parameters using the current federated learning round. The formulas for calculating the local model parameters of each edge computing device are as follows (the formulas show the process of updating the local model parameters of each edge computing device):

[0081] ;

[0082] in, The value can be [0, ], Indicates the first After the first round of federated learning The parameters of the local model of an edge computing device, when middle When the value is 0, it indicates that during the first round of federated learning... The local model parameters (i.e., initial parameters) of each edge computing device. Indicates the learning rate. express gradient, Indicates the first Global parameters in round-fed learning Indicates the first In the first round of federated learning The amount of local model training data for each edge computing device Indicates the first The sum of local model training data across all edge computing devices in each round of federated learning.

[0083] The local model of each edge computing device is then processed The second training phase of the round, those who have been trained Local models of edge computing devices, and from trained Choose one of the local models of each edge computing device to input the power load data within a preset historical time period;

[0084] in, The local model of an edge computing device was processed During the second training phase of the first round, The local model parameters of the edge computing device are as follows: After rounds of federated learning Parameters of the local model of an edge computing device;

[0085] For each edge computing device's local model, the parameters of the local model for the current edge computing device are selected based on the KNN algorithm (i.e., the K-nearest neighbor algorithm). The local model parameters of the most similar other edge computing devices (in this embodiment, the central server uses the KNN algorithm to select the local model parameters of the most similar other edge computing devices for each edge computing device, and aggregates the selected parameters in a second way as the parameters of the teacher model, wherein the structure of the teacher model is the same as the structure of the local model).

[0086] For the local model of current edge computing devices The local model parameters of the most similar other edge computing devices are aggregated to obtain aggregate parameters, and the aggregate parameters are used as the teacher model parameters of the current edge computing device. The teacher model also includes encoders and decoders connected in sequence.

[0087] The local model of the current edge computing device learns from the teacher model of the current edge computing device through knowledge distillation (knowledge distillation enables knowledge transfer between the teacher model and the local model, allowing the local model to imitate the behavior of the teacher model). At the same time, the local model of the current edge computing device is trained independently based on the training data and updates the parameters of the local model of the current edge computing device based on the gradient descent algorithm. The updated parameters are used as the parameters of the local model of the current edge computing device in the second training phase of the next round.

[0088] This cycle repeats. Rounds, to get trained A local model of an edge computing device.

[0089] In the second training phase of each round, The local model parameters of each edge computing device are represented as follows: ;in, Indicates the first The local model parameters of the th edge computing device in the second training phase of each round are as follows: The range of values ​​for is [1, ... ];

[0090] The KNN algorithm is used for selecting local model parameters for the current edge computing device. The formula for calculating the local model parameters of the closest other edge computing devices is as follows:

[0091] ;

[0092] in, Indicates the first In the second training phase of the round The local model of the first edge computing device The local model parameters of the most similar other edge computing devices, and ;

[0093] The calculation formula for the second aggregation is as follows:

[0094] ;

[0095] in, Indicates the first In the second training phase of the round Aggregated parameters of the local model of an edge computing device Indicates the first The sum of local model training data from all edge computing devices in the second training phase of the round. Indicates the first In the second training phase of the round The number of local model training data for the nearest edge computing device.

[0096] Among them, the total loss during the learning process from the local model to the teacher model in current edge computing devices is... The calculation formula is as follows:

[0097] The formula for calculating the total loss function in the second training phase is as follows:

[0098] ;

[0099] in,

[0100] ;

[0101] in, Indicates the first After the second training phase of the round The total loss of the local model of each edge computing device Indicates the first After the second training phase of the round Knowledge distillation loss of the local model of an edge computing device Indicates the first After the second training phase of the round The supervised learning loss of the local model of an edge computing device (i.e., the loss when the local model of the edge computing device is trained independently based on the training data, specifically using mean squared error loss or mean absolute error loss). To balance and The coefficient.

[0102] To evaluate the effectiveness of the proposed short-term power load forecasting method based on KNN federated distillation learning and Seq2Seq from different perspectives, this embodiment uses three evaluation metrics: mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE). The short-term power load forecasting method based on KNN federated distillation learning and Seq2Seq is also compared with the following baseline models:

[0103] FedGRU (Liu, Y., Yu, JJQ, Kang, JW, Niyato, D., & Zhang, SY(2020). Privacy-preserving traffic flow prediction: A federated learning approach. IEEE Internet of Things Journal, 7(8), 7751-7763.): A neural network combining GRU and FedAvg.

[0104] LSTM-FedAvg (Fekri, MN, Grolinger, K., & Mir, S. (2022). Distributedload forecasting using smart meter data: Federated learning with recurrent neural networks. International Journal of Electrical Power & Energy Systems, 137, 107669.): A neural network that combines LSTM and FedAvg.

[0105] LSTM-FedSGD (Fekri, MN, Grolinger, K., & Mir, S. (2022). Distributedload forecasting using smart meter data: Federated learning with recurrent neural networks. International Journal of Electrical Power&Energy Systems ,137, 107669.): A neural network that combines LSTM and Federated Stochastic Gradient Descent (FedSGD) algorithms.

[0106] LSTM-CC (Gholizadeh, N., & Musilek, P. (2022). Federated learning with hyperparameter-based clustering for electrical load forecasting. Internet of Things, 17, 100470.): A neural network that combines LSTM with a federated learning algorithm based on edge computing device clustering.

[0107] IcingFL (Cheng, X., Shi, F., Liu, YP, Liu, XF, & Huang, LZ(2022). Wind turbine blade icing detection: A federated learning approach. Energy, 254, 124441.): A neural network that combines LSTM with a federated learning algorithm based on the importance of edge computing device models.

[0108] (1) When all errors (regardless of size) are of equal importance, MAE performs well. The formula for calculating MAE is as follows:

[0109] ;

[0110] in, It is the sample size. It is the actual value. It is a predicted value;

[0111] (2) When evaluating a model, RMSE places greater emphasis on large errors. If the model's predictions have significant biases, RMSE will assign higher weights to these large biases. The formula for calculating RMSE is as follows:

[0112] ;

[0113] (3) MAPE can show the degree of error between the predicted value and the true value. The formula for calculating MAPE is as follows:

[0114] .

[0115] Experimental setup: The power load data in the dataset was split into a training set, a validation set, and a test set, accounting for 60%, 20%, and 20% of the total data, respectively.

[0116] Experimental Environment: The short-term power load forecasting method and baseline model based on KNN federated distillation learning and Seq2Seq were implemented using PyTorch. All experiments were conducted on a computer equipped with an Intel i5-8300H CPU and an NVIDIA GeForce GTX 1050 Ti GPU.

[0117] Hyperparameter settings: The number of regions selected from GEFCom2012 was set to 10, 15, and 20 respectively. The number of regions selected from GEFCom2017 was set to 20, 60, and 80 respectively. The input window was set to 24 hours (i.e., power load data within the historical 24 hours), and the output window was set to 3, 7, and 12 hours (i.e., power load data within the next 3, 7, and 12 hours respectively). The learning rate was set to 0.001, the weight decay rate was set to 0.0001, the batch size was set to 64, the number of training epochs was set to 100 (the total number of epochs in the first and second training phases was 100), the loss function was set to MAE, and the optimizer was set to Adam. To ensure fairness, the hyperparameters of the baseline models were set according to their original references.

[0118] The following compares the performance of our short-term power load forecasting method (S3TKFDL) based on KNN federated distillation learning and Seq2Seq with the baseline model on two datasets, as shown in Tables 1 and 2 (both Tables 1 and 2 show the average results of five experiments of our short-term power load forecasting method based on KNN federated distillation learning and Seq2Seq and the baseline model; Table 1 shows the performance on the GEFCom2012 dataset, and Table 2 shows the performance on the GEFCom2017 dataset):

[0119] Table 1

[0120]

[0121] Note: The values ​​shown in bold are the optimal values, and the values ​​shown in underline are the suboptimal values.

[0122] Table 2

[0123]

[0124] Note: The values ​​shown in bold are the optimal values, and the values ​​shown in underline are the suboptimal values.

[0125] According to Tables 1 and 2:

[0126] Overall, S3TKFDL outperforms most baseline models on both power datasets, validating the effectiveness of S3TKFDL.

[0127] As the number of nodes increases, the performance of S3TKFDL improves, indicating that spatiotemporal characteristics play an important role in short-term power load forecasting.

[0128] As the output window increases, S3TKFDL still performs well on most evaluation metrics, indicating that S3TKFDL has good robustness.

[0129] Ablation Experiment: This embodiment also compares different variants of S3TKFDL to verify the effectiveness of each module. Specifically, the variant design of this embodiment is as follows:

[0130] Variant 1: S3TKFDL without S3T (i.e., without the structure of the local model), does not capture spatiotemporal features, but uses LSTM to capture temporal features. In other words, Variant 1 no longer focuses on the spatial features of the load data, but only relies on LSTM to process temporal features.

[0131] Variant 2: S3TKFDL without KFDL (i.e. without the first and second training phases), which does not use federated learning to train the model, but instead uses a centralized approach to train the local model.

[0132] Variant 3: S3TKFDL without knowledge distillation, which does not use knowledge distillation to transfer knowledge between edge computing devices, but uses FedAvg to update local and global models.

[0133] To verify the necessity of each module, ablation experiments were conducted in this embodiment. Tables 3 and 4 show the average results of five experiments for each variant (Table 3 shows the ablation experiment results on the GEFCom2012 dataset, and Table 4 shows the ablation experiment results on the GEFCom2017 dataset).

[0134] Table 3

[0135]

[0136] Note: The values ​​shown in bold are the optimal values, and the values ​​shown in underline are the suboptimal values.

[0137] Table 4

[0138]

[0139] Note: Values ​​in bold are the optimal values, and values ​​in underline are the suboptimal values.

[0140] According to Tables 3 and 4:

[0141] S3TKFDL outperforms Variant 1, demonstrating the crucial role of spatiotemporal characteristics in short-term power load forecasting. The S3T module alleviates model heterogeneity issues, helping S3TKFDL capture latent patterns more effectively.

[0142] Variant 2 employs a centralized method for training the model, thus avoiding the impact of model heterogeneity or data heterogeneity, and therefore outperforms S3TKFDL. However, Variant 2 carries the risk of privacy breaches.

[0143] S3TKFDL outperforms variant 3, demonstrating that knowledge distillation is effective in federated learning. By transferring knowledge between edge computing devices, KFDL uses a global model to update local models, thus mitigating the data heterogeneity problem.

[0144] This short-term power load forecasting method, based on KNN federated distillation learning and Seq2Seq, adopts... The local models of each edge computing device are trained together, employing federated learning and knowledge distillation during training. The local models use an encoder to extract spatiotemporal features, and a decoder to convert these features into output for prediction. The KNN algorithm is used to select the parameters of the most similar local models from each edge computing device, and these parameters are aggregated to serve as the parameters of the teacher model. Knowledge distillation is achieved by transmitting knowledge across edge computing devices, rather than simply using a global model to update local models, thus reducing the impact of data heterogeneity. Experimental results also show that the proposed S3TKFDL exhibits better performance and robustness compared to the baseline model.

[0145] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A short-term power load forecasting method based on KNN federated distillation learning and Seq2Seq, characterized in that: Obtain historical power load data within a preset time period, input it into the local model of the trained edge computing device, and obtain power load data for a future preset time period; The local model of the edge computing device includes an encoder and a decoder connected in sequence. The encoder includes a convolutional neural network, an activation layer, a batch normalization layer, an LSTM network, a temporal attention mechanism, a graph convolutional neural network, a spatial attention mechanism, and a first fully connected layer connected in sequence from data input to output. The decoder includes a preset number of hidden layers and a second fully connected layer connected in sequence. Among them, first The local model of an edge computing device was processed The first training phase of the round, and the first training phase uses federated learning, yields... The parameters of the local model of each edge computing device; The local model of each edge computing device is then processed The second training phase of the round, those who have been trained Local models of edge computing devices, and from trained Choose one of the local models of each edge computing device to input the power load data within a preset historical time period; In the second stage of training, the local model parameters of multiple closest other edge computing devices are selected for the local model of each edge computing device based on the K-nearest neighbor algorithm. The local model parameters of multiple closest other edge computing devices are aggregated and used as the teacher model parameters of each edge computing device. The local model of each edge computing device learns from the corresponding teacher model through knowledge distillation. At the same time, the local model of each edge computing device is trained independently based on the training data.

2. The short-term power load forecasting method based on KNN federated distillation learning and Seq2Seq as described in claim 1, characterized in that: In the local model of the trained edge computing device, the power load data within a preset historical time period is first converted into a graph representation through spatiotemporal embedding, and then the converted graph is used as the input of the encoder.

3. The short-term power load forecasting method based on KNN federated distillation learning and Seq2Seq as described in claim 2, characterized in that: In the encoder, the transformed image is sequentially passed through a convolutional neural network, an activation layer, and a batch normalization layer to obtain the first feature. The first feature is then fused with the transformed image to obtain the second feature. The second feature is then passed through an LSTM network and a temporal attention mechanism to obtain the third feature. The output of the LSTM network is then fused with the third feature to obtain the fourth feature. The fourth feature is then passed through a graph convolutional neural network and a spatial attention mechanism to obtain the fifth feature. The output of the graph convolutional neural network is then fused with the fifth feature to obtain the sixth feature. The sixth feature is then passed through the first fully connected layer to obtain the encoder output.

4. The short-term power load forecasting method based on KNN federated distillation learning and Seq2Seq as described in claim 3, characterized in that: In the decoder, the encoder output passes through a preset number of hidden layers in sequence, and the outputs of each hidden layer are spliced ​​together to obtain the seventh feature. The seventh feature then passes through the second fully connected layer to obtain the power load data for a preset time period in the future.

5. The short-term power load forecasting method based on KNN federated distillation learning and Seq2Seq as described in claim 1, characterized in that: During the training process, the local model of the edge computing device first collects a dataset and performs preprocessing. The preprocessed dataset is partitioned using a density-based spatial clustering algorithm, resulting in the electricity load data in the preprocessed dataset being divided into... Training data for local models on edge computing devices.

6. The short-term power load forecasting method based on KNN federated distillation learning and Seq2Seq as described in claim 5, characterized in that: The local model of an edge computing device in In the rounds of federated learning, the first round of federated learning begins with initialization. Local model parameters of each edge computing device, and The local model parameters of each edge computing device are aggregated to obtain the global parameters for the current federated learning round. Then, the global parameters for the current federated learning round are used to update the parameters. The local model parameters of each edge computing device, and the updated parameters are the same as those after the first round of federated learning. The local model parameters of each edge computing device are used, and the updated parameters are used as the basis for the second round of federated learning. The local model parameters of each edge computing device are cyclically processed. Rounds, get The parameters of the local model of each edge computing device; The calculation formula for the first aggregation is as follows: ； This involves updating the global parameters using the current federated learning round. The formula for calculating the local model parameters of an edge computing device is as follows: ； in, The value can be [0, ], Indicates the first After the first round of federated learning The parameters of the local model of an edge computing device, when middle When the value is 0, it indicates that during the first round of federated learning... Local model parameters of each edge computing device Indicates the learning rate. express gradient, Indicates the first Global parameters in round-fed learning Indicates the first In the first round of federated learning The amount of local model training data for each edge computing device Indicates the first The sum of local model training data across all edge computing devices in each round of federated learning.

7. The short-term power load forecasting method based on KNN federated distillation learning and Seq2Seq as described in claim 6, characterized in that: The local model of an edge computing device was processed During the second training phase of the first round, The local model parameters of the edge computing device are as follows: After rounds of federated learning Parameters of the local model of an edge computing device; For each edge computing device's local model, parameters are selected based on the KNN algorithm for the current edge computing device's local model. The local model parameters of the closest other edge computing device; For the local model of current edge computing devices The local model parameters of the most similar other edge computing devices are aggregated in the second aggregation to obtain the aggregated parameters, and the aggregated parameters are used as the teacher model parameters of the current edge computing device. The local model of the current edge computing device learns from the teacher model of the current edge computing device through knowledge distillation. At the same time, the local model of the current edge computing device is trained independently based on the training data and updated with the local model parameters of the current edge computing device based on the gradient descent algorithm. The updated parameters are used as the local model parameters of the current edge computing device in the second training stage of the next round. This cycle repeats. Rounds, to get trained A local model of an edge computing device.

8. The short-term power load forecasting method based on KNN federated distillation learning and Seq2Seq as described in claim 7, characterized in that: In the second training phase of each round, The local model parameters of each edge computing device are represented as follows: ;in, Indicates the first The local model parameters of the th edge computing device during the second training phase of each round are as follows: The range of values ​​for is [1, ... ]; The KNN algorithm is used for selecting local model parameters for the current edge computing device. The formula for calculating the local model parameters of the closest other edge computing devices is as follows: ； in, Indicates the first In the second training phase of the round The local model of the first edge computing device The local model parameters of the most similar other edge computing devices, and ; The calculation formula for the second aggregation is as follows: ； in, Indicates the first In the second training phase of the round Aggregated parameters of the local model of an edge computing device Indicates the first The sum of local model training data from all edge computing devices in the second training phase of the round. Indicates the first In the second training phase of the round The number of local model training data for the nearest edge computing device.

9. The short-term power load forecasting method based on KNN federated distillation learning and Seq2Seq as described in claim 8, characterized in that: The total loss during the learning process from the local model to the teacher model in current edge computing devices The calculation formula is as follows: The formula for calculating the total loss function in the second training phase is as follows: ； in, ； in, Indicates the first After the second training phase of the round The total loss of the local model of each edge computing device Indicates the first After the second training phase of the round Knowledge distillation loss of the local model of an edge computing device Indicates the first After the second training phase of the round Supervised learning loss of the local model on an edge computing device To balance and The coefficient.

10. The short-term power load forecasting method based on KNN federated distillation learning and Seq2Seq as described in claim 3, characterized in that: The fusion operation is an addition operation.

Images