Federal power load prediction method based on dynamic clustering and layered personalized aggregation

Abstract

The invention relates to the technical field of power load prediction, and discloses a federated power load prediction method based on dynamic clustering and layered personalized aggregation, comprising the following steps: constructing a transverse federated learning framework; each client carries out local load data independent training and uploads updated model parameters to the server; the server calculates similarity by using cosine similarity so as to dynamically adjust the structure of the clusters, and deletes inactive clusters by judging whether the clusters are active or not according to training rounds and setting an active period so as to adjust the number of the clusters; the server executes hierarchical federal aggregation on each cluster model based on the current clustering result; and the server respectively issues the aggregated cluster model to each client of the corresponding cluster for next training, thereby realizing continuous dynamic optimization. The method has obvious advantages in key indexes such as prediction precision and convergence rate, and the load prediction performance under the federated framework is improved.

Description

Technical Field

[0001] This invention relates to the field of power load forecasting technology, and more specifically, to a federal power load forecasting method based on dynamic clustering and hierarchical personalized aggregation. Background Technology

[0002] Short-term power load forecasting (STLF) is crucial for the efficient operation of smart grids, especially with the increasing intermittent and unpredictable supply of renewable energy. Accurate STLF forecasting is becoming increasingly critical for smart grid construction and stable dispatch. Residential electricity consumption, as a key component of the smart grid, significantly impacts the grid's rational dispatch planning and stable operation.

[0003] Federated Learning (FL), a distributed machine learning method, allows multiple devices to store their own data locally and jointly train the system by uploading only local model parameters or gradients. This results in a global model applicable to all devices without directly exchanging raw data. This mechanism avoids the risk of raw data leakage, protects privacy, and offers higher scalability than centralized learning, and has been applied in the field of power load forecasting. To address the prevalent non-identical distribution (Non-IID) characteristic of residential load data, existing solutions often introduce clustering mechanisms into federated learning, training clients with similar data distributions in clusters. These include one-time clustering methods with fixed cluster divisions before training, and dynamic clustering methods that dynamically update cluster identities during training.

[0004] However, existing dynamic clustering methods still have two key limitations: 1) These methods perform federated learning training independently within each cluster, with clusters completely independent and without interference from each other, lacking an inter-cluster information exchange mechanism. 2) The number of clusters needs to be manually specified in advance. Too many clusters may lead to a waste of computing resources, while too few clusters cannot fully capture the diversity of data distribution. This reliance on prior knowledge is often difficult to satisfy in practical applications. Summary of the Invention

[0005] To address the aforementioned technical issues, this invention provides a federated power load forecasting method based on dynamic clustering and hierarchical personalized aggregation. This method demonstrates significant advantages in key indicators such as forecast accuracy and convergence rate, effectively mitigating the impact of Non-IID load data on federated learning and improving load forecasting performance within the federated framework.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] The federal power load forecasting method based on dynamic clustering and hierarchical personalized aggregation includes the following steps:

[0008] S1. Construct a horizontal federated learning framework, which includes N resident clients and 1 central server. Each resident client stores a private electricity load dataset, which includes electricity load data, humidity data, temperature data and dew point temperature data.

[0009] S2. Local training and model upload: After receiving the model parameters sent by the central server, each resident client performs independent local training based on the local private electricity load dataset to obtain updated model parameters, and uploads the updated model parameters to the central server.

[0010] S3. Dynamic Clustering: The central server receives updated model parameters uploaded by each resident client, calculates the similarity between clients using cosine similarity, and completes the initial cluster division based on the similarity. During subsequent federated training, the similarity between clients and each cluster is calculated based on cosine similarity to dynamically adjust the cluster structure. Simultaneously, the cluster activity period is set according to the training round, and inactive clusters are deleted to dynamically adjust the number of clusters by judging whether each cluster meets the activity criteria.

[0011] S4. Model hierarchical aggregation: Based on the current clustering results obtained in S3, the central server performs hierarchical federated aggregation on the models of each cluster: global aggregation is performed on the shared layer parameters of all clusters to achieve global knowledge sharing; intra-cluster aggregation is performed on the personalized layer parameters within each cluster to maintain the feature differences and personalization capabilities between different clusters.

[0012] S5. Model distribution and iterative update: The central server distributes the cluster models aggregated in S4 to each resident client in the corresponding cluster, returns to S2 and repeats the process until the training reaches the preset stopping condition, and obtains the final federal power load prediction model.

[0013] S6. Using the final federal power load forecasting model, input 48 multi-feature data points at 30-minute intervals from the previous day, and output 48 power load forecast values ​​at 30-minute intervals from the next day.

[0014] As a preferred embodiment of the present invention, the model described in S2 is a TCN-BiLSTM hybrid network model. The structure of the TCN-BiLSTM hybrid network model includes: a TCN layer, a BiLSTM layer, and a fully connected layer (FC layer). The TCN layer is used to extract the temporal features of the power load data, the BiLSTM layer is used to capture the global contextual time information of the power load data, and the fully connected layer is used to map the features to the target output dimension.

[0015] As a preferred embodiment of the present invention, the shared layer is a TCN layer, and the personalized layer is a combination of a BiLSTM layer and a fully connected layer (FC layer) (BiLSTM-FC layer); the TCN layer of the shared layer is used to extract common temporal features shared by each cluster, and the BiLSTM-FC layer of the personalized layer is used to adapt to the unique data distribution features of each cluster.

[0016] As a preferred embodiment of the present invention, the cosine similarity in S3 is calculated as follows:

[0017]

[0018] In the formula: and They represent the client respectively and client The model parameter vector; " represents the vector dot product operation. The L2 norm operation of vectors is represented; the closer the cosine similarity value is to 1, the more similar the data distributions of the two resident clients are.

[0019] As a preferred embodiment of the present invention, the specific steps of S3 include:

[0020] Initial cluster structure construction: In the first training round, the central server updates the model parameters based on all resident clients. Construct a similarity matrix using cosine similarity Hierarchical clustering method based on matrix The initial cluster structure is obtained by partitioning all clients. and the initial model parameters of each cluster ;

[0021] Subsequent cluster structure updates: In non-initial training rounds, the central server randomly selects some resident clients to participate in training, calculates the cosine similarity between the local update parameters of each selected resident client and the current cluster model parameters, and assigns the resident client to the cluster with the highest similarity, thus completing the dynamic update of the cluster structure.

[0022] As a preferred embodiment of the present invention, an inactive cluster deletion mechanism is introduced in the subsequent cluster structure update stage to periodically check... If any cluster is inactive, it is deleted.

[0023] The active state determination period is denoted as If a cluster exists In continuous After each training round, no clients joined, thus satisfying the state condition. , If the cluster is inactive, then the cluster is considered to be in an inactive state; at this time, the first... All clusters that meet this condition are combined to form a set of inactive cluster indexes. The server then removes the corresponding cluster from the current cluster structure based on this index set. The deletion process can be represented as follows: At the same time, update the total number of clusters to... .

[0024] As a preferred embodiment of the present invention, the specific steps of intra-cluster aggregation are as follows:

[0025] The server performs intra-cluster aggregation for each cluster to maintain the model parameters for each cluster. For all clients i, the intra-cluster aggregation process is as follows:

[0026] ,

[0027] In the formula: In the first Cluster in Wheel Model parameters, For clusters Chinese client weights, For clusters after the t-th round of training Chinese client The server records the cluster model parameters as follows: ,in and These are the clusters after the t-th round of training. The model parameters for the shared layer and the personalized layer.

[0028] As a preferred embodiment of the present invention, the specific steps of the inter-cluster global aggregation are as follows:

[0029] The specific steps for inter-cluster global aggregation are as follows:

[0030] The server performs global aggregation of shared layer parameters for all clusters on a cluster-by-cluster basis to update the global shared layer, specifically as follows:

[0031]

[0032] in, This represents the updated global model parameters of the shared layer. For the first Cluster in Wheel Shared layer parameters, Global aggregation time cluster The weight parameters, Indicates the first The number of clusters in the wheel.

[0033] As a preferred embodiment of the present invention, before performing independent local training based on the local private power load dataset, a data preprocessing step is also included:

[0034] Linear interpolation was used to correct the missing humidity, temperature, and dew point temperature data in the private power load dataset.

[0035] The modified meteorological and power load data were normalized using the maximum-minimum normalization method. The normalization formula is as follows: ;in, The original data, For normalized data, The minimum value in the original data. Original number The data is based on the maximum value in the data set. The corrected meteorological data includes humidity, temperature, and dew point temperature.

[0036] The present invention also provides a federal power load forecasting system based on dynamic clustering and hierarchical personalized aggregation, for implementing the above-mentioned federal power load forecasting method. The system includes a central server and N residential clients.

[0037] The central server includes:

[0038] Clustering module: Used to perform dynamic clustering operations, including calculating the cosine similarity of model parameters between clients, constructing a similarity matrix, dynamically adjusting the cluster structure, setting the active period, and deleting inactive clusters;

[0039] The aggregation module is used to perform hierarchical federated aggregation operations, specifically including global aggregation of shared layer parameters across all clusters and intra-cluster aggregation of individual layer parameters within each cluster.

[0040] Model management module: Used to store initial model parameters and cluster model parameters for each round of training. The initial model parameters are TCN-BiLSTM hybrid network model parameters. It sends model parameters to resident clients and receives updated model parameters uploaded by resident clients.

[0041] The resident client includes:

[0042] Data preprocessing module: Used to perform data preprocessing operations, including linear interpolation to correct missing meteorological data and maximum-minimum normalization of data;

[0043] Local training module: Used to independently train local models based on preprocessed local private power load datasets and model parameters distributed by the central server, to obtain updated model parameters;

[0044] Parameter Upload Module: This module uploads updated model parameters to the central server's model management module. Updated model parameters include shared layer parameters and personalized layer parameters.

[0045] The beneficial technical effects of this invention are:

[0046] By utilizing cosine similarity to measure the similarity of client-side model parameter updates, the composition of clusters is adaptively and dynamically adjusted, and the number of clusters is dynamically adjusted according to the training epochs, thus avoiding the limitations of relying on prior settings for the number of clusters. The prediction model is decomposed into shared and personalized layers, and different aggregation methods are employed to effectively integrate global knowledge into the clusters. This allows the client to learn both cluster-specific knowledge and global knowledge simultaneously, ensuring optimal performance.

[0047] A dynamic clustering mechanism is designed to address the Non-IID characteristics of power load data. By using a clustering method based on client similarity to adaptively adjust the cluster structure and dynamically optimize the number of clusters by deleting inactive clusters, the flexibility of the federated learning framework in Non-IID scenarios is enhanced.

[0048] Based on dynamic clustering, a hierarchical personalized aggregation method is introduced, which divides the load prediction network model into a shared layer and a personalized layer. Global aggregation of the shared layer and intra-cluster aggregation of the personalized layer are performed. Based on the shared layer, global knowledge is introduced into intra-cluster learning, which effectively alleviates the problem of information isolation between clusters. Attached Figure Description

[0049] Figure 1 This is the overall prediction framework diagram of the present invention.

[0050] Figure 2 This is a schematic diagram of an electricity load forecasting model.

[0051] Figure 3 This is a diagram illustrating the load distribution on the client side.

[0052] Figure 4 This is a diagram illustrating the data volume distribution on the client side.

[0053] Figure 5 This is a schematic diagram of the convergence curves of training loss for different active periods.

[0054] Figure 6 This is a comparison chart of training loss.

[0055] Figure 7 This is a visualization of the ablation results.

[0056] Figure 8a This is a comparison chart of prediction results from random client 1.

[0057] Figure 8bThis is a comparison chart of prediction results from random client 2.

[0058] Figure 9 This is a comparison table of model hierarchical results.

[0059] Figure 10 This is the ablation experiment results table.

[0060] Figure 11 This is a table comparing the model's hierarchical results with those of the benchmark method. Detailed Implementation

[0061] In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, the specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0062] Combination Figures 1-11 The present invention provides the following embodiments:

[0063] The federal power load forecasting method based on dynamic clustering and hierarchical personalized aggregation includes the following steps:

[0064] S1. Construct a horizontal federated learning framework, which includes N resident clients and 1 central server. Each resident client stores a private electricity load dataset, which includes electricity load data, humidity data, temperature data and dew point temperature data.

[0065] S2. Local training and model upload: After receiving the model parameters sent by the central server, each resident client performs independent local training based on the local private electricity load dataset to obtain updated model parameters, and uploads the updated model parameters to the central server.

[0066] S3. Dynamic Clustering: The central server receives updated model parameters uploaded by each resident client, calculates the similarity between clients using cosine similarity, and completes the initial cluster division based on the similarity. During subsequent federated training, the similarity between clients and each cluster is calculated based on cosine similarity to dynamically adjust the cluster structure. Simultaneously, the cluster activity period is set according to the training round, and inactive clusters are deleted to dynamically adjust the number of clusters by judging whether each cluster meets the activity criteria.

[0067] S4. Model hierarchical aggregation: Based on the current clustering results obtained in S3, the central server performs hierarchical federated aggregation on the models of each cluster: global aggregation is performed on the shared layer parameters of all clusters to achieve global knowledge sharing; intra-cluster aggregation is performed on the personalized layer parameters within each cluster to maintain the feature differences and personalization capabilities between different clusters.

[0068] S5. Model distribution and iterative update: The central server distributes the cluster models aggregated in S4 to each resident client in the corresponding cluster, returns to S2 and repeats the process until the training reaches the preset stopping condition, and obtains the final federal power load prediction model.

[0069] S6. Using the final federal power load forecasting model, input 48 multi-feature data points at 30-minute intervals from the previous day, and output 48 power load forecast values ​​at 30-minute intervals from the next day.

[0070] Furthermore, the model described in S2 is a TCN-BiLSTM hybrid network model. The structure of the TCN-BiLSTM hybrid network model includes: a TCN layer, a BiLSTM layer, and a fully connected layer. The TCN layer is used to extract the temporal features of the power load data, the BiLSTM layer is used to capture the global contextual time information of the power load data, and the fully connected layer is used to map the features to the target output dimension.

[0071] Furthermore, the shared layer is a TCN layer, and the personalized layer is a combination of a BiLSTM layer and a fully connected layer; the TCN layer of the shared layer is used to extract common temporal features shared by each cluster, and the BiLSTM-FC layer of the personalized layer is used to adapt to the unique data distribution features of each cluster.

[0072] It can more effectively achieve global knowledge sharing and local personalized modeling. This is because the temporal features extracted by TCN have strong universality, while the BiLSTM-FC structure enhances the adaptability to intra-cluster feature differences, thereby obtaining better prediction performance.

[0073] As a preferred embodiment of the present invention, the cosine similarity in S3 is calculated as follows:

[0074]

[0075] In the formula: and They represent the client respectively and client The model parameter vector; " represents the vector dot product operation. The L2 norm operation of vectors is represented; the closer the cosine similarity value is to 1, the more similar the data distributions of the two resident clients are, indicating that they have the same data distribution.

[0076] Cosine similarity is used to measure the similarity between different clients, thus achieving effective quantification of the similarity of client distributions. The higher the cosine similarity, the more similar the data features of the two clients are.

[0077] Furthermore, the specific steps of S3 include:

[0078] Initial cluster structure construction: In the first training round, the central server updates the model parameters based on all resident clients. Construct a similarity matrix using cosine similarity Hierarchical clustering method based on matrix The initial cluster structure is obtained by partitioning all clients. and the initial model parameters of each cluster ;

[0079] Subsequent cluster structure updates: In non-initial training rounds, the central server randomly selects some resident clients to participate in training, calculates the cosine similarity between each selected resident client and the current cluster model parameters, and assigns the resident client to the cluster with the highest similarity, thus completing the dynamic update of the cluster structure.

[0080] Each client may be reassigned to a different cluster after each round of communication. As training continues, the client assignments gradually stabilize, meaning that the cluster to which most clients belong no longer changes over multiple consecutive rounds, eventually forming a convergent and reasonable cluster structure.

[0081] Furthermore, an inactive cluster deletion mechanism is introduced during subsequent cluster structure update phases to periodically check... If any cluster is inactive, it is deleted.

[0082] The active state determination period is denoted as If a cluster exists In continuous After each training round, no clients joined, thus satisfying the state condition. , If the cluster is inactive, then the cluster is considered to be in an inactive state; at this time, the first... All clusters that meet this condition are combined to form a set of inactive cluster indexes. The server then removes the corresponding cluster from the current cluster structure based on this index set. The deletion process can be represented as follows: At the same time, update the total number of clusters to... .

[0083] As federated learning (FL) iterates through training, the distribution of clients may gradually become concentrated, causing some clusters to no longer receive any clients after several consecutive updates. These clusters are referred to as inactive clusters. Without proper handling, this leads to cluster redundancy and resource waste, longer training times, weakens the effectiveness of cluster partitioning, and ultimately reduces the overall training efficiency and convergence performance of the model. This application introduces an inactive cluster deletion mechanism during the dynamic update phase of the cluster structure. This mechanism can dynamically optimize the cluster structure and number of clusters in Non-IID load data scenarios, ensuring the overall convergence performance and stability of federated learning.

[0084] Furthermore, the specific steps of the intra-cluster aggregation are as follows:

[0085] The server performs intra-cluster aggregation for each cluster to maintain the parameters of each cluster model. For a cluster... For all clients i, the intra-cluster aggregation process is as follows:

[0086] , ;

[0087] In the formula: In the first Cluster in Wheel Model parameters, For clusters Chinese client weights, In the first After round training cluster Chinese client The server records the cluster model parameters as follows: ,in and They are the first After round training cluster The model parameters for the shared layer and the personalized layer.

[0088] The specific steps for inter-cluster global aggregation are as follows:

[0089] The server accesses shared layer parameters across all clusters on a cluster-by-cluster basis. Global aggregation is used to update the global shared layer, specifically as follows:

[0090]

[0091] in, This represents the updated global model parameters of the shared layer. For the first Cluster in Wheel Shared layer parameters, Global aggregation time cluster The weight parameters, Indicates the first The number of clusters in the wheel.

[0092] When performing FL aggregation, the prediction model for each cluster is split into a shared layer and a personalized layer. The shared layer performs global aggregation across all clusters, enabling information sharing between clusters; while the personalized layer performs local aggregation only within each cluster's client, preserving the personalized characteristics of each cluster. Introducing a hierarchical federated aggregation method into the clustering framework achieves knowledge sharing between clusters, striking a balance between the generalization of the global model and the adaptability of the local model. This improves overall prediction performance while overcoming the limitations of completely independent intra-cluster models when processing Non-IID data.

[0093] Furthermore, before conducting independent local training based on the local private electricity load dataset, a data preprocessing step is also included:

[0094] Linear interpolation was used to correct the missing humidity, temperature, and dew point temperature data in the private power load dataset.

[0095] The modified meteorological and power load data were normalized using the maximum-minimum normalization method. The normalization formula is as follows: ;in, The original data, For normalized data, The minimum value in the original data. This represents the maximum value in the original data. The corrected meteorological data includes humidity, temperature, and dew point temperature.

[0096] The present invention also provides a federal power load forecasting system based on dynamic clustering and hierarchical personalized aggregation, for implementing the above-mentioned federal power load forecasting method. The system includes a central server and N residential clients.

[0097] The central server includes:

[0098] Clustering module: Used to perform dynamic clustering operations, including calculating the cosine similarity of model parameters between clients, constructing a similarity matrix, dynamically adjusting the cluster structure, setting the active period, and deleting inactive clusters;

[0099] The aggregation module is used to perform hierarchical federated aggregation operations, specifically including global aggregation of shared layer parameters across all clusters and intra-cluster aggregation of individual layer parameters within each cluster.

[0100] Model management module: Used to store initial model parameters and cluster model parameters for each round of training. The initial model parameters are TCN-BiLSTM hybrid network model parameters. It sends model parameters to resident clients and receives updated model parameters uploaded by resident clients.

[0101] The resident client includes:

[0102] Data preprocessing module: Used to perform data preprocessing operations, including linear interpolation to correct missing meteorological data and maximum-minimum normalization of data;

[0103] Local training module: Used to independently train local models based on preprocessed local private power load datasets and model parameters distributed by the central server, to obtain updated model parameters;

[0104] Parameter Upload Module: This module uploads updated model parameters to the central server's model management module. Updated model parameters include shared layer parameters and personalized layer parameters.

[0105] For specific implementation details of this invention, please refer to the following:

[0106] In the horizontal federated learning approach, it is assumed that there are N resident clients and a central service, where each client possesses its own private electricity dataset. ,in Represents a multi-feature input matrix. The target vector represents the training vector, and the overall power load prediction framework is as follows: Figure 1 As shown.

[0107] The main process of power load forecasting is as follows: ① Local training and model upload: Each client independently trains its local load data based on the received model parameters and uploads the updated model parameters to the server; ② Dynamic clustering: The server calculates similarity using cosine similarity based on the received client model parameters to dynamically adjust the cluster structure. Based on the training rounds, an active period is set, and inactive clusters are deleted to adjust the number of clusters; ③ Hierarchical model aggregation: Based on the current clustering results, the server performs hierarchical federated aggregation on each cluster model. Specifically, global aggregation is performed on shared layer parameters across all clusters to achieve global knowledge sharing; simultaneously, intra-cluster aggregation is performed on personalized layer parameters within each cluster to maintain feature differences and personalization capabilities between different clusters; ④ Model distribution and iterative updates: The server distributes the aggregated cluster models to each client in the corresponding cluster for the next round of training, achieving continuous dynamic optimization. While ensuring personalized expression within cluster models, knowledge interaction between different clusters is achieved, effectively mitigating the training instability problem caused by Non-IID data distribution.

[0108] In local training, a TCN-BiLSTM hybrid network is used to build the prediction model. The main structure of TCN can be divided into causal convolution suitable for time series and dilated convolution plus residual module model suitable for historical data memory. Considering that the role of TCN in the power load prediction model is to capture the long-term dependency relationship of time series data, this embodiment only adopts the structure of dilated convolution plus residual module.

[0109] Electricity load forecasting models such as Figure 2 As shown, the TCN model consists of a dilated convolutional network, a ReLU function, a normalization layer, and a Dropout layer. The feature values ​​extracted by the TCN are input into a BiLSTM to obtain global contextual time information. The BiLSTM network contains two LSTM layers with opposite transmission directions, which are better suited for modeling long-term series compared to unidirectional LSTM. Finally, the prediction results are mapped to the target output dimension through a fully connected layer (FC layer). The input data for the prediction network consists of 48 data points at 30-minute intervals from the previous day, including four types: electricity load data (kWh), humidity (%), temperature (°C), and dew point temperature (°C). The output dimension of the prediction network is 48 load data points at 30-minute intervals from the next day, achieving multi-step load sequence prediction. In data processing, linear interpolation is used to correct missing meteorological data.

[0110] Clustering is implemented by the server. In each training round, clients are randomly selected to participate in the training, and similarity metrics between clients are calculated. The process is carried out in three stages: cluster structure initialization, cluster structure update, and cluster number adjustment, in order to obtain a dynamic clustering result.

[0111] Cosine similarity is used to measure the similarity between different clients, thus achieving effective quantification of the similarity of client distributions. A higher cosine similarity indicates greater similarity in the data features of the two clients. For any two clients i and j, the cosine similarity is defined as:

[0112] (5)

[0113] In the formula: and They represent the client respectively and client The model parameter vector. Cosine similarity. The closer the value is to 1, the stronger the client's... With the client The more similar the update direction, the more likely they are to have the same data distribution.

[0114] During the cluster structure initialization phase, the central server distributes global model parameters to all clients. Each client receives parameters and performs E rounds of local training to obtain local model updates. The data is then uploaded, and the server obtains the similarity matrix for all clients according to Formula 5. Hierarchical clustering method based on matrix Divide all clients into groups to obtain the initial cluster structure. ,and ,in The initial number of clusters, Indicates the first There are 1 cluster; and model parameters are calculated for each cluster. .

[0115] During the cluster structure update phase, for the t-th round of training, the central server randomly selects a subset of clients to participate in the training; let the selected client set be denoted as . At this time, the client The cluster to which it belongs has not yet been updated. (Client) The currently received model parameters This refers to the cluster model parameters based on its history. The server follows...

[0116] (6)

[0117] For client i, the cluster with the highest similarity is selected, and cluster assignment is adjusted. In this way, each client may be reassigned to a different cluster after each round of communication. As training continues, the client assignments gradually stabilize, meaning that the cluster affiliation of most clients no longer changes in consecutive rounds, eventually forming a convergent and reasonable cluster structure.

[0118] It should be noted that as FL iterates through training, the distribution of clients may gradually become concentrated, causing some clusters to no longer receive any clients after several consecutive updates. These clusters are called inactive clusters. If left unaddressed, this will lead to cluster redundancy and resource waste, longer training time, and weaken the effectiveness of cluster partitioning, thereby reducing the overall training efficiency and convergence performance of the model. This embodiment introduces an inactive cluster deletion mechanism during the dynamic update phase of the cluster structure, periodically checking... If any cluster is inactive, it is deleted. The period for determining the active state is denoted as . If a cluster exists In continuous After each training round, no clients joined, thus satisfying the state condition. , If the cluster is inactive, then it is considered to be in an inactive state. At this time, the... All clusters that meet this condition are combined to form a set of inactive cluster indexes. The server then removes the corresponding cluster from the current cluster structure based on this index set. The deletion process can be represented as follows: At the same time, the total number of updated clusters is... .

[0119] By employing the dynamic clustering and inactive cluster deletion mechanisms mentioned above, the cluster structure and number of clusters can be dynamically optimized in the context of Non-IID load data, ensuring the convergence performance and stability of the overall federated learning.

[0120] During FL aggregation, the prediction model for each cluster is split into a shared layer and a personalized layer. The shared layer performs global aggregation across all clusters, enabling information sharing between clusters; while the personalized layer performs local aggregation only within each cluster's client, preserving the individual features of each cluster. After the t-th round of local training, the client uploads new shared layer and personalized layer parameters, which are then used for global aggregation and intra-cluster aggregation, respectively, thus completing one round of federated learning.

[0121] During the t-th training round, record the model parameters on the client side. For the cluster to which it belongs client In terms of, among them For all clusters Global shared layer parameters, For clusters The personalized layer parameters. It's worth noting that all clients within the same cluster use the same cluster model parameters. In round t, each cluster... Client in After local training is complete, the updated local parameter variables will be... The uploaded model parameters are then sent to the server. The server updates the parameters of different layers on the client using both global and intra-cluster aggregation methods. The specific aggregation steps are as follows.

[0122] First, the server performs intra-cluster aggregation for each cluster to maintain the parameters of each cluster's model. For each cluster... All clients The intra-cluster aggregation process is as follows:

[0123] , .

[0124] In the formula: In the first Cluster in Wheel Model parameters, For clusters Chinese client weights, In the first After round training cluster Chinese client The server records the cluster model parameters as follows: ,in and They are the first After round training cluster The model parameters for the shared layer and the personalized layer.

[0125] Next, the server processes the shared layer parameters for all clusters on a cluster-by-cluster basis. Global aggregation is used to update the global shared layer, specifically as follows:

[0126] (8)

[0127] in, This represents the updated global model parameters of the shared layer. For the first Cluster in Wheel Shared layer parameters, Global aggregation time cluster The weight parameters, Indicates the first The number of clusters in the round. This global aggregation method enables information exchange between different clusters, achieving the sharing of global knowledge among the clusters.

[0128] Finally, the server, for all client i participating in the training, according to the shared layer parameters... and cluster-specific layer parameters Complete model parameters , as the initial parameters for its next round of training. .

[0129] By introducing a hierarchical federated aggregation method into the clustering framework, knowledge sharing between clusters is achieved, striking a balance between the generalization of the global model and the adaptability of the local model. This not only improves the overall prediction performance but also overcomes the limitations of completely independent intra-cluster models when processing Non-IID data.

[0130] This embodiment provides the following experiment:

[0131] Experimental Dataset

[0132] The dataset for this experiment came from the Low Carbon London project dataset provided by UKPowerNetworks. This dataset primarily collected smart meter readings from 5,567 London households, covering the period from November 2011 to February 2014. Each household's smart meter readings include electricity consumption over several months to three years within the data collection period, with load data sampled at 30-minute intervals. In addition to electricity load data (kWh), hourly meteorological information was also included, such as humidity (%), temperature (°C), and dew point temperature (°C).

[0133] To conduct a detailed comparative study of different federated learning methods, this embodiment randomly selected 50 households from all participating households as the research subjects, and divided the load data of each electricity user into a training set and a test set in a 7:3 ratio. Five households were randomly selected from the 50 households, and their hourly electricity load consumption was as follows: Figure 3 As shown. From Figure 3 It is evident that there are significant differences in load pattern distribution among households, indicating that the load data exhibits a clear Non-IID characteristic. Furthermore, to simulate a Non-IID scenario for client-held data, a certain proportion of load data was removed from a random selection of clients out of the 50 clients, while the data held by the remaining electricity users remained unchanged. The data volume for each electricity user is as follows: Figure 3 As shown.

[0134] Experimental setup

[0135] The experimental hardware was an NVIDIA GeForce RTX 4090, and the software platform used Python 3.8. The network was built and trained using the PyTorch framework in Anaconda3, and the training framework of federated learning was simulated.

[0136] In such Figure 2 In the power load forecasting model shown, maximum-minimum normalization is used when the load data is input, and the load and weather data are processed separately to meet the input requirements of the forecasting model. The TCN layer uses a 3-layer dilated convolutional network to extract features from the input load sequence, with the dilation rate of each convolutional network set to 1, 2, and 2 respectively; the kernel size is 3 for all layers, and the number of channels is 16, 32, and 32 respectively. The inactivation rate of the Dropout layer is 0.2, and the number of neurons per layer of the BiLSTM network is 32. The learning rate of the forecasting network is set to 0.001, the training batch size is 32, with 3 rounds of local iteration and 80 communication rounds.

[0137] To measure the effectiveness of the prediction, this embodiment uses mean square error (MSE) and mean absolute error (MAE) as evaluation metrics, with the specific expressions as follows:

[0138] (10)

[0139] (11)

[0140] In the formula: This represents the true value of the i-th load data point; N is the total number of sample points in the load dataset. This represents the i-th load forecast value.

[0141] Setting experimental parameters

[0142] 1) Active state determination period

[0143] To measure the active cycle The impact on the training model was investigated by setting three values: 10, 15, and 20. The iterative curve of the global training loss value under the given value is as follows: Figure 5 As shown. From Figure 5 It can be seen from this that When the value is 10, the global training loss converges the fastest and has the smallest convergence value. When the training loss value is 20, the global training loss curve in the later stages of training does not show a clear convergence trend, which indicates that... Setting the value too high will require more training epochs for FL to converge. Experimental results show that... The smaller the value, the faster the model converges.

[0144] 2) Personalized tiering

[0145] To determine the optimal network layer partitioning for the prediction model in this embodiment, detailed experiments were conducted. The model layer structure set up is as follows:

[0146] Method 1: TCN is used as the shared layer, and BiLSTM-FC is used as the personalized layer;

[0147] Method 2: TCN-BiLSTM is used as the shared layer, and FC is used as the personalized layer;

[0148] Method 3: TCN-BiLSTM-FC shared layer, no personalized layer.

[0149] Table 1 shows the comparison results of MAE and MSE for different methods. As can be seen from Table 1, Method 1 achieved the best performance across all evaluation metrics, with its MAE and MSE significantly outperforming the other two structural configurations. This indicates that the hierarchical model structure using TCN as the sharing layer and BiLSTM-FC as the personalization layer can more effectively achieve global knowledge sharing and local personalized modeling. This is because the temporal features extracted by TCN have strong generality, while the BiLSTM-FC structure enhances the adaptability to intra-cluster feature differences, thus achieving better predictive performance.

[0150] In summary, all subsequent experiments were conducted using... The model structure, with =10 and TCN as shared layers and BiLSTM-FC as personalized layers, can more effectively adapt to the distribution of Non-IID load data.

[0151] ablation experiment

[0152] To verify the effectiveness of different mechanisms in DC-FL-LA, this embodiment conducted ablation experiments, and the results are shown in Table 2. Method 4: uses dynamic clustering structure and Fedavg aggregation method, where the cluster structure and number of clusters change with each training round; Method 5: uses fixed clustering and hierarchical personalized aggregation method, adjusting only the cluster structure without changing the number of clusters; DC-FL-LA (proposed method): the method proposed in this paper.

[0153] As shown in Table 2, 1) compared to Method 1, DC-FL-LA reduced MAE by 4.75% and MSE by 7.93%. This indicates that, compared to simple intra-cluster averaging (FedAvg), the hierarchical personalized aggregation strategy proposed in this embodiment achieves global fusion of shared information between clusters, effectively promoting inter-cluster knowledge sharing while preserving intra-cluster personalized features, thus achieving better prediction performance. 2) Compared to Method 2, DC-FL-LA reduced MAE by 11.32% and MSE by 12.57%. This is because dynamically changing the number of clusters avoids the negative impact of artificially setting a fixed number of clusters on the experiment, and thus better adapts to the complexity of Non-IID load data distribution.

[0154] The training loss convergence curves for different experimental settings are as follows: Figure 6 As shown. By Figure 6It can be seen that: 1) In the initial stage of training, all methods show a relatively fast convergence speed, and the training loss function is constantly decreasing. As the number of training rounds increases, the convergence speed gradually slows down, and the loss curve gradually becomes stable. However, DC-FL-LA maintains a faster downward trend throughout the entire training process, and the training loss is the lowest in the later stage of convergence, showing significantly better performance than other baseline methods; 2) Compared with method 4, the DC-FL-LA method proposed in this embodiment, after removing hierarchical personalized aggregation, has a significantly slower convergence speed and decreased stability in its loss curve, and the loss value after convergence is also higher than that of the method in this embodiment. This verifies that even if the clustering structure is dynamically optimized, if only simple FedAvg is used for intra-cluster averaging, without the exchange of shared information between clusters, the training loss of the model is still difficult to achieve better results; 3) Comparing method 5 and the method in this embodiment (DC-FL-LA), it can be seen that changing the number of cluster centers has a certain effect as training progresses. Towards the end of training, the clustering structure stabilized, and all clients were assigned to clusters with more similar feature distributions. This verifies that appropriately reducing the number of cluster centers during federated clustering can effectively improve the model's fitting accuracy and overall performance.

[0155] To further verify the feasibility of federated learning and the method proposed in this embodiment, a client was randomly selected and set up under the same complete workload dataset. A comparative experiment was conducted to compare the prediction performance of the localized learning method without federated learning, Method 1, Method 2, and the method proposed in this embodiment. The comparison results are as follows: Figure 7 As shown. Figure 7 In the diagram, the true value and the localized learning result are represented in black and orange, respectively, while Method 1, Method 2, and the method proposed in this embodiment under the federated learning method are represented in other colors.

[0156] Through the Figure 7 Analysis reveals that, compared to localized learning, FL, as a local learning method, effectively captures the nonlinear changes in residential consumer electricity consumption patterns. The method proposed in this embodiment, however, better matches the actual load curve in both overall trends and local fluctuations. This indicates that the proposed method can balance global patterns with individual differences when processing Non-IID load data, effectively improving load forecasting accuracy and further validating the effectiveness of the proposed framework in processing Non-IID load data.

[0157] Comparative experiment

[0158] To verify the effectiveness of the proposed federal electricity load forecasting framework, three common methods were set as benchmark methods, and their MAE and MSE indices were compared with those methods. The experimental results of the different comparison methods are shown in Table 1. The following is a brief description of the three benchmark methods:

[0159] FedAvg: In each round of communication, a subset of clients are randomly selected to train the model, and then the parameters are returned to the central server for aggregation, training a global model for all clients.

[0160] FL+HC: Determines the cluster structure for subsequent training through a one-time clustering method, aggregates within the clusters, and updates the structure of the clustering model.

[0161] IFCA: The server maintains multiple cluster models. In each round of communication, the client performs local training based on all the received cluster models and selects the optimal cluster to join based on the loss value, thereby achieving dynamic clustering.

[0162] As shown in Table 1, 1) in the Non-IID power load forecasting FL scenario, the DC-FL-LA method proposed in this embodiment significantly improves the convergence stability and prediction accuracy of FL through the collaborative design of similarity-based dynamic client clustering, inactive cluster deletion mechanism, and hierarchical personalized aggregation of the model, achieving excellent results; 2) compared with FedAvg, FL-HC, and IFCA, DC-FL-LA reduces MAE by 16.50%, 10.76%, and 5.68%, respectively, and MSE by 22.19%, 9.84%, and 6.13%, respectively. The reasons are: 1) dynamic clustering and inactive cluster deletion mechanism solve the problem of inaccurate client distribution caused by one-time clustering method, and also alleviate the resource waste caused by the fixed number of clusters in the dynamic clustering process; 2) the hierarchical personalized aggregation method separates the shared layer parameters and personalized layer parameters of each cluster model parameters, realizing global knowledge sharing between clusters while maintaining the personalized expression ability of the model within the cluster, thus achieving a better balance between global generalization and local adaptability. In summary, the federal load forecasting method proposed in this embodiment has better forecasting accuracy and reliability.

[0163] To illustrate the problem more clearly, Figure 8 shows a visualization of the prediction graphs for two randomly selected clients. The results demonstrate that the proposed method outperforms the benchmark methods compared to it. This method effectively utilizes data resources from all residents and captures the nonlinear variations in load data from different clients, thereby improving prediction accuracy while protecting the privacy and security of client load data.

[0164] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A federal power load forecasting method based on dynamic clustering and hierarchical personalized aggregation, characterized in that, Includes the following steps: S1. Construct a horizontal federated learning framework, which includes N resident clients and 1 central server. Each resident client stores a private electricity load dataset, which includes electricity load data, humidity data, temperature data and dew point temperature data. S2. Local training and model upload: After receiving the model parameters sent by the central server, each resident client performs independent local training based on the local private electricity load dataset to obtain updated model parameters, and uploads the updated model parameters to the central server. S3. Dynamic Clustering: The central server receives updated model parameters uploaded by each resident client, calculates the similarity between clients using cosine similarity, and completes the initial cluster division based on the similarity. During subsequent federated training, the similarity between clients and each cluster is calculated based on cosine similarity to dynamically adjust the cluster structure. Simultaneously, the cluster activity period is set according to the training round, and inactive clusters are deleted to dynamically adjust the number of clusters by judging whether each cluster meets the activity criteria. S4. Model hierarchical aggregation: Based on the current clustering results obtained in S3, the central server performs hierarchical federated aggregation on the models of each cluster: global aggregation is performed on the shared layer parameters of all clusters to achieve global knowledge sharing; intra-cluster aggregation is performed on the personalized layer parameters within each cluster to maintain the feature differences and personalization capabilities between different clusters. S5. Model distribution and iterative update: The central server distributes the cluster models aggregated in S4 to each resident client in the corresponding cluster, returns to S2 and repeats the process until the training reaches the preset stopping condition, and obtains the final federal power load prediction model. S6. Using the final federal power load forecasting model, input 48 multi-feature data points at 30-minute intervals from the previous day, and output 48 power load forecast values ​​at 30-minute intervals from the next day.

2. The federal power load forecasting method based on dynamic clustering and hierarchical personalized aggregation according to claim 1, characterized in that, The model described in S2 is a TCN-BiLSTM hybrid network model. The structure of the TCN-BiLSTM hybrid network model includes: a TCN layer, a BiLSTM layer, and a fully connected layer. The TCN layer is used to extract the temporal features of the power load data, the BiLSTM layer is used to capture the global contextual time information of the power load data, and the fully connected layer is used to map the features to the target output dimension.

3. The federal power load forecasting method based on dynamic clustering and hierarchical personalized aggregation according to claim 2, characterized in that, The shared layer is a TCN layer, and the personalized layer is a combination of a BiLSTM layer and a fully connected layer. The TCN layer of the shared layer is used to extract common time-series features shared by each cluster, and the BiLSTM-FC layer of the personalized layer is used to adapt to the unique data distribution features of each cluster.

4. The federal power load forecasting method based on dynamic clustering and hierarchical personalized aggregation according to claim 1, characterized in that, The cosine similarity calculation method described in S3 is as follows: ， In the formula: and They represent the client respectively and client The model parameter vector; " represents the vector dot product operation. The L2 norm operation of vectors is represented; the closer the cosine similarity value is to 1, the more similar the data distributions of the two resident clients are.

5. The federal power load forecasting method based on dynamic clustering and hierarchical personalized aggregation according to claim 3, characterized in that, The specific steps of S3 include: Initial cluster structure construction: In the first training round, the central server updates the model parameters based on all resident clients. Construct a similarity matrix using cosine similarity Hierarchical clustering method based on matrix The initial cluster structure is obtained by partitioning all clients. and the initial model parameters of each cluster ; Subsequent cluster structure updates: In non-initial training rounds, the central server randomly selects some resident clients to participate in training, calculates the cosine similarity between the local update parameters of each selected resident client and the current cluster model parameters, and assigns the resident client to the cluster with the highest similarity, thus completing the dynamic update of the cluster structure.

6. The federal power load forecasting method based on dynamic clustering and hierarchical personalized aggregation according to claim 5, characterized in that, S3 also includes an inactive cluster deletion mechanism introduced during subsequent cluster structure update phases, periodically checking... If any cluster is inactive, it is deleted. The active state determination period is denoted as If a cluster exists In continuous After each training round, no clients joined, thus satisfying the state condition. , If the cluster is inactive, then the cluster is considered to be in an inactive state; at this time, the first... All clusters that meet this condition are combined to form a set of inactive cluster indexes. The server then removes the corresponding cluster from the current cluster structure based on this index set. The deletion process can be represented as follows: At the same time, update the total number of clusters to... .

7. The federal power load forecasting method based on dynamic clustering and hierarchical personalized aggregation according to claim 1, characterized in that, The specific steps for intra-cluster aggregation are as follows: The server performs intra-cluster aggregation for each cluster to maintain the model parameters for each cluster. All clients The intra-cluster aggregation process is as follows: ， ， In the formula: In the first Cluster in Wheel Model parameters, For clusters Chinese client weights, In the first After round training cluster Chinese client The server records the cluster model parameters as follows: ,in and They are the first After round training cluster The model parameters for the shared layer and the personalized layer.

8. The federal power load forecasting method based on dynamic clustering and hierarchical personalized aggregation according to claim 1, characterized in that, The specific steps for inter-cluster global aggregation are as follows: The server performs global aggregation of shared layer parameters for all clusters on a cluster-by-cluster basis to update the global shared layer, specifically as follows: ， in, This represents the updated global model parameters of the shared layer. For the first Cluster in Wheel Shared layer parameters, Global aggregation time cluster The weight parameters, Indicates the first The number of clusters in the wheel.

9. The federal power load forecasting method based on dynamic clustering and hierarchical personalized aggregation according to claim 1, characterized in that, Before conducting independent local training based on the local private electricity load dataset, a data preprocessing step is also included: Linear interpolation was used to correct the missing humidity, temperature, and dew point temperature data in the private power load dataset. The maximum-minimum normalization method was used to normalize the corrected meteorological data and power load data.

10. A federal power load forecasting system based on dynamic clustering and hierarchical personalized aggregation, used to implement the federal power load forecasting method according to any one of claims 1-9, characterized in that, The system includes a central server and N resident clients; The central server includes: Clustering module: Used to perform dynamic clustering operations, including calculating the cosine similarity of model parameters between clients, constructing a similarity matrix, dynamically adjusting the cluster structure, setting the active period, and deleting inactive clusters; The aggregation module is used to perform hierarchical federated aggregation operations, specifically including global aggregation of shared layer parameters across all clusters and intra-cluster aggregation of individual layer parameters within each cluster. Model management module: Used to store initial model parameters, cluster model parameters for each training round, send model parameters to resident clients, and receive updated model parameters uploaded by resident clients; The resident client includes: Data preprocessing module: Used to perform data preprocessing operations, including linear interpolation to correct missing meteorological data and maximum-minimum normalization of data; Local training module: Used to independently train local models based on preprocessed local private power load datasets and model parameters distributed by the central server, to obtain updated model parameters; Parameter Upload Module: This module is used to upload updated model parameters to the central server for model management.

Images