Method for achieving load prediction based on federated learning of multi-party samples

Abstract

The invention relates to a method for achieving load prediction based on federated learning of multi-party samples, and the method comprises the steps: S1, carrying out the local model training of multi-party samples of federated learning; s2: federated learning model aggregation: specially designing an aggregation model for a load prediction scene, wherein the construction of the aggregation model needs to carry out analysis and trend detection on a prediction result of a global model; s3, federal learning privacy data protection: a differential privacy protection technology is adopted, the technology protects local data by adding noise to data or training model output, original privacy data is prevented from being obtained through reverse calculation, and a noise adding mode should be determined according to data characteristics and privacy protection requirements; the load prediction accuracy is high, and a more reliable result can be provided no matter short-term load fluctuation or long-term load trend prediction. The model training efficiency is high, and the training efficiency of the whole federated learning load prediction model is improved, so that support can be provided for load prediction of the power distribution network more timely.

Description

Technical Field

[0001] The present invention belongs to the technical field of parameter perception and fault handling methods of calibration systems, and specifically relates to a method for achieving load forecasting based on federated learning of multi-party samples. Background Art

[0002] With the large-scale application of big data models in power distribution networks, load forecasting model training requires a large amount of basic power data. However, data from different sources (such as different provincial grid companies and individual users) cannot be directly exchanged due to privacy and security constraints. Existing technologies fail to effectively train load forecasting models while meeting data privacy protection requirements, so an innovative approach is needed to solve this problem. Summary of the invention

[0003] In order to solve the technical problems in the above-mentioned background technology, the present invention provides a method for load forecasting of federated learning based on multi-party samples, including: S1: local model training of multi-party samples of federated learning: performing deep feature analysis on the data of local clients participating in model training, including extracting features, identifiers and signs of the data to determine the distribution characteristics of the data samples, and then selecting a suitable federated learning mode from horizontal federated learning, vertical federated learning and federated transfer learning modes according to these characteristics;

[0004] Determine the training method, process, learning requirements, model deployment method and development method of federated learning according to the selected federated learning model, and determine the adopted topology structure according to the requirements and model of federated learning, where the topology structure includes but is not limited to centralized learning mode, distributed learning mode or hierarchical learning mode;

[0005] During the training process, multiple candidate global models can be obtained by using different model structures and hyperparameters, and the distribution end has the right to select local clients to participate in model updates. The distribution end broadcasts the global model to the selected clients, and the clients train based on local data and upload intermediate results. The end aggregates and updates the global model based on the results uploaded by the clients.

[0006] S2: Federated learning model aggregation: An aggregation model is designed specifically for load forecasting scenarios. The construction of this aggregation model requires analysis and trend detection of the forecast results of the global model;

[0007] Model combination strategies are used for model aggregation, including but not limited to randomly extracting data subsets from the original data, training independent models on each subset, and then aggregating by voting or averaging or according to model weights;

[0008] Study the convergence technology of the aggregated model. This convergence technology needs to combine the model aggregation goals and computing resource constraints, and use weighted average or gradient descent or other equivalent methods to achieve the optimal convergence result;

[0009] S3: Federated learning privacy data protection: Differential privacy protection technology is used to protect local data by adding noise to the data or training model output to prevent the original private data from being obtained through reverse calculation. The noise addition method should be determined according to the characteristics of the data and the privacy protection requirements.

[0010] Using homomorphic encryption, the technology allows model training to be performed directly on encrypted data, and the training effect is similar to that of training directly on the original data, ensuring that even if the data is obtained during the entire training process, it cannot be restored to the original data;

[0011] Multi-party computing privacy protection technology is used, which allows multiple participants to jointly calculate the agreed function results without sharing their original data, so that all parties can participate in model training while protecting their own privacy data.

[0012] In federated learning multi-party sample local model training:

[0013] When the local data of local client i is represented as {(xi1,yi1),(xi2,yi2),…,(xin,yin;)}, for the simple linear regression model y=θ0+θ1x, the loss function of local client i is And the update formula of θ0 is The update formula of θ1 is Where α is the learning rate, and is not limited to simple linear regression models. Other model structures should also have corresponding loss function definitions and parameter update formulas.

[0014] In federated learning model aggregation:

[0015] If the prediction results of m independent models are aggregated using simple average The aggregation result is

[0016] If weighted average aggregation is used, assuming the weight of model k is wk, the aggregation result is

[0017] In the weighted average convergence process, let the parameter of the global model be θ, and the model parameter uploaded by client i in the tth iteration be Its weight is Then the update formula of the global model is Where N is the number of clients participating in the aggregation, and is not limited to the above aggregation and convergence formulas. Other equivalent aggregation and convergence methods should also be within the protection scope of this claim.

[0018] Convergence Method

[0019] The convergence method of the model after aggregation focuses on learning weighted average and gradient descent methods. In the process of model aggregation, the load is considered. The combined algorithm of the prediction model combines the specific goals of model aggregation and the limitations of computing resources to achieve the optimal convergence result. For example, for weighted average convergence, let the parameters of the global model be θ, and the model parameters uploaded by client i in the tth iteration be Its weight is Then the update formula of the global model is Where N is the number of clients participating in the aggregation.

[0020] In the federated learning privacy data protection method:

[0021] For differential privacy protection technology, when adding noise e (€ obeys a certain probability distribution) to the original data x or calculation results, such as when calculating the gradient, the original gradient is g, and the gradient after adding noise is g'=g+€, and similar noise addition principles should also be followed in other calculation processes involving privacy protection;

[0022] For homomorphic encryption technology, it should be satisfied that for plaintext data m1 and m2, the encryption function is E, the homomorphic encryption operation E(m1)⊙E(m2)=E(m1⊙m2) (⊙ represents a certain operation), and other homomorphic encryption operation rules and encryption functions should also be within the scope of protection of this claim;

[0023] For multi-party computing privacy protection technology, there are several participants, whose data are x1, x2, ..., xn. Each participant calculates f(xi), and the results are jointly input into the multi-party computing platform to obtain the final result F=f(f1(x1), f2(x2), ..., fn(xn)), and other equivalent multi-party computing functions and calculation processes should also be within the scope of protection of this claim.

[0024] The beneficial effects of the present invention are: good data privacy protection

[0025] Advantages of differential privacy protection technology By adding noise to the original data or training model output, differential privacy protection technology can effectively prevent the background center from obtaining the original private data through data back-calculation. This method greatly protects the privacy of data from all parties without affecting the overall training effect of the model. For example, even in the face of complex hacker attacks or malicious data acquisition attempts, due to the added noise interference, it is difficult for attackers to accurately back-calculate the original sensitive power data from the model parameters and gradient values, such as the specific power load data and power consumption time patterns of each client (different grid companies or individual users).

[0026] Effect of Homomorphic Encryption Technology Homomorphic encryption technology allows model training to be performed directly on encrypted data, and the training effect is close to that of plaintext training. This means that the data of all parties remains encrypted throughout the training process of the load forecasting model of federated learning. Even if the data is stolen during transmission or maliciously obtained on the server side, only the encrypted ciphertext data is obtained, and the original basic power data cannot be restored. This provides a strong guarantee for the privacy protection of power data, allowing all parties to safely participate in the training of federated learning models without worrying about the risk of data leakage.

[0027] The significance of multi-party computing privacy protection technology Multi-party computing privacy protection technology allows multiple participants to jointly calculate the agreed method and obtain the final result without sharing their original data. This feature is particularly important in the power load forecasting scenario, because different provincial grid companies or individual users may have their own sensitive power data, such as the peak and valley patterns of power consumption in a specific area, the power consumption habits of a specific user group, etc. Through multi-party computing privacy protection technology, all parties can protect their own privacy data while jointly providing data support for the training of the load forecasting model, thereby achieving a balance between data privacy protection and data value mining.

[0028] High load forecasting accuracy

[0029] Improved accuracy of local model training In the process of federated learning multi-party sample local model training, the present invention can select a more suitable federated learning method (horizontal, vertical or transfer learning mode) by in-depth research on the data characteristics of local clients participating in model training, including the extraction and analysis of data identification, signs and features. This selection of precise learning mode based on data characteristics can better utilize the distribution characteristics of local data, making local model training more accurate. For example, for data from different network provincial companies with similar load characteristics but different sample numbers, selecting a horizontal federated learning mode can make full use of the data samples of all parties and improve the local model's ability to fit local power load data, thereby providing a more accurate local model foundation for the construction of a global model.

[0030] The optimization effect of the model aggregation strategy The federated learning model aggregation method specially designed for the load forecasting scenario takes into account the quality of multi-party data samples, not just the number of samples. By studying the prediction result analysis and trend detection of the global model, and adopting model combination strategies (including randomly extracting data subsets to train independent models and voting or weighted average aggregation, etc.), the prediction performance and generalization ability of the model can be improved. Compared with the traditional method of determining the aggregation weight based only on the number of samples, the model aggregation strategy of the present invention can more reasonably integrate the advantages of each local model, thereby improving the accuracy of the global model in load forecasting. For example, when dealing with the differences in the prediction contributions of different client data to different time periods (such as peak load periods and valley load periods), different weights can be assigned according to the actual prediction contributions, so that the load forecast of the global model in each time period is more accurate.

[0031] Convergence technology guarantees accuracy During the model aggregation process, the convergence technologies such as weighted average and gradient descent are studied, combined with the specific goals of model aggregation and the limitations of computing resources to achieve the best convergence results. This optimized convergence technology can ensure that the model converges stably to an ideal state during the aggregation process, avoiding model deviations caused by improper convergence. Stable and accurate convergence results enable the global model to more accurately reflect the actual power load situation during load forecasting, and provide more reliable results for both short-term load fluctuations and long-term load trend forecasts.

[0032] High model training efficiency

[0033] Optimizing the efficiency of local model training In the training of multi-party sample local models in federated learning, selecting appropriate learning modes, topological structures, model structures and hyperparameters according to data characteristics can improve the efficiency of local model training. For example, for local power load data with specific data structures and distribution characteristics, selecting appropriate model structures (such as simple linear regression models or specific neural network structures) and hyperparameters (such as learning rates, etc.) can reduce unnecessary consumption of computing resources and speed up the training of local models. At the same time, the selection of reasonable topological structures (such as centralized learning modes, distributed learning methods or hierarchical learning methods) can optimize the data interaction and model update process between the local client and the distribution end, further improving the efficiency of local model training.

[0034] Improvement of model aggregation efficiency The model aggregation strategy and convergence technology studied in the present invention help to improve the efficiency of model aggregation. By adopting an effective model combination strategy (such as training independent models based on data subsets and performing rapid aggregation), the computational complexity in the model aggregation process can be reduced. At the same time, optimized convergence techniques (such as weighted average and gradient descent methods) can converge the model faster while ensuring model accuracy, reducing the number of iterations and computing resources required for model aggregation. This enables the global model to be built and updated more quickly, improves the training efficiency of the entire federated learning load forecasting model, and thus can provide support for load forecasting of the power distribution network in a more timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 : Module diagram of the present invention;

[0036] Figure 2 : Diagram of the training process of the federated learning model of the present invention;

[0037] Figure 3 : Aggregation diagram of the global model of federated learning of the present invention. DETAILED DESCRIPTION

[0038] A method for achieving load forecasting based on federated learning of multi-party samples. With the large-scale application of power distribution network big data models, a large amount of basic power data is needed to provide basic support for load forecasting model training. However, due to constraints on data privacy and security, multiple data sources (such as different provincial grid companies, individual users) cannot directly exchange basic power data. This problem needs to be solved through an AI machine learning model training solution that can protect the privacy data of all parties.

[0039] In order to solve the above problems, this project chose a federated learning model training solution that supports the distributed learning paradigm. Federated learning can enable multiple clients to collaborate under the coordination of the server to solve the data privacy and security issues of machine learning. The data of each client is always stored in the local device, and each client can complete the model training by simply exchanging the locally learned model.

[0040] Federated learning multi-party sample local model training

[0041] First, we study the feature analysis of the data that the local client uses to participate in model training, study the characteristics of the samples, and extract the identifiers, signs, and features of the data. According to the characteristics of the data samples, we study the local data overlap method used by the local device and select the federated learning method of the device. According to the analysis of the local client data samples, we can choose the horizontal or vertical learning mode, or the federated transfer learning mode according to the dimension of the data characteristics. Finally, based on the research content of the previous part, we design the topology used between the local model and the server, and study whether the topology adopts a centralized learning mode, a distributed learning mode, or a hierarchical learning mode.

[0042] Federated Learning Model Aggregation

[0043] Model aggregation of federated learning is a key step in federated learning technology, which involves merging model updates of multiple local clients into a global model. First, the federated learning model gradient is studied. In machine learning, a neural network model can be composed of one or more functions. A function may have one or more parameters that require the study of the optimal solution of the function parameters. The client trains its own data locally, and uploads the model parameters and gradients after training. Secondly, the distribution end models the modeling method of local model gradient information, and aggregates the models updated across different clients to generate a single global model. The simple average method, weighted average method, and federated average method are studied. Finally, the convergence technology and strategy of the global model are studied, and the model gradients and parameters collected from each client are stably converged to an ideal state. This topic mainly studies the model weighting strategy and method of multiple rounds of data iteration, updates the model weights according to the weights of different clients, and model compression technology to reduce the bandwidth requirements during transmission.

[0044] Federated Learning Privacy Data Protection

[0045] The main research is on the cryptographic protection technology of private data in federated learning. First, the protection technology combined with differential privacy is studied. Differential privacy mainly protects the original data by adding a certain amount of noise to the original data. In order to prevent the background center from obtaining the original private data through data inversion through model parameters and gradient values, the differential privacy method is used to introduce noise into the calculation results to hide the original sensitive information of individuals. Secondly, the homomorphic encryption technology is studied. Homomorphic encryption technology uses a homomorphism to represent the characteristics in cryptographic operations, that is, there is a rule of synchronous transformation between plaintext and password, and finally achieves the use of ciphertext for model training, and the training effect is close to the plaintext training effect. In this way, even if the original training data is obtained through data inversion, it is also encrypted ciphertext, and the original data cannot be obtained. Finally, the multi-party privacy computing technology is studied, that is, multi-party computing technology. This technology allows multiple participants to jointly calculate an agreed method without sharing their original data, and input their respective calculation results into the multi-party computing platform to obtain the final result. In this way, the common calculation results of multiple participants are obtained, and one party will not obtain the private data of other participants.

[0046] Embodiment 1: Federated learning multi-party sample local model training implements data collection and preprocessing: Each local client collects its own basic power data, and performs preprocessing operations such as cleaning and normalization on the data to ensure data quality.

[0047] Feature analysis and learning mode determination: According to the above data feature analysis technology, analyze the characteristics, identification and signs of local data in detail. According to the analysis results, select the appropriate federated learning mode (horizontal, vertical or transfer learning mode). For example, if the analysis finds that the data of different network provincial companies are similar in load feature dimension but with different sample numbers, determine to use the horizontal federated learning mode.

[0048] Training process: After determining the learning mode and topology, the local client starts training. Training is performed with the selected model structure (such as the simple linear regression model mentioned above or a more complex neural network model) and hyperparameters, and calculations are performed according to the corresponding algorithm formula (such as the gradient descent method parameter update formula). During the training process, the global model broadcast by the distribution end is received and corrected, and the intermediate results (such as gradients and parameters) calculated are uploaded to the distribution end.

[0049] Federated learning model aggregation implements aggregation model construction: Build an aggregation model based on the load forecasting scenario and analyze the impact of different client data on load forecasting. For example, through historical data mining, the contribution weights of different clients to load forecasting in different time periods (peak, trough, etc.)

[0050] Aggregation strategy application: Select a model combination strategy based on actual conditions. If you select a weighted average aggregation strategy, determine the weight of each model. For example, assign weights based on the performance of the model on different test data sets, with models with better performance having larger weights. Perform model aggregation calculations according to the selected aggregation strategy formula (such as the weighted average formula).

[0051] Convergence technology implementation: In the process of model aggregation, convergence techniques such as weighted average or gradient descent are used. According to the computing resources and specific model aggregation goals, convergence parameters such as learning rate are adjusted. According to the convergence formula (such as the global model update formula), iterative calculations are performed until the model converges to the ideal state.

[0052] Federated learning privacy data protection technology implements differential privacy protection: Determine the method and parameters for adding noise during local client data processing and model training. For example, determine the scale parameters of Laplace noise based on the sensitivity and privacy requirements of the data. Add noise according to differential privacy technology during the calculation of gradients, model parameters, etc. to ensure that the privacy of the original data can be protected even if the data is maliciously acquired.

[0053] Homomorphic encryption protection: Select an appropriate homomorphic encryption algorithm to encrypt the original data of the local client. Perform model training operations on the encrypted data to ensure that the encrypted calculation results are functionally equivalent to the plaintext calculation results. For example, for the multiplication operation of power load data, the encrypted data meets the homomorphic property under the multiplication homomorphic encryption algorithm, ensuring data privacy during the training process.

[0054] Multi-party computing privacy protection: When multiple participants (such as different grid companies or user groups) need to jointly calculate functions related to load forecasting (such as calculating the statistical characteristics of the overall load), each participant calculates part of the results locally according to the multi-party computing privacy protection technology, and then inputs the results into the multi-party computing platform. The platform calculates the final result according to the predetermined calculation rules. In this process, the data privacy of each participant is protected.

Claims

1. A federated learning load forecasting method based on multi-party samples, characterized by: S1: Federated learning multi-party sample local model training: Perform in-depth feature analysis on the data participating in the model training of the local client, including extracting the features, identifiers and signs of the data to determine the distribution characteristics of the data samples, and then select the appropriate federated learning mode from horizontal federated learning, vertical federated learning and federated transfer learning modes based on these characteristics; Determine the training method, process, learning requirements, model deployment method and development method of federated learning according to the selected federated learning model, and determine the adopted topology structure according to the requirements and model of federated learning, where the topology structure includes but is not limited to centralized learning mode, distributed learning mode or hierarchical learning mode; During the training process, multiple candidate global models can be obtained by using different model structures and hyperparameters, and the distribution end has the right to select local clients to participate in model updates. The distribution end broadcasts the global model to the selected clients, and the clients train based on local data and upload intermediate results. The end aggregates and updates the global model based on the results uploaded by the clients. S2: Federated learning model aggregation: An aggregation model is designed specifically for load forecasting scenarios. The construction of this aggregation model requires analysis and trend detection of the forecast results of the global model; Model combination strategies are used for model aggregation, including but not limited to randomly extracting data subsets from the original data, training independent models on each subset, and then aggregating by voting or averaging or according to model weights; Study the convergence technology of the aggregated model. This convergence technology needs to combine the model aggregation goals and computing resource constraints, and use weighted average or gradient descent or other equivalent methods to achieve the optimal convergence result; S3: Federated learning privacy data protection: Differential privacy protection technology is used to protect local data by adding noise to the data or training model output to prevent the original private data from being obtained through reverse calculation. The noise addition method should be determined according to the characteristics of the data and the privacy protection requirements. Using homomorphic encryption, the technology allows model training to be performed directly on encrypted data, and the training effect is similar to that of training directly on the original data, ensuring that even if the data is obtained during the entire training process, it cannot be restored to the original data; Multi-party computing privacy protection technology is used, which allows multiple participants to jointly calculate the agreed function results without sharing their original data, so that all parties can participate in model training while protecting their own privacy data.

2. The method for predicting federated learning load based on multi-party samples according to claim 1, characterized in that: In federated learning multi-party sample local model training: When the local data of local client i is represented as {(xi1,yi1),(xi2,yi2),…,(xin,yin;)}, for the simple linear regression model y=θ0+θ1x, the loss function of local client i is And the update formula of θ0 is The update formula of θ1 is Where α is the learning rate, and is not limited to simple linear regression models. Other model structures should also have corresponding loss function definitions and parameter update formulas.

3. The method for predicting federated learning load based on multi-party samples according to claim 1, characterized in that: In federated learning model aggregation: If the prediction results of m independent models are aggregated using simple average The aggregation result is If weighted average aggregation is used, assuming the weight of model k is wk, the aggregation result is In the weighted average convergence process, let the parameter of the global model be θ, and the model parameter uploaded by client i in the tth iteration be Its weight is Then the update formula of the global model is Where N is the number of clients participating in the aggregation, and is not limited to the above aggregation and convergence formulas. Other equivalent aggregation and convergence methods should also be within the protection scope of this claim.

4. The method for predicting load of federated learning based on multi-party samples according to claim 1, characterized in that: In the federated learning privacy data protection method: For differential privacy protection technology, when adding noise e (€ obeys a certain probability distribution) to the original data x or calculation results, such as when calculating the gradient, the original gradient is g, and the gradient after adding noise is g'=g+€, and similar noise addition principles should also be followed in other calculation processes involving privacy protection; For homomorphic encryption technology, it should be satisfied that for plaintext data m1 and m2, the encryption function is E, the homomorphic encryption operation E(m1)⊙E(m2)=E(m1⊙m2) (⊙ represents a certain operation), and other homomorphic encryption operation rules and encryption functions should also be within the scope of protection of this claim; For multi-party computing privacy protection technology, there are several participants, whose data are x1, x2, ..., xn. Each participant calculates f(xi), and the results are jointly input into the multi-party computing platform to obtain the final result F=f(f1(x1), f2(x2), ..., fn(xn)), and other equivalent multi-party computing functions and calculation processes should also be within the scope of protection of this claim.

5. The method for predicting federated learning load based on multi-party samples according to claim 1, characterized in that: This method should be able to adapt to different types of power load data, including but not limited to large-scale power data from different grid companies and small-scale power data from individual users. When new data sources are added, such as data from emerging distributed energy resources or new electricity users, they can be easily integrated into the federated learning load forecasting system without major modifications to the overall framework, so as to reflect the applicability and scalability of this method.

6. The method for predicting load of federated learning based on multi-party samples according to claim 1, characterized in that: In the process of federated learning model aggregation, the model weight determination method adopted by the model combination strategy should be determined based on the model's performance on different test data sets, its contribution to load forecasting in different time periods, or other factors related to load forecasting accuracy, so as to ensure the rationality and accuracy of model aggregation.

7. The method for predicting federated learning load based on multi-party samples according to claim 1, characterized in that: In the local model training of multi-party samples in federated learning, factors such as the number of local clients, data distribution, computing resources, and communication costs should be considered when selecting the topology structure to optimize the local model training efficiency and the overall federated learning effect.

8. The method for predicting federated learning load based on multi-party samples according to claim 1, characterized in that: In the federated learning privacy data protection technology, the noise scale parameters in the differential privacy protection technology should be dynamically adjusted according to the sensitivity of the data, the privacy protection requirements, and the impact on the model accuracy, so as to minimize the impact on the accuracy of the load forecasting model while ensuring data privacy.

Images