Federal learning optimization method for trusted sandbox extension

Abstract

The invention provides a federated learning optimization method for trusted sandbox expansion, and the method comprises the steps: carrying out the training of a global model in a sandbox, and obtaining a local model updating parameter; the global model is used for predicting power business; sending the local model updating parameters to an aggregation node; and the aggregation node is used for updating parameters based on the local model and determining the updated global model. According to the method provided by the embodiment of the invention, the security of federal learning related to power business prediction is effectively ensured.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a federated learning optimization method for trusted sandbox extension. Background Art

[0002] As a new branch of machine learning technology, federated learning can efficiently conduct joint modeling and model training among multiple users without leaving the local machine, thus fully releasing the potential value of data. Therefore, it has brought new development momentum to various industries in recent years, and has broad application prospects in fields such as industrial Internet, smart cities, electricity, and retail.

[0003] In the related technologies, traditional federated learning related to power business forecasting ignores the security issues of data and models during training and transmission. Therefore, how to prevent the destruction of malicious programs and the leakage of data and improve the security of federated learning related to power business forecasting are technical problems that technical personnel in this field urgently need to solve. Summary of the invention

[0004] The present invention provides a federated learning optimization method for trusted sandbox extension, which puts the model training and parameter transfer processes of federated learning tasks related to power business forecasting into a computing sandbox. That is, while ensuring the credibility of the computing and transmission processes, the client can send local model update parameters to an aggregation node, and aggregate the local model update parameters sent by multiple clients through the aggregation node, so that the updated global model can be accurately obtained, effectively ensuring the security of federated learning related to power business forecasting.

[0005] In a first aspect, the present invention provides a trusted sandbox-extended federated learning optimization method, which is applied to a client participating in a federated learning task, comprising: The global model is trained in the sandbox to obtain local model update parameters; the global model is used for prediction of power business; The local model update parameters are sent to an aggregation node; the aggregation node is used to determine an updated global model based on the local model update parameters.

[0006] According to a trusted sandbox extended federated learning optimization method provided by the present invention, after sending the local model update parameters to the aggregation node, the method further includes: The local model update parameters are hashed to obtain a first hash result, and the result is uploaded to the blockchain; the first hash result on the blockchain is used by the aggregation node to determine whether to discard the received local model update parameters.

[0007] According to a trusted sandbox extended federated learning optimization method provided by the present invention, after sending the local model update parameters to the aggregation node, the method further includes: Receiving a test gradient sent by the aggregation node; the test gradient is constructed based on a deep deterministic strategy; According to the test gradient, a response gradient is determined and sent to the aggregation node; the direction deviation and amplitude deviation between the test gradient and the response gradient are used by the aggregation node to identify whether the client is a malicious node.

[0008] In a second aspect, the present invention provides a trusted sandbox-extended federated learning optimization method, which is applied to an aggregation node participating in a federated learning task, comprising: Receiving local model update parameters sent by multiple clients; the local model update parameters are obtained by the clients training the global model in the sandbox; the global model is used for prediction of power business; The local model update parameters sent by the multiple clients are aggregated to determine an updated global model.

[0009] According to a trusted sandbox-extended federated learning optimization method provided by the present invention, before aggregating the local model update parameters sent by the multiple clients, the method further includes: Obtaining a first hash result uploaded by the client from the blockchain; the first hash result is obtained by the client performing hash processing on the local model update parameter; Performing hash processing on the local model update parameter sent by the client to obtain a second hash result; Determine whether to discard the local model update parameters sent by the client according to the first hash result and the second hash result.

[0010] According to a trusted sandbox-extended federated learning optimization method provided by the present invention, before aggregating the local model update parameters sent by the multiple clients, the method further includes: Sending a test gradient to the client; the test gradient is constructed based on a deep deterministic strategy; According to the direction deviation and amplitude deviation between the test gradient and the response gradient, whether the client is a malicious node is identified; the response gradient is determined by the client based on the test gradient.

[0011] The present invention also provides a client, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the federated learning optimization method for the trusted sandbox extension as described in any one of the first aspects above is implemented.

[0012] The present invention also provides an aggregation node, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the federated learning optimization method for the trusted sandbox extension as described in any one of the second aspects above is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a federated learning optimization method for a trusted sandbox extension as described in any one of the above.

[0014] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned federated learning optimization methods for trusted sandbox extensions.

[0015] The trusted sandbox extended federated learning optimization method provided by the present invention places the model training and parameter transfer processes of the federated learning tasks related to power business forecasting into the computing sandbox. That is, while ensuring that the computing and transmission processes are trustworthy, the client can send the local model update parameters to the aggregation node, and aggregate the local model update parameters sent by multiple clients through the aggregation node, so that the updated global model can be accurately obtained, effectively ensuring the security of the federated learning related to power business forecasting. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 This is one of the schematic diagrams of the trusted sandbox extended federated learning optimization method provided by the present invention.

[0018] Figure 2 This is the second schematic diagram of the trusted sandbox extended federated learning optimization method provided by the present invention.

[0019] Figure 3 This is the third schematic diagram of the trusted sandbox extended federated learning optimization method provided by the present invention.

[0020] Figure 4 This is the fourth schematic diagram of the trusted sandbox extended federated learning optimization method provided by the present invention.

[0021] Figure 5 This is the fifth schematic diagram of the trusted sandbox extended federated learning optimization method provided by the present invention.

[0022] Figure 6 It is one of the schematic diagrams of the test comparison results provided by the present invention.

[0023] Figure 7 This is the second schematic diagram of the test comparison results provided by the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] Combine the following Figure 1-Figure 7 The present invention describes a federated learning optimization method for trusted sandbox extension.

[0026] In order to facilitate a clearer understanding of the technical solutions of the embodiments of the present application, some technical contents related to the embodiments of the present application are first introduced.

[0027] As a new branch of machine learning technology, federated learning can meet the requirements of efficient joint modeling and model training among multiple users without leaving the local machine, thus fully releasing the potential value of data. Therefore, it has brought new development momentum to various industries in recent years, and has broad application prospects in fields such as industrial Internet, smart cities, electricity, and retail. However, traditional federated learning ignores the security issues of data, models, and results during training and transmission. An important security threat faced by federated learning is poisoning attacks. For poisoning attacks, according to the attack method, poisoning attacks can be divided into data poisoning attacks and model poisoning attacks. Data poisoning attacks are attacks in which the attacker pollutes the training data set so that the model learns the wrong correspondence during the training process, thereby reducing the accuracy of the model. Model poisoning attacks are achieved by controlling the training phase of the client and replacing or tampering with the uploaded local model or gradient, thereby manipulating the global model. During the federated learning calculation process, malicious clients may attack normal clients to tamper with their gradients or disguise themselves as normal clients to submit custom gradients to cause errors, making the final model invalid.

[0028] Figure 1 This is one of the flow charts of the trusted sandbox extended federated learning optimization method provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 101: Train the global model in the sandbox to obtain local model update parameters; the global model is used for prediction of power business.

[0029] Specifically, in the embodiment of the present application, the client can create a trusted computing sandbox, and put the model training and parameter transfer process of the federated learning task related to the power business forecast into the computing sandbox, that is, the global model related to the power business forecast is trained in the sandbox to obtain the local model update parameters, so as to ensure the trustworthiness of the computing and transmission process. Optionally, the global model is used for the forecast of power business, such as load forecasting, credit assessment, power consumption forecasting, query retrieval, etc.

[0030] For example, the technical solution in the embodiment of the present application is as follows Figure 2 As shown, it is composed of multiple Internet of Things devices participating in federated learning. The Internet of Things devices have limited computing and communication resources. Assume that there are M Internet of Things devices, represented by M = {1,..,m,..,M}, and define the Internet of Things device that requests the task as m'. Assume that the set of Internet of Things devices M has a corresponding training data set D = {D1,..,Dm,..,DM}, and the computing task is to train the task model according to the data set D and return it to the target model required by the task requester. Optionally, in this architecture, the trusted computing sandbox aims to create an encrypted space as a secure area for model training and aggregation to protect the code and data therein from being tampered with or leaked by other processes. Optionally, Intel SGX technology can be used to create a protected memory area isolated from any other code in the system according to a series of instructions, called Enclave. Strict access control and encryption operations are implemented in this protected space to provide protection for program data confidentiality and code integrity, thereby achieving the effect of ensuring the trustworthiness of the computing and transmission processes.

[0031] Step 102: Send the local model update parameters to the aggregation node; the aggregation node is used to determine the updated global model based on the local model update parameters.

[0032] Specifically, by placing the model training and parameter transfer processes of federated learning tasks related to power business forecasting into the computing sandbox, that is, while ensuring the credibility of the computing and transmission processes, the client can send local model update parameters to the aggregation node, and aggregate the local model update parameters sent by multiple clients through the aggregation node, so that the updated global model can be accurately obtained, effectively ensuring the security of federated learning related to power business forecasting.

[0033] Alternatively, if Figure 3As shown, the technical solution of the present application is applicable to the power business of the new power system, such as load forecasting, credit assessment, power consumption forecasting, and query retrieval tasks in demand-side response, source-network interaction, and intelligent scheduling. By building a trusted sandbox-extended federated learning architecture for such businesses, the problem of data security sharing between different subject systems is solved, and a solution is provided for the privacy and security issues of large-scale data aggregation scenarios. Its architecture can be divided into perception layer, network layer, platform layer, and application layer. The perception layer includes a variety of edge smart devices in the smart grid, and is used to obtain power business data. Through resource sharing such as communication and computing, notification perception access and edge intelligent processing are realized at the source end, mainly referring to charging piles, electric meters, electric vehicles, etc. The network layer refers to various network devices, such as routers and base stations. The platform layer refers to the system platform for different users on different businesses, mainly referring to the Internet of Vehicles platform, charging station platform, power grid system, regulatory agency, etc. The trusted sandbox-extended federated learning architecture realizes the trusted data interaction between various platforms at this layer, and uses deep learning models to realize various businesses. The application layer includes various new power businesses. This mainly refers to load forecasting, credit assessment, power consumption forecasting, query retrieval, etc.

[0034] For example, the federated learning architecture extended by the trusted sandbox in the embodiment of the present application can be used for load forecasting tasks at electric vehicle charging stations. With the surge in electric vehicles, the energy demand in the power market has surged. When a large number of electric vehicles are charged at the same time, energy transmission will be blocked. In order to solve this problem, energy can be reserved in advance at the charging station to meet the real-time needs of electric vehicles. Optionally, the historical records of energy consumption and charging duration of connected electric vehicles can be captured as a data set, and then stored on the charging station power supplier (remote server) for training. However, this method requires the electric vehicles connected to the charging station and the power supplier to share charging data, which may encounter privacy and security issues. The method of the embodiment of the present application can solve these problems well. In the sandbox, the energy demand of the charging station is predicted based on federated learning. The charging station records the relevant information of the local charging car, but does not share the real data set with the central server. Only the model trained in the local data set of the charging station is shared. The central server executes the model aggregation algorithm and trains the prediction model after several iterations. The energy demand of the charging station is predicted by the prediction model, and the decision on whether to pre-store energy and how much energy to store in the charging station is made according to the prediction results.

[0035] The method of the above embodiment places the model training and parameter transfer processes of the federated learning tasks related to power business forecasting into a computing sandbox. That is, while ensuring the reliability of the computing and transmission processes, the client can send the local model update parameters to the aggregation node, and aggregate the local model update parameters sent by multiple clients through the aggregation node, so that the updated global model can be accurately obtained, effectively ensuring the security of federated learning related to power business forecasting.

[0036] In one embodiment, after sending the local model update parameters to the aggregation node, the method further includes: The local model update parameters are hashed to obtain a first hash result, which is uploaded to the blockchain; the first hash result on the blockchain is used by the aggregation node to determine whether to discard the received local model update parameters.

[0037] Specifically, in an embodiment of the present application, after the client sends the local model update parameters to the aggregation node, the local model update parameters are further hashed to obtain a first hash result, and uploaded to the blockchain. After receiving the local model update parameters sent by the client, the aggregation node can hash the local model update parameters sent by the client to obtain a second hash result; and compare the first hash result and the second hash result. Optionally, when the first hash result on the blockchain and the second hash result calculated by the aggregation node are inconsistent, indicating that the federated learning related to the power business forecast has been attacked, it is necessary to discard the local model update parameters sent by the client, thereby improving the security of the federated learning related to the power business forecast.

[0038] For example, Figure 4 As shown, first assume that in the tth round of federated learning training related to power business forecasting, the i-th training node performs local training in the trusted computing sandbox and obtains local model updates Then the training node gets the local model update After that, it is hashed and its hash value is Upload to the blockchain. The training node updates the local model Transmitted to the aggregation node through the secure channel between sandboxes. Finally, when the aggregation node obtains the local model update of all training nodes After that, the aggregation node downloads the updated hash values ​​of the local models of all training nodes from the blockchain. . Furthermore, the aggregation node hashes the local model updates it obtains, and compares the hash values ​​calculated by each training node with the downloaded hash values ​​one by one. If the comparison is consistent, the next aggregation operation is carried out normally. If the comparison is inconsistent, it means that the local model update has been tampered with by the attacker, so the local model update results of the training node in this round need to be discarded. That is, in the embodiment of the present application, by combining the sandbox and the blockchain, the integrity and authenticity of the data during the transmission process are effectively guaranteed to prevent it from being maliciously tampered with by attackers, thereby achieving reliable verification of model updates and improving the security of federated learning related to power business forecasting.

[0039] In the method of the above embodiment, after the client sends the local model update parameters to the aggregation node, the local model update parameters are further hashed to obtain a first hash result, and uploaded to the blockchain. After receiving the local model update parameters sent by the client, the aggregation node can hash the local model update parameters sent by the client to obtain a second hash result; and compare the obtained first hash result with the second hash result to determine whether it is necessary to discard the local model update parameters sent by the client. That is, in the embodiment of the present application, by combining the sandbox and the blockchain, the integrity and authenticity of the data during the transmission process are effectively ensured to prevent it from being maliciously tampered with by attackers, thereby achieving trusted verification of model updates and improving the security of federated learning related to power business forecasting.

[0040] In one embodiment, after sending the local model update parameters to the aggregation node, the method further includes: Receive the test gradient sent by the aggregation node; the test gradient is built based on the deep deterministic strategy; According to the test gradient, the response gradient is determined and sent to the aggregation node; the direction deviation and amplitude deviation between the test gradient and the response gradient are used by the aggregation node to identify whether the client is a malicious node.

[0041] Specifically, in order to resist poisoning attacks, the implementation of this application detects and identifies training nodes periodically, removes malicious nodes, and improves the security of the federated learning platform related to power business forecasting. Assume that in the federated learning framework with trusted sandbox extension, there are a total of n training nodes, which are represented as , which contains x malicious nodes and y normal nodes, and Based on the aggregation rules of federated learning, the training results uploaded by the training nodes are weighted averaged. Optionally, a weight list of each training node needs to be maintained. , and use it as the aggregation rule to aggregate the global model. When federated learning is initialized, the initial weight of each training node is set to 1.

[0042] Optionally, in order not to excessively affect the federated learning process and cause excessive increase in the delay of federated learning training, the training mode is divided into two modes, namely detection mode and normal mode. Assume that in a certain iteration round f of federated learning, when it is in normal mode, the gradients of each training node are aggregated using formula (1) according to the aggregation rule it maintains: (1) in, is the global model update for the fth iteration of federated learning, The local model of the i-th training node of the f-th iteration of federated learning is updated. Then the global model of federated learning is updated, and the updated global model is: (2) in, is the global model update for the fth iteration of federated learning, is the global model of the fth iteration of federated learning, is the global model of the f-1th iteration of federated learning, is the learning rate for federated learning.

[0043] Assuming that the federated learning is in detection mode at a certain iteration round t, the DDPG agent first learns the test gradient rule based on the current environment of the federated learning system, and uses this rule and the local model update of the training node in the previous iteration round to construct the corresponding test gradient. The constructed test gradient is then sent to the corresponding training node for local training as the global model update of the new iteration. Then, by observing the reaction of the training node, that is, observing the difference between the local model update returned by it and the test gradient, poisoning detection is performed to determine whether it is a malicious node. Finally, based on the result of the judgment, the aggregate rule weight list of federated learning is The workflow is mainly divided into the following four steps: building test gradients, obtaining training node responses, poisoning detection, building new aggregation strategies, and updating the agent.

[0044] Alternatively, if Figure 5 As shown, the first step is to construct the test gradient and set the state space in reinforcement learning to ,in is the weight value of the i-th training node in the t-th round of federated learning. The DDPG agent outputs the action space based on the state space , the action output by the DDPG agent is defined as the rule for constructing the test gradient, where Indicates the size change. represents the change in direction, where Therefore, the test gradient constructed for the i-th training node is , Satisfies the test gradient construction formula (3).

[0045] (3) in, For the The training node is Local model update of the wheel. To construct the intermediate amount produced during the test gradient process, Represents a vector Norm.

[0046] Then, the test gradient constructed in the action space is As the global model update for round t of federated learning The training nodes are sent to the corresponding training nodes for testing to obtain the response gradient of the training nodes. For the i-th training node, the training node updates the local model in the following way: (4) in, Represents a training node The learning rate. Indicates that the training node is Optionally, the training node trains the model on a local dataset. , and get the local model update , that is, the response gradient of the training node is obtained.

[0047] Optionally, upon receiving the response gradient from the training node, the local model update After that, the direction and amplitude characteristics of the gradient change before and after the training node are detected to determine whether it is a malicious node. Malicious nodes usually manipulate the uploaded gradient of the node to make the global gradient deviate from the original direction. The gradient direction of a benign node usually points to the distribution of its local data set, while the gradient direction of a malicious node is usually random or contrary to the global gradient direction. Therefore, it is necessary to measure the direction change of the gradient before and after the test. Optionally, cosine similarity is used in the implementation of this application. To measure the degree of change in direction, the formula is shown in (5): (5) Optionally, the gradient size of the malicious node also dominates the poisoning effect. In order to make the poisoning effect appear as soon as possible, the malicious node often changes the gradient drastically, especially when the gradient is larger than the benign gradient. Therefore, it is necessary to measure the magnitude of the gradient change. The norm is used to measure the size of the gradient, using the ratio of the gradient size before and after the test To measure the magnitude of its change, the formula is shown in (6).

[0048] (6) in, For the The training node is The local model update of the round, that is, the response gradient. The test gradient constructed for the i-th training node at epoch t.

[0049] Optionally, in the embodiment of the present application, the abnormality of the training node is redefined by the degree of deviation in direction and amplitude. The definition of is shown in formula (7): (7) in, Controlling the exception The positive and negative When the value of is negative, it means that the test gradient and response gradient of the training node are in opposite directions, and the change is huge. =1. When the value of is positive, The bigger, The closer the value of is to 1, The smaller. = 1, indicating that the deviation of the gradient magnitude before and after the test is almost negligible. = 0. Similarly, and Control the impact of directional deviation and size deviation on outliers, and adjust to ensure that the magnitude of the two is at the same level so that they have equal impact The size of Some appropriate adjustments are made to the outliers. In summary, the definition of outliers encourages gradients to be updated within a smaller range, thereby avoiding the impact of malicious updates with large changes on federated learning.

[0050] Finally, it is assumed that after each round of detection, a constant baseline value will be deducted from the suspected poisoner in the poisoning detection. Among them, for the i-th training node, the updated weight is for: (8) in, is the weight value updated for the i-th training node after the t-th round of detection. is the weight value of the i-th training node before the t-th round of detection. Indicates the outlier value of the training node in this detection round. Indicates the set constant baseline value. When the weight value of a training node is deducted to less than or equal to 0, it is marked as a malicious node, and the node's gradient is no longer involved in the aggregation process, that is, it is forced to exit the federated learning process. After normalization, a new model weight list is generated .

[0051] Optionally, a new list of model weights can be , aggregate the training nodes again to obtain a new global model, and set the reward of reinforcement learning , The calculation of is as follows: (9) in is the accuracy of the new global model on a clean validation dataset. a and c are used to adjust the order of magnitude and other outliers. After obtaining the reward value, the DDPG agent is updated and the above process is repeated until the reinforcement learning algorithm reaches the iteration termination condition. After that, the new model weight list obtained when the iteration termination condition is reached is The current model weight list is updated as a result of the reinforcement learning based model safety aggregation strategy.

[0052] In the method of the above embodiment, the aggregation node constructs a test gradient and sends it to the client, and then based on the direction deviation and amplitude deviation between the test gradient and the response gradient sent by the client, it can update the weights of each client and accurately identify whether the client is a malicious node, thereby effectively ensuring the integrity and authenticity of the data during the transmission process, realizing trusted verification of the model update, and effectively improving the security of federated learning related to power business forecasting.

[0053] The embodiment of the present application also provides a trusted sandbox-extended federated learning optimization method, which is applied to aggregation nodes participating in federated learning tasks, including: Step 201, receiving local model update parameters sent by multiple clients; the local model update parameters are obtained by the client training the global model in the sandbox; the global model is used for prediction of power business.

[0054] Specifically, in the embodiment of the present application, the aggregation node first receives local model update parameters sent by multiple clients; wherein, the local model update parameters are obtained by the client training the global model in the sandbox, that is, the client trains the global model related to the power business prediction in the sandbox to obtain the local model update parameters, which can effectively ensure the reliability of the calculation and transmission process. Optionally, the global model is used for the prediction of power business, such as load prediction, credit assessment, power consumption prediction, query retrieval, etc.

[0055] Step 202: Aggregate the local model update parameters sent by multiple clients to determine an updated global model.

[0056] Specifically, after the aggregation node receives the local model update parameters sent by multiple clients, it can aggregate the local model update parameters sent by multiple clients, so as to accurately obtain the updated global model while ensuring the reliability of the calculation and transmission process, and effectively ensure the security of federated learning related to power business forecasting.

[0057] The method of the above embodiment places the model training and parameter transfer processes of the federated learning tasks related to power business forecasting into a computing sandbox. That is, while ensuring the reliability of the computing and transmission processes, the aggregation node aggregates the local model update parameters sent by multiple clients, and thus accurately obtains the updated global model, thereby effectively ensuring the security of federated learning related to power business forecasting.

[0058] In one embodiment, before aggregating the local model update parameters sent by multiple clients, the method further includes: Obtain the first hash result uploaded by the client from the blockchain; the first hash result is obtained by the client performing hash processing on the local model update parameters; Performing hash processing on the local model update parameters sent by the client to obtain a second hash result; According to the first hash result and the second hash result, it is determined whether to discard the local model update parameters sent by the client.

[0059] Specifically, in an embodiment of the present application, after receiving the local model update parameters sent by the client, the aggregation node may hash the local model update parameters sent by the client to obtain a second hash result, and compare the first hash result with the second hash result. Optionally, the first hash result is obtained after the client hashes the local model update parameters, and the first hash result is uploaded to the blockchain to improve the security of federated learning. Optionally, when the first hash result on the blockchain is inconsistent with the second hash result calculated by the aggregation node, indicating that the federated learning related to the power business forecast has been attacked, the aggregation node is required to discard the local model update parameters sent by the client, thereby improving the security of the federated learning related to the power business forecast.

[0060] In the method of the above embodiment, after receiving the local model update parameters sent by the client, the aggregation node can hash the local model update parameters sent by the client to obtain a second hash result; and compare the first hash result obtained from the blockchain with the calculated second hash result to determine whether it is necessary to discard the local model update parameters sent by the client. That is, in the embodiment of the present application, by combining the sandbox and the blockchain, the integrity and authenticity of the data during the transmission process are effectively guaranteed to prevent it from being maliciously tampered with by attackers, thereby achieving trusted verification of model updates and improving the security of federated learning related to power business forecasting.

[0061] In one embodiment, a test gradient is sent to a client; the test gradient is constructed based on a deep deterministic strategy; According to the direction deviation and amplitude deviation between the test gradient and the response gradient, whether the client is a malicious node is identified; the response gradient is determined by the client based on the test gradient.

[0062] Specifically, in order to resist poisoning attacks, the embodiments of the present application periodically detect and identify training nodes, remove malicious nodes therein, and improve the security of the federated learning platform related to power business forecasting.

[0063] Optionally, we first construct the test gradient, setting the state space in reinforcement learning to ,in is the weight value of the i-th training node in the t-th round of federated learning. The DDPG agent outputs the action space based on the state space , the action output by the DDPG agent is defined as the rule for constructing the test gradient, where Indicates the size change. represents the change in direction, where Therefore, the test gradient constructed for the i-th training node is , Satisfy the test gradient construction formula.

[0064] in, For the The training node is Local model update of the wheel. To construct the intermediate amount produced during the test gradient process, Represents a vector Norm.

[0065] Then, the test gradient constructed in the action space is As the global model update for round t of federated learning The training nodes are sent to the corresponding training nodes for testing to obtain the response gradient of the training nodes. For the i-th training node, the training node updates the local model in the following way: in, Represents a training node The learning rate. Indicates that the training node is Optionally, the training node trains the model on a local dataset. , and get the local model update , that is, the response gradient of the training node is obtained.

[0066] Optionally, upon receiving the response gradient from the training node, the local model update After that, the direction and amplitude characteristics of the gradient change before and after the training node are detected to determine whether it is a malicious node. Malicious nodes usually manipulate the uploaded gradient of the node to make the global gradient deviate from the original direction. The gradient direction of a benign node usually points to the distribution of its local data set, while the gradient direction of a malicious node is usually random or contrary to the global gradient direction. Therefore, it is necessary to measure the direction change of the gradient before and after the test. Optionally, cosine similarity is used in the implementation of this application. To measure the degree of change in direction.

[0067] Optionally, the gradient size of the malicious node also dominates the poisoning effect. In order to make the poisoning effect appear as soon as possible, the malicious node often changes the gradient drastically, especially when the gradient is larger than the benign gradient. Therefore, it is necessary to measure the magnitude of the gradient change. The norm is used to measure the size of the gradient, using the ratio of the gradient size before and after the test to measure the magnitude of its change.

[0068] Optionally, whether a training node is a malicious node can be determined based on the directional deviation and amplitude deviation between the test gradient and the response gradient and a preset threshold, thereby effectively ensuring the integrity and authenticity of the data during transmission and improving the security of federated learning related to power business forecasting.

[0069] In the method of the above embodiment, the aggregation node constructs a test gradient and sends it to the client, and then based on the direction deviation and amplitude deviation between the test gradient and the response gradient sent by the client, it can update the weights of each client and accurately identify whether the client is a malicious node, thereby effectively ensuring the integrity and authenticity of the data during the transmission process, realizing trusted verification of the model update, and effectively improving the security of federated learning related to power business forecasting.

[0070] Exemplarily, the present application provides a method for optimizing federated learning with a trusted sandbox extension, and the specific process is as follows: (1) First, the client participating in the federated learning task related to power business forecasting must first register as a training node on the blockchain system. Participants submit identity information, summary information describing their own data, data set hash information, and computing resource information, including data set size, computing unit operation rate, storage space size, etc., to facilitate blockchain scheduling and authentication.

[0071] (2) The blockchain system authenticates the client, integrates the resources provided by the client, and establishes a resource pool. The blockchain system maintains a virtual resource status table of all nodes, records the allocation and usage of resources, and facilitates the instantiation of subsequent federated learning tasks.

[0072] (3) A training node initiates a federated learning task request and uploads the relevant federated learning parameters to the blockchain. The blockchain instantiates the federated learning task based on the relevant parameters of the federated learning task and the resource information of the node, sends the global model to the training node, and broadcasts the training request.

[0073] (4) The training node creates a trusted computing sandbox locally, loads the data and training program code into the trusted computing sandbox for training, obtains the local model update of the federated learning task, and transmits it to the aggregation node through the secure channel between the sandboxes.

[0074] (5) After all training nodes have completed training, the aggregation node aggregates the local models of the training nodes to obtain a new global model. Repeat steps 4-5 above, and continue the federated learning training and parameter transfer process in the trusted computing sandbox until the federated learning task iteration conditions are met. Finally, the blockchain sends an iteration termination command, and the node destroys the trusted computing sandbox and releases resources.

[0075] The method of the above embodiment, by combining blockchain technology and sandbox mechanism, ensures the trustworthiness of the federated learning training process related to power business forecasting, while reducing the risk of model poisoning attacks. Moreover, by constructing a suitable test gradient, malicious nodes in the federated learning system can be detected and deleted in time, effectively resisting the impact of data poisoning attacks.

[0076] For example, in a federated learning system related to power business forecasting, a deep convolutional neural network model ResNet18 is used to implement image classification tasks on the CIFAR10 dataset. For each training node in the system, the ratio of the training set to the test set is 9:1.

[0077] Dataset: CIFAR-10 is a small dataset for identifying universal objects. It contains RGB color images of 10 categories: airplanes, cars, birds, cats, deer, dogs, frogs, horses, ships, and trucks. The size of each image in the CIFAR-10 dataset is 32 × 32, and there are 6,000 images in each category. There are a total of 50,000 training images and 10,000 test images in the dataset.

[0078] Experimental environment: In the embodiment of this application, a federated learning framework based on a trusted computing sandbox is used to model the environment involved in resisting poisoning attacks. The open source framework Pytorch is used for federated learning experiments, and the security aggregation strategy in the embodiment of this application is simulated. In addition, the trusted computing sandbox is simulated using a docker container. All experiments are performed on an Ubuntu 20.04.6 LTS 64-bit server, where the CPU is a 128 AMD EPYC 776364-Core Processor, the memory is 52GB, and the number of cores is 64.

[0079] During the experiment, in order to simulate the real environment, unless otherwise specified, it is assumed that there are 20% malicious nodes. When dividing the data set, the hyperparameter of the Dirichlet distribution is set to 0.5. In order to simulate the poisoning attack environment, a poisoning attack is performed every 10 rounds during the training of the federated learning system.

[0080] Optionally, the proposed method is compared with two existing defense measures, Fltrust and RECESS. In order to better observe the impact of poisoning attacks, the common FedAvg aggregation rule is used to conduct federated learning experiments with and without poisoning, and use them as a benchmark to observe the effects of different defense measures. During the experiment, the initial action space of the DDPG agent is set to Regarding the implementation method of constructing the test gradient, for the magnitude change, the gradient is directly scaled, and for the direction change, noise is added to the gradient element, and iterative adjustment is made until the cosine similarity before and after reaches the direction change. The training hyperparameter settings in the experiment are shown in Table 1.

[0081] Table 1

[0082] Accuracy can be used to measure the performance of model training. The higher the accuracy, the better the defense measures. In addition, the effect of the attack can also be measured by the degree of accuracy reduction. For the two classic poisoning methods, label flipping attack (FLIP) and model replacement attack (MR), we evaluated the average accuracy changes of the global model obtained by different iterations of federated learning on the test data set with different defense measures. The experimental results are as follows: Figure 6 , Figure 7 shown.

[0083] It should be noted that the no-attack curve represents the solution of using FedAvg to conduct federated learning experiments in the absence of poisoning attacks. By observing the above figure, it can be seen that, in general, the model accuracy of the solution of the present application is high, and it has advantages in reducing the loss of precision caused by poisoning attacks. In addition, it is closer to the case without poisoning attacks, which means that the defense effect of the solution of the present application is better. In addition, since the present application is set to carry out poisoning attacks every 10 rounds. It can be found from the figure that the accuracy of the model has dropped significantly every 10 rounds. As can be seen from the curve in the figure, the curve of the solution of the present application changes more gently, which shows that the accuracy of the solution of the present application before and after the poisoning attack decreases less, and it has advantages in reducing the loss of precision caused by poisoning attacks.

[0084] The present application also provides a client, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and a federated learning optimization method for implementing a trusted sandbox extension when the processor executes the program.

[0085] The present application also provides an aggregation node, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and a federated learning optimization method for implementing a trusted sandbox extension when the processor executes the program.

[0086] The present application also provides a federated learning system, including a client and an aggregation node.

[0087] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the trusted sandbox extended federated learning optimization method provided by the above methods, and the method includes: training the global model in the sandbox to obtain local model update parameters; the global model is used for predicting power business; sending the local model update parameters to the aggregation node; the aggregation node is used to determine the updated global model based on the local model update parameters; or, receiving local model update parameters sent by multiple clients; the local model update parameters are obtained by the client training the global model in the sandbox; the global model is used for predicting power business; aggregating the local model update parameters sent by multiple clients to determine the updated global model.

[0088] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a federated learning optimization method for executing the trusted sandbox extension provided by the above methods, the method comprising: training a global model in a sandbox to obtain local model update parameters; the global model is used for prediction of power business; sending the local model update parameters to an aggregation node; the aggregation node is used to determine an updated global model based on the local model update parameters; or, receiving local model update parameters sent by multiple clients; the local model update parameters are obtained by the client training the global model in the sandbox; the global model is used for prediction of power business; aggregating the local model update parameters sent by multiple clients to determine an updated global model.

[0089] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0090] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A trusted sandbox extended federated learning optimization method, characterized in that: Applicable to clients participating in federated learning tasks, including: The global model is trained in the sandbox to obtain local model update parameters; the global model is used for prediction of power business; The local model update parameters are sent to an aggregation node; the aggregation node is used to determine an updated global model based on the local model update parameters.

2. The method for optimizing federated learning with trusted sandbox extension according to claim 1, characterized in that: After sending the local model update parameters to the aggregation node, the method further includes: The local model update parameters are hashed to obtain a first hash result, and the result is uploaded to the blockchain; the first hash result on the blockchain is used by the aggregation node to determine whether to discard the received local model update parameters.

3. The method for optimizing federated learning of a trusted sandbox extension according to claim 1 or 2, characterized in that: After sending the local model update parameters to the aggregation node, the method further includes: Receiving a test gradient sent by the aggregation node; the test gradient is constructed based on a deep deterministic strategy; According to the test gradient, a response gradient is determined and sent to the aggregation node; the direction deviation and amplitude deviation between the test gradient and the response gradient are used by the aggregation node to identify whether the client is a malicious node.

4. A trusted sandbox extended federated learning optimization method, characterized in that: Aggregation nodes used to participate in federated learning tasks include: Receiving local model update parameters sent by multiple clients; the local model update parameters are obtained by the clients training the global model in the sandbox; the global model is used for prediction of power business; The local model update parameters sent by the multiple clients are aggregated to determine an updated global model.

5. The method for optimizing federated learning with trusted sandbox extension according to claim 4, characterized in that: Before aggregating the local model update parameters sent by the multiple clients, the method further includes: Obtaining a first hash result uploaded by the client from the blockchain; the first hash result is obtained by the client performing hash processing on the local model update parameter; Performing hash processing on the local model update parameter sent by the client to obtain a second hash result; Determine whether to discard the local model update parameters sent by the client according to the first hash result and the second hash result.

6. The method for optimizing federated learning with trusted sandbox extension according to claim 4 or 5, characterized in that: Before aggregating the local model update parameters sent by the multiple clients, the method further includes: Sending a test gradient to the client; the test gradient is constructed based on a deep deterministic strategy; According to the direction deviation and amplitude deviation between the test gradient and the response gradient, whether the client is a malicious node is identified; the response gradient is determined by the client based on the test gradient.

7. A client, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the federated learning optimization method for the trusted sandbox extension as described in any one of claims 1 to 3 is implemented.

8. An aggregation node, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the federated learning optimization method for the trusted sandbox extension as described in any one of claims 4 to 6 is implemented.

9. A federated learning system, characterized in that: It comprises a client as claimed in claim 7 and an aggregation node as claimed in claim 8.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the federated learning optimization method for the trusted sandbox extension as described in any one of claims 1 to 3, or implements the federated learning optimization method for the trusted sandbox extension as described in any one of claims 4 to 6.

Images