Method for verifying model updates
By verifying model updates using hash functions and public-key cryptography, the problem of insufficient trust and robustness of the coordination device in federated learning systems is solved, achieving accuracy and privacy protection in model updates while reducing computational and communication costs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- INVENTEC PUDONG TECH CORPOARTION
- Filing Date
- 2022-08-30
- Publication Date
- 2026-05-29
AI Technical Summary
In federated learning systems, the reliability and robustness of the coordinating mechanism are insufficient, especially when the coordinating mechanism fails or is malicious, leading to inaccurate model updates and data privacy risks. Existing encryption schemes also have high computational and communication costs.
A lightweight encryption protocol is designed using hash functions and public-key cryptography. The correctness and privacy of model updates are verified through client devices. Homomorphic encryption and secret sharing techniques are used to improve the trustworthiness and robustness of the coordination device.
It improves the credibility and robustness of the coordination mechanism in the federated learning system, ensures the accuracy of model updates and data privacy, and reduces computation and communication costs.
Smart Images
Figure CN117675179B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to federated learning systems, and more particularly to a method for verifying model updates. Background Technology
[0002] When data becomes crucial to ensuring the effectiveness of machine learning models in solving real-world problems, allowing multiple parties with different data to jointly train models becomes more attractive to the public. Federated Learning (FL) has shown increasing success in decentralized, multimodal feature learning. However, due to the distributed nature of FL, security and privacy risks threaten the entire FL architecture. Another problem arises when the coordinating device needs to evaluate the model and release it to consumers. Because the coordinating device cannot access the private data of client devices, representative data is lost when client devices stop collaborating, leading to significantly skewed FL gradient updates and long-term training degradation. The simple approach of memorizing gradients during training is not a suitable solution, as gradients quickly become unrepresentative as iterations progress.
[0003] The coordinator aggregates gradient updates from client devices. Most existing collaborative training (FL) algorithms assume the coordinator is trustworthy and honest, and only apply to settings with a single coordinator. However, requiring client devices to completely trust the coordinator represents a vulnerability on the client side. Furthermore, this assumption is strong when considering data privacy from individual and corporate entities. Research on how to protect the coordinator from corruption or failure is limited. In the former case, the coordinator may not follow the prescribed aggregation procedure, thus producing an incorrect model; in the latter case, the coordinator may disconnect or dropout due to network instability or hardware failure. Coordinator failure (or even simple dropout) can abruptly halt the training process, ultimately disrupting collaborative training.
[0004] On the other hand, while general encryption schemes and mechanisms exist, such as multi-party computation and verifiable computation, they require significant computational and communication costs. Summary of the Invention
[0005] In view of the shortcomings of the prior art described above, this invention proposes a method for verifying model updates to improve the credibility and robustness of the coordination device in a federated learning system. The proposed method for verifying model updates includes several embodiments, some of which employ lightweight encryption protocols.
[0006] A method for verifying model updates according to an embodiment of the present invention is applicable to a federated learning system comprising a coordinating device and multiple client devices. The method includes each client device performing the following steps: receiving a hash function and a general model from the coordinating device; training a client model based on the general model and raw data; calculating the difference between the general model and the client model as an update parameter; sending the update parameter to the coordinating device so that the coordinating device receives multiple update parameters; inputting the update parameter to the hash function to generate a hash value; sending the hash value to the other client devices besides itself, and receiving multiple hash values other than the hash value generated by itself; summing the hash value generated by itself and the received multiple hash values to generate a trust value; receiving an aggregation parameter from the coordinating device, the aggregation parameter being calculated by the coordinating device based on the multiple update parameters; inputting the aggregation parameter to the hash function to generate a value to be verified; and comparing the value to be verified with the trust value, and updating the client model based on the aggregation parameter when the value to be verified equals the trust value.
[0007] A method for verifying model updates according to an embodiment of the present invention is applicable to a federated learning system comprising multiple coordinating devices and multiple client devices. The method includes each client device performing the following steps: receiving a general model from the federated learning system; training a client model based on the general model and raw data; calculating the difference between the general model and the client model; encrypting the difference using a public key to generate an update parameter; sending the update parameter to the multiple coordinating devices so that each coordinating device receives multiple update parameters; receiving multiple aggregation parameters from the multiple coordinating devices, each aggregation parameter being calculated by each coordinating device based on the multiple update parameters; finding a mode among the multiple aggregation parameters; when the number of modes exceeds half the number of the multiple coordinating devices, decrypting based on the mode and a private key to generate a decryption result; and updating the client model based on the decryption result.
[0008] A method for verifying model updates according to an embodiment of the present invention is applicable to a federated learning system comprising multiple coordinating devices and multiple client devices. The method includes each client device performing the following steps: receiving a hash function and a general model from the federated learning system; training a client model based on the general model and raw data; calculating the difference between the general model and the client model; encrypting the difference using a public key to generate an update parameter; sending the update parameter to the multiple coordinating devices, so that each coordinating device receives multiple update parameters; inputting the update parameter to the hash function to generate a hash value; and sending the hash value to the remaining client devices (excluding itself). The system sets and receives multiple hash values other than the hash value it generates; it sums the hash value it generates and the multiple hash values received to generate a trust value; it receives multiple aggregation parameters from the multiple coordination devices, each of which is calculated by the coordination device based on the multiple update parameters; it inputs each aggregation parameter into the hash function to generate multiple values to be verified; it sequentially compares each value to be verified with the trust value; and when a value to be verified that is equal to the trust value is found, it decrypts the aggregation parameter corresponding to the value to be verified that is equal to the trust value based on a private key, and updates the customer model based on the decrypted aggregation parameter.
[0009] In summary, the verification model update method provided by the embodiments of the present invention is beneficial to improving the credibility and robustness of the coordination device in a federated learning system.
[0010] The foregoing description of the invention and the following description of the embodiments are intended to demonstrate and explain the spirit and principles of the invention, and to provide a further explanation of the scope of the patent application. Attached Figure Description
[0011] Figures 1A to 5A These are schematic diagrams of the architecture of the federal system applicable to the first to fifth embodiments of the present invention.
[0012] Figures 1B to 5B These are flowcharts of the verification model update method according to the first to fifth embodiments of the present invention.
[0013] Figure 6 This is a comparison chart of the average maximum test accuracy of the general model.
[0014] Component designation explanation
[0015] Steps S1-S10
[0016] Steps T1-T10
[0017] U1-U8 Steps
[0018] Steps V1-V9
[0019] Steps W1-W13 Detailed Implementation
[0020] The following detailed description of the features and characteristics of the present invention in the embodiments is sufficient to enable anyone skilled in the art to understand the technical content of the present invention and implement it accordingly. Based on the disclosure of this specification, the scope of the patent applications, and the drawings, anyone skilled in the art can easily understand the related concepts and features of the present invention. The following embodiments are used to further illustrate the viewpoints of the present invention, but are not intended to limit the scope of the present invention in any way.
[0021] The detailed description of the embodiments of the present invention includes several technical terms, the definitions of which are as follows:
[0022] Moderator: A service provider that collects models from multiple clients to aggregate them into a common model for providing services; also known as a coordinating device.
[0023] Client: An endpoint that provides data to be added to distributed training or federated learning; also known as a client device.
[0024] Raw data: Data held by the customer and requiring protection; also known as private data.
[0025] Federated Learning (FL): A collaborative training architecture for training machine learning models without sharing customer data to protect data privacy.
[0026] Machine learning: a field of research that enables computers to learn without explicitly writing programs.
[0027] Homomorphic Encryption (HE): A form of encryption that allows users to perform operations on encrypted data without first decrypting it.
[0028] For a trustworthy FL system, several key security requirements must be met: the reliability and robustness of the coordination device, and customer privacy, with the following objectives:
[0029] (1) Reliable aggregation results: How to verify the correctness of aggregation results on client devices, which involves the concept of verifiable computing;
[0030] (2) Customer privacy: How to protect the updated parameters of each customer device, which involves data encryption; and
[0031] (3) Robust Coordinator: How to perform computation when the coordinator exits or fails during training, which involves the concept of Multi-Party Computation (MPC).
[0032] The method for verifying model updates proposed in this invention includes multiple embodiments, each applicable to FL systems with different configurations. The correspondence between the embodiments and system configurations is shown in Table 1. Based on the number of coordinating devices, FL systems can be classified as single-coordinating-device or multi-coordinating-device systems. Based on whether the multiple update parameters received by the coordinating device from multiple client devices are encrypted, FL systems can be classified as non-private aggregations (update parameters are not encrypted) and private aggregations (update parameters are encrypted). The configuration of the FL system is a combination of the above two classification methods.
[0033] Table 1 shows an example of the method for validating model updates and the applicable FL system configuration.
[0034]
[0035] Figure 1A This is an architecture diagram of the federated system to which the first embodiment of the present invention applies. Figure 1B This is a flowchart of a verification model update method according to a first embodiment of the present invention. In this invention, the first embodiment is also referred to as a Single Verifiable Moderator (SVM). SVM is suitable for FL systems with a single moderator and multiple client devices, where the data sent from the client devices to the moderator is non-private data. In SVM, each client device executes its own... Figure 1B The method is used to verify the model updates provided by the coordinating unit. Since each client unit performs the same steps, the client unit mentioned below can be any client unit in the FL system.
[0036] In step S1, the client device receives the hash function and the general model from the coordinating device. Regarding the general model, during the first FL training, the coordinating device initializes the general model itself and broadcasts it to all client devices. During subsequent FL training, the coordinating device updates the general model based on feedback from the client devices before broadcasting it to all client devices. Regarding the hash function, in one embodiment, the hash function is an additively homomorphic hash (AHH) function.
[0037] In step S2, the client device trains the client model based on the general model and the original data.
[0038] In step S3, the customer calculates the difference between the general model and the customer model as the update parameter.
[0039] In step S4, the client devices send update parameters to the coordinating device, thereby the coordinating device receives multiple update parameters sent by all client devices. Steps S2 to S4 are common operations in FL systems, therefore their details are not described here. As for the update parameters in step S3, they can be model weights or model gradients, which are not limited in this invention.
[0040] In one embodiment of step S3, after the client device calculates the difference, it performs a quantization procedure to convert this difference from a floating-point number to an integer number. Specifically, all client devices coordinate the parameters used for quantization, including the zero-point and scale, thereby converting the floating-point number into an integer of a specified bit length as the update parameter.
[0041] In step S4, all client devices send their own update parameters to the coordinating device, so the coordinating device receives multiple update parameters from multiple client devices.
[0042] In step S5, the client device inputs update parameters to the hash function to generate a hash value. In one embodiment, it is assumed that H:X→Y is a collision-resistant hash function, meaning that computationally it is impossible to find a hash function such that x1≠x2 such that H(x1)=H(x2), where x1 and x2 represent data. If for any x1,...,x... k , satisfying H(∑ i∈[k] x i )=∑ i∈[k] H(x i If k is a parameter, then the hash function H is additively homomorphic. In one embodiment, AHH can be instantiated according to the lattice problem Ring-SIS in the document "C. Peikert, "A Decade of Lattice Cryptography", "Found. Trends Theor. Comput. Sci., vol. 10, p. 283–424, 2016". Specifically, it is assumed that R is a cyclotomic ring with dimension N, where N is a power of 2, i.e.: And for some moduli Q, R Q =R / QR, then the process of establishing AHH is as follows: Define function H, whose input domain is For a certain Moreover, the output domain is The description of function H is a matrix ring. For input x∈X, where x can be interpreted as R l The ring vector in H(x). H(x) outputs h = AxmodQ, where h is the hash value. It should be noted that the above process is only an example, and this invention does not limit the functions used to implement AHH.
[0043] In step S6, the client device sends hash values to client devices other than itself, thereby receiving multiple hash values in addition to the hash value it generated. In other words, each client device in the FL system broadcasts its own hash value to other client devices. Therefore, each client device receives hash values generated by all client devices.
[0044] In step S7, the client device sums up all hash values to generate a trust value.
[0045] In step S8, the client device receives aggregation parameters from the coordinating device. Specifically, the coordinating device performs aggregation operations based on all updated parameters to update the general model. This updated general model is referred to as the aggregated model, and the weights or gradients of this aggregated model are called aggregation parameters. Step S8 is a standard operation of the FL system.
[0046] In step S9, the client device inputs aggregation parameters into the hash function to generate a value to be verified.
[0047] In step S10, the client device compares the value to be verified with the trust value. When the value to be verified equals the trust value, it means that the aggregation parameters generated by the coordination device are normal. Therefore, the client device can update the client model based on the aggregation parameters.
[0048] Overall, SVM proposes a Verifiable Aggregation Protocol (VAP) in steps S4 to S10. By using AHH, the first embodiment of this invention designs a VAP to combat malicious coordination devices. Consider the following scenario: n honest client devices C1,...,C n Each has its own (non-private) inputs x1,...,x n (i.e., the updated parameters described in step S3). The coordinating device M verifies the aggregation of all inputs. In steps S4 and S6, each client device C... i The hash value h of the broadcast update data i =H(x) i ), and will update data x i The data is sent to the coordinating device M. In step S8, the coordinating device M broadcasts the aggregation parameter z to all client devices C. iIn steps S7, S9, and S10, each customer device C i Check ∑ i∈[k] h i Whether H(z) holds true, that is, whether the value to be verified, H(z), is equal to ∑ i∈[k] h i This determines whether the aggregation parameter z is correct in the aggregation result.
[0049] Please refer to Table 1. When the update parameters transmitted from the client device to the coordinating device do not need to be encrypted, the SVM of the first embodiment can be directly applied to an FL system with multiple coordinating devices. This is because, under normal circumstances, the aggregation operation performed by each coordinating device is the same. Therefore, executing SVM for each of the multiple coordinating devices can both satisfy the configuration of a multi-coordinating-device, non-private aggregation FL system and achieve the effect of verifying the aggregation results of each coordinating device.
[0050] Figure 2A This is an architecture diagram of the federated system to which the second embodiment of the present invention applies. Figure 2B This is a flowchart of a method for updating a verification model according to a second embodiment of the present invention. In this invention, the second embodiment is also referred to as a Single Secure and Verifiable Moderator (SSVM). Please refer to... Figure 1B and Figure 2B The steps T1 to T10 of SSVM are mostly the same as those of SVM. The differences between the two are described below.
[0051] During step T3, when the client device calculates the difference between the general model and the client model as update parameters, the client device encrypts the difference using the public key to generate the update parameters. Additionally, during step T10, before updating the client model based on the aggregate parameters, the client device decrypts the aggregate parameters using the private key, and then updates the client model based on the decrypted aggregate parameters.
[0052] In one embodiment, each client device performs the KeyGen key generation protocol in Threshold Additive Homomorphic Encryption (TAHE) to generate a public and private key. See “S. Gordon, F.-H. Liu and E. Shi, “Constant-Round MPC with Fairness and Guarantee of Output Delivery,” in CRYPTO, 2015.” TAHE consists of the KeyGen protocol, Enc(.), and Dec(.). The KeyGen protocol generates a public key pk and sends the corresponding private key sk to each party (e.g., the client device). Enc(x, pk) encrypts x into ciphertext. ciphertext Decryption is x, and the above function only works when a certain percentage of private key sk holders are involved.
[0053] Overall, the second embodiment uses client device C i Private input held To extend the scenarios of SVM. The private input These are encrypted update parameters. Client device C i The goal is to perform verifiable FL training via the coordinating device M, without using their private inputs. Therefore, at the start of training, i.e., during steps T1 to T3, each client device C... i Executing the KeyGen protocol in TAHE generates a public key pk and their respective private keys sk. i Then, during the period from step T4 to step T10, the client device C... i The coordination device M performs VAP. Specifically, in step T4, each customer device C... i Transmit encrypted update parameters To the coordinating device M. In step T5, the client device C... i Use encrypted update parameters Calculate hash value In step T8, customer device C i Receive aggregation parameters from the coordinating device M Aggregate parameters It is also the encrypted result. If it passes the consistency check from step T6 to step T10, in other words, Customer device C i The decryption program can then be executed. This generates the decrypted aggregation parameters, which are then used to update the client model. Otherwise (if the consistency check fails), client device C...i It will be suspended if abnormal behavior of the coordinating device M is detected.
[0054] In SVM and SSVM, the premise is that a single coordinating device will always complete its work. This invention proposes a more robust protocol to prevent potential coordinating device failures (malicious or random failures), thus relaxing this premise. For robustness, redundant computation is introduced into the FL system. In other words, the task of a single coordinating device is delegated to multiple coordinating devices, each handling its own task. As in the previous embodiments, this invention assumes that client devices are honest (e.g., no security threats, no hardware failures, no network outages, etc.). Please refer to Table 1. This invention refers to embodiments three through five collectively as Multiple Secure and Verifiable Moderator (MSVM). The FL systems mentioned in embodiments three through five all have multiple coordinating devices, each of which needs to perform private aggregation. In practice, one of embodiments three, four, or five can be selected for execution depending on the number of failed coordinating devices.
[0055] The third embodiment of this invention is referred to as MSVM-2 / 3. The applicable scenario for the third embodiment is as follows: assuming all coordinating devices {M j} j∈[m] The quantity is m, at least An honest coordinating mechanism will get the job done, while others... The coordination mechanism may fail or disconnect at any time. Please refer to the following documents: "M. Ben-Or, S. Goldwasser and A. Wigderson, "Completeness Theorems for Non-Cryptographic Fault-Tolerant Distributed Computation (Extended Abstract)," in STOC, 1988; and "A. Shamir, "How to Share a Secret," Commun. ACM, vol. 22, pp. 612–613, 1979." The MSVM-2 / 3 proposed in this invention applies the BGW protocol mentioned in the above documents, as well as Shamir secret sharing.
[0056] The (n,t) secret sharing algorithm consists of two parts: Share(.) and Recon(.). Share(x) distributes the input x across n devices. Recon(.) can only recover the original input x if data is collected from t of these devices. Otherwise, x is theoretically hidden information. Shamir secret sharing achieves this goal based on polynomial evaluation and interpolation.
[0057] Figure 3A This is an architecture diagram of the federated system to which the third embodiment of the present invention applies. Figure 3B This is a flowchart of a verification model update method according to a third embodiment of the present invention. Steps U1 to U3 are largely the same as steps S1 to S3 in the first embodiment. The difference is that the third embodiment does not require the use of a hash function; therefore, in step U1, the client device receives only the general model from the coordinating device.
[0058] In step U4, based on the number m of coordinating devices, a degree is generated. The polynomial, and satisfying the update parameter x i =P i (0).
[0059] In step U5, the customer device C i Through Share(x) i ) The polynomial P i (j) Send to all coordinating devices M j .
[0060] In step U6, the customer device C i From all coordination devices M j Receives multiple aggregation parameters p j =∑ i∈[n] P i (j).
[0061] In step U7, the client device C i Check from all coordination devices M j Received aggregation parameters (p1, p2, ..., p m And through Recon(p1,p2,...,p) m The polynomial P is reconstructed. Based on Shamir's homomorphism and robustness, as long as it exceeds... The aggregation parameter p j If this is correct, then the unique polynomial P can be reconstructed, and then the aggregation parameter P(0) = ∑ i∈[n] x i Otherwise, it represents m coordinating devices M. j In China, more than Quantity coordination device M j It is abnormal.
[0062] In step U8, the client device C i Update the customer model based on the aggregation parameter z.
[0063] It is worth noting that the third embodiment is information-theoretic secure and does not require the use of encryption tools.
[0064] Figure 4A This is an architecture diagram of the federated system to which the fourth embodiment of the present invention applies. Figure 4B This is a flowchart of a verification model update method according to a fourth embodiment of the present invention. The fourth embodiment further relaxes the conditions of the third embodiment, assuming that the FL system has at least... An honest coordinating device C i The fourth embodiment is referred to as MSVM-half in this invention.
[0065] In step V1, the customer device C i A general model is received from the federated learning system. For example, the general model is received from one of multiple coordinating devices. Provided that all coordinating devices and all client devices in the federated learning system receive the same general model, this invention does not limit which device in the federated learning system is responsible for broadcasting this general model. In step V2, client device C... i A customer model is trained based on the general model and the original data. In step V3, the difference x between the general model and the customer model is calculated. i Steps V1 to V3 are essentially the same as steps S1 to S3 in the first embodiment.
[0066] In step V4, the client device C i Encryption of differences based on public key pk x i To generate updated parameters In one embodiment, the client device C i Before step V4, the KeyGen protocol in TAHE is executed to generate the public key pk and the private key sk. i .
[0067] In step V5, the client device C i Send updated parameters to all coordinating devices M j Thus, each coordinating device M j Multiple update parameters were received.
[0068] In step V6, the client device C i From multiple coordination devices M j Receive multiple aggregation parameters Each aggregation parameter By each coordinating device M j Calculated based on the aforementioned multiple update parameters, i.e.
[0069] In step V7, the client device C i All aggregation parameters Finding the mode In a set of data, the data that appears most frequently is called the mode. In other words, in step V7, client device C... i examine majority consistency.
[0070] In step V8, the number of people The number exceeds the aggregation parameters When the number is half, that is, more than half of the aggregation parameters. When the values are the same, customer device C i Based on the mode and private key sk i Decryption produces the decryption result z.
[0071] In step V9, the client device C i Update the customer model based on the decryption results.
[0072] In the fourth embodiment, MSVM-half, the prerequisite is that the FL system has a coordinating mechanism with an honest majority. Therefore, a majority vote is sufficient to guarantee the sum of the ciphertext. The correctness of the SVM in the first embodiment is achieved without using the VAP.
[0073] The fifth embodiment considers the worst-case scenario: in a multi-coordinating FL system, only one coordinating device is honest. Figure 5A This is an architecture diagram of the federated system to which the fifth embodiment of the present invention applies. Figure 5B This is a flowchart of a method for updating a verification model according to the fifth embodiment of the present invention. The fifth embodiment is referred to as MSVM-one.
[0074] In step W1, the client device receives a hash function H and a general model from the federated learning system. For example, the general model may be received from one of multiple coordinating devices. In one embodiment, the hash function is AHH. In step W2, the client device trains a client model based on the general model and the original data. In step W3, the client device calculates the difference x between the general model and the client model. i Steps W1 to W3 are essentially the same as steps S1 to S3 in the first embodiment.
[0075] In step W4, the client device encrypts the difference based on the public key pk. To generate updated parameters In one embodiment, the client device executes the KeyGen protocol in TAHE to generate a public key pk and a private key sk before step W4. i .
[0076] In step W5, the customer device C i Send update parameters To all coordinating devices M j Thus, each coordinating device M j Received multiple update parameters
[0077] In step W6, the customer device C i Input update parameters To generate a hash value using the hash function H.
[0078] In step W7, the client device C i Send hash value To other client devices, thereby receiving hash values other than those generated by the device itself. Multiple hash values other than those mentioned above.
[0079] In step W8, the client device C i Sum all hash values To generate trust value
[0080] In step W9, the customer device C i From all coordination devices M j Receive multiple aggregation parameters Each aggregation parameter Each coordination device M j Based on multiple update parameters Calculated, i.e.
[0081] In step W10, the customer device C i Enter each aggregation parameter separately. The hash function H is used to generate multiple values to be verified.
[0082] In step W11, each value to be verified is compared sequentially. With trust value In other words, for j = 1,...,m, the client device C i examine Is it true or false? Overall, steps W10 and W11 are hash consistency checks performed by the client device.
[0083] In step W12, when the first value to be verified equal to the trust value is found, all client devices C i Based on their respective private keys sk i Decrypt the aggregate parameter corresponding to this value to be verified, i.e. In other words, It is the first ciphertext to pass the hash consistency check.
[0084] In step W13, the customer device C i Update the client model based on the decrypted aggregation parameter z.
[0085] This invention employs various cryptographic tools to construct a secure and trustworthy coordinating device. In a first embodiment, the invention uses Additive Homomorphic Hashing (AHH) to construct a verifiable aggregation protocol (VAP) in a federated learning (FL) system with a single coordinating device, allowing client devices to verify the correctness of aggregate parameter calculations. In FL systems with multiple coordinating devices, the invention uses robust threshold secret sharing. In cases where the coordinating device may be dishonest, the invention uses Threshold Additive Homomorphic Encryption (TAHE) to preserve the privacy of updated parameters for client devices.
[0086] The following section explains the proposed method for verifying model updates from two aspects: the performance (test accuracy) of the general model and the training time.
[0087] Figure 6 This is a comparison of the average maximum test accuracy of the general model. The four bars from left to right correspond to different methods for training the general model: conventional intensive training (Cen), conventional quantized training (Cen(Q)), conventional federated learning (FL), and quantized federated learning (FL(Q)). The intervals on each bar represent the standard deviation of five independent experiments. Overall, the maximum accuracy is stable across experiments.
[0088] from Figure 6 It can be observed that: (1) the quantization of updated parameters has little impact on accuracy; (2) the accuracy of the FL method is slightly lower than that of the intensive training method; and (3) the magnitude of data skewness does affect the accuracy, and its parameter value is positively correlated with the accuracy. Specifically, the test accuracies of SVM, SSVM, MSVM-half, and MSVM-one are 0.742 (0.009), 0.741 (0.009), 0.736 (0.002), and 0.734 (0.002), respectively, where the values in parentheses represent the standard deviation of the five experiments.
[0089] Table 2 shows the average computation time and cost (in seconds per iteration) for training a network with 1,250,858 updated parameters.
[0090]
[0091] Table 2 shows the time cost of several embodiments of the invention and centralized training for each procedure. Fields marked with "-" indicate that the method does not execute that procedure. Training time includes the time spent executing forward and backward procedures to update the general model. Consistency checks include the time spent on hash consistency checks, majority consistency checks, and aggregation equality checks. Verifiable computations such as AHH and consistency checks add approximately 4 seconds per iteration. Compared to centralized training, each iteration increases training time by 30%. If the updated parameters are encrypted, the verifiable computations require additional time, depending on the amount of data added to the ciphertext relative to the plaintext. Overall, using homomorphic addition to encrypted updated parameters significantly increases training time, but this can be reduced with hardware support and better implementation. In embodiments with multiple coordinators, aggregation time increases, but the increment is relatively smaller than the time spent on encrypted parameter updates. Consistency check time increases significantly in MSVM-one, taking approximately 8.5 seconds for two hash consistency checks, because 4 were randomly assigned honest coordinators in the experiments. Checking an aggregation parameter The corresponding hash value takes approximately 4.3 seconds. MSVM-half spends very little time on consistency checks because checking majority consistency is very fast. Overall, cryptographically updating parameters significantly increases training time.
[0092] The proposed method for verifying model updates includes multiple embodiments, applicable to various scenarios with different security requirements. In particular, this invention considers two important design requirements: (1) whether the client's gradient needs to be kept confidential; and (2) whether more coordination mechanisms are available during federated learning. Specifically, this invention proposes protocols to address the following three cases:
[0093] (1) Single Verifiable Coordinator (SVM): Assuming the update parameters of the client device are non-private, the goal of the SVM is to check the correctness of the aggregation results of the coordinator. This protocol can be extended to federated learning systems with multiple coordinators because the client device only needs to run the SVM with each coordinator. As long as one of the coordinators does not exit during the training of the federated learning, the client device can receive the expected aggregation parameters.
[0094] (2) Single Secure and Verifiable Coordination Device (SSVM): Its application scenario is similar to SVM, but the updates of the client device are private and should be protected to achieve verifiable private aggregation.
[0095] (3) Multiple Secure and Verifiable Coordinating Devices (MSVMs): MSVMs emphasize the robustness of the coordinating devices, where aggregation operations are not interrupted by the failure of the coordinating devices. To achieve this, the present invention distributes a single coordinating device into multiple coordinating devices so that they can run some distributed protocol to prevent interruptions caused by the failure of the coordinating device.
[0096] In summary, this invention evaluated the specific performance and cost of multiple federated learning systems with progressive security levels in configurations, demonstrating the practicality of the invention, enhancing protection from the client device perspective, and taking an important step towards developing robust and reliable federated learning systems for current and future applications.
[0097] While the present invention has been disclosed above with reference to the foregoing embodiments, it is not intended to limit the invention. Any modifications and refinements made without departing from the spirit and scope of the invention are within the scope of patent protection of the present invention. For details regarding the scope of protection defined in the present invention, please refer to the appended claims.
Claims
1. A method for verifying model updates, characterized in that, Applicable to a federated learning system comprising a coordinating device and multiple client devices, including performing the following steps on each of the client devices: Receive a hash function and a general model from the coordination device; A customer model is trained based on the general model and a set of raw data; The difference between the general model and the customer model is calculated as an update parameter; The update parameters are sent to the coordination device so that the coordination device receives multiple update parameters; The update parameters are input into the hash function to generate a hash value; Send the hash value to other client devices besides itself, and receive multiple hash values other than the hash value generated by itself; The hash value generated by the process itself and the received hash values are summed to produce a trust value; A aggregation parameter is received from the coordinating device, the aggregation parameter being calculated by the coordinating device based on the plurality of update parameters; Input the aggregation parameters into the hash function to generate a value to be verified; as well as The value to be verified is compared with the trust value. When the value to be verified is equal to the trust value, the customer model is updated according to the aggregation parameters.
2. The method for verifying model updates according to claim 1, characterized in that, The hash function is an additive homomorphic hash function.
3. The method for verifying model updates according to claim 1, characterized in that, Each of the aforementioned client devices calculates the difference between the general model and the client model as the update parameter, including: Calculate the difference between the general model and the customer model; and Perform a quantization procedure to convert the difference from a floating-point number to an integer.
4. The method for verifying model updates according to claim 1, characterized in that: Calculating the difference between the general model and the client model as the update parameter in each of the client devices includes: each of the client devices encrypting the difference based on a public key to generate the update parameter; and Updating the client model based on the aggregation parameters includes: each client device decrypting the aggregation parameters using a private key, and updating the client model based on the decrypted aggregation parameters.
5. The method for verifying model updates according to claim 4, characterized in that, Also includes: Each of the client devices performs a key generation protocol in threshold additive homomorphic encryption to generate the public key and the private key.
6. A method for verifying model updates, characterized in that, Applicable to a federated learning system comprising multiple coordinating devices and multiple client devices, including performing the following steps on each of the client devices: Receive a general model from the federated learning system; A customer model is trained based on the general model and a set of raw data; Calculate the difference between the general model and the customer model; The difference is encrypted using a public key to generate an update parameter; The update parameters are sent to the plurality of coordination devices so that each of the coordination devices receives the plurality of update parameters; Multiple aggregation parameters are received from the multiple coordination devices, and each aggregation parameter is calculated by each of the multiple update parameters based on the multiple coordination devices. Find a mode among the multiple aggregation parameters; When the number of the mode exceeds half the number of the plurality of coordinating devices, a decryption result is generated based on the mode and a private key. as well as The customer model is updated based on the decryption result.
7. The method for verifying model updates according to claim 6, characterized in that, Also includes: Each of the client devices performs a key generation protocol in threshold additive homomorphic encryption to generate the public key and the private key.
8. A method for verifying model updates, characterized in that, Applicable to a federated learning system comprising multiple coordinating devices and multiple client devices, including performing the following steps on each of the client devices: Receive a hash function and a general model from the federated learning system; A customer model is trained based on the general model and a set of raw data; Calculate the difference between the general model and the customer model; The difference is encrypted using a public key to generate an update parameter; The update parameters are sent to the plurality of coordination devices so that each of the coordination devices receives the plurality of update parameters; The update parameters are input into the hash function to generate a hash value; Send the hash value to other client devices besides itself, and receive multiple hash values other than the hash value generated by itself; The hash value generated by the process itself is summed with the received hash values to produce a trust value; Multiple aggregation parameters are received from the multiple coordination devices, and each of the multiple aggregation parameters is calculated by each of the multiple coordination devices based on the multiple update parameters; Each of the aggregation parameters is input into the hash function to generate multiple values to be verified. Compare each of the values to be verified with the trust values in sequence; as well as When a value to be verified that is equal to the trust value is found, the aggregate parameter corresponding to the value to be verified that is equal to the trust value is decrypted according to a private key, and the customer model is updated according to the decrypted aggregate parameter.
9. The method for verifying model updates according to claim 8, characterized in that, The hash function is an additive homomorphic hash function.
10. The method for verifying model updates according to claim 8, characterized in that, Also includes: Each of the client devices performs a key generation protocol in threshold additive homomorphic encryption to generate the public key and the private key.