Verifiable Active Security Aggregation Method and System for Federated Learning

Abstract

The present invention belongs to the technical field of secure federated learning, and specifically discloses a verifiable active secure aggregation method and system for federated learning, the method comprising the following process: the client generates a local model update using a local data set for training, and sends the model update to an aggregation server through homomorphic encryption, with verification information attached, the server calculates and sends the encrypted aggregate model update and verification value to each client, the client decrypts after verification to obtain a partial aggregation result, encrypts it using a shared secret with other users and sends it to the server to propagate to the corresponding client, the client decrypts to obtain a sufficient number of partial decryption results, and then aggregates to obtain the final global model update. The solution provided by the present invention protects the privacy of model updates and provides the verifiability of aggregation results under active security.

Description

Technical Field

[0001] The present invention relates to the technical field of secure federated learning, and in particular to a verifiable active secure aggregation method and system for federated learning. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Federated learning is a distributed machine learning paradigm that aims to solve the data silo problem while ensuring data privacy. This approach allows machine learning tasks to be performed collaboratively across many clients (such as mobile devices) without moving data away from the client, thereby protecting data privacy. In this setting, the main responsibility of the service provider is to coordinate the operations of multiple clients, receive model parameters trained locally by each client, and update the global aggregate model to ensure its effectiveness. In order to fully realize the advantages of federated learning, a series of challenges must be overcome to ensure data privacy, verifiability of results, and robustness of user exit.

[0004] First, data privacy is a core concern of federated learning. Although the data does not leave the client, local model updates may still leak some sensitive information. To solve this problem, researchers have proposed a variety of technical solutions, including homomorphic encryption, differential privacy, and secure multi-party computing protocols. Homomorphic encryption is particularly noteworthy in federated learning because it allows ciphertext to be calculated without decrypting the data, thereby providing stronger privacy protection. Differential privacy ensures the privacy of a single data point by introducing noise into the data, which will affect the accuracy of the model but can greatly reduce the computational overhead. Secure multi-party computing protocols allow multiple participants to collaborate on computations without leaking their respective data, thereby ensuring the privacy of the data and the correctness of the computation.

[0005] Verification of aggregation results is also key to the success of federated learning. In a distributed environment, it is crucial to ensure that the final global model accurately reflects the contributions of each client. To this end, service providers need to effectively coordinate the operations of clients to receive and correctly aggregate local model parameters. In addition, providing a verification mechanism to ensure the transparency and credibility of the aggregation process and results is also crucial for user trust and system robustness.

[0006] In addition, the federated learning system must be robust against user exits. The power and network connection of mobile devices are usually unstable, and user participation is unpredictable. Therefore, the system needs to be designed to continue to operate normally when some users exit, and to maintain the consistency of the model training process and the effectiveness of the final model. This robustness requires not only that the system can dynamically adjust the participating clients, but also that the model can still be effectively trained and updated when some data is lost.

[0007] More seriously, federated learning faces security and privacy threats from external malicious actors. By active adversaries, we mean parties (clients or servers) that deviate from the protocol, send incorrect and / or arbitrarily chosen messages to honest users, give up, omit messages, and share their entire view of the protocol with each other and the server (if the server is also an active adversary). On the other hand, adversarial servers also pose a threat to federated learning, as curious server-side actors may leak private data by reverse engineering local model parameters received by the server.

[0008] Federated learning has significant advantages in protecting data privacy and solving data silo problems. However, in order to fully realize its potential, it must be continuously optimized and improved in terms of privacy protection, result verifiability, and robustness of user exit. Summary of the invention

[0009] In order to solve the above problems, the present invention proposes a verifiable active security aggregation method and system for federated learning, which protects the privacy of model updates and provides the verifiability of aggregation results under active security.

[0010] In some embodiments, the following technical solutions are adopted:

[0011] A verifiable active security aggregation method for federated learning includes the following process:

[0012] The server generates the private key and public key of the homomorphic encryption algorithm and sends the private key share and the public key Distribute to each client;

[0013] Each client uses the public key to encrypt the local model update to obtain encrypted information ; Each client generates a random number , for the random number Perform secret sharing and calculate the ciphertext shares of other clients ; The random number and ciphertext share Send to the server;

[0014] The server sends all encrypted ciphertext shares to each client. , the client decrypts and aggregates the decrypted plaintext, encrypts the aggregated plaintext and sends it to the server;

[0015] After receiving messages from a sufficient number of clients, the server aggregates and encrypts the model updates and sends them to the corresponding clients;

[0016] Each client receives the encrypted aggregate information from the server and verifies it. After verification, it uses its own private key to decrypt the aggregate information and obtains a partial decrypted value.

[0017] Each client encrypts part of the decrypted value using a secret shared with other clients and sends the encrypted information to the server. After the server collects all the encrypted information, it sends the ciphertext information of each client back to each client. After receiving enough ciphertext information, each client decrypts it to obtain the final decrypted information and updates the global model.

[0018] As a further solution, each client Use your own local data based on the global model Train the local model update for this round , using the private key share pair Perform encryption calculation to obtain encrypted information ;

[0019] After receiving the private key share, each client generates , sent to the server, the server generates a random number , and then broadcast them together.

[0020] Each client randomly selects a random number , and using the generated random numbers and encrypted information , calculate the verification value , the verification value Send to the server.

[0021] As a further solution, calculate the ciphertext shares of other clients , specifically:

[0022] Client calculate .

[0023] As a further solution, each client receives the encrypted aggregate information from the server and performs verification, specifically:

[0024] Each client After receiving the encrypted aggregate information from the server, a secret recovery operation is performed to obtain ; Then verify and Are they equal? ​​If they are equal, use the private key share. right Decrypt and get part of the decrypted value ;in, Update aggregation information for the model, is the verification value.

[0025] As a further solution, each client encrypts part of the decrypted value using a secret shared with other clients and sends the encrypted information to the server, specifically:

[0026] When the client Get partial decrypted value Afterwards, use the client The shared secret Encrypted The shared secret is generated between clients through the Diffie-Hellman key negotiation protocol. Sent to the server.

[0027] As a further solution, after receiving enough ciphertext information, each client performs decryption to obtain the final decrypted information, which is specifically:

[0028] Each client Receive at least Encrypted ciphertext After that, use the client The shared secret Decrypt and obtain partial decrypted information of other clients ; ;

[0029] Decryption on the client side yields at least Decrypted values ​​of other clients After obtaining the information, Lagrange interpolation is used to calculate the interpolation coefficients , and then through the interpolation coefficient Calculate the final decrypted information , and then update the global model;

[0030] in, Indicates The partial decrypted value of the client, is the identifier of the corresponding client. It is a variable in the interpolation calculation, used to construct the interpolation coefficient , As a reference point in the interpolation process, it is used to calculate each interpolation coefficient. is the value of other clients, Is the value of the current client.

[0031] In other embodiments, the following technical solutions are adopted:

[0032] A verifiable active security aggregation system for federated learning includes: a server and multiple clients, wherein the server is configured to implement the following process:

[0033] Generate the private key and public key of the homomorphic encryption algorithm and share the private key and the public key Distribute to each client;

[0034] Receive random numbers sent by each client and ciphertext share ; Send all encrypted ciphertext shares to each client , so that the client can decrypt and aggregate the decrypted plaintext;

[0035] Receive the encrypted aggregated plaintext sent by each client; after receiving messages from a sufficient number of clients, aggregate the model updates and encrypt them before sending them to the corresponding clients; so that each client receives the encrypted aggregated information from the server and then verifies it. After verification, use their own private key to decrypt the aggregated information to obtain a partial decrypted value;

[0036] Receive the partial decrypted value encrypted by each client using the secret shared with other clients, collect all the encrypted information, and then send the ciphertext information of each client back to each client; so that each client can decrypt after receiving enough ciphertext information, obtain the final decrypted information, and update the global model.

[0037] Each client is configured to perform the following process:

[0038] Receive the share of the private key sent by the server and the public key ;

[0039] The public key is used to encrypt the local model update to obtain encrypted information. ; Each client generates a random number , for the random number Perform secret sharing and calculate the ciphertext shares of other clients ; The random number and ciphertext share Send to the server;

[0040] Receive all ciphertext shares sent by the server for the client , decrypt and aggregate the decrypted plaintext, encrypt the aggregated plaintext and send it to the server;

[0041] Receive the encrypted model update aggregate information and verify it. After verification, use their respective private keys to decrypt the aggregate information to obtain a partial decrypted value.

[0042] Use the secret shared with other clients to encrypt part of the decrypted value and send the encrypted information to the server; receive the ciphertext information returned by the server, and after receiving enough ciphertext information, decrypt it to obtain the final decrypted information and update the global model.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] (1) The server of the method of the present invention cannot learn the specific value of a certain client information from the information uploaded by the client. The information is transmitted in ciphertext form, which effectively solves the problem of being vulnerable to gradient reversal attacks.

[0045] (2) The method of the present invention uses a distributed decryption strategy to obtain the final decryption result without any of the participants knowing the complete decryption key. Even if an active adversary controls some clients, the private information of other clients cannot be obtained.

[0046] (3) The client of the present invention generates verifiable information about the local gradient, and the server sends it to the client after calculation, so that the client can verify the sent aggregate value.

[0047] Other features and advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 Schematic diagram of a verifiable active security aggregation method for federated learning in an embodiment of the present invention;

[0049] Figure 2 Schematic diagram of a verifiable active security aggregation system for federated learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0051] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0052] Embodiment 1

[0053] In one or more embodiments, a verifiable active security aggregation method for federated learning is disclosed, which is used to realize data security interaction between multiple clients and a server; wherein each client trains model parameters locally, and sends the model update to the aggregation server through homomorphic encryption, with verification information attached, the server calculates and sends the encrypted aggregate model update and verification value to each client, the client decrypts after verification to obtain partial aggregation results, encrypts them with a shared secret with other users and sends them to the server to propagate to the corresponding client, the client decrypts to obtain a sufficient number of partial decryption results, and then aggregates to obtain the final global model update. The aggregation method of this embodiment can protect the privacy of model updates and provide the verifiability of aggregation results under active security.

[0054] As a specific implementation method, Figure 1 The verifiable active security aggregation method for federated learning in this embodiment specifically includes the following process:

[0055] (1) Initialization phase: This phase mainly involves the following processes:

[0056] (1-1) Give each party a security parameter k, the number of clients n and a threshold t; generate the public parameter pp honestly; the threshold t is used to ensure that the secret cannot be recovered when there are less than t clients; all users also have a private authenticated channel with the server.

[0057] (1-2) The server generates a homomorphic encryption algorithm public key for encrypting the client's gradient information and private key , split the private key into t-out-of-n threshold secrets , where n is the number of clients, then broadcast the public key to all clients and send each client a share of the private key After sending, the server will send the local private key Crush.

[0058] (1-3) After receiving the private key share, the client generates , sent to the server, the server generates a random number , and then broadcast them together; among them, Use your own It is generated and used for calculation in the morning when key negotiation KA.Agree is used later.

[0059] (2) Encryption aggregation stage: This stage mainly involves the following processes:

[0060] (2-1) Each client Use your own local data based on the global model Train the local model update for this round , using the public key right Perform encryption calculation to obtain .

[0061] (2-2) Each client randomly selects a random number , and use the random number generated by the server Generate verification value , and then the random number Perform t-out-of-n secret sharing and get ;

[0062] For each additional client , Client calculate ,in Indicates authentication encryption. Indicates key negotiation, so that the client Multiple ciphertext shares from other clients j besides itself can be calculated; Represents each client For the shares of client j, j represents other clients.

[0063] Then update the encrypted model 、Ciphertext share And the validation value Send to the server.

[0064] (2-3) The server receives at least After receiving the message from each client, all the encrypted ciphertexts for it are sent to each client. After receiving it, the client decrypts it to get the corresponding plaintext , after decrypting all After that, the decrypted plaintext is aggregated to obtain ,Right now of which , and then use key negotiation The obtained key is encrypted and returned to the server.

[0065] (2-4) The server receives at least After receiving the message from a client, all clients will be collected Aggregate the local updates of , the aggregated results and encrypted information after calculation Sent to the corresponding client.

[0066] (3) Partial decryption stage: This stage mainly involves the following processes:

[0067] (3-1) Each client After receiving the encrypted aggregate information from the server, a secret recovery operation is performed to obtain , then verify and Are they equal? ​​If they are equal, the verification is successful and the private key share is encrypted using its homomorphic encryption right Decrypt and get part of the decrypted value .

[0068] (3-2) When the client Decrypt to obtain partial decrypted value Afterwards, use the client The shared secret Encrypted ,The shared secret is generated between the clients through the Diffie-Hellman key agreement protocol, and the encrypted information is sent to the server.

[0069] (3-3) After receiving multiple encrypted messages from different clients, the server sends all the encrypted ciphertext messages to each client, which is the client send , and proceed to the next step.

[0070] (4) Final decryption stage: This stage mainly involves the following processes:

[0071] (4-1) Each client Receive at least Encrypted ciphertext After that, use the client The shared secret Decrypt and obtain partial decrypted information of other clients .

[0072] (4-2) Decryption on the client side yields at least Decrypted values ​​of other clients information, including Indicates The partial decrypted value of the client, is the identifier of the corresponding client, and the interpolation coefficient is calculated using Lagrange interpolation , and then through the interpolation coefficient Calculate the final decrypted information , and then update the global model. Among them, It is a variable in the interpolation calculation, used to construct the interpolation coefficient , As a reference point in the interpolation process, it is used to calculate each interpolation coefficient. is the value of other clients, Is the value of the current client.

[0073] The method of this embodiment is based on homomorphic encryption and combined with secure multi-party computing, which protects the privacy of model updates and provides the verifiability of aggregation results under active security.

[0074] Embodiment 2

[0075] In one or more embodiments, a verifiable active security aggregation system for federated learning is disclosed, combined with Figure 2 , specifically comprising: a server and multiple clients, wherein the server is configured to implement the following process:

[0076] Generate the private key and public key of the homomorphic encryption algorithm and share the private key and the public key Distribute to each client;

[0077] Receive random numbers sent by each client and ciphertext share ; Send all encrypted ciphertext shares to each client , so that the client can decrypt and aggregate the decrypted plaintext;

[0078] Receive the encrypted aggregated plaintext sent by each client; after receiving messages from a sufficient number of clients, aggregate the model updates and encrypt them before sending them to the corresponding clients; so that each client receives the encrypted aggregated information from the server and then verifies it. After verification, use their own private key to decrypt the aggregated information to obtain a partial decrypted value;

[0079] Receive the partial decrypted value encrypted by each client using the secret shared with other clients, collect all the encrypted information, and then send the ciphertext information of each client back to each client; so that each client can decrypt after receiving enough ciphertext information, obtain the final decrypted information, and update the global model.

[0080] Each client is configured to perform the following process:

[0081] Receive the share of the private key sent by the server and the public key ;

[0082] The public key is used to encrypt the local model update to obtain encrypted information. ; Each client generates a random number , for the random number Perform secret sharing and calculate the ciphertext shares of other clients ; The random number and ciphertext share Send to the server;

[0083] Receive all ciphertext shares sent by the server for the client , decrypt and aggregate the decrypted plaintext, encrypt the aggregated plaintext and send it to the server;

[0084] Receive the encrypted model update aggregate information and verify it. After verification, use their respective private keys to decrypt the aggregate information to obtain a partial decrypted value.

[0085] Use the secret shared with other clients to encrypt part of the decrypted value and send the encrypted information to the server; receive the ciphertext information returned by the server, and after receiving enough ciphertext information, decrypt it to obtain the final decrypted information and update the global model.

[0086] The specific implementation method of the above process is the same as that in Example 1 and will not be described in detail.

[0087] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A verifiable active security aggregation method for federated learning, characterized in that: The process includes the following: The server generates a private key and a public key for the homomorphic encryption algorithm used to encrypt the client's gradient information, and sends the private key share and the public key Distribute to each client; Each client uses the public key to encrypt the local model update to obtain encrypted information ; Each client generates a random number , for the random number Perform secret sharing and calculate the ciphertext shares of other clients ; The random number and ciphertext share Send to the server; After receiving the private key share, each client generates , sent to the server, the server generates a random number , and then broadcast them together; among them, Use your own to generate; Each client randomly selects a random number , and using the generated random numbers and encrypted information , calculate the verification value , the verification value Send to the server; Calculate the ciphertext shares of other clients , specifically: Client calculate ; in, Indicates authentication encryption. Indicates key negotiation, so that the client Calculate the number of clients other than yourself Multiple ciphertext shares of ; Represents each client For Clients share, Indicates other clients; The server sends all encrypted ciphertext shares to each client. , the client decrypts and aggregates the decrypted plaintext, encrypts the aggregated plaintext and sends it to the server; After receiving messages from a sufficient number of clients, the server aggregates and encrypts the model updates and sends them to the corresponding clients; Each client receives the encrypted aggregate information from the server and verifies it. After verification, it uses its own private key to decrypt the aggregate information and obtains a partial decrypted value. Each client receives the encrypted aggregate information from the server and performs verification, specifically: Each client After receiving the encrypted aggregate information from the server, a secret recovery operation is performed to obtain ; Then verify and Are they equal? ​​If they are equal, use the private key share. right Decrypt and get part of the decrypted value ;in, Update aggregation information for the model, To verify the value, for share; Each client encrypts part of the decrypted value using a secret shared with other clients and sends the encrypted information to the server. After the server collects all the encrypted information, it sends the ciphertext information of each client back to each client. After receiving enough ciphertext information, each client decrypts it to obtain the final decrypted information and updates the global model.

2. A verifiable active security aggregation method for federated learning as claimed in claim 1, characterized in that: Each client Use your own local data based on the global model Train the local model update for this round , using the private key share pair Perform encryption calculation to obtain encrypted information .

3. A verifiable active security aggregation method for federated learning as claimed in claim 1, characterized in that: Each client encrypts part of the decrypted value using a secret shared with other clients and sends the encrypted information to the server, specifically: When the client Get partial decrypted value Afterwards, use the client The shared secret Encrypted The shared secret is generated between clients through the Diffie-Hellman key negotiation protocol. Sent to the server.

4. A verifiable active security aggregation method for federated learning as claimed in claim 1, characterized in that: After receiving enough ciphertext information, each client decrypts it and obtains the final decrypted information, which is as follows: Each client Receive at least Encrypted ciphertext After that, use the client The shared secret Decrypt and obtain partial decrypted information of other clients ; ; Decryption on the client side yields at least Decrypted values ​​of other clients After obtaining the information, Lagrange interpolation is used to calculate the interpolation coefficients , and then through the interpolation coefficient Calculate the final decrypted information , and then update the global model; in, Indicates The partial decrypted value of the client, is the identifier of the corresponding client. It is a variable in the interpolation calculation, used to construct the interpolation coefficient , As a reference point in the interpolation process, it is used to calculate each interpolation coefficient. is the value of other clients, Is the value of the current client.

5. A verifiable active security aggregation system for federated learning, using a verifiable active security aggregation method for federated learning as claimed in claim 1, characterized in that: include: A server and multiple clients, wherein the server is configured to implement the following process: Generate the private key and public key of the homomorphic encryption algorithm and share the private key and the public key Distribute to each client; Receive random numbers sent by each client and ciphertext share ; Send all encrypted ciphertext shares to each client , so that the client can decrypt and aggregate the decrypted plaintext; Receive the encrypted aggregated plaintext sent by each client; after receiving messages from a sufficient number of clients, aggregate the model updates and encrypt them before sending them to the corresponding clients; so that each client receives the encrypted aggregated information from the server and then verifies it. After verification, use their own private key to decrypt the aggregated information to obtain a partial decrypted value; Receive the partial decrypted value encrypted by each client using the secret shared with other clients, collect all the encrypted information, and then send the ciphertext information of each client back to each client; so that each client can decrypt after receiving enough ciphertext information, obtain the final decrypted information, and update the global model.

6. A verifiable active security aggregation system for federated learning as claimed in claim 5, characterized in that: Each client is configured to perform the following process: Receive the share of the private key sent by the server and the public key ; The public key is used to encrypt the local model update to obtain encrypted information. ; Each client generates a random number , for the random number Perform secret sharing and calculate the ciphertext shares of other clients ; The random number and ciphertext share Send to the server; Receive all ciphertext shares sent by the server for the client , decrypt and aggregate the decrypted plaintext, encrypt the aggregated plaintext and send it to the server; Receive the encrypted model update aggregate information and verify it. After verification, use their respective private keys to decrypt the aggregate information to obtain a partial decrypted value. Use the secret shared with other clients to encrypt part of the decrypted value and send the encrypted information to the server; receive the ciphertext information returned by the server, and after receiving enough ciphertext information, decrypt it to obtain the final decrypted information and update the global model.

Images