A federated learning method based on high-availability non-interactive secure aggregation

Abstract

The application discloses a kind of federal learning methods based on high availability non-interactive security aggregation scheme, its characteristics are based on the security aggregation method of pairing mask, through asynchronous public subset consensus algorithm, it is realized on distributed server cluster non-interactive high availability security aggregation federal learning, specifically includes: (A) initialization stage (B) aggregation stage epoch 0, first round, user;(C) aggregation stage epoch 0, first round, server;(D) aggregation stage epoch 0, second round, user;(E) aggregation stage epoch 0, second round, server;(F) aggregation stage epoch k (k>0), user and (G) aggregation stage epoch k (k>0), server etc.Steps.The application has high availability security aggregation, small amount of calculation, resists malicious adversary and other advantages compared with prior art, realizes the privacy protection of gradient information, and guarantees the correctness of federal learning training result, is established in the asynchronous network model closest to real network model, with higher practical application value.

Description

Technical Field

[0001] This invention relates to the field of secure aggregation technology, and in particular to a federated learning method for highly available non-interactive secure aggregation based on pairing masks. Background Technology

[0002] Today, artificial intelligence technologies based on machine learning are shining brightly in various fields. Training machine learning models requires significant computing power, time, and, most importantly—a vast amount of data. However, in certain areas, data privacy is particularly important to data owners, such as electronic medical records, yet this data also possesses high analytical value. Patients, on the one hand, do not want to expose the privacy of their individual data, but on the other hand, they want to provide data to researchers for analysis. This problem is known in the industry as the "data silo problem," and this paradoxical situation has led to the emergence of federated learning technology.

[0003] In 2016, Google proposed the first federated learning scheme, using input method data distributed across mobile devices for training, effectively utilizing data resources while ensuring the privacy and security of individual users. Based on the overlap in the datasets owned by different data owners, federated learning can be mainly divided into three categories: horizontal federated learning, vertical federated learning, and federated transfer learning. Horizontal federated learning refers to a training method used when the features of the data provided by different parties have a high degree of overlap, but the data source overlap is low; vertical federated learning refers to a training method used when the data source of the data provided by different parties has a high degree of overlap, but the feature overlap is low; transfer learning is a training method used when the overlap between the two is limited.

[0004] Unless otherwise specified, federated learning can be assumed to be horizontal federated learning. In traditional federated learning training methods, the federated averaging algorithm is the most widely used training scheme. This involves each party training its model locally using its own data, uploading the gradient parameters trained in that epoch, averaging the gradients from all parties, and then updating their local models based on this average. This ultimately achieves consistent model training across all participants and significantly increases the amount of training data. However, uploading gradient information without protection poses a risk of data privacy leakage for users. Differentiating the gradient values ​​reveals the individual data owner's label information. Therefore, a secure aggregation algorithm is needed to protect the privacy of the gradient information during averaging. Currently, secure aggregation technologies mainly fall into four categories: additive homomorphic encryption, secret sharing, multi-party secure computation, and pairwise masking.

[0005] During federated learning training, users may drop out midway due to the large number of participants and the unavoidable latency or interruptions in real-world network environments. Therefore, it's essential to implement a secure aggregation scheme that maintains good performance without compromising security, even when users might drop out. Schemes based on additive homomorphic encryption and secret-sharing naturally possess this advantage, while schemes based on pairwise masks require additional rounds to tolerate dropouts. In more realistic asynchronous network models, increasing the number of interaction rounds means increasing communication time, as each interaction requires a network communication session, introducing a network latency. Therefore, implementing non-interactive secure aggregation schemes or minimizing interaction rounds is crucial for performance optimization.

[0006] The aggregation of gradient information requires cloud servers to undertake computational tasks. When cloud servers are corrupted by malicious adversaries, the aggregation results may be tampered with, thereby modifying the model training results and enabling backdoor attacks on the model. Therefore, ensuring the verifiability of the aggregation results is also a research hotspot. Existing verifiability solutions mainly fall into two categories: using homomorphic hashing and using blockchain technology.

[0007] Most existing solutions employ a single-server or dual-server setup, which carries the risk of a single point of failure. A single point of failure means that if any part of the system fails, the entire system will become unusable and forced to shut down. Therefore, some research utilizes blockchain technology, employing redundant server clusters to achieve high availability for the entire system. The core of blockchain technology is the consensus algorithm, including computation-based and communication-based consensus algorithms. However, existing solutions do not effectively integrate security aggregation technology with consensus algorithm technology, and the chosen consensus algorithms may not fully meet the needs of security aggregation solutions.

[0008] In summary, while existing federated learning technologies have solved some of the problems, they do not cover all of these important characteristics and cannot fully meet the scenarios of secure aggregation solutions. In terms of functionality and security, they clearly cannot meet the usage requirements. Summary of the Invention

[0009] The purpose of this invention is to address the shortcomings of existing technologies by providing a federated learning method based on highly available, non-interactive, secure aggregation. It employs a combination of secure aggregation based on paired masks and the Byzantine consensus algorithm to achieve a non-interactive, secure aggregation that guarantees result correctness and privacy. Federated learning is implemented based on this aggregation method, featuring non-interactive characteristics. A secondary consensus mechanism further optimizes communication volume between nodes, and a backup key mechanism is added, allowing users who have temporarily exited the process to quickly participate in training, further improving user experience and data utilization. This achieves high availability, privacy protection, correct aggregation results, and resistance to malicious adversaries. The method ensures the correct execution of the protocol by the server cluster through the Byzantine consensus algorithm, thereby guaranteeing the correctness of the aggregation results. Furthermore, secure data aggregation is achieved through paired masks, ensuring privacy protection of the aggregation results. We have combined these two methods and optimized the number of communication rounds and communication overhead to implement a non-interactive secure aggregation scheme between users and servers, minimizing the impact of network latency. Finally, a backup key mechanism is added to facilitate the quick return of disconnected users to participate in aggregation, further optimizing the scheme. Based on paired masks, this method has the advantage of low computational cost and has been optimized in terms of interaction rounds. It achieves non-interactive, highly available, and secure aggregation in a server cluster setting. This federated learning method is suitable for distributed systems and real-world asynchronous network environments. It can protect the privacy of gradient information and ensure the correctness of federated learning training results. It is built on an asynchronous network model that is closest to the real-world network model and has high practical application value.

[0010] The objective of this invention is achieved as follows: a federated learning method based on highly available, non-interactive, secure aggregation, characterized by combining a secure aggregation scheme based on pairing masks with a Byzantine consensus algorithm to achieve non-interactive, secure aggregation that guarantees result correctness and privacy, and then implementing federated learning based on this aggregation scheme, specifically including the following steps:

[0011] (A) Initialization phase: Given the system security parameter κ, the number of users n, the number of server nodes N, and the number of malicious server nodes the system can tolerate f (3f+1≤N); the trusted institution generates public parameters for the threshold encryption scheme, a set of public and private key fragments, sends the private key fragments to the corresponding server nodes, and publishes the public key; the trusted institution generates an initial model and publishes the initial model to each user.

[0012] (B) Aggregation phase epoch 0, first round, users: participants set their local model as the initial model, use local data to train the model, and obtain the gradient plaintext vector of this epoch; users generate key pairs, backup key pairs, self-mask seeds, and backup self-mask seeds, encrypt the private key and self-mask seeds (including backup) using a threshold encryption scheme, and send the encrypted ciphertext, public key, and backup public key to all server nodes.

[0013] (C) Aggregation Phase Epoch 0, Round 1: Server: The server node collects the ciphertext, public key, and backup public key uploaded by each user, using them as input to the asynchronous common subset consensus algorithm to achieve consensus. The user set after consensus is denoted as... After reaching a consensus, each user's public key is then sent to all other users.

[0014] (D) Aggregation phase epoch 0, second round, user: The user negotiates the key with the public key of other users and their own private key, expands it into a pair mask using a pseudo-random generator, expands it into a self-mask using a pseudo-random generator according to the self-mask seed, encrypts the plaintext vector using the pair mask and the self-mask, and finally uploads the masked ciphertext.

[0015] (E) Aggregation Phase Epoch 0, Round 2, Servers: Each server collects the masked ciphertext uploaded by users. First, each server samples a portion of the masked ciphertext from users and uses these users as input to the asynchronous common subset consensus algorithm for consensus. After consensus is reached, each server then performs asynchronous common subset consensus on the remaining masked ciphertexts of all users who did not reach consensus. Finally, the union of the user sets from the two consensuses is denoted as... Each node of the server will The symmetric mask private key ciphertext of the user (who quits midway), and The self-mask seed ciphertext of online users is partially decrypted, and the partial decryption results are sent to other server nodes. After receiving a certain number of partial decryption results, each server node can recover the corresponding user's symmetric mask private key and self-mask seed, thereby recovering the paired mask of users who left the server midway, and the self-mask of online users. These are then substituted into a formula to complete the aggregation decryption. After aggregation, each node sends the decryption results and the set of offline users to the server nodes. The system sends its backup public key to each user and discards users who have gone offline. The public key, private key, and self-masked seed ciphertext of the given key are replaced with its backup public key, private key, and self-masked seed ciphertext.

[0016] (F) Aggregation Phase epoch k (k>0): For honest users, the aggregation results of one epoch on the server node are collected. When f+1 identical aggregation results are received, the aggregation result is considered a correct calculation result. Local data is then used to train the model, obtaining the gradient plaintext vector for this epoch. For users whose disconnection and reconnection time is greater than or equal to 2 epochs: a new key pair, backup key pair, self-masking seed, and backup self-masking seed are generated. The private key and self-masking seed are then encrypted using a threshold encryption scheme. The encrypted ciphertext, public key, and backup public key are sent to all server nodes. These users cannot participate in aggregation in this epoch and must wait for the next epoch.

[0017] For users who disconnect and reconnect within one epoch: Discard the original key pair and self-masking seed, and replace them with a backup key pair and backup self-masking seed. Regenerate a new backup key pair and backup self-masking seed, encrypt the backup private key and backup self-masking seed using a threshold encryption scheme, and then upload the ciphertext to the server.

[0018] For all online users (including those who disconnect and reconnect within one epoch): Discard the public key of disconnected user U1 / U2, replace it with their backup public key, update the pairing mask, and encrypt the plaintext vector before uploading.

[0019] (G) Aggregation phase epoch k (k>0), server: consistent with step (E), i.e., the operation of step (E).

[0020] The user uploads their private key information to the server using threshold encryption, eliminating the need to send it to other users and reducing communication rounds and overhead.

[0021] After receiving information uploaded by users, the server cluster uses an asynchronous common subset consensus protocol to unify the view. This two-stage consensus mechanism reduces communication overhead while preventing data loss and achieving high availability. Servers interact to decrypt user mask information and then perform independent calculations. If a user receives the same information from f+1 server nodes, they consider the aggregation result reliable, and no further computation is needed to verify it. By optimizing the interaction by moving it to the previous epoch, the number of communication rounds is further reduced, achieving a non-interactive secure aggregation scheme.

[0022] This invention employs a server cluster setup to achieve high availability. A consensus algorithm is used among the server nodes to unify the view. The server cluster provides redundant computation, ensuring that the majority of honest nodes can cooperate to achieve consistency in the computational view and provide consistent computation results while limiting the number of malicious server nodes. Users receive aggregated results from each server. When f+1 identical aggregated results are received, the result is considered correct and can be used for model updates.

[0023] The system constructed in this invention mainly includes: a server cluster, users, and a trusted third party that generates keys. The server cluster consists of multiple servers in a peer-to-peer relationship. The servers need to reach a consensus on the messages uploaded by users and then independently complete the computation on their respective nodes. High availability is provided through redundant computation. The server nodes are denoted as P. i Let N be the number of nodes. To ensure successful message upload, each user needs to broadcast messages to multiple servers and download the communication results from multiple servers. This incurs additional communication costs for high availability. The paired mask security aggregation scheme also requires communication between users, denoted as C. i The number of nodes is denoted as n.

[0024] The threat model employed in this invention, within a distributed system, exhibits adversary objectives and capabilities largely consistent with the dual-server solution, but with limitations on the number of participants it can control. The security model defines a malicious adversary (or arbitrary adversary). A malicious adversary can violate the agreement and perform arbitrary actions. (Arbitrary adversary) There are two attack objectives: (i) to calculate the gradient privacy information of honest clients; and (ii) to forge a fake aggregation result and successfully pass verification; and (iii) to define any adversary. The capabilities are as follows:

[0025] 1. It can eavesdrop on all channels and obtain messages during communications between all parties.

[0026] 2. Static Byzantine Fault Node: It can control up to f server nodes, at which point the number of server nodes must be at least 3f+1.

[0027] 3. There are at least two users who are not controlled by the adversary.

[0028] 4. The attack capability is limited to polynomial time.

[0029] Compared with existing technologies, this invention improves user experience and data utilization, achieves high availability, privacy protection, and correctness of aggregation results, and can resist malicious adversaries. The method is based on pairing masks, which has the advantage of low computational cost, and optimizes the number of interaction rounds. It achieves non-interactive, highly available, and secure aggregation in a server cluster setting. This federated learning is suitable for distributed systems and real-world asynchronous network environments, can protect the privacy of gradient information, and ensures the correctness of federated learning training results. It is built on an asynchronous network model that is closest to real-world network models and has high practical application value. Detailed Implementation

[0030] I. Explanation of the mathematical theory applied in this invention:

[0031] 1. Key Negotiation

[0032] Local devices perform pairwise Diffie-Hellman key negotiation. The key negotiation steps are as follows: First, determine the public cyclic group G, with generator g. Device C i Randomly generate the negotiated private key sk locally. i Calculate the negotiated public key And publicly disclose its negotiated public key. Device C j Received device C i public key pk i Then, the negotiation result was calculated. Thus, C i With C j A consensus was reached through successful negotiation. ij Furthermore, this result is not known to outsiders. The Key Agreement (KA) scheme comprises three parts: an initialization function, a key generation function, and a negotiation function, denoted as KA.Setup, KA.KeyGen, and KA.Agree, respectively, as shown below:

[0033] 1) Initialization function KA.Setup(1 κ →KApp: The algorithm takes the safety parameter κ as input and outputs the common parameter KApp. For the sake of brevity, KApp is omitted in the functions below.

[0034] 2) Key generation function: KA.KeyGen()→(pk, sk): The algorithm generates a public-private key pair (pk, sk).

[0035] 3) Negotiation function KA.Agree(sk i , pk j )→ak i，j The algorithm is based on two non-corresponding public and private keys (sk i , pk jNegotiate the negotiation key ak i，j The negotiated key satisfies the following properties:

[0036] ak i，j =KA.Agree(sk i , pk j =KA.Agree(sk j , pk i ).

[0037] 2. Public Key Threshold Encryption Scheme

[0038] The Threshold Public Key Encryption (TPKE) scheme consists of four parts: an initialization function, an encryption function, a fragment decryption function, and a decryption function, denoted as TPKE.Setup, TPKE.Enc, TPKE.DecShare, and TPKE.Dec, respectively, as shown below:

[0039] 1) TPKE.Setup(1 λ →PK, {SK i The algorithm generates a system master public key PK for encryption and generates private key fragments SK for N decryption parties. i .

[0040] 2) TPKE.Enc(PK, m) → C: The encryptor uses the public key PK to encrypt the plaintext m to obtain the ciphertext C.

[0041] 3) TPKE.DecShare(SK i C)→σ i : Participants in decryption Decrypt the ciphertext portion, generate the i-th decrypted fragment and output it (if C is forged, it cannot be decrypted correctly, and the algorithm outputs ⊥).

[0042] 4) TPKE.Dec(PK, C, {i, σ} i})→m: If at least t decryption fragments σ are collected i If the plaintext m is output (if there is an invalid decryption fragment that does not correspond to C, the algorithm will identify it).

[0043] 3. Asynchronous Common Subset Consensus Protocol

[0044] ACS is implemented using the RBC and ABA schemes. The Asynchronous Byzantine Binary Agreement (ABA) is used to allow participants to negotiate a binary value to achieve consensus among all parties.

[0045] The ABA protocol satisfies the following three characteristics:

[0046] 1) Consistency: If a correct node outputs the bit value b, then all correct nodes will output b.

[0047] 2) Termination: If an honest node participates in the consensus, then a consensus result will definitely be obtained.

[0048] 3) Validity: If all honest nodes take b as input, then the final output will also be b.

[0049] In an asynchronous network, the Asynchronous Common Set (ACS) consensus protocol provides a set U to each participant. i As input, then consensus is reached to produce a set.

[0050] The Reliable Broadcast Channel (RBC) protocol solves the problem of broadcast consistency. Its main principle is based on the echo forwarding mechanism and the principle of majority rule. It unifies broadcast messages based on the broadcast messages received by the majority of honest parties, and satisfies the following characteristics.

[0051] 1) Consistency: (Even if the sender is malicious) If two correct nodes deliver two values ​​v and v′ respectively, then v = v′.

[0052] 2) Validity: If the sender is correct and has sent the value v, then all correct nodes should deliver v.

[0053] The ACS protocol execution process is as follows, by the node Parallel execution:

[0054] 1) Let {RBC} i} N For N instances of the reliable broadcast protocol, where RBCi represents the broadcast protocol sent by the sender. The instance that was initiated. Similarly, let {ABA} i} N For N instances of the asynchronous Byzantine binary consensus protocol, where ABAi represents the node-to-node relationship. A consensus on whether the provided set can be accepted.

[0055] 2) Receive input v from the user i Initiate instance RBC i , will v i As input for an instance.

[0056] 3) When the instance RBC is obtained j The output result v jIf no instance v was previously created j Given the input, then take 1 as ABA. j Input.

[0057] 4) When Nf (3f+1≤N) ABA instances with a consensus result of 1 are obtained, input 0 is provided for all ABA instances that have not provided input.

[0058] 5) Once all ABA instances have completed, denote set C as the set of indices of all ABA consensus results that are 1. Wait for all instances to complete RBC. j Once j∈C is completed, calculate and output v=U j∈C v j .

[0059] To understand the technical means, creative features, objectives, and effects of this invention, the invention will be further described below in conjunction with specific embodiments.

[0060] Example 1

[0061] This invention employs a secure aggregation method based on paired masks. Through an asynchronous common subset consensus algorithm, it achieves non-interactive, highly available, secure aggregation federated learning in a distributed server cluster setting. Specifically, it includes: (A) initialization phase; (B) aggregation phase epoch 0, first round, user; (C) aggregation phase epoch 0, first round, server; (D) aggregation phase epoch 0, second round, user; (E) aggregation phase epoch 0, second round, server; (F) aggregation phase epoch k (k>0), user; (G) aggregation phase epoch k (k>0), server. Users and servers calculate and upload different information based on their own disconnection status in different aggregation phases. Single-server and dual-server architectures pose a risk of single point of failure. In scenarios with ample resources or high reliability requirements, a distributed system architecture is necessary. Therefore, this invention proposes a federated learning scheme under a distributed system, with a highly available, non-interactive, secure aggregation scheme at its core. This scheme uses a Byzantine consensus algorithm to ensure the correct execution of the protocol by the server cluster, thereby guaranteeing the correctness of the aggregation results. Furthermore, it achieves secure data aggregation through pairing masks, thus protecting the privacy of the aggregation results. We combine these two approaches and optimize the number of communication rounds and communication overhead to implement a non-interactive, secure aggregation scheme between users and servers, minimizing the impact of network latency. Finally, a backup key mechanism is added to facilitate the quick return of disconnected users to participate in aggregation, further optimizing the scheme.

[0062] The initialization phase (A) is as follows: given the system security parameters, the number of users and the number of server nodes, the trusted institution generates threshold encryption public parameters and a set of public and private key fragments, and sends the private key fragments to the corresponding server nodes respectively; the public key is made public, the trusted institution generates an initial model and makes it public to each user.

[0063] The given system security parameters are as follows:

[0064] A1: Let the security parameter be κ, the number of users be n, the number of server nodes be N, and the upper limit of the number of malicious server nodes be f. Let the threshold encryption scheme TPKE have a threshold of f+1, satisfying the condition N≥3f+1.

[0065] The trusted institution generates the following public parameters for threshold encryption:

[0066] A2: Generate the initial model W <0> and W <0> It is made available to all users.

[0067] A3: Generate the public key PK for the threshold encryption scheme. TP With private key Will Send to the corresponding node

[0068] In the aggregation phase (B), epoch 0, the first round, the user and participating parties set their local model as the initial model, use local data to train the model, and obtain the gradient plaintext vector for this epoch. The user generates a key pair, a backup key pair, a self-masking seed, and a backup self-masking seed. The user encrypts the private key and the self-masking seed (including the backup) using a threshold encryption scheme, and sends the encrypted ciphertext, public key, and backup public key to all server nodes.

[0069] In step (B), the user performs the following local calculations:

[0070] B1: Set the local model W <0> =W <0> .

[0071] B2: Train the local model W<> using your own local data to obtain the gradient vector v<>.

[0072] The user calculates the public / private key and self-masking seed as follows:

[0073] B3: User generates key pair With backup key pair Generate self-masking seed b and alternative self-masking seed in

[0074] B4: Encrypt the private key and the self-mask seed to obtain... Encryption result Public Key Backup public key Send to all server nodes.

[0075] (C) Aggregation Phase Epoch 0, Round 1: Server: The server node collects the ciphertext, public key, and backup public key uploaded by each user, using them as input to the asynchronous common subset consensus algorithm to achieve consensus. The user set after consensus is denoted as... After reaching a consensus, each user's public key is then sent to all other users.

[0076] In step (C), the server performs the following operations:

[0077] C1: Server nodes collect the data uploaded by each user. Then, it is used as input to the ACS algorithm for consensus. The user set obtained after consensus is denoted as... Each server will use the user's public key Send to All users in the country.

[0078] (D) Aggregation phase epoch 0, second round, user: The user negotiates the key with the public key of other users and their own private key, expands it into a pair mask using a pseudo-random generator, expands it into a self-mask using a pseudo-random generator according to the self-mask seed, encrypts the plaintext vector using the pair mask and the self-mask, and finally uploads the masked ciphertext.

[0079] In step (D), each user performs the following calculations:

[0080] D1: User uses a pairing mask in Self-masking PRG(b) i ), for plaintext vector v i Encryption, y i =x i +p i +∑ i＜j P i，j -∑ i＜j P i，j , will y i Upload.

[0081] (E) Aggregation Phase, epoch 0, second round: Servers: Each server collects the masked ciphertext uploaded by users. First, each server samples a portion of the users' masked ciphertext and uses these users as input to the asynchronous common subset consensus algorithm for consensus. After consensus is reached, each server then performs asynchronous common subset consensus on the remaining masked ciphertext of all users who did not reach consensus. After consensus is reached, the union of the user sets from the two consensuses is denoted as... Each node of the server will The symmetric mask private key ciphertext of the user (who quits midway), and The self-mask seed ciphertext of online users is partially decrypted, and the partial decryption results are sent to other server nodes. After receiving a certain number of partial decryption results, each server node can recover the corresponding user's symmetric mask private key and self-mask seed, thereby recovering the paired mask of users who left the server midway, and the self-mask of online users. These are then substituted into a formula to complete the aggregation decryption. After aggregation, each node sends the decryption results and the set of offline users to the server nodes. The system sends its backup public key to each user and discards users who have gone offline. The public key, private key, and self-masked seed ciphertext of the given key are replaced with its backup public key, private key, and self-masked seed ciphertext.

[0082] In step (E), the server performs the following operations:

[0083] E1: The server collects user-uploaded data. i First, a random sampling of b / N ciphertext is used as input to the ACS algorithm for consensus. Then, all remaining ciphertext that did not reach consensus is grouped into a set and input into the ACS algorithm again for consensus. The union of the user sets obtained after the two consensus processes is denoted as . In E1, the sampling algorithm probability b / N, when the number of malicious adversaries is at its maximum, gives the following approximate probability of a ciphertext being sampled by all users. To achieve consensus on all ciphertexts within two consensus rounds, thus minimizing time consumption and reducing the outward communication overhead for a single user to the greatest extent, we ensure that the number of samples in the first consensus round is consistent with the remaining number in the second consensus round, and that no further sampling is performed in the second consensus round, resulting in the following equation (a):

[0084]

[0085] According to equation (a) above, an approximate value of b can be calculated when N is determined. For example, when N=8, b=2 can minimize the upper limit of outward communication between the two consensuses.

[0086] E2: Each node of the server will Symmetric mask private key ciphertext of the user who quits midway and Self-masked seed ciphertext of online users Partial decryption, Then partially decrypted result e t Send to other server nodes.

[0087] E3: The server has received at least f+1 e t Decryption yields and Recover the corresponding self-mask Pairing mask Substitute the results into equation (b) below to aggregate and decrypt them.

[0088]

[0089] E4: The server will decrypt the result z, the set of disconnected users. and the backup public keys of all users in the set Send to each user. If a user's backup public key has already been used and no new backup public key has been uploaded, then the public key will be represented by NULL.

[0090] E5: The server discards users who have gone offline. Replace with

[0091] Encrypt the backup private key and backup self-mask seed using a threshold encryption scheme, and then upload the ciphertext to the server. For all online users (including users who disconnected and reconnected within one epoch): discard the public key of the disconnected user U1 / U2 and replace it with their backup public key. The following steps are consistent with the second round of the aggregation phase epoch 0.

[0092] (F) Aggregation Phase epoch k (k>0): For honest users, when they receive f+1 identical aggregation results from the server nodes for one epoch, they consider the aggregation result to be correct. Then, they use local data to train the model and obtain the gradient plaintext vector for this epoch. For users whose disconnection and reconnection time is greater than or equal to 2 epochs: A new key pair, backup key pair, self-masking seed, and backup self-masking seed are generated. The private key and self-masking seed are then encrypted using a threshold encryption scheme. The encrypted ciphertext, public key, and backup public key are sent to all server nodes. These users cannot participate in aggregation in this epoch and must wait for the next epoch. For users who disconnect and reconnect within one epoch: The original key pair and self-masking seed are discarded and replaced with a backup key pair and backup self-masking seed. A new backup key pair and backup self-masking seed are generated. The backup private key and backup self-masking seed are encrypted using a threshold encryption scheme, and the ciphertext is uploaded to the server. For all online users (including those who disconnect and reconnect within one epoch): Discard the public key of disconnected user U1 / U2, replace it with their backup public key, update the pairing mask, and encrypt the plaintext vector before uploading.

[0093] In step (F), each user performs the following calculations:

[0094] F1: The honest user collects the aggregated results from the server nodes. <k-1>When f+1 identical aggregation results are received, the aggregation result is considered to be a correct calculation result.

[0095] F2: The aggregation result is the aggregated gradient vector v. <k-1> ＝R <k-1>Calculate the average gradient vector based on the aggregated gradient vector. Finally, the local model is updated based on the learning rate α of this round.

[0096] F3: Use your own local data to create a local model W i <k>The training yields the gradient vector v. i <k>.

[0097] Users whose disconnection and reconnection time is greater than or equal to 2 epochs:

[0098] F4: Regenerate Public Key Backup public key Once uploaded to the server, aggregation can no longer be performed in this epoch, so the process jumps to the next epoch.

[0099] For users who disconnect and reconnect within one epoch: discard the original connection. Replace with And regenerate a new one

[0100] F5: Same as step B4.

[0101] All online users (including those who disconnect and reconnect within one epoch):

[0102] F6: Discard the public keys of offline users U1 and U2 Replace it with its backup public key

[0103] F7: Recalculate the p-value between the disconnected user and the user. i，j For plaintext vector v i Encryption, y i =x i +p i +∑ i＜j p i，j -∑ i＜ j p i，j , will y i Upload.

[0104] (G) Aggregation phase epoch k (k>0), server:

[0105] In step (G), the server operation is the same as in (E).

[0106] The above embodiments are merely illustrative of the present invention and are not intended to limit the scope of the present invention. All equivalent implementations of the present invention should be included within the scope of the claims of the present invention.< / k> < / k> < / k-1>

Claims

1. A federated learning method based on high-availability non-interactive secure aggregation, characterized in that, A secure aggregation method based on pairing mask is adopted, and through an asynchronous public subset consensus algorithm, a non-interactive high-availability secure aggregation federated learning is realized on a distributed server cluster, which specifically includes the following steps: (A) initialization stage Given system security parameters , number of users , number of server nodes , and number of malicious server nodes that the system can tolerate , the trusted authority generates public parameters and a set of public key and private key shards for threshold encryption, and sends the private key shards to the corresponding server nodes respectively; the public key is disclosed, and the trusted authority generates an initial model and discloses it to each user; (B) aggregation stage epoch 0 first round of users The participants set the local model as the initial model, train the model by using the local data, obtain the gradient plaintext vector of the current epoch, generate a key pair, a backup key pair, a self-mask seed and a backup self-mask seed, encrypt the private key, the self-mask seed and the backup self-mask seed by using a threshold, and then send the encrypted ciphertext, the public key and the backup public key to all server nodes; (C) aggregation stage epoch 0 first round of servers The server node collects the ciphertext, the public key and the backup public key uploaded by each user as the input of the asynchronous public subset consensus algorithm for consensus, and sends the public key of each user to all other users after consensus, and records the user set after consensus as ; (D) aggregation stage epoch 0 second round of users The users perform key negotiation according to the received public keys of other users and the private key of the user, expand the pairing mask by using a pseudo-random generator, expand the self-mask by using a pseudo-random generator according to the self-mask seed, encrypt the plaintext vector by using the pairing mask and the self-mask, and finally upload the mask ciphertext; (E) aggregation stage epoch 0 second round of servers Each server collects the user uploaded mask ciphertext, first each sample part of the user's mask ciphertext, these users as asynchronous public subset consensus algorithm input consensus, consensus after the remaining all users mask ciphertext consensus of the user not consensus, consensus after the two consensus user set union set as , on behalf of the online user set; midway out of the user set is , the server will All users of the symmetric mask private key ciphertext, and All users of the self-mask seed ciphertext are partially decrypted, and then the partial decryption result is sent to other server nodes; after receiving the partial decryption result, each server node can recover the corresponding user's symmetric mask private key and self-mask seed, thereby recovering the pairing mask of the midway out of the user and the self-mask of the online user, which is substituted into the following (b) formula to aggregate and decrypt the result: （b）； wherein, is a user uploaded mask ciphertext; is a user self-mask; is a pairing mask when , ; when , ; when , ; After the aggregation and decryption are completed, each node will send the decryption results and the set of disconnected users to the network. The system sends its backup public key to each user and discards users who have gone offline. The public key, private key, and self-masked seed ciphertext are replaced with the spare public key, private key, and self-masked seed ciphertext. (F) aggregation stage epoch k(k>0) of users When honest users collect data from each server node If the same aggregation result is obtained for multiple epochs, the aggregation result is considered a correct calculation result. This result is then used to train the model using local data to obtain the gradient plaintext vector for this epoch. For users whose disconnection and reconnection time is greater than or equal to two epochs: a new key pair, backup key pair, self-masking seed, and backup self-masking seed are generated. The private key and self-masking seed are then encrypted using threshold encryption. The encrypted ciphertext, public key, and backup public key are sent to all server nodes. These users cannot participate in aggregation in this epoch and must wait for the next epoch. For users who disconnect and reconnect within one epoch: the original key pair and self-masking seed are discarded and replaced with a backup key pair and backup self-masking seed. A new backup key pair and backup self-masking seed are generated, and the backup private key and backup self-masking seed are encrypted using threshold encryption before the ciphertext is uploaded to the server. For all online users, including those who disconnect and reconnect within one epoch: disconnected users are discarded. Replace the public key with the backup public key, update the pairing mask, and encrypt the plaintext vector before uploading; (G) aggregation stage epoch k(k>0) of servers In step (G), the server operation is the same as that in step (E).

2. The federated learning method based on high-availability non-interactive secure aggregation according to claim 1, characterized in that, After receiving the information uploaded by the users, the server cluster adopts an asynchronous public subset consensus protocol to unify the view and a twice consensus mechanism.

Images