A federated learning secure aggregation method and system based on MDS coding

Abstract

The application discloses a kind of federated learning security aggregation methods and systems based on MDS coding.The method is based on federated learning computing framework based on security aggregation, using data encryption, data splitting etc., ensure that user participates in federated learning training while, private data is not disclosed.At the same time, the method solves the problem of invalid aggregation result caused by user drop-out in federated learning based on security aggregation through server scheduling, MDS coding etc., ensures the availability of final aggregation result, and improves the robustness of the whole system.Compared with the existing method, the method has lower computing overhead, improves the learning efficiency of federated learning.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of federated learning, and particularly relates to a federated learning secure aggregation method and system based on MDS coding. BACKGROUND

[0002] Federated learning is a kind of distributed machine learning, and its core idea is that multiple users with local data complete the training of a global model by exchanging only models and gradients without exchanging local data. The advantage of federated learning is that it can train a model by using local data of multiple different users while protecting the privacy of the user data, thereby achieving better model training effect. It can be said that federated learning is a distributed machine learning computing framework that takes into account both privacy protection and model training effect.

[0003] However, although users do not need to exchange local data under the computing framework of federated learning, the users will upload the updated gradients calculated by themselves in each round of federated learning, and the original data information of the users is contained in these gradients. Research shows that the original data of the users can be easily recovered from the gradient values uploaded by the users by using means such as an adversarial generative network, and therefore additional privacy protection means need to be introduced in the computing framework of federated learning to protect the data privacy of the users.

[0004] The existing privacy protection of federated learning mainly falls into three categories: differential privacy, trusted execution environment and secure aggregation. Among them, differential privacy means that the user adds noise subject to a certain distribution to cover his own original data when uploading the gradient, for example, adding noise subject to Gaussian distribution, and when there are enough users uploading data, the noise is cancelled out because it is subject to Gaussian distribution, so it will not affect the final result. Its advantage is that the calculation and communication overhead is small, but its disadvantage is that it lacks strict privacy protection effectiveness, and the attacker can still recover the original data from the uploaded data, and the method of differential privacy needs the participation of a large number of users to eliminate the influence of noise on the final result, otherwise it will affect the learning effect of the model. The trusted execution environment refers to an independent processing environment with computing and storage functions that can provide security and integrity protection. The basic idea is to allocate a separate memory for sensitive data in the hardware, and all sensitive data calculations are performed in this memory, and no other part of the hardware can access the information in this isolated memory except the authorized interface. In this way, the privacy calculation of sensitive data is realized, and the advantage is that the calculation and communication overhead is small, and the disadvantage is that the implementation depends on the underlying hardware architecture and lacks a unified industry standard. Secure aggregation means that the user adds a mask to protect the privacy of the local data, and the sum of all user masks is 0, so the server can recover a useful result. The work in the field of secure aggregation has the strongest privacy protection effectiveness and is not constrained by the number of users, but the disadvantage is that the calculation and communication cost is high, and in the framework of secure aggregation, user dropouts will have a very serious impact, and the problem of user dropouts is unavoidable in a distributed system. SUMMARY

[0005] In view of the above problems of the prior art, the purpose of the present application is to provide a federated learning training method based on secure aggregation, which can protect user data privacy while reducing calculation overhead by combining MDS encoding, and solve the problem of affecting the learning effect due to user dropouts in federated learning based on secure aggregation.

[0006] The present application also provides a secure aggregation federated learning training system based on MDS encoding.

[0007] In order to achieve the above-mentioned purpose of the application, the following technical solutions are adopted:

[0008] In the first aspect, a secure aggregation federated learning training method based on MDS encoding comprises the following steps:

[0009] (1) User i generates a pair of public and private keys locally The public key is The public keys are sent to the server. After receiving the public keys of at least n1 online users, the server will send the collected public key set S. pk Distribute to all online users, n1≤n, where n is the number of users, and (n-n1) / n>t, where t is the maximum disconnection rate allowed by the system;

[0010] (2) User i uses their private key and the public keys of other users collected Calculate the shared key S between yourself and other users. i，j ;

[0011] (3) User i splits the gradient values ​​that he / she needs to upload and obtains several data blocks through MDS encoding. And using the previously calculated shared key S i，j For data blocks Encryption, resulting in encrypted data blocks. Then encrypt the data block Uploaded to the server;

[0012] (4) After the server collects encrypted data blocks from at least n2 online users, it distributes these encrypted data blocks to the corresponding users, where n2≤n. After receiving the encrypted data block, user i uses the shared key S. i，j Decrypt and sum the results before uploading to the server;

[0013] (5) After collecting data from no less than t users, the server obtains the aggregation gradient value for this round by decoding, completes the safe aggregation for this round, and uses the obtained aggregation gradient to update the model.

[0014] Furthermore, in step (1), user i uses the KA.gen algorithm in the KeyAgreement protocol to generate a public-private key pair. and public key Uploaded to the server, the server collects the public keys of at least n1 users, denoted as set N1, which is the set of public keys of all online users. The server will use the public key set S pk Send to all online users.

[0015] Furthermore, in step (2), user i collects the public key set S of other users. pk Then, the KA.agree algorithm in the KeyAgreement protocol is used to generate the result. S i，j This is the shared key between user i and user j. For user j, the public key and the shared key S i，j It has the property: S i，j =Sj，i .

[0016] Further, step (3) includes:

[0017] User i randomly generates t-1 random numbers. And calculate Where a i The original gradient is calculated for user i, and then n1 new encoded data blocks are generated from these t numbers using (n1, k) MDS codes, denoted as set t. The (n1, k) MDS code refers to encoding the task into n1 parts, from which any k parts can be used to recover the original task; all users use the same encoding method, for each encoded data block Encrypting it using the corresponding shared key yields n1 new encrypted data blocks, denoted as set. User i will set C i Send to the server.

[0018] Further, in step (4), the server collects encrypted data block sets from at least n² users, and (n-n²) / n > t. These users are denoted as set N². The server then reassembles the collected encrypted data blocks to obtain n² integrated encrypted data block sets. Where set D i The elements in are composed of The server will collect D i Send to the corresponding user i.

[0019] Furthermore, in step (4), user i applies the formula to the received encrypted data block. Decrypt and calculate. m i Uploaded to the server.

[0020] Further, step (5) includes:

[0021] The server collects data uploaded by at least t users, and then uses the decoding matrix to recover the aggregated result x of this round of federated learning, where a i The original gradient calculated for user i.

[0022] Secondly, a secure aggregation federated learning training system based on MDS encoding includes a server node and several user nodes. The server node possesses the initial federated learning model, and the user nodes possess the training data required for federated learning. In each round of federated learning, the user nodes are configured to: train the model distributed by the server using local data; and generate a public-private key pair locally. public key Send to the server; receive public keys from other users distributed by the server, and use your own private key. and the public keys of other users collected Calculate the shared key S between yourself and other users. i，j ; Split the gradient values ​​that need to be uploaded into several data blocks and encode them using MDS. And using the shared key S i，j For data blocks Encryption, resulting in encrypted data blocks. Then encrypt the data block Uploaded to the server; and, after receiving the encrypted data block from the server, according to the shared key S i，j Decrypt and sum the results before uploading to the server;

[0023] The server is configured to: distribute the latest model to users; and, after receiving the public keys of at least n1 online users, distribute the collected public key set S. pk Distribute the data to all online users, where n1 ≤ n, n is the number of users, and (n-n1) / n > t, where t is the maximum allowed disconnection rate of the system; after collecting encrypted data blocks from at least n2 online users, distribute these encrypted data blocks to the corresponding users, where n2 ≤ n, and (n-n2) / n > t; after collecting data from no less than t users, decode the data to obtain the aggregation gradient value for this round, complete the secure aggregation for this round, and use the obtained aggregation gradient to update the model.

[0024] Beneficial effects: This invention addresses federated learning based on secure aggregation. It utilizes data splitting to complete the federated learning process while protecting user privacy. Furthermore, it employs MDS encoding to resolve issues arising from user disconnections during federated learning within the secure aggregation framework. Compared to other secure aggregation-based federated learning methods, this invention has lower computational overhead. Moreover, under the same user disconnection rate, this method achieves higher utilization of disconnected users, significantly improving the training effect of federated learning while protecting user privacy and reducing the overall computational cost of the learning process. Attached Figure Description

[0025] Figure 1 This is the overall flowchart of the secure aggregation federated learning method based on MDS encoding of the present invention;

[0026] Figures 2(a) to 2(c) This is a schematic diagram illustrating the working process of the secure aggregation federated learning system based on MDS encoding according to the present invention;

[0027] Figure 3 This is a schematic diagram of data transmission during the secure aggregation process of the present invention. Detailed Implementation

[0028] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to specific embodiments and accompanying drawings.

[0029] Federated learning systems typically consist of a server node and several user nodes. The server node possesses the initial federated learning model, while the user nodes possess the training data required for federated learning. In this invention, the server node is also referred to as the server, and the user nodes as users. In each round of federated learning, the server distributes the latest model to the users, who then train the model using their local data. After completing one round of training, the users obtain the latest local update gradient, which the server then aggregates to update the model. However, the data security of the interaction between the two parties is a crucial issue. To address privacy protection and potential user disconnection during aggregation, this invention proposes a secure aggregation federated learning training method based on MDS encoding. After all users complete one round of training, a secure aggregation process begins. Users split, encode, and encrypt their local update gradients before uploading them to the server. The server collects the data and distributes it to the corresponding users. Users then integrate the received data, sum it, and upload it to the server. The server decodes the data to obtain the aggregated gradient for this round and uses the aggregated gradient to update the model, completing this round of federated learning. This process is repeated until the model converges.

[0030] The concept of this invention is based on considerations of the following characteristics of server nodes: (1) The server is semi-trustworthy, that is, the server may infer the user's original data from the user's uploaded data, but will not deviate from the protocol process, nor will it engage in malicious acts, such as maliciously tampering with user data, sending incorrect data or models to the user, etc. (2) The server is stable, and the server will not experience problems such as disconnection or failure during the entire federated learning process.

[0031] According to an embodiment of the present invention, the server is mainly responsible for: (1) sending the latest model to the user; (2) undertaking the task of transmission scheduling during the secure aggregation process, and acting as a relay station between user communications; and (3) calculating the aggregation gradient and using the aggregation gradient to update the model.

[0032] The concept of this invention is also based on consideration of the following characteristics of user nodes: (1) Users are honest, that is, users will not try to deduce other users' original data through the data they obtain, nor will they deviate from the protocol process or engage in malicious acts. (2) User nodes may disconnect during the protocol process. User disconnection may occur at any stage of the protocol. Once a user disconnects, it is considered to be permanently invalid in this round of the protocol and will not participate in the subsequent process of this round of the protocol.

[0033] According to an embodiment of the present invention, the user is mainly responsible for: (1) training the model using local data to obtain local updated gradients; (2) splitting, encoding, encrypting and re-integrating the data; and (3) sending the integrated gradient values ​​to the server.

[0034] like Figure 1 As shown, the present invention proposes a secure aggregation federated learning training method based on MDS encoding, which includes the following steps:

[0035] Step S1: The server sends the latest model to the user;

[0036] Step S2: The user trains the model using local data and obtains the local update gradient;

[0037] Step S3: The user generates a public-private key pair and sends the public key to the server;

[0038] According to an embodiment of the present invention, user i uses the KA.gen algorithm in the KeyAgreement protocol to generate a public-private key pair. and public key Send to the server;

[0039] Step S4: The server combines the public key and sends it to the user, who then calculates the shared key.

[0040] According to an embodiment of the present invention, the server collects the public keys of at least n1 users, denoted as set N1, where n1 ≤ n, n is the number of users, and (n-n1) / n > t, where t is the maximum allowed disconnection rate of the system. At this point, the system obtains the set of public keys of all online users. The server will store the public key set Send to all online users.

[0041] According to an embodiment of the present invention, user i obtains a shared key using the KA.agree algorithm in the KeyAgreement protocol, wherein the shared key S between user i and user j is... i，j The calculation method is as follows:

[0042] Step S5: The user splits, encodes, and encrypts the local gradient before sending it to the server.

[0043] According to an embodiment of the present invention, the user splits the original gradient by: user i randomly generates t-1 random numbers. And calculate Where a i The original gradient is calculated for the user. The database is then split into t parts.

[0044] According to an embodiment of the present invention, encrypting encoded data by a user includes: user i using (n1, k) MDS codes to generate n1 new encoded data blocks from t split data blocks, where the (n1, k) MDS code refers to encoding the task into n1 parts, and any k parts can be used to recover the original task. Let be denoted as set. All users use the same encoding method.

[0045] According to an embodiment of the present invention, encrypting the encoded database by a user includes: user i for each encoded data block Encrypting it using the corresponding shared key yields n1 new encrypted data blocks, denoted as set. User i will set C i Send to the server.

[0046] Step S6: The server reassembles the collected data and sends it to the corresponding user.

[0047] According to an embodiment of the present invention, the server collects a set of encrypted data blocks from at least n² users, where n² ≤ n and (n-n²) / n > t. These users are denoted as set N². The server then reassembles the collected encrypted data blocks to obtain n² integrated sets of encrypted data blocks. Where set D i The elements in are composed of The server will collect D i Send to the corresponding user i.

[0048] Step S7: The user decrypts the received data, sums it, and sends it to the server.

[0049] According to an embodiment of the present invention, user i uses the formula to process the received encrypted data block. Decrypt and calculate. m i Uploaded to the server.

[0050] In step S8, the server decodes the aggregated gradients and updates the model.

[0051] According to an embodiment of the present invention, the server collects data uploaded by at least t users, and then uses a decoding matrix to recover the aggregated result x of this round of federated learning, wherein... The server updates the model using aggregated gradient x, repeating steps S1-S8 until the model converges.

[0052] Figures 2(a) to 2(c)The diagram illustrates an example of the working process of the secure aggregated federated learning system based on MDS encoding in this invention. Figure 2(a) shows the data preparation stage, where the main task is to generate a shared key between any two users to prepare for subsequent data transmission. Figure 2(b) shows the data distribution stage, where users process their local update gradients and upload the processed data blocks to the server. The server collects all users' data blocks and sends each data block to the corresponding user. Figure 2(c) shows the data upload stage, where users decrypt the received data blocks and upload the summed result to the server. The server recovers the aggregated gradient value using the decoding matrix and uses the aggregated gradient to update the model, completing this round of federated learning.

[0053] Figure 3 This diagram illustrates data transmission during the secure aggregation process of the present invention. (A1, A2...A...) n (a1, a2, ..., an) represents the n user nodes participating in federated learning. n () represents the local update gradient of each user. Each user decomposes their own local update gradient. Taking user A1 as an example, it decomposes its local update gradient a1 into () Each row in the diagram represents a data block obtained after each user completes the data splitting process. The data is then distributed among users. Taking user A1 as an example, the data blocks... Keep it yourself, data block Data block sent to user A2 Send to user A n Each user performs the same operation. Each column in the diagram represents the data received by each user after the data distribution is complete. Each user sums up their newly received data to obtain (m1, m2...m...). n The final value uploaded to the server is the value of m, which does not reveal any user's original data. However, since ∑m = ∑a, the value of ∑m can be used for gradient updates in federated learning, thus completing the entire training process of federated learning without leaking user privacy.

[0054] Based on the description of the above method embodiments, the secure aggregation federated learning training system based on MDS encoding provided by the present invention configures user nodes to: train a model distributed by a server using local data; and generate a public-private key pair locally. public key Send to the server; receive public keys from other users distributed by the server, and use your own private key. and the public keys of other users collected Calculate the shared key S between yourself and other users. i，j ; Split the gradient values ​​that need to be uploaded into several data blocks and encode them using MDS. And using the shared key S i，j For data blocks Encryption, resulting in encrypted data blocks. Then encrypt the data block Uploaded to the server; and, after receiving the encrypted data block from the server, according to the shared key S i，j Decrypt and sum the results before uploading to the server;

[0055] The server is configured to: distribute the latest model to users; and, after receiving the public keys of at least n1 online users, distribute the collected public key set S. pk Distribute the data to all online users, where n1 ≤ n, n is the number of users, and (n-n1) / n > t, where t is the maximum allowed disconnection rate of the system; after collecting encrypted data blocks from at least n2 online users, distribute these encrypted data blocks to the corresponding users, where n2 ≤ n; after collecting enough data, decode the data to obtain the aggregation gradient value for this round, complete the secure aggregation for this round, and use the obtained aggregation gradient to update the model.

[0056] This invention utilizes data splitting to complete the federated learning process while protecting user privacy. It also uses MDS encoding to solve the problem of user disconnection in federated learning under the secure aggregation framework, while having low computational overhead. Furthermore, under the same user disconnection rate, this invention has a higher utilization rate for disconnected users. It greatly improves the training effect of federated learning while protecting user privacy and reduces the computational overhead of the entire learning process.

Claims

1. A federated learning secure aggregation method based on MDS coding, characterized in that, The method comprises the following steps: (1) User i generates a public-private key pair locally. public key The public keys are sent to the server. After receiving the public keys of at least n1 online users, the server will collect the public key set. Distribute to all online users, n1≤n, where n is the number of users, and (n-n1) / n>t, where t is the maximum disconnection rate allowed by the system; (2) User i computes the shared key S between himself and other users according to his own private key and the public keys of other users collected i,j ;​​ (3) User i splits the gradient value he needs to upload and gets several data blocks through MDS encoding and uses the shared key S calculated before i,j Encrypts the data blocks , and gets encrypted data blocks Then uploads the encrypted data blocks to the server; (4) The server distributes the encrypted data blocks to the corresponding users after collecting them from at least n2 online users, n2≤n, and user i decrypts and sums the received encrypted data blocks according to the shared key S i,j and uploads the sum to the server. (5) The server obtains the aggregated gradient value of the round by decoding after collecting data of no less than t users, completes the secure aggregation of the round, and updates the model using the obtained aggregated gradient.

2. The method of claim 1, wherein, In step (1), the user i generates a pair of public and private keys using the KA.gen algorithm in the Key Agreement protocol and uploads the public key to the server. The server collects the public keys of at least n1 users, denoted as set N1, and the set of public keys of all online users The server sends the set of public keys to all online users.

3. The method of claim 1, wherein, In the step (2), the user i collects the set of public keys of other users After that, the KA.agree algorithm in the KeyAgreement protocol is used to generate S i,j is the shared key between the user i and the user j, is the public key of the user j, and the shared key S i,j has the property: S i,j = S j,i .

4. The method of claim 1, wherein, The step (3) comprises: User i randomly generates t-1 random numbers and calculates where a i is the original gradient calculated by user i, and then uses an (n1, k) MDS code to generate n1 new encoded data blocks for the t numbers, denoted as a set where the (n1, k) MDS code means that the task is encoded into n1 parts, and any k parts can be used to recover the original task; all users use the same encoding method, and for each encoded data block uses the corresponding shared key to encrypt it to obtain n1 new encrypted data blocks, denoted as a set User i sends the set C i to the server.

5. The method of claim 1, wherein, In the step (4), the server collects the encrypted data block sets of at least n2 users, and (n-n2) / n>t, and records these users as a set N2, and recombines the collected encrypted data blocks to obtain n2 integrated encrypted data block sets where the elements in set D i are composed of The server sends the set D i to the corresponding user i.

6. The method of claim 5, wherein, In step (4), the user i decrypts the received encrypted data block using the formula and computes and uploads m i to the server.

7. The method of claim 5, wherein, The step (5) comprises: The server collects the data uploaded by at least t users, and then uses the decoding matrix to recover the aggregation result x of this round of federated learning, where a i The original gradient calculated for user i.

8. A secure aggregation federated learning training system based on MDS encoding, characterized in that, It consists of one server node and several user nodes. The server node has the initial federated learning model, and the user nodes have the training data required for federated learning. In each round of federated learning, the user nodes are configured to: train the model distributed by the server using local data; and generate a public-private key pair locally. public key Send to the server; receive public keys from other users distributed by the server, and use your own private key. and the public keys of other users collected Calculate the shared key S between yourself and other users. i,j ; Split the gradient values ​​that need to be uploaded into several data blocks and encode them using MDS. And using the shared key S i,j For data blocks Encryption, resulting in encrypted data blocks. Then encrypt the data block Uploaded to the server; and, after receiving the encrypted data blocks from the server, the shared key S is used to decrypt and sum the data blocks, and then the sum is uploaded to the server. i,j decrypt and sum the data blocks, and then the sum is uploaded to the server. The server is configured to: distribute the latest model to users; and, after receiving the public keys of at least n1 online users, collect the set of public keys. Distribute the data to all online users, where n1 ≤ n, n is the number of users, and (n-n1) / n > t, where t is the maximum allowed disconnection rate of the system; after collecting encrypted data blocks from at least n2 online users, distribute these encrypted data blocks to the corresponding users, where n2 ≤ n; after collecting data from no less than t users, decode the data to obtain the aggregation gradient value for this round, complete the secure aggregation for this round, and use the obtained aggregation gradient to update the model.

Images