A federated learning method, system and device with high availability in a personalized differential privacy scenario

Abstract

A kind of federated learning method, system and equipment under personalized differential privacy scene high availability, its method is through local differential privacy technology to realize the privacy protection of the local model of participant, simultaneously, based on model availability theorem, more useful information can be extracted from the local model with higher availability by federated aggregation algorithm, to improve the availability of global model, in the algorithm, homomorphic encryption and secure multi-party computation method are used to protect the privacy of local model availability parameter;Its system and equipment can realize the privacy protection of the local model of participant based on the above method, improve the availability of aggregated model under the condition of ensuring the privacy of model availability parameter, with the advantages of good personalized privacy protection, high precision.

Description

Technical Field

[0001] This invention belongs to the field of information security technology, and in particular relates to a highly available federated learning method, system and device for personalized differential privacy scenarios. Background Technology

[0002] Federated learning is an emerging distributed machine learning framework that enables model training by exchanging intermediate parameters without data leaving the local machine, thus protecting the privacy of data owners. Currently, federated learning is widely used in various fields, such as smart finance, smart healthcare, autonomous driving, wireless communication, and object detection.

[0003] While federated learning protects participant data privacy by exchanging model parameters between participants and the server, research has found that the exchanged model parameters may also leak the original training data. A common technique to prevent information leakage is differential privacy, and existing work on differential privacy-based federated learning includes local differential privacy and differential privacy based on stochastic gradient descent. In local differential privacy, each participant adds a perturbation of varying magnitude to its local model parameters according to its local privacy budget and sends the perturbed model parameters to the server, thus protecting the participant's privacy. Existing work on differential privacy-based federated learning assumes that all participants have a uniform privacy budget. However, in practice, participants' privacy budgets are personalized. Due to different privacy policies or different privacy preferences of data subjects, different data subjects may have different privacy needs, leading to different cross-subject privacy budgets. However, existing federated averaging aggregation algorithms weight the local model based on the client's dataset size, which is not suitable for personalized differential privacy scenarios because clients with smaller privacy budgets add larger perturbations, resulting in lower model usability. In such scenarios, using federated averaging aggregation algorithms will lead to low usability of the aggregated model. Furthermore, how to improve model availability by performing global model aggregation based on model availability parameters while protecting the privacy of these parameters is an extremely challenging problem, and currently there is no good solution.

[0004] The paper [R.Hu,Y.Guo,H.Li,Q.Pei and Y.Gong, "Personalized Federated Learning With Differential Privacy," in IEEE Internet of Things Journal, vol.7, no.10, pp.9530-9539, Oct.2020, doi:10.1109 / JIOT.2020.2991416.] proposes a personalized federated learning scheme that completes multi-task learning by learning user characteristics, but this scheme does not consider the security of the federated learning process.

[0005] The paper [G. Yang, S. Wang and H. Wang, "Federated Learning with Personalized Local Differential Privacy," 2021 IEEE 6th International Conference on Computer and Communication Systems (ICCCS), Chengdu, China, 2021, pp. 484-489, doi:10.1109 / ICCCS52626.2021.9449232.] proposes a personalized differential privacy federated learning scheme in which users perturb the data by using their own privacy preferences. However, this scheme does not consider the impact of user-added noise on the global data, nor does it quantify the degree of privacy protection. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, the present invention aims to provide a highly available federated learning method, system, and device for personalized differential privacy scenarios. It achieves privacy protection for the local models of participating parties through local differential privacy technology. Simultaneously, based on model availability theory, a federated aggregation algorithm can extract more useful information from local models with higher availability, thereby improving the availability of the global model. This algorithm employs homomorphic encryption and secure multi-party computation techniques to protect the privacy of local model availability parameters. The present invention can improve the availability of the aggregated model while ensuring the privacy of model availability parameters.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A highly available federated learning method for personalized differential privacy scenarios is proposed. Each participant trains a local model using a local dataset, and local differential privacy technology is used to protect the privacy of the participant's local model. Simultaneously, each participant calculates the availability of their local model based on model availability theory and encrypts the plaintext model availability parameters using homomorphic encryption and secure multi-party computation. Then, the participants upload their local models protected by differential privacy to an aggregation server, and simultaneously upload all model availability parameters encrypted using homomorphic encryption and secure multi-party computation to a utility server. The utility server uses the received encrypted model availability parameters to calculate the local model aggregation weights in ciphertext, and then sends these ciphertext aggregation weights to the aggregation server. The aggregation server decrypts the ciphertext aggregation weights, performs model aggregation based on these weights, and finally distributes the aggregated model to each participant.

[0009] The highly available federated learning method for personalized differential privacy scenarios described above is based on the following system assumptions:

[0010] 1) There are n participants in the system, and the set of participants is denoted as u = {u1, u2, ..., u3}. n The local datasets of each participant are {D1, D2, ..., D}. n The dataset for all participants is represented as D;

[0011] 2) The local neural network models of each participant are of the same type and structure, but the privacy budgets of each participant can be different, and the local datasets can be either independent and identically distributed or non-independent and identically distributed.

[0012] 3) Aggregation servers and utility servers are honest, curious, and do not collude.

[0013] A highly available federated learning method for personalized differential privacy scenarios includes the following steps:

[0014] Step 1: System initialization: This includes generating public and private key pairs for homomorphic encryption, key negotiation between participants, and generating a mask; the server initializes the global model and then distributes it to the local participants.

[0015] Step 2: Each participant trains the global model distributed by the server locally, implements differential privacy protection for the local model information based on the local privacy budget, and uploads the local model parameters with differential privacy protection to the aggregation server.

[0016] Step 3: Each participant calculates the local model availability with differential privacy protection implemented in Step 2 based on the model availability theory, and protects the privacy of the local model availability parameters through homomorphic encryption and masking techniques, and uploads the results to the utility server;

[0017] Step 4: The utility server obtains the aggregate weight under the ciphertext based on the information obtained in Step 3, and sends the aggregate weight under the ciphertext to the aggregation server;

[0018] Step 5: The aggregation server decrypts the received aggregation weights using its private key, performs model aggregation based on the decrypted aggregation weights and local model parameters, and finally distributes the aggregated model to each participant.

[0019] Step 6: Repeat steps 2 to 5 to perform the next round of model training until the number of training rounds reaches the predefined number of training rounds or the model converges.

[0020] The specific method for system initialization in step 1 is as follows:

[0021] Using KeyGen(1 λ The function (pk, sk) is used to generate a public-private key pair (pk, sk), where λ is a security parameter. The output is a shared public key pk and private key sk, generated by all participants. The private key sk is held by the aggregation server, and the public key pk is held by each participant and the utility server. i and u j i≠j, through KeyGen(1 λ The agreement and mask generation algorithm generated a series of masks. The server initializes the global model and distributes the model information to each participant; each participant selects its own local privacy budget.

[0022] The specific method for achieving differential privacy protection of local model information based on local privacy budget in step 2 is as follows:

[0023]

[0024] According to the DPSGD algorithm, given a data sampling rate q and a training epoch T, and with constants c1 and c2, for any privacy budget ε < c1q, 2 T, if noise standard deviation is chosen Algorithm 1 then satisfies (ε, δ)-differential privacy.

[0025] The specific method for step 3 is as follows:

[0026] Define local model availability Of which, the local loss is t is the current training epoch of the model, D t This represents the data randomly sampled by each participant from their local dataset D during the t-th round of training, where the size of the sampled dataset is j. t =|D t |,w* These are the optimal model parameters that minimize the local loss;

[0027] According to the model availability theorem, for μ smooth and λ strongly convex models, the loss L(w) is... t D t ), making the learning rate d is the dimension of the model parameters, C is the regularization boundary of the gradient, and σ 2 The variance of the added Gaussian noise is considered, and the usability of the local model after training t rounds using the DPSGD algorithm satisfies:

[0028]

[0029] As stated in the above theorem, the availability of a participant's local model is proportional to...

[0030] The local model parameters of each participant are determined based on the availability of their local model. The weights in the aggregation process, i.e., the weights of the participants u i The aggregate weight in round t is:

[0031]

[0032] in, For participating party u i Model availability in round t, The sum of model availability for all participants in round t;

[0033] Calculation of participant u i Availability of local models Afterwards, the participating party u i Model availability parameters Adding it to the local mask yields the mask-protected model availability parameter. Simultaneously using homomorphic encryption Obtain the model availability parameters in encrypted form. Then, the model availability parameters protected by the mask. and model availability parameters protected by homomorphic encryption Upload to the utility server.

[0034] The specific method for step 4 is as follows:

[0035] First, the utility server receives all masked model availability parameters. Adding them together eliminates the effect of the mask, that is:

[0036]

[0037] Then, the utility server uses an encryption function. Use public key pk to analyze plaintext Encrypt, obtain Then, through the ciphertext multiplication function get And aggregate weights Send to the aggregation server.

[0038] The specific method for step 5 is as follows:

[0039] First, the aggregation server will obtain the data from the utility server. Decryption is performed to obtain the aggregate weights of each participant's local model: i.e., the decryption function. Use key sk to ciphertext After decryption, the aggregate weight is obtained. Then, the aggregation server adjusts the parameters based on the local model parameters from the participants. and aggregate weight Perform model aggregation, that is:

[0040]

[0041] Finally, the global model parameters are distributed to all participants.

[0042] Systems based on the highly available federated learning methods in the aforementioned personalized differential privacy scenarios include:

[0043] The system initialization module is used for the system initialization described in step 1, including generating public-private key pairs for homomorphic encryption, key negotiation between participants, generating a mask, and the server initializing the global model and then distributing it to the local participants.

[0044] The local model training module is used by each participant in step 2 to train the initialized global model locally, implement differential privacy protection of local model information based on the local privacy budget, and upload the local model parameters that implement differential privacy protection to the aggregation server.

[0045] The availability calculation and protection module is used to calculate the availability of the local model as described in step 3, and to protect the availability parameters for privacy using masking and homomorphic encryption.

[0046] The aggregation weight calculation module is used to calculate the aggregation weight according to the method described in step 4 without disclosing availability privacy.

[0047] The model aggregation module is used to perform model aggregation as described in step 5.

[0048] Systems based on the highly available federated learning methods in the aforementioned personalized differential privacy scenarios include:

[0049] Memory, used to store computer programs;

[0050] A processor is used to execute the computer program, which, when executed by the processor, enables the highly available federated learning method in the personalized differential privacy scenario described in steps 1 to 6.

[0051] A computer-readable storage medium is provided for storing a computer program that, when executed by a processor, can achieve privacy protection of the local models of participating parties according to the highly available federated learning method in the personalized differential privacy scenario described in steps 1 to 6, thereby improving the availability of the aggregated model while ensuring the privacy of model availability parameters.

[0052] Compared with the prior art, the present invention has the following advantages:

[0053] 1. This invention addresses the problem of insufficient availability of aggregated models in personalized differential privacy scenarios. Based on the model availability theorem, a new federated aggregation method is designed. Compared with existing methods, this invention enables the server to extract more useful information from models with high availability, resulting in a highly available aggregated model in personalized differential privacy scenarios. At the same time, homomorphic encryption and secure multi-party computation techniques are used to protect the privacy of local model availability parameters during this process.

[0054] 2. This invention utilizes the model availability theorem to determine the local model weights of each participant by introducing model availability parameters during the model aggregation process. This enables the server to extract more useful information from models with higher availability, thereby obtaining an aggregated model with higher availability.

[0055] 3. This invention employs homomorphic encryption and secure multi-party computation techniques to protect the privacy of local model availability parameters, thereby improving the availability of the aggregated model while ensuring the privacy of these parameters.

[0056] In summary, this invention has the advantages of good personalized privacy protection and high accuracy. Attached Figure Description

[0057] Figure 1 It is a federated learning system framework designed for personalized differential privacy scenarios.

[0058] Figure 2 It is the accuracy of the aggregation model using the federated average aggregation algorithm.

[0059] Figure 3 It is the accuracy of the aggregation model where the privacy budget follows a mixed distribution 1, where, Figure 3 (a) represents the accuracy of the aggregation model when α is 0.1. Figure 3 (b) represents the accuracy of the aggregation model when α is 0.3. Figure 3 (c) represents the accuracy of the aggregation model when α is 0.5. Figure 3 (d) represents the accuracy of the aggregation model when α is 0.7. Figure 3 (e) represents the accuracy of the aggregation model when α is 0.9.

[0060] Figure 4 The accuracy of the aggregation model for privacy budgets following a mixed distribution 2, where, Figure 4 (a) represents the accuracy of the aggregation model when α is 0.1. Figure 4 (b) represents the accuracy of the aggregation model when α is 0.3. Figure 4 (c) represents the accuracy of the aggregation model when α is 0.5. Figure 4 (d) represents the accuracy of the aggregation model when α is 0.7. Figure 4 (e) represents the accuracy of the aggregation model when α is 0.9.

[0061] Figure 5 The accuracy of the aggregation model for privacy budgets following a mixed distribution 3, where, Figure 5 (a) The accuracy of the aggregation model with α, β, and γ values ​​of (20%, 20%, 60%). Figure 5 (b) shows the accuracy of the aggregation model with α, β, and γ values ​​of (20%), 60%, and 20%, respectively. Figure 5 (c) represents the accuracy of the aggregation model with α, β, and γ values ​​of (60%, 20%, 20%). Figure 5 (d) represents the accuracy of the aggregation model with values ​​of α, β, and γ of (30%, 30%, and 40%). Detailed Implementation

[0062] The present invention will now be described in further detail with reference to the accompanying drawings.

[0063] The theoretical and technical foundation of highly available federated learning in personalized differential privacy scenarios involved in this invention is as follows:

[0064] A highly available federated learning method for personalized differential privacy scenarios includes the following steps:

[0065] See Figure 1 Step 1: Initialization

[0066] Using KeyGen(1 λ The function (pk, sk) is used to generate a public-private key pair (pk, sk), where λ is a security parameter. The output is a shared public key pk and private key sk, generated by all participants. The private key sk is held by the aggregation server, and the public key pk is held by each participant and the utility server.i and u j i≠j, through KeyGen(1 λ The agreement and mask generation algorithm generated a series of masks. The server initializes the global model and distributes the model information to each participant; each participant selects its own local privacy budget.

[0067] Step 2: Each participant trains its local model based on its local data, the latest global model, and the DSGD algorithm. It implements differential privacy protection for local model information based on the local privacy budget and uploads the local model parameters that achieve local differential privacy to the aggregation server.

[0068] The specific method for achieving differential privacy protection of local model information based on local privacy budget is as follows:

[0069]

[0070] According to the DPSGD algorithm, given a data sampling rate q and the number of training epochs T, there exist constants c1 and c2 such that for any privacy budget ε < c1q... 2 T, if noise standard deviation is chosen Algorithm 1 then satisfies (ε, δ)-differential privacy.

[0071] Step 3: Define local model availability Of which, the local loss is t is the current training epoch of the model, D t This represents the data randomly sampled by each participant from their local dataset D during the t-th round of training, where the size of the sampled dataset is j. t =|D t |,w * These are the optimal model parameters that minimize the local loss;

[0072] According to model availability theory, for μ smooth and λ strongly convex models, the loss L(w) is... t D t ), making the learning rate d is the dimension of the model parameters, C is the regularization boundary of the gradient, and σ 2 The variance of the added Gaussian noise is considered, and the usability of the local model after training t rounds using the DPSGD algorithm satisfies:

[0073]

[0074] As can be seen from the above theory, the availability of a participant's local model is proportional to...

[0075] The weights of each participant's local model parameters in the aggregation process are determined based on the availability of each participant's local model, i.e., participant u. i The aggregate weight in round t is:

[0076]

[0077] in, For participating party u i Model availability in round t, The sum of model availability for all participants in round t;

[0078] Calculation of participant u i Availability of local models Afterwards, the participating party u i Model availability parameters Adding it to the local mask yields the mask-protected model availability parameter. Simultaneously using homomorphic encryption Obtain the model availability parameters in encrypted form. Then, the model availability parameters protected by the mask. and model availability parameters protected by homomorphic encryption Upload to the utility server.

[0079] Step 4: First, the utility server displays all masked model availability parameters it receives. Adding them together eliminates the effect of the mask, that is:

[0080]

[0081] Then, the utility server through get After that, through get And Send to the aggregation server.

[0082] Step 5: The aggregation server performs model aggregation. First, the aggregation server will obtain the data from the utility server. Decryption is performed to obtain the aggregate weights of each participant's local model, i.e. Then, the aggregation server adjusts the parameters based on the local model it received. and aggregate weight Perform model aggregation, that is:

[0083]

[0084] Finally, the global model parameters are distributed to all participants.

[0085] Step 6: Repeat steps 2 to 5 to perform the next round of model training until the number of training rounds reaches the predefined number of training rounds or the model converges.

[0086] Systems based on the highly available federated learning methods in the aforementioned personalized differential privacy scenarios include:

[0087] The system initialization module is used to initialize the federated learning structure, participant keys, and mask as described in step 1.

[0088] The local model training module is used by each participant in step 2 to train the initialized global model locally, implement differential privacy protection of local model information based on the local privacy budget, and upload the local model parameters that implement differential privacy protection to the aggregation server.

[0089] The availability calculation and protection module is used to calculate the availability of the local model as described in step 3, and to protect the availability parameters for privacy using masking and homomorphic encryption.

[0090] The aggregation weight calculation module is used to calculate the aggregation weight according to the method described in step 4 without disclosing availability privacy.

[0091] The model aggregation module is used to perform model aggregation as described in step 5.

[0092] Devices based on the highly available federated learning methods for personalized differential privacy scenarios include:

[0093] Memory, used to store computer programs;

[0094] A processor is used to execute the computer program, which, when executed by the processor, enables the highly available federated learning method in the personalized differential privacy scenario described in steps 1 to 6.

[0095] A computer-readable storage medium is provided for storing a computer program that, when executed by a processor, can achieve privacy protection of the local models of participating parties according to the highly available federated learning method in the personalized differential privacy scenario described in steps 1 to 6, thereby improving the availability of the aggregated model while ensuring the privacy of model availability parameters.

[0096] The following section verifies the effectiveness of the proposed scheme through a series of experiments. Compared with the FedAvg algorithm, which does not consider personalized differential privacy preferences, the algorithm proposed in this invention can achieve higher accuracy in the aggregation model.

[0097] Experimental environment: Processor: Intel(R) Core(TM) i5-10400 CPU @ 2.90GHz, RAM: 16GB, Operating system: Windows 11 Home Edition. Programming environment: Python 3.9.13, PyTorch 1.7.7, TensorFlow 0.3.12, TensorFlow-privacy 0.8.7.

[0098] All experiments used the MNIST dataset, with 10 clients and each participant holding a dataset size of 200. The model trained by the system was a convolutional neural network containing two convolutional layers and two linear layers.

[0099] Privacy preferences: To simulate the possible privacy preferences of the participants, we consider that the participants' privacy preferences follow a Gaussian distribution and a multimodal distribution (a mixture of two or more different Gaussian distributions). The multimodal distribution indicates that the participants have multiple privacy budgets (denoted as N1, N2 and N3 respectively). Table 1 shows the distribution of the participants' privacy preferences considered in the experiment.

[0100] Table 1: Distribution of Client Privacy Preferences

[0101]

[0102] The experimental results are as follows:

[0103] Figure 2 The above demonstrates the model accuracy obtained using the federated averaging algorithm when the privacy preferences of the participants are Gaussian distribution 1, Gaussian distribution 2, and Gaussian distribution 3. Figure 2 As shown, the accuracy of the aggregation model improves with the increase of the privacy budget. When the privacy budget is 0.5, the accuracy of the aggregation model is about 10%, indicating that the small privacy budget leads to excessive noise, making the model unusable. When the privacy budget is 10, the accuracy of the aggregation model reaches 96%. At this point, the larger privacy budget results in less noise, leading to higher model accuracy.

[0104] Figure 3 This demonstrates the model accuracy obtained using the availability-weighted aggregation algorithm and the federated average aggregation algorithm when the privacy preferences of the participants follow a mixed distribution 1, with α values ​​of 0.1, 0.3, 0.5, 0.7, and 0.9. Figure 3 It can be seen that the accuracy of the aggregated models obtained by the availability-weighted aggregation algorithm is higher than that of the models obtained by the federated average aggregation algorithm. As α increases, the effect of the federated average aggregation algorithm approaches that of the availability-weighted aggregation algorithm. As α decreases, the availability-weighted aggregation algorithm is significantly better than the federated average aggregation algorithm.

[0105] Figure 4 This demonstrates the model accuracy obtained using the availability-weighted aggregation algorithm and the federated average aggregation algorithm when the privacy preferences of the participants follow a mixed distribution 2, with α values ​​of 0.1, 0.3, 0.5, 0.7, and 0.9. Figure 4 As can be seen, the accuracy of the aggregated models obtained by the availability-weighted aggregation algorithm is higher than that of the models obtained by the federated average aggregation algorithm. When the proportion of clients with a privacy budget of 10 is less than 50%, the accuracy of the aggregated models obtained by the federated average aggregation algorithm is about 10%, indicating that the participation of participants with a privacy budget of 0.5 in the federated learning system will reduce the accuracy of the aggregated models. On the other hand, the availability-weighted aggregation algorithm can significantly reduce the impact of participants with a privacy budget of 0.5 uploading models, thereby improving the accuracy of the aggregated models.

[0106] Figure 5 This demonstrates the model accuracy obtained using the availability-weighted aggregation algorithm and the federated average aggregation algorithm when the privacy preferences of the participants follow a mixed distribution 3, with α, β, and γ taking values ​​of (20%, 20%, 60%), (20%, 60%, 20%), (60%, 20%, 20%), and (30%, 30%, 40%), respectively. Figure 5 As can be seen, when the participants in federated learning have three different privacy preferences, the accuracy of the aggregated model obtained by the availability-weighted aggregation algorithm is higher than that of the model obtained by the federated average aggregation algorithm.

Claims

1. A highly available federated learning method for personalized differential privacy scenarios, characterized in that, Each participant trains a local model using a local dataset. Local differential privacy is used to protect the privacy of the participant's local model. At the same time, each participant calculates the availability of the local model based on the model availability theory and uses homomorphic encryption to encrypt the model availability parameters in plaintext. Secure multi-party computation is used to encrypt the model availability parameters in plaintext. Then, the participants will upload the local model protected by differential privacy to the aggregation server, and at the same time, upload all the model availability parameters encrypted with homomorphic encryption and secure multi-party computation methods to the utility server. The utility server uses the received encrypted model availability parameters to calculate the local model aggregation weight under the ciphertext, and then sends the local model aggregation weight under the ciphertext to the aggregation server. The aggregation server decrypts the encrypted aggregation weights, performs model aggregation based on the aggregation weights, and finally distributes the aggregated model to each participant. The highly available federated learning algorithm for personalized differential privacy scenarios is based on the following system assumptions: 1) The system has a total of n There are __ participants, and the set of participants is denoted as ___. The local datasets of each participant are as follows: The dataset of all participants is represented as ; 2) The local neural network models of each participant are of the same type and structure, but the privacy budgets of each participant can be different, and the local datasets can be either independent and identically distributed or non-independent and identically distributed; 3) Aggregation servers and utility servers are honest, curious, and non-colluding; Specifically, the following steps are included: Step 1: System initialization: This includes generating public and private key pairs for homomorphic encryption, key negotiation between participants, and generating a mask; The server initializes the global model and then distributes it to the local participants. Step 2: Each participant trains the global model distributed by the server locally, implements differential privacy protection for the local model information based on the local privacy budget, and uploads the local model parameters with differential privacy protection to the aggregation server. Step 3: Each participant calculates the local model availability with differential privacy protection implemented in Step 2 based on the model availability theory, and protects the privacy of the local model availability parameters through homomorphic encryption and masking techniques, and uploads the results to the utility server; Step 4: The utility server obtains the aggregate weight under the ciphertext based on the information obtained in Step 3, and sends the aggregate weight under the ciphertext to the aggregation server; Step 5: The aggregation server decrypts the received aggregation weights using its private key, performs model aggregation based on the decrypted aggregation weights and local model parameters, and finally distributes the aggregated model to each participant. Step 6: Repeat steps 2 to 5 to perform the next round of model training until the number of training rounds reaches the predefined number of training rounds or the model converges.

2. The highly available federated learning method for personalized differential privacy scenarios according to claim 1, characterized in that, The specific method for system initialization in step 1 is as follows: use Key generation function, For security parameters, the function output is the public key shared by all participants. With private key Generate a public-private key pair private key The public key is held by the aggregation server. Held by each participant and the utility server; Participants and , ,pass , A series of masks were generated by the mask generation algorithm. ; The server initializes the global model and distributes the model information to each participant; each participant selects its own local privacy budget.

3. The highly available federated learning method for personalized differential privacy scenarios according to claim 1, characterized in that, The specific method for achieving differential privacy protection of local model information based on local privacy budget in step 2 is as follows: according to The algorithm can obtain: given data sampling rate and number of training rounds And there exists a constant. and For any privacy budget If noise standard deviation is selected Then Algorithm 1 satisfies - Differential privacy.

4. The highly available federated learning method for personalized differential privacy scenarios according to claim 1, characterized in that, The specific method for step 3 is as follows: Define local model availability Of which, the local loss is , This is the current training epoch of the model. Indicates the first During each training round, each participant uses the local dataset. The data was randomly sampled from the dataset, and the size of the sampled dataset was [size missing]. , These are the optimal model parameters that minimize the local loss; According to the model availability theorem, we can obtain: For Smooth and Strongly convex model loss Make the learning rate , For the dimensions of the model parameters, The gradient regularization boundary is... To calculate the variance of the added Gaussian noise, use Algorithm Training The availability of the local model after the round satisfies: As stated in the above theorem, the availability of a participant's local model is proportional to... ; Based on the availability of local models for each participant, the local model parameters of each participant are determined. Weights in the aggregation process, i.e., the participants In the The aggregate weight of the round is: in, As a participant In the Wheel model availability, For all participants in the first The sum of the availability of wheel models; Calculation of participants Availability of local models Afterwards, the participating parties Model availability parameters Adding it to the local mask yields the mask-protected model availability parameter. Simultaneously through homomorphic encryption Obtain the model availability parameters in encrypted form. Then, the model availability parameters protected by the mask. and model availability parameters protected by homomorphic encryption Upload to the utility server.

5. A highly available federated learning method for personalized differential privacy scenarios according to claim 1, characterized in that, The specific method for step 4 is as follows: First, the utility server receives all masked model availability parameters. ( Adding them together eliminates the effect of the mask, that is: Then, the utility server uses an encryption function. Using public key plaintext encryption, arrive Then, through the ciphertext multiplication function get and aggregate weights ( Send to the aggregation server.

6. The highly available federated learning method for personalized differential privacy scenarios according to claim 1, characterized in that, The specific method for step 5 is as follows: First, the aggregation server will obtain the data from the utility server. Decryption is performed to obtain the aggregate weights of each participant's local model: i.e., the decryption function. ( ), using a key For ciphertext After decryption, the aggregate weight is obtained. ( Then, the aggregation server adjusts the parameters based on the local model parameters from the participants. and aggregate weight Perform model aggregation, that is: Finally, the global model parameters are distributed to all participants.

7. A system based on the highly available federated learning method for personalized differential privacy scenarios as described in any one of claims 1 to 6, characterized in that, include: The system initialization module is used for the system initialization described in step 1, including generating a public-private key pair for homomorphic encryption, key negotiation between participants, and generating a mask. The server initializes the global model and then distributes it to the local participants. The local model training module is used by each participant in step 2 to train the initialized global model locally, implement differential privacy protection of local model information based on the local privacy budget, and upload the local model parameters that implement differential privacy protection to the aggregation server. The availability calculation and protection module is used to calculate the availability of the local model as described in step 3, and to protect the availability parameters for privacy using masking and homomorphic encryption. The aggregation weight calculation module is used to calculate the aggregation weight according to the method described in step 4 without disclosing availability privacy. The model aggregation module is used to perform model aggregation as described in step 5.

8. A device based on the highly available federated learning method for personalized differential privacy scenarios according to any one of claims 1 to 6, characterized in that, include: Memory, used to store computer programs; A processor is used to execute the computer program, which, when executed by the processor, enables the highly available federated learning method in the personalized differential privacy scenario described in steps 1 to 6.

Images