Convolutional Neural Network Training Method Based on Non-intersection Differential Privacy Federated Learning

Abstract

This invention discloses a convolutional neural network training method based on non-intersection differential privacy federated learning, which primarily addresses the problems of low privacy protection capabilities and low model classification accuracy of convolutional neural network models trained using existing technologies. Its implementation scheme is as follows: constructing a federated learning system; the client initializes the federated learning training dataset; the client initializes the convolutional neural network model; the client initializes the local convolutional neural network model and the local privacy budget; the client iteratively trains the local convolutional neural network model; the client calculates the non-intersection elements of the trained local model; the client adds noise to the quantized local model based on the non-intersection element results; and a central server iteratively trains the convolutional neural network model by aggregating the noisy local convolutional neural network models to obtain a trained convolutional neural network model. The convolutional neural network trained by this invention has strong privacy protection capabilities and high classification accuracy, and can be used for image classification.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to a convolutional neural network training method that can be used to classify images. Background Art

[0002] Image classification is a common machine learning task that accurately identifies the target category of an image based on its features. Convolutional neural networks are often used in machine learning to solve image classification tasks. Training an image classification model requires a large amount of training data. However, this training data often comes from multiple data sources and contains sensitive personal information. As people become more aware of privacy, the requirements for personal privacy are becoming increasingly stringent.

[0003] To address privacy concerns, McMahan et al. proposed the concept of federated learning in 2017, balancing efficiency and security. Federated learning has become a popular distributed machine learning paradigm, reducing communication overhead and enhancing privacy protection during communication by sharing model updates, such as gradients and model parameters. However, researchers have discovered that gradients and model parameters can be analyzed, for example, in model inversion attacks and membership inference attacks, often infringing user privacy. Deploying differential privacy technology in federated learning can meet privacy requirements, but this approach can also reduce the model's predictive classification performance or system efficiency. Therefore, in federated learning based on differential privacy, improving the model's predictive classification performance and reducing the risk of privacy leakage are urgent issues that need to be addressed in order to achieve an efficient and secure federated learning system.

[0004] The patent application document with application publication number CN113762525A discloses "a method for training a federated learning model with differential privacy protection", and its implementation steps are: (1) establishing a connection between the model and the terminal, selecting a learning model, and establishing a connection between the learning model and the terminal server; (2) distinguishing and distributing the model, distinguishing the model at the sample level and the user level, and the terminal server selects the device participating in this round of federated learning according to the standard after the distinguished model; (3) conducting local training, each device loads the current global model, uses its own training data for training, and obtains a new local model; (4) establishing privacy protection, according to the distinction of model categories, using differential privacy protection to encrypt and protect information in different ways; (5) updating the global model, each device uploads the updated value of the model to the server, the server collects the updated value and aggregates it to obtain the aggregated result, and then updates the global model; (6) monitoring and feedback, monitoring the updated value of the data, judging whether the trained model leaks a specific sample or user participates in the training, and feeding back the obtained result to the terminal server; (7) data backup record, backing up and archiving the updated global model data to avoid damage during data transmission that affects data acquisition. This method controls the perturbation degree of the federated learning model through the privacy budget value, and assigns equal privacy budget values ​​to all model weight parameters of the federated learning model, so that the perturbation degree of all weight parameters is the same. It does not take into account the different privacy protection requirements of the client's shared weight parameters and non-shared weight parameters. As a result, the perturbation degree of the shared weight parameters with low privacy protection requirements is too large, and the perturbation degree of the non-shared weight parameters with high privacy protection requirements is too small, thereby reducing the model's prediction effect and privacy protection ability, and affecting the image classification accuracy. Summary of the Invention

[0005] The purpose of the present invention is to address the shortcomings of the above-mentioned existing technologies and propose a convolutional neural network training method based on non-intersection differential privacy federated learning, so as to reduce the impact of different privacy protection requirements on the mismatch of the perturbation degree of the two types of models: client shared weight parameters and non-shared weight parameters, improve the prediction effect and privacy protection ability of the network model, and thus improve the image classification accuracy.

[0006] The technical approach to achieving the objectives of the present invention is to have each client first calculate the shared intersection parameters and non-shared intersection parameters of all local model weight parameters during each iteration, and then assign different privacy budget values ​​to different weight parameters based on different privacy protection requirements. That is, a smaller degree of disturbance is achieved for the shared intersection parameters and a larger degree of disturbance is achieved for the non-shared intersection parameters, thereby improving the prediction effect and privacy protection capability of the network model.

[0007] According to the above ideas, the implementation steps of the present invention are as follows:

[0008] (1) Initialization includes the central server, the private set intersection server and N clients C = {c1, c2, ..., c n ,…,c N} of the federated learning system, where c n represents the nth client, n={1,2,…,N}, N≥2;

[0009] (2) Each client c n Initialize a local training dataset of M images containing L target categories and annotate the target in each image, where L ≥ 2 and M ≥ 100.

[0010] (3) Each client c n Get the convolutional neural network model X consisting of I model layers from the central server n , the X n The weight parameter is ω n , X n The weight parameter of the i-th model layer is ω n,i ;

[0011] (4) Each client initializes the local convolutional neural network model set up The privacy budget value and weight parameters are and The weight parameter of the i-th model layer is

[0012] (5) Client-side local convolutional neural network model Perform iterative training:

[0013] Each client c n Randomly select B training data from the local training dataset with replacement as the local convolutional neural network model Input, get B predicted labels Using this label, the weight parameters are adjusted using stochastic gradient descent. Perform iterative update and obtain the weight parameter as The trained local model

[0014] (6) The client calculates the trained local model The non-intersecting elements of:

[0015] (6a) Each client c n The weight parameters of each layer of the updated local model Perform quantitative calculations and set the weight parameters The decimal part of is precisely fixed length, and the quantized weight parameters are obtained.

[0016] (6b) Each client c n Calculate the model weight parameters after quantization of each layer Hash value And upload to the private set intersection server, the private set intersection server according to each client c n Uploaded hash value Calculate the non-intersection element results of the weight parameters of each layer of the local model after client quantization And then summarize it and send it to each client;

[0017] (7) The client adds noise to the quantized local model based on the non-intersection element results:

[0018] (7a) Each client c n Based on the aggregated non-intersection element results and the local model Privacy budget Calculate the noise value containing two different differential privacy guarantees

[0019] (7b) Each client c n The noise value Weight parameters added to the quantized local model The weight parameters after perturbation are Local model The perturbed local model Upload to the central server;

[0020] (8) The central server trains the convolutional neural network model:

[0021] (8a) The central server initializes the aggregation round to t = 0 and sets the maximum aggregation round to T ≥ 100;

[0022] (8b) Randomly select V t Client c v Uploaded perturbed local model The weight parameter Perform aggregation and get the current aggregation result ω t , and judge whether t=T is true:

[0023] If so, the weight parameter is ω T The trained convolutional neural network model X T ;

[0024] Otherwise, let t = t + 1 and send the aggregation result to each client cn , return to step (3).

[0025] Compared with the prior art, the present invention has the following advantages:

[0026] First, the client of the present invention allocates different disturbance noises to different weight parameters of the local convolutional neural network model based on the different privacy protection requirements of the shared weight parameters and the unshared weight parameters of the local model, thereby being able to accurately control the degree of disturbance of the local model weight parameters;

[0027] Second, because the client of the present invention calculates Gaussian noise values ​​that meet differential privacy guarantees for the non-intersection elements of the local model weight parameters, it can avoid the defects of the existing technology caused by excessively large privacy budget values, such as too small perturbation noise, too low local model perturbation degree, and low privacy protection ability, and effectively improve the privacy protection ability of the local model;

[0028] Third, since the client of the present invention calculates the noise value that does not meet the differential privacy guarantee for the intersection elements of the local model weight parameters, it can avoid the defects of the existing technology caused by too small privacy budget value, such as excessive disturbance noise, too high disturbance degree of local model and low classification accuracy of convolutional neural network model, and effectively improve the prediction classification accuracy of the convolutional neural network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flow chart for implementing the present invention.

[0030] Figure 2 It is a sub-flowchart of non-intersection elements for calculating weight parameters of local model by the client in the present invention;

[0031] Figure 3 This is a sub-flowchart of the present invention in which the client performs noise addition according to the non-intersection element results;

[0032] Figure 4 The results of the simulation training of the MNIST data set using the present invention and the existing method are shown in FIG;

[0033] Figure 5 A comparison chart of the effects of privacy attacks on convolutional neural network models trained using the present invention and existing methods. DETAILED DESCRIPTION

[0034] The embodiments and effects of the present invention are further described in detail below with reference to the accompanying drawings.

[0035] Reference Figure 1 The implementation steps of this example include the following:

[0036] Step 1: Build a federated learning system.

[0037] Suppose there is a central server, a private set intersection server and N clients C={c1,c2,…,c n ,…,c N}, each client c n Establish transmission channels with the central server and the private set intersection server respectively to form a federated learning system, where N≥2, c n Indicates the nth client, in this example N=100.

[0038] Step 2: The client initializes the federated learning training dataset.

[0039] Each client c n Initialize a local training dataset of M images containing L target categories and annotate the target in each image, where L ≥ 2 and M ≥ 100.

[0040] The dataset used in this example is the MNIST handwritten digit image dataset. The number of target categories corresponding to this dataset is L = 10, which are the digits 0-9, and a total of 60,000 images. The pixel size of each image is 28×28, and there are 6,000 images for each target category. In this example, the local training dataset of each client is set to contain M = 600 images.

[0041] Step 3: The client initializes the convolutional neural network model.

[0042] Each client c n Obtain a convolutional neural network model X from the central server, which consists of I model layers consisting of K sequentially stacked composite layers and F fully connected layers. n , each composite layer includes a convolutional layer, a ReLU layer, a Dropout layer, and a pooling layer stacked in sequence;

[0043] Initialize X n The weight parameter is ω n , X n The weight parameter of the i-th model layer is ω n,i ;

[0044] In this example, the number of composite layers of the convolutional neural network is K=2, the number of fully connected layers is F=2, and I=K+F=4.

[0045] Step 4: The client initializes the local convolutional neural network model and the local privacy budget value.

[0046] Each client initializes a local convolutional neural network model

[0047] set up The privacy budget is initialization The weight parameter is

[0048] set up The weight parameter of the i-th model layer is

[0049] In this example,

[0050] Step 5: The client performs local convolutional neural network model Perform iterative training.

[0051] 5.1) Each client c n Randomly select B training data from the local training dataset with replacement as the local convolutional neural network model Input, get B predicted labels In this example, B=32;

[0052] 5.2) Each client c n For the obtained B predicted labels The cross entropy loss function is used to calculate each predicted label Its corresponding real label The loss value

[0053]

[0054] 5.3) Each client c n Utilization loss value For local convolutional neural network models The weight parameter Find the partial derivative to adjust the weight parameters Update and get the current updated weight parameters

[0055]

[0056] Where η>0 represents the learning rate. In this example, η=0.01.

[0057] 5.4) Each client c n Repeat steps 5.1) to 5.3) for τ times, and finally get the weight parameter: The trained local model τ≥10, in this example, τ=10.

[0058] Step 6: The client calculates the trained local model The non-intersecting elements of .

[0059] Reference Figure 2 , the implementation steps of this step include the following:

[0060] 6.1) Each client c n By updating the weight parameters of each layer of the local model calculate and judge The range of values ​​to be taken to calculate the quantization threshold ι:

[0061]

[0062] in, is the weight parameter of each layer of the local model after update, Represents the mth weight parameter of the i-th model layer of the local model; s is the weight parameter The fixed length value of the decimal part is set by the client. M is the number of weight parameters of the i-th layer of the quantized model. In this example, set s = 5;

[0063] 6.2) Each client c n The weight parameters of each layer of the updated local model are updated using the quantization threshold ι Compute random value function The output probability value It represents a random value function The probability of outputting ι / s;

[0064] 6.3) Each client c n According to the probability value and weight parameter Calculate the random value function The output is:

[0065] when hour,

[0066] when season

[0067] 6.4) Each client c n use The output result of the updated local model weight parameters of each layer Perform quantitative calculations, that is, weight parameters The decimal part of is exactly the fixed length s, and the quantized weight parameters are obtained.

[0068]

[0069] Among them, ‖·‖2 is the L2 norm operator, sgn(·) is the sign function;

[0070] 6.5) Each client c n Negotiate and select any hash function H:

[0071] H:{0,1} * →{0,1} κ

[0072] Among them, → is the calculation process of the hash function, which outputs a set of binary strings of arbitrary length into a set of values ​​of length κ; * is the length of the hash function input value, which is an arbitrary value; κ is the length of the hash function output value, κ>0,

[0073] In this example, each client c n Select SHA-1 function as hash function H, and the length of the hash function output value κ = 160;

[0074] 6.6) Each client c n Each weight parameter of the quantized model Convert to binary format and input hash function H to calculate the model weight parameters after quantization of each layer The hash value is:

[0075]

[0076] 6.7) Each client c n Quantize the model weight parameters of each layer Hash value Upload to the private set intersection server, the private set intersection server judges the parity of the number of clients N, and calculates the number of clients c according to each client. n Uploaded hash value Calculate the non-intersection element results of the weight parameters of each layer of the local model after client quantization

[0077] If N is an even number, go to step 6.8);

[0078] If N is an odd number, go to step 6.9);

[0079] 6.8) The private set intersection server calculates the XOR value of all hash values ​​as:

[0080]

[0081] Among them, the XOR value The position m corresponds to the weight parameter in the local model after quantization are intersection elements, and the weight parameters in the local model after quantization at the remaining positions are non-intersection elements;

[0082] 6.9) Private Set Intersection Server randomly selects a client c k Uploaded hash value And calculate the XOR value of the remaining N-1 hash values

[0083]

[0084]

[0085] 6.10) Private set intersection server determines XOR value Is it 0?

[0086] like Does not change The value of

[0087] like Then the private set intersection server calculates the position m corresponding to client c k Hash value With any client c j Hash value XOR value of

[0088]

[0089]

[0090] 6.11) Private set intersection server determines client c k Hash value With any client c j Hash value XOR value of Is it 0?

[0091] like Then The value of is changed to 1;

[0092] like Does not change The value of

[0093] 6.12) The private set intersection server will modify the XOR value The position m corresponds to the weight parameter in the local model after quantization As the intersection elements, the weight parameters in the local model corresponding to the remaining positions after quantization are taken as non-intersection elements;

[0094] 6.13) The private set intersection server aggregates the obtained intersection elements and non-intersection elements and sends them to each client c n ;

[0095] In this example, N=100, and each client c n Execute steps 6.1) to 6.8), and then execute step 6.13) to obtain the non-intersection element results of the quantized local model weight parameters.

[0096] In step 7, the client adds noise to the quantized local model based on the non-intersection element results.

[0097] Reference Figure 3 , the implementation steps of this step include the following:

[0098] 7.1) Each client c n Using the aggregated non-intersection element results and local models Privacy budget Calculate the Gaussian noise scale value that satisfies differential privacy guarantee for the non-intersection elements of the quantized local model

[0099]

[0100] in, represents the sensitivity, δ represents the probability that the local model does not meet the differential privacy technology after adding noise after quantization; in this example,

[0101] 7.2) Each client c n Using Gaussian noise scale value Calculate the noise value that satisfies differential privacy guarantees

[0102]

[0103] Among them, s represents a random number, μ represents the mathematical expectation of Gaussian noise. In this example, μ = 0;

[0104] 7.3) Each client c n Calculate the noise value for the intersection element that does not meet the differential privacy guarantee and will and The noise value is obtained by combining the corresponding positions of the intersection elements and the non-intersection elements

[0105] In this step, the larger the noise value added to the non-intersection elements of the quantized local model, the higher the degree of disturbance to the non-intersection elements, the lower the probability of leaking local training data, and the higher the privacy protection capability. The smaller the noise value added to the intersection elements of the quantized local model, the lower the degree of disturbance to the intersection elements, the smaller the impact on the model's prediction and classification results, and the higher the model's prediction and classification accuracy.

[0106] 7.4) Each client c n The noise value Weight parameters added to the quantized local model The weight parameters after perturbation are Local model The perturbed local model Upload to the central server.

[0107] Step 8: The central server trains the convolutional neural network model.

[0108] 8.1) The central server initializes the aggregation round to t=0 and sets the maximum aggregation round to T≥100. In this example, the maximum aggregation round is set to T=100.

[0109] 8.2) The central server randomly selects V t Client c v Uploaded perturbed local model The weight parameter Perform aggregation to get the current aggregation result ω t :

[0110]

[0111] Among them, ω t is the result after aggregation. In this example, V t =10;

[0112] 8.3) The central server determines whether t=T:

[0113] If so, the weight parameter is ω T The trained convolutional neural network model X T ;

[0114] Otherwise, let t = t + 1, and aggregate the result ω t Sent to each client c n , return to step 3.

[0115] The numbers of the above steps are for more clearly describing the implementation of the present invention, and the order of the numbers is not limited.

[0116] The effect of the present invention can be further illustrated by the following simulation experiments:

[0117] 1. Simulation conditions

[0118] The simulation experiment is implemented using Tensorflow and Keras libraries in a computing environment with Windows 10 system, Intel(R)Core(TM)i5-10400 CPU@2.90GHz, NVIDIAGeForce RTX 3070, 16GB RAM, and 4T memory.

[0119] 2. Simulation Content

[0120] Simulation 1: The convolutional neural network model is simulated and trained using the MNIST dataset using the present invention and the existing federated learning model training method. The classification accuracy of the convolutional neural network model in each training process is compared with the associative classification accuracy of the convolutional neural network model trained without adding noise. The results are as follows: Figure 4 shown.

[0121] from Figure 4 As can be seen, the convolutional neural network model trained without adding noise has the highest associative classification accuracy, but the model lacks privacy protection capabilities. This can be used as a reference standard for evaluating model classification performance: the closer the model is to the ideal result, the better the model's prediction and classification performance. The convolutional neural network model trained using the present invention has a higher classification accuracy than the convolutional neural network model trained using existing methods and is closer to the ideal result. This shows that the present invention can effectively improve the prediction and classification accuracy of convolutional neural network models.

[0122] Simulation 2, using the present invention and the existing federated learning model training method to perform privacy attacks on the convolutional neural network model trained on the MNIST dataset, and obtain the respective attack results, such as Figure 5 As shown. Among them,

[0123] Figure 5 (a) is the client's original training data sample subgraph, which can be used as a reference standard for evaluating the privacy protection ability of the model. That is, the closer the attack result is to the original training data sample subgraph, the worse the model's privacy protection ability is.

[0124] Figure 5 (b) Result graph of privacy attack on the convolutional neural network model trained using existing methods;

[0125] Figure 5 (c) is a graph showing the result of performing a privacy attack on the convolutional neural network model trained using the present invention.

[0126] from Figure 5As can be seen, when performing a privacy attack on a convolutional neural network model trained using existing methods, the attack results are very close to the original training data samples. This is because the existing methods perturb the local model too little, resulting in insufficient privacy protection for the local model. However, when performing a privacy attack on a convolutional neural network model trained using the present invention, the attack results differ significantly from the original training data samples, making it impossible for the attacker to obtain the original training data samples through the privacy attack. This is because the present invention perturbs the local model to a greater extent, resulting in a higher privacy protection capability for the local model. This demonstrates that the present invention can effectively improve the privacy protection capability of the local model.

[0127] The above description is only a specific example of the present invention and does not constitute any limitation to the present invention. Obviously, for professionals in this field, after understanding the content and principles of the present invention, it is possible to make various modifications and changes in form and details without departing from the principles and structure of the present invention. However, these modifications and changes based on the ideas of the present invention are still within the scope of protection of the claims of the present invention.

Claims

1. A convolutional neural network training method based on non-intersection differential privacy federated learning, characterized by: These include: (1) Initialization includes the central server, the private set intersection server and N clients C = {c1, c2, ..., c n ,,c N } of the federated learning system, where c n represents the nth client, n={1,2,…,N}, N≥2; (2) Each client c n Initialize a local training dataset of M images containing L target categories and annotate the target in each image, where L ≥ 2 and M ≥ 100. (3) Each client c n Get the convolutional neural network model X consisting of I model layers from the central server n , the X n The weight parameter is ω n , X n The weight parameter of the i-th model layer is ω n,i ; (4) Each client initializes the local convolutional neural network model set up The privacy budget value and weight parameters are and The weight parameter of the i-th model layer is (5) Client-side local convolutional neural network model Perform iterative training: Each client c n Randomly select B training data from the local training dataset with replacement as the local convolutional neural network model Input, get B predicted labels Using this label, the weight parameters are adjusted using stochastic gradient descent. Perform iterative update and obtain the weight parameter as The trained local model (6) The client calculates the trained local model The non-intersecting elements of: (6a) Each client c n The weight parameters of each layer of the updated local model Perform quantitative calculations and set the weight parameters The decimal part of is precisely fixed length, and the quantized weight parameters are obtained. (6b) Each client c n Calculate the model weight parameters after quantization of each layer Hash value And upload to the private set intersection server, the private set intersection server according to each client c n Uploaded hash value Calculate the non-intersection element results of the weight parameters of each layer of the local model after client quantization And then summarize it and send it to each client; (7) The client adds noise to the quantized local model based on the non-intersection element results: (7a) Each client c n Based on the aggregated non-intersection element results and the local model Privacy budget Calculate the noise value containing two different differential privacy guarantees (7b) Each client c n The noise value Weight parameters added to the quantized local model The weight parameters after perturbation are Local model The perturbed local model Upload to the central server; (8) The central server trains the convolutional neural network model: (8a) The central server initializes the aggregation round to t = 0 and sets the maximum aggregation round to T ≥ 100; (8b) Randomly select V t Client c v Uploaded perturbed local model The weight parameter Perform aggregation and get the current aggregation result ω t , and judge whether t=T is true: If so, the weight parameter is ω T The trained convolutional neural network model X T ; Otherwise, let t = t + 1 and send the aggregation result to each client c n , return to step (3).

2. The method according to claim 1, wherein: The I model layers described in step (3) include K composite layers and F fully connected layers stacked in sequence, that is, I = K + F; each composite layer includes a convolutional layer, a ReLU layer, a Dropout layer and a pooling layer stacked in sequence, where K ≥ 2 and F ≥ 2.

3. The method according to claim 1, wherein: In step (5), use the tag The weight parameters are adjusted using stochastic gradient descent Perform iterative updates, and the implementation steps include the following: (5a) Each client c n For the obtained B predicted labels The cross entropy loss function is used to calculate each predicted label Its corresponding real label The loss value (5b) Utilization loss value For local convolutional neural network models The weight parameter The partial derivative of the weight parameter Update and get the current updated weight parameters Among them, η>0 represents the learning rate; (5c) Repeat (5a) and (5b) for τ times, and finally get the weight parameter: The trained local model 4. The method according to claim 1, wherein: The quantized weight parameters obtained in step (6a) are It is expressed as follows: in, is the weight parameter of each layer of the local model after update, Represents the mth weight parameter of the i-th model layer of the local model; ‖·‖2 is the L2 norm operator, sgn(·) is the sign function, when hour, The definition is as follows: s is the weight parameter The fixed length value of the decimal part, ι is an integer, 0≤ι<s, when season 5. The method according to claim 1, wherein: In step (6b), each client c n Calculate the model weight parameters after quantization of each layer Hash value The implementation steps include the following: (6b1) Each client c n Negotiate and select any hash function H: H:{0,1} * →{0,1} κ Among them, → is the calculation process of the hash function, which outputs a set of binary strings of arbitrary length into a set of values ​​of length κ; * is the length of the hash function input value, which is an arbitrary value; κ is the length of the hash function output value, κ>0; (6b2) The quantized model weight parameters Each weight parameter in Convert to binary format and input into the hash function H to calculate the weight parameters of each layer of quantization model The hash value is: M is the number of weight parameters in the i-th layer of the quantized model.

6. The method according to claim 1, wherein: In step (6b), the private set intersection server calculates the value of each client c n Uploaded hash value Calculate the non-intersection element results of the weight parameters of each layer of the local model after client quantization The implementation steps include the following: (6b3) Determine the parity of the number of clients N: If N is even, execute (6b4); If N is an odd number, execute (6b5); (6b4) The private set intersection server calculates the XOR value of all hash values ​​as follows: Among them, the XOR value The position m corresponds to the weight parameter in the local model after quantization are intersection elements, and the weight parameters in the local model after quantization at the remaining positions are non-intersection elements; (6b5) The private set intersection server randomly selects a client c k Uploaded hash value And calculate the XOR value of the remaining N-1 hash values (6b6) Determine the XOR value Is it 0? like Does not change The value of like Then the private set intersection server calculates the position m corresponding to client c k Hash value With any client c j Hash value XOR value of (6b7) Determine client c k Hash value With any client c j Hash value XOR value of Is it 0? like Then The value of is changed to 1; like Does not change The value of (6b8) XOR the modified value The position m corresponds to the weight parameter in the local model after quantization As the intersection elements, the weight parameters in the local model corresponding to the remaining positions after quantization are taken as non-intersection elements.

7. The method according to claim 1, wherein: In step (7a), each client c n Compute the noise value with two different differential privacy guarantees The implementation steps include the following: (7a1) Each client c n Leveraging Privacy Budgets Calculate the Gaussian noise scale value that satisfies differential privacy guarantee for the non-intersection elements of the quantized local model in, represents the sensitivity, δ represents the probability that the local model after quantization does not meet the differential privacy technology after adding noise; (7a2) Using Gaussian noise scale value Calculate the noise value that satisfies differential privacy guarantees Among them, s represents a random number, μ represents the mathematical expectation of Gaussian noise; (7a3) Calculate the noise value for the intersection element that does not meet the differential privacy guarantee and will and The noise value is obtained by combining the corresponding positions of the intersection elements and the non-intersection elements 8. The method according to claim 1, wherein: In step (8b), V is randomly selected t Client c v Uploaded perturbed local model The weight parameter To perform aggregation, the formula is as follows: Among them, ω t is the result after polymerization, 1≤V t ≤N.

Images