Gradient leakage defense method based on model layering and characterization disturbance

Abstract

The invention discloses a gradient leakage defense method based on model layering and characterization disturbance. The method comprises the following steps: initializing model parameters of a global model and a local model; performing model parameter iteration updating on the local model to obtain updated universal layer model parameters and personalized layer model parameters of the local model; performing aggregation updating on the universal layer model parameters after the local model is updated to obtain universal layer model parameters after the global model is updated; and replacing the universal layer model parameters of the local model with the universal layer model parameters of the global model, and carrying out model parameter iteration updating on the local model until the training of the global model is completed. According to the method, the model is divided into the personalized layer and the universal layer, and the universal layer is subjected to characterization disturbance, so that the federal learning calculation overhead is reduced while the privacy protection requirement is met, and the performance of the model is improved. According to the method, effective information is difficult to extract from the image recovered by the gradient leakage attack, the gradient leakage attack can be effectively defended, and a better defense effect is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of federated learning, and in particular to a gradient leakage defense method based on model stratification and representation disturbance. Background Art

[0002] Federated learning is a distributed machine learning framework that uses data samples from multiple clients to train a global model by sharing local model update gradients. In the process of federated learning, clients do not need to share local private data, which helps protect data privacy. At the same time, data in federated learning is stored on local devices rather than central servers, which reduces the cost of data storage and management and reduces the burden of network transmission. However, the security of federated learning has not yet been fully guaranteed. Among them, gradient leakage attack refers to an attack technique in which an attacker infers client private data by accessing or analyzing the gradient information of the local model. In order to resist the security threats brought by various types of gradient leakage attacks, its defense technology is also constantly developing. According to the main defense objects in the process of federated learning, the existing defense methods can be roughly divided into three categories: defense methods based on original data, training models, and shared gradients.

[0003] In the defense method based on raw data, since the purpose of gradient leakage attack is to recover the client's private training data, the raw data can be protected directly before training. In the paper "A Framework For Evaluating Client Privacy Leakages In Federated Learning", Wei et al. found that using larger image batches for local training can alleviate the risk of gradient leakage. In addition, data augmentation technology increases the diversity of training data and improves the defense capability of local models by randomly transforming and expanding training data. In the paper "Automatic Transformation Search Against Deep Leakage From Gradients", Huang et al. proposed an automatic transformation search method to search for the best data augmentation strategy in the augmentation library to maintain model performance and prevent gradient leakage.

[0004] In the defense method based on training models, Wei et al. pointed out that controlling the number of local training iterations before gradient sharing can make it more difficult to reconstruct client private data. In addition, modifying the network structure can also effectively defend against gradient leakage attacks. In the paper "Dropout Against Deep Leakage From Gradients", Zheng et al. added an additional random dropout layer between the encoder and the classifier, reducing the attacker's success rate in reconstructing client private data.

[0005] In the defense method based on shared gradient, the use of encryption technology can ensure the security and privacy of gradients without affecting the performance of the model. In the paper "Homomorphic Encryption-Based Privacy-Preserving Federated Learning in IoT-Enabled Healthcare System", Zhang et al. proposed a federated learning masking method based on homomorphic encryption and secure multi-party computing, which effectively ensured the security of federated learning. In addition, the gradient compression algorithm compresses gradient information in different ways to reduce the possibility of leaking client private data. At the same time, perturbing gradient information is also a kind of defense method. In the paper "Federated Learning With Local Differential Privacy", Truex et al. first perform differential privacy protection on gradient information, and then selectively filter the perturbed gradient in the server to achieve defense. In the paper "Protect Privacy From Gradient Leakage Attack In Federated Learning", Wang et al. randomly select a certain proportion of slices from the shared gradient to add noise in each round of training, and maintain model convergence through global update compensation.

[0006] However, the above method still has shortcomings:

[0007] 1) Mainstream defense technology focuses on protecting the gradient information transmitted during the training process. When the attacker extracts prior information to compensate for the information loss caused by gradient defense, the security of the defense is greatly reduced, and it is difficult to resist advanced gradient leakage attacks.

[0008] 2) Mainstream defense techniques usually create perturbations on gradient information to provide a defensive effect, but this operation also damages local and global model performance, making it difficult to achieve a good balance between model performance and defense strength.

[0009] 3) Encryption-based defense methods have poor operating efficiency and high computational overhead, making them difficult to use in resource-constrained federated learning scenarios. Summary of the invention

[0010] The purpose of the present invention is to provide a gradient leakage defense method based on model stratification and representation disturbance.

[0011] The technical solution for achieving the purpose of the present invention is as follows:

[0012] A gradient leakage defense method based on model stratification and representation disturbance includes a global model stored in a cloud server and local models stored in two or more edge devices, wherein the local model has the same structure as the global model; the global model is an image classification model, which is trained in a federated learning manner;

[0013] The global model is composed of d network layers cascaded, namely {L1,L2,…,L d}; where {L1,L2,…L ε} is the general layer, {L ε+1 ,…L d-1 ,L d} is the personalization layer, ε∈{2,3,…,d-1};

[0014] include:

[0015] Step 1: Initialize the model parameters of the global model to obtain its general layer model parameters and personalized layer model parameters; the cloud server broadcasts the global model to the edge device; the edge device initializes the model parameters of the local model according to the global model to obtain its general layer model parameters and personalized layer model parameters;

[0016] Step 2, iteratively update the model parameters of all local models for S rounds to obtain the updated general layer model parameters and personalized layer model parameters of all local models;

[0017] Step 3: All updated general layer model parameters of local models are sent back to the cloud server for aggregate update to obtain updated general layer model parameters of the global model;

[0018] Step 4: The cloud server broadcasts the updated global model to the edge device, replaces the general layer model parameters of all local models with the general layer model parameters of the current global model, and returns to step 2;

[0019] Perform steps 2 to 4 for T rounds to complete the training of the global model;

[0020] The step 2 is to iteratively update the model parameters of the local model for S rounds to obtain the general layer model parameters and the personalized layer model parameters after the local model is updated, which are specifically:

[0021] 2.1 Freeze the current general layer model parameters, perform iterative updates on the local model for the first S-1 rounds, and obtain updated personalized layer model parameters; wherein, the iterative update of each round is: using the stochastic gradient descent algorithm, with minimizing the average cross entropy loss as the optimization goal on the local data set D, to update the personalized layer model parameters;

[0022] 2.2 Freeze the personalized layer model parameters updated in step 2.1, and update the local model for the Sth round to obtain the updated general layer model parameters; specifically, use the stochastic gradient descent algorithm to minimize the average cross entropy loss on the local data set D as the optimization goal, and update the general layer model parameters as follows:

[0023] The general layer {L1,L2,…L ε}Divided into middle layer and defense layer

[0024] The image data x in the local dataset D i Input the local model and calculate the cross entropy loss through the forward propagation process; where i is the sequence number of the image data;

[0025] In the forward propagation process, the image data x i Through the middle layer The output vector is used as the representation vector Perform perturbation and obtain the perturbed representation vector Among them, Q is the dimension of the representation vector;

[0026] According to the cross entropy loss, the general layer model parameters are updated in combination with the stochastic gradient descent algorithm; in the stochastic gradient descent algorithm, the representation vector R i Replaced with the perturbed representation vector

[0027] The image data x i Through the middle layer The output vector is used as the representation vector Perform perturbation and obtain the perturbed representation vector Specifically:

[0028] Calculate image data x i For the representation vector R i The Euclidean norm of the gradient of constitutes the representation vector R i Different elements in the image data x i The importance of the set in,

[0029]

[0030] In the formula, ri,q Represents image data x i The representation vector R i The qth element of i,h Represents image data x i The h-th pixel value of represents the operation of finding partial derivatives using the chain rule, M×N represents the image resolution, and c represents the number of color channels;

[0031] Calculate the variance of the model parameters of each network layer in the defense layer

[0032]

[0033]

[0034] In the formula, w l,u is the value of the u-th network parameter in the l-th network layer, is the mean of all network parameters in the lth network layer, n l is the number of network parameters in the lth network layer, l∈{ε-K l +1,ε-K l +2,…,ε};

[0035] Computing the network layer in the defense layer The standardized variance of the model parameters

[0036]

[0037] In the formula, is the maximum value of the variance of the model parameters of all network layers in the defense layer;

[0038] Computing the network layer in the defense layer The pruning rate Among them, p is the preset initial pruning rate;

[0039] Adjust the pruning rate according to the current round t Adjusted pruning rate Among them, λ t is the t-th power of the preset hyperparameter λ, t∈[1,T];

[0040] According to the collection The element values ​​in the set Sort the elements in descending order, and then put them in front The representation vector R corresponding to the elements i =[r i,1 ,…,r i,Q ] are set to 0, representing the vector R i =[ri,1 ,…,r i,Q The values ​​of other elements in ] are retained to obtain the perturbed representation vector

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] 1. The present invention divides the model into a personalized layer and a general layer, and performs characterization perturbations on the general layer, thereby meeting the privacy protection requirements while reducing the computational overhead of federated learning and improving the performance of the model.

[0043] 2. The present invention makes it difficult to extract valid information from images restored by gradient leakage attacks, and can effectively defend against gradient leakage attacks. Compared with existing gradient leakage defense technologies, the present invention has better defense effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is the overall framework diagram of gradient leakage defense based on model layering and representation perturbation.

[0045] Figure 2 Schematic diagram of the model layering and perturbation characterization structure. DETAILED DESCRIPTION

[0046] The present invention provides a gradient leakage defense method based on model stratification and representation disturbance, and the technical solution is as follows:

[0047] Step 1: Initialize the model parameters of the global model and the local model.

[0048] In the federated learning process, the neural network model stored on a single cloud server is called a global model, while the neural network model stored on the edge device is called a local model. The local model and the global model are required to have the same network structure and are used to classify the content of the input image data.

[0049] like Figure 1 As shown in the figure, the cloud server randomly initializes the model parameters of the global model and records the model parameters as θ, which are transmitted to the edge device through the channel as the local model. The model parameters of the local model are recorded as F; the local model and the global model use the same method to divide the network layer of the model into a personalized layer and a general layer. The global model is composed of d network layers cascaded, represented as {L1,L2,…,L d}, and then stratify it to get two subsets. Among them, the subset ({L1,L2,…L ε}) is the general layer, and the subset ({L ε+1 ,…L d-1 ,L d}) is a personalized layer, where ε is a predefined parameter, ε∈{2,3,…,d-1}. To simplify the representation, the model parameter of the universal layer of the global model is denoted by α G , and the model parameter of its personalized layer is denoted as α L , so the global model parameter θ=α G ∪α L ; Correspondingly, the general layer model parameters of the local model are recorded as β G , and the model parameter of the personalized layer is denoted as β L , that is, the local model parameter F = β G ∪β L After the model parameters are initialized, the global model parameters are Local model parameters

[0050] Step 2: During the federated learning process, the cloud server and edge devices execute steps 2.1 to 2.9 for a total of T rounds. T is the preset maximum number of federated learning rounds. For the tth round of federated learning update, the edge device first receives the global model from the cloud server, and then performs S rounds of iterative updates of local model parameters under the representation disturbance defense measures. After completing the iterative update of the local model, all edge devices transmit the common layer parameters of the local model back to the cloud server for aggregate update of the global model.

[0051] 2.1 For an edge device, it has a local dataset D. (x,y)∈D represents image data x with category label y. y∈{0,1} K×1 , M×N represents the image resolution, and K represents the total number of categories of image content. In the tth round of federated learning update during the training phase, the general layer parameters of the global model that completes the aggregate update in the t-1th round are received from the cloud server To update the common layer parameters of the local model of the edge device

[0052]

[0053] Therefore, the updated local model parameters are

[0054] 2.2 The local model performs S rounds of iterative model parameter updates locally. The first S-1 rounds update the personalized layer parameters, and the Sth round updates the general layer parameters. Among them, during the first S-1 rounds of updates, the general layer parameters of the local model are frozen so that they do not participate in the model parameter update. The stochastic gradient descent algorithm is used to minimize the average cross entropy loss on the local dataset to update the personalized layer parameters of the local model. The above process can be expressed as the following formula:

[0055]

[0056]

[0057] Where s∈{1,2,…,S-1} represents the round number of the local model parameter iteration; represents the personalized layer parameters of the local model after the sth iteration update round; |·| represents the number of elements in the set; k is the category index of the image data in D; (x i ,y i ) is the i-th image data in the dataset and its actual category label; y i,k is the value of the kth category of the actual category label. M(x i ,F) indicates that the i-th image data is input into the local model with model parameter F for forward propagation calculation and prediction results. It is the value of the kth category of the model output prediction result.

[0058] 2.3 After completing the first S-1 rounds of local model personalization layer parameter updates, the output vector of the model's middle layer is used as the representation vector for representation perturbation.

[0059] like Figure 2 As shown, the defense layer includes K l Network Layer Input data x into the local model and pass through the middle layer of the model Output representation vector R. Representation vector Q is the dimension of the representation vector. Then, the Euclidean norm of the input data for the representation vector gradient is calculated to measure the importance of different elements in the representation vector to the input data. i , and its calculation formula is:

[0060]

[0061] Among them, r i,q Represents the input data x i The qth element of the representation vector R, x i,h Represents the input data x i The h-th pixel value of represents the operation of finding partial derivatives using the chain rule. For the i-th input data, the set of importance of different elements in the vector to the input data is

[0062] 2.4 Calculating the variance of the defense layer model parameters

[0063] The calculation formula for the variance of model parameters of each network layer in the defense layer is:

[0064]

[0065]

[0066] in, is the value of the u-th network parameter in the l-th network layer, is the mean of all network parameters in the lth network layer, n l is the number of network parameters in the lth network layer, l∈{ε-K l +1,ε-K l +2,…,v}.

[0067] 2.5 Given the initial pruning rate p∈(0,1), according to step 2.4, the set of variances of network parameter values ​​of all network layers in the defense layer is obtained as Calculate the maximum of the above variances After that, the above variance is standardized to the range of [0,1] using the following formula:

[0068]

[0069] in is the standardized variance of the parameter value of the lth network layer. Update the pruning rate of different network layers

[0070] 2.6 Given a hyperparameter λ∈(0,1), adjust the pruning rate according to the number of training rounds t Specifically, the pruning rate of the tth round of updates can be expressed as follows:

[0071]

[0072] Steps 2.5 and 2.6 can also be done by only adding the network layer in the defense layer The parameter value variance is standardized, and then the pruning rate of the network layer is updated and adjusted according to the current number of training rounds t, that is,

[0073]

[0074]

[0075]

[0076] 2.7 For the representation vector R = [r1,…,r Q ] q , according to the set of importance Sort the corresponding elements in descending order. Take the one with the largest importance value elements, set the corresponding elements in the representation vector to 0, and keep the original values ​​of other elements. Indicates the rounding down operation. This step can be expressed by the following formula: Use this value as the demarcation to create a pruning mask:

[0077]

[0078] Therefore, the representation vector after representation disturbance is

[0079] 2.8 Freeze the personalized layers of the local model in the Sth round of local training By minimizing the average cross entropy loss on the local dataset D, the common layer parameters of the local model are updated using the stochastic gradient descent algorithm The stochastic gradient descent algorithm uses the output vectors of each network layer during the model forward propagation process. In this round of the stochastic gradient descent algorithm, the output vector of the middle layer is replaced with the perturbed representation vector R * .

[0080] 2.9 After local training, update the general layer parameters The data is sent back to the cloud server for aggregation update.

[0081] Step 3: For the tth round of federated learning updates, when all edge devices complete local update iterations, the cloud server receives the common layer parameters from all edge devices. Perform aggregate update, which can be expressed as:

[0082]

[0083] Get the global model after aggregation update C is the total number of edge devices.

[0084] Provide embodiments of the present invention below:

[0085] Step 1. The cloud server randomly initializes the model parameters of the global model and records the model parameters as θ, which are transmitted to the edge device through the channel as the local model. The model parameters of the local model are recorded as F; the local model and the global model use the same method to divide the network layer of the model into a personalized layer and a general layer.

[0086] In this embodiment, the global model and the local model are Resnet-18 models, and d is 10. Specifically, the global model is composed of d network layers cascaded, represented as {L1, L2, ..., L d}, and then stratify it to get two subsets. Among them, the subset ({L1,L2,…Lε}) is the general layer, and the subset ({L ε+1 ,…L d-1 ,L d}) is the personalization layer, and in this embodiment, ε=8.

[0087] To simplify the representation, the model parameters of the general layer of the global model are denoted as α G , and the model parameter of its personalized layer is denoted as α L , so the global model parameter θ=α G ∪α L ; Correspondingly, the general layer model parameters of the local model are recorded as β G , and the model parameter of the personalized layer is denoted as β L , that is, the local model parameter F = β G ∪β L After the model parameters are initialized, the global model parameters are Local model parameters

[0088] Step 2. In the federated learning process, the cloud server and edge device perform steps 2 to 10 for a total of T rounds, where T is set to 50. For an edge device, it has a local dataset D. (x,y)∈D represents image data x with category label y. y∈{0,1} K×1 , M×N represents the image resolution, and K represents the total number of categories of image content. In this embodiment, the CIFAR-100 dataset is used, M×N is 32×32, and K is 100. In the tth round of federated learning update in the training phase, the general layer parameters of the global model that completes the aggregation update in the t-1th round are received from the cloud server To update the common layer parameters of the local model of the edge device

[0089]

[0090] Therefore, the updated local model parameters are

[0091] Step 3. The local model performs S rounds of iterative model parameter updates locally, where S=5 in this embodiment. Specifically, the first S-1 rounds update the personalized layer parameters, and the Sth round updates the general layer parameters. In the first S-1 rounds of updates, the general layer parameters of the local model are frozen so that they do not participate in the model parameter update. The local model's personalized layer parameters are updated using the stochastic gradient descent algorithm with the optimization goal of minimizing the average cross entropy loss on the local dataset. The above process can be expressed as the following formula:

[0092]

[0093]

[0094] Where s∈{1,2,…,S-1} represents the round number of the local model parameter iteration; represents the personalized layer parameters of the local model after the sth iteration update round; |·| represents the number of elements in the set; k is the category index of the image data in D; (x i ,y i ) is the i-th image data in the dataset and its actual category label; y i,k is the value of the kth category of the actual category label. M(x i ,F) indicates that the i-th image data is input into the local model with model parameter F for forward propagation calculation and prediction results. It is the value of the kth category of the model output prediction result.

[0095] Step 4. After completing the first S-1 rounds of local model personalization layer parameter updates, use the output vector of the model intermediate layer as the representation vector for representation perturbation. Specifically, the defense layer includes K l Network Layer In this embodiment, K l = 2. Input data x into the local model and pass through the model's intermediate layer Output representation vector R. Representation vector Q is the dimension of the representation vector. Then, the Euclidean norm of the input data for the representation vector gradient is calculated to measure the importance of different elements in the representation vector to the input data. i , and its calculation formula is:

[0096]

[0097] Among them, r i,q Represents the input data x i The qth element of the representation vector R, x i,h Represents the input data x i The h-th pixel value of represents the operation of finding partial derivatives using the chain rule. For the i-th input data, the set of importance of different elements in the vector to the input data is

[0098] Step 5. Calculate the variance of the defense layer model parameters Specifically, the calculation formula for the variance of the model parameters of each network layer in the defense layer is:

[0099]

[0100]

[0101] in, is the value of the u-th network parameter in the l-th network layer of the general layer, is the mean of all network parameters in the lth network layer, n l is the number of network parameters in the lth network layer, l∈{ε-K l +1,ε-K l +2,…,ε}.

[0102] Step 6. Given an initial pruning rate p∈(0,1), in this embodiment p=0.8, according to the above steps, the set of variances of network parameter values ​​of all network layers in the defense layer is obtained as Calculate the maximum of the above variances After that, the above variance is standardized to the range of [0,1] using the following formula:

[0103]

[0104] in is the standardized variance of the parameter value of the lth network layer. Update the pruning rate of different network layers

[0105] Step 7. Given a hyperparameter λ∈(0,1), λ=0.95 in this example, adjust the pruning rate according to the number of training rounds t. Specifically, the pruning rate of the tth round of updates can be expressed as follows:

[0106]

[0107] Step 8. For the representation vector R = [r1,…,r Q ] q , according to the set of importance Sort the corresponding elements in descending order. Take the one with the largest importance value elements, set the corresponding elements in the representation vector to 0, and keep the original values ​​of other elements. Indicates the rounding down operation. This step can be expressed by the following formula: Use this value as the demarcation to create a pruning mask:

[0108]

[0109] Therefore, the representation vector after representation disturbance is

[0110] Step 9. Freeze the personalization layers of the local model in the fifth round of local training By minimizing the average cross entropy loss on the local dataset D, the common layer parameters of the local model are updated using the stochastic gradient descent algorithm The stochastic gradient descent algorithm uses the output vectors of each network layer during the model forward propagation process. In this round of the stochastic gradient descent algorithm, the output vector of the middle layer is replaced with the perturbed representation vector R * .

[0111] Step 10. After local training, update the general layer parameters The cloud server receives the common layer parameters from all edge devices after all edge devices complete the local update iterations. Perform aggregate update, which can be expressed as:

[0112]

[0113] Get the global model after aggregation update C is the total number of edge devices, and in this embodiment, C=100.

[0114] In order to illustrate the performance of the present invention, relevant experiments were carried out.

[0115] Experimental parameter reference: The datasets selected are CIFAR-10, CIFAR-100 and CelebA, which are common in the field of federated learning, to conduct experiments on model performance and defense effect respectively. DLG attack, IG attack and GGL attack are used as gradient leakage attack methods to test the defense effect of the scheme proposed in this invention, where DLG attack comes from the document "Deep Leakage From Gradients", IG attack comes from the document "Inverting Gradients-How Easy Is It To Break Privacy In Federated Learning?", and GGL attack comes from the document "Auditing Privacy Defenses In Federated Learning Via Generative Gradient Leakage". The parameter settings are the same as the original text. Four types of gradient leakage defense methods, namely gradient noise, gradient compression, gradient clipping and Soteria, are used as defense baselines, among which gradient noise and gradient compression come from the paper "Deep Leakage From Gradients", gradient clipping comes from the paper "Gradient-Leakage Resilient Federated Learning", and Soteria comes from the paper "Soteria: Provable Defense Against Privacy Leakage In Federated Learning From Representation Perspective"; the federated learning standard algorithm FedAvg is used as a baseline to test the model performance of the proposed scheme of the present invention, among which FedAvg comes from the paper "Communication-Efficient Learning Of Deep Networks From Decentralized Data". The experiment specifically assumes that there are 100 clients and 1 central server. Under the Non-IID setting, each client will obtain 2 types of randomly selected data samples. The experiment conducts 50 rounds of communication training on the CIFAR-100 dataset. Among them, the experiment sets the epoch of local training to 5, the batch-size to 10, and the client selection ratio to 0.1, that is, 10 users are selected for communication in each round. The experiment selected the stochastic gradient descent optimizer for federated learning training, and the learning rate was set to 0.01.The experiment mainly tests the defense effect and model performance of the present invention, using mean square error (MSE), peak signal-to-noise ratio (PSNR), learned perceptual image patch similarity (LPIPS) and structure similarity index measure (SSIM) as defense effect evaluation indicators, and the accuracy of the global model on the test set as the model performance evaluation indicator.

[0116] Table 1 Model performance test

[0117]

[0118] Table 1 shows the model performance of this experiment, selecting the final accuracy of the 50th communication round and the average accuracy of the next 10 communication rounds for display, and the best result is in bold. The experimental results show that the scheme proposed in the present invention achieves the best model performance.

[0119] Table 2 Defense effect test under GGL attack

[0120]

[0121] Table 2 shows the defense effect under GGL attack in this experiment. The best result is in bold. The "↑" in the index indicates that the larger the value, the better the defense effect; "↓" indicates that the smaller the value, the better the defense effect. The experimental results show that the technical solution provided by the present invention has achieved the best defense effect.

Claims

1. A gradient leakage defense method based on model stratification and representation disturbance, characterized in that: It includes a global model stored in a cloud server and local models stored in two or more edge devices respectively, and the local model has the same structure as the global model; The global model is an image classification model, which is trained in a federated learning manner; The global model is composed of d network layers cascaded, namely {L1,L2,…,L d }; Among them, {L1,L2,…L ε } is the general layer, {L ε+1 ,…L d-1 ,L d } is the personalization layer, ε∈{2,3,…,d-1}; include: Step 1: Initialize the model parameters of the global model to obtain its general layer model parameters and personalized layer model parameters; the cloud server broadcasts the global model to the edge device; the edge device initializes the model parameters of the local model according to the global model to obtain its general layer model parameters and personalized layer model parameters; Step 2, iteratively update the model parameters of all local models for S rounds to obtain the updated general layer model parameters and personalized layer model parameters of all local models; Step 3: All updated general layer model parameters of local models are sent back to the cloud server for aggregate update to obtain updated general layer model parameters of the global model; Step 4: The cloud server broadcasts the updated global model to the edge device, replaces the general layer model parameters of all local models with the general layer model parameters of the current global model, and returns to step 2; Perform steps 2 to 4 for T rounds to complete the training of the global model; The step 2 is to iteratively update the model parameters of the local model for S rounds to obtain the general layer model parameters and the personalized layer model parameters after the local model is updated, which are specifically: 2.1 Freeze the current general layer model parameters, perform iterative updates on the local model for the first D-1 rounds, and obtain updated personalized layer model parameters; wherein, the iterative update of each round is: using the stochastic gradient descent algorithm, with the minimization of the average cross entropy loss as the optimization goal on the local data set D, to update the personalized layer model parameters; 2.2 Freeze the personalized layer model parameters updated in step 2.1, and update the local model for the Sth round to obtain the updated general layer model parameters; specifically: use the stochastic gradient descent algorithm to minimize the average cross entropy loss on the local data set L as the optimization goal, and update the general layer model parameters as follows: The general layer {L1,L2,…L ε }Divided into middle layer and defense layer The image data x in the local dataset D i Input the local model and calculate the cross entropy loss through the forward propagation process; where i is the sequence number of the image data; In the forward propagation process, the image data x i Through the middle layer The output vector is used as the representation vector Perform perturbation and obtain the perturbed representation vector Among them, Q is the dimension of the representation vector; According to the cross entropy loss, the general layer model parameters are updated in combination with the stochastic gradient descent algorithm; in the stochastic gradient descent algorithm, the representation vector R i Replaced with the perturbed representation vector The image data x i Through the middle layer The output vector is used as the representation vector Perform perturbation and obtain the perturbed representation vector Specifically: Calculate image data X i For the representation vector R i The Euclidean norm of the gradient of constitutes the representation vector R i Different elements in the image data x i The importance of the set in, In the formula, r i,q Represents image data x i The representation vector R i The qth element of i,h Represents image data x i The h-th pixel value of represents the operation of finding partial derivatives using the chain rule, M×N represents the image resolution, and c represents the number of color channels; Calculate the variance of the model parameters of each network layer in the defense layer In the formula, w l,u is the value of the u-th network parameter in the l-th network layer, is the mean of all network parameters in the lth network layer, n l is the number of network parameters in the lth network layer, l∈{ε-K l +1,ε-K l +2,…,ε}; Computing the network layer in the defense layer The standardized variance of the model parameters In the formula, is the maximum value of the variance of the model parameters of all network layers in the defense layer; Computing the network layer in the defense layer The pruning rate Among them, p is the preset initial pruning rate; Adjust the pruning rate according to the current round t Adjusted pruning rate Among them, λ t is the t-th power of the preset hyperparameter λ, t∈[1,T]; According to the collection The element values ​​in the set Sort the elements in descending order, and then put them in front The representation vector R corresponding to the elements i =[r i,1 ,…,r i,Q ] are set to 0, representing the vector R i =[r i,1 ,…,r i,Q The values ​​of other elements in ] are retained to obtain the perturbed representation vector

Images