A double-strategy collaborative optimization method for privacy protection federated learning

Abstract

The application discloses a double-strategy cooperative optimization method for privacy protection federated learning, comprising the following steps: a center server constructs a global model and issues the current global model to a client; pruning actions and privacy actions of the global model are obtained based on a deep deterministic policy gradient method, the client performs structured pruning, gradient clipping and Gaussian noise adding operations, and obtains a perturbation update which is uploaded to the center server; the center server performs weighted aggregation on the perturbation updates uploaded by all participating clients to obtain a new round of global model; until the iteration is completed, the final global model is returned; the final global model is issued to each client, and each client processes the local data set held by the client by using the final global model, and transmits the data processing result protected by privacy to the center server to complete data publishing under privacy protection. The application has the characteristics of improving test precision, reducing communication overhead and ensuring training stability.

Description

Technical Field

[0001] This invention relates to the field of privacy protection technology, and more specifically, to a dual-policy collaborative optimization method for privacy-preserving federated learning. Background Technology

[0002] With the development of federated learning, McMahan et al. proposed the FedAvg algorithm, which enables multiple clients to collaboratively train the model without uploading the original data by weighted averaging of the local models on the clients. This has become a classic method in federated learning. However, under conditions of non-independent and identically distributed data and client differences, the local update directions of different clients are prone to shift, affecting the convergence speed and training stability of the global model. To alleviate these problems, FedProx limits the deviation of the local model by adding a proximal term to the local objective function of the clients, and SCAFFOLD corrects client drift and reduces local update variance by controlling variables. Some studies have also improved the convergence efficiency and system stability of heterogeneous federated learning from the perspective of hierarchical asynchronous aggregation. However, the above methods mainly focus on aggregation bias and convergence stability, and do not adequately consider the communication overhead caused by multiple rounds of model transmission and the performance degradation under privacy perturbations.

[0003] To reduce communication overhead during federated learning training, methods such as FedDST, PruneFL, FedMef, SparsyFed, and SpaFL introduce dynamic sparse training, model pruning, or adaptive sparse training into federated learning to reduce the storage, computation, and communication burden during model training. While these methods can effectively reduce communication volume or model complexity, most of them focus on compression ratio, sparsity ratio, or lightweight training, without fully considering the impact of differential privacy noise on effective gradient information and the quality of aggregated updates.

[0004] In terms of privacy protection, differential privacy limits the impact of a single sample or client update on the model output through gradient clipping and random noise injection, and is a commonly used privacy protection mechanism in federated learning. DP-SGD provides a differential privacy protection approach for deep learning model training by clipping gradients and adding Gaussian noise. DP-SCAFFOLD introduces differential privacy constraints into the SCAFFOLD algorithm, considering both privacy protection and client drift correction under heterogeneous data conditions. However, these methods usually focus on adjusting privacy parameters and are difficult to coordinate with model compression strategies for optimization.

[0005] As research has progressed, existing studies have shown that model compression and differential privacy have the potential to be combined. In heterogeneous federated training scenarios, fixed pruning strategies struggle to simultaneously address the resource constraints and model expression needs of different clients. Furthermore, fixed differential privacy parameters can easily introduce excessive perturbations on some clients, leading to decreased model accuracy and training instability. Moreover, model pruning and differential privacy are not independent: the former alters the structure and scale of model updates, while the latter affects update quality and aggregation effects. They are clearly coupled during training, but existing methods still primarily rely on fixed parameters, empirical rules, or single-module adaptive adjustment, lacking a unified modeling of the coupling relationship between pruning strength and privacy strength. Especially when structured pruning and differential privacy act simultaneously on model updates, pruning alters the effective parameter space and gradient distribution, and privacy noise further impacts the quality of aggregation updates. Optimizing only the compression ratio or privacy strength can easily lead to over-compression, excessive perturbation, or inconsistent optimization objectives. Therefore, the key to improving the overall performance of privacy-preserving federated learning lies in how to dynamically adjust the layer-by-layer channel retention rate, gradient pruning threshold, and Gaussian noise intensity based on the training state, and in establishing a unified trade-off between model accuracy, communication overhead, and privacy overhead. Summary of the Invention

[0006] The purpose of this invention is to design and develop a dual-policy collaborative optimization method for privacy-preserving federated learning. Through a collaborative mode of centralized decision-making by the server and local execution by the client, the method can improve the strength of privacy protection and reduce the cumulative communication volume.

[0007] The technical solution provided by this invention is as follows:

[0008] A dual-policy collaborative optimization method for privacy-preserving federated learning includes the following steps:

[0009] Step 1: The central server builds a global model and distributes it to a subset of clients. The current global model is distributed to each client in the middle;

[0010] Step 2: Obtain the pruning and privacy actions of the global model based on the deep deterministic policy gradient method. The client performs structured pruning, gradient pruning and Gaussian noise addition operations, and uploads the perturbation update to the central server.

[0011] Step 3: The central server performs weighted aggregation on all perturbation updates uploaded by participating clients to obtain a new round of global model;

[0012] Step 4: When the preset total number of communication rounds is reached, the algorithm terminates and returns to the final global model;

[0013] Step 5: The final global model is distributed to each client. Each client uses the final global model to process its local dataset, obtains privacy-protected data processing results, and then transmits them to the central server to complete the data release under privacy protection.

[0014] Preferably, the global model includes an input layer, at least two pruning layers, and an output layer, wherein the at least two pruning layers include convolutional layers and / or fully connected layers.

[0015] Preferably, step two specifically includes:

[0016] Step 1, construct the first The client in the first Shared state of the wheel:

[0017] ;

[0018] in, For the first Evaluation accuracy of global model data For the first Evaluation loss of the global model data. For the first Heterogeneous characteristics of individual clients For the first Training feedback information for each client in historical rounds;

[0019] Step 2: Update the pruning agent based on the deep deterministic policy gradient method to obtain the pruning action;

[0020] Step 3: The client completes structured pruning and local training based on the pruning actions to obtain the updated local model;

[0021] Step 4: Update the privacy agent based on the deep deterministic policy gradient method to obtain privacy actions;

[0022] Step 5: The client performs model cropping based on privacy actions to obtain the cropped local model;

[0023] Step 6: Inject Gaussian noise into the effective update after pruning to obtain the final perturbation update for uploading.

[0024] Preferably, the first The heterogeneous characteristics of each client are used to describe the differences between different clients in terms of data scale, computing power and communication conditions;

[0025] The data scale is obtained by counting the number of local samples on the client, the computing power is estimated by the local training time on the client, the number of samples processed per unit time or the computing power of the device, and the communication conditions can be estimated by the model upload time, historical communication latency or available bandwidth.

[0026] Preferably, the first The training feedback information of each client in historical rounds includes the first... Pruning actions of each client in previous rounds Privacy actions Communication costs Privacy expenses Compressed revenue and performance feedback after model update , This indicates the historical iteration identifier in the accumulation operation.

[0027] Preferably, the deep deterministic policy gradient method for updating the pruning agent in step 2 specifically includes:

[0028] The state space is the first The client in the first Shared state of wheels ;

[0029] The action space is a continuous vector. ;

[0030] The reward function is the first The client in the first Joint rewards of the round ;

[0031] State transition to ;

[0032] The Actor network takes the shared state as input and outputs the first... The client in the first The pruning action of the wheel The mapping relationship is expressed as follows: ;

[0033] Critic network with shared state and the first The client in the first The pruning action of the wheel is the input, and the output is a state-action value function. The Critic network is updated by minimizing the temporal difference error, and the Actor network is updated by policy gradient ascent.

[0034] in, The policy network representing the pruning agent. This represents the network parameters for the pruning strategy. For the first The client in the first Round targeting the first The retention rate of the prunable layer , .

[0035] Preferably, the first The client in the first Joint rewards of the round satisfy:

[0036] ;

[0037] In the formula, Indicates the first Global test accuracy increment. This represents the normalized communication cost. This represents the normalized privacy overhead. This indicates normalized compressed returns. , , , All coefficients greater than 0 are trade-off coefficients.

[0038] Preferably, the structured pruning specifically includes:

[0039] Using the output channels of convolutional layers or the output neurons of fully connected layers as pruning units, the client calculates the importance score of each pruning unit based on the channel parameter norm or neuron parameter norm, and sorts them from high to low importance scores. The retention rate generated by the pruned agent is then considered. Determine the number of channels or neurons that need to be retained in this layer. The system retains the channels or neurons with the highest importance scores, while masking the rest. This generates a structured mask that preserves the labeling information corresponding to the output channels of the convolutional layer or the output neurons of the fully connected layer. The client applies the structured mask to the global model distributed by the central server to obtain the compressed model for the current round. After the client completes local training on the compressed model, it obtains the locally updated model.

[0040] Preferably, the privacy action is as follows:

[0041] ;

[0042] In the formula, For privacy-preserving intelligent agents in the first The first round Gradient clipping thresholds generated by each client. For privacy-preserving intelligent agents in the first The first round Gaussian noise intensity coefficients generated by each client.

[0043] Preferably, steps 2 and 4 further include:

[0044] Transition sample ( The samples are stored in the experience replay pool B, and the policy network and value network of the pruned agent and privacy agent are updated using the small batch of samples in the experience replay pool.

[0045] The beneficial effects of this invention are as follows:

[0046] (1) The present invention designs and develops a dual-strategy collaborative optimization method for privacy-preserving federated learning. It adopts a collaborative mode of "centralized decision-making by the server and local execution by the client". The model compression strategy and privacy protection strategy are constructed on the server side. The structured pruning control and differential privacy parameter adjustment are incorporated into the unified optimization process. Through the sharing of training state and joint feedback mechanism, the layer-by-layer channel retention rate, gradient pruning threshold and Gaussian noise intensity are adaptively adjusted so that the model compression intensity and privacy protection intensity can be dynamically matched with the training process.

[0047] (2) The dual-strategy collaborative optimization method for privacy-preserving federated learning designed and developed in this invention improves the strength of differential privacy protection, enhances the stability during global training, effectively reduces the cumulative communication volume, and maintains good model accuracy and training stability under different privacy budget conditions, thus verifying the effectiveness of collaborative control between model compression strategy and privacy protection strategy. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of the optimization framework for the dual-policy collaborative optimization method for privacy-preserving federated learning described in this invention.

[0049] Figure 2 This is a schematic diagram showing the comparison of test accuracy curves of different methods described in this invention on the MNIST dataset.

[0050] Figure 3 This is a schematic diagram showing the test loss comparison curves of different methods described in this invention on the MNIST dataset.

[0051] Figure 4 This is a schematic diagram showing the cumulative communication volume comparison curves of different methods described in this invention on the MNIST dataset.

[0052] Figure 5 This is a schematic diagram showing the comparison of test accuracy curves of different methods described in this invention on the FEMNIST dataset.

[0053] Figure 6 This is a schematic diagram showing the test loss comparison curves of different methods described in this invention on the FEMNIST dataset.

[0054] Figure 7 This is a schematic diagram showing the cumulative communication volume comparison curves of different methods described in this invention on the FEMNIST dataset.

[0055] Figure 8 This is a schematic diagram showing the comparison of test accuracy curves of different methods described in this invention on the CIFAR-10 dataset.

[0056] Figure 9 This is a schematic diagram showing the test loss comparison curves of different methods described in this invention on the CIFAR-10 dataset.

[0057] Figure 10 This is a schematic diagram showing the cumulative communication volume comparison curves of different methods described in this invention on the CIFAR-10 dataset. Detailed Implementation

[0058] The present invention will now be described in further detail with reference to the accompanying drawings, so that those skilled in the art can implement it based on the description.

[0059] This invention provides a dual-policy collaborative optimization method (DAPP-FL) for privacy-preserving federated learning, which uses a federated learning system consisting of a central server and The client consists of 1 client, the 1st Each client holds a local dataset. , Due to differences in data scale, data distribution, computing power, and communication bandwidth among clients, the federated training process exhibits significant heterogeneity. Let w be the global model parameter in the t-th round. t The server selects a subset S from the client set to participate in the training. t ⊆{1,2,…,N} and distribute the global model to it.

[0060] like Figure 1 As shown, the DAPP-FL framework described in this invention constructs a federated pruning and differential privacy co-optimization framework for heterogeneous clients, jointly adjusting the model compression strength and privacy protection strength during unified training. Specifically, it includes the following steps:

[0061] Step 1: The central server builds a global model. and to a subset of clients The current global model is distributed to each client in the middle;

[0062] The global model ( ) is a deep neural network, specifically including an input layer, at least two pruning layers and an output layer. The pruning layers include convolutional layers and / or fully connected layers, and L represents the total number of pruning layers in the model.

[0063] Initialization is required before the global model is deployed, i.e. Standard parameter initialization methods (such as Xavier initialization or Kaiming initialization) are used to randomly initialize all trainable parameters of the model (convolutional kernel weights of convolutional layers, weight matrices of fully connected layers, and bias vectors), and the random seed is fixed to ensure the reproducibility of the experiment. This does not include cases where pre-trained weights are loaded or parameter values ​​are manually specified.

[0064] Client subset , The set of all clients in the federated learning system. The central server is in the Wheel from all client sets The selected subset of clients participating in the training;

[0065] Step 2: Obtain the global model using the deep deterministic policy gradient (DDPG) method based on Actor-Critic. The client performs structured pruning, gradient pruning, and Gaussian noise addition operations to obtain perturbation updates and upload them to the central server.

[0066] A pruning agent and a privacy agent are deployed on the central server side to build a dual-agent collaborative decision-making mechanism. The pruning agent is responsible for generating client-level pruning actions, while the privacy agent is responsible for generating client-level privacy actions. The two agents share the training state and perform collaborative updates under the constraint of a unified reward function, thereby avoiding the pruning strategy and the privacy strategy from pursuing local optima separately.

[0067] Therefore, step two specifically includes:

[0068] Step 1: To describe the decision-making basis of the two agents on the server side, construct the first... The client in the first Shared state of the wheel:

[0069] ;

[0070] in, For the first Evaluation accuracy of global model data For the first Evaluation loss of the global model data. For the first Heterogeneous characteristics of individual clients For the first Training feedback information for each client in historical rounds;

[0071] and Used to reflect the overall training state of the current global model, the central server performs validation and evaluation on the global model after each round of aggregation. It performs forward inference once on the reserved validation set and calculates the validation accuracy and validation loss as global state information.

[0072] ;

[0073] ;

[0074] In the formula, The total number of samples in the validation set, Predict labels for the model, For real labels, Here, C is the indicator function; C is the total number of categories. For the model to sample Predicted as category The probability, One-hot encoding of the actual label;

[0075] Heterogeneous features This feature describes the differences among clients in terms of data size, computing power, and communication conditions. It is obtained from observable client statistics before or during training. The data size is determined by the number of local samples on each client. Computing power is estimated by the client's local training time, the number of samples processed per unit time, or the device's computing power. All three methods satisfy the consistent monotonicity that "the faster the processing speed, the stronger the computing power." One of the three methods should be used.

[0076] ① When estimating based on local training time, the benchmark test... The actual time taken for a client to complete one round of local training (seconds), and thus the computing power satisfies:

[0077] ;

[0078] In the formula, For computing power, This represents the median of the benchmark test results before the first training for all clients.

[0079] ② When processing sample size estimation per unit time, the computational capability meets the following requirements:

[0080] ;

[0081] ;

[0082] In the formula, For the first Total number of local samples (samples) for each client;

[0083] ③ When estimating based on equipment computing power, the computing power is directly proportional to the equipment's theoretical FLOPS;

[0084] Communication conditions can be estimated from model upload time, historical communication delay, or available bandwidth. All three methods satisfy the consistent monotonicity that "the faster the transmission speed, the better the communication conditions." One of the three methods can be selected for use.

[0085] ① When estimating the upload time based on the model, measure the first... The actual time taken for each client to upload a model with a unit parameter quantity Communication conditional components and Inversely proportional, that is:

[0086] ;

[0087] In the formula, for This represents the median of the benchmark test results before the first training for all clients.

[0088] ② When estimating based on historical communication delays, the statistics for the first... Average round-trip latency of each client in previous rounds Communication conditional components and Inversely proportional, that is:

[0089] ;

[0090] In the formula, This represents the median historical latency for all clients.

[0091] ③ When estimating based on available bandwidth, measure the first... Network available bandwidth for each client (Mbps), conditional communication components and Proportional, that is:

[0092] ;

[0093] In the formula, This represents the maximum bandwidth for all clients in the benchmark test.

[0094] Training feedback information This data characterizes the client's past pruning performance, privacy perturbation intensity, and communication behavior. It is maintained by the central server based on historical records during federated training and primarily includes the client's pruning actions in previous rounds. Privacy actions Communication costs Privacy expenses Compressed revenue and performance feedback after model update ,in, This indicates the historical iteration identifier in the accumulation operation.

[0095] Among them, performance feedback That is, the first The client participated in the first The increase in test accuracy of the global model on the validation set after each round of training. For the first The validation accuracy of the global model after round aggregation. This represents the accuracy of the previous round of validation; specifically, it is obtained by the server performing weighted aggregation in each round to obtain a new global model. Then, perform forward inference once on the reserved validation set, and obtain the proportion of correctly predicted samples. Read the stored The difference is obtained by taking the subtraction. And store the value in the client. Historical feedback records;

[0096] For the initial value of the shared state, in the initial round At that time, A0 and ℓ0 are determined by the global model w 0 The values ​​are obtained through random initialization, where A0 is the random guessing accuracy (A0 = 1 / C in C-class classification tasks) (e.g., 10% for MNIST and 10% for CIFAR-10); ℓ0 is the corresponding cross-entropy loss value (e.g., approximately 2.30 on MNIST).

[0097] Before the first round of federated communications begins, the central server issues benchmark test instructions to each client, and each client performs the following probe operations to determine heterogeneous characteristics. :

[0098] 1. Data Scale: Directly calculate the total number of samples in the client's local dataset;

[0099] 2. Computing power: The client uses the initial global model issued by the server. Based on this, a mini-training run (e.g., 1 epoch or 1 batch) is performed on the local dataset, and the actual time taken for this local training is measured and reported; the server uses this to estimate the number of samples processed by the client per unit time or the equivalent computing power level.

[0100] 3. Communication conditions: The client uploads a standard-sized probe file (e.g., a 1MB virtual model parameter file) to the server, measures and reports the actual transmission time; the server uses this to estimate the client's available uplink bandwidth or equivalent communication capacity.

[0101] The above benchmark test is executed only once during the initialization phase, and the results are as follows: This heterogeneous feature is stored in the server-side state record as a client-side feature, and used for subsequent dual-agent state construction.

[0102] In cases where there are insufficient initial training rounds or historical information, h k,0 Initialize it as a zero vector, i.e., h k,0 =0, or use 0.5 times the measured value of the first round as the initial default value after the first round of training is completed.

[0103] If there is insufficient initial training rounds or historical information, the default value or the statistical mean of existing rounds can be used for initialization.

[0104] Shared states need to be normalized before being input into the agent's policy network to eliminate the influence of different dimensions on the decision-making process.

[0105] Step 2: Update the pruning agent based on DDPG to obtain pruning actions, specifically:

[0106] The pruning agent is trained using the Deep Deterministic Policy Gradient (DDPG) framework, and its state space is a shared state. The action space is a continuous vector. The reward function is a joint reward. The state transition is Among them, the Actor network is in a shared state. For input, output the first... The pruning action of the wheel The mapping relationship is expressed as follows: Critic network and Input and output state-action value functions The Critic network is updated by minimizing the temporal difference error, and the Actor network is updated by policy gradient ascent.

[0107] in, The policy network representing the pruning agent. This represents the network parameters for the pruning strategy.

[0108] In the formula, For the first The client in the first Round targeting the first The retention rate of the prunable layer , ,Right now It is an action vector The Each component is given in the form of a layer vector, which can perform differentiated compression for different clients and different network layers, thereby improving the method's adaptability to heterogeneous resource conditions.

[0109] Step 3, the client according to After completing structured pruning and local training, the locally updated model is obtained. .

[0110] During the model compression stage, structured channel pruning is employed, using the output channels of convolutional layers or the output neurons of fully connected layers as pruning units, without deleting entire layers. This is to avoid significant changes in network topology and affecting training stability. For the th There are pruning layers, assuming they have a total of _____ layers. For each output channel or output neuron, the client calculates the importance score of each pruned unit based on the channel parameter norm, neuron parameter norm, or other importance evaluation methods, and sorts them from highest to lowest importance score. Then, based on the retention rate generated by the pruned agent... Determine the number of channels or neurons that need to be retained in this layer. The system retains the channels or neurons with the highest importance scores, while masking the rest. This generates a structured mask that preserves the identification information corresponding to the output channels of the convolutional layer or the output neurons of the fully connected layer. The client applies the structured mask to the global model sent by the central server to obtain the compressed model for the current round. The structured masks of each layer jointly determine the effective computational units (output channels or output neurons with a mask value of 1, i.e., the computational units that are retained after pruning and participate in forward inference and backward propagation) and effective upload parameters (model weight parameters corresponding to the positions with a mask value of 1, i.e., the weight values ​​of the retained channels or neurons) in the compressed model, thereby reducing the scale of effective parameters in local computation and upload communication.

[0111] Specifically, the client calculates the importance score of each pruned unit based on the L2 norm of the convolution kernel parameters corresponding to each output channel:

[0112] For the The first pruning layer The importance score for each output channel is calculated using the following formula:

[0113] ;

[0114] In the formula, For the first The first pruning layer The importance score of each output channel or output neuron This is the input channel index, used to traverse the connection weights between this output channel and all input channels; and These are the element position indices in the height and width directions of the convolution kernel, respectively, used to locate the specific weight parameters within the convolution kernel matrix; This is the convolution kernel weight tensor corresponding to that channel. Input the number of channels. and The height and width of the convolution kernel, For the input dimension index, For the first The first pruning layer The output neuron and the first Weight parameters between inputs, For input dimensions;

[0115] The structured mask m k,t The mask is a binary matrix with the same dimension as the model parameters, and its elements take values ​​of 0 or 1, where 1 indicates that the positional parameter is preserved, and 0 indicates that the positional parameter is pruned and masked. For convolutional layers, the mask is generated according to the output channel dimension, and all convolutional kernel parameters within the same output channel share the same mask value. For fully connected layers, the mask is generated according to the output neuron dimension, and all weight parameters corresponding to the same output neuron share the same mask value. The mask is determined by the retention rate. Together with the importance score ranking results, only the mask positions corresponding to channels or neurons are set to 1, and the rest are set to 0.

[0116] After the client completes local training on the compressed model, it obtains the locally updated model. .

[0117] Step 4: Calculate the locally updated model The gradient norm, gradient mean, gradient variance, cumulative noise intensity injected by the client in previous rounds, and amount of privacy budget consumed:

[0118] ;

[0119] ;

[0120] ;

[0121] ;

[0122] ;

[0123] In the formula, For locally updated models The gradient norm, For locally updated models The One parameter element, For locally updated models The gradient mean, For locally updated models gradient variance, This represents the cumulative noise intensity injected by the client in previous rounds. For the first The Gaussian noise intensity coefficient of the wheel, For the first The gradient clipping threshold is used to limit the norm of the local model update on the client. When the local model's gradient norm exceeds this clipping threshold, the update is scaled proportionally. This indicates the traversal identifier of the previous iteration in the accumulation operation, which needs to be... The client in round 0 to round 1 The historical values ​​of each round are accumulated; when the local model update norm does not exceed the pruning threshold, the original update remains unchanged; This represents the amount of privacy budget already consumed by the client in previous rounds. For the first The product of the privacy budget consumed in each round and the two. For the first The standard deviation of actual wheel noise.

[0124] By the The client in the first Model update after local training The gradient norm, gradient mean, and gradient variance constitute the local update statistical features. The historical privacy overhead consists of the cumulative noise intensity injected by the client in previous rounds and the amount of privacy budget consumed. The privacy agent is also trained using the Deep Deterministic Policy Gradient (DDPG) framework, sharing state... Extended state consisting of local update statistical features and historical privacy overhead. The state space and action space are two-dimensional continuous vectors. The reward function is a joint reward shared with the pruning agent. The Actor network of privacy-preserving intelligent agents is in an extended state. Input, output privacy actions Critic network and Input, output state-action value function Furthermore, the Critic network is updated by minimizing the temporal difference error, and the Actor network is updated by policy gradient ascent. It shares the same experience replay pool and joint reward signal with the pruned agent, thereby achieving adaptive adjustment of the privacy protection strength based on the local update magnitude of the client and historical privacy consumption.

[0125] Unlike the approach of uniformly setting privacy parameters, this invention generates privacy control variables for different clients to adapt to the differentiated privacy protection needs under the constraints of non-independent and identically distributed data and heterogeneous resources.

[0126] Since pruning alters the model update structure and gradient distribution, while privacy perturbations affect update quality and aggregation effects, these two types of actions need to be collaboratively optimized through shared state and joint rewards. This invention constructs a joint reward function to comprehensively measure changes in model accuracy, communication costs, privacy overhead, and compression benefits.

[0127] ;

[0128] In the formula, As a reward signal, Indicates the first The global test accuracy increment is specifically the difference between the test accuracy of the current round and the test accuracy of the previous round. This represents the normalized communication cost, specifically the number of valid uploaded parameters by the client in this round compared to the number of parameters in the complete dense model (determined by the global model). The ratio of (the network structure is determined); This represents the normalized privacy overhead, specifically the ratio of the noise variance in the current round to the preset maximum noise variance. ); This represents the normalized compression gain, specifically the ratio of the difference between the number of parameters in the complete dense model and the number of effective uploaded parameters to the total number of parameters in the complete model. , , , All coefficients greater than 0 are trade-off coefficients.

[0129] In this embodiment, the Take 1.0, Take the upper bound of the initial L2 norm of the model parameters (e.g., 10.0), and the product of the two is the maximum standard deviation of the noise that can be injected;

[0130] This reward function establishes a unified trade-off between model accuracy, communication efficiency, and privacy protection: the reward increases when model accuracy improves and communication costs decrease; the reward decreases when privacy perturbations are too strong or model performance degrades. The pruning agent and the privacy agent share the reward signal defined in the above formula, thus ensuring that they update collaboratively around the same optimization objective. Unlike strategies that independently adjust the pruning rate or privacy parameters, this invention incorporates pruning and privacy actions into a unified state feedback and reward constraint for collaborative updates. This allows model compression, privacy perturbations, and performance feedback to work together under the same optimization objective, thereby enhancing the coordination between the pruning and privacy strategies.

[0131] Step 5: To limit the impact of individual client updates on the aggregation result, L2 pruning is first performed to obtain the pruned model:

[0132] ;

[0133] Step 6: Inject Gaussian noise into the effective update after pruning to obtain the final uploaded perturbation update:

[0134] ;

[0135] ;

[0136] In the formula, Indicates the first The client in the first Update the model obtained after local training. Indicates the first The perturbation model update that is ultimately uploaded to the server by each client. This represents a structured mask generated by the pruning action, used to identify the locations of valid channels, valid neurons, or valid parameters that are retained in the current round; This indicates element-wise multiplication, used to apply the structured mask separately to the trimmed model update and Gaussian noise. Indicates the first The client in the first The Gaussian noise vector generated by the round, The identity matrix, consistent with the model update dimension, is used to characterize the covariance structure of the Gaussian noise vector, ensuring that noise in different parameter dimensions is independent and that the variance of noise in each dimension is determined by... and To be determined jointly.

[0137] This shows that the present invention does not uniformly apply perturbation to the update of the complete dense model, but performs pruning and noise addition processing within the effective parameter subspace determined by the structured mask.

[0138] Steps 2 and 4 also include:

[0139] Transition sample ( The samples are stored in the experience replay pool B, and the policy network and value network of the pruned agent and privacy agent are updated using the small batch of samples in the experience replay pool.

[0140] Step 3: The central server performs weighted aggregation on all perturbation updates uploaded by participating clients to obtain a new round of global model:

[0141] ;

[0142] In the formula, Indicates the first Local sample size for each client, Indicates that it is in the first The perturbation update in each round shows that the present invention does not change the basic paradigm of federated learning, which is "local training - server aggregation". Instead, it introduces adaptive pruning and differential privacy collaborative control in the update generation stage, thereby reducing communication costs and enhancing privacy protection capabilities while ensuring model trainability.

[0143] Step 4: When the preset total number of communication rounds is reached, the algorithm terminates and returns to the global model. ;

[0144] Step 5: The final global model is distributed to each client. Each client uses the final global model to process its local dataset, obtains privacy-protected data processing results, and then transmits them to the central server to complete the data release under privacy protection.

[0145] The total number of communication rounds is set to 100, 120, or 150 depending on the task complexity.

[0146] In the first round of communication At that time, the first round of pruning action of the pruning agent is conservatively initialized by the central server to a retention rate of 0.5% for each layer. of dimensional vector, i.e. At this point, the structured mask The matrix consists entirely of 1s; the first round of privacy actions of the privacy agent is conservatively initialized by the central server. Gradient clipping threshold The Gaussian noise intensity coefficient is a preset empirical constant (or the statistical mean of the first round of local update norms measured during the testing phase). This indicates that no Gaussian noise is injected in the first round; from the... The pruning agent starts by pruning based on the shared state. Autonomously generates pruning actions; the privacy agent bases its actions on the extended state. Autonomously generate privacy actions.

[0147] Therefore, the algorithm flow of DAPP-FL described in this invention is as follows:

[0148]

[0149] To verify the performance of DAPP-FL described in this invention, its effectiveness was validated on the MNIST, FEMNIST, and CIFAR-10 datasets. MNIST was used to test the method's performance on basic image classification tasks, FEMNIST was used to evaluate the method's adaptability in natural heterogeneous client scenarios, and CIFAR-10 was used to examine its generalization ability in complex image classification tasks. Regarding the model, for the MNIST dataset, the global model used was LeNet, containing 2 convolutional layers and 3 fully connected layers, for a total of 5 trainable layers; for the FEMNIST dataset, a lightweight convolutional neural network was used, containing 2 convolutional layers and 2 fully connected layers; for the CIFAR-10 dataset, Re... sNet18 consists of one initial convolutional layer, eight residual blocks (a total of 16 convolutional layers), and one fully connected classification layer. For data partitioning, MNIST and CIFAR-10 use a non-IID partitioning method with label distribution offset to construct client-side local data, simulating the scenario of inconsistent client-side data distribution in federated learning. FEMNIST retains the original writer / user partitioning method of the dataset, treating different user samples as different client-side local data, thereby constructing a naturally heterogeneous federated learning scenario. To ensure the fairness of the experimental comparison, all methods use the same random seed, initial model parameters, data partitioning method, model structure, and federated training settings. The complete dense model of LeNet has 44,426 parameters, and the complete dense model of ResNet18 has 11,173,962 parameters.

[0150] The experiment was conducted on Ubuntu 20.04, using Python 3.8 and PyTorch 1.13.1. The hardware environment consisted of an NVIDIA GeForce RTX 4060 GPU. In federated training, the total number of clients was set to 10, the client participation ratio per round was set to 1.0, the number of local training rounds was set to 2, the batch size was set to 64, and the learning rate was set to 0.01 on MNIST and 0.001 on FEMNIST and CIFAR-10. The momentum coefficient for CIFAR-10 was set to 0.9, the weight decay coefficient was set to 5×10−4, and the number of global communication rounds was set to 100, 120, and 150 on MNIST, FEMNIST, and CIFAR-10, respectively. The communication volume was calculated based on the number of valid model update parameters uploaded by the clients, and the model updates were represented using 32-bit floating-point.

[0151] The dual-agent strategy in DAPP-FL is trained using DDPG. Both the pruning agent and the privacy agent employ an Actor-Critic architecture. Both the Actor and Critic networks consist of two fully connected layers with hidden layer dimensions of 128 and 64, respectively. The learning rate for the Actor network is set to 1×10⁻⁴, and for the Critic network, 1×10⁻³. The discount factor is set to 0.99, the soft update coefficient for the target network is set to 0.005, the experience replay pool capacity is set to 10000, the mini-batch sampling size is set to 64, and the weight coefficients in the reward function are... , , , The values ​​were set to 1.0, 0.3, 0.3 and 0.2 respectively. Exploration noise was added during the training process to enhance the continuous action search ability, and the exploration intensity was gradually reduced as the number of training rounds increased.

[0152] To verify the effectiveness of DAPP-FL, FedAvg, SCAFFOLD, DP-SCAFFOLD, FedMef, SparsyFed, and Fed-SMP were selected as comparison methods. All methods were run under the same data partitioning, model structure, and communication rounds to ensure the comparability of experimental results.

[0153] like Figures 2-10 As shown in Tables 1-3, the experimental results of DAPP-FL, FedAvg, SCAFFOLD, DP-SCAFFOLD, FedMef, SparsyFed, and Fed-SMP on the MNIST, FEMNIST, and CIFAR-10 datasets are presented. The experimental results show that, under the federated training scenario set in this invention, DAPP-FL can maintain high test accuracy and reduce cumulative communication volume on all three datasets, while exhibiting good convergence performance under the same training progress, thus verifying the effectiveness of the proposed collaborative control mechanism in multi-objective trade-offs.

[0154] Table 1. Comparison of final test accuracy of different methods on various datasets

[0155]

[0156] Table 2 Comparison of cumulative communication volume of different methods on various datasets

[0157]

[0158] Table 3. Comparison of test accuracy of different methods under uniform training schedule

[0159]

[0160] As shown in Table 1, DAPP-FL achieves test accuracies of 98.68%, 90.24%, and 81.53% on the MNIST, FEMNIST, and CIFAR-10 datasets, respectively, all outperforming the comparative methods. This result demonstrates that even with differential privacy perturbations, the proposed method maintains strong model discriminative ability and good convergence quality. The main reason for this is that DAPP-FL does not directly apply noise to the original high-dimensional dense update. Instead, it first compresses redundant parameters through a pruning strategy, allowing subsequent privacy noise to primarily affect the more contributing effective parameter subspace. This mitigates the adverse impact of differential privacy on model expressiveness. Compared to methods that directly apply noise to the dense update space, this strategy effectively reduces noise diffusion on irrelevant parameters, thus preserving more effective gradient information under privacy constraints. Especially in the FEMNIST and CIFAR-10 scenarios, DAPP-FL maintains high test accuracy, indicating that the proposed method still possesses good adaptability and generalization performance even with highly heterogeneous data or high task complexity.

[0161] Regarding communication overhead, Table 2 shows that the cumulative communication volume of DAPP-FL on the three datasets is 102.730MB, 192.779MB, and 1838.290MB, respectively, which is 67.91%, 66.54%, and 72.58% lower than FedAvg. This result indicates that the proposed method can significantly reduce the overall transmission cost in federated training. This is because DAPP-FL uses a pruning strategy to structurally compress model updates, reducing the size of effective parameters that need to be transmitted in each round. At the same time, the dual agents dynamically adjust the pruning intensity based on the shared training state, so that the compression ratio is no longer a fixed parameter, but can adapt to the training stage and model state. Therefore, the communication advantage of DAPP-FL does not come from static sparsity, but from dynamic compression control oriented towards the training process. Although SparsyFed and FedMef have lower cumulative communication volume in some scenarios, their test accuracy and test loss are not as good as DAPP-FL, indicating that the proposed method does not simply pursue the lowest communication cost, but achieves communication compression while ensuring high model performance.

[0162] Regarding convergence efficiency, Table 3 presents a comparison of test accuracy under a uniform training schedule. MNIST, FEMNIST, and CIFAR-10 are observed at rounds 80, 96, and 120, respectively. It can be seen that DAPP-FL maintains the highest accuracy on all three datasets, indicating that it not only excels in final performance but also enters the effective convergence interval faster under the same training schedule. Figures 2 to 10The accuracy and loss curves also show that DAPP-FL exhibits a more stable convergence trend and a lower loss level in the later stages of training. Compared with DP-SCAFFOLD, DAPP-FL reduces the persistent performance suppression caused by fixed noise perturbation by dynamically adjusting the privacy strength and pruning strength. Compared with methods that only emphasize sparse compression, DAPP-FL avoids excessive weakening of the model's expressive ability by excessive pruning. Therefore, pruning and privacy control are not independently optimized in the method described in this invention, but are synergistically adjusted through a unified strategy.

[0163] In summary, DAPP-FL demonstrates good overall performance in terms of test accuracy, cumulative communication volume, and test accuracy under the same training progress. This indicates that the proposed method can reduce communication overhead while maintaining model availability and mitigate the impact of differential privacy perturbations on training stability through the coordinated adjustment of pruning control and privacy control.

[0164] This invention presents a dual-strategy collaborative optimization method for privacy-preserving federated learning. It employs a collaborative model of "centralized server decision-making and local client execution," constructing model compression and privacy protection strategies on the server side. Structured pruning control and differential privacy parameter adjustment are integrated into a unified optimization process. Through shared training states and a joint feedback mechanism, the method adaptively adjusts the layer-by-layer channel retention rate, gradient pruning threshold, and Gaussian noise intensity, ensuring dynamic matching between model compression and privacy protection strengths throughout the training process. Theoretical analysis shows that under norm pruning and Gaussian noise mechanisms, client-side updates achieve differential privacy protection; and under bounded pruning errors and noise perturbations, the global training process exhibits a stable convergence trend. Experimental results demonstrate that the proposed method effectively reduces cumulative communication traffic on multiple datasets and maintains good model accuracy and training stability under different privacy budgets, validating the effectiveness of the collaborative control of model compression and privacy protection strategies and improving applicability and robustness.

[0165] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and embodiments shown and described herein.

Claims

1. A dual-policy collaborative optimization method for privacy-preserving federated learning, characterized in that, Includes the following steps: Step 1: The central server builds a global model and distributes it to a subset of clients. The current global model is distributed to each client in the middle; Step 2: Obtain the pruning and privacy actions of the global model based on the deep deterministic policy gradient method. The client performs structured pruning, gradient pruning and Gaussian noise addition operations, and uploads the perturbation update to the central server. Step 3: The central server performs weighted aggregation on all perturbation updates uploaded by participating clients to obtain a new round of global model; Step 4: When the preset total number of communication rounds is reached, the algorithm terminates and returns to the final global model; Step 5: The final global model is distributed to each client. Each client uses the final global model to process its local dataset, obtains privacy-protected data processing results, and then transmits them to the central server to complete the data release under privacy protection.

2. The dual-policy collaborative optimization method for privacy-preserving federated learning according to claim 1, characterized in that, The global model includes an input layer, at least two pruning layers, and an output layer, wherein the at least two pruning layers include convolutional layers and / or fully connected layers.

3. The dual-policy collaborative optimization method for privacy-preserving federated learning according to claim 2, characterized in that, Step two specifically includes: Step 1, construct the first The client in the first Shared state of the wheel: ； in, For the first Evaluation accuracy of global model data For the first Evaluation loss of the global model data. For the first Heterogeneous characteristics of individual clients For the first Training feedback information for each client in historical rounds; Step 2: Update the pruning agent based on the deep deterministic policy gradient method to obtain the pruning action; Step 3: The client completes structured pruning and local training based on the pruning actions to obtain the updated local model; Step 4: Update the privacy agent based on the deep deterministic policy gradient method to obtain privacy actions; Step 5: The client performs model cropping based on privacy actions to obtain the cropped local model; Step 6: Inject Gaussian noise into the effective update after pruning to obtain the final perturbation update for uploading.

4. The dual-policy collaborative optimization method for privacy-preserving federated learning according to claim 3, characterized in that, The first The heterogeneous characteristics of each client are used to describe the differences between different clients in terms of data scale, computing power and communication conditions; The data scale is obtained by counting the number of local samples on the client, the computing power is estimated by the local training time on the client, the number of samples processed per unit time or the computing power of the device, and the communication conditions can be estimated by the model upload time, historical communication latency or available bandwidth.

5. The dual-policy collaborative optimization method for privacy-preserving federated learning according to claim 4, characterized in that, The first The training feedback information of each client in historical rounds includes the first... Pruning actions of each client in previous rounds Privacy actions Communication costs Privacy expenses Compressed revenue and performance feedback after model update , This indicates the historical iteration identifier in the accumulation operation.

6. The dual-policy collaborative optimization method for privacy-preserving federated learning according to claim 5, characterized in that, The method for updating the deep deterministic policy gradient of the pruning agent in step 2 specifically includes: The state space is the first The client in the first Shared state of wheels ; The action space is a continuous vector. ; The reward function is the first The client in the first Joint rewards of the round ; State transition to ; The Actor network takes the shared state as input and outputs the first... The client in the first The pruning action of the wheel The mapping relationship is expressed as follows: ; Critic network with shared state and the first The client in the first The pruning action of the wheel is the input, and the output is a state-action value function. The Critic network is updated by minimizing the temporal difference error, and the Actor network is updated by policy gradient ascent. in, The policy network representing the pruning agent. This represents the network parameters for the pruning strategy. For the first The client in the first Round targeting the first The retention rate of the prunable layer , .

7. The dual-policy collaborative optimization method for privacy-preserving federated learning according to claim 6, characterized in that, The first The client in the first Joint rewards of the round satisfy: ； In the formula, Indicates the first Global test accuracy increment. Represents the normalized communication cost. Indicates normalized privacy overhead. This indicates normalized compressed returns. , , , All coefficients greater than 0 are trade-off coefficients.

8. The dual-policy collaborative optimization method for privacy-preserving federated learning according to claim 7, characterized in that, The structured pruning specifically includes: Using the output channels of convolutional layers or the output neurons of fully connected layers as pruning units, the client calculates the importance score of each pruning unit based on the channel parameter norm or neuron parameter norm, and sorts them from high to low importance scores. The retention rate generated by the pruned agent is then considered. Determine the number of channels or neurons that need to be retained in this layer. The system retains the channels or neurons with the highest importance scores, while masking the rest. This generates a structured mask that preserves the labeling information corresponding to the output channels of the convolutional layer or the output neurons of the fully connected layer. The client applies the structured mask to the global model distributed by the central server to obtain the compressed model for the current round. After the client completes local training on the compressed model, it obtains the locally updated model.

9. The dual-policy collaborative optimization method for privacy-preserving federated learning according to claim 8, characterized in that, The privacy action is as follows: ； In the formula, For privacy-preserving intelligent agents in the first The first round Gradient clipping threshold generated by each client. For privacy-preserving intelligent agents in the first The first round Gaussian noise intensity coefficients generated by each client.

10. The dual-policy collaborative optimization method for privacy-preserving federated learning according to claim 9, characterized in that, Steps 2 and 4 also include: Transition sample ( The samples are stored in the experience replay pool B, and the policy network and value network of the pruned agent and privacy agent are updated using the small batch of samples in the experience replay pool.

Images