Differential privacy federated learning method based on noise scale distribution and related equipment

Abstract

The invention belongs to the technical field of model training, and provides a differential privacy federated learning method based on noise scale distribution and related equipment. According to the embodiment of the invention, the method comprises the steps: obtaining a total noise scale sequence of a target client under a total training step number, and distributing noise scale subsequences for a plurality of local training steps of the target client in a current round of global iteration based on the total noise scale sequence; the total noise scale sequence is an incremental sequence determined by minimizing the convergence upper bound of a global model corresponding to the server under the constraint of the total privacy budget of the target client; and executing the current round of global iteration through the target client based on the progressively decreased learning rate sequence and a small-batch stochastic gradient descent algorithm, and performing gradient noise adding processing according to the noise scale subsequence. And finally, updating the current global model based on the aggregated accumulated noise-adding gradient. The method and device can be suitable for differential privacy federated learning of small-batch local training, and the model performance is improved while privacy protection is achieved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of federated learning, and particularly relates to a differential privacy federated learning method based on noise scale allocation and related equipment. BACKGROUND

[0002] At present, federated learning has attracted extensive attention because it can train a global model while keeping the privacy data of clients locally. During training, the client only needs to share the model gradient with the central parameter server (PS), which avoids the direct leakage of training data. However, the model gradient exposed during transmission can still be stolen by attackers and used to perform privacy attacks to illegally obtain the privacy data of the client.

[0003] In order to resist these privacy attacks, differential privacy federated learning (DPFL) is proposed, which uses zero-mean random noise generated by the differential privacy (DP) mechanism to disturb the model gradient. Although the noisy gradient has been proven to effectively reduce the success rate of the above privacy attacks, the noisy gradient also brings the challenge of performance degradation of the global model, which seriously hinders the practical application of differential privacy federated learning.

[0004] Gaussian mechanism (GM) is a commonly used differential privacy mechanism, and the noise size is determined by the noise scale. The larger the noise scale, the greater the probability of random noise deviating from the mean value of zero, the better the privacy protection effect, but the worse the usability of the model gradient. Under the given privacy budget and total training steps, the noise scale of each training step is usually determined by two aspects of audit method and noise allocation strategy. Among them, the audit method determines the form of privacy loss variable, and the noise allocation strategy further refines the noise scale of the training process on the basis of the audit method.

[0005] In related technologies, an adaptive noise scale allocation strategy based on zero-concentrated differential privacy (zCDP) audit method is proposed, that is, a smaller noise scale is allocated to the later training stage to ensure the stable convergence of the model. However, this adaptive noise scale allocation strategy is designed for centralized machine learning and is not applicable to the federated learning scenario of small batch local training (i.e. the sampling probability q is less than 1).

[0006] Therefore, how to provide a differential privacy federated learning suitable for small batch local training to balance the relationship between privacy protection and model performance has become a key problem to be solved. SUMMARY

[0007] Therefore, the embodiments of the present application provide a differential privacy federated learning method based on noise scale allocation and related equipment, which can be applied to differential privacy federated learning with small batch local training, and can improve model performance while protecting privacy.

[0008] The first aspect of the embodiments of the present application provides a differential privacy federated learning method based on noise scale allocation, comprising: determining a learning rate sequence of each local training step under a total training step number, and sending the learning rate sequence to all clients; the total training step number corresponds to multiple rounds of global iterations, and the learning rate sequence is a decreasing sequence; for each round of global iteration, determining a target client participating in the round of global iteration from the all clients, and sending a current global model to the target client; obtaining a total noise scale sequence of the target client under the total training step number, and based on the total noise scale sequence, allocating a noise scale sub-sequence for multiple local training steps of the target client in the round of global iteration, and sending the noise scale sub-sequence to the target client; the total noise scale sequence is an increasing sequence determined by minimizing the convergence upper bound of the global model corresponding to the server under the constraint of the total privacy budget of the target client; by the target client, based on the learning rate sequence and the small batch stochastic gradient descent algorithm, performing multiple local training steps in the round of global iteration, and respectively adding noise to the model gradient obtained by each local training step according to the noise scale sub-sequence to obtain a cumulative noise gradient; obtaining the cumulative noise gradient uploaded by the target client, and updating the current global model based on the aggregated cumulative noise gradient.

[0009] In some embodiments, the obtaining the total noise scale sequence of the target client under the total training step number comprises: determining an upper bound of noise variance of the target client using a Gaussian mechanism at a local training step; determining the degree of sample distribution of the target client according to the difference between the local model gradient and the global model gradient of the target client; based on the upper bound of noise variance and the degree of sample distribution, deriving a model convergence upper bound under the total training step number; using a Laplace differential privacy mechanism, quantifying a first cumulative privacy budget consumed by the noise scale of each target client under the total training step number; determine a target constraint condition of a total noise scale sequence of the target client based on the first cumulative privacy budget and a total privacy budget of the target client respectively; minimize a convergence upper bound of a global model corresponding to the server under the target constraint condition to calculate the total noise scale sequence of the target client at the total training step number.

[0010] In some embodiments, the determination of the target constraint condition of the total noise scale sequence of the target client based on the first cumulative privacy budget and the total privacy budget of the target client respectively includes: obtain a conversion relationship between the RAPPOR differential privacy mechanism and the differential privacy mechanism, and convert the first cumulative privacy budget into a second cumulative privacy budget under the differential privacy mechanism based on the conversion relationship; determine a target constraint condition of a total noise scale sequence of the target client based on the second cumulative privacy budget and a total privacy budget of the target client respectively.

[0011] In some embodiments, the determination of the target constraint condition of the total noise scale sequence of the target client based on the second cumulative privacy budget being equal to the total privacy budget of the target client respectively includes: set the second cumulative privacy budget and the total privacy budget of the target client being equal as a theoretical constraint condition of the total noise scale sequence of the target client; obtain a target optimal order parameter value determined based on a noise scale average allocation algorithm; replace an optimal order parameter in the theoretical constraint condition based on the target optimal order parameter value to obtain the target constraint condition of the total noise scale sequence of the target client.

[0012] In some embodiments, the minimization of the convergence upper bound of the global model corresponding to the server under the target constraint condition to calculate the total noise scale sequence of the target client at the total training step number includes: determine a total noise upper bound of the target client based on the convergence upper bound of the global model corresponding to the server; the total noise upper bound is directly proportional to a learning rate and a noise scale; convert the minimization of the convergence upper bound of the global model corresponding to the server into the minimization of the total noise upper bound of the target client; minimize the total noise upper bound under the target constraint condition to calculate the total noise scale sequence of the target client at the total training step number.

[0013] In some embodiments, the minimizing the total noise upper bound under the target constraint condition, the total noise scale sequence of the target client under the total training step is calculated, comprising: Neglecting the different noise scales allocated by the plurality of local training steps in each round of global iteration, the total noise upper bound is converted into a first noise upper bound of the noise scale for each round of global iteration, and the target constraint condition is converted into a first constraint condition for each round of global iteration; Minimizing the first noise upper bound under the first constraint condition, the optimal Riel privacy budget of the target client in each round of global iteration is calculated; The total noise upper bound is converted into a second noise upper bound of the noise scale for the plurality of local training steps in each round of global iteration; Minimizing the second noise upper bound under the constraint of the optimal Riel privacy budget, the total noise scale sequence of the target client under the total training step is obtained.

[0014] The second aspect of the embodiments of the application provides a differential privacy federated learning method based on noise scale allocation, comprising: Obtaining a learning rate sequence of each local training step under a total training step and a current global model, the total training step corresponding to a plurality of global iterations, and the learning rate sequence being a decreasing sequence; For each round of global iteration, a noise scale sub-sequence allocated in a plurality of local training steps in the round of global iteration is obtained; the noise scale sub-sequence is allocated based on a total noise scale sequence under the total training step; the total noise scale sequence is determined by minimizing the convergence upper bound of the global model corresponding to the server under the constraint of the total privacy budget of the target client; Based on the learning rate sequence and the mini-batch stochastic gradient descent algorithm, the plurality of local training steps in the round of global iteration are executed, and the model gradient obtained by each local training step is respectively added noise based on the noise scale sub-sequence to obtain a cumulative added noise gradient; The cumulative added noise gradient is uploaded to the server, and the current global model is updated by the server based on the aggregated cumulative added noise gradient.

[0015] The third aspect of the embodiments of the application provides a differential privacy federated learning device based on noise scale allocation, comprising: A determination module is configured to determine a learning rate sequence of each local training step under a total training step, and send the learning rate sequence to a global client; the total training step corresponds to a plurality of global iterations, and the learning rate sequence is a decreasing sequence; The screening module is configured to determine, for each round of global iteration, target clients participating in the current round of global iteration from the plurality of clients, and send the current global model to the target clients; The noise allocation module is configured to obtain a total noise scale sequence of the target clients under the total training steps, and allocate a noise scale sub-sequence for each of the plurality of local training steps in the current round of global iteration based on the total noise scale sequence, and send the noise scale sub-sequence to the target clients; the total noise scale sequence is an increasing sequence determined by minimizing the convergence upper bound of the global model corresponding to the server under the constraint of the total privacy budget of the target clients; The first training module is configured to perform the plurality of local training steps in the current round of global iteration by the target clients based on the learning rate sequence and the mini-batch stochastic gradient descent algorithm, and respectively add noise to the model gradient obtained by each local training step according to the noise scale sub-sequence to obtain the cumulative noise gradient. The model updating module is configured to obtain the cumulative noise gradient uploaded by the target clients, and update the current global model based on the aggregated cumulative noise gradient.

[0016] A fourth aspect of the embodiments of the present application provides a differential privacy federated learning device based on noise scale allocation, the device comprising: The first obtaining module is configured to obtain a learning rate sequence of each local training step under a total training step and a current global model, the total training step corresponding to a plurality of rounds of global iteration, and the learning rate sequence being a decreasing sequence; The second obtaining module is configured to, for each round of global iteration, obtain a noise scale sub-sequence allocated for the plurality of local training steps in the current round of global iteration; the noise scale sub-sequence is allocated based on a total noise scale sequence under the total training steps; the total noise scale sequence is an increasing sequence determined by minimizing the convergence upper bound of the global model corresponding to the server under the constraint of the total privacy budget of the target clients; The second training module is configured to perform the plurality of local training steps in the current round of global iteration based on the learning rate sequence and the mini-batch stochastic gradient descent algorithm, and respectively add noise to the model gradient obtained by each local training step according to the noise scale sub-sequence to obtain the cumulative noise gradient; The transmission module is configured to upload the cumulative noise gradient to the server, and update the current global model based on the aggregated cumulative noise gradient by the server.

[0017] The fifth aspect of the embodiments of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method for federated learning based on differential privacy with noise scale allocation provided in the first aspect when executing the computer program.

[0018] The sixth aspect of the embodiments of the present application provides a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the steps of the method for federated learning based on differential privacy with noise scale allocation provided in the first aspect.

[0019] The seventh aspect of the embodiments of the present application provides a computer program product, which, when executed on a processor, causes the processor to implement the steps of the method for federated learning based on differential privacy with noise scale allocation provided in the first aspect.

[0020] The method for federated learning based on differential privacy with noise scale allocation provided in the first aspect of the embodiments of the present application determines the learning rate sequence of each local training step under the total training step number, and sends the learning rate sequence to the full quantity of clients; the total training step number corresponds to multiple global iterations, and the learning rate sequence is a decreasing sequence; for each global iteration, target clients participating in the global iteration are determined from the full quantity of clients, and the current global model is sent to the target clients; the total noise scale sequence of the target clients under the total training step number is obtained, and noise scale subsequences are allocated to multiple local training steps of the target clients in the global iteration based on the total noise scale sequence, and the noise scale subsequences are sent to the target clients; the total noise scale sequence is an increasing sequence determined by minimizing the convergence upper bound of the global model corresponding to the server under the constraint of the total privacy budget of the target clients; the target clients perform multiple local training steps in the global iteration based on the learning rate sequence and the mini-batch stochastic gradient descent algorithm, and respectively add noise to the model gradient obtained by each local training step according to the noise scale subsequence to obtain the cumulative noise gradient; the cumulative noise gradient uploaded by the target clients is obtained, and the current global model is updated based on the aggregated cumulative noise gradient.

[0021] Therefore, the application obtains a total noise scale sequence of the target client under the total training steps in advance, and allocates a noise scale sub-sequence for the target client in the plurality of local training steps in the current global iteration based on the total noise scale sequence. Since the total noise scale sequence is an increasing sequence determined by minimizing the convergence upper bound of the global model corresponding to the server under the constraint of the total privacy budget of the target client, that is, by using the optimal noise scale increasing allocation strategy, smaller noise scales are allocated in the early training stage to enable the model to quickly establish an accurate strong correlation between the data features and the labels, thereby reducing the influence of noise on the final global model and improving the model accuracy. Larger noise scales are allocated in the later training stage to ensure that the privacy loss of the target client meets the privacy budget requirement of each client. In addition, since the total noise scale sequence is an increasing sequence, the noise scales of the plurality of local training steps in each global iteration are different in the small-batch local training process of each global iteration, which realizes more fine-grained noise scale allocation and is suitable for differential privacy federated learning with small-batch local training. While protecting privacy, the model performance is improved.

[0022] It can be understood that the beneficial effects of the above-mentioned second aspect to the seventh aspect can be referred to the related description in the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0024] Figure 1 is a comparison diagram of the sensitivity of noise scale at different training stages provided by the embodiments of the present application; Figure 2 is a flowchart of a differential privacy federated learning method based on noise scale allocation provided by the embodiments of the present application; Figure 3 is a flowchart of a method for determining a total noise scale sequence of a target client under total training steps provided by the embodiments of the present application; Figure 4 is a flowchart of another differential privacy federated learning method based on noise scale allocation provided by the embodiments of the present application; Figure 5 is a structural block diagram of a differential privacy federated learning device based on noise scale allocation provided by the embodiments of the present application; Figure 6is a structural block diagram of another device for federated learning with differential privacy based on noise scale allocation provided by an embodiment of the present application. Figure 7 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0025] In the following description, for the purposes of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc., in order to provide a thorough understanding of the embodiments of the application. However, it will be apparent to those skilled in the art that the embodiments of the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the embodiments of the application with unnecessary detail.

[0026] It is to be understood that the terminology “includes”, “comprises”, “has”, “contains”, “involves”, and / or “comprising” when used in this disclosure specifies the presence of stated features, integers, steps, operations, elements, and / or components but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0027] It is also to be understood that the terminology “and / or” when used in this disclosure, refers to at least one of the items listed or a combination of any two or more of the items listed, and includes all possible combinations.

[0028] As used in this application, the term “if’ can be construed to mean “when” or “once” or “in response to determining” or “in response to detecting”, depending on the context. Similarly, the phrase “if it is determined” or “if [a described condition or event] is detected” can be construed to mean “once it is determined” or “in response to determining” or “once [the described condition or event] is detected” or “in response to detecting [a described condition or event]”, depending on the context.

[0029] In addition, in the description of the application, the terms “first”, “second”, “third”, etc. are used only to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0030] Reference in the specification to “one embodiment” or “some embodiments” etc., means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrases “in one embodiment”, “in some embodiments”, “in other embodiments”, “in additional embodiments”, etc., in various places in the specification are not necessarily all referring to the same embodiment, although they can. The terms “comprise”, “comprises”, “comprising”, “include”, “includes”, “including” and the like are synonymous with the term “contain”, “contains”, “containing” or “has”, “has”, “having” and are used in this application in the same way as the term “contain”, “contains”, “containing” or “has”, “has”, “having”.

[0031] First, some of the terms involved in the embodiments of the present application are explained and described, so as to facilitate understanding by those skilled in the art.

[0032] Federated Learning (FL): A machine learning framework that uses data for learning and training while meeting privacy and data security requirements. As a distributed machine learning paradigm, federated learning can effectively solve the data silo problem, allowing participants to jointly model without sharing data, and can technically break down data silos and enable collaborative training.

[0033] Differential Privacy (DP): By introducing controllable random noise in data queries or model training, it ensures that the query results on adjacent data sets are indistinguishable from the query results on the original data set, ensuring that attackers cannot infer the existence or non-existence of individual data from the output results.

[0034] Differentially Private Federated Learning (DPFL): A method that applies differential privacy technology to the federated learning framework to protect client data privacy.

[0035] Renyi Differential Privacy (RDP): A more rigorous and refined generalization of differential privacy, which uses Renyi divergence to measure the difference between two probability distributions (i.e., the output distribution of the algorithm on adjacent data sets).

[0036] Gaussian Mechanism (GM): One of the commonly used techniques for implementing differential privacy noise perturbation. Its core operation is to add noise sampled from a Gaussian (normal) distribution to data queries or model training.

[0037] Federated Averaging Algorithm (FedAvg): The core model training algorithm for federated learning. By training a shared global model collaboratively with multiple clients holding local data, without the need to centralize the original data to the server.

[0038] Stochastic Gradient Descent (SGD): An iterative algorithm for optimizing machine learning models, which uses the gradient of only one sample (or a small batch of samples) at a time to update model parameters, thereby accelerating convergence and reducing computational cost.

[0039] Currently, federated learning has attracted extensive attention due to its ability to train a global model while keeping the privacy data of clients locally. During training, the client only needs to share the model gradient with the central parameter server (PS), which avoids the direct leakage of training data. However, the model gradient exposed during transmission can still be stolen by attackers and used to perform privacy attacks to illegally obtain the privacy data of the client. Attackers can use the uniqueness of the gradient to iteratively restore the original training sample that generates the gradient, or use the gradient to train a shadow model to simulate the target model and infer whether a given sample is a training sample member of the target model.

[0040] To resist these privacy attacks, differentially private federated learning (DPFL) is proposed, which uses zero-mean random noise generated by a differentially private (DP) mechanism to disturb the model gradient. Although the noisy gradient has been proven to effectively reduce the success rate of the above privacy attacks, the noisy gradient also brings the challenge of performance degradation of the global model, which seriously hinders the practical application of differentially private federated learning.

[0041] The Gaussian mechanism (GM) is a commonly used differential privacy mechanism, and the noise size is determined by the noise scale (i.e. the standard deviation of the Gaussian distribution). The larger the noise scale, the greater the probability that the random noise deviates from the mean value of zero (i.e. the larger the noise), the better the privacy protection effect, but the worse the usability of the model gradient. Under the given privacy budget and total number of training steps, the noise scale of each training step is usually determined by two aspects: the audit method and the noise allocation strategy. Among them, the audit method determines the form of the privacy loss variable, and the noise allocation strategy further refines the noise scale of the training process on the basis of the audit method.

[0042] In related technologies, an adaptive noise scale allocation strategy based on the zero-concentrated differential privacy zCDP audit method is proposed, that is, a smaller noise scale is allocated to the later training stage to ensure the stable convergence of the model. However, this adaptive noise scale allocation strategy is designed for centralized machine learning and is not applicable to the federated learning scenario of small batch local training (i.e. the sampling probability q is less than 1). Specifically, first, the performance of these allocation strategies is very sensitive to hyperparameters (such as the initial noise scale and the noise scale decay rate), so grid search is needed to find the optimal hyperparameters to achieve the best performance, which is not allowed in federated learning because each search will leak privacy. Second, the audit method zCDP used by these allocation strategies is not suitable for the small batch training scenario (i.e. the sampling probability q is less than 1) commonly used in federated learning, because the zCDP audit method is designed for the centralized machine learning scenario (i.e. the sampling probability q is equal to 1). However, the above methods cannot achieve the optimal privacy auditing. Most importantly, these allocation strategies are heuristic and cannot provide theoretical convergence guarantees or optimality.

[0043] Therefore, how to provide a differential privacy federated learning suitable for small batch local training to balance the relationship between privacy protection and model performance has become a key problem to be solved.

[0044] First, introduce the general differential privacy federated learning system (DPFL system), which includes a parameter server (Parameter Server, PS) and a client, denoted as . Among them, the PS is semi-honest and curious, that is, it satisfies the following properties: (1) the PS will honestly sample the client and aggregate the model gradient collected from the selected client; (2) the PS will not modify or delete any information used for model aggregation; (3) the PS tries to infer the sensitive information of the corresponding client from the received model gradient. Therefore, the DP mechanism can be used to resist both the PS and malicious external attackers. Each client has an independent data set of size , denoted as . All clients cooperate to train a global model, and the training goal is to minimize the global loss function , denoted as . Among them, is the model training parameter with dimension d, is the local loss function of client , that is, the average of the loss functions of all data samples, denoted as .

[0045] Federated averaging algorithm (FedAvg) is one of the most popular training algorithms in federated learning, and most of the current DPFL frameworks are based on the improvement of federated averaging algorithm (FedAvg). The existing DPFL framework generates DP random noise that satisfies a specific Gaussian distribution to disturb the gradient after the local training calculation of the FedAvg algorithm. However, the original DP auditing method cannot accurately track the privacy loss in the training process, and the noise scale average allocation strategy cannot consider the difference in model performance improvement in different training stages of the training process, which leads to poor performance of the existing DPFL and makes it difficult to be applied in practice.

[0046] Currently, there are a large number of research works devoted to improving the auditing method or the noise scale allocation strategy to reduce the impact of the noise scale, so as to improve the performance of the DPFL. On the one hand, since the Renyi divergence can measure the difference between any two distributions, it is used to define the privacy loss variable, that is, the difference between the output distributions on a pair of adjacent data sets. Some researchers relax the order of the Renyi divergence from infinity (DP definition) to a real number greater than 1, so as to define the upper bound of the Renyi divergence of all orders as the zCDP loss variable. Compared with the original DP auditing method, the zCDP can more accurately track the combined privacy loss of multiple (such as ) differential privacy mechanisms, so as to reduce the noise scale from ) to ), wherein is the total number of training steps.

[0047] The inventors found that although the above-mentioned zCDP auditing method greatly reduces the order of magnitude of the noise scale, they ignore the inherent dynamics of the training process, so as to allocate the same noise scale for each training step. In fact, the relevant researchers observed that the model gradient gradually becomes smaller as the training process progresses, and in order to ensure the stable convergence of the model, they proposed a noise decreasing allocation strategy based on zCDP. Specifically, some researchers proposed to reduce the noise scale based on the local loss, and they believed that if the local loss of the noisy local model obtained after one local training step is higher than the local loss of the model before training, it means that the noise scale is too large, so they use a factor less than 1 to reduce the noise scale until the local loss after training is lower than the local loss before training.

[0048] However, the above-mentioned adaptive noise scale allocation strategies are designed for centralized machine learning, and face various limitations in federated learning. First of all, their algorithm performance is sensitive to the setting of hyperparameters. Too large initial noise scale or too fast noise scale decay rate can cause the model to learn nothing in the early stage of training. Therefore, these algorithms usually need to grid search the optimal hyperparameter setting to achieve the highest model accuracy, which is not allowed in federated learning, because the search process requires the client and the server to interact with the gradient, which will consume the privacy budget. Secondly, these algorithms all use the zCDP auditing method to track the combined privacy loss of multiple training steps. However, small batch local training is a common setting in federated learning, and in this scenario, zCDP cannot reduce the noise scale according to the batch sampling rate. Most importantly, these algorithms are essentially heuristic algorithms, because their design is not based on theoretical analysis, so they cannot guarantee the reasonableness of the noise allocation and the optimality of the model performance.

[0049] We demonstrate why the existing noise scale decreasing allocation strategy is against the theory of key learning stages in training process through an experiment. Specifically, we conduct DPFL training on MNIST dataset and divide the training process into four stages (i.e. , , and ) averagely. We set the noise scale of one stage to be twice of the other three stages in turn, and record the accuracy of the global model aggregated from each round of global iteration on the test set, as shown in Figure 1 . As shown in Figure 1 , when the noise scale of the first stage is twice of the other three stages (black line), the model performs much slower in the first stage. Moreover, from the performance of the final global model, the performance loss caused by the first stage cannot be compensated in the later small noise scale training. Therefore, these results show that in the early stage of training, the model quickly establishes a strong correlation between the features of the training data and the labels, and this relationship will last throughout the training process. If the noise is too large in the early stage of training, the model cannot establish an accurate correlation. In this case, even if small noise is used in the later training, the model cannot correct this inaccurate correlation. Therefore, the early stage of training is more sensitive to noise scale, and smaller noise scale is beneficial to the model to quickly establish an accurate relationship, thereby improving the performance of the model.

[0050] Inspired by the sensitivity of different training stages to noise scale, in this application, we propose an optimal noise scale increasing allocation strategy, i.e., allocating smaller noise scale (i.e., smaller noise) to the early stage of training to promote the model to quickly establish an accurate correlation in the early stage of training, thereby reducing the impact of noise on the final global model while ensuring privacy, i.e., improving the accuracy of the model.

[0051] As shown in Figure 2 , the differential privacy federated learning method based on noise scale allocation provided by an embodiment of the present application includes the following steps S201 to S205: Step S201, determining a learning rate sequence for each local training step under a total training step number, and sending the learning rate sequence to all clients; the total training step number corresponds to multiple rounds of global iteration, and the learning rate sequence is a decreasing sequence.

[0052] Specifically, the server determines a learning rate sequence for each local training step under a total training step number , and sends the learning rate sequence to all clients. All clients refer to all clients participating in differential privacy federated learning. In addition, the server also initializes a global model , and initializes a candidate set of available clients as and initialize the index of local training steps Each client owns a private independent dataset of size with a privacy budget set according to its own privacy requirement . The total number of training steps corresponds to multiple global iterations, and each global iteration includes multiple local training steps. Each client uses its own independent dataset as the training dataset to train the global model. The data type of the training dataset is related to the downstream task to which the global model is applied. For example, if the downstream task includes an image recognition task, the training dataset can include but is not limited to at least one of image sample data, video sample data, etc. If the downstream task includes a video-text understanding task, the training dataset can include but is not limited to multi-modal data such as video sample data, text sample data, etc.

[0053] Suppose the total number of training steps is T, and each client performs local training steps in each global iteration. Among them, is the number of training periods, is the mini-batch sampling rate, so a training period contains local training steps to traverse the entire dataset. Accordingly, the learning rate sequence is set. The learning rate sequence is a decreasing sequence, that is, as the number of local training steps increases, the learning rate gradually decreases.

[0054] Step S202, for each global iteration, determine the target client participating in the current global iteration from the total number of clients, and send the current global model to the target client.

[0055] Specifically, before each global iteration starts (i.e. ), the server randomly selects K clients from the available client candidate set as target clients to participate in the current global iteration, denoted as . Then, the server can distribute the current global model to the selected target clients, and record the current training step index .

[0056] Step S203, obtaining a total noise scale sequence of the target client at the total training step number, and assigning a noise scale sub-sequence to each local training step in the current round of global iteration for the target client based on the total noise scale sequence, and sending the noise scale sub-sequence to the target client; the total noise scale sequence is an increasing sequence determined by minimizing the convergence upper bound of the global model corresponding to the server under the constraint of the total privacy budget of the target client.

[0057] Specifically, the server obtains a total noise scale sequence of the target client at the total training step number , and then determines a noise scale sub-sequence corresponding to the current round of global iteration from the total noise scale sequence according to the step arrangement order of the local training steps in the current round of global iteration among the total training step number, that is, assigns a noise scale sub-sequence to each local training step in the current round of global iteration for the target client. Then, the server sends the noise scale sub-sequence to the target client, so that the target client performs gradient noise disturbance based on each noise scale in the noise scale sub-sequence, and guarantees the data privacy of the target client.

[0058] In the embodiment, the total noise scale sequence is an increasing sequence, that is, the noise scale of each local training step in the total noise scale sequence increases with the increase of the local training step. That is, a smaller noise scale is used in the early training period, and a larger noise scale is used in the later training period. The total noise scale sequence is determined by minimizing the convergence upper bound of the global model corresponding to the server under the constraint of the total privacy budget of the target client.

[0059] In order to determine the optimal noise scale increasing allocation strategy, an improved DPFL framework is proposed, which is different from the traditional DPFL framework in which the same noise scale is set for each local training step of the client in each iteration. In the improved DPFL framework, the noise scale of each training step of the client is a variable.

[0060] In addition, considering that the aforementioned zCDP auditing method cannot reduce the noise scale according to the batch sampling rate, and cannot be applied to differential privacy federated learning with small batch local training, the RDP auditing method is used instead of the original DP auditing method and the zCDP auditing method to track the cumulative privacy consumption in the training process. Unlike the zCDP auditing method, in the RDP auditing method, if the upper bound of each order Roldi divergence is defined as an RDP loss variable, since the tail of the distribution of the RDP loss variable satisfies the sub-exponential distribution, the privacy loss can be further relaxed under the subsampling condition, that is, the noise scale is only ) to achieve a given privacy protection level, where is the batch sampling probability, This represents the total number of training steps.

[0061] To facilitate understanding, the relevant knowledge of the differential privacy mechanism involved in this application is introduced below.

[0062] Since the privacy loss variables for both RDP and the original DP can be defined by Rayleigh divergence, we will first introduce Rayleigh divergence here: Definition 1. (Rayli divergence). For any two distributions... and Given order The corresponding Ruili divergence is defined as: (1) According to this definition, Rayleigh divergence describes two distributions. and With the same value The expected probability difference of Rayleigh divergence can be used to represent the probability difference of differential privacy algorithms obtaining the same result on adjacent datasets (differences of at most one sample), i.e., the privacy loss variable. A smaller Rayleigh divergence indicates that the probability of obtaining the same result on adjacent datasets is closer, making it more difficult to distinguish which dataset the result comes from, i.e., a smaller privacy loss. Next, we will introduce DP and RDP from the perspective of Rayleigh divergence.

[0063] Assumption and It is a pair of adjacent datasets, that is and At most one sample is different, denoted as .make To represent the differential privacy algorithm, then and They represent exist and The output of the differential privacy algorithm is a distribution, therefore let... and Representing the algorithms respectively exist and The output is obtained from the above. The probability of DP. Based on this, DP is defined as follows: Definition 2. (( )-DP). When ,algorithm satisfy( )-DP.

[0064] in, For DP privacy loss variables, For DP privacy budget, The maximum probability that a DP loss variable exceeds the DP privacy budget. From this definition, we can see that The smaller or The smaller means the algorithm The closer the probability of getting the same output on neighboring datasets, the better the privacy protection effect.

[0065] In this paper, we use Gaussian mechanism (GM) to achieve differential privacy. GM achieves privacy protection by adding Gaussian random noise to the gradient. Since the noise adding process is irreversible, attackers cannot use the noisy gradient to restore the original gradient to conduct effective gradient inversion attacks or membership inference attacks. We describe each local training step as a d-dimensional gradient query on the local dataset, denoted as where denotes the model parameters before training, denotes the local dataset. GM generates Gaussian random noise to perturb , i.e., the output is where is a d-dimensional identity matrix, is the noise scale, is the L2-sensitivity of the gradient Since different audit methods have different definitions of loss variables, the noise scale required by them is also different.

[0066] Theorem 1. (Gaussian noise scale under DP privacy audit). When using DP audit methods and the noise scale satisfies and , the Gaussian mechanism satisfies ( )-DP.

[0067] Theorem 1 can guarantee that each local training step achieves ( )-DP. If local training steps are required, the cumulative privacy consumption should be calculated according to Theorem 2.

[0068] Theorem 2. (Composition theorem of DP). Let each local training step, satisfy ( )-DP, then satisfies ( )-DP.

[0069] Theorem 1 and Theorem 2 both assume that the entire dataset is used for training in each local training step. However, in FL, only a small batch of data is usually selected to calculate the stochastic gradient, so in this case the actual privacy consumption should be calculated according to Theorem 3.

[0070] Theorem 3. (Subsampling DP). Suppose a differentially private algorithm satisfies ( )-DP on the entire dataset, and denotes the sampling rate of mini-batch, then satisfies ( )-DP on the subsampled dataset, where and .

[0071] However, since DP requires the R'eli divergence to have a constant upper bound at the infinity order, which is an overly strict constraint, its composition theorem (Theorem 2) would amplify the privacy consumption during the training process. To track the cumulative privacy consumption more precisely during the training process, RDP only requires the upper bound of the R'eli divergence at each order to be represented by the corresponding order. Similar to DP, we introduce the definition of RDP and related theorems as follows.

[0072] Definition 2. ((ε, δ)-RDP). Given an order , a randomized algorithm satisfies -RDP if the corresponding R'eli divergence satisfies .

[0073] where is the RDP privacy loss variable, is the RDP privacy budget and is a function of . According to the R'eli divergence, there is the following conversion relationship between RDP and DP.

[0074] Lemma 1. (Conversion relationship between RDP and DP). A randomized algorithm that satisfies -RDP also satisfies -DP, where .

[0075] Through this conversion relationship, the RDP privacy budget can find the optimal conversion to the minimum DP privacy budget. Similarly, we introduce the Gaussian noise scale, composition theorem, and subsampling theorem of RDP in turn.

[0076] Theorem 4. (Gaussian noise scale under RDP privacy auditing). When using the RDP auditing method and the noise scale satisfies , the Gaussian mechanism satisfies -RDP.

[0077] Theorem 5. (Composition theorem of RDP). Let each local training step​ respectively satisfy -RDP, then satisfy -RDP.

[0078] Theorem 6. (Sub-sampling RDP). Assume a differentially private algorithm satisfies -RDP over the entire dataset, and denotes the sampling rate of mini-batches, then satisfies -RDP over the sub-sampled dataset, where .

[0079] Theorem 7. (Relationship between Gaussian noise scale and sub-sampling RDP). Assume a Gaussian noise scale , a mini-batch sampling rate , and a given order , the RDP privacy budget of this Gaussian mechanism is:

[0080] where and .

[0081] In this application, in order to explore the influence of the Gaussian noise scale allocation strategy under the RDP privacy audit on the performance of the DPFL model, we analyze the convergence of the above-mentioned DPFL framework. Referring to the related theoretical analysis work, we analyze the upper bound of the variance of the Gaussian noise to quantify the influence of the noise scale allocation strategy on the convergence rate, and then obtain the total noise scale sequence of the target client under the total training step.

[0082] In some embodiments, as shown in Figure 3 , the obtaining the total noise scale sequence of the target client under the total training step comprises: Step S301, determining the upper bound of the noise variance of the Gaussian mechanism used by the target client at the local training step.

[0083] Specifically, let denote the upper bound of the L2 norm of the original gradient of a single data sample, then the L2-sensitivity of the gradient is , so the Gaussian noise used by the target client at the i-th local training step has an upper bound of the noise variance of . .

[0084] ​Step S302, determining a sample distribution degree of the target client according to a difference between a local model gradient of the target client and a global model gradient.

[0085] Specifically, the sample distribution between clients in federated learning is generally Non-Identical and Independently Distributed (Non-IID). In this application, we define the difference between the local gradient and the global gradient to represent the Non-IID degree of the sample distribution of the client. Based on this, we further define the upper bound of its expectation as .

[0086] Step S303, deriving a model convergence upper bound at the total number of training steps based on the noise variance upper bound and the sample distribution degree.

[0087] Specifically, similar to related convergence analysis work, we also make some general assumptions about the trained model. We assume that the loss function of all clients satisfies L-smoothness and Lipschitz continuity, and the upper bound of the variance of the stochastic gradient and the true gradient of a single sample is represented as .

[0088] Based on the calculation of the above noise variance upper bound, the definition of the Non-IID degree of the sample distribution, and various model-related assumptions, and based on the set learning rate , where , we can derive the convergence result of the above improved DPFL framework after local training steps as follows:

[0089]

[0090] where, , . is a model sampled from all global model sequences with a probability . In addition, is the global loss value calculated on the initial global model, and is the minimum global loss value.

[0091] Step S304, using the RDP mechanism to quantify the first cumulative privacy budget consumed by the noise scale of each target client at the total number of training steps.

[0092] Specifically, since in the actual system, only a target client participates in training, which means that the probability of the client participating in each training step is s. Therefore, according to Theorem 7 above, the expectation of the first cumulative privacy budget consumed by each target client in the entire global training process can be expressed as:

[0093] wherein is the noise scale sequence that achieves the optimal Rényi order of the minimum DP privacy budget.

[0094] In step S305, a target constraint condition of the total noise scale sequence of the target client is determined based on the first cumulative privacy budget and the total privacy budget of each target client.

[0095] Specifically, the target constraint condition of the total noise scale sequence of the target client is determined according to the relationship between the first cumulative privacy budget and the total privacy budget of each target client. Under this target constraint condition, the sum of the cumulative privacy budgets consumed by the noise scales of all local training steps of the client can be equal to the total privacy budget.

[0096] In some embodiments, the determination of the target constraint condition of the total noise scale sequence of the target client based on the first cumulative privacy budget and the total privacy budget of each target client comprises: obtaining a conversion relationship between the Rényi differential privacy mechanism and the differential privacy mechanism, and converting the first cumulative privacy budget into a second cumulative privacy budget under the differential privacy mechanism based on the conversion relationship; determining the target constraint condition of the total noise scale sequence of the target client based on the second cumulative privacy budget and the total privacy budget of each target client.

[0097] Specifically, considering that the noise scale is also constrained by the total privacy budget of the client, the first cumulative privacy budget is converted into a second cumulative privacy budget under the differential privacy mechanism according to the conversion relationship between the Rényi differential privacy mechanism RDP and the differential privacy mechanism DP (i.e., Lemma 1 above), that is, the RDP privacy budget is converted into the minimum DP privacy budget under the given . Then, the target constraint condition of the total noise scale sequence of the target client is defined based on the total privacy budget of the target client under the differential privacy mechanism ( ) and the second cumulative privacy budget. Generally, the sum of the cumulative privacy budgets consumed by the noise scales of all local training steps of the client can be equal to the total privacy budget as the target constraint condition, that is, the determined noise scale sequence needs to satisfy the following constraint condition:

[0098] wherein, is the participation probability of the client at each training step, is the sampling rate.

[0099] Step S306, under the target constraint condition, minimizing the convergence upper bound of the global model corresponding to the server, calculating the total noise scale sequence of the target client under the total training step number.

[0100] Specifically, under the target constraint condition, by adjusting the noise scale of each local training step, the convergence upper bound of the global model corresponding to the server is minimized, and the total noise scale sequence of the target client under the total training step number is solved.

[0101] In some embodiments, the total noise scale sequence of the target client under the total training step number is calculated by minimizing the convergence upper bound of the global model corresponding to the server under the target constraint condition, comprising: determining the total noise upper bound of the target client based on the convergence upper bound of the global model corresponding to the server; the total noise upper bound is proportional to the learning rate and the noise scale; convert the minimization of the convergence upper bound of the global model corresponding to the server into the minimization of the total noise upper bound of the target client; minimizing the total noise upper bound under the target constraint condition, calculating the total noise scale sequence of the target client under the total training step number.

[0102] Specifically, the convergence result of the above equation (3) shows the influence of the noise scale on the convergence of the DPFL training model. Through the analysis of the convergence result, we can draw the following three conclusions: 1) The first term ( ) and the second term ( ) in the above convergence result are the convergence loss caused by the non-differential privacy federated learning training, which is consistent with the convergence upper bound obtained by the previous research work.

[0103] 2) The third term ( ) in the above convergence result is the sum of the noise upper bounds of all target clients. Since each client independently allocates the noise scale of each training step according to its total privacy budget, the noise upper bound of different clients can be optimized independently by the client, i.e. each client only needs to consider optimizing the total noise upper bound , so as to minimize the convergence upper bound.

[0104] 3) The total noise upper bound It is revealed that the existing noise scale diminishing distribution strategies cannot minimize the convergence error under diminishing learning rate, so their designs are not optimal.

[0105] Thus, combining Equations (3) and (5), each target client can establish the following optimization problem to adjust the noise scale of each local training step to achieve the minimum convergence error upper bound:

[0106]

[0107] where the optimization objective function is . Since the Hessian matrix of is positive definite, the is a convex function about .

[0108] However, the constraint condition of problem is not an affine function, so problem is a non-convex optimization problem, which can be solved by the existing IPOPT (a nonlinear optimization solver) optimization algorithm. According to the properties of the objective function and the constraint condition, it can be concluded that when the learning rate is diminishing, the optimal noise scale distribution strategy should be increasing, which is consistent with the previous motivation experiment conclusion that smaller noise scales, i.e., smaller noise, should be allocated in the early stage of training.

[0109] However, the time complexity of directly solving problem in practical application scenarios is very high, mainly for two reasons. On the one hand, the optimal RDP order depends on the noise scale sequence , which means is a complex function about . Therefore, it takes a long time to calculate the derivative of this constraint about during optimization. On the other hand, the time complexity of solving problem presents an exponential growth with the increase of the number of optimization variables . In addition, in the actual training process, complex models usually require thousands (or even more) of training steps to ensure that the model achieves good performance or reaches convergence. In order to improve the solving efficiency of problem , we propose the DPFL-STNS (DPFL with SmartTuning of Noise Scale) algorithm.

[0110] First, given the total privacy budget and the total number of training steps, DPFL-STNS finds an optimal RDP order that is close to and independent of the sequence of noise scales to simplify the problem under the constraint condition.

[0111] In some embodiments, the determining the target constraint condition of the total noise scale sequence of the target clients based on the second cumulative privacy budget being equal to the total privacy budget of each of the target clients comprises: setting the second cumulative privacy budget being equal to the total privacy budget of each of the target clients as a theoretical constraint condition of the total noise scale sequence of the target clients; obtaining a target optimal order parameter value determined based on a noise scale average allocation algorithm; replacing the optimal order parameter in the theoretical constraint condition with the target optimal order parameter value to obtain the target constraint condition of the total noise scale sequence of the target clients.

[0112] Specifically, the second cumulative privacy budget is set to be equal to the total privacy budget of each of the target clients as a theoretical constraint condition of the total noise scale sequence of the target clients, i.e., the above equation (5) is taken as the theoretical constraint condition. Then, a target optimal order parameter value determined based on a noise scale average allocation algorithm is obtained, and the target optimal order parameter value is replaced with the optimal order parameter in the theoretical constraint condition as the target constraint condition of the total noise scale sequence of the target clients.

[0113] In practical applications, existing research results show that when the noise scale fluctuates within a small range, the corresponding optimal RDP order changes very little. Referring to the existing noise scale average allocation algorithm based on RDP (i.e., DP-SGD), the problem is solved to obtain an optimal noise scale allocation strategy that adjusts within a small range of average noise scale. Therefore, the DPFL-STNS can use the optimal RDP order parameter value of the DP-SGD algorithm to approximate the problem in the above equation (5). Experimental results show that using the optimal noise scale allocation strategy obtained by using the approximate value for training, the actual optimal RDP order of more than 95% of the clients is equal to the optimal RDP order of the DP-SGD, which proves the effectiveness of the approximation process.

[0114] In some embodiments, the calculating the total noise scale sequence of the target clients under the total number of training steps under the target constraint condition comprises: ​ignoring the different noise scales assigned to the multiple local training steps in each round of global iteration, to convert the total noise upper bound to a first noise upper bound for the noise scale of each round of global iteration, and to convert the target constraint condition to a first constraint condition for each round of global iteration; minimizing the first noise upper bound under the first constraint condition, to calculate an optimal RDP privacy budget of the target client in each round of global iteration; converting the total noise upper bound to a second noise upper bound for the noise scale of the multiple local training steps in each round of global iteration; minimizing the second noise upper bound under the constraint of the optimal RDP privacy budget, to obtain a total noise scale sequence of the target client under the total number of training steps.

[0115] Specifically, DPFL-STNS proposes a two-stage method to reduce the number of optimization variables, so that the problem can be solved efficiently in complex training tasks (large number of training steps).

[0116] In the first stage, it is assumed that the noise scales assigned to the training steps in each round of global iteration are the same, that is, satisfying . That is, ignoring the different noise scales assigned to the multiple local training steps in each round of global iteration, to convert the total noise upper bound to a first noise upper bound for the noise scale of each round of global iteration, and to convert the target constraint condition to a first constraint condition for each round of global iteration, define the problem as minimizing the first noise upper bound under the first constraint condition. Then the problem can be converted to:

[0117]

[0118] By solving the problem , the optimal RDP privacy budget of each round of global iteration can be obtained, where is the optimal solution of the problem .

[0119] On this basis, in the second stage, the total noise upper bound is converted to a second noise upper bound for the noise scale of the multiple local training steps in each round of global iteration. Define the problem as minimizing the second noise upper bound under the constraint of the optimal RDP privacy budget. Under the constraint of the optimal RDP privacy budget , the optimal RDP privacy budget noise scales for each training step, i.e., solving the following problem :

[0120]

[0121] By the above two-stage method, solving the problem with optimization variables can be decomposed into solving the problem with optimization variables and solving the problem with optimization variables times , which greatly reduces the time complexity required in the solving process. The optimal noise scale allocation strategy calculated based on this method can be used to generate the corresponding random noise for a given local training step when the target client is selected to participate in training, and to update the current privacy consumption of the client in real time according to Lemma 1 and Theorem 7.

[0122] Step S204, performing a plurality of local training steps in the current global iteration by the target client based on the learning rate sequence and the mini-batch stochastic gradient descent algorithm, and respectively adding noise to the model gradient obtained by each local training step according to the noise scale sub-sequence to obtain a cumulative noisy gradient.

[0123] Specifically, for each global iteration, the selected target client updates the local model after receiving the global model, i.e., . Then, the target client performs local training steps locally. In each training step , the target client randomly samples data samples, denoted as , and performs a mini-batch stochastic gradient descent algorithm to obtain the total gradient of the batch . Then, the target client n generates Gaussian noise to disturb and average the model gradient, i.e., . Based on this gradient, the local model is updated, i.e., . After completing local training steps, the cumulative noisy gradient is uploaded to the server. At the same time, the target client calculates its consumed RDP privacy budget as according to equation (2), and then finds the optimal order * using the above Lemma 1 to convert the RDP privacy budget into a given the minimum DP privacy budget under the total training steps If , it means that the privacy budget of the target client has been used up, the server will remove the client from the candidate set , that is, the target client no longer participates in global training.

[0124] Step S205, obtaining the cumulative noisy gradient uploaded by the target client, and updating the current global model based on the aggregated cumulative noisy gradient.

[0125] Specifically, the server obtains the cumulative noisy gradient uploaded by each target client, and aggregates the cumulative noisy gradient collected from the selected target client in an average manner to obtain the aggregated cumulative noisy gradient. Then, the current global model is updated based on the aggregated cumulative noisy gradient, that is, . Then, the server will loop back to step S204 to perform the next round of global iteration until the total training steps or other model training conditions are reached.

[0126] In the embodiments of the present application, the total noise scale sequence of the target client under the total training steps is obtained in advance, and the noise scale sub-sequence is allocated to the target client for multiple local training steps in the current global iteration based on the total noise scale sequence. Since the total noise scale sequence is an increasing sequence determined by minimizing the convergence upper bound of the corresponding global model of the server under the constraint of the total privacy budget of the target client, that is, by using the optimal noise scale increasing allocation strategy, smaller noise scale is allocated in the early training period to make the model quickly establish an accurate strong correlation between data features and labels, thereby reducing the influence of noise on the final global model and improving the model accuracy. Larger noise scale is allocated in the later training period to ensure that the privacy loss of the target client meets the privacy budget requirements of each client. In addition, since the total noise scale sequence is an increasing sequence, the noise scales of the multiple local training steps in each global iteration are different in the small-batch local training process of each global iteration, which realizes more fine-grained noise scale allocation and is suitable for differential privacy federated learning with small-batch local training. While protecting privacy, the model performance is improved.

[0127] In addition, to determine the optimal noise scale increasing allocation strategy, we propose a general DPFL framework, in which the noise scale of each local training step is a variable. Moreover, the clients use mini-batch SGD for local training, and the RDP auditing method is used to track the privacy consumption during training. On this basis, we analyze the convergence upper bound of the DPFL framework under non-convex loss function, which reveals the influence of noise scale on model performance. By taking the noise scale of each training step as an optimization variable, we establish an optimization problem, which aims to optimize the noise scale of each training step to minimize the convergence upper bound under the constraint of total privacy budget. In addition, by proposing the DPFL-STNS algorithm, we efficiently solve the optimization problem to obtain the optimal noise scale allocation strategy. Since our allocation strategy can minimize the theoretically derived convergence error, our algorithm is the optimal noise scale allocation strategy. Specifically, DPFL-STNS greatly reduces the time complexity of solving the original optimization problem by simplifying the constraint condition and using a two-stage solving method. Finally, we conduct a large number of experiments on the commonly used public datasets MNIST, FEMNIST and CIFAR-10, and the results show that: compared with the latest noise scale allocation strategy, the noise increasing allocation strategy of the DPFL-STNS algorithm proposed in this application achieves faster convergence speed and higher final model accuracy, for example, it can improve the model accuracy by an average of 2.37%, and DPFL-STNS is more robust and stable in performance under different training system settings.

[0128] As Figure 4 shown, another noise scale allocation based differential privacy federated learning method provided by the embodiments of the application includes the following steps S401 to S404: Step S401, obtaining a learning rate sequence of each local training step under a total training step number and a current global model, the total training step number corresponding to multiple global iterations, and the learning rate sequence being a decreasing sequence.

[0129] Step S402, for each global iteration, obtaining a noise scale sub-sequence allocated to multiple local training steps in the global iteration; the noise scale sub-sequence is allocated based on a total noise scale sequence under the total training step number; the total noise scale sequence is an increasing sequence determined by minimizing the convergence upper bound of the global model corresponding to the server under the constraint of the total privacy budget of the target client.

[0130] Step S403, based on the learning rate sequence and the mini-batch stochastic gradient descent algorithm, performing a plurality of local training steps in the current global iteration, and respectively adding noise to the model gradient obtained by each local training step according to the noise scale sub-sequence to obtain a cumulative noisy gradient.

[0131] Step S404, uploading the cumulative noisy gradient to the server, and updating the current global model based on the aggregated cumulative noisy gradient through the server.

[0132] The embodiments of the present application obtain a noise scale sub-sequence for a plurality of local training steps in the current global iteration, which is obtained based on a total noise scale sequence under a total number of training steps. Since the total noise scale sequence is an increasing sequence determined by minimizing the convergence upper bound of the global model corresponding to the server under the constraint of the total privacy budget of the target client, i.e., by using an optimal noise scale increasing allocation strategy, smaller noise scales are allocated in the early training stage to enable the model to quickly establish an accurate strong correlation between data features and labels, thereby reducing the impact of noise on the final global model and improving the accuracy of the model. Larger noise scales are allocated in the later training stage to ensure that the privacy loss of the target client meets the privacy budget requirements of the respective clients. In addition, since the total noise scale sequence is an increasing sequence, the noise scales of the plurality of local training steps in each global iteration are different in the mini-batch local training process of each global iteration, which realizes more fine-grained noise scale allocation and is suitable for differential privacy federated learning with mini-batch local training. The privacy is protected while the model performance is improved.

[0133] It should be noted that the specific technical details and beneficial effects of steps S401 to S404 described above are described in detail in the foregoing method embodiments, and will not be repeated here.

[0134] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0135] Corresponding to the differential privacy federated learning method based on noise scale allocation described in the foregoing embodiments, Figure 5 Fig. 1 shows a structural block diagram of a differential privacy federated learning device based on noise scale allocation provided by an embodiment of the present application. For ease of illustration, only the parts related to the embodiments of the present application are shown. As shown in Fig. 1, Figure 5 The differential privacy federated learning device 500 based on noise scale allocation comprises: The determining module 501 is configured to determine a learning rate sequence of each local training step under a total training step number, and send the learning rate sequence to full clients; the total training step number corresponds to multiple rounds of global iterations, and the learning rate sequence is a decreasing sequence; The screening module 502 is configured to determine, for each round of global iteration, a target client participating in the current round of global iteration from the full clients, and send a current global model to the target client; The noise allocation module 503 is configured to obtain a total noise scale sequence of the target client under the total training step number, and allocate a noise scale sub-sequence for the target client in multiple local training steps in the current round of global iteration based on the total noise scale sequence, and send the noise scale sub-sequence to the target client; the total noise scale sequence is an increasing sequence determined by minimizing the convergence upper bound of the global model corresponding to the server under the constraint of the total privacy budget of the target client; The first training module 504 is configured to perform multiple local training steps in the current round of global iteration by the target client based on the learning rate sequence and a mini-batch stochastic gradient descent algorithm, and respectively add noise to the model gradient obtained by each local training step according to the noise scale sub-sequence to obtain a cumulative noise gradient. The model updating module 505 is configured to obtain the cumulative noise gradient uploaded by the target client, and update the current global model based on the aggregated cumulative noise gradient.

[0136] In some embodiments, the noise allocation module 503 is further configured to: determine an upper bound of noise variance of the target client using a Gaussian mechanism at a local training step; determine the degree of sample distribution of the target client according to the difference between the local model gradient and the global model gradient of the target client; deduce a model convergence upper bound under the total training step number based on the upper bound of noise variance and the degree of sample distribution; quantify a first cumulative privacy budget consumed by the noise scale of each of the target clients under the total training step number using a Laplace differential privacy mechanism; determine a target constraint condition of the total noise scale sequence of the target client based on the first cumulative privacy budget and the total privacy budget of each of the target clients; under the target constraint condition, minimize the convergence upper bound of the global model corresponding to the server to calculate the total noise scale sequence of the target client under the total training step number.

[0137] In some embodiments, the noise allocation module 503 is further configured to: obtain a conversion relationship between the RAPP mechanism and the differential privacy mechanism, and convert the first cumulative privacy budget into a second cumulative privacy budget under the differential privacy mechanism based on the conversion relationship; determine a target constraint condition of a total noise scale sequence of the target client based on the second cumulative privacy budget and a total privacy budget of each of the target clients.

[0138] In some embodiments, the noise allocation module 503 is further configured to: set the second cumulative privacy budget to be equal to the total privacy budget of each of the target clients as a theoretical constraint condition of the total noise scale sequence of the target client; obtain a target optimal order parameter value determined based on the noise scale average allocation algorithm; replace an optimal order parameter in the theoretical constraint condition based on the target optimal order parameter value to obtain the target constraint condition of the total noise scale sequence of the target client.

[0139] In some embodiments, the noise allocation module 503 is further configured to: determine a total noise upper bound of the target client based on a convergence upper bound of a global model corresponding to the server; the total noise upper bound is directly proportional to a learning rate and a noise scale; convert minimizing the convergence upper bound of the global model corresponding to the server into minimizing the total noise upper bound of the target client; minimize the total noise upper bound under the target constraint condition to calculate the total noise scale sequence of the target client at the total training step number.

[0140] In some embodiments, the noise allocation module 503 is further configured to: ignore that noise scales allocated by multiple local training steps in each round of global iteration are different, to convert the total noise upper bound into a first noise upper bound of a noise scale for each round of global iteration, and convert the target constraint condition into a first constraint condition for each round of global iteration; minimize the first noise upper bound under the first constraint condition to calculate an optimal RAPP privacy budget of the target client for each round of global iteration; convert the total noise upper bound into a second noise upper bound of a noise scale for multiple local training steps in each round of global iteration; minimize the second noise upper bound under the constraint of the optimal RAPP privacy budget to obtain the total noise scale sequence of the target client at the total training step number.

[0141] Figure 6A structural block diagram of another noise scale allocation based differential privacy federated learning device provided by the embodiments of the present application is shown. For ease of illustration, only parts related to the embodiments of the present application are shown. As shown in Figure 6 The noise scale allocation based differential privacy federated learning device 600 comprises: A first obtaining module 601 is configured to obtain a learning rate sequence of each local training step and a current global model under a total training step number, wherein the total training step number corresponds to multiple global iterations, and the learning rate sequence is a decreasing sequence. A second obtaining module 602 is configured to, for each global iteration, obtain a noise scale sub-sequence allocated for multiple local training steps in the global iteration; the noise scale sub-sequence is allocated based on a total noise scale sequence under the total training step number; the total noise scale sequence is an increasing sequence determined by minimizing a convergence upper bound of a global model corresponding to a server under a constraint of a total privacy budget of a target client. A second training module 603 is configured to perform the multiple local training steps in the global iteration based on the learning rate sequence and a mini-batch stochastic gradient descent algorithm, and to respectively add noise to model gradients obtained by each local training step according to the noise scale sub-sequence to obtain cumulative noise-added gradients. A transmission module 604 is configured to upload the cumulative noise-added gradients to the server, and to update the current global model based on aggregated cumulative noise-added gradients by the server.

[0142] In applications, each module in the noise scale allocation based differential privacy federated learning device described above can be a software program module, can be realized by different logical circuits integrated in a processor, or can be realized by multiple distributed processors.

[0143] As shown in Figure 7 The embodiments of the present application also provide an electronic device 700, which comprises at least one processor 701 (only one processor is shown in the figure), a memory 702, and a computer program 703 stored in the memory 702 and executable on the at least one processor 701, wherein the processor 701 implements the steps in any method embodiment described above when executing the computer program 703. Figure 7

[0144] In applications, the electronic device 700 can be a desktop computer, a notebook computer, a palm computer, a cloud server, and other computing devices. The electronic device can include, but is not limited to, a processor and a memory. Those skilled in the art can understand that Figure 7 ​The electronic device is merely an example and does not limit the electronic device, which can include more or fewer components than illustrated, or combine some components, or different components, for example, can also include a face capture device, an input / output device, a network access device, etc.

[0145] In applications, the processor can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0146] In applications, the memory can be an internal storage unit in some embodiments, for example, a hard disk or a memory. The memory can also be an external storage device of the bionic robot in other embodiments, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the bionic robot. Further, the memory can include both the internal storage unit and the external storage device. The memory is used to store an operating system, an application program, a boot loader, data and other programs, for example, program codes of computer programs, etc. The memory can also be used to temporarily store data that has been output or will be output.

[0147] It should be noted that the information interaction, execution process, etc. between the above apparatus / module, since based on the same concept as the method embodiments of the present application, the specific functions and the technical effects brought by it can be referred to the method embodiments part, and will not be repeated here.

[0148] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional modules is taken as an example, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. Each functional module in the embodiment can be integrated in one processing module, or each module can be physically present separately, or two or more modules can be integrated in one module. The above integrated module can be realized in the form of hardware or in the form of software functional module. In addition, the specific name of each functional module is only for the convenience of mutual distinction, and does not limit the protection scope of the present application. The specific working process of the module in the above device can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0149] The embodiment of the present application further provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the steps in each method embodiment.

[0150] The embodiment of the present application provides a computer program product, when the computer program product is run on a processor, so that the processor is executed to realize the steps in each method embodiment.

[0151] If the integrated module is realized in the form of a software functional module and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the present application realizes all or part of the processes in the above embodiment methods, which can be completed by a computer program to instruct related hardware, and the computer program can be stored in a computer readable storage medium. The computer program is executed by the processor to realize the steps in each method embodiment. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium at least includes any entity or device capable of carrying the computer program code to the device / bionic robot, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, according to legislation and patent practice, the computer readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0152] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0153] Those skilled in the art can understand that the modules and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0154] In the embodiments provided in the present application, it should be understood that the disclosed apparatuses / devices and methods can be implemented in other ways. For example, the above-described apparatus / device embodiments are merely illustrative, for example, the division of modules is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of modules or components can be combined or integrated into another device, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed each other can be indirect coupling or communication connection through some interface, device or module, and can be electrical, mechanical or other forms.

[0155] The modules described as separate components can or can not be physically separated, and the components shown as modules can or can not be physical modules, i.e. can be located in one place, or can be distributed to multiple network modules. Part or all of the modules can be selected to achieve the purpose of the embodiments according to actual needs.

[0156] The above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for differentially private federated learning based on noise scale assignment, the method comprising: The method comprises: determining a learning rate sequence of each local training step under a total training step number, and sending the learning rate sequence to a full quantity client; the total training step number corresponds to multiple rounds of global iterations, and the learning rate sequence is a decreasing sequence; for each round of global iteration, determining a target client participating in the round of global iteration from the full quantity client, and sending a current global model to the target client; obtaining a total noise scale sequence of the target client under the total training step number, and based on the total noise scale sequence, assigning a noise scale sub-sequence to multiple local training steps of the target client in the round of global iteration, and sending the noise scale sub-sequence to the target client; the total noise scale sequence is an increasing sequence determined by minimizing the convergence upper bound of the global model corresponding to the server under the constraint of the total privacy budget of the target client; by the target client, based on the learning rate sequence and the mini-batch stochastic gradient descent algorithm, performing multiple local training steps in the round of global iteration, and respectively adding noise to the model gradient obtained by each local training step according to the noise scale sub-sequence to obtain a cumulative noise gradient; obtaining the cumulative noise gradient uploaded by the target client, and updating the current global model based on the aggregated cumulative noise gradient.

2. The noise scale based differential privacy federated learning method of claim 1, wherein, The method comprises: determining an upper bound of the noise variance of the Gaussian mechanism adopted by the target client at a local training step; determining the degree of sample distribution of the target client according to the difference between the local model gradient and the global model gradient of the target client; based on the noise variance upper bound and the sample distribution degree, deducing the model convergence upper bound under the total training step number; quantifying the first cumulative privacy budget consumed by the noise scale of each target client under the total training step number by using the Rényi differential privacy mechanism; based on the first cumulative privacy budget and the total privacy budget of each target client, determining the target constraint condition of the total noise scale sequence of the target client; under the target constraint condition, minimizing the convergence upper bound of the global model corresponding to the server to calculate the total noise scale sequence of the target client under the total training step number.

3. The noise scale based differential privacy federated learning method of claim 2, wherein, The method comprises: obtaining the conversion relationship between the Rényi differential privacy mechanism and the differential privacy mechanism, and based on the conversion relationship, converting the first cumulative privacy budget into a second cumulative privacy budget under the differential privacy mechanism; based on the second cumulative privacy budget and the total privacy budget of each target client, determining the target constraint condition of the total noise scale sequence of the target client.

4. The noise scale based differential privacy federated learning method of claim 3, wherein, The method comprises: The second cumulative privacy budget is equal to the total privacy budget of the target client, and is set as a theoretical constraint condition of the total noise scale sequence of the target client; Obtain the target optimal order parameter value determined based on the noise scale average allocation algorithm; Replace the optimal order parameter in the theoretical constraint condition with the target optimal order parameter value to obtain a target constraint condition of the total noise scale sequence of the target client.

5. The noise scale based differential privacy federated learning method according to any one of claims 2-4, characterized in that, The method comprises: Based on the convergence upper bound of the global model corresponding to the server, determine the total noise upper bound of the target client; the total noise upper bound is proportional to the learning rate and the noise scale; Convert the minimization of the convergence upper bound of the global model corresponding to the server into the minimization of the total noise upper bound of the target client; Minimize the total noise upper bound under the target constraint condition to calculate the total noise scale sequence of the target client under the total training step number.

6. The noise scale based differential privacy federated learning method of claim 5, wherein, The method comprises: Ignore the different noise scales allocated by the multiple local training steps in each global iteration to convert the total noise upper bound into a first noise upper bound of the noise scale for each global iteration, and convert the target constraint condition into a first constraint condition for each global iteration; Minimize the first noise upper bound under the first constraint condition to calculate the optimal Riel privacy budget of the target client in each global iteration; Convert the total noise upper bound into a second noise upper bound of the noise scale for the multiple local training steps in each global iteration; Minimize the second noise upper bound under the constraint of the optimal Riel privacy budget to obtain the total noise scale sequence of the target client under the total training step number.

7. A method for differentially private federated learning based on noise scale allocation, comprising: The method comprises: Obtain the learning rate sequence of each local training step and the current global model under the total training step number; the total training step number corresponds to multiple global iterations, and the learning rate sequence is a decreasing sequence; For each global iteration, obtain a noise scale subsequence allocated by the multiple local training steps in the current global iteration; the noise scale subsequence is obtained based on the total noise scale sequence under the total training step number; the total noise scale sequence is an increasing sequence determined by minimizing the convergence upper bound of the global model corresponding to the server under the constraint of the total privacy budget of the target client; Based on the learning rate sequence and the mini-batch stochastic gradient descent algorithm, execute the multiple local training steps in the current global iteration, and respectively add noise to the model gradient obtained by each local training step according to the noise scale subsequence to obtain a cumulative noise gradient; Upload the cumulative noise gradient to the server, and update the current global model based on the aggregated cumulative noise gradient by the server.

8. A differential privacy federated learning device based on noise scale allocation, characterized in that, The device comprises: The determining module is configured to determine a learning rate sequence of each local training step under a total training step number, and send the learning rate sequence to the full-volume clients; the total training step number corresponds to multiple rounds of global iterations, and the learning rate sequence is a decreasing sequence; The screening module is configured to determine, for each round of global iteration, a target client participating in the current round of global iteration from the full-volume clients, and send a current global model to the target client; The noise allocation module is configured to obtain a total noise scale sequence of the target client under the total training step number, and allocate a noise scale sub-sequence for the target client in multiple local training steps in the current round of global iteration based on the total noise scale sequence, and send the noise scale sub-sequence to the target client; the total noise scale sequence is an increasing sequence determined by minimizing the convergence upper bound of the global model corresponding to the server under the constraint of the total privacy budget of the target client; The first training module is configured to perform multiple local training steps in the current round of global iteration by the target client based on the learning rate sequence and a mini-batch stochastic gradient descent algorithm, and respectively add noise to the model gradient obtained by each local training step according to the noise scale sub-sequence to obtain a cumulative noisy gradient. The model updating module is configured to obtain the cumulative noisy gradient uploaded by the target client, and update the current global model based on the aggregated cumulative noisy gradient.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the steps of the noise scale allocation based differential privacy federated learning method in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to realize the steps of the noise scale allocation based differential privacy federated learning method in any one of claims 1 to 7.

Images