Differentially private episodic adaptive mean estimation optimization method

Abstract

The application discloses a differential privacy phased adaptive moment estimation optimization method, relates to the optimization technical field of privacy protection machine learning, and comprises the following steps: acquiring the input of a moment estimation optimization model; wherein the input of the moment estimation optimization model comprises a data set, a step length hyperparameter, an iteration number, a differential privacy noise parameter and an initialized model parameter value; optimizing the model parameter in stages; in different stages, the gradient of the model parameter in each iteration process is calculated based on different learning rates, and random noise based on the differential privacy noise parameter is added to the gradient; the random noise added to the gradient is used for protecting the privacy in the data set; the model parameter in each iteration process is updated based on the gradient of the model parameter in each iteration process; and the final model parameter of the moment estimation optimization model is output. The application can guarantee the availability of the model under the privacy and non-privacy settings under the non-convex loss.

Description

Technical Field

[0001] This invention belongs to the field of optimization technology in privacy-preserving machine learning, specifically relating to a differential privacy-preserving staged adaptive moment estimation optimization method. Background Technology

[0002] With the advent of the big data era, the privacy leakage of sensitive information contained in a large number of training data samples has become an increasingly concerning issue. In order to provide strict privacy protection, Differential Privacy (DP) has become a standard technique for SNCO privacy protection and ERM training, hereinafter referred to as DP-SNCO and DP-ERM.

[0003] The working principle of dynamic programming (DP) is to inject additional perturbations during training to hide the probability of the presence or absence of any individual data sample. For example, one of the most popular DP algorithms in deep learning is Differential Privacy Stochastic Gradient Descent (DP-SGD). This algorithm protects the privacy of the data by adding calibrated Gaussian noise to the stochastic gradient in each iteration. Compared with non-privacy models, the perturbations in DP inevitably reduce the usability of privacy-preserving models.

[0004] Therefore, there is an urgent need to propose an optimization algorithm to effectively improve the usability of SNCO and ERM under differential privacy. Summary of the Invention

[0005] To address the aforementioned problems in the existing technology, this invention provides a differential privacy-focused staged adaptive moment estimation optimization method. The technical problem to be solved by this invention is achieved through the following technical solution:

[0006] Firstly, this application provides a differential privacy-focused staged adaptive moment estimation optimization method, including:

[0007] Obtain the input to the moment estimation optimization model; where the input to the moment estimation optimization model includes the dataset, step size hyperparameter, number of iterations, differential privacy noise parameter, and initial model parameter values;

[0008] The model parameters are optimized in stages. At different stages, based on different learning rates, the gradient of the model parameters is calculated in each iteration, and random noise based on differential privacy noise parameters is added to the gradient. Random noise is added to the gradient to protect the privacy of the dataset.

[0009] Update the model parameters in each iteration based on the gradients of the model parameters obtained in each iteration.

[0010] The output moments estimate the final model parameters of the optimized model.

[0011] Optionally, the process of updating the model parameters in each iteration includes:

[0012] Uniformly and randomly select small batch indexes in, B is the batch size, and n is the total number of data items in the dataset.

[0013] Set the noise for differential privacy as in, For Gaussian noise, 0 represents the mean of the Gaussian noise, and π represents the mean of the Gaussian noise. 2 I d Let I be the variance. d It is a d-dimensional vector of all 1s;

[0014] Obtaining the noisy gradient in, Let t be the number of rounds of small batch data, B be the size of the small batch data, and z be the number of rounds of small batch data. i for One of the datasets, For parameter w t Regarding z i The gradient;

[0015] Obtain the optimization parameters w in the t-th round during the K-th iteration. t Based on the optimization parameters w in the t-th round t Obtain the optimization parameters w for the (t+1)th round. t+1 ,in, Where γ is a small constant used to stabilize the step size, and η is the step size;

[0016] Obtain the optimization parameters ω1,…,ω in round T during the K-th iteration. T One parameter is randomly selected as the optimization parameter for the Kth iteration process.

[0017] Optionally, privacy guarantees can be achieved by adding random noise based on differential privacy noise parameters to the gradient, satisfying the following conditions:

[0018] Set constants c1 and c2, where c1 > 0 and c2 > 0;

[0019] Set sampling ratio Total number of iterations For any For any δ, the following holds:

[0020]

[0021] Where B is the number of mini-batch samples, n is the number of samples, and L is the Lipschitz constant;

[0022] When any ∈ and any δ satisfy the condition, the moment estimation optimization model satisfies (∈,δ)-differential privacy.

[0023] The beneficial effects of this invention are:

[0024] This invention provides a differential privacy-based staged adaptive moment estimation optimization method. To reduce the utility difference of the algorithm under privacy and non-privacy settings, on the one hand, it proposes a privacy-preserving staged learning rate scheduling method in stochastic gradient descent, introducing differential privacy noise parameters into the input of the moment estimation optimization model to protect the privacy of the dataset. On the other hand, in a certain iteration of the moment estimation optimization model, considering that the original gradient descent method uses the same learning rate for all parameters, affecting the usability of the model, a staged adaptive learning rate strategy is proposed. The model has different learning rates in different iteration stages, ensuring the usability of the model under non-convex loss and privacy settings.

[0025] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0026] Figure 1 This is a flowchart of a differential privacy-focused staged adaptive moment estimation optimization method provided in an embodiment of the present invention;

[0027] Figure 2 This is a schematic diagram of the experimental results provided in an embodiment of the present invention;

[0028] Figure 3 This is another schematic diagram of the experimental results provided in the embodiments of the present invention;

[0029] Figure 4 This is another schematic diagram of the experimental results provided in the embodiments of the present invention;

[0030] Figure 5 This is another schematic diagram of the experimental results provided in the embodiments of the present invention. Detailed Implementation

[0031] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0032] In existing applications, for convex loss functions, studies have shown that the results of stochastic convex optimization under differential privacy (DP-SCO) can be close to those of ordinary stochastic convex optimization. This indicates that under most parameter conditions, DP-SCO may be able to obtain the same optimization results as the non-privacy version.

[0033] Despite the widespread and successful application of DP-ERM and DP-SNCO in various applications, theoretical understanding of the usability of non-convex loss functions is scarce, and analysis of their generalization risk is even rarer. Existing research has used different usability measures, such as gradient norm, but gradient norm cannot effectively measure model usability. Furthermore, existing research on generalization risk and empirical risk under non-convex loss functions has the following problems.

[0034] Existing studies on excess empirical risk in nonconvex losses analyze the Langevin dynamics (GLD) of DP gradients and, for general nonconvex and smooth losses, obtain... Furthermore, this is meaningful under T→+∞ and log(n)≥d; moreover, through a detailed analysis of the same DP-GLD algorithm, the additional empirical risk under the iteration number T→+∞ of τ∈(0,1) is increased to [value missing]. Given that the loss satisfies the additional assumption of Polyak-Lojasiewicz (PL), analysis of DP full gradient descent can yield the following results. Although the above studies provide for general nonconvex and smooth losses The algorithm for excess empirical risk matches the bound in general convex environments, but its exponential computational complexity is high. Therefore, it is not applicable in practice.

[0035] Existing research addresses the overgeneralization risk of nonconvex losses by analyzing DP-GLD and applying it to general nonconvex and smooth losses. For the two special nonconvex losses and robust regression of the generalized linear model, existing research has obtained the results by analyzing the DP Frank Wolfe algorithm. The generalization loss, if the constraint is an l1 norm ball, can be improved to...

[0036] In summary, existing research has some key limitations, leading to the following three gaps:

[0037] The first gap lies between the privacy-preserving algorithms analyzed above and those used in practical applications. The algorithms analyzed may not be suitable for current large-scale machine learning problems, such as deep neural networks (DNNs). In fact, these algorithms require full gradient calculations, resulting in high computational costs per iteration and poor scalability. In practical applications, optimization algorithms based on mini-batch random sampling are more popular for such large-scale problems. For example, DP-SGD is already provided by TensorFlow for training privacy-preserving deep neural networks. While DP-SGD-based algorithms are scalable and practical for convex loss functions, their effectiveness in terms of practicality and scalability for non-convex loss functions remains unclear.

[0038] The second gap lies in the difference between stochastic gradient descent (SGD) algorithms under privacy and non-privacy settings. For non-privacy SGD, many popular algorithm designs can further improve its performance. For example, non-privacy SGD typically utilizes phased learning rate adjustments (also known as geometric step decay) to enhance optimization, i.e., the learning rate is reduced at regular intervals. This phased step size scheduling is widely used in practice (e.g., in deep learning frameworks like TensorFlow and PyTorch). However, existing SGD-based algorithms, under both convex and non-convex loss functions, only analyze constant step size settings and do not consider dynamic learning rate scenarios. Furthermore, adaptive learning rate techniques are popular in non-privacy SGD, but there are no similar optimization algorithms under privacy settings. Moreover, while SGD has achieved considerable success in deep learning, some problems remain to be addressed. For instance, the dynamic adjustment of the learning rate is limited by pre-defined adjustment rules, and applying the same learning rate to various parameters and in the optimization of highly non-convex functions can easily lead to local saddle points. Among them, adaptive moment estimation (Adam) is widely used in DNN training as an adaptive learning rate algorithm. However, research on this optimization method with adaptive technology is still limited in the field of privacy optimization. It is currently unclear how the design of the phase step size and adaptive technology affects privacy-preserving algorithms based on stochastic gradients.

[0039] The third gap is the difference in usability between privacy-preserving and non-privacy-preserving algorithms. Under convex loss functions, research shows that, for most parameter conditions, the usability of algorithms under privacy settings can match that under non-privacy settings. However, for non-convex loss problems, there is a significant gap in usability between privacy-preserving and non-privacy settings. Algorithms under privacy settings sacrifice usability to provide privacy protection. Whether this gap can be narrowed as much as possible through algorithm design, at least for well-conditional non-convex problems, remains unknown, as it is possible to design privacy-preserving optimization algorithms that match the usability of algorithms under non-privacy settings.

[0040] In view of this, this application provides a differential privacy-based staged adaptive moment estimation optimization method. On the one hand, by introducing a noise parameter for differential privacy, the privacy of the dataset can be effectively protected. On the other hand, by introducing DP-Adam with a staged step size design, the gap between popular non-privacy algorithm designs in practice and privacy algorithms analyzed in theory is narrowed.

[0041] Please see Figure 1 , Figure 1 This is a flowchart of a differential privacy-focused staged adaptive moment estimation optimization method provided in this invention. The differential privacy-focused staged adaptive moment estimation optimization method provided in this application includes:

[0042] S101. Obtain the input of the moment estimation optimization model; wherein, the input of the moment estimation optimization model includes the dataset, step size hyperparameter, number of iterations, differential privacy noise parameter, and initial model parameter values;

[0043] S102. Optimize model parameters in stages. At different stages, based on different learning rates, calculate the gradient of the model parameters in each iteration process, and add random noise based on the differential privacy noise parameters to the gradient; adding random noise to the gradient is used to protect the privacy of the dataset.

[0044] S103. Based on the gradient of the model parameters obtained in each iteration, update the model parameters in each iteration.

[0045] S104, Output moment estimation optimization model final model parameters.

[0046] Specifically, this embodiment provides a differential privacy-based staged adaptive moment estimation optimization method. To reduce the difference in utility between the algorithm under privacy and non-privacy settings, on the one hand, it proposes a privacy-preserving staged learning rate scheduling method in stochastic gradient descent, introducing differential privacy noise parameters into the input of the moment estimation optimization model to protect the privacy of the dataset. On the other hand, in a certain iteration of the moment estimation optimization model, considering that the original gradient descent method uses the same learning rate for all parameters, affecting the usability of the model, a staged adaptive learning rate strategy is proposed. The model has different learning rates in different iteration stages, ensuring the usability of the model under both privacy and non-privacy settings with non-convex loss.

[0047] In one optional embodiment of this application, the process of updating the model parameters in each iteration includes:

[0048] Uniformly and randomly select small batch indexes in, B is the batch size, and n is the total number of data items in the dataset.

[0049] Set the noise for differential privacy as Where, ξ t To obey Distributed Gaussian noise, For Gaussian noise, 0 represents the mean of the Gaussian noise, and π represents the mean of the Gaussian noise. 2 I d Let I be the variance. d It is a d-dimensional vector of all 1s, which can also be understood as each dimension having a default variance of 1; optionally, the differential privacy noise is randomly sampled from the noise distribution.

[0050] Obtaining the noisy gradient in, Let t be the number of rounds of small batch data, B be the size of the small batch data, and z be the number of rounds of small batch data. i for One of the datasets, For parameter w t Regarding z i The gradient; optionally, the gradient is obtained to optimize the parameters. Noise is added to the gradient to protect the privacy of the data, because clean gradient data contains implicit user data information, which attackers can use, for example, member inference attacks to infer user data.

[0051] Obtain the optimization parameters w in the t-th round during the K-th iteration. t Based on the optimization parameters w in the t-th round t Obtain the optimization parameters w for the (t+1)th round. t+1 ,in, Where γ is a small constant used to stabilize the step size, and η is the step size;

[0052] Obtain the optimization parameters ω1,…,ω in round T during the K-th iteration. T One parameter is randomly selected as the optimization parameter for the Kth iteration process.

[0053] It should be noted that to add noise to gradient data, the gradient representation must first be obtained, where... m t and v t Let them represent the first and second moment variables, respectively. λ is v t The upper bound of g is given by β1 and β2, which are the exponential decay rates of the moment estimate, β1∈[0,1], β2∈[0,1], generally β1=0.99, β2=0.999, g j Let be the gradient of the j-th round, j≤t≤T; the gradient of the parameters with respect to the dataset is obtained through the above formula, and the gradient of the parameters is obtained by differentiating the loss function with respect to the parameters. The function changes the value fastest in the gradient direction. Therefore, optimization needs to be performed along the negative direction of the gradient. So we need to find the gradient and update the parameters in this direction.

[0054] In an optional embodiment of this application, regarding privacy protection, the algorithm can provide a privacy guarantee by adding random noise based on the differential privacy noise parameters to the gradient, satisfying the following conditions:

[0055] Set constants c1 and c2, where c1 > 0 and c2 > 0;

[0056] Set sampling ratio Total number of iterations For any For any δ, the following holds:

[0057]

[0058] Where B is the number of mini-batch samples, n is the number of samples, and L is the Lipschitz constant, which can also be understood as the loss function satisfying the L-Lipschitz continuous condition.

[0059] When any ∈ and any δ satisfy the conditions, the moment estimation optimization model satisfies (∈,δ)-differential privacy. By setting privacy guarantee conditions, the degree of privacy protection of the algorithm can be measured.

[0060] It should be noted that the L-Lipschitz continuity condition applies to a function f on a subset of the real number set: If there exists a constant L such that |f(a)-f(b)|≤L|ab|, Then f is said to satisfy the Lipshitz condition, and the smallest constant L for f is called the Lipshitz constant of f.

[0061] In an optional embodiment of this application, Algorithm 1 is used to demonstrate the full picture of DP-Adam with a phased learning rate. It iterates K times in the outer layer, which can also be understood as dividing the optimization process into K stages, with each stage having an η-level learning rate. k Iterative optimization of T under learning rate k In a specific stage K, Algorithm 2 is used, which adds noise to the gradient according to certain rules to ensure that the algorithm is differentially private. Then, the noisy gradient is used to execute the moment estimation optimization algorithm. Here, the certain rules refer to the moment estimation optimization satisfying the condition of differential privacy.

[0062] Algorithm 1 and Algorithm 2 are respectively:

[0063]

[0064]

[0065] In one optional embodiment of this application, the application is verified by specific experiments.

[0066] The experiment was conducted on the MNIST dataset, using a convolutional neural network (2-layer CNN). The ELU activation function was used, and the assumptions made for the theoretical analysis roughly met the empirical requirements. This embodiment was implemented on an Intel(R) Core(TM) i5-9400F 2.9GHz CPU, an NVIDIA RTX2070 Super graphics card, and an Ubuntu 18.04 operating system, based on the Tensorflow 1.15 deep learning framework.

[0067] Please see the experimental results. Figure 2 , Figure 2 This is a schematic diagram of the experimental results provided in an embodiment of the present invention. Please refer to [link / reference]. Figure 3 , Figure 3 This is another schematic diagram of the experimental results provided in the embodiments of the present invention. Please refer to [link / reference]. Figure 4 , Figure 4 This is another schematic diagram of the experimental results provided in the embodiments of the present invention. Please refer to [link / reference]. Figure 5 , Figure 5 This is another schematic diagram of the experimental results provided in this embodiment of the invention. According to the specific implementation of this experiment, the algorithm proposed in this embodiment and all its comparison algorithms are plotted with the number of iterations as the horizontal axis and training accuracy and testing accuracy as the vertical axes, respectively. The experiment selects DP parameters as (∈=4,δ=10). -5 ), (∈=8,δ=10 -5Experiments were conducted separately.

[0068] This invention provides a differential privacy-preserving staged adaptive moment estimation optimization method, apparatus, and electronic device. To reduce the utility difference of the algorithm under privacy and non-privacy settings, on the one hand, it proposes a privacy-preserving staged learning rate scheduling method in stochastic gradient descent by introducing differential privacy noise parameters into the input of the moment estimation optimization model to protect the privacy of the dataset. On the other hand, in a certain iteration of the moment estimation optimization model, considering that the original gradient descent method uses the same learning rate for all parameters, which affects the usability of the model, a staged adaptive learning rate strategy is proposed. The model has different learning rates in different iteration stages, ensuring the usability of the model under non-convex loss and privacy settings.

[0069] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A method for differentially private phased adaptive matrix estimation optimization, comprising: include: Obtain the input of the moment estimation optimization model; wherein, the input of the moment estimation optimization model includes the dataset, step size hyperparameter, number of iterations, differential privacy noise parameter, and initial model parameter values; The model parameters are optimized in stages. At different stages and based on different learning rates, the gradient of the model parameters is calculated for each iteration, and random noise based on the differential privacy noise parameters is added to the gradient. This random noise is added to the gradient to protect the privacy of the dataset. , , and Let them represent the first and second moment variables, respectively. , , for The upper realm, and These are the exponential decay rates estimated by moments, , , For the first Wheel gradient, , The total number of rounds is given; the gradient of the parameters with respect to the dataset is obtained by taking the derivative of the loss function with respect to the parameters; Based on the gradients of the model parameters obtained in each iteration, the model parameters in each iteration are updated; the process of updating the model parameters in each iteration includes: Uniformly randomly selecting a mini-batch index wherein, , is the size of the mini-batch data, is the total number of data in the data set; Set the noise for differential privacy as ,in, It is Gaussian noise. The mean of Gaussian noise, For variance, for A 1-dimensional vector; Obtaining the noisy gradient , ,in, for Small batches of data, For the size of small batches of data, for One of the datasets, For parameters about The gradient; The output moments estimate the final model parameters of the optimized model.

2. The differentially private phase-adaptive matrix estimation optimization method according to claim 1, characterized in that, The process of updating the model parameters in each iteration also includes: Get the The first round of iteration Optimization parameters of the wheel Based on the first Round optimization parameters , obtain the Round optimization parameters ,in, ,in, This is a constant used to stabilize the step size. Step size; acquire the first optimization parameters in each iteration optimization parameters in each iteration , randomly select one as the first optimization parameters in each iteration 3. The differentially private phase-adaptive matrix estimation optimization method of claim 1, wherein, The privacy guarantee achieved by adding random noise based on the differential privacy noise parameters to the gradient satisfies the following condition: Set constant and , and , ; Setting the sampling ratio and the total number of iterations , for any , for any , satisfies: ； wherein is a Lipschitz constant; When any and any If the condition is met, the matrix estimation optimization model satisfies differential privacy.

Images