A model privacy protection method, device, equipment and storage medium
By combining momentum and differential privacy mechanisms in distributed learning to filter and encrypt gradients, the convergence insufficiency and privacy protection challenges of zero-order optimization methods in non-convex settings are addressed, achieving efficient privacy protection and model convergence.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- OCEAN UNIV OF CHINA
- Filing Date
- 2026-04-16
- Publication Date
- 2026-07-10
Smart Images
Figure CN122365576A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of privacy protection, and in particular to a model privacy protection method, a model privacy protection device, an electronic device, and a computer-readable storage medium. Background Technology
[0002] Zero-order methods are widely used in distributed learning and other black-box machine learning tasks. Existing zero-order optimization methods are mostly based on finite difference and stochastic search. Several zero-order algorithms have been proposed for both convex and non-convex optimization problems, but their theoretical analysis and convergence guarantees under non-convex settings remain insufficient. Meanwhile, while some distributed learning methods have made progress in reducing communication overhead, they often rely on complex perturbation mechanisms or strict assumptions about data distribution, limiting their widespread application in practice. Data privacy is also crucial in distributed learning scenarios. Existing techniques typically introduce differential privacy mechanisms into the gradient, such as adding Gaussian noise, to meet the privacy budget requirements of different clients. However, these methods rely on gradient information and are difficult to apply when the gradient cannot be directly obtained or the problem is a black-box optimization. Summary of the Invention
[0003] The purpose of this invention is to provide a model privacy protection method, apparatus, device, and storage medium, applicable to the field of [unclear - possibly related to a specific field]. This method combines momentum mechanisms with differential privacy mechanisms during the distributed training process of the model, thereby improving model convergence efficiency while achieving privacy protection.
[0004] To address the aforementioned technical problems, this invention provides a model privacy protection method, comprising: At the start of the current model training round, the global model parameters in the server are sent to all clients; Candidate model parameter sets are determined from the model parameter sets of each client based on the compressed sensing algorithm; The gradient of each candidate model parameter set is determined based on the zero-order optimization algorithm, and the gradient is encrypted based on the differential privacy algorithm to obtain the encrypted gradient. The client's local model is updated based on the cryptographic gradient, and the cryptographic gradient is uploaded to the server. The global model parameters in the server are then updated based on the momentum mechanism and the cryptographic gradient.
[0005] Optionally, a candidate model parameter set is determined from the model parameter set of each client based on a compressed sensing algorithm, including: The model parameter set is divided into multiple model parameter groups, and a parameter label is set for each model parameter in the model parameter group; Construct a random perturbation vector, and perform two function value queries on each of the model parameters based on the random perturbation vector to obtain the function value query results; Based on the function value query results, the target ratio is determined, and the model parameters in each model parameter group whose parameter labels are the same as the target ratio are added to the candidate model parameter set.
[0006] Optionally, determining the target ratio based on the function value query result includes: The difference between the query result of the first function value and the query result of the unperturbed function value is taken as the numerator; The sum of the second function value query result and the unperturbed function value query result is used as the denominator. The target ratio is determined based on the ratio of the numerator to the denominator.
[0007] Optionally, the gradient of each candidate model parameter set is determined based on a zero-order optimization algorithm, including: The parameter gradients of each model parameter in the candidate model parameter set are determined based on the zero-order optimization algorithm. The mean of the parameter gradients of each candidate model parameter set is determined as the gradient of each candidate model parameter set.
[0008] Optionally, the gradient is encrypted using a differential privacy algorithm to obtain an encrypted gradient, including: Gradient clipping is performed on the gradients of each client to obtain clipped gradients, and Gaussian noise is added to the clipped gradients to obtain encrypted gradients.
[0009] Optionally, updating the global model parameters in the server based on the momentum mechanism and the cryptographic gradient includes: The average gradient is obtained by aggregating the encryption gradients of each client, and the momentum factor of the current model training round is determined based on the average gradient. The global model parameters for the current training round are obtained based on the momentum factor and the global model parameters updated in the previous training round.
[0010] Optionally, determining the momentum factor for the current model training epoch based on the average gradient includes: Determine the momentum update factor, and sum the product of the momentum update factor and the momentum factor of the previous model training round with the average gradient to obtain the momentum factor of the current model training round.
[0011] Accordingly, the global model parameters for the current training epoch are updated based on the momentum factor and the global model parameters from the previous training epoch, including: The learning rate for the current model training epoch is determined by subtracting the product of the momentum factor and the learning rate for the current model training epoch from the global model parameters of the previous model training epoch.
[0012] To address the aforementioned technical problems, this invention provides a model privacy protection device, comprising: The first module is used to send the global model parameters from the server to all clients at the start of the current model training round. The second module is used to determine the candidate model parameter set from the model parameter set of each client based on the compressed sensing algorithm; The third module is used to determine the gradient of each candidate model parameter set based on the zero-order optimization algorithm, and to encrypt the gradient based on the differential privacy algorithm to obtain the encrypted gradient. The fourth module is used to update the client's local model based on the cryptographic gradient, upload the cryptographic gradient to the server, and update the global model parameters in the server based on the momentum mechanism and the cryptographic gradient.
[0013] To solve the above-mentioned technical problems, the present invention provides an electronic device, comprising: Memory, used to store computer programs; A processor is used to implement the model privacy protection method described above when executing the computer program.
[0014] To address the aforementioned technical problems, the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the aforementioned model privacy protection method.
[0015] As can be seen, this invention sends the global model parameters from the server to all clients at the start of the current model training round; determines candidate model parameter sets from the model parameter sets of each client based on a compressed sensing algorithm; determines the gradients of each candidate model parameter set based on a zero-order optimization algorithm; encrypts the gradients using a differential privacy algorithm to obtain encrypted gradients; updates the local model on the client based on the encrypted gradients; uploads the encrypted gradients to the server; and updates the global model parameters on the server based on the momentum mechanism and the encrypted gradients. This invention combines the momentum mechanism and differential privacy mechanism during distributed model training, improving model convergence efficiency while achieving privacy protection. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0017] Figure 1 A flowchart illustrating a model privacy protection method provided in an embodiment of the present invention; Figure 2 This is a structural block diagram of a model privacy protection device provided in an embodiment of the present invention. Detailed Implementation
[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0019] In high-dimensional zero-order optimization problems, traditional zero-order methods face high optimization costs, leading to inefficient local training. In distributed systems, data is typically distributed across multiple clients or nodes. However, standard distributed learning methods often neglect privacy protection when handling such data, increasing the risk of data leakage. Differential privacy ensures strict privacy protection by introducing random perturbations and noise. However, this randomness and noise inevitably compromise model utility. Therefore, maximizing model utility while ensuring privacy remains a key challenge in distributed machine learning.
[0020] In zero-order optimization, data sampling and stochastic gradient estimation suffer from large noise variance, leading to poor convergence performance and low query efficiency. Existing work applying stochastic gradient descent and its variants to zero-order optimization is hampered by the challenges of high-dimensional and non-convex problems. This is mainly because gradient estimation based solely on function value queries introduces noise dependence and high variance, thus slowing down convergence and reducing query efficiency. In distributed zero-order optimization scenarios, the application of differential privacy introduces random perturbations and biases into zero-order sparse gradient estimation. These perturbations and biases exacerbate noise and training oscillations, compromising the model's convergence stability and generalization ability.
[0021] However, as AI models continue to grow in scale and computational demands increase, obtaining gradient information remains quite challenging in practical applications. For example, in fields such as healthcare and financial risk modeling, it is often only possible to query function values, making gradient-based optimization infeasible. Zero-order methods, on the other hand, estimate gradients from function values. Therefore, zero-order methods have become the primary approach for machine learning and black-box optimization problems.
[0022] Zero-order methods are widely used in distributed learning and other black-box machine learning tasks, especially in high-dimensional, computationally expensive scenarios where gradient information is difficult to obtain. Existing zero-order optimization methods are mostly based on finite difference and stochastic search, using linear interpolation strategies and reusing historical iteration queries to reduce the complexity of gradient estimation, thereby improving query efficiency. In recent years, various zero-order algorithms for convex and non-convex optimization problems have been proposed, but their theoretical analysis and convergence guarantees under non-convex settings remain insufficient. Meanwhile, although some distributed learning methods have made progress in reducing communication overhead, they often rely on complex perturbation mechanisms or strict assumptions about data distribution, limiting their widespread application in practice.
[0023] In distributed learning scenarios, data privacy is equally crucial. Existing techniques typically introduce differential privacy mechanisms into gradients, such as adding Gaussian noise, to meet the privacy budget requirements of different clients. However, these methods rely on gradient information, making them unsuitable when gradients are not directly available or the problem involves black-box optimization. To address this issue, existing solutions attempt to combine zero-order optimization with differential privacy by injecting random noise into model updates or outputs uploaded by clients to achieve privacy protection. However, while ensuring privacy, these solutions still suffer from slow convergence speeds or insufficient query efficiency.
[0024] On the other hand, momentum techniques, as an important component of optimization methods, play a crucial role in improving the convergence and stability of algorithms. Classical momentum methods, such as the heavy ball method and accelerated gradient method, have been widely applied in both convex and non-convex optimization. In zero-order optimization, researchers have also introduced momentum mechanisms to alleviate the high variance problem of gradient estimation and accelerate convergence. Furthermore, in distributed learning scenarios, momentum mechanisms can achieve linear acceleration of communication and computation, thereby effectively improving overall training efficiency. However, in existing zero-order distributed methods, the integration of momentum with differential privacy mechanisms is still insufficient, and there is a lack of unified consideration for query efficiency, convergence, and privacy protection.
[0025] The following combination Figure 1 , Figure 1 A flowchart of a model privacy protection method provided in an embodiment of the present invention, the method may include: S101: At the start of the current model training round, send the global model parameters from the server to all clients.
[0026] Edge computing is a distributed computing paradigm that moves computing, storage, and networking resources from traditional centralized cloud data centers to network edge nodes closer to data sources or terminal devices. By performing local data processing, analysis, and decision-making at the edge, edge computing can significantly reduce network transmission latency, save backbone bandwidth, and alleviate cloud load, while meeting real-time, privacy, and reliability requirements. This embodiment can push distributed machine learning down to a three-tier "cloud-edge-device" architecture, enabling terminal / edge nodes to complete sub-model training locally, uploading only encrypted gradients or quantization parameters, achieving low-latency, low-bandwidth, and high-privacy collaborative learning.
[0027] In this embodiment, at the start of the current model training epoch, the global model parameters from the server can be sent to all clients. These global model parameters are those from the previous training epoch after the update. Upon receiving the global model parameters, the clients can use the DP-ZOCompress (Differentially Private Zeroth-Order Compressed Sensing) algorithm to calculate gradients locally and update their local models. After the update, the gradients are uploaded to the server so that the server can aggregate gradients and update the global model parameters.
[0028] S102: Determine the candidate model parameter set from the model parameter set of each client based on the compressed sensing algorithm.
[0029] This embodiment can determine the candidate model parameter set from the model parameter set of each client based on the compressed sensing algorithm. The compressed sensing algorithm can reduce the dimensionality of the model parameters that need to be processed, thereby reducing the amount of added differential privacy noise and ultimately improving the model utility.
[0030] Specifically, in this embodiment, the model parameter set can be divided into multiple model parameter groups, and a parameter label can be set for each model parameter in the model parameter group; a random perturbation vector can be constructed, and the function value query results can be obtained by performing two function value queries on each model parameter based on the random perturbation vector; the target ratio can be determined based on the function value query results, and the model parameters in each model parameter group whose parameter labels are the same as the target ratio can be added to the candidate model parameter set.
[0031] For example, for any client, the candidate model parameter set R can be initialized to be empty.
[0032] The model parameter set S is divided into multiple model parameter groups s. j s jThis refers to the j-th model parameter group obtained from the partitioning.
[0033] The model parameter set S contains global model parameters sent by the server, with a dimension of d. In this embodiment, the model parameter set can be randomly shuffled and divided into several model parameter groups, each with a size of n.
[0034] Furthermore, a corresponding parameter label can be set for each model parameter in the model parameter group. For example, the parameter label for the i-th model parameter in the j-th model parameter group can be h. ij .
[0035] This embodiment can construct a random perturbation vector, where the perturbation vector of the i-th model parameter in the j-th model parameter group can be u. ij and v ij The calculation method can be shown in the following formula: ; ; In the formula, u ij Let v be the perturbation vector of the i-th model parameter in the j-th model parameter set in the u direction. ij Let μ be the perturbation vector of the i-th model parameter in the j-th model parameter set in the v direction, and let r be the finite difference step size. ij Let be a random variable representing the direction of random perturbation of the i-th model parameter in the j-th model parameter group.
[0036] The function value query result is obtained by performing two function value queries on each model parameter based on the random perturbation vector, where x ij For the i-th model parameter in the j-th model parameter group, the corresponding L(x) ij The result of the query for the unperturbed function value is L(x). The result of the query for the first function value after perturbation is L(x). ij +v ij The query result for the first function value after perturbation is L(x). ij +u ij ), where L is the loss function.
[0037] This embodiment can determine the corresponding target ratio based on the function value query results of each model parameter. Specifically, the difference between the first function value query result and the unperturbed function value query result is used as the numerator; the sum of the second function value query result and the unperturbed function value query result is used as the denominator; and the target ratio is determined based on the ratio of the numerator and the denominator.
[0038] The formula for calculating the target ratio is as follows: ; In the formula, q ijThis represents the target ratio corresponding to the i-th model parameter in the j-th model parameter group, where round is the rounding function.
[0039] This embodiment can filter model parameters into a candidate model parameter set based on a target ratio. Specifically, for any model parameter x on the client side... ij When its target ratio is equal to the parameter label, that is, q ij =h ij If so, the model parameter can be added to the candidate model parameter set.
[0040] This embodiment can perform the above judgment process on the model parameters in each model parameter group to obtain the final candidate model parameter set.
[0041] This embodiment can execute the above process for each client to obtain the candidate model parameter set corresponding to each client.
[0042] S103: The gradient of each candidate model parameter set is determined based on the zero-order optimization algorithm, and the gradient is encrypted based on the differential privacy algorithm to obtain the encrypted gradient.
[0043] This embodiment can determine the gradient of each candidate model parameter set based on a zero-order optimization algorithm. Specifically, the parameter gradient of each model parameter in the candidate model parameter set is determined based on the zero-order optimization algorithm; the mean of the parameter gradients of each candidate model parameter set is determined as the gradient of each candidate model parameter set.
[0044] For example, for the nth client, its corresponding candidate model parameter set is R. n This refers to the nth candidate model parameter set. In this embodiment, zero-order optimization can be performed on each model parameter in each candidate model parameter set to obtain the corresponding parameter gradient. The calculation method is as follows: ; In the formula, g mn For the m-th model parameter x in the n-th candidate model parameter set mn The gradient of the parameters, L(x) mn +μe mn L(x) represents the query result of the function after finite difference estimation. mn ) is x mn Unperturbed query results, e mn For x mn The corresponding unit vector.
[0045] After obtaining the parameter gradients, the mean of the parameter gradients of each candidate model parameter set is further determined as the gradient of each candidate model parameter set, for example, g. n Let the parameter set of the candidate model be R n The gradient of the average gradient of each model parameter.
[0046] This embodiment can encrypt the gradient based on the differential privacy algorithm to obtain the encrypted gradient. Specifically, the gradient of each client can be clipped to obtain the clipped gradient, and Gaussian noise can be added to the clipped gradient to obtain the encrypted gradient.
[0047] After the client uses the DP-ZOCompress algorithm to filter out non-zero gradients, it applies differential privacy techniques to perform gradient clipping, as shown in the following equation: ; In the formula, gc n Let the parameter set of the candidate model be R n gradient g n The clipping gradient after clipping, max is the maximum value function, C is the gradient clipping threshold, and ||g n ||2 represents the gradient g n The 2-norm.
[0048] Furthermore, Gaussian noise is added to the clipped gradient to obtain the encrypted gradient, as shown in the following equation: gs n =gc n +z n ; In the formula, gs n For clipping gradient gc n The encrypted gradient after Gaussian noise encryption, z n For GC n Gaussian noise.
[0049] S104: Update the client's local model based on encrypted gradients and upload the encrypted gradients to the server. Update the global model parameters on the server based on the momentum mechanism and encrypted gradients.
[0050] This embodiment can update the local model of each client based on the encryption gradient and upload the encryption gradient to the server.
[0051] This embodiment can update the global model parameters in the server based on the momentum mechanism and encrypted gradient. Specifically, it can aggregate the encrypted gradients of each client to obtain the average gradient, determine the momentum factor of the current model training round based on the average gradient, and update the global model parameters of the current model training round based on the momentum factor and the global model parameters of the previous model training round.
[0052] This embodiment does not limit the specific method for determining the momentum factor. Generally, the momentum update factor can be determined, and the momentum factor of the current model training round is obtained by summing the product of the momentum update factor and the momentum factor of the previous model training round with the average gradient.
[0053] For example, the gradient of the aggregation client: ; In the formula, Let N be the average gradient and N be the total number of clients.
[0054] The momentum factor can be calculated as follows: ; In the formula, m t m is the momentum factor for the current model training epoch. t-1 β is the momentum factor from the previous model training epoch, and β is the momentum update factor. The average gradient for the current model training epochs.
[0055] This embodiment can determine the learning rate of the current model training epoch. The global model parameters of the current model training epoch are determined by subtracting the product of the momentum factor and the learning rate of the current model training epoch from the global model parameters of the previous model training epoch, as shown in the following formula: ; In the formula, The global model parameters for the current training epoch. These are the global model parameters from the previous training epoch. The learning rate for the current training epoch of the model.
[0056] The above process can be used to update the model parameters for the current round. In this embodiment, the model can be trained in the same way in each training round until the model converges or reaches the preset training round.
[0057] To address the issue of high computational overhead, this solution employs an algorithm based on compressed sensing principles on the client side to filter out non-zero elements in the candidate set, thereby reducing computational complexity. To address the privacy leakage problem in distributed algorithms, this solution utilizes differential privacy technology to add noise to the filtered elements, performing gradient clipping and incorporating Gaussian noise. To resolve the issues of high variance and instability introduced by differential privacy noise and zero-order methods, this solution introduces a momentum mechanism on the server side for updates, accumulating historical gradients.
[0058] Compared to existing technologies, this method relies solely on function values rather than gradient information during the optimization process, allowing for direct application in black-box optimization and non-differentiable objective function scenarios, thus overcoming the limitations of first-order methods. Furthermore, it introduces a differential privacy mechanism in zero-order estimation and incorporates momentum concepts to smooth gradient estimation from noise effects, significantly improving the algorithm's convergence speed and stability in distributed environments while further reducing the risk of privacy leaks, achieving a balance between optimization performance and privacy protection.
[0059] Based on the above embodiments, the present invention combines momentum mechanism and differential privacy mechanism in the distributed training process of the model, thereby improving the model convergence efficiency while achieving privacy protection.
[0060] The following combination Figure 2 , Figure 2 This is a structural block diagram of a model privacy protection device provided in an embodiment of the present invention. The device may include: The first module 100 is used to send the global model parameters from the server to all clients at the start of the current model training round. The second module 200 is used to determine the candidate model parameter set from the model parameter set of each client based on the compressed sensing algorithm; The third module 300 is used to determine the gradient of the parameter set of each candidate model based on the zero-order optimization algorithm, and to encrypt the gradient based on the differential privacy algorithm to obtain the encrypted gradient. The fourth module 400 is used to update the local model on the client based on the encrypted gradient and upload the encrypted gradient to the server. It updates the global model parameters on the server based on the momentum mechanism and the encrypted gradient.
[0061] Based on the above embodiments, the present invention combines momentum mechanism and differential privacy mechanism in the distributed training process of the model, thereby improving the model convergence efficiency while achieving privacy protection.
[0062] Based on the above embodiments, the second module 200 may include: The first unit is used to divide the model parameter set into multiple model parameter groups and set the parameter label for each model parameter in the model parameter group; The second unit is used to construct a random perturbation vector, and to perform two function value queries on each model parameter based on the random perturbation vector to obtain the function value query results; The third unit is used to determine the target ratio based on the function value query results, and adds the model parameters in each model parameter group whose parameter labels are the same as the target ratio to the candidate model parameter set.
[0063] Based on the above embodiments, the third unit may include: The first sub-unit is used to take the difference between the query result of the first function value and the query result of the unperturbed function value as the numerator; The second sub-unit is used to use the sum of the second function value query result and the unperturbed function value query result as the denominator; The third subunit is used to determine the target ratio based on the ratio of the numerator to the denominator.
[0064] Based on the above embodiments, the third module 300 may include: The fourth unit is used to determine the parameter gradients of each model parameter in the candidate model parameter set based on the zeroth-order optimization algorithm; The fifth unit is used to determine the mean of the parameter gradients of each candidate model parameter set as the gradient of each candidate model parameter set.
[0065] Based on the above embodiments, the third module 300 may include: The sixth unit is used to perform gradient clipping on the gradients of each client to obtain clipped gradients, and to add Gaussian noise to the clipped gradients to obtain encrypted gradients.
[0066] Based on the above embodiments, the fourth module 400 may include: The seventh unit is used to aggregate the encrypted gradients from each client to obtain the average gradient, and to determine the momentum factor of the current model training round based on the average gradient. The eighth unit is used to obtain the global model parameters for the current model training round based on the momentum factor and the global model parameters updated in the previous model training round.
[0067] Based on the above embodiments, the seventh unit may include: The fourth sub-unit is used to determine the momentum update factor. The momentum update factor is obtained by multiplying the momentum update factor with the momentum factor of the previous model training round and summing it with the average gradient.
[0068] Correspondingly, the eighth unit may include: The fifth sub-unit is used to determine the learning rate for the current model training epoch. It determines the global model parameters for the current model training epoch by subtracting the product of the momentum factor and the learning rate for the current model training epoch from the global model parameters of the previous model training epoch.
[0069] Based on the above embodiments, the present invention also provides an electronic device, which may include a memory and a processor. The memory stores a computer program, and when the processor calls the computer program in the memory, it can implement the steps provided in the above embodiments. Of course, the device may also include various necessary network interfaces, a power supply, and other components.
[0070] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by an execution terminal or processor, can implement the method provided in the embodiments of the present invention; the storage medium may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0071] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
Claims
1. A model privacy protection method, characterized in that, include: At the start of the current model training round, the global model parameters in the server are sent to all clients; Candidate model parameter sets are determined from the model parameter sets of each client based on the compressed sensing algorithm; The gradient of each candidate model parameter set is determined based on the zero-order optimization algorithm, and the gradient is encrypted based on the differential privacy algorithm to obtain the encrypted gradient. The client's local model is updated based on the cryptographic gradient, and the cryptographic gradient is uploaded to the server. The global model parameters in the server are then updated based on the momentum mechanism and the cryptographic gradient.
2. The model privacy protection method according to claim 1, characterized in that, Based on the compressed sensing algorithm, a candidate model parameter set is determined from the model parameter set of each client, including: The model parameter set is divided into multiple model parameter groups, and a parameter label is set for each model parameter in the model parameter group; Construct a random perturbation vector, and perform two function value queries on each of the model parameters based on the random perturbation vector to obtain the function value query results; Based on the function value query results, the target ratio is determined, and the model parameters in each model parameter group whose parameter labels are the same as the target ratio are added to the candidate model parameter set.
3. The model privacy protection method according to claim 2, characterized in that, Determining the target ratio based on the function value query result includes: The difference between the query result of the first function value and the query result of the unperturbed function value is taken as the numerator; The sum of the second function value query result and the unperturbed function value query result is used as the denominator. The target ratio is determined based on the ratio of the numerator to the denominator.
4. The model privacy protection method according to claim 1, characterized in that, The gradient of each candidate model parameter set is determined based on a zero-order optimization algorithm, including: The parameter gradients of each model parameter in the candidate model parameter set are determined based on the zero-order optimization algorithm. The mean of the parameter gradients of each candidate model parameter set is determined as the gradient of each candidate model parameter set.
5. The model privacy protection method according to claim 1, characterized in that, The encrypted gradient is obtained by encrypting the gradient using a differential privacy algorithm, including: Gradient clipping is performed on the gradients of each client to obtain clipped gradients, and Gaussian noise is added to the clipped gradients to obtain encrypted gradients.
6. The model privacy protection method according to claim 1, characterized in that, Updating the global model parameters in the server based on the momentum mechanism and the cryptographic gradient includes: The average gradient is obtained by aggregating the encryption gradients of each client, and the momentum factor of the current model training round is determined based on the average gradient. The global model parameters for the current training round are obtained based on the momentum factor and the global model parameters updated in the previous training round.
7. The model privacy protection method according to claim 6, characterized in that, Determining the momentum factor for the current model training epoch based on the average gradient includes: Determine the momentum update factor, and sum the product of the momentum update factor and the momentum factor of the previous model training round with the average gradient to obtain the momentum factor of the current model training round. Accordingly, the global model parameters for the current training epoch are updated based on the momentum factor and the global model parameters from the previous training epoch, including: The learning rate for the current model training epoch is determined by subtracting the product of the momentum factor and the learning rate for the current model training epoch from the global model parameters of the previous model training epoch.
8. A model privacy protection device, characterized in that, include: The first module is used to send the global model parameters from the server to all clients at the start of the current model training round. The second module is used to determine the candidate model parameter set from the model parameter set of each client based on the compressed sensing algorithm; The third module is used to determine the gradient of each candidate model parameter set based on the zero-order optimization algorithm, and to encrypt the gradient based on the differential privacy algorithm to obtain the encrypted gradient. The fourth module is used to update the client's local model based on the cryptographic gradient, upload the cryptographic gradient to the server, and update the global model parameters in the server based on the momentum mechanism and the cryptographic gradient.
9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for implementing the model privacy protection method as described in any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the model privacy protection method as described in any one of claims 1 to 7.