Data distributed encryption sharing and multi-party collaborative learning method

Abstract

The invention discloses a data distributed encryption sharing and multi-party collaborative learning method. The method comprises the following steps: initializing a fault detection model and issuing the fault detection model to a client; utilizing the equipment data to obtain a fault detection model after the current round of training; uploading the corresponding first model parameters to a server; calculating sensitivity according to historical model parameters; cutting a second model parameter obtained by aggregating the first model parameters, and adding differential privacy noise to obtain a third model parameter; and issuing the third model parameter to the client, and performing next round of federated learning training on the fault detection model by using the third model parameter until a preset termination condition is reached. According to the method, the sensitivity is obtained according to the importance and the privacy degree, the intensity of differential privacy noise is dynamically adjusted, the privacy protection and model accuracy are effectively balanced, a double-cloud server architecture is adopted and combined with a secret sharing mechanism, and it is ensured that privacy protection is achieved in the whole process of model training.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of federated learning, and particularly relates to a data distributed encryption sharing and multi-party collaborative learning method. BACKGROUND

[0002] Federated learning is a new collaborative machine learning paradigm that is widely used in medical, financial and other fields that require high data privacy. The main advantage of federated learning is that multiple participants can share model parameters without directly exchanging data, thereby improving model performance while protecting data privacy. The federated learning process consists of three iterative steps: 1) clients train their local data to generate individual local models and transmit model parameters to the central server; 2) the central server aggregates model parameters according to a predefined aggregation rule to update the global model; 3) the updated global model is transmitted back to the client for the next round of learning. However, federated learning faces serious user privacy leakage problems during model training. On the one hand, although the client does not directly share data, an honest and curious server can analyze the local model uploaded by each client and use inference attacks to infer the sensitive information of the client. On the other hand, a malicious client can also launch inference attacks on the aggregated global model to infer the sensitive data of other clients. SUMMARY

[0003] In order to solve the above problems existing in the prior art, the application provides a data distributed encryption sharing and multi-party collaborative learning method.

[0004] The technical problem to be solved by the application is solved by the following technical scheme: In a first aspect, the application provides a data distributed encryption sharing and multi-party collaborative learning method, which comprises: The server initializes a fault detection model and distributes the fault detection model to the clients participating in collaborative learning; The client trains the fault detection model using device data to obtain the fault detection model after the current round of training; The client uploads the first model parameter of the fault detection model after the current round of training to the server; The server calculates the sensitivity according to the historical model parameter; The server clips the second model parameter obtained by aggregating the first model parameter uploaded by each client, and adds the differential privacy noise to the clipped second model parameter to obtain the third model parameter; wherein the differential privacy noise is calculated according to the sensitivity; The server sends the third model parameters to the client, and the client uses the third model parameters to perform the next round of federated learning training on the fault detection model until the preset termination condition is reached.

[0005] Optionally, the server calculates sensitivity based on historical model parameters, including: The server calculates the privacy and importance of the current round based on the historical model parameters; The server determines the sensitivity based on the privacy level and the importance level.

[0006] Optionally, the server prunes the second model parameters obtained by aggregating the first model parameters uploaded by each client, and adds the differential privacy noise to the pruned second model parameters to obtain the third model parameters, including: The server determines the pruning threshold based on the historical model parameters; The server prunes the second model parameters obtained by aggregating the first model parameters uploaded by each client according to the pruning threshold, and obtains the pruned second model parameters. The server obtains the trimmed sensitivity based on the sensitivity and the modulus of the sensitivity. The server obtains the noise level based on the trimmed sensitivity. The server obtains the differential privacy noise based on the second model parameters and the noise level; The server adds the differential privacy noise to the cropped second model parameters to obtain the third model parameters.

[0007] Optionally, the privacy level is represented as follows: ; in, This indicates the level of privacy. Indicates the first Second model parameters in rounds The first in One parameter, Represents variance. This represents the updated parameter values ​​of the historical model parameters. As a preset constant, Indicates the second model parameters The Middle The parameters are listed first. Average update volume per round Indicates the number of iterations.

[0008] Optionally, the importance is expressed as follows: ; in, This indicates the degree of importance.

[0009] Optionally, the sensitivity is expressed as follows: ; in, Indicates the first Second model parameters in rounds The first in Sensitivity of each parameter This is used to avoid extremely small numbers with a denominator of 0.

[0010] The third model parameters are represented as follows: ; in, This represents the parameters of the third model. This represents the parameters of the second model after cropping. Let be differential privacy noise, representing noise with a mean of 0 and a variance of . Gaussian noise, Indicates the second model parameters The first in The noise level of each parameter.

[0011] Optionally, the second model parameters The first in Noise level of each parameter It is expressed as follows: ; in, This indicates the number of parameters in the second model. Indicates the basic noise level. This indicates the sensitivity after cropping.

[0012] Optionally, the server can be a dual-cloud server based on secure multi-party computation.

[0013] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects: In the above technical solution, this invention proposes the concepts of importance and privacy, derives sensitivity based on importance and privacy, and dynamically adjusts the intensity of differential privacy noise, effectively balancing privacy protection and model accuracy. Furthermore, by employing a dual-cloud server architecture based on secure multi-party computation combined with a secret-sharing mechanism, privacy protection is ensured throughout the entire model training process. Compared with existing privacy-preserving federated methods, this invention demonstrates superior performance in terms of both model accuracy and privacy protection.

[0014] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0015] Figure 1 This is a flowchart of a data distributed encrypted sharing and multi-party collaborative learning method provided by an embodiment of the present invention; Figure 2a This is a schematic diagram illustrating the effect of a clipping threshold provided in an embodiment of the present invention; Figure 2b This is a schematic diagram illustrating the effect of another clipping threshold provided in an embodiment of the present invention; Figure 2c This is a schematic diagram illustrating the effect of another clipping threshold provided in an embodiment of the present invention; Figure 3a This is a schematic diagram illustrating the influence of a sensitive factor provided in an embodiment of the present invention; Figure 3b This is a schematic diagram illustrating the influence of another sensitive factor provided in an embodiment of the present invention; Figure 3c This is a schematic diagram illustrating the influence of another sensitive factor provided in an embodiment of the present invention; Figure 4a This is a schematic diagram illustrating the results under different communication rounds provided in an embodiment of the present invention; Figure 4b This is a schematic diagram illustrating the results under another different communication round provided in an embodiment of the present invention; Figure 4c This is a schematic diagram illustrating the results under different communication rounds provided in this embodiment of the invention; Figure 5a This is a bar chart showing the computation and storage overhead across three datasets, provided by an embodiment of the present invention. Figure 5b This is a schematic diagram showing the measurement results of the accuracy of the three method models provided in the embodiments of the present invention. Detailed Implementation

[0016] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0017] Figure 1 This is a flowchart of a data distributed encrypted sharing and multi-party collaborative learning method provided by an embodiment of the present invention, such as... Figure 1 As shown, the method may include the following steps: S1. The server initializes the fault detection model and distributes the fault detection model to the clients participating in collaborative learning.

[0018] S2. The client uses device data to train the fault detection model and obtains the fault detection model after the current round of training.

[0019] Understandably, in a federated learning system with centralized differential privacy, consider a server. and a group of distributed clients The number of clients is Each client Both can access the local device training set. ,in, Indicates client The first in the equipment training set One sample, Indicates the client The first in the equipment training set The label corresponding to each sample This represents the number of samples. Initial fault detection model parameters. Propagated by the server. In the t-th iteration of federated learning, the local model parameters are represented as... .

[0020] Client Using stochastic gradient descent for optimization, the loss function can be expressed as follows: ; in, Denotes the sub-loss function. This represents the model's predicted output.

[0021] S3. The client uploads the first model parameters of the fault detection model after the current round of training to the server.

[0022] S4. The server calculates sensitivity based on historical model parameters.

[0023] Optionally, S4 may include: The server calculates the privacy and importance of the current round based on historical model parameters; The server's sensitivity is determined based on its privacy and importance.

[0024] Understandably, the server learns the privacy and importance of model parameters based on historical model parameters, and designs fine-grained sensitivity based on the heterogeneity of the parameters.

[0025] Optionally, the privacy level is represented as follows: ; in, Indicates privacy level. Indicates the first Second model parameters in rounds The first in One parameter, Represents variance. a parameter update value representing a historical model parameter, is a preset constant, a parameter update value representing a second model parameter the average update amount of the th parameter in the previous round, the average update amount of the th parameter in the previous round,

[0026] may be represented as follows: ; Optionally, the importance is represented as follows: ; wherein, represents the importance, the importance reflects the degree of influence of the parameter on the model prediction output, specifically, if a slight change in the parameter will cause a significant change in the model output, it is considered that the parameter has high importance and corresponds to a larger importance measure.

[0027] Optionally, the sensitivity is represented as follows: ; wherein, represents the sensitivity of the th parameter in the second model parameter in the th round, is a very small number of quantity used to avoid the denominator being 0.

[0028] S5, the server trims the second model parameter obtained by aggregating the first model parameter uploaded by each client, and adds differential privacy noise to the trimmed second model parameter to obtain a third model parameter; wherein the differential privacy noise is calculated according to the sensitivity.

[0029] Optionally, S5 can include: The server determines a trimming threshold according to the historical model parameter; The server trims the second model parameter obtained by aggregating the first model parameter uploaded by each client according to the trimming threshold to obtain a trimmed second model parameter; The server obtains the trimmed sensitivity according to the sensitivity and the modulus of the sensitivity; The server obtains the noise quantity according to the trimmed sensitivity; The server obtains the differential privacy noise according to the second model parameter and the noise quantity; The server adds differential privacy noise to the trimmed second model parameter to obtain a third model parameter.

[0030] It can be understood that for each round The model uploaded by the client forms the global model parameter of this round after aggregation by the server In each round of measurement, the latest second model parameter .

[0031] In the design of adaptive differential privacy mechanism, the sensitivity of each model parameter should be estimated first. The sensitivity depends on the difference between the current round model parameter and the last round model parameter . To achieve differential privacy and control the impact of each parameter update, a sensitivity factor is introduced to determine the clipping threshold by scaling the parameter update . To control the magnitude of each update and prevent sensitive information leakage, the update of the second model parameter needs to be clipped, and the clipped second model parameter can be represented as: ; This ensures that the updated parameter remains within the sensitivity determined range and enhances the robustness to abnormal parameters. Sensitivity estimation is a key step in achieving differential privacy, and the inclusion of privacy degree and importance in the calculation of sensitivity can further optimize the privacy protection strategy, making it more adaptable. Considering the two prior assumptions that: 1. Since parameters with high privacy are more likely to leak sensitive information, more noise protection should be introduced for these parameters; 2. Parameters with high importance have a greater impact on model output, and excessive noise may severely affect model performance.

[0032] Considering that the introduction of importance and privacy may lead to uncontrollable sensitivity, the overall sensitivity needs to be clipped to limit it within a reasonable range to ensure differential privacy properties even in the presence of outliers, expressed as: ; ; where represents the modulus of sensitivity, represents the clipped sensitivity, represents the lower limit of the overall sensitivity, represents the upper limit of the overall sensitivity.

[0033] Under the Gaussian mechanism, to ensure that the algorithm satisfies the given differential privacy parameters and , the determination of the basic noise scale is crucial. First, using the Rényi differential privacy (RDP) definition, the cumulative privacy loss is calculated as where denotes the aggregation round, denotes the standard deviation of the noise, is the order of the Rényi divergence. According to the relationship between RDP and traditional differential privacy, the privacy budget must satisfy the following condition: where is an integer and must satisfy: ; Since , to ensure that the inequality always holds, the following condition must be met: ; Let the value of , assuming that at , the value of the expression is equal to , then if the minimum value is taken here, the condition is met; if it is not the minimum value at , but the minimum value is still less than , the condition is still valid. So to simplify the operation, by solving the following equation: ; the size of the basic noise amount can be obtained. And the noise amount of the th parameter in the second model parameter is represented as follows: ; where, denotes the number of parameters in the second model parameter, denotes the basic noise amount, denotes the clipped sensitivity.

[0034] The third model parameter is represented as follows: ; where, denotes the third model parameter, denotes the clipped second model parameter, is the differential privacy noise, which represents Gaussian noise with a mean of 0 and a variance of , denotes the noise amount of the th parameter in the second model parameter .

[0035] Here is to ensure that the algorithm meets the predetermined The result of optimizing the balance between the amount of noise and the data utility while maintaining the privacy protection level. The noise is added in each parameter update to ensure full differential privacy, and the third model parameters are obtained by adding differential privacy noise to the second model parameters after pruning.

[0036] S6、The server pushes the third model parameters to the client, and the client uses the third model parameters to perform the next round of federated learning training on the fault detection model until the preset termination condition is reached.

[0037] It can be understood that the preset termination condition can be that the training round reaches a preset upper limit or the fault detection model training is completed, and can also be adjusted according to actual conditions.

[0038] Optionally, the server is a double-cloud server based on secure multi-party computation.

[0039] In an embodiment, the previous steps are equivalent to not using a double-cloud server, and a common federated learning uses 1 server to complete aggregation and noise addition, but does not defend against attacks from the server; the application can also be deployed on a double-cloud server to complete the aggregation and noise addition process through the secret sharing of two servers. Due to the characteristics of secret sharing, some calculation processes such as multiplication and division cannot be directly calculated on the double-cloud server as on a single server. In short, the calculation on the double-cloud server is a ciphertext value while the calculation on the single server is a plaintext value. A security measurement protocol and a secure pruning and noise addition protocol based on secure multi-party computation are proposed. In order to realize the above two protocols, a series of basic operation sub-protocols for data secret sharing are proposed, including a secure division protocol, a secure absolute value protocol, and a secure square root protocol. For the secure division protocol, the calculation is which is equivalent to , so as to solve the reciprocal of We use the Newton iteration method to approximately solve it, and the iterative calculation is , and the initial value is . To calculate the exponential part, the definition of the secure exponential protocol is as follows: the participant initializes , the participant initializes , and the double-cloud server calculates using the secure multiplication protocol . Based on the secure exponential protocol, the secure division protocol is defined as follows: the input , that is, the input , and the output , which is calculated first , 10 iterations are calculated, and then the three-multiplication multiplication protocol is called to obtain ​Finally, the security multiplication protocol is calculated .

[0040] For the security square root protocol, it is defined as: , that is, the input , and the output . The Newton iteration method is used to approximate the square root, and the calculation formula is: . First, calculate , and then set the iteration number to 10. Call the security division protocol, and .

[0041] For the security absolute value protocol, it is defined as: , that is, the input , and the output . First, the participants and perform the addition secret sharing operation on 0 to obtain the secret sharing shares and . Then, the two servers set call the security comparison protocol and compare and . If , let ; otherwise, let . Finally, share to the participants and in the form of secret sharing, so as to obtain .

[0042] Based on the above basic operation sub-protocols, the security measurement protocol is introduced. The input includes the second model parameters of N rounds, and the output is the privacy degree and the importance degree.

[0043] The flow of the security measurement protocol is divided into two parts: calculating the privacy degree and calculating the importance degree. When calculating the privacy degree, the participant first calculates the parameter change locally, and then the participant calculates the mean of the parameter change , and then calls the security multiplication protocol to calculate , and finally takes the mean of the parameter transformation variance to obtain the privacy degree under the security protocol: ; When calculating the importance degree, the key is to calculate the absolute value of the parameter change, Si call the security absolute value protocol to safely solve the absolute value of . Then take the average of the absolute values of the last round parameter changes, which is the importance degree: ; The secure clipping and noise adding protocol mainly uses the secure division protocol and the secure absolute value operation protocol . The input includes the privacy degree of the participants , the importance degree , the current global model , the parameters and the number of clients participating in training . The output is the global model after adding noise .

[0044] The flow divides the secure clipping and noise adding protocol into three parts: sensitivity estimation, adaptive clipping and noise adding. When estimating the sensitivity of secure computation , the participants first calculate . The basic operation sub-protocol is called to calculate as the numerator through the three-input multiplication protocol, and then the denominator is calculated using the secure addition . Next obtains through the secure division protocol, and then calculates the overall sensitivity through the formula and adjusts by comparing and . After the final scaling, obtains .

[0045] When determining the clipping range, calculates through the secure multiplication protocol and the secure absolute value protocol. Then determines the clipping threshold through the comparison protocol. Specifically, calls the secure less than protocol to compare the size of and , selects the smaller value and records it as , and then calls the secure greater than or equal to protocol to compare the size of and , selects the larger value, which is the final .

[0046] When adding noise, first, the participants collaborate to generate a random number . Then, the participants ​Compute the noise standard deviation by the secure multiplication protocol locally: ; And securely add the generated noise to the clipped second model parameters by secret sharing, to obtain the third model parameters under the secure protocol.

[0047] To verify the effectiveness and superiority of the present application, comparative experiments are set up. The present algorithm uses Pytorch to code the scheme, and shows the TACC results of model training under various privacy protection methods under different privacy budgets. Other similar methods used for comparison with the present application are DP-FedAvg, BS-DP, and FedDPA. In addition, to verify the superiority of the training efficiency, overhead, and model accuracy of the differential privacy scheme FedPIDP+ combined with secure multi-party computation, simulation experiments are set up, which show the calculation overhead, storage overhead, and model accuracy of federated learning under double cloud servers.

[0048] Three standard data sets MNIST, EMNIST, and CIFAR10 can be used to evaluate the performance of FedPIDP and PedPIDP+. The privacy budget is a key indicator for evaluating the strength of privacy protection, which is defined by the parameters (ε, δ), where represents the level of privacy protection, is used as an auxiliary parameter to specify the acceptable failure probability in differential privacy. To evaluate the learned global model, the test accuracy (TACC) is used, which is the proportion of instances accurately classified by the global model.

[0049] In the experiment combined with secure multi-party computation, the computation time and communication overhead are two key performance indicators for evaluating the protocol. The computation time refers to the duration required by each participant to perform local computation during the execution of the protocol, usually in seconds (s) or milliseconds (ms). On the other hand, the communication overhead represents the amount of data exchanged between participants during the execution of the protocol to complete the computing task, usually in kilobytes (KB), megabytes (MB), or gigabytes (GB).

[0050] Comparative experiments for adaptive differential privacy methods: For MNIST, a fully connected network (FC) with layer sizes {784, 100, 10} can be used as the global model architecture. For EMNIST, a simple convolutional neural network (CNN) is adopted, which includes two 3x3 convolutional layers (the first one has 30 channels, and the second one has 5 channels), each followed by a ReLU activation function and a 2x2 max-pooling operation, and two fully connected layers after these convolutional layers. For CIFAR-10, we use ResNet18 as the global model. The following tables show the model accuracy of the four differential privacy methods on the three datasets under different privacy budgets: The accuracy on the MNIST dataset is shown in Table 1: Table 1

[0051] The accuracy on the EMNIST dataset is shown in Table 2: Table 2

[0052] Figure 2a is an influence diagram of a clipping threshold provided by an embodiment of the present application, Figure 2b is another influence diagram of a clipping threshold provided by an embodiment of the present application, Figure 2c is yet another influence diagram of a clipping threshold provided by an embodiment of the present application, Figure 3a is an influence diagram of a sensitive factor provided by an embodiment of the present application, Figure 3b is another influence diagram of a sensitive factor provided by an embodiment of the present application, Figure 3c is yet another influence diagram of a sensitive factor provided by an embodiment of the present application, Figure 4a is a result diagram under different communication rounds provided by an embodiment of the present application, Figure 4b is another result diagram under different communication rounds provided by an embodiment of the present application, Figure 4c is yet another result diagram under different communication rounds provided by an embodiment of the present application.

[0053] Simulation experiment of the present application under the dual-cloud server based on secure multi-party computation: The computational and storage overheads on the three datasets are shown in Table 3: Table 3

[0054] Figure 5a is a column chart of the computational and storage overheads on the three datasets provided by an embodiment of the present application, Figure 5b is a measurement result diagram of the model accuracy of the three methods provided by an embodiment of the present application, Figure 5aThe column chart shows the calculation and storage overhead of the application on three data sets, and evaluates the TACC of FedAvg, FedPIDP and FedPIDP+ on 3 data sets. Figure 5b The measurement results of the model accuracy of the three methods are shown as Figure 5b As shown in the figure, FedAvg has the highest accuracy, FedPIDP+ (the application under the double cloud server based on secure multi-party computation) has the lowest accuracy, and FedPIDP (the application under the double cloud server not based on secure multi-party computation) is between the two. This is because FedAvg is not affected by any noise, allowing it to maintain the best performance, although this is at the expense of privacy. In contrast, FedPIDP directly adds noise to the plaintext data, and FedPIDP+ introduces noise after secret sharing of model parameters. Compared with FedPIDP, the secret sharing process will nonlinearly transform the model parameters, making the noise addition process complex, but as can be seen from the figure, FedPIDP+ achieves privacy protection for the entire process of federated learning with acceptable loss of model accuracy. It embodies the superiority of the method of the application in the trade-off between privacy protection and model accuracy.

[0055] The application proposes the concepts of importance and privacy degree, obtains sensitivity according to the importance and privacy degree, and dynamically adjusts the strength of differential privacy noise, effectively balancing privacy protection and model accuracy. In addition, the double cloud server architecture based on secure multi-party computation is adopted and combined with the secret sharing mechanism to ensure that the entire process of model training achieves privacy protection. Compared with existing privacy protection federated methods, the application performs superior performance in model accuracy and privacy protection degree.

[0056] It should be noted that the terms "first", "second", and the like are used to distinguish similar objects, and do not necessarily have to be used to describe a particular order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. The implementation described in the following exemplary embodiments does not represent all implementations consistent with the application. Rather, they are only examples of devices and methods consistent with some aspects of the application.

[0057] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described can be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples described in the specification.

[0058] Although the present application is described herein in conjunction with various embodiments, those skilled in the art, with the benefit of the drawings and the disclosure, can understand and implement other variations of the disclosed embodiments in the implementation of the claimed application. In the description of the present application, the word "comprising" does not exclude other components or steps, "a" or "one" does not exclude a plurality, and "plurality" means two or more, unless otherwise expressly specified. In addition, some measures are described in different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0059] The above is a further detailed description of the present application in conjunction with specific preferred embodiments, and cannot be considered as limiting the specific implementation of the present application to these descriptions. For those skilled in the art, without departing from the concept of the present application, a number of simple deductions or substitutions can be made, which should be considered as falling within the scope of protection of the present application.

Claims

1. A method for distributed encryption sharing and multi-party collaborative learning of data, characterized in that, The method comprises: The server initializes a fault detection model and distributes the fault detection model to clients participating in collaborative learning; The client trains the fault detection model using device data to obtain the fault detection model after current round training; The client uploads first model parameters of the fault detection model after current round training to the server; The server calculates a sensitivity according to historical model parameters; The server prunes second model parameters obtained by aggregating the first model parameters uploaded by each client, and adds differential privacy noise to the pruned second model parameters to obtain third model parameters; wherein the differential privacy noise is calculated according to the sensitivity; The server distributes the third model parameters to the client, and the client uses the third model parameters to perform federated learning training on the fault detection model in the next round until a preset termination condition is reached.

2. The method of claim 1, wherein, The server calculates a sensitivity according to historical model parameters, comprising: The server calculates a privacy degree and an importance degree according to the historical model parameters; The server obtains the sensitivity according to the privacy degree and the importance degree.

3. The method of claim 2, wherein, The server prunes second model parameters obtained by aggregating the first model parameters uploaded by each client, and adds differential privacy noise to the pruned second model parameters to obtain third model parameters, comprising: The server determines a pruning threshold according to the historical model parameters; The server prunes second model parameters obtained by aggregating the first model parameters uploaded by each client according to the pruning threshold to obtain the pruned second model parameters; The server obtains a pruned sensitivity according to the sensitivity and a modulus of the sensitivity; The server obtains a noise amount according to the pruned sensitivity; The server obtains the differential privacy noise according to the second model parameters and the noise amount; The server adds the differential privacy noise to the pruned second model parameters to obtain third model parameters.

4. The method of claim 3, wherein, The privacy degree is represented as follows: ； wherein, denotes the privacy degree, denotes the historical model parameter, denotes the first parameter in the second model parameter in the i-th round, denotes the i-th parameter in the second model parameter denotes the variance, denotes the parameter update value of the historical model parameter, is a preset constant, denotes the average update amount of the i-th parameter in the second model parameter in the previous rounds, denotes the average update amount of the i-th parameter in the second model parameter in the previous rounds, denotes the iteration number.

5. The method of claim 4, wherein, The importance degree is represented as follows: ； wherein represents the importance degree.

6. The method of claim 5, wherein, The sensitivity is represented as follows: ； wherein, represents the sensitivity of the i-th parameter in the second model parameter at the j-th round, to the i-th parameter in the first model parameter at the j-th round, is a small number used to avoid division by zero.

7. The method of claim 6, wherein, The third model parameters are represented as follows: ； wherein, denotes the third model parameter, denotes the second model parameter after clipping, is a differential privacy noise, denotes a Gaussian noise with mean 0 and variance , denotes the noise amount of the th parameter in the second model parameter .

8. The method of claim 7, wherein, the second model parameter the first parameter in the second model parameter the noise amount of the first parameter in the second model parameter is represented as follows: ； wherein denotes the number of parameters in the second model parameters, denotes the amount of underlying noise, denotes the sensitivity after clipping.

9. The method of claim 1, wherein, The server is a double-cloud server based on secure multi-party computation.

Images