A method and apparatus for calculating global model parameters.

Abstract

This invention provides a method and apparatus for calculating global model parameters. The method involves constructing an initial global parameter model and determining a first loss function for that model; controlling a client to train the initial global parameter model based on the first loss function, generating first and second trained parameters; generating a first Paillier algorithm key and sending the first Paillier algorithm key and the first trained parameters encrypted with it to a second server, receiving a second Paillier algorithm key from the second server; determining constant values ​​based on the first Paillier algorithm key, the first and second trained parameters; generating target global model parameters and a target loss function; and controlling the client to iterate over the target global model parameters based on the target loss function to generate the final global model parameters. This improves the accuracy and reliability of global model parameter calculation and further ensures data security.

Description

Technical Field

[0001] This invention relates to the field of computational technology for global model parameters, and in particular to a method for calculating global model parameters, a device for calculating global model parameters, an electronic device, and a computer-readable storage medium. Background Technology

[0002] Federated learning, as a distributed machine learning paradigm, does not require data to leave the local client, thus offering some protection for data privacy. However, in practice, due to its distributed nature, federated learning is vulnerable to Byzantine attacks, which can damage the model and lead to inaccurate training results. Traditional attack mitigation methods lack privacy protection strategies, requiring secure multi-party computation between clients, which cannot guarantee that clients are merely curious and not malicious during secure computation. Furthermore, these methods require the server to possess a portion of the training data, making implementation difficult. Summary of the Invention

[0003] The present invention provides a method, apparatus, electronic device, and computer-readable storage medium for calculating global model parameters to overcome or at least partially solve the above-mentioned problems.

[0004] This invention discloses a method for calculating global model parameters. The method is applied to a first server, which is configured with a corresponding second server. The first server and the second server are not allowed to collude with each other, and the method may include:

[0005] Construct an initial global parameter model and determine a first loss function for the initial global parameter model;

[0006] The initial global parameter model is sent to multiple clients to control the clients to train the initial global parameter model based on the first loss function, and generate a first training parameter and a second training parameter that include the local model parameter increment, a second loss function for the local model parameter increment, and the number of data.

[0007] A first Paillier algorithm key is generated, and the first Paillier algorithm key and the first training parameters encrypted with the first Paillier algorithm key are sent to the second server. The second server receives a second Paillier algorithm key generated by the second server. The second server is configured to determine a constant value based on the first Paillier algorithm key, the first training parameters and the second training parameters, and send the constant value to the first server.

[0008] Based on the second Paillier algorithm key and the constant value, the target global model parameters and the target loss function are generated, and the client is controlled to iterate the target global model parameters based on the target loss function to generate the final global model parameters.

[0009] Optionally, the step of controlling the client to train the initial global parameter model based on the first loss function, and generating a first training parameter and a second training parameter that include the local model parameter increment, a second loss function for the local model parameter increment, and the number of data points may include:

[0010] The client is controlled to train the initial global parameter model based on the first loss function to obtain the local model parameter increment, the second loss function for the local model parameter increment, and the number of data.

[0011] Determine a first weight parameter and a second weight parameter for the local model parameter increment, the second loss function, and the number of data points;

[0012] Based on the first weight parameter and the second weight parameter, the local model parameter increment is divided to generate a first local model parameter increment secret shared value and a second local model parameter increment secret shared value.

[0013] The second loss function is divided based on the first weight parameter and the second weight parameter to generate the secret shared value of the first partitioning loss function and the secret shared value of the second partitioning loss function;

[0014] The number of data items is divided based on the first weight parameter and the second weight parameter to generate a first secret sharing value for the number of data items and a second secret sharing value for the number of data items.

[0015] The first local model parameter increment secret shared value, the first partition loss function secret shared value, and the first data number secret shared value are determined as the parameters obtained from the first training.

[0016] The second local model parameter increment secret shared value, the second partition loss function secret shared value, and the second data number secret shared value are determined as the second training parameters.

[0017] Optionally, the step of dividing the local model parameter increment based on the first weight parameter and the second weight parameter to generate a first local model parameter increment secret shared value and a second local model parameter increment secret shared value may include:

[0018]

[0019] The step of dividing the second loss function based on the first weight parameter and the second weight parameter to generate the first and second partitioning loss function secret shared values ​​includes:

[0020] F i =Δ x F i +Δ y F i ,

[0021] The step of dividing the number of data points based on the first weight parameter and the second weight parameter to generate a first data point secret sharing value and a second data point secret sharing value includes:

[0022] n i =Δ x n i +Δ y n i ,

[0023] Among them, w i，d This corresponds to the d-th coordinate size of the local model parameter increment for the i-th client. The increment of the local model parameters, Δ x Let Δ be the first weight parameter. y This is the second weight parameter. This is the incremental secret shared value of the first local model parameters. F is the incremental secret shared value of the second local model parameters. i Let Δ be the second loss function. x F i Let Δ be the secret shared value of the first partitioning loss function. y F i Let n be the secret shared value of the second partitioning loss function. i Δ is the number of data points. x n i The secret shared value for the first number of data items, Δ y n i The second data number is a secret shared value.

[0024] Optionally, the step of generating a first Paillier algorithm key and sending the first Paillier algorithm key and the first training parameters encrypted with the first Paillier algorithm key to the second server may include:

[0025] Calculate the parameter difference obtained from the first training based on the incremental secret shared value of the first local model parameters;

[0026] The parameter difference obtained from the first training is encrypted using the first Paillier algorithm key;

[0027] The encrypted parameter difference obtained from the first training is sent to the second server.

[0028] Optionally, the step of calculating the first training parameter difference based on the first local model parameter increment secret shared value and encrypting the first training parameter difference using the first Paillier algorithm key may include:

[0029]

[0030] Where E1(m) represents the encryption of m using the first Paillier algorithm key. The difference between the parameters obtained from the first training is... For the j-th coordinate size corresponding to the local model parameter increment of the i1-th client, The j-th coordinate size corresponds to the local model parameter increment of the i2-th client.

[0031] Optionally, the second server can be configured to perform the step of determining the constant value based on the first Paillier algorithm key, the first trained parameters, and the second trained parameters in the following manner:

[0032] The parameter difference obtained from the second training is calculated based on the incremental secret shared value of the second local model parameters;

[0033] Determine the first and second random numbers;

[0034] The constant value is determined based on the first random number, the second random number, the difference between the parameters obtained from the first training, and the difference between the parameters obtained from the second training.

[0035] Optionally, the step of determining the constant value based on the first random number, the second random number, the difference between the first trained parameters, and the difference between the second trained parameters may include:

[0036]

[0037] Where val is the constant value, r1 is the first random number, and r2 is the second random number.

[0038]

[0039] The difference between the parameters obtained from the second training is denoted by mod.

[0040]

[0041] Divide by The remainder, where N is a constant.

[0042] Optionally, the step of generating target global model parameters and a target loss function based on the second Paillier algorithm key and the constant value, and controlling the client to iterate the target global model parameters based on the target loss function to generate the final global model parameters may include:

[0043] The maximum and minimum values ​​are determined from the incremental secret shared value of the first local model parameters based on the constant values;

[0044] The maximum and minimum values ​​are removed from the local model parameter increments to generate a client index set;

[0045] A first intermediate value is generated based on the client index set, the second local model parameter incremental secret shared value, and the second Paillier algorithm key, and sent to the second server. The second server is configured to generate a second intermediate value based on the client index set, the first local model parameter incremental secret shared value, and the first Paillier algorithm key, and the second server sends the second intermediate value to the first server.

[0046] The third intermediate value is determined based on the second intermediate value;

[0047] The second server is configured to determine a fourth intermediate value based on the first intermediate value; the second server sends the fourth intermediate value to the first server;

[0048] The index set values ​​are determined based on the third intermediate value and the fourth intermediate value;

[0049] Target global model parameters are generated based on the numerical values ​​of the index set;

[0050] The target loss function is determined based on the target global model parameters;

[0051] The target global model parameters are iterated based on the target loss function to generate the final global model parameters.

[0052] Optionally, the step of generating a first intermediate value based on the client index set, the second local model parameter incremental secret shared value, and the second Paillier algorithm key may include:

[0053]

[0054] The second server is configured to perform the step of generating a second intermediate value based on the client index set, the first local model parameter incremental secret shared value, and the first Paillier algorithm key in the following manner:

[0055]

[0056] The step of determining the third intermediate value based on the second intermediate value includes:

[0057]

[0058] The second server is configured to perform the step of determining a fourth intermediate value based on the first intermediate value in the following manner:

[0059]

[0060] The step of determining the index set value based on the third intermediate value and the fourth intermediate value includes:

[0061]

[0062] The step of generating target global model parameters based on the index set values ​​includes:

[0063]

[0064]

[0065] The step of determining the target loss function based on the target global model parameters includes:

[0066]

[0067] Wherein, INDS is the client index set, Tmp1 is the first intermediate value, Tmp2 is the second intermediate value, Tmp3 is the third intermediate value, Tmp4 is the fourth intermediate value, E2(m) is the encryption of m using the first Paillier algorithm key, and Δ y w i，j This is the incremental secret shared value of the second local model parameters. represents the target global model parameters; F represents the target loss function.

[0068] This invention also discloses a method for calculating global model parameters. The method is applied to a second server, which is configured with a corresponding first server. The first and second servers are not collusive. The first server is configured to construct an initial global parameter model and determine a first loss function for the initial global parameter model. The initial global parameter model is sent to multiple clients to control the clients to train the initial global parameter model based on the first loss function, generating a first training parameter and a second training parameter that include local model parameter increments, a second loss function for the local model parameter increments, and the number of data points. A first Paillier algorithm key is generated, and the first Paillier algorithm key and the first training parameter encrypted with the first Paillier algorithm key are sent to the second server. The second server receives a second Paillier algorithm key generated by the second server. The method may include:

[0069] The constant value is determined based on the first Paillier algorithm key, the first training parameters, and the second training parameters;

[0070] The constant value is sent to the first server.

[0071] This invention also discloses a calculation device for global model parameters. The device is applied to a first server, which is configured with a corresponding second server. The first server and the second server are not allowed to collude with each other. The device may include:

[0072] The first loss function determination module is used to construct an initial global parameter model and determine a first loss function for the initial global parameter model.

[0073] The first training parameter generation module is used to send the initial global parameter model to multiple clients to control the clients to train the initial global parameter model based on the first loss function, and generate first training parameters and second training parameters that include local model parameter increments, a second loss function for the local model parameter increments, and the number of data points.

[0074] A first Paillier algorithm key generation module is used to generate a first Paillier algorithm key and send the first Paillier algorithm key and the first training parameters encrypted with the first Paillier algorithm key to the second server, and receive a second Paillier algorithm key generated by the second server; the second server is configured to determine a constant value based on the first Paillier algorithm key, the first training parameters and the second training parameters, and send the constant value to the first server.

[0075] The final global model parameter generation module is used to generate target global model parameters and target loss function based on the second Paillier algorithm key and the constant value, and to control the client to iterate the target global model parameters based on the target loss function to generate the final global model parameters.

[0076] This invention also discloses a calculation device for global model parameters. The device is applied to a second server, which is configured with a corresponding first server. The first and second servers are not collusive. The first server is configured to construct an initial global parameter model and determine a first loss function for the initial global parameter model; send the initial global parameter model to multiple clients to control the clients to train the initial global parameter model based on the first loss function; generate first and second training parameters containing local model parameter increments, a second loss function for the local model parameter increments, and the number of data points; generate a first Paillier algorithm key; and send the first Paillier algorithm key and the first training parameters encrypted with the first Paillier algorithm key to the second server; and receive a second Paillier algorithm key generated by the second server. The device may include:

[0077] The constant value determination module is used to determine the constant value based on the first Paillier algorithm key, the first training parameters, and the second training parameters;

[0078] A constant value sending module is used to send the constant value to the first server.

[0079] This invention also discloses an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0080] The memory is used to store computer programs;

[0081] When the processor executes a program stored in the memory, it implements the method described in the embodiments of the present invention.

[0082] This invention also discloses a computer-readable storage medium storing instructions that, when executed by one or more processors, cause the processors to perform the methods described in this invention.

[0083] The embodiments of the present invention have the following advantages:

[0084] In this embodiment of the invention, an initial global parameter model is constructed, and a first loss function is determined for the initial global parameter model. The initial global parameter model is sent to multiple clients to control the clients to train the initial global parameter model based on the first loss function, generating a first training parameter and a second training parameter that include the local model parameter increment, a second loss function for the local model parameter increment, and the number of data points. A first Paillier algorithm key is generated, and the first Paillier algorithm key and the first training parameter encrypted with the first Paillier algorithm key are sent to a second server, which receives a second Paillier algorithm key generated by the second server. The second server is configured to determine a constant value based on the first Paillier algorithm key, the first training parameter, and the second training parameter, and send the constant value to the first server. Based on the second Paillier algorithm key and the constant value, target global model parameters and a target loss function are generated, and the clients are controlled to iterate on the target global model parameters based on the target loss function to generate the final global model parameters. This improves the accuracy and reliability of the calculation of global model parameters and further ensures data security. Attached Figure Description

[0085] Figure 1 This is a flowchart of the steps of a method for calculating global model parameters provided in Embodiment 1 of the present invention;

[0086] Figure 2 This is a flowchart of another method for calculating global model parameters provided in Embodiment 1 of the present invention;

[0087] Figure 3 This is a flowchart of the steps for calculating global model parameters provided in Embodiment 2 of the present invention;

[0088] Figure 4 This is a structural block diagram of a calculation device for global model parameters provided in Embodiment 3 of the present invention;

[0089] Figure 5 This is a structural block diagram of a calculation device for global model parameters provided in Embodiment 4 of the present invention;

[0090] Figure 6 This is a hardware structure block diagram of an electronic device provided in various embodiments of the present invention;

[0091] Figure 7 This is a schematic diagram of a computer-readable medium provided in an embodiment of the present invention. Detailed Implementation

[0092] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0093] To enable those skilled in the art to better understand the embodiments of the present invention, the technical terms involved in the embodiments of the present invention will be described below.

[0094] Byzantine attacks are a type of network attack in which an intruder (node) attacks a distributed system (comprising many nodes). These attacks send misleading information to disrupt normal operation, thereby compromising the normal functioning of the entire system.

[0095] Robustness refers to a system's ability to survive abnormal and dangerous situations. Robust federated learning means that even if malicious clients send corrupted model parameters to the service, the system can still learn an accurate model.

[0096] Secure multi-party computation refers to the process by which participants, without disclosing their own private data, engage in confidential computation to collaboratively complete a computational task. This technology can meet people's needs for confidential computation using private data, effectively resolving the contradiction between data confidentiality and data sharing.

[0097] FedAvg aggregation method: a commonly used federated learning algorithm that aggregates model parameters through weighted averaging.

[0098] RFA aggregation method: Robust federated aggregation, which relies on updating using geometric median aggregation.

[0099] FOLD aggregation method: An aggregation method that removes corrupted clients by comparing the cosine distance between the client and server gradients.

[0100] With the rapid popularization of smart terminal devices, a large amount of personal information data is collected by various online platforms. In real life, the leakage of personal information is commonplace. In recent years, not only have individuals become increasingly concerned about the confidentiality of their privacy data, but government departments have also successively introduced many laws and regulations concerning the protection of personal privacy and the construction of cybersecurity. Traditional machine learning requires collecting all information on a central server before model training, making it easy for client-side privacy data to be leaked. Federated learning, as a distributed machine learning model, does not require data to leave the local client, thus providing a certain degree of protection for data privacy. In practice, it first establishes a global model on a cloud server and sends it to each client. Then, the model is trained on the client. After training, the client uploads its local gradients or model parameters to the cloud server. The cloud server then aggregates the uploaded local gradients or model parameters and calculates a new global model. Finally, repeat the above process until the loss function converges or the required number of iterations is reached. However, due to the distributed nature of federated learning, it is vulnerable to Byzantine attacks. An adversary can control one or more clients in the system, tamper with the gradients or model parameters of these clients, and then send the tampered parameters to the server for aggregation, thereby damaging the model or implanting a backdoor. For the classic FedAvg aggregation method, even controlling only one client, Byzantine attacks can easily manipulate the training process, resulting in an inaccurate model after training. To prevent federated learning from Byzantine attacks, robust aggregation methods are needed. Common robust aggregation methods include: RFA aggregation, which uses weighted geometric median as global model parameters; and FOLD aggregation, which removes corrupted clients by comparing the cosine distance between the client and server gradients. To prevent curious servers from accessing users' private information, secure multi-party computation (MPC) techniques are needed to protect user privacy. In the RFA aggregation method described above, secure MPC is required between clients, but this must ensure that the clients are curious but not malicious during the secure computation process. In the FOLD aggregation method, the server must possess a portion of the training data, which is difficult to achieve in many cases. Furthermore, traditional aggregation methods do not provide relevant privacy protection strategies when they are proposed. The embodiments of this invention combine the first server and the second server to calculate the global model parameters, thereby improving the security and reliability of the calculation of global model parameters.

[0101] Example 1

[0102] Reference Figure 1 The diagram illustrates a flowchart of a method for calculating global model parameters provided in Embodiment 1 of the present invention, which may specifically include the following steps:

[0103] Step 101: Construct an initial global parameter model and determine a first loss function for the initial global parameter model;

[0104] Step 102: Send the initial global parameter model to multiple clients to control the clients to train the initial global parameter model based on the first loss function, and generate a first training parameter and a second training parameter that include the local model parameter increment, a second loss function for the local model parameter increment, and the number of data.

[0105] Step 103: Generate a first Paillier algorithm key and send the first Paillier algorithm key and the first training parameters encrypted with the first Paillier algorithm key to the second server, and receive a second Paillier algorithm key generated by the second server; the second server is configured to determine a constant value based on the first Paillier algorithm key, the first training parameters and the second training parameters, and send the constant value to the first server.

[0106] Step 104: Generate target global model parameters and target loss function based on the second Paillier algorithm key and the constant value, and control the client to iterate the target global model parameters based on the target loss function to generate the final global model parameters.

[0107] In practical applications, the embodiments of the present invention can be applied to a first server, which is configured with a corresponding second server. The first server and the second server are not collusive. For example, the first server can be server A and the second server can be server B. Server A and server B will not collude with each other. For example, server A will not send server A's private information to server B, and server B will not send server B's private information to server A.

[0108] In a specific implementation, embodiments of the present invention can construct an initial global parameter model and determine a first loss function for the initial global parameter model; send the initial global parameter model to multiple clients to control the clients to train the initial global parameter model based on the first loss function, generating a first training parameter and a second training parameter that include the local model parameter increment, a second loss function for the local model parameter increment, and the number of data points; generate a first Paillier algorithm key and send the first Paillier algorithm key and the first training parameter encrypted with the first Paillier algorithm key to a second server, and receive a second Paillier algorithm key generated by the second server; the second server is configured to determine a constant value based on the first Paillier algorithm key, the first training parameter, and the second training parameter, and send the constant value to the first server; generate target global model parameters and a target loss function based on the second Paillier algorithm key and the constant value, and control the clients to iterate on the target global model parameters based on the target loss function to generate the final global model parameters. For example, when the first server is server A and the second server is server B, the global parameter model generated by server A can be used as the initial global parameter model, denoted as... The loss function corresponding to the initial global parameter model can be used as the first loss function, denoted as . in, Then, server A can randomly select m clients to participate in this round of training. Each client can use local data to train an initial global parameter model based on the first loss function. For example, in federated learning using the Trimmed-mean aggregation method (a relatively easy aggregation algorithm given by Dong Yin et al.), a model weighted average is used. The client uses the Trimmed-mean aggregation method to calculate the model parameter increment, which is then used as the local model parameter increment. For example, the local model parameter increment of the i-th client can be denoted as... Simultaneously, a second loss function can be determined for the local model parameter increments, denoted as F. i Let n be the number of local data points of the i-th client. iThe local model parameter increments, the second loss function, and the number of data points generated by m clients can be divided into parameters obtained from the first training and parameters obtained from the second training. Server A can generate a Palliier algorithm (a homomorphic encryption algorithm) key 1, where key 1 can include public key 1 and private key 1. For example, server A can generate {p1, q1, N1, g1} and send {N1, g1} to server B. Server A can encrypt the parameters obtained from the first training iteration based on key 1. The encrypted data can be denoted as E1(m). Then, public key 1 and E1(m) can be sent to server B. Simultaneously, server A can receive a key 2 generated by server B, where key 2 can include public key 2 and private key 2. For example, server B generates {p2, q2, N2, g2}, and server A receives {N2, g2} from server B. A second server is configured to determine a constant value, denoted as val, based on public key 1, the parameters obtained from the first training iteration, and the parameters obtained from the second training iteration. Server A can then send val to server A. Server A can use val to calculate the magnitude relationship of the corresponding coordinates of the model parameter increments for multiple clients. For example, when the decrypted val value, denoted as D1(val), is greater than 0, it can be considered... This indicates that the j-th coordinate in the local model parameter increment of the i1-th client is larger. This can be achieved by comparing the j-th coordinates of the local model parameter increments of any two clients and using a sorting algorithm to adjust the order of the j-th coordinates of the local model parameter increments of the m clients according to their size. In the federated learning of the Trimmed-means aggregation method, assuming the number of clients is m0 and the number of clients selected in each training round is m, where the proportion of corrupted sides β < 1 / 2, and the number of corrupted sides cannot exceed half the number of selected clients, it is necessary to remove the top-ranked clients before aggregation. The smallest number, and the last The largest number, therefore, is eliminated. The smallest number and After obtaining the largest data point, servers A and B continue to perform calculations on their respective data. Server B sends its calculation result to server A, which then calculates the sum of the two results to obtain the j-th coordinate of the new round of model parameter increments, which can be denoted as... Other coordinates are calculated in the same way, thus obtaining... The target global model parameters are used as the target global model parameters, and the target loss function corresponding to the target global model parameters is determined, denoted as F. Then, the target global model parameters can be used as the initial global parameter model for a new round of training, and the target global model parameters and target loss function for the next round are obtained according to the above process until the target loss function converges. The result is then used as the final global model parameters. Optionally, the number of iterations can be set. When the number of iterations is reached, the iteration stops and the final global model parameters are obtained.

[0109] In this embodiment of the invention, an initial global parameter model is constructed, and a first loss function is determined for the initial global parameter model. The initial global parameter model is sent to multiple clients to control the clients to train the initial global parameter model based on the first loss function, generating a first training parameter and a second training parameter that include the local model parameter increment, a second loss function for the local model parameter increment, and the number of data points. A first Paillier algorithm key is generated, and the first Paillier algorithm key and the first training parameter encrypted with the first Paillier algorithm key are sent to a second server, which receives a second Paillier algorithm key generated by the second server. The second server is configured to determine a constant value based on the first Paillier algorithm key, the first training parameter, and the second training parameter, and send the constant value to the first server. Based on the second Paillier algorithm key and the constant value, target global model parameters and a target loss function are generated, and the clients are controlled to iterate on the target global model parameters based on the target loss function to generate the final global model parameters. This improves the accuracy and reliability of the calculation of global model parameters and further ensures data security.

[0110] Based on the above embodiments, modified embodiments of the above embodiments are proposed. It should be noted that, in order to keep the description brief, only the differences from the above embodiments are described in the modified embodiments.

[0111] In an optional embodiment of the present invention, the step of controlling the client to train the initial global parameter model based on the first loss function, generating a first training parameter and a second training parameter that include the local model parameter increment, a second loss function for the local model parameter increment, and the number of data points includes:

[0112] The client is controlled to train the initial global parameter model based on the first loss function to obtain the local model parameter increment, the second loss function for the local model parameter increment, and the number of data.

[0113] Determine a first weight parameter and a second weight parameter for the local model parameter increment, the second loss function, and the number of data points;

[0114] Based on the first weight parameter and the second weight parameter, the local model parameter increment is divided to generate a first local model parameter increment secret shared value and a second local model parameter increment secret shared value.

[0115] The second loss function is divided based on the first weight parameter and the second weight parameter to generate the secret shared value of the first partitioning loss function and the secret shared value of the second partitioning loss function;

[0116] The number of data items is divided based on the first weight parameter and the second weight parameter to generate a first secret sharing value for the number of data items and a second secret sharing value for the number of data items.

[0117] The first local model parameter increment secret shared value, the first partition loss function secret shared value, and the first data number secret shared value are determined as the parameters obtained from the first training.

[0118] The second local model parameter increment secret shared value, the second partition loss function secret shared value, and the second data number secret shared value are determined as the second training parameters.

[0119] Optionally, the step of dividing the local model parameter increment based on the first weight parameter and the second weight parameter to generate a first local model parameter increment secret shared value and a second local model parameter increment secret shared value includes:

[0120]

[0121] The step of dividing the second loss function based on the first weight parameter and the second weight parameter to generate the first and second partitioning loss function secret shared values ​​includes:

[0122] F i =Δ x F i +Δ y F i ,

[0123] The step of dividing the number of data points based on the first weight parameter and the second weight parameter to generate a first data point secret sharing value and a second data point secret sharing value includes:

[0124] n i =Δ x n i +Δ y ni ,

[0125] Among them, w i，d This corresponds to the d-th coordinate size of the local model parameter increment for the i-th client. The increment of the local model parameters, Δ x Let Δy be the first weight parameter and Δy be the second weight parameter. This is the incremental secret shared value of the first local model parameters. F is the incremental secret shared value of the second local model parameters. i Let Δ be the second loss function. x F i Let Δ be the secret shared value of the first partitioning loss function. y F i Let n be the secret shared value of the second partitioning loss function. i Δ is the number of data points. x n i The secret shared value for the first number of data items, Δ y n i The second data number is a secret shared value.

[0126] In a specific implementation, embodiments of the present invention can control the client to train an initial global parameter model based on a first loss function to obtain the local model parameter increment, a second loss function for the local model parameter increment, and the number of data points; determine a first weight parameter and a second weight parameter for the local model parameter increment, the second loss function, and the number of data points; divide the local model parameter increment based on the first weight parameter and the second weight parameter to generate a first local model parameter increment secret shared value and a second local model parameter increment secret shared value; divide the second loss function based on the first weight parameter and the second weight parameter to generate a first partitioning loss function secret shared value and a second partitioning loss function secret shared value; divide the number of data points based on the first weight parameter and the second weight parameter to generate a first number of data points secret shared value and a second number of data points secret shared value; determine the first local model parameter increment secret shared value, the first partitioning loss function secret shared value, and the first number of data points secret shared value as the first training parameters; determine the second local model parameter increment secret shared value, the second partitioning loss function secret shared value, and the second number of data points secret shared value as the second training parameters. For example, when the first server is server A, the second server is server B, and the initial global parameter model is... The first loss function is Server A can select m clients to participate in this round of training. Based on the first loss function, the initial global parameter model is trained T times to obtain the trained local parameter model. The local model parameter increment, the second loss function for the local model parameter increment, and the number of data points are calculated. Then, the first weight parameter and the second weight parameter for the local model parameter increment, the second loss function, and the number of data points can be determined. The first weight parameter can be denoted as Δ. x The second weighting parameter can be denoted as Δ. y It can be calculated according to the following formula 1 based on the first weight parameter Δ x The second weighting parameter Δ y Divide the local model parameter increments:

[0127] Formula 1:

[0128]

[0129] The second loss function can be partitioned based on the first and second weight parameters according to the following formula 2, generating the secret shared value of the first partitioning loss function and the secret shared value of the second partitioning loss function:

[0130] Formula 2:

[0131] F i =Δ x F i +Δ y F i

[0132] The number of data points can be divided based on the first and second weight parameters according to Formula 3, generating a first data point secret sharing value and a second data point secret sharing value.

[0133] Formula 3:

[0134] n i =Δ x n i +Δ y n i

[0135] Among them, w i，d This corresponds to the d-th coordinate size of the local model parameter increment for the i-th client. The increment of the local model parameters, Δ x Let Δ be the first weight parameter. y This is the second weight parameter. Let Δ be the incremental secret shared value of the first local model parameters. x w i，1 , ..., Δ x w i，d , The second local model parameter increment secret shared value, expanded and denoted as Δ y w i，1 , ..., Δ y w i，d F i Let Δ be the second loss function. x F i Let Δ be the secret shared value of the first partitioning loss function. y F i Let n be the secret shared value of the second partitioning loss function. i Δ is the number of data points. x n i The secret shared value for the first number of data items, Δ y n i The second data quantity is a secret shared value. Then, the first local model parameter increment secret shared value can be... First partitioning loss function secret shared value Δ x F i Secretly share value Δ with the first data number x n i As the parameters obtained from the first training, the incremental values ​​of the second local model parameters can be secretly shared. The second partitioning loss function secret shared value Δ y F i And the second data number secret shared value Δ y n i The parameters obtained from the second training are determined. Preferably, when sending the parameters obtained from the first training to the first server and / or sending the parameters obtained from the second training to the second server, the transmission can be private, and any encryption algorithm can be selected, such as AES (Advanced Encryption Standard), 3DES (Triple Data Encryption Algorithm), etc., to encrypt the data.

[0136] Optionally, in extreme cases (where an attacker can simultaneously replay attacks on most legitimate clients), to prevent replay attacks (intercepting client-sent information, modifying it, and then sending it to the server), the following approach can be adopted: The client creates a key for an asymmetric encryption algorithm (e.g., RSA, Elgamal, Pallier, etc.) and sends the public key to the server. Before sending information, the client creates a hash value for the information to be sent and uses this hash value along with its private key as input to a digital signature algorithm to generate a digital signature. Then, after encrypting the information using AES or 3DES, the digital signature is appended and sent to the server. Upon receiving the information and signature, the server first decrypts the information and then uses the hash value and the client's public key as input to a digital signature verification algorithm to verify whether the information was sent by a specific client. This method not only ensures user data privacy but also effectively defends against replay attacks.

[0137] In this embodiment of the invention, by determining a first weight parameter and a second weight parameter for the local model parameter increment, the second loss function, and the number of data points; dividing the local model parameter increment based on the first weight parameter and the second weight parameter to generate a first secret shared value for the local model parameter increment and a second secret shared value for the local model parameter increment; dividing the second loss function based on the first weight parameter and the second weight parameter to generate a first secret shared value for the partitioning loss function and a second secret shared value for the partitioning loss function; dividing the number of data points based on the first weight parameter and the second weight parameter to generate a first secret shared value for the number of data points and a second secret shared value for the number of data points; determining the first secret shared value for the local model parameter increment, the first secret shared value for the partitioning loss function, and the first secret shared value for the number of data points as first trained parameters; and determining the second secret shared value for the local model parameter increment, the second secret shared value for the partitioning loss function, and the second secret shared value for the number of data points as second trained parameters, the data is partitioned, providing data support for subsequent computation using different servers.

[0138] In an optional embodiment of the present invention, the step of generating a first Paillier algorithm key and sending the first Paillier algorithm key and the first training parameters encrypted with the first Paillier algorithm key to the second server includes:

[0139] Calculate the parameter difference obtained from the first training based on the incremental secret shared value of the first local model parameters;

[0140] The parameter difference obtained from the first training is encrypted using the first Paillier algorithm key;

[0141] The encrypted parameter difference obtained from the first training is sent to the second server.

[0142] Optionally, the step of calculating the first training parameter difference based on the first local model parameter increment secret shared value, and encrypting the first training parameter difference using the first Paillier algorithm key, includes:

[0143] Formula 4:

[0144]

[0145] Where E1(m) represents the encryption of m using the first Paillier algorithm key. The difference between the parameters obtained from the first training is... The j-th coordinate size corresponds to the increment of the local model parameters of the i1-th client. The j-th coordinate size corresponds to the local model parameter increment of the i2-th client.

[0146] In practical applications, to compare the magnitude of the j-th coordinate of the local model parameter increment of two clients, such as the i1-th and i2-th clients, we can set the j-th coordinate of the i1-th client as... Let the coordinates of the i2th client be j-th coordinates.

[0147] In practical applications, the first Paillier algorithm key and the second Paillier algorithm key in this embodiment of the invention may include a public key and a private key. The public key can be used for encryption, and the private key can be used for decryption.

[0148] In a specific implementation, embodiments of the present invention can calculate the first training parameter difference based on the first local model parameter increment secret shared value; encrypt the first training parameter difference using a first Paillier algorithm key; and send the encrypted first training parameter difference to a second server. For example, when the first local model parameter increment secret shared value is Δ... x w i，1 , ..., Δ x w i，d When the first Paillier algorithm key is E1(m), then the j-th coordinate of the i1-th client can be set to... Let the coordinates of the i2th client be j-th coordinates. Will As the parameter difference obtained from the first training, the parameter difference obtained from the first training is encrypted using the first Paillier algorithm key, resulting in... Then, you can Send to the second server.

[0149] In this embodiment of the invention, a first training parameter difference is calculated based on the first local model parameter increment secret shared value; the first training parameter difference is encrypted using a first Paillier algorithm key; and the encrypted first training parameter difference is sent to the second server, thereby further ensuring data information security.

[0150] In an optional embodiment of the present invention, the second server is configured to perform the step of determining the constant value based on the first Paillier algorithm key, the first trained parameters, and the second trained parameters in the following manner:

[0151] The parameter difference obtained from the second training is calculated based on the incremental secret shared value of the second local model parameters;

[0152] Determine the first and second random numbers;

[0153] The constant value is determined based on the first random number, the second random number, the difference between the parameters obtained from the first training, and the difference between the parameters obtained from the second training.

[0154] Optionally, the step of determining the constant value based on the first random number, the second random number, the difference between the first trained parameters, and the difference between the second trained parameters includes:

[0155]

[0156] Where val is the constant value, r1 is the first random number, and r2 is the second random number.

[0157]

[0158] The difference between the parameters obtained from the second training is denoted by mod.

[0159]

[0160] Divide by The remainder, where N is a constant.

[0161] In practical applications, lcm(a, b) represents the least common multiple of a and b, and gcd(a, b) represents the greatest common divisor of a and b. If gcd(a, b) = 1, a and b are said to be coprime. Let {0, 1, ..., N-1} be the set. Represents a set The set of elements coprime to N. a mod c is read as a modulo c, representing the remainder when a is divided by c. a = b mod c means that a and b have the same remainder when divided by c. If ab = 1 mod c, it means that b is the multiplicative inverse of a modulo c, denoted as b = a. -1 mod c, where a is also the multiplicative inverse of b modulo c. If For set S N ={u|u=1modn}, define S N The function L is shown in Formula 5 below:

[0162] Formula 5:

[0163]

[0164] Represents the space of real numbers. Let x represent an n-dimensional Euclidean space. If x is a real number, This indicates approximate rounding, which is the integer obtained by rounding according to the rounding method.

[0165] In a specific implementation, embodiments of the present invention can calculate the second training parameter difference based on the second local model parameter increment secret shared value; determine the first random number and the second random number; and determine a constant value based on the first random number, the second random number, the first training parameter difference, and the second training parameter difference. For example, when the second server is server B, the second local model parameter increment secret shared value is Δ. y w i，1 , ..., Δ y w i，d The difference in parameters obtained from the second training can be calculated using the following formula 6:

[0166] Formula 6:

[0167]

[0168] Among them, the j-th coordinate of the i1-th client received by server B is The coordinates of the i2th client received by server B are:

[0169] Server B selects two random numbers r1 and r2, which are used as the first and second random numbers respectively, where r1 > r2 > 0. Then, a constant value, denoted as val, can be determined based on the first random number, the second random number, the difference between the parameters obtained from the first training, and the difference between the parameters obtained from the second training, according to the following formula 7:

[0170] Formula 7:

[0171]

[0172] Where mod is Divide by The remainder, where N is a constant. This indicates the use of the first Paillier algorithm key pair. The encryption result, This indicates the use of the first Paillier algorithm key pair. The encryption result is used to calculate the val value. Preferably, server B can send the val value to the first server.

[0173] In this embodiment of the invention, the difference between the second trained parameters is calculated based on the secret shared value of the second local model parameter increment; a first random number and a second random number are determined; and a constant value is determined based on the first random number, the second random number, the difference between the first trained parameters, and the difference between the second trained parameters, thereby determining a constant value and providing data support for subsequent sorting of the local model parameter increments.

[0174] In an optional embodiment of the present invention, the step of generating target global model parameters and a target loss function based on the second Paillier algorithm key and the constant value, and controlling the client to iterate the target global model parameters based on the target loss function to generate the final global model parameters includes:

[0175] The maximum and minimum values ​​are determined from the incremental secret shared value of the first local model parameters based on the constant values;

[0176] The maximum and minimum values ​​are removed from the local model parameter increments to generate a client index set;

[0177] A first intermediate value is generated based on the client index set, the second local model parameter incremental secret shared value, and the second Paillier algorithm key, and sent to the second server. The second server is configured to generate a second intermediate value based on the client index set, the first local model parameter incremental secret shared value, and the first Paillier algorithm key, and the second server sends the second intermediate value to the first server.

[0178] The third intermediate value is determined based on the second intermediate value;

[0179] The second server is configured to determine a fourth intermediate value based on the first intermediate value; the second server sends the fourth intermediate value to the first server;

[0180] The index set values ​​are determined based on the third intermediate value and the fourth intermediate value;

[0181] Target global model parameters are generated based on the numerical values ​​of the index set;

[0182] The target loss function is determined based on the target global model parameters;

[0183] The target global model parameters are iterated based on the target loss function to generate the final global model parameters.

[0184] Optionally, the step of generating a first intermediate value based on the client index set, the second local model parameter incremental secret shared value, and the second Paillier algorithm key includes:

[0185]

[0186] The second server is configured to perform the step of generating a second intermediate value based on the client index set, the first local model parameter incremental secret shared value, and the first Paillier algorithm key in the following manner:

[0187]

[0188] The step of determining the third intermediate value based on the second intermediate value includes:

[0189]

[0190] The second server is configured to perform the step of determining a fourth intermediate value based on the first intermediate value in the following manner:

[0191]

[0192] The step of determining the index set value based on the third intermediate value and the fourth intermediate value includes:

[0193]

[0194] The step of generating target global model parameters based on the index set values ​​includes:

[0195]

[0196]

[0197] The step of determining the target loss function based on the target global model parameters includes:

[0198]

[0199] Wherein, INDS is the client index set, Tmp1 is the first intermediate value, Tmp2 is the second intermediate value, Tmp3 is the third intermediate value, Tmp4 is the fourth intermediate value, E2(m) is the encryption of m using the first Paillier algorithm key, and Δ y w i，j This is the incremental secret shared value of the second local model parameters. represents the target global model parameters; F represents the target loss function.

[0200] In a specific implementation, embodiments of the present invention can determine the maximum and minimum values ​​from the first local model parameter increment secret shared value based on a constant value; remove the maximum and minimum values ​​from the local model parameter increment to generate a client index set; generate a first intermediate value based on the client index set, the second local model parameter increment secret shared value, and the second Paillier algorithm key, and send it to the second server. The second server is configured to generate a Tmp2 value based on the client index set, the first local model parameter increment secret shared value, and the first Paillier algorithm key, and send the Tmp2 value to the first server; determine the Tmp3 value based on the Tmp2 value; determine the Tmp4 value based on the first intermediate value; send the Tmp4 value to the first server; determine the index set value based on the Tmp3 and Tmp4 values; generate target global model parameters based on the index set value; determine the target loss function based on the target global model parameters; iterate the target global model parameters based on the target loss function to generate the final global model parameters. For example, when the first server is server A, the second server is server B, the constant value is val, and the first local model parameter increment secret shared value is Δ... x w i，1 , ..., Δ x w i，d The first Paillier algorithm key is E1(m), and the decryption result of the first Paillier algorithm key is D1(m). The second Paillier algorithm key is E2(m), and the decryption result of the second Paillier algorithm key is D2(m). The coordinates of the j-th location of the i-th client are... Let the coordinates of the i2th client be j-th coordinates. Then, the maximum and minimum values ​​can be determined from the incremental secret shared values ​​of the first local model parameters based on val. For example, server A can decrypt val to obtain D1(val). When D1(val) > 0, it can be considered that... This indicates that the j-th coordinate of the i1-th client model parameter increment is larger, and vice versa. That is, if the j-th coordinate of the model parameter increment of the i2-th client is larger, by comparing the size of the j-th coordinate in any two clients, a sorting algorithm can be used to sort the j-th coordinate of the secret shared value of the first local model parameter increment of the m clients, thus determining... The smallest number and The largest data points are taken as the maximum and minimum values, respectively. The maximum and minimum values ​​can be removed from the local model parameter increment. The client index set corresponding to the remaining data is taken as the client index set, denoted as INDS. If the i-th client data is not deleted, then i∈INDS.

[0201] Preferably, the server A determines Know After determining the size relationship, the size relationship can be sent to server B. Since server B calculates based on the first Paillier algorithm key... The size, that is Therefore, server B can... and The size relationship is verified. At the same time, since server A cannot know the values ​​of the first random number r1 and the second random number r2, it cannot infer the information in server B. And since server B only knows the ciphertext of the data in server A, it cannot infer the information in server A, so as to protect data privacy.

[0202] Once the client index set INDS is determined, server A can secretly share the first local model parameter increment value Δ. x w i，j Encrypted using the first Paillier algorithm key as E1(Δ) x w i，j The data is sent to server B, and server B can then secretly share the second local model parameter increment value Δ. y w i，j Encrypted using the second Paillier algorithm key as E2(Δ) y w i，j ), and send it to server A.

[0203] Server A can then calculate the first intermediate value according to the following formula 8, and denot it as Tmp1:

[0204] Formula 8:

[0205]

[0206] Then, server A can send Tmp1 to server B, and server B can calculate the second intermediate value according to the following formula 9, denoted as Tmp2, and send it to server A:

[0207] Formula 9:

[0208]

[0209] Server A can decrypt the second intermediate value to obtain D1(Tmp2), and then calculate the third intermediate value, denoted as Tmp3, according to the following formula 10:

[0210] Formula 10:

[0211]

[0212] Server B can decrypt Tmp1 to obtain D2(Tmp1), and calculate the fourth intermediate value according to the following formula 11, denoted as Tmp4, and send it to server A:

[0213] Formula 11:

[0214]

[0215] Then, the index set value can be calculated and determined based on the third and fourth intermediate values ​​according to the following formula 12, denoted as ∑. i∈INDS (Δw i，j ·n i ):

[0216] Formula 12:

[0217]

[0218] The target global model parameters can be calculated using the index set numerical values ​​according to Formula 13, denoted as follows. And send it to each client:

[0219] Formula 13:

[0220]

[0221]

[0222] Then, it can be based on Δ x w i，j Δ y w i，j Replace with Δ x F i Δ y F i Then the target loss function can be obtained, as shown in Formula 14 below:

[0223] Formula 14:

[0224]

[0225] Where INDS is the client index set, Tmp1 is the first intermediate value, Tmp2 is the second intermediate value, Tmp3 is the third intermediate value, Tmp4 is the fourth intermediate value, E2(m) is the encryption of m using the first Paillier algorithm key, and Δ y w i，j This is the secret shared value for the incremental parameters of the second local model. represents the target global model parameters; F represents the target loss function.

[0226] It can be based on the target global model parameters The target loss function F is iteratively calculated, and the target global model parameters are used as the initial global parameter model for training in the next round. The target global model parameters and target loss function are obtained in the next round according to the above process until the target loss function converges. The result is then used as the final global model parameters. Optionally, the number of iterations can be set. When the number of iterations is reached, the iteration stops and the final global model parameters are obtained.

[0227] In this embodiment of the invention, the maximum and minimum values ​​are determined from the first local model parameter increment secret shared value based on the constant value; the maximum and minimum values ​​are removed from the local model parameter increment to generate a client index set; a first intermediate value is generated based on the client index set, the second local model parameter increment secret shared value, and the second Paillier algorithm key, and sent to the second server; the second server is configured to generate a second intermediate value based on the client index set, the first local model parameter increment secret shared value, and the first Paillier algorithm key, and the second server sends the second intermediate value to the first server; a third intermediate value is determined based on the second intermediate value; the second server is configured to determine a fourth intermediate value based on the first intermediate value; the second server sends the fourth intermediate value to the first server; an index set value is determined based on the third and fourth intermediate values; target global model parameters are generated based on the index set value; a target loss function is determined based on the target global model parameters; and the target global model parameters are iterated based on the target loss function to generate the final global model parameters, thereby realizing the calculation of global model parameters and further improving the reliability and security of the calculation results.

[0228] To enable those skilled in the art to better understand the embodiments of the present invention, a complete example is provided below to illustrate the embodiments of the present invention.

[0229] refer to Figure 2 , Figure 2A flowchart illustrating the steps of another method for calculating global model parameters provided in Embodiment 1 of the present invention is shown.

[0230] The specific process is as follows:

[0231] Symbolic representation:

[0232] lcm(a, b) represents the least common multiple of a and b, and gcd(a, b) represents the greatest common divisor of a and b. If gcd(a, b) = 1, a and b are said to be coprime. Let {0, 1, ..., N-1} be the set. Represents a set The set of elements coprime to N. a mod c is read as a modulo c, representing the remainder when a is divided by c. a = b mod c means that a and b have the same remainder when divided by c. If ab = 1 mod c, it means that b is the multiplicative inverse of a modulo c, denoted as b = a. -1 mod c, where a is also the multiplicative inverse of b modulo c. If For set S N ={u|u=1modn}, define S N The function L on is

[0233]

[0234] Represents the space of real numbers. Let x represent an n-dimensional Euclidean space. If x is a real number, This indicates approximate rounding, which is the integer obtained by rounding according to the rounding method.

[0235] In practical applications, secret sharing is an important technique in secure multi-party computation. Due to its relative simplicity, it is widely used in the field of privacy protection. In additive secret sharing, data x is randomly split into the sum of two data points (x0, x1), i.e., x = x0 + x1. Then, x0 and x1 are stored in the hands of participants P0 and P1, respectively. Data x can only be recovered by combining the data from participants P0 and P1.

[0236] Let's assume participant P0 possesses data x, and another participant P1 possesses data y. Following the process below, we can achieve secret sharing in addition.

[0237] 1. P0 generates a random number x0 and calculates x1 = x - x0, then sends x1 to P1.

[0238] 2. P1 generates a random number y0 and calculates y1 = y - y0, then sends y0 to P0.

[0239] 3. P0 calculates z0 = x0 + y0, and P1 calculates z1 = x1 + y1.

[0240] To calculate the sum of x and y, simply sum z0 and z1 to the requester, resulting in z0 + z1. This prevents the leakage of data x and y, protecting data privacy.

[0241] Homomorphic encryption is a fundamental technology for secure multi-party computation. It allows operations to be performed directly on the ciphertext, and the decrypted result is equivalent to the operation performed directly on the plaintext.

[0242] The Paillier algorithm (a public-key encryption algorithm) is as follows:

[0243] Key generation: Choose two large prime numbers p and q, let N = pq and λ = 1 / cm(p-1, q-1), which satisfy gcd(λ, N) = 1. Choose g such that...

[0244] gcd(L(g λ modN 2 ), N)=1,

[0245] We will use (N, g) as the public key and λ as the private key.

[0246] Encryption: for any Select random number The ciphertext is

[0247] c = gmr N modN 2 .

[0248]

[0249] If we use E(m) to represent the encryption of plaintext m and D(c) to represent the decryption of ciphertext c, then the above algorithm is homomorphic, meaning it is homomorphic for any... have

[0250] D(E(m1)E(m2))=m1+m2modN,

[0251] if Therefore, D(E(m1)E(m2)) = m1 + m2, which shows that when multiple data are added together, that is, if There are D(E(m1)…E(m) k ))=m1+…+m k This satisfies semantic safety.

[0252] Choose a sufficiently large N such that the absolute value of the sum of the data does not exceed N / 2. Then, add N to any data less than 0, converting all data into natural numbers, and directly encrypt them using the Paillier algorithm. After decryption using the Paillier algorithm, if the data is greater than N / 2, subtract N from it; if the data is less than N / 2, do not process it further, thus obtaining the final decrypted result. Based on the additive homomorphism of the Paillier algorithm, it is easy to verify that the algorithm generalized to the integer range still satisfies the additive homomorphism.

[0253] Meanwhile, in federated learning using the Trimmed-mean aggregation method, gradient-weighted averaging is employed. Assuming the number of clients is m0, and m clients are selected in each training round, with the proportion of corrupted clients β < 1 / 2 (meaning the number of corrupted clients cannot exceed half the number of selected clients), the previous clients need to be removed before aggregation. The smallest number, and the last The largest number is used to aggregate the remaining numbers, that is:

[0254] The server generates a global parameter model, initial values ​​for the global model parameters, and a loss function, and sends them to the client.

[0255] Let the initial values ​​of the global model parameters be... The loss function is

[0256] The server randomly selects m clients to participate in this round of training. Each client uses local data for training, obtains local gradients, calculates local loss function values, and sends the local gradients, local loss function values, and the number of data points to the server.

[0257] Let the gradient be calculated by the i-th client. Local loss function value F i The number of local data items n i And send them to the server.

[0258] The server sorts the gradient parameters and removes... The smallest number and The largest number is used to aggregate the remaining data to obtain a new set of global model parameters, which are then sent to the client. The global loss function value is also calculated.

[0259] Let's take the j-th coordinate of the gradient as an example. The j-th component g of the i-th client gradient... i，j For all customers and remove The smallest number and The largest number is sorted by the j-th component of the gradient, which is equivalent to sorting g. 1，j , ..., g m，jSort the data. Assume that removing... The smallest number and After determining the largest number, the set of client indices corresponding to the remaining numbers is called INDS. That is, if the i-th client data has not been deleted, then i ∈ INDS. The server then updates the j-th coordinate of the global model as follows:

[0260]

[0261] Where η is the step size chosen by the stochastic gradient descent method. Other coordinates can be calculated in the same way, thus obtaining...

[0262]

[0263] And the global loss function value is calculated.

[0264]

[0265] Repeat the calculation until the required number of iterations is reached or the loss function converges.

[0266] Compared to gradient-weighted averaging, model parameter-weighted averaging can effectively reduce the number of communication rounds, thus achieving higher efficiency. When using federated learning with model-weighted averaging via the Trimmed-mean aggregation method, the following can be obtained:

[0267] The server generates a global parameter model, initial values ​​for the global model parameters, and a loss function, and sends them to the client.

[0268] Let the initial values ​​of the global model parameters be... The loss function is

[0269] The server selects m clients to participate in this round of training. Each client uses local data for training, performs T training iterations using stochastic gradient descent, obtains the trained local model parameters, calculates the local model parameter increments and the local loss function value, and sends the local model parameter increments, local loss function values, and the number of data points to the server.

[0270] The local model parameter increment equals the local model parameters after this round of training minus the global model parameters. Let the local model parameter increment calculated by the i-th client be... Local loss function value F i The number of local data items n i And send them to the server.

[0271] The server incrementally sorts the model parameters and removes... The smallest number and The largest number is used to aggregate the remaining numbers to obtain the new set of global model parameters. The global loss function value is then calculated and sent to the client.

[0272] Let's take the j-th coordinate of the model parameter increment as an example. The j-th component Δw of the i-th client-side model parameter increment... i，j Sort the j-th component of the parameter increments of these client models, that is, sort Δw 1，j ,…,Δw m，j Sort the data. Then remove the sorted data. The smallest number and Let the largest number be the set of client indices corresponding to the remaining numbers, and let INDS be the set of indices. That is, if the i-th client data has not been deleted, then i ∈ INDS. The server then updates the j-th coordinate of the global parameter model as follows:

[0273]

[0274] Other coordinates can be calculated in the same way, and then...

[0275]

[0276] And the global loss function value is calculated.

[0277]

[0278] Repeat the calculation until the required number of iterations is reached or the loss function converges.

[0279] When there are two servers that do not collude with each other, to protect user privacy, in each round of federated learning, the client can use additive secret sharing to divide the private data into two parts, encrypt them separately and send them to the two servers, and then use secure multi-party computation between the two servers to obtain the global parameter model. The process is as follows:

[0280] When two servers that do not collude with each other are server A and server B.

[0281] Server A generates Pallier algorithm key 1 and sends public key 1 to server B. Server B generates Pallier algorithm key 2 and sends public key 2 to server A.

[0282] Server A generates {p1, q1, N1, g1} and sends {N1, g1} to server B. Server B generates {p2, q2, N2, g2} and sends {N2, g2} to server A. Typically, the chosen values ​​for p1, q1, p2, and q2 are very large (e.g., 1024-bit numbers), therefore the following data will generally satisfy the condition that the sum of the values ​​is within a certain range. The requirements are as follows. In the following process, we use E1(m) and D1(c) to represent the encryption of m with public key 1 and the decryption of c with key 1, respectively, and we use E2(m) and D2(c) to represent the encryption of m with public key 2 and the decryption of c with key 2, respectively. Of course, the encryption and decryption here refer to encryption and decryption of integers.

[0283] Server A generates a global parameter model, initial values ​​for the global model parameters, and a loss function, and sends them to the client.

[0284] Let the initial values ​​of the global model parameters be... The loss function is

[0285] Server A selects m clients to participate in this round of training. Each client uses local data for training, performs T training iterations using stochastic gradient descent, obtains the local model parameters after training, calculates the local model parameter increment and the local loss function value, and these clients use addition to secretly share the sum of the local model parameter increment, the local loss function, and the number of data points into two parts, and then secretly send them to servers A and B respectively.

[0286] The local model parameter increment equals the local model parameters after this round of training minus the global model parameters. Let the local model parameter increment calculated by the i-th client be... Local loss function value F i The number of local data items n i Obtained through secret sharing

[0287]

[0288] F i =Δ x F i +Δ y F i ,

[0289] n i =Δ x n i +Δ y n i ,

[0290] Then the first part of the divided data, that is Δ x F i Δ x n i The second part of the divided data is sent privately to server A. Δ y F i Δy n i Send the data privately to server B. For private sending, you can choose any encryption algorithm, such as AES, 3DES, etc.

[0291] This achieves privacy protection for the user's model parameters, as well as for the number of client-side data points and the local loss function value. Preferably, when a high level of privacy protection is not required, the processing of the number of client-side data points and the local loss function value can be omitted.

[0292] Servers A and B use secure multi-party computation to incrementally sort the model parameters and remove... The smallest data and The largest dataset is used to aggregate the remaining data using secure multi-party computation to obtain a new set of global model parameters and global loss function values. Server A then sends the global model parameters to the client.

[0293] Preferably, for ease of calculation, decimals (including Δ) are usually decimalized. x w i，j Δ x w i，j Δ x F i Δ y F i First, multiply the data by a factor S to convert it to integers before performing calculations (data is usually processed only within a certain precision range, so multiplying by S converts the data to integers). After the calculations are complete, divide the global model parameters and the global loss function value by S again. For ease of explanation, the same notation will be used for the data after multiplying by S as for the data before multiplication.

[0294] Then, compare the j-th coordinate of the model parameter increments for the two clients (e.g., the i1-th and i2-th clients). Server A possesses the data. Server B has data

[0295] First, server A will Encryption obtained And then send it to server B.

[0296] Secondly, server B selects two random numbers r1>r2>0, and then calculates...

[0297]

[0298] That is, calculate the following two numbers modulo... Multiplication in the sense of the first number: the first number is a product of the first number. The encrypted result, the second number is the data sent to server A. Find r to the power of 1. Then server B sends the value val to server A.

[0299] Next, server A decrypts val, which gives D1(val). If D1(val) > 0, then... That is, the j-th coordinate of the i1-th client model parameter increment is larger; otherwise That is, the j-th coordinate of the i2-th client model parameter increment is larger. Finally, server A sends the relationship between the two coordinates to server B.

[0300] In fact, in step b, server B performed the computation on the ciphertext. The magnitudes of r1 and r2 are readily verifiable, thus confirming the correct conclusion regarding their relationship in step c. Since server A does not know the values ​​of r1 and r2, it cannot deduce any information about the data in server B; and since server B only knows the ciphertext of the data in server A, it cannot deduce any information about the data in server A. Therefore, the above process effectively protects user privacy.

[0301] Using the above method, we can compare the size of the j-th coordinate of any two client-side model parameter increments. We can then use a sorting algorithm to sort the j-th coordinates of m client-side model parameter increments (i.e., the process of repeatedly adjusting the data order based on data size). Assuming sorting and removing... The smallest number and After the largest data point, the set of client indices corresponding to the remaining data is called INDS. That is, if the i-th client data point has not been deleted, then i ∈ INDS.

[0302]

[0303] Where, ∑ i∈INDS n i The value can be obtained by calculating ∑ on server A. i∈INDS Δ x n i Server B calculates ∑ i∈INDS Δ y n i And send it to server A, then server A calculates the sum of the two. For ∑ i∈INDS (Δw i，j ·n i The calculation of ) can be done in the following way:

[0304] First, server A will Δ x w i，j Encryption yields E1(Δ) x wi，j ), and then send it to server B. Server B will Δ y w i，j Encryption yields E2(Δ) y w i，j Then send it to server A. Where i∈INDS.

[0305] Secondly, server A calculates And send it to server B, server B calculates... And send it to server A.

[0306] Next, server A decrypts Tmp2 to obtain D1(Tmp2) and calculates...

[0307]

[0308] Server B decrypts Tmp1 to obtain D2(Tmp1), and calculates...

[0309]

[0310] And send it to server A.

[0311] Then, server A can calculate ∑ i∈INDS (Δw i，j ·n i ) = Tmp3 + Tmp4.

[0312] Therefore, server A can calculate the j-th coordinate of the new round of model parameter increments.

[0313]

[0314] Other coordinates can be calculated in the same way, thus obtaining...

[0315]

[0316] The new round of model parameters Send to each client.

[0317] And in the above process Δ x w i，j Δ y w i，j Replace with Δ x F i Δ y F i The global loss function value can be calculated.

[0318]

[0319] The calculation results can be used as the initial global parameter model for training in the next round, and the target global model parameters and target loss function can be obtained in the next round according to the above process until the target loss function converges, and the result can be used as the final global model parameters. Optionally, the number of iterations can be set, and the iteration can be stopped when the required number of iterations is reached to obtain the final global model parameters.

[0320] In federated learning, the server first builds a global parameter model and sends it to the client. After the client trains and obtains the incremental parameters of its local model, it divides this into two parts using secret sharing, and then secretly sends each part to the two servers. The servers then use multi-party secure computation to remove the larger and smaller client data, and use the remaining data to calculate the new global model parameters. These parameters are then sent to the client for the next round of computation, until the required number of iterations is reached or the overall loss function converges. This effectively protects the client's privacy and satisfies robustness requirements.

[0321] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0322] Example 2

[0323] Reference Figure 3 The diagram illustrates a flowchart of a method for calculating global model parameters according to Embodiment 2 of the present invention, which may specifically include the following steps:

[0324] Step 301: Determine the constant value based on the first Paillier algorithm key, the first training parameters, and the second training parameters;

[0325] Step 302: Send the constant value to the first server.

[0326] In practical applications, embodiments of the present invention can be applied to a second server, which is configured with a corresponding first server. The first server and the second server are not collusive. The first server is configured to construct an initial global parameter model and determine a first loss function for the initial global parameter model; send the initial global parameter model to multiple clients to control the clients to train the initial global parameter model based on the first loss function, generate a first training parameter and a second training parameter that include the local model parameter increment, a second loss function for the local model parameter increment, and the number of data points; generate a first Paillier algorithm key, and send the first Paillier algorithm key and the first training parameter encrypted with the first Paillier algorithm key to the second server, and receive a second Paillier algorithm key generated by the second server.

[0327] As for Embodiment 2, since it is basically similar to Embodiment 1, the description is relatively simple. For relevant details, please refer to the description of the method embodiment.

[0328] Example 3

[0329] Reference Figure 4 The diagram shows a structural block diagram of a calculation device for global model parameters provided in Embodiment 3 of the present invention, which may specifically include the following modules:

[0330] The first loss function determination module 401 is used to construct an initial global parameter model and determine a first loss function for the initial global parameter model.

[0331] The first training parameter generation module 402 is used to send the initial global parameter model to multiple clients to control the clients to train the initial global parameter model based on the first loss function, and generate first training parameters and second training parameters that include the local model parameter increment, as well as a second loss function and the number of data for the local model parameter increment.

[0332] The first Paillier algorithm key generation module 403 is used to generate a first Paillier algorithm key and send the first Paillier algorithm key and the first training parameters encrypted with the first Paillier algorithm key to the second server, and receive a second Paillier algorithm key generated by the second server; the second server is configured to determine a constant value based on the first Paillier algorithm key, the first training parameters and the second training parameters, and send the constant value to the first server.

[0333] The final global model parameter generation module 404 is used to generate target global model parameters and target loss function based on the second Paillier algorithm key and the constant value, and to control the client to iterate the target global model parameters based on the target loss function to generate the final global model parameters.

[0334] As for Embodiment 3, since it is basically similar to Embodiment 1, the description is relatively simple. For relevant details, please refer to the description of the method embodiment.

[0335] Example 4

[0336] Reference Figure 5 The diagram illustrates a structural block diagram of a calculation device for global model parameters provided in Embodiment 4 of the present invention, which may specifically include the following modules:

[0337] The constant value determination module 501 is used to determine the constant value based on the first Paillier algorithm key, the first training parameters and the second training parameters;

[0338] The constant value sending module 502 is used to send the constant value to the first server.

[0339] As for Embodiment 4, since it is basically similar to Embodiment 1, the description is relatively simple. For relevant details, please refer to the description of the method embodiment.

[0340] In addition, this invention also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the various processes of the above-described calculation method embodiment for global model parameters and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0341] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the above-described calculation method embodiments for global model parameters, achieving the same technical effect. To avoid repetition, it will not be described again here. The computer-readable storage medium may include read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc.

[0342] Figure 6 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0343] The electronic device 600 includes, but is not limited to, components such as: a radio frequency unit 601, a network module 602, an audio output unit 603, an input unit 604, a sensor 605, a display unit 606, a user input unit 607, an interface unit 608, a memory 609, a processor 610, and a power supply 611. Those skilled in the art will understand that... Figure 6 The electronic device structures shown are not intended to limit the electronic device. An electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements. In embodiments of the present invention, the electronic device includes, but is not limited to, mobile phones, tablet computers, laptops, PDAs, in-vehicle terminals, wearable devices, and pedometers.

[0344] It should be understood that, in this embodiment of the invention, the radio frequency unit 601 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink data from the base station and processes it with the processor 610; additionally, it transmits uplink data to the base station. Typically, the radio frequency unit 601 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, etc. Furthermore, the radio frequency unit 601 can also communicate with networks and other devices through a wireless communication system.

[0345] The electronic device provides users with wireless broadband internet access through the network module 602, such as helping users send and receive emails, browse web pages, and access streaming media.

[0346] The audio output unit 603 can convert audio data received by the radio frequency unit 601 or the network module 602 or stored in the memory 609 into audio signals and output them as sound. Furthermore, the audio output unit 603 can also provide audio output related to specific functions performed by the electronic device 600 (e.g., call signal reception sound, message reception sound, etc.). The audio output unit 603 includes a speaker, a buzzer, and a receiver, etc.

[0347] Input unit 604 is used to receive audio or video signals. Input unit 604 may include a graphics processing unit (GPU) 6041 and a microphone 6042. GPU 6041 processes image data of still images or videos acquired by an image capture device (such as a camera) in video capture mode or image capture mode. The processed image frames can be displayed on display unit 606. The image frames processed by GPU 6041 can be stored in memory 609 (or other storage medium) or transmitted via radio frequency unit 601 or network module 602. Microphone 6042 can receive sound and process such sound into audio data. The processed audio data can be converted into a format that can be transmitted to a mobile communication base station via radio frequency unit 601 in telephone call mode.

[0348] The electronic device 600 also includes at least one sensor 605, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 6061 according to the ambient light level, and the proximity sensor can turn off the display panel 6061 and / or backlight when the electronic device 600 is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used to identify the posture of the electronic device (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc. The sensor 605 may also include a fingerprint sensor, pressure sensor, iris sensor, molecular sensor, gyroscope, barometer, hygrometer, thermometer, infrared sensor, etc., which will not be described in detail here.

[0349] The display unit 606 is used to display information input by the user or information provided to the user. The display unit 606 may include a display panel 6061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0350] User input unit 607 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of electronic devices. Specifically, user input unit 607 includes a touch panel 6071 and other input devices 6072. Touch panel 6071, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near touch panel 6071). Touch panel 6071 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 610, which receives and executes commands from the processor 610. In addition, touch panel 6071 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. Besides touch panel 6071, user input unit 607 may also include other input devices 6072. Specifically, other input devices 6072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, joysticks, etc., which will not be described in detail here.

[0351] Furthermore, the touch panel 6071 can cover the display panel 6061. When the touch panel 6071 detects a touch operation on or near it, it transmits the information to the processor 610 to determine the type of touch event. Subsequently, the processor 610 provides corresponding visual output on the display panel 6061 based on the type of touch event. Although in Figure 6 In this embodiment, the touch panel 6071 and the display panel 6061 are two independent components to realize the input and output functions of the electronic device. However, in some embodiments, the touch panel 6071 and the display panel 6061 can be integrated to realize the input and output functions of the electronic device. The specific implementation is not limited here.

[0352] Interface unit 608 serves as an interface for connecting external devices to electronic device 600. For example, external devices may include a wired or wireless headphone port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, and so on. Interface unit 608 can be used to receive input from external devices (e.g., data, power, etc.) and transmit the received input to one or more components within electronic device 600, or it can be used to transmit data between electronic device 600 and external devices.

[0353] The memory 609 can be used to store software programs and various data. The memory 609 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback, image playback, etc.), etc.; the data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). Furthermore, the memory 609 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0354] The processor 610 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 609, and by calling data stored in the memory 609, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. The processor 610 may include one or more processing units; preferably, the processor 610 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 610.

[0355] The electronic device 600 may also include a power supply 611 (such as a battery) for supplying power to various components. Preferably, the power supply 611 is logically connected to the processor 610 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system.

[0356] In addition, the electronic device 600 includes some functional modules not shown, which will not be described in detail here.

[0357] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0358] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0359] like Figure 7 As shown, in another embodiment of the present invention, a computer-readable storage medium 701 is also provided, which stores instructions that, when run on a computer, cause the computer to execute the calculation method for global model parameters described in the above embodiments.

[0360] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

[0361] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0362] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0363] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0364] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0365] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0366] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0367] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A computing method for global model parameters, characterized in that, The method is applied to a first server, the first server is configured with a corresponding second server, the first server and the second server have uncollusion, comprising: building an initial global parameter model, and determining a first loss function for the initial global parameter model; sending the initial global parameter model to a plurality of clients to control the clients to train the initial global parameter model based on the first loss function, generate local model parameter increments, and first and second training parameters of a second loss function and a data quantity for the local model parameter increments; generating a first Paillier algorithm key, and sending the first Paillier algorithm key and the first training parameters encrypted by the first Paillier algorithm key to the second server, and receiving a second Paillier algorithm key generated by the second server; the second server is configured to determine a constant value based on the first Paillier algorithm key, the first training parameters and the second training parameters, and send the constant value to the first server; generating target global model parameters and a target loss function based on the second Paillier algorithm key and the constant value, and controlling the clients to iterate the target global model parameters based on the target loss function to generate final global model parameters.

2. The method of claim 1, wherein, The step of controlling the clients to train the initial global parameter model based on the first loss function, generate local model parameter increments, and first and second training parameters of a second loss function and a data quantity for the local model parameter increments comprises: controlling the clients to train the initial global parameter model based on the first loss function to obtain local model parameter increments, a second loss function for the local model parameter increments and a data quantity; determining first and second weight parameters for the local model parameter increments, the second loss function and the data quantity; dividing the local model parameter increments based on the first and second weight parameters to generate first and second local model parameter increment secret sharing values; dividing the second loss function based on the first and second weight parameters to generate first and second divided loss function secret sharing values; dividing the data quantity based on the first and second weight parameters to generate first and second data quantity secret sharing values; determining the first local model parameter increment secret sharing value, the first divided loss function secret sharing value and the first data quantity secret sharing value as the first training parameters; determining the second local model parameter increment secret sharing value, the second divided loss function secret sharing value and the second data quantity secret sharing value as the second training parameters.

3. The method of claim 2, wherein, The step of dividing the local model parameter increment based on the first weight parameter and the second weight parameter to generate a first local model parameter increment secret sharing value and a second local model parameter increment secret sharing value comprises: The step of dividing the second loss function based on the first weight parameter and the second weight parameter to generate a first divided loss function secret sharing value and a second divided loss function secret sharing value comprises: F i = Δ x F i + Δ y F i , The step of dividing the number of data based on the first weight parameter and the second weight parameter to generate a first number of data secret sharing value and a second number of data secret sharing value comprises: n i = Δ x n i + Δ y n i , wherein, w i，d is the dth coordinate size of the local model parameter increment corresponding to the ith client, is the local model parameter increment, Δ x is the first weight parameter, Δ y is the second weight parameter, is the first local model parameter increment secret sharing value, is the second local model parameter increment secret sharing value, F i is the second loss function, Δ x F i is the first partition loss function secret sharing value, Δ y F i is the second partition loss function secret sharing value, n i is the data number, Δ x n i is the first data number secret sharing value, Δ y n i is the second data number secret sharing value.

4. The method of claim 3, wherein, The step of generating a first Paillier algorithm key and sending the first Paillier algorithm key and the first training parameter encrypted by the first Paillier algorithm key to the second server comprises: calculating a first training parameter difference value based on the first local model parameter increment secret sharing value; encrypting the first training parameter difference value by using a first Paillier algorithm key; sending the encrypted first training parameter difference value to the second server.

5. The method of claim 4, wherein, The step of calculating a first training parameter difference value based on the first local model parameter increment secret sharing value and encrypting the first training parameter difference value by using a first Paillier algorithm key comprises: wherein E1(m) is an encryption of m using the first Paillier algorithm key, is the parameter difference of the first training, is the jth coordinate size of the local model parameter increment corresponding to the ith client, is the jth coordinate size of the local model parameter increment corresponding to the ith client.

6. The method of claim 5, wherein, The second server is configured to perform the step of determining a constant value based on the first Paillier algorithm key, the first training parameter and the second training parameter by the following way: calculating a second training parameter difference value based on the second local model parameter increment secret sharing value; determining a first random number and a second random number; determining a constant value based on the first random number, the second random number, the first training parameter difference value and the second training parameter difference value.

7. The method of claim 6, wherein, The step of determining a constant value based on the first random number, the second random number, the first training parameter difference value and the second training parameter difference value comprises: wherein val is the constant value, r1 is the first random number, r2 is the second random number, is the second training parameter difference value, and mod is The remainder of dividing by N, where N is a constant.

8. The method of claim 7, wherein, The step of generating a target global model parameter and a target loss function based on the second Paillier algorithm key and the constant value, and controlling the client to iterate the target global model parameter based on the target loss function to generate a final global model parameter comprises: determining a maximum value and a minimum value from the first local model parameter increment secret sharing value based on the constant value; eliminating the maximum value and the minimum value from the local model parameter increment to generate a client index set; generating a first intermediate value based on the client index set, the second local model parameter increment secret sharing value and the second Paillier algorithm key, and sending to the second server, the second server being configured to generate a second intermediate value based on the client index set, the first local model parameter increment secret sharing value and the first Paillier algorithm key, the second server sending the second intermediate value to the first server; determining a third intermediate value based on the second intermediate value; the second server being configured to determine a fourth intermediate value based on the first intermediate value; the second server sending the fourth intermediate value to the first server; determining an index set value based on the third intermediate value and the fourth intermediate value; generating a target global model parameter based on the index set value; determining the target loss function based on the target global model parameter; iterating the target global model parameter based on the target loss function to generate a final global model parameter.

9. The method of claim 8, wherein, the step of generating a first intermediate value based on the client index set, the second local model parameter increment secret sharing value and the second Paillier algorithm key comprises: the second server being configured to perform the step of generating a second intermediate value based on the client index set, the first local model parameter increment secret sharing value and the first Paillier algorithm key by: the step of determining a third intermediate value based on the second intermediate value comprises: the second server being configured to perform the step of determining a fourth intermediate value based on the first intermediate value by: the step of determining an index set value based on the third intermediate value and the fourth intermediate value comprises: the step of generating a target global model parameter based on the index set value comprises: the step of determining the target loss function based on the target global model parameter comprises: INDS = E2 (Tmp1) + E2 (Tmp2) + E2 (Tmp3) + E2 (Tmp4), wherein INDS is the client index set, Tmp1 is the first intermediate value, Tmp2 is the second intermediate value, Tmp3 is the third intermediate value, Tmp4 is the fourth intermediate value, E2(m) is encryption of m using the first Paillier algorithm key, Δ y w i，j is the second local model parameter increment secret sharing value, is the target global model parameter; F is the target loss function.

10. A computing method for global model parameters, characterized in that, the method is applied to a second server, the second server being configured with a corresponding first server, the first server and the second server having uncollusion, the first server being configured to build an initial global parameter model and determine a first loss function for the initial global parameter model; sending the initial global parameter model to a plurality of clients to control the clients to train the initial global parameter model based on the first loss function to generate a local model parameter increment, and a second loss function for the local model parameter increment and a first training result parameter and a second training result parameter of a data quantity; generating a first Paillier algorithm key and sending the first Paillier algorithm key and the first training result parameter encrypted by the first Paillier algorithm key to the second server, receiving a second Paillier algorithm key generated by the second server, the method comprising: determining a constant value based on the first Paillier algorithm key, the first training result parameter and the second training result parameter; send the constant value to the first server.

11. A computing device for global model parameters, characterized in that, The device is applied to a first server, the first server is configured with a corresponding second server, the first server and the second server have uncollusion, comprising: A first loss function determination module is configured to construct an initial global parameter model and determine a first loss function for the initial global parameter model. A first trained parameter generation module is configured to send the initial global parameter model to a plurality of clients to control the clients to train the initial global parameter model based on the first loss function, generate first trained parameters and second trained parameters containing local model parameter increments, second loss functions for the local model parameter increments, and data quantities. A first Paillier algorithm key generation module is configured to generate a first Paillier algorithm key, send the first Paillier algorithm key and the first trained parameters encrypted by the first Paillier algorithm key to the second server, and receive a second Paillier algorithm key generated by the second server; the second server is configured to determine a constant value based on the first Paillier algorithm key, the first trained parameters, and the second trained parameters, and send the constant value to the first server. A final global model parameter generation module is configured to generate target global model parameters and a target loss function based on the second Paillier algorithm key and the constant value, and control the clients to iterate the target global model parameters based on the target loss function to generate final global model parameters.

12. A computing device for global model parameters, characterized in that, The device is applied to a second server, the second server is configured with a corresponding first server, the first server and the second server have uncollusion, the first server is configured to construct an initial global parameter model and determine a first loss function for the initial global parameter model; send the initial global parameter model to a plurality of clients to control the clients to train the initial global parameter model based on the first loss function, generate first trained parameters and second trained parameters containing local model parameter increments, second loss functions for the local model parameter increments, and data quantities; generate a first Paillier algorithm key, and send the first Paillier algorithm key and the first trained parameters encrypted by the first Paillier algorithm key to the second server, and receive a second Paillier algorithm key generated by the second server, the device comprising: A constant value determination module is configured to determine a constant value based on the first Paillier algorithm key, the first trained parameters, and the second trained parameters. A constant value sending module is configured to send the constant value to the first server.

13. An electronic device, comprising: comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory accomplish mutual communication through the communication bus; the memory, configured to store a computer program; the processor, configured to execute the program stored on the memory, so as to realize the method in any one of claims 1-9 or 10.

14. A computer readable storage medium having stored thereon instructions which, when executed by one or more processors, cause the processors to perform the method in any one of claims 1-9 or 10.

Images