Federal learning system and method based on optimized multi-party computing

Abstract

The invention discloses a federated learning system based on optimized multi-party computing, which comprises a local processing module, a central aggregation platform, a local model of each participant platform is trained based on global model parameters to obtain a model updating gradient, compression and secure transformation processing are performed to obtain a transformed gradient vector, and the gradient vector is subjected to multi-party computing; each participant platform generates a random scalar mask and executes an encryption operation, and generates identity authentication information at the same time; the gradient aggregation module verifies the identity of each participant platform through the central aggregation platform, performs plaintext aggregation on the transformed gradient vectors of all participant platforms passing verification, performs secure aggregation after decrypting encrypted mask parameters by using secure multi-party calculation, and finally obtains a global aggregation gradient through inverse transformation reconstruction; and the gradient updating module updates the local model by using the global aggregation gradient through each participant platform. According to the method, the security of data privacy is guaranteed, and the calculation efficiency of federal learning is improved.

Description

Technical Field

[0001] This invention relates to the field of multi-party secure computation technology, specifically to a federated learning system and method based on optimized multi-party computation. Background Technology

[0002] Federated learning, as a distributed machine learning paradigm, allows multiple participating platforms to jointly train models without sharing their local raw data. While this solves the "data silo" problem by avoiding the sharing of raw data, the model parameters or gradient information exchanged during federated learning training may be used to infer sensitive data from participating platforms. Therefore, existing technologies typically introduce secure multi-party computation to protect gradient confidentiality, such as using arithmetic secret sharing techniques to split local gradients into secret shares before secure aggregation. However, federated learning schemes based on secure multi-party computation have significant performance bottlenecks. Implementing nonlinear operations such as multiplication requires multiple rounds of communication, leading to excessively long training times in environments with high network latency. Furthermore, the nonlinear activation functions widely used in complex models require computationally intensive polynomial approximations within the secure multi-party computation framework, further exacerbating computation and communication latency. In addition, the high communication overhead, incomplete security mechanisms, and lack of participant identity authentication make the system vulnerable to adversarial attacks such as model poisoning or data contamination. Some solutions attempt to address these issues but introduce dependence on trusted third parties, which can easily introduce single points of failure and security risks. Summary of the Invention

[0003] The purpose of this invention is to provide a federated learning system and method based on optimized multi-party computation. This invention significantly improves the computational efficiency and security of federated learning while ensuring data privacy and security.

[0004] To achieve this objective, the present invention provides a federated learning system based on optimized multi-party computation, comprising: The local processing module is used to train the local model corresponding to each participating platform based on the global model parameters of the central aggregation platform to obtain the corresponding model update gradient, compress the model update gradient of each participating platform to obtain the quantized sparse gradient, perform a secure transformation on the quantized sparse gradient to obtain the transformed gradient vector, generate a random scalar mask for each participating platform and perform an encryption operation to obtain the encryption mask parameter, and generate its own identity authentication information. The gradient aggregation module is used to verify the identity of each participating platform through the central aggregation platform using the identity authentication information, perform plaintext aggregation on the transformed gradient vectors of all verified participating platforms to obtain the global aggregated transformation gradient, decrypt the encrypted mask parameters of all verified participating platforms using secure multi-party computation, perform secure aggregation on the decrypted plaintext to obtain the global mask aggregation gradient, and perform inverse transformation reconstruction on the global aggregated transformation gradient and the global mask aggregation gradient to obtain the global aggregated gradient. The gradient update module is used by each participating platform to update its own local model using the global aggregated gradient, thereby obtaining the updated local model parameters.

[0005] Preferably, the specific process of training the local model corresponding to each participating platform to obtain the corresponding model update gradient based on the global model parameters of the central aggregation platform is as follows: Each participating platform Receive global model parameters from the central aggregation platform. The global model parameters are the current global model of the central aggregation platform. Weighting of each participating platform; Using local datasets The local model is trained using stochastic gradient descent to obtain an updated local model: in, For the central aggregation platform t The global model is updated after the local model. For the first i Local datasets of each participating platform Indicates performing in the local model k Rounds of stochastic gradient descent training, For the first i The local model updated by each participating platform; Compensation for residuals introduced by sparsification during the (t-1)th round of local model updates: ,in This represents the sparsed residual of the local model update in round t-1. This represents the updated gradient of the model after compensation; Thus, the model update gradient can be calculated: in, Indicates the first i The participating platform in the first t The model update gradient of the local model after the first round of updates.

[0006] Preferably, the specific process of compressing the model update gradient of each participating platform to obtain the quantized sparse gradient is as follows: Update gradients for the model Perform sparsification and random quantization sequentially to generate a compressed gradient vector: First update the gradient of the model Sparsification is performed to obtain sparse gradients. : in, K These are preset positive integer hyperparameters used to represent the model update gradient. The number of elements retained during sparsification; This represents the sparsity selection algorithm; Further compression processing is applied to sparse gradients. Perform random quantization to obtain the quantized sparse gradient. For each element j Quantized sparse gradient for: in, For elements j The sparse gradient to be quantized For elements j Quantized sparse gradient, The preset quantization step size, This indicates the floor function. This indicates the rounding up operation.

[0007] Preferably, the process of performing a secure transformation on the quantized sparse gradient to obtain the transformed gradient vector, and each participating platform generating a random scalar mask and performing an encryption operation to obtain the encrypted mask parameters, is as follows: Perform a stochastic orthogonal transformation on the quantized sparse gradient using a randomly generated orthogonal matrix. and random scalar mask The transformed gradient vector is calculated. : in, This is the transformed gradient vector. Randomly generated orthogonal matrix, The quantized sparse gradient, For the first i Each participating platform independently generates a random scalar mask. For all elements are 1 M 3D column vector; Using a semantically secure encryption function based on the discrete logarithm problem to mask random scalars Perform the encryption operation to obtain the encryption mask parameter. : in, The first part represents the encryption mask parameter. , This represents the shared mask used in the encryption process. This represents a group element that contains plaintext information with a mask. The second part represents the encryption mask parameter. , Indicates a semantically secure encryption function; Each participating platform will transmit the transformed gradient vector. With encryption mask parameters Uploaded to the central aggregation platform.

[0008] Preferably, each participating platform generates its own identity authentication information, and the specific process of using this identity authentication information to verify the identity of each participating platform through the central aggregation platform is as follows: Calculate each participating platform With the central aggregation platform S Shared secret value : in, For the first i Individual Participant Platforms private key, Indicates central aggregation platform S public key, This indicates the calculation of the shared key. H This represents the key-derived hash function. Used to calculate the shared secret value; Shared secret value Combined with the current timestamp Generate a hash function using the authentication token. Generate identity authentication token : in, Indicates the first i Identity authentication tokens for each participating platform. Indicates the first i The participating platform in the first t The current timestamp of the round-up update, Indicates the hash function used to generate the authentication token; Each participating platform Transformed gradient vector Encryption mask parameters Identity authentication token and timestamp Merged into a unified data packet Uploaded to the central aggregation platform: For each received data packet The central aggregation platform uses its own private key and storage for each participating platform public key Calculate the platform of each participating party Expected authentication token : in, Central aggregation platform S private key, For the first i Individual Participant Platforms public key, Used to calculate the shared secret value. The expected shared secret value is independently calculated by the central aggregation platform. This indicates an expectation to share the secret value. With timestamp Concatenate the first and last characters in order to form a new string; Central aggregation platform comparison i Individual Participant Platforms Identity authentication token and expected authentication token Are they the same? If they are the same, then the first... i Individual Participant Platforms Authentication passed.

[0009] Preferably, the process of performing plaintext aggregation on the transformed gradient vectors of all verified participating platforms to obtain the global aggregated transformation gradient, decrypting the encrypted mask parameters of all verified participating platforms using secure multi-party computation, and performing secure aggregation on the decrypted plaintext to obtain the global mask aggregated gradient, and then performing inverse transformation on the global aggregated transformation gradient and the global mask aggregated gradient to reconstruct the global aggregated gradient is as follows: For all participating platforms that have passed the verification, extract the transformed gradient vector. The plaintext is aggregated using the aggregation weights of each participating platform to obtain the global aggregation transformation gradient. : in, Indicates the first i Individual Participant Platforms Aggregate weights; encryption mask parameters Using secure multi-party computation decryption, the central aggregation platform utilizes its own private key. calculate ,in This represents the private key of the central aggregation platform. This represents the first part of the encryption mask parameter. The first part represents the encryption mask parameter. , r i Indicates the first i Individual Participant Platforms Generated random numbers, This represents a shared masking factor used to eliminate randomness during the decryption process. This indicates that the central aggregation platform uses its private key to access the first part of the encryption mask parameters. Perform exponentiation. express The specific unfolding form; encryption mask parameters After decryption, the mask plaintext located at the exponent position is obtained. : in, This refers to the masking term introduced during the encryption process. Indicates the decryption factor; Then, the plaintext mask is solved using a discrete logarithmic algorithm. The plain text was recovered from the middle : plaintext The global mask aggregation gradient is obtained by performing weighted security aggregation. : Gradient transformation from global aggregation Subtract global mask aggregate gradient The effect is that: in, For all elements are 1 M 3D column vector, Randomly generated an orthogonal matrix; Then, the inverse of the stochastic orthogonal transformation is performed to reconstruct the global aggregate gradient. : in, Orthogonal matrix The transpose of .

[0010] Preferably, the specific process by which each participating platform updates its own local model using the global aggregated gradient to obtain the updated local model parameters is as follows: The central aggregation platform will reconstruct the global aggregation gradient. The data is distributed to each participating platform, and each participating platform utilizes the global aggregation gradient. The corresponding local model is updated to obtain the updated local model parameters, using the following formula: in, For the present t Local model parameters for each round, For the updated local model parameters, For the present t Learning rate per round.

[0011] To achieve this objective, the present invention provides a federated learning system for neural network models containing nonlinear activation functions, utilizing optimized multi-party computation, comprising: The local processing module is used to train the neural network model corresponding to each participating platform based on the global model parameters of the central aggregation platform to obtain the corresponding model update gradient. The model update gradient of each participating platform is compressed to obtain the quantized sparse gradient. The quantized sparse gradient is subjected to a secure transformation to obtain the transformed gradient vector. Each participating platform generates a random scalar mask and performs an encryption operation to obtain the encryption mask parameter. The input value of the nonlinear activation function in the neural network model is converted into a secret shared value. Each participating platform also generates its own identity authentication information. The gradient aggregation module is used to authenticate the identity of each participating platform through the central aggregation platform using the identity authentication information. It performs plaintext aggregation on the transformed gradient vectors of all verified participating platforms to obtain the global aggregated transformation gradient. It decrypts the encrypted mask parameters of all verified participating platforms using secure multi-party computation and performs secure aggregation on the decrypted plaintext to obtain the global mask aggregate gradient. It performs inverse transformation on the global aggregated transformation gradient and the global mask aggregate gradient to reconstruct the global aggregated sparse gradient of the neural network model. It performs polynomial aggregation computation on the secret shared value of the nonlinear activation function using an optimized multi-party computation protocol and pre-generated auxiliary data to obtain the nonlinear aggregate gradient of the neural network model. It obtains the global aggregated gradient based on the global aggregated sparse gradient of the neural network model and the nonlinear aggregate gradient of the neural network model. The gradient update module is used by each participating platform to update its own neural network model using the global aggregated gradient, thereby obtaining the updated neural network model parameters.

[0012] Preferably, when the global model and the local model are neural network models and contain non-linear activation functions, the specific process of obtaining the global aggregated gradient includes: A set of multivariate Beaver triples related to the highest power K to be determined is generated in advance for all participating platforms and distributed in a secret sharing manner as auxiliary data. The auxiliary data format is as follows: in, For K independent random values Secret shared collection, The secret shared set of any pairwise product of any two random values. For any j A secret shared set of products of random values. Secret sharing of the total product of K random values; Auxiliary data Including secret sharing of K independent random values And the secret sharing of all possible cross-product terms consisting of random values; For the input value of the non-linear activation function Each participating platform utilizes pre-shared random values For input values Perform masking and calculate the difference. Secret shared value : in, The input value represents the intermediate data processed by the non-linear activation function; It is a random value; In the single communication round, all participating platforms simultaneously publish and reconstruct the plaintext result of the difference: ; Each participating platform is based on the same publicly disclosed difference. and pre-shared auxiliary data All power terms are computed locally in parallel using linear operations. The secret shared value, in which The input value of the non-linear activation function The secret sharing result of the power of K; Each participating platform shares a secret value for each power term and preset polynomial coefficients. By performing a linear combination, the secret shared value of the calculation result of the nonlinear activation function is obtained: For the secret shared value of the nonlinear activation function, an optimized multi-party computation protocol and pre-generated auxiliary data are used to perform polynomial aggregation computation based on secret sharing to obtain the nonlinear aggregate gradient. The global aggregate sparse gradient of the neural network model is directly used for global model update, and the nonlinear aggregate gradient is used to influence the model update gradient generated by the training of the neural network model through the nonlinear activation function, thereby obtaining the global aggregate gradient.

[0013] A federated learning method based on optimized multi-party computation includes the following steps: Based on the global model parameters of the central aggregation platform, the local model corresponding to each participating platform is trained to obtain the corresponding model update gradient. The model update gradient of each participating platform is compressed to obtain the quantized sparse gradient. The quantized sparse gradient is subjected to a secure transformation to obtain the transformed gradient vector. Each participating platform generates a random scalar mask and performs an encryption operation to obtain the encryption mask parameter. Each participating platform also generates its own identity authentication information. The central aggregation platform uses the identity authentication information to verify the identities of each participating platform. The transformed gradient vectors of all verified participating platforms are aggregated in plaintext to obtain the global aggregation transformation gradient. The encrypted mask parameters of all verified participating platforms are decrypted using secure multi-party computation. The decrypted plaintext is then aggregated securely to obtain the global mask aggregation gradient. Finally, the global aggregation transformation gradient and the global mask aggregation gradient are reconstructed by inverse transformation to obtain the global aggregation gradient. Each participating platform updates its local model using the globally aggregated gradient to obtain updated local model parameters. A computer program product includes a computer program that, when executed by a processor, implements the steps of the above method.

[0014] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0015] The beneficial effects of this invention are: This invention trains the local model through participating platforms and compresses the model update gradients, effectively reducing the amount of communication data in each training round. Each participating platform uploads the processed data using secure multi-party computation, and a built-in authentication mechanism ensures communication security and the legitimacy of the participating platform's identity, avoiding reliance on trusted third parties. Through gradient compression, secure multi-party computation optimization, and authentication mechanisms, this invention significantly improves the computational efficiency and security of federated learning while reducing latency, ensuring data privacy and security. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the structure of the present invention; Figure 2 This is a flowchart of the present invention. Detailed Implementation

[0017] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments: Example 1 A federated learning system based on optimized multi-party computation, such as Figure 1 As shown, it includes: The local processing module is used to train the local model corresponding to each participating platform based on the global model parameters of the central aggregation platform to obtain the corresponding model update gradient. The model update gradient of each participating platform is compressed to obtain the quantized sparse gradient. The quantized sparse gradient is then subjected to a secure transformation to obtain the transformed gradient vector. Each participating platform generates a random scalar mask and performs an encryption operation to obtain the encryption mask parameter (each participating platform generates only one encryption mask parameter in the federated learning system). In addition, each participating platform generates its own identity authentication information. This design reduces the amount of data transmitted by compressing the model update gradient of the participating platforms and encrypts only preset elements, which can enhance data privacy protection and reduce computational costs. The gradient aggregation module is used by the central aggregation platform to authenticate the identity information of each participating platform. It performs plaintext aggregation on the transformed gradient vectors of all verified participating platforms to obtain the global aggregated transformation gradient. It decrypts the encrypted mask parameters of all verified participating platforms using secure multi-party computation and performs secure aggregation on the decrypted plaintext to obtain the global masked aggregation gradient. It performs inverse transformation on the global aggregated transformation gradient and the global masked aggregation gradient to reconstruct the global aggregated gradient (all participating platforms receive and use the same global aggregated gradient in each round of training). This design can ensure the legitimacy of participating platforms through the identity authentication mechanism and prevent malicious attacks. By aggregating and reconstructing the unencrypted and encrypted gradients, it improves the processing efficiency of the central aggregation platform, reduces computational latency, and enhances security. The gradient update module is used by each participating platform to update its local model using the global aggregated gradient, and obtain the updated local model parameters (the updated local model parameters are used for the training of the local model in the next iteration). This design uses the global aggregated gradient issued by the central aggregation platform to directly update the local model parameters of each participating platform. Through weighted averaging and learning rate adjustment, the local model is gradually aligned with the global state, which can ensure the continuity and convergence of the local model iterative optimization and avoid performance degradation caused by delay or inconsistency during the learning process.

[0018] Some optimized technical solutions include: the central aggregation platform refers to the server-side entity responsible for coordinating the participating platforms, distributing global model parameters, updating aggregated gradients, and executing secure computing protocols in the federated learning network, including high-performance computing cluster servers deployed within power companies or hospitals, Alibaba Cloud servers, Amazon AWS cloud platforms, and Tencent Cloud servers; the participating platforms refer to client devices or institutional servers that hold local private data, train local models locally, and upload encrypted gradients, including smartphones, onboard computing units of electric vehicles, smart wearable devices, and local database servers of power companies or hospitals; the local models include deep neural networks (DNN), convolutional neural networks (CNN), recurrent neural networks (RNN), logistic regression models, and multilayer perceptrons (MLP).

[0019] In the above technical solution, the specific process of training the local model corresponding to each participating platform to obtain the corresponding model update gradient based on the global model parameters of the central aggregation platform is as follows: Each participating platform Receive global model parameters from the central aggregation platform. The global model parameters are the current global model of the central aggregation platform. The weights (global models include Deep Neural Networks (DNN), Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Logistic Regression models, and Multilayer Perceptrons (MLP)); each participating platform Using local datasets The local model is trained using stochastic gradient descent to obtain an updated local model: in, For the central aggregation platform t The global model is updated after the local model. For the first i Local datasets of each participating platform Indicates performing in the local model k Rounds of stochastic gradient descent training, For the first i The local model updated by each participating platform; Compensation for residuals introduced by sparsification during the (t-1)th round of local model updates: ,in This represents the sparsed residual of the local model update in round t-1. This represents the gradient of the model update after compensation; Thus, the model update gradient can be calculated: in, Indicates the first i The participating platform in the first t The local model updates its gradient after each round of updates. The above design compensates the residuals from the previous round back to the current round. While pursuing communication efficiency (sparseness), it retains as much complete gradient information as possible, which can avoid error accumulation and effectively alleviate the information loss problem caused by gradient sparsity, thereby improving the accuracy and convergence stability of local model training.

[0020] In the above technical solution, the specific process of compressing the model update gradient of each participating platform to obtain the quantized sparse gradient is as follows: Update gradients for the model The sparsity processing (Top-K sparsity processing) and random quantization processing are performed sequentially to generate a compressed gradient vector: First update the gradient of the model Sparsification is performed to obtain sparse gradients. : in, K Preset positive integer hyperparameters (K (Can take a value of 100) to represent the model update gradient. The number of elements retained during sparsification; This represents the sparsity selection algorithm; Further compression processing is applied to sparse gradients. Perform random quantization to obtain the quantized sparse gradient. For each element j Quantized sparse gradient for: in, For elements j The sparse gradient to be quantized For elements j Quantized sparse gradient, The preset quantization step size ( (This determines the precision after quantization) This indicates the floor function. This represents the floor function; the above design uses Top-K sparsity to retain only the most important gradients from the model update. K The element transforms the high-dimensional model update gradient into a sparse gradient. By performing random quantization to further reduce the gradient value of the sparse gradient, the amount of gradient data that needs to be transmitted can be reduced.

[0021] In the above technical solution, the transformed gradient vector is obtained by performing a secure transformation on the quantized sparse gradient, and each participating platform generates a random scalar mask and performs an encryption operation to obtain the encrypted mask parameters. The specific process is as follows: Perform a stochastic orthogonal transformation on the quantized sparse gradient using a randomly generated orthogonal matrix. and random scalar mask The transformed gradient vector is calculated. : in, This is the transformed gradient vector. Randomly generated orthogonal matrix, The quantized sparse gradient, For the first i Each participating platform independently generates a random scalar mask. For all elements are 1 M 3D column vector; Using a semantically secure encryption function based on the discrete logarithm problem to mask random scalars Perform the encryption operation to obtain the encryption mask parameter. : in, The first part represents the encryption gradient. , This represents the shared mask used in the encryption process. This represents a group element that contains plaintext information with a mask. The second part represents the encryption gradient. , Indicates a semantically secure encryption function; Each participating platform will transmit the transformed gradient vector. With encryption mask parameters The data is uploaded to the central aggregation platform. The above design scrambles gradient information through random orthogonal transformation and encrypts only preset elements (key mask parameters), which enhances data privacy and security while reducing computational costs.

[0022] In the above technical solution, each participating platform generates its own identity authentication information. The specific process of using this identity authentication information to verify the identity of each participating platform through the central aggregation platform is as follows: Calculate each participating platform With the central aggregation platform S Shared secret value : in, For the first i Individual Participant Platforms private key, Indicates central aggregation platform S public key, This indicates the calculation of the shared key. H This represents the key-derived hash function. Used to calculate the shared secret value; Shared secret value Combined with the current timestamp Generate a hash function using the authentication token. Generate authentication token : in, Indicates the first i Identity authentication tokens for each participating platform. Indicates the first i The participating platform in the first t The current timestamp of the round-up update, Indicates the hash function used to generate the authentication token; Each participating platform Transformed gradient vector Encryption mask parameters Identity authentication token and timestamp Merged into a unified data packet Uploaded to the central aggregation platform: For each received data packet The central aggregation platform uses its own private key and storage for each participating platform public key Calculate the platform of each participating party Expected authentication token : in, Central aggregation platform S private key, For the first i Individual Participant Platforms public key, Used to calculate the shared secret value. The expected shared secret value is independently calculated by the central aggregation platform. This indicates an expectation to share the secret value. With timestamp Concatenate the first and last characters in order to form a new string; Central aggregation platform comparison i Individual Participant Platforms Identity authentication token and expected authentication token Are they the same? If they are the same, then the first... i Individual Participant Platforms The identity verification is successful; the above design allows participating platforms and a central aggregation platform to independently calculate the same shared secret using each other's public keys and their own private keys, and generate tokens by combining them with timestamps, ensuring the uniqueness and timeliness of identity authentication information. The authentication process does not require third-party intervention, thus improving information security.

[0023] In the above technical solution, the process of performing plaintext aggregation on the transformed gradient vectors of all verified participating platforms to obtain the global aggregated transformation gradient, decrypting the encrypted mask parameters of all verified participating platforms using secure multi-party computation, and performing secure aggregation on the decrypted plaintext to obtain the global mask aggregated gradient, and then performing inverse transformation on the global aggregated transformation gradient and the global mask aggregated gradient to reconstruct the global aggregated gradient is as follows: For all participating platforms that have passed the verification, extract the transformed gradient vector. The plaintext is aggregated using the aggregation weights of each participating platform to obtain the global aggregation transformation gradient. : in, Indicates the first i Individual Participant Platforms Aggregate weights; encryption mask parameters Using secure multi-party computation decryption, the central aggregation platform utilizes its own private key. calculate ,in This represents the private key of the central aggregation platform. This represents the first part of the encryption mask parameter. The first part represents the encryption mask parameter. , r i Indicates the first i Individual Participant Platforms Generated random numbers, This represents a shared masking factor used to eliminate randomness during the decryption process. This indicates that the central aggregation platform uses its private key to access the first part of the encryption mask parameters. Perform exponentiation. express The specific unfolding form; encryption mask parameters After decryption, the mask plaintext located at the exponent position is obtained. (Group elements containing plaintext mask information): in, This refers to the masking term introduced during the encryption process. Indicates the decryption factor; Then, the plaintext mask is solved using a discrete logarithmic algorithm. The plain text was recovered from the middle : plaintext The global mask aggregation gradient is obtained by performing weighted security aggregation. : Gradient transformation from global aggregation Subtract global mask aggregate gradient The effect is that: in, For all elements are 1 M 3D column vector, Randomly generated an orthogonal matrix; Then perform the inverse transformation of the random orthogonal transformation (multiply by the orthogonal matrix). (transpose of the given information) to reconstruct the global aggregated gradient. : in, Orthogonal matrix The above design performs efficient plaintext weighted aggregation on unencrypted gradients, and decrypts encrypted gradients using secure multi-party computation before aggregation. The aggregation result is then subjected to the inverse of a random orthogonal transformation. This approach correctly reconstructs the global aggregation gradient while protecting the data privacy of each participating platform, thus improving processing efficiency and security.

[0024] In the above technical solution, the specific process by which each participating platform updates its own local model using the global aggregated gradient to obtain the updated local model parameters is as follows: The central aggregation platform will reconstruct the global aggregation gradient. The data is distributed to each participating platform, and each participating platform utilizes the global aggregation gradient. The corresponding local model is updated to obtain the updated local model parameters, using the following formula: in, For the present t Local model parameters for each round, For the updated local model parameters, For the present t The learning rate for each round; the above design ensures that the local models of all participating platforms optimize in the same direction after each iteration through the global aggregation gradient issued by the central aggregation platform, thereby enabling the local models of all participating platforms to evolve synchronously and consistently.

[0025] Among the above technical solutions, a federated learning system for neural network models containing nonlinear activation functions, using optimized multi-party computation, is characterized by: The local processing module is used to train the neural network model corresponding to each participating platform based on the global model parameters of the central aggregation platform to obtain the corresponding model update gradient. The model update gradient of each participating platform is compressed to obtain the quantized sparse gradient. The quantized sparse gradient is then subjected to a secure transformation to obtain the transformed gradient vector. Each participating platform generates a random scalar mask and performs encryption to obtain the encryption mask parameters. The input value of the nonlinear activation function in the neural network model (the input value specifically refers to the intermediate data processed by the nonlinear activation function during the training of the neural network model) is converted into a secret shared value (the secret shared value is the secret shared share). Each participating platform also generates its own identity authentication information. This design, by converting the input value of the nonlinear activation function into a secret shared value, enables subsequent complex calculations to be performed in a secret shared form, thus isolating the risk of data privacy leakage from the data source. The gradient aggregation module is used to authenticate the identity information of each participating platform through the central aggregation platform. It performs plaintext aggregation on the transformed gradient vectors of all verified participating platforms to obtain the global aggregated transformation gradient. It decrypts the encrypted mask parameters of all verified participating platforms using secure multi-party computation and performs secure aggregation on the decrypted plaintext to obtain the global masked aggregation gradient. It performs inverse transformation on the global aggregated transformation gradient and the global masked aggregation gradient to reconstruct the global aggregated sparse gradient of the neural network model. It performs polynomial aggregation computation on the secret shared value of the nonlinear activation function using an optimized multi-party computation protocol and pre-generated auxiliary data to obtain the nonlinear aggregated gradient of the neural network model. Based on the global aggregated sparse gradient of the neural network model and the nonlinear aggregated gradient of the neural network model, the global aggregated gradient is obtained. This design uses pre-generated multivariate Beaver triples as auxiliary data to transform the nonlinear computation that originally required multiple rounds of interaction (such as the polynomial approximation of the nonlinear activation function) into a single-round communication operation, which greatly improves the computational efficiency of data processing. The gradient update module is used by each participating platform to update its own neural network model using the global aggregated gradient to obtain the updated neural network model parameters. This design uses the global aggregated gradient issued by the central aggregation platform to directly update the local model parameters of each participating platform. Through weighted averaging and learning rate adjustment, the local model is gradually aligned with the global state, which can ensure the continuity and convergence of the local model iterative optimization and avoid performance degradation caused by delay or inconsistency during the learning process.

[0026] For cases where both the global and local models are neural network models and contain non-linear activation functions, some optimization techniques include: the specific process for obtaining the global aggregated gradient includes: A set of multivariate Beaver triples related to the highest power K to be determined is generated in advance for all participating platforms and distributed in a secret sharing manner as auxiliary data. The auxiliary data format is as follows: in, For K independent random values Secret shared collection, The secret shared set of any pairwise product of any two random values. For any j A secret shared set of products of random values. Secret sharing of the total product of K random values; Auxiliary data Including secret sharing of K independent random values And the secret sharing of all possible cross-product terms consisting of random values; For the input value of the non-linear activation function Each participating platform utilizes pre-shared random values For input values Perform masking and calculate the difference. Secret shared value : in, The input value represents the intermediate data processed by the non-linear activation function; The value is a random number (a random number generated in advance through a secret sharing method among the participating platforms). In the single communication round, all participating platforms simultaneously publish and reconstruct the plaintext result of the difference: ; Each participating platform is based on the same publicly disclosed difference. and pre-shared auxiliary data All power terms are computed locally in parallel using linear operations. The secret shared value, in which The input value of the non-linear activation function The secret sharing result of the power of K; Each participating platform shares a secret value for each power term and preset polynomial coefficients. By performing a linear combination, the secret shared value of the calculation result of the nonlinear activation function is obtained: For the secret-shared value of the nonlinear activation function, an optimized multi-party computation protocol and pre-generated auxiliary data are used to perform polynomial aggregation computation based on secret sharing to obtain the nonlinear aggregated gradient. The global aggregated sparse gradient of the neural network model is directly used for global model updates, while the nonlinear aggregated gradient is used to influence the model update gradient generated during neural network model training through the nonlinear activation function, thereby obtaining the global aggregated gradient. This design reduces the multiple rounds of communication required for the nonlinear activation function to only one round, which can reduce the training time of the neural network model caused by network latency. All power terms of the input value of the nonlinear activation function are calculated securely in one operation. The secret shared value enables parallel processing and improves data processing efficiency; at the same time, the input value of the nonlinear activation function always exists in the form of secret sharing, and no participating platform can obtain its plaintext. Based on secret sharing and multivariate Beaver triples, the confidentiality of data is ensured.

[0027] Example 2 A federated learning method based on optimized multi-party computation, such as Figure 2 As shown, based on the global model parameters of the central aggregation platform, the local model of each participating platform is trained to obtain the model update gradient, which is then compressed and securely transformed to obtain the transformed gradient vector. Each participating platform generates a random scalar mask and performs encryption, while simultaneously generating identity authentication information. After the central aggregation platform verifies the identity of each participating platform, the transformed gradient vectors of all verified participating platforms are aggregated in plaintext. The encrypted mask parameters are decrypted using secure multi-party computation and then securely aggregated. Finally, the global aggregated gradient is reconstructed through inverse transformation. Each participating platform then uses the global aggregated gradient to update its local model.

[0028] The specific method of federated learning based on optimized multi-party computation includes the following steps: Based on the global model parameters of the central aggregation platform, the local model corresponding to each participating platform is trained to obtain the corresponding model update gradient. The model update gradient of each participating platform is compressed to obtain the quantized sparse gradient. The quantized sparse gradient is subjected to a secure transformation to obtain the transformed gradient vector. Each participating platform generates a random scalar mask and performs an encryption operation to obtain the encryption mask parameter. Each participating platform also generates its own identity authentication information. The central aggregation platform uses the identity authentication information to verify the identities of each participating platform. The transformed gradient vectors of all verified participating platforms are aggregated in plaintext to obtain the global aggregation transformation gradient. The encrypted mask parameters of all verified participating platforms are decrypted using secure multi-party computation. The decrypted plaintext is then aggregated securely to obtain the global mask aggregation gradient. Finally, the global aggregation transformation gradient and the global mask aggregation gradient are reconstructed by inverse transformation to obtain the global aggregation gradient. Each participating platform updates its local model using the global aggregated gradient to obtain updated local model parameters.

[0029] Example 3 A computer program product includes a computer program that, when executed by a processor, implements the steps of the method described in Embodiment 2.

[0030] Example 4 A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method as described in Embodiment 2.

[0031] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0032] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A system that specifies functions in one or more boxes.

[0033] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction set implemented in a process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0034] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0035] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit its scope of protection. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that after reading the present invention, they can still make various changes, modifications or equivalent substitutions to the specific implementation of the invention, but these changes, modifications or equivalent substitutions are all within the scope of protection of the pending claims of the invention.

[0036] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

Claims

1. A federated learning system based on optimized multi-party computation, characterized in that, It includes: The local processing module is used to train the local model corresponding to each participating platform based on the global model parameters of the central aggregation platform to obtain the corresponding model update gradient, compress the model update gradient of each participating platform to obtain the quantized sparse gradient, perform a secure transformation on the quantized sparse gradient to obtain the transformed gradient vector, generate a random scalar mask for each participating platform and perform an encryption operation to obtain the encryption mask parameter, and generate its own identity authentication information. The gradient aggregation module is used to verify the identity of each participating platform through the central aggregation platform using the identity authentication information, perform plaintext aggregation on the transformed gradient vectors of all verified participating platforms to obtain the global aggregated transformation gradient, decrypt the encrypted mask parameters of all verified participating platforms using secure multi-party computation, perform secure aggregation on the decrypted plaintext to obtain the global mask aggregation gradient, and perform inverse transformation reconstruction on the global aggregated transformation gradient and the global mask aggregation gradient to obtain the global aggregated gradient. The gradient update module is used by each participating platform to update its own local model using the global aggregated gradient, thereby obtaining the updated local model parameters.

2. The federated learning system based on optimized multi-party computation according to claim 1, characterized in that: The specific process of training the local model corresponding to each participating platform to obtain the corresponding model update gradient based on the global model parameters of the central aggregation platform is as follows: Each participating platform Receive global model parameters from the central aggregation platform. The global model parameters are the current global model of the central aggregation platform. Weighting of each participating platform; Using local datasets The local model is trained using stochastic gradient descent to obtain an updated local model: in, For the central aggregation platform t The global model is updated after the local model. For the first i Local datasets of each participating platform Indicates performing in the local model k Rounds of stochastic gradient descent training, For the first i The local model updated by each participating platform; Compensation for residuals introduced by sparsification during the (t-1)th round of local model updates: ,in This represents the sparsed residual of the local model update in round t-1. This represents the gradient of the model update after compensation; Thus, the model update gradient can be calculated: in, Indicates the first i The participating platform in the first t The model update gradient of the local model after the first round of updates.

3. A federated learning system based on optimized multi-party computation according to claim 2, characterized in that: The specific process of compressing the model update gradients of each participating platform to obtain the quantized sparse gradients is as follows: Update gradients for the model Perform sparsification and random quantization sequentially to generate a compressed gradient vector: First update the gradient of the model Sparsification is performed to obtain sparse gradients. : in, K These are preset positive integer hyperparameters used to represent the model update gradient. The number of elements retained during sparsification; This represents the sparsity selection algorithm; Further compression processing is applied to sparse gradients. Perform random quantization to obtain the quantized sparse gradient. For each element j Quantized sparse gradient for: in, For elements j The sparse gradient to be quantized For elements j Quantized sparse gradient, The preset quantization step size, This indicates the floor function. This indicates the rounding up operation.

4. A federated learning system based on optimized multi-party computation according to claim 3, characterized in that: The process of performing a secure transformation on the quantized sparse gradient to obtain the transformed gradient vector, and each participating platform generating a random scalar mask and performing an encryption operation to obtain the encrypted mask parameters is as follows: Perform a stochastic orthogonal transformation on the quantized sparse gradient using a randomly generated orthogonal matrix. and random scalar mask The transformed gradient vector is calculated. : in, This is the transformed gradient vector. Randomly generated orthogonal matrix, The quantized sparse gradient, For the first i Each participating platform independently generates a random scalar mask. For all elements are 1 M 3D column vector; Using a semantically secure encryption function based on the discrete logarithm problem to mask random scalars Perform the encryption operation to obtain the encryption mask parameter. : in, The first part represents the encryption mask parameter. , This represents the shared mask used in the encryption process. This represents a group element that contains plaintext information with a mask. The second part represents the encryption mask parameter. , Encryption functions that represent semantic security; Each participating platform will transmit the transformed gradient vector. With encryption mask parameters Uploaded to the central aggregation platform.

5. A federated learning system based on optimized multi-party computation according to claim 4, characterized in that: Each participating platform generates its own identity authentication information. The specific process of using this identity authentication information to verify the identities of each participating platform through the central aggregation platform is as follows: Calculate each participating platform With the central aggregation platform S Shared secret value : in, For the first i Individual Participant Platforms private key, Indicates a central aggregation platform S public key, This indicates the calculation of the shared key. H This indicates the key-derived hash function. Used to calculate the shared secret value; Shared secret value Combined with the current timestamp Generate a hash function using the authentication token. Generate authentication token : in, Indicates the first i Identity authentication tokens for each participating platform. Indicates the first i The participating platform in the first t The current timestamp of the round-up update, Indicates the hash function used to generate the authentication token; Each participating platform Transformed gradient vector Encryption mask parameters Identity authentication token and timestamp Merged into a unified data packet Uploaded to the central aggregation platform: For each received data packet The central aggregation platform uses its own private key and storage for each participating platform public key Calculate the platform of each participating party Expected authentication token : in, Central aggregation platform S private key, For the first i Individual Participant Platforms public key, Used to calculate the shared secret value. The expected shared secret value is independently calculated by the central aggregation platform. This indicates an expectation to share the secret value. With timestamp Concatenate the first and last characters in order to form a new string; Central aggregation platform comparison i Individual Participant Platforms Identity authentication token and expected authentication token Are they the same? If they are the same, then the first... i Individual Participant Platforms Authentication passed.

6. A federated learning system based on optimized multi-party computation according to claim 5, characterized in that: The process of performing plaintext aggregation on the transformed gradient vectors of all verified participating platforms to obtain the global aggregated transformation gradient, decrypting the encrypted mask parameters of all verified participating platforms using secure multi-party computation, and performing secure aggregation on the decrypted plaintext to obtain the global masked aggregated gradient, and then performing inverse transformation on the global aggregated transformation gradient and the global masked aggregated gradient to reconstruct the global aggregated gradient is as follows: For all participating platforms that have passed the verification, extract the transformed gradient vector. The plaintext is aggregated using the aggregation weights of each participating platform to obtain the global aggregation transformation gradient. : in, Indicates the first i Individual Participant Platforms Aggregate weights; encryption mask parameters Using secure multi-party computation decryption, the central aggregation platform utilizes its own private key. calculate ,in This represents the private key of the central aggregation platform. This represents the first part of the encryption mask parameter. The first part represents the encryption mask parameter. , r i Indicates the first i Individual Participant Platforms Generated random numbers, This represents a shared masking factor used to eliminate randomness during the decryption process. This indicates that the central aggregation platform uses its private key to access the first part of the encryption mask parameters. Perform exponentiation. express The specific unfolding form; encryption mask parameters After decryption, the mask plaintext located at the exponent position is obtained. : in, This refers to the masking term introduced during the encryption process. Indicates the decryption factor; Then, the plaintext mask is solved using a discrete logarithmic algorithm. The plain text was recovered from the middle : plaintext The global mask aggregation gradient is obtained by performing weighted security aggregation. : Gradient transformation from global aggregation Subtract global mask aggregate gradient The effect is that: in, For all elements are 1 M 3D column vector, Randomly generated an orthogonal matrix; Then, the inverse of the stochastic orthogonal transformation is performed to reconstruct the global aggregate gradient. : in, Orthogonal matrix The transpose of .

7. A federated learning system based on optimized multi-party computation according to claim 1, characterized in that: The specific process by which each participating platform updates its own local model using the global aggregated gradient to obtain the updated local model parameters is as follows: The central aggregation platform will reconstruct the global aggregation gradient. The data is distributed to each participating platform, and each participating platform utilizes the global aggregation gradient. The corresponding local model is updated to obtain the updated local model parameters, using the following formula: in, For the present t Local model parameters for each round, For the updated local model parameters, For the present t Learning rate per round.

8. A federated learning system for neural network models containing nonlinear activation functions, using optimized multi-party computation, characterized in that: The local processing module is used to train the neural network model corresponding to each participating platform based on the global model parameters of the central aggregation platform to obtain the corresponding model update gradient. The model update gradient of each participating platform is compressed to obtain the quantized sparse gradient. The quantized sparse gradient is subjected to a secure transformation to obtain the transformed gradient vector. Each participating platform generates a random scalar mask and performs an encryption operation to obtain the encryption mask parameter. The input value of the nonlinear activation function in the neural network model is converted into a secret shared value. Each participating platform also generates its own identity authentication information. The gradient aggregation module is used to authenticate the identity of each participating platform through the central aggregation platform using the identity authentication information. It performs plaintext aggregation on the transformed gradient vectors of all verified participating platforms to obtain the global aggregated transformation gradient. It decrypts the encrypted mask parameters of all verified participating platforms using secure multi-party computation and performs secure aggregation on the decrypted plaintext to obtain the global mask aggregate gradient. It performs inverse transformation on the global aggregated transformation gradient and the global mask aggregate gradient to reconstruct the global aggregated sparse gradient of the neural network model. It performs polynomial aggregation computation on the secret shared value of the nonlinear activation function using an optimized multi-party computation protocol and pre-generated auxiliary data to obtain the nonlinear aggregate gradient of the neural network model. It obtains the global aggregated gradient based on the global aggregated sparse gradient of the neural network model and the nonlinear aggregate gradient of the neural network model. The gradient update module is used by each participating platform to update its own neural network model using the global aggregated gradient, thereby obtaining the updated neural network model parameters.

9. A federated learning system for neural network models containing nonlinear activation functions, as described in claim 8, characterized in that: The specific process of obtaining the global aggregated gradient includes: A set of multivariate Beaver triples related to the highest power K to be determined is generated in advance for all participating platforms and distributed in a secret sharing manner as auxiliary data. The auxiliary data format is as follows: in, For K independent random values Secret shared collection, The secret shared set of any pairwise multiplication of any two random values. For any j A secret shared set of products of random values. Secret sharing of the total product of K random values; Auxiliary data Including secret sharing of K independent random values And the secret sharing of all possible cross-product terms consisting of random values; For the input value of the non-linear activation function Each participating platform utilizes pre-shared random values For input values Perform masking and calculate the difference. Secret shared value : in, The input value represents the intermediate data processed by the non-linear activation function; It is a random value; In the single communication round, all participating platforms simultaneously publish and reconstruct the plaintext result of the difference: ; Each participating platform is based on the same publicly disclosed difference. and pre-shared auxiliary data All power terms are computed locally in parallel using linear operations. The secret shared value, in which The input value of the non-linear activation function The secret sharing result of the power of K; Each participating platform shares a secret value for each power term and preset polynomial coefficients. By performing a linear combination, the secret shared value of the calculation result of the nonlinear activation function is obtained: For the secret shared value of the nonlinear activation function, an optimized multi-party computation protocol and pre-generated auxiliary data are used to perform polynomial aggregation computation based on secret sharing to obtain the nonlinear aggregate gradient. The global aggregate sparse gradient of the neural network model is directly used for global model update, and the nonlinear aggregate gradient is used to influence the model update gradient generated by the training of the neural network model through the nonlinear activation function, thereby obtaining the global aggregate gradient.

10. A federated learning method based on optimized multi-party computation, characterized in that, It includes the following steps: Based on the global model parameters of the central aggregation platform, the local model corresponding to each participating platform is trained to obtain the corresponding model update gradient. The model update gradient of each participating platform is compressed to obtain the quantized sparse gradient. The quantized sparse gradient is subjected to a secure transformation to obtain the transformed gradient vector. Each participating platform generates a random scalar mask and performs an encryption operation to obtain the encryption mask parameter. Each participating platform also generates its own identity authentication information. The central aggregation platform uses the identity authentication information to verify the identities of each participating platform. The transformed gradient vectors of all verified participating platforms are aggregated in plaintext to obtain the global aggregation transformation gradient. The encrypted mask parameters of all verified participating platforms are decrypted using secure multi-party computation. The decrypted plaintext is then aggregated securely to obtain the global mask aggregation gradient. Finally, the global aggregation transformation gradient and the global mask aggregation gradient are reconstructed by inverse transformation to obtain the global aggregation gradient. Each participating platform updates its local model using the global aggregated gradient to obtain updated local model parameters.

Images