A privacy protection training method of a generative adversarial network under outsourcing computation

Abstract

The application discloses a privacy protection training method of a generative adversarial network under outsourcing computing and belongs to the field of privacy calculation; specifically, first, an edge communication scene including a server and a user end is built, then based on the built security protocol, the user end issues split input shares and random noise shares to the server, the server executes a security generation process in cooperation with both parties to calculate the share of the synthesized data; the real data share and the synthesized data share are respectively executed in a security discrimination process to obtain the output share of the discriminator. The server calls the built security protocol to calculate the loss share of the discriminator and the generator; then based on the ASS, the security back propagation is executed, the gradient share of each layer parameter of the neural network is calculated and post-processed, and the parameters of the generator and the discriminator are synchronously updated; finally, the server returns the generated data share to the user end, and the data share is reconstructed into plaintext locally. The application significantly reduces the online communication overhead.

Description

Technical Field

[0001] This invention belongs to the field of privacy computing and relates to a privacy-preserving training method for generative adversarial networks under outsourced computing. Background Technology

[0002] Generative Adversarial Network (GAN) Capable of synthesizing highly realistic data, it has attracted significant attention in the field of machine learning and is widely used in various areas such as image generation, data augmentation, adversarial attack generation and detection. In particular, Deep Convolutional GAN ​​(DCGAN) significantly improves the stability of the model and the quality of the generated samples by introducing a convolutional architecture.

[0003] However, building an accurate GAN model requires enormous computational and memory resources during the training phase. This challenge has led to the adoption of outsourced computing solutions, entrusting the training of GANs to third-party servers with greater processing power. However, with the continuous growth of data volume on computing servers, data privacy issues are becoming increasingly prominent. Transmitting large amounts of unencrypted sensitive datasets to these servers exposes user-end data to significant privacy risks. Therefore, there is an urgent need for effective solutions to protect data privacy.

[0004] The training process of GANs often requires direct access to users' raw data, which poses a serious risk of data leakage in privacy-sensitive scenarios such as connected vehicles, healthcare, and finance. At the same time, GAN training itself involves significant computational and communication overhead; therefore, achieving efficient training while protecting data privacy has become a core research focus.

[0005] In recent years, researchers have proposed various privacy-preserving GAN training methods from two directions: Differential Privacy (DP) and Secure Multi-Party Computation (MPC).

[0006] In research based on differential privacy, Abadi et al. were the first to propose injecting noise during the backpropagation process to limit the risk of privacy leaks. Subsequently, Xie... Xu et al. proposed DPGAN, which effectively suppresses privacy leakage and maintains model performance by introducing noise related to Wasserstein distance into the gradient. Further, researchers proposed the GANobfuscator, which performs gradient pruning and noise injection only on the discriminator side, thereby more precisely controlling privacy loss. Recently, Ma... Researchers proposed RDP-GAN, which directly introduces noise at the loss function level and combines it with an adaptive noise adjustment mechanism to reduce accuracy loss while ensuring privacy. In research based on secure multi-party computation, Yang... LP-GAN, proposed by researchers, utilizes Additive Secret Sharing (ASS) to perform secure training in a two-party environment. This method significantly reduces online communication overhead by moving random number generation to an offline stage and designs secure approximation protocols for nonlinear functions such as LeakyReLU and Sigmoid, improving scalability. Subsequently, SecGAN... With SecureGAN Replicated Secret Sharing (RSS) reduces the reliance on Beaver triples and is more computationally efficient than LP-GAN, but it is difficult to deploy directly in edge computing environments due to the need for third-party participation.

[0007] In summary, the DP method seeks a balance between noise control and model accuracy, while the MPC method reduces computational and communication burdens through protocol optimization. Although significant theoretical and experimental progress has been made, the adversarial architecture of the generator and discriminator in GAN models places enormous computational pressure on servers, and the privacy-preserving training process itself involves substantial data communication between servers. Therefore, how to efficiently train GAN models while protecting data privacy remains a challenging problem in the current technological context.

[0008] The state-of-the-art solution LP-GAN is a privacy-preserving GAN framework based on two-party computation and secret sharing, such as... Figure 1 As shown, LP-GAN can train a GAN without compromising privacy. However, its core protocol design has inherent flaws, leading to significant online computation and communication overhead, specifically in the following aspects:

[0009] (1) The SecCmp security comparison protocol of LP-GAN requires iterative computation of the secure AND gate operation to obtain the final comparison result. This design introduces multiple rounds of interaction, which is the main source of performance overhead in the online phase. The protocol requires additional communication rounds to handle special cases. For example, when the LP-GAN comparison protocol processes certain input values ​​(e.g., when the input is not greater than zero), it will produce a preliminary result that is the opposite of the ideal value. The protocol requires both parties to communicate additionally to correct this error, which directly leads to an increase in communication overhead.

[0010] (2) The inefficient comparison mechanism described above makes the safe computation of key nonlinear functions (such as LeakyReLU and BCELoss) a performance bottleneck. Since the computation of LeakyReLU and the handling of extreme values ​​in BCELoss both rely heavily on the above-mentioned high-overhead safe comparison protocol, huge communication overhead is generated, which becomes the performance bottleneck of the entire framework.

[0011] (3) Poor overall performance, excessively long training time, and lack of practicality. The aforementioned defects at the protocol level are ultimately reflected in the overall performance. Experimental data shows that LP-GAN requires tens of hours to complete the training of the entire dataset, which highlights the severe challenges it faces in terms of communication and computational overhead, making it difficult to deploy in real-world applications that require timeliness.

[0012] The references are as follows:

[0013] [1]I.Goodfellow, J. Pouget-Abadie, M.Mirza, B.Xu, D.Warde-Farley, S.Ozair, A.Courville, andY.Bengio, "Generative adversarial networks," Commun.ACM, vol.63, no.11, pp.139–144, 2020.

[0014] [2]M. Abadi, A. Chu, I. Goodfellow, HB McMahan, I. Mironov, K.Talwar, and L. Zhang, “Deep Learning with Differential Privacy,” inProceedings of the 2016 ACM SIGSAC Conference on Computer and CommunicationsSecurity, 2016, pp. 308–318.

[0015] [3]L. Xie, K. Lin, S. Wang, F. Wang, and J. Zhou, “DifferentiallyPrivate Generative Adversarial Network,” ArXiv, 2018.

[0016] [4]C. Xu, J. Ren, D. Zhang, Y. Zhang, Z. Qin, and K. Ren, “GANobfuscator: Mitigating Information Leakage Under GAN via Differential Privacy,”IEEE Transactions on Information Forensics and Security, vol. 14, no. 9, pp.2358–2371, 2019.

[0017] [5]C. Ma, J. Li, M. Ding, B. Liu, K. Wei, J. Weng, and H. V. Poor,“RDP-GAN: A R´ enyi-Differential Privacy Based Generative AdversarialNetwork,” IEEE Transactions on Dependable and Secure Computing, vol. 20, no.6, pp. 4838–4852, 2023.

[0018] [6]Y. Yang, K. Mu, and R. H. Deng, “Lightweight Privacy-PreservingGAN Framework for Model Training and Image Synthesis,” IEEE Transactions onInformation Forensics and Security, vol. 17, pp. 1083 1098, 2022.

[0019] [7] Y. Yang, L. Liu, S. Fu, J.-J. Huang, and Y. Luo, “SecGAN: HonestMajority Maliciously 3PC Framework for Privacy-Preserving Image Synthesis,” in 2023 IEEE 22nd International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), 2023, pp. 1119–1126.

[0020] [8] S. Cao, H. Zhang, Y. Wang, J. Lin, F. Kong, and L. Yu, “SecureGAN: Secure Three-Party GAN Training,” in 2023 32nd International Confer ence on Computer Communications and Networks (ICCCN), 2023, pp. 1–7. Summary of the Invention

[0021] To address the problems of low efficiency and high communication and computational overhead in the online phase of existing technologies (especially LP-GAN schemes) when handling nonlinear operations, this invention proposes a privacy-preserving training method for generative adversarial networks under outsourced computation. Based on the fast privacy-preserving GAN training framework of Function Secret Sharing (FSS), it constructs an efficient, practical, and secure training scheme without sacrificing model accuracy.

[0022] The privacy-preserving training method for generative adversarial networks under outsourced computation comprises the following steps:

[0023] Step 1: Set up two servers that do not collude with each other. and And edge communication computing scenarios on the user end;

[0024] The user is a trusted entity. After inputting a plaintext dataset and random noise, it secretly shares the data to obtain two input shares. and Send to the server respectively and ;

[0025] The two servers use neural network models, and each initializes its generator share. Share with the discriminator The neural network model is jointly trained under a secure multi-party computation protocol. The server returns its respective data share to the user, who then reconstructs it into plaintext locally.

[0026] Step 2: Set up the secure LeakyReLU protocol and the secure BCELoss protocol used for the training server;

[0027] Each protocol is divided into an offline phase and an online phase. In the offline phase, the user generates the key and random materials such as Beaver triples required for secure computation and inputs them into the server. In the online phase, the server uses the offline resources to perform hierarchical secure convolution (Conv), transposed convolution (TConv), batch normalization (BN), and nonlinear activation operations, and ensures that all intermediate values ​​are kept in a secret shared state throughout the entire forward and backward propagation.

[0028] Operations such as convolution, transposed convolution, and batch normalization are evaluated using Additive Secret Sharing (ASS). Nonlinear operations like ReLU and LeakyReLU are handled by Function Secret Sharing (FSS). Function Secret Sharing (FSS) includes a key generation function. and evaluation function ;

[0029] The specific process of the offline phase is as follows:

[0030] The user first generates a random number r as a mask and constructs a key parameter vector. , where s is the negative slope of LeakyReLU;

[0031] Then, the user calls the key generation function. , using parameters and Generate a pair of function keys .

[0032] Finally, the user terminal will share the DCF key. Share of random mask and the share of the parameter vector Packaged into a single key packet Distribute to the corresponding server (where j is 0 or 1).

[0033] The specific process of the online phase is as follows:

[0034] Two servers and Upon receiving the input share After a round of collaborative communication, the input value hidden by the random mask r is reconstructed. .

[0035] Subsequently, each server Call the evaluation function independently on the local machine. Use your own key and public value Determine whether The judgment result is encoded in the output share in the form of a secret sharing mechanism. middle.

[0036] Finally, each server Use the result from the previous step Vector shares, calculating the final share expression ;

[0037] The above expression utilizes As a flag: if The value of the expression converges to the share of x; if Then converges to . share.

[0038] Step 3: The user sends the split input share and random noise share to the server to obtain the noise vector of the secret sharing. and the actual share of data The two servers collaborate to execute a secure generation process and calculate the share of the synthesized data. ;

[0039] Step 4: Both servers, based on the secure LeakyReLU protocol, determine the actual data share. and synthetic data share The security discrimination process is executed separately to obtain the discriminator's output share for the two types of data. .

[0040] Step 5: Based on the secret sharing state, the server calls the secure BCELoss protocol to calculate the share of discriminator loss and generator loss respectively;

[0041] The formula for the loss share of the discriminator is:

[0042]

[0043] The formula for the generator's loss share is:

[0044] .

[0045] Step Six: Both servers perform secure backpropagation, using Additive Secret Sharing (ASS) and locally computed Jacobian matrices or partial derivatives to calculate the gradient share of each layer's parameters in the neural network. And perform safety gradient pruning and normalization;

[0046] Safe gradient clipping refers to the L2 norm of gradient calculation using a safe multiplication protocol. and with the preset maximum norm threshold Perform a safety comparison; if the norm exceeds the threshold, reduce the gradient share proportionally to prevent gradient explosion.

[0047] Safe gradient normalization refers to the process by which a safe multiplication protocol normalizes gradient fractions to unit vectors, keeping their direction unchanged while maintaining uniform scale.

[0048] Step 7: Apply safe gradient descent to the post-processed gradient shares to synchronously update the generator holdings of both parties. and discriminator The model parameter share, the server will generate the data share The data is returned to the user's device, where it is reconstructed into plaintext locally.

[0049] The advantages of this invention are:

[0050] 1. A high-efficiency secure LeakyReLU protocol is constructed using FSS technology. Based on the Distributed Comparison Function (DCF), the protocol generates function-based keys offline through a preprocessing mechanism, transferring a large amount of computation from the online stage to the offline stage. Compared with the original FSS scheme, the secure LeakyReLU protocol proposed in this invention only requires one call to DCF, reducing communication and computation rounds, thereby greatly improving the online evaluation efficiency of the activation function.

[0051] 2. By utilizing FSS technology to construct an efficient and secure BCELoss protocol, a secure input value processing mechanism was designed. This mechanism can effectively handle extreme input values ​​approaching 0 or 1 without compromising privacy, thereby solving the gradient explosion or vanishing problem commonly encountered in fixed-point arithmetic and ensuring the stability of the training process.

[0052] 3. This invention organically integrates the proposed high-efficiency LeakyReLU and BCELoss protocols with ASS-based linear operation protocols to form a complete end-to-end hybrid protocol framework. Attached Figure Description

[0053] Figure 1This is a flowchart of a privacy-preserving training method for generative adversarial networks under outsourced computation, according to the present invention.

[0054] Figure 2 This is a system architecture diagram for an edge communication computing scenario built according to the present invention;

[0055] Figure 3 This is a schematic diagram of the secure multi-party computation protocol built for this invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0057] The privacy-preserving training method for generative adversarial networks under outsourced computation aims to address the efficiency problem of existing solutions being difficult to deploy in resource-constrained environments such as edge computing due to the huge communication and computational overhead. Figure 1 As shown, the specific steps are as follows:

[0058] Step 1: Set up two servers that do not collude with each other. and And edge communication computing scenarios on the user end;

[0059] The client is a trusted entity responsible for holding the plaintext dataset and the random noisy input, and for secretly sharing these inputs to obtain two shares. and Send to the server respectively and .

[0060] The two servers use neural network models, and each initializes the proportion of the generator and discriminator. and The neural network model is jointly trained under a secure multi-party computation protocol. The server returns its respective data share to the user client, which then reconstructs it into plaintext locally.

[0061] The interaction between the client and the server mainly involves the secret sharing of data transmission. The client sends the split input segments and random noise segments to the server, while the servers exchange necessary intermediate results through a secure channel to complete the protocol calculations.

[0062] Step 2: Set up the secure LeakyReLU protocol and the secure BCELoss protocol used for the training server;

[0063] Each protocol is divided into an offline phase and an online phase. In the offline phase, the user generates the keys and random materials such as Beaver triples required for secure computation and inputs them into the server; this process is independent of the actual data. In the online phase, the server receives the input shares and performs the actual secure computation. The server utilizes offline resources to perform hierarchical secure convolutions (Conv), transposed convolutions (TConv), batch normalization (BN), and nonlinear activation operations, ensuring that all intermediate values ​​throughout the forward and backward propagation remain secretly shared.

[0064] Operations such as convolution, transposed convolution, and batch normalization are evaluated using Additive Secret Sharing (ASS). Nonlinear operations like ReLU and LeakyReLU are handled by Function Secret Sharing (FSS). Function Secret Sharing (FSS) includes a key generation function. and evaluation function ;

[0065] The specific process of the offline phase is as follows:

[0066] The user first generates a random number r as a mask and constructs a key parameter vector. , where s is the negative slope of LeakyReLU;

[0067] Then, the user calls the key generation function. , using parameters and Generate a pair of function keys .

[0068] Finally, the user terminal will share the DCF key. Share of random mask and the share of the parameter vector Packaged into a single key packet Distribute to the corresponding server (where j is 0 or 1).

[0069] The specific process of the online phase is as follows:

[0070] Two servers and Upon receiving the input share After a round of collaborative communication, the input value hidden by the random mask r is reconstructed. .

[0071] Subsequently, each server Call the evaluation function independently on the local machine. Use your own key and public value Determine whether The judgment result is encoded in the output share in the form of a secret sharing mechanism. middle.

[0072] Finally, each server Use the result from the previous step Vector shares, calculating the final share expression ;

[0073] The above expression utilizes As a flag: if The value of the expression converges to the share of x; if Then converges to . share.

[0074] Step 3: The user sends the split input share and random noise share to the server to obtain the noise vector of the secret sharing. and the actual share of data The two servers collaborate to execute a secure generation process and calculate the share of the synthesized data. ;

[0075] Step 4: Both servers, based on the secure LeakyReLU protocol, determine the actual data share. and synthetic data share The security discrimination process is executed separately to obtain the discriminator's output share for the two types of data. .

[0076] Step 5: Based on the secret sharing state, the server calls the secure BCELoss protocol to calculate the share of discriminator loss and generator loss respectively;

[0077] The formula for the loss share of the discriminator is:

[0078]

[0079] The formula for the generator's loss share is:

[0080] .

[0081] Step Six: Both servers perform secure backpropagation, using Additive Secret Sharing (ASS) and locally computed Jacobian matrices or partial derivatives to calculate the gradient share of each layer's parameters in the neural network. And perform safety gradient pruning and normalization;

[0082] To prevent gradient explosion and numerical anomalies, a safe GAN training method is introduced, which uses ibDCF-based safe interval pruning and logarithmic calculation mechanism. Stable parameter updates in dense states are achieved through safe multiplication, gradient pruning and normalization protocol.

[0083] Safe gradient clipping refers to the L2 norm of gradient calculation using a safe multiplication protocol. and with the preset maximum norm threshold Perform a safety comparison; if the norm exceeds the threshold, reduce the gradient share proportionally to prevent gradient explosion.

[0084] Safe gradient normalization refers to the process by which a safe multiplication protocol normalizes gradient fractions to unit vectors, keeping their direction unchanged while maintaining uniform scale.

[0085] Step 7: Apply safe gradient descent to the post-processed gradient shares to synchronously update the generator holdings of both parties. and discriminator The model parameter share, the server will generate the data share The data is returned to the user's device, where it is reconstructed into plaintext locally.

[0086] Example:

[0087] This invention proposes a privacy-preserving training method for GAN models under edge computing, which includes two types of entities: computing servers and users. The system architecture is as follows: Figure 2 As shown:

[0088] (1) Computing server: Two servers that do not collude with each other process the user’s training requests according to a customized protocol and provide the user with image shares generated based on the trained model.

[0089] (2) User: The user is a trusted entity. The GAN model and input are divided into shares using ASS technology. These shares are then provided to two servers to provide privacy-preserving training services. The computing server returns the shares of the generated images to the user, and the user reconstructs them into plaintext locally.

[0090] The FPGAN solution proposed in this invention enables server-side implementation. and The main protocols involved in safe GAN training, such as Figure 3As shown on the right, the red dashed line indicates that data always flows securely within the protocol in the form of secret sharing. Specifically, each party performs layer-by-layer secure computation, and each protocol is divided into an offline phase (for key and triple generation) and an online phase (for security evaluation). This example employs a hybrid protocol design: operations such as convolution (Conv), transposed convolution (TConv), and batch normalization (BN) are evaluated via additive secret sharing (ASS). Nonlinear operations like ReLU and LeakyReLU are handled by function secret sharing (FSS). Furthermore, the BCELoss loss function is also securely computed via FSS and used to update the model. and Therefore, by enabling secure computation for each layer of the model, the FPGAN scheme allows a GAN model to be used in both directions. They are trained and used safely.

[0091] For nonlinear operations in the model (such as the LeakyReLU activation function and the BCELoss loss function), this scheme introduces an efficient protocol based on FSS. To clarify the subsequent protocol description, FSS and its two key primitives used in this scheme are formally defined here:

[0092] (1) Function Secret Sharing (FSS)

[0093] FSS is a technique that splits a function f into two function shares. and Cryptographic techniques enable the encryption of any publicly available input. , And any single function share No information about the function will be disclosed. Any information. A two-party FSS scheme typically includes the following two core algorithms:

[0094] Key generation algorithm This is an algorithm that runs offline. It takes a security parameter λ and a description of the function f as input and outputs a pair of function keys. Each key , The function share is implicitly defined. .

[0095] Function evaluation algorithm This is an algorithm that runs in the online phase. Participants Receive your own key share Given a public input x, compute and output the result share locally and independently. ,in The only communication overhead during the online phase is used solely for the final reconstruction result. .

[0096] (2) Distributed Comparison Function (DCF)

[0097] DCF is a concrete implementation of FSS used to securely compare a public input. and In this scheme, it is used to build an efficient and secure LeakyReLU protocol. Its core functionality can be described as: for a given... and Function output (if ) or 0 (if The corresponding generation and evaluation algorithms are: and .

[0098] To protect input Introducing an offset function, in the form of: In other words, the input consists of an offset. The mask, and the output is determined by the offset. Mask. (In the following text,) Indicates input The offset mask is the secret value, i.e. .

[0099] (3) Incremental Boolean DCF (ibDCF)

[0100] ibDCF is a key-size-optimized, one-dimensional DCF variant used to efficiently determine whether an input value is greater than 0. It is the underlying tool for building the secure BCELoss protocol in this scheme. It outputs 1 (if x > 0) or 0 (if x ≤ 0). Its two included algorithms are defined as follows:

[0101] Key generation algorithm This is a randomized key generation algorithm. It accepts a security parameter λ and an input mask. Output mask The input consists of a partition point α (an 1-bit value) and an output bit β, and the output is a pair of function keys. .

[0102] Function evaluation algorithm This is a deterministic evaluation algorithm. It receives the participant index j, the key share kⱼ, and the input mask. Hidden input value As input, output a result share. .

[0103] After formally defining FSS and the core cryptographic primitives upon which this scheme relies, the next step will be to utilize these building blocks to implement a complete end-to-end privacy-preserving GAN training process, divided into two core stages: offline deployment and online computation.

[0104] (1) Offline deployment stage.

[0105] First, in order to provide the service, users need to pre-deploy the encrypted parameters of the GAN model to the server. Specifically, users need to execute ASS to encrypt and distribute the model parameters to two non-colluding servers to complete the deployment. This solution constructs a LeakyReLU activation function based on DCF and a BCELoss loss function using ibDCF. Simultaneously, the random number mask needed for multiplication operations and data transmission is also generated at this stage.

[0106] (2) Construction of LeakyReLU activation function based on DCF.

[0107] The secure LeakyReLU protocol is used to compute LeakyReLU activation functions while preserving privacy. Based on DCF (Distributed Dynamics Function), its core objective is to complete the computation using only one round of online communication and one DCF function call, significantly outperforming existing schemes that require multiple rounds of communication.

[0108] The implementation of this protocol relies on the following two key innovations, which are specifically defined in Algorithm 1:

[0109] a. Offline phase

[0110] This stage, performed by the user or a trusted entity before online computation begins, generates all the keys and random numbers required for subsequent computation. The user first generates a random number *r* as a mask and constructs a key parameter vector. , where s is the negative slope of LeakyReLU. This β vector precodes the branching logic of the LeakyReLU function (i.e., multiplying by 1 when the input is greater than 0, otherwise multiplying by s) into the cryptographic parameters. Client-side call Using the above parameters and Generate a pair of function keys Finally, the user terminal will share the DCF key. Share of random mask and the share of the parameter vector Packaged into a single key packet Distribute to the corresponding server (where j is 0 or 1).

[0111] b. Online phase

[0112] This stage involves two servers. and Upon receiving the input share Post-cooperative execution. Server and The first round of communication aims to safely reconstruct the input value hidden by the random mask r. Since r is random, it is public. No information about the original input will be disclosed. The information. Subsequently, each server... Call independently on the local machine Use your own key and public value Perform the calculation. This step requires no communication and is designed to ensure safe completion. The judgment is made, and the judgment result is encoded in the output share in the form of a secret sharing. middle.

[0113] Finally, each server Use the result from the previous step Vector shares are calculated using a predefined, branchless arithmetic expression to determine the final share. This expression utilizes (Right now The judgment result is used as a flag: if The value of the expression converges to the share of x; if Then converges to The share. The remaining items. and This is used to offset the random masks and parameters introduced during the calculation process, ensuring the correctness of the results.

[0114] Through the above design, the secure LeakyReLU protocol transforms complex condition judgments into efficient offline key generation and online computation of single-round communication, greatly improving the processing efficiency of nonlinear activation functions in privacy-preserving GAN training.

[0115]

[0116] (3) Construction of BCELoss loss function based on FSS and ASS hybrid protocol.

[0117] In privacy-preserving GAN training, the use of fixed-point precision makes the calculation of loss values ​​highly susceptible to numerical instability. For example, when the input value is very close to 0 or 1, and Terms may tend towards extreme values ​​or produce zero gradients, leading to gradient explosion or gradient vanishing. This phenomenon severely affects training stability and convergence speed. To address this issue, this example proposes an efficient and numerically stable safe BCELoss protocol.

[0118] This protocol is based on the ibDCF primitive in FSS and ensures computational robustness by introducing a safe input truncation mechanism. The specific implementation of this protocol is defined in detail in Algorithm 2.

[0119] a. Offline phase

[0120] At this stage, the client generates all the necessary ibDCF function keys for subsequent online input truncation operations.

[0121] The user receives the input for secret sharing. For each input, the key generation algorithm is called for each of the two numerical stability boundaries. Generate two sets of keys: one set of keys. Used for online phase judgment Is it less than Another set of keys Used for judgment Is it greater than The client packages these generated keys along with other random number shares and distributes them to the corresponding servers. .

[0122] b. Online phase

[0123] This stage is handled by the server. and Upon receiving the input share Post-cooperative execution mainly includes two core steps:

[0124] Safe input truncation: This is a crucial step in ensuring numerical stability. Its goal is to truncate all input values. All are safely confined within the interval. Inside. First, for each input... The server performs a round of communication to reconstruct the masked value. Next, each server The evaluation algorithm is invoked locally and independently using the ibDCF key generated in the offline phase. Two calls. The first call calculates a flag share. , used to indicate Is it less than The second call calculates another flag share. , used to indicate Is it greater than The plaintext flags are exchanged and reconstructed on the server. and Then, they safely update the input shares using a branchless arithmetic expression. .if If true, Safely replaced with ;if If true, it will be replaced with Otherwise, remain unchanged.

[0125] Safe Loss Calculation: After all input values ​​have been safely truncated to a stable range, the server begins calculating the final loss. The protocol checks for a skip flag. If it is False, the calculation continues. The server initializes a loss accumulator share. The result is 0. For each sample i, the server invokes the secure multiplication protocol and the secure logarithm protocol to calculate... and The share, and accumulated to middle.

[0126] Finally, invert the summed total loss share and divide it by the sample size n to obtain the final BCELoss share. .

[0127] (4) End-to-end GAN security training.

[0128] By organically integrating the aforementioned security protocol modules (such as secure LeakyReLU and secure BCELoss), a complete end-to-end GAN privacy-preserving training scheme was constructed.

[0129] The security training process is an iterative cycle, mainly including the following steps:

[0130] i. Secure forward propagation: Given a secret-shared noise vector The two servers collaborate to execute a secure generation process, securely calculating the share of the synthesized image. Subsequently, the two servers agreed on the actual image share. and the share of the synthesized image The security discrimination process is executed separately to obtain the discriminator's output share for the two types of images. .

[0131] ii. Security Loss Calculation: The server calls the security BCELoss protocol proposed in this invention to calculate the discriminator loss respectively. and the share of generator loss The entire calculation process is conducted in a completely secret and shared state, without revealing any intermediate values ​​or the final loss value.

[0132] iii. Secure Backpropagation and Update: After obtaining the loss share, both servers execute the secure backpropagation process. Unlike forward propagation, this process does not invoke complex function security protocols. Instead, it primarily utilizes an efficient secure multiplication protocol, combined with locally computed Jacobian matrices or partial derivatives, to implement the chain rule and securely calculate the gradient share of each layer's parameters. After obtaining the original gradient shares... Afterwards, to ensure the stability of the training, two additional safety post-processing steps will be performed in sequence.

[0133] iv. Safe gradient clipping: This protocol safely computes the L2 norm of the gradient. and with a preset maximum norm threshold Perform a safety comparison. If the norm exceeds a threshold, scale down the gradient share proportionally to prevent gradient explosion.

[0134] v. Safe gradient normalization: This protocol further normalizes gradient shares to unit vectors, keeping their direction unchanged while maintaining a uniform scale, thereby improving training stability.

[0135] Finally, using the post-processed gradient shares, the generators held by both parties are synchronously updated via safe gradient descent. and discriminator The proportion of model parameters.

[0136] (5) Return of complete process results.

[0137] In the privacy-preserving computation framework proposed in this example, all core computation results (e.g., the loss value for each round of training, or the final generated synthetic image) are always stored separately on two computation servers in a secret-sharing manner. and Above. To obtain the final plaintext result, the process is as follows:

[0138] The server returns the share: After the computation task is completed, the server... its share of generated images The result is returned to the user. Meanwhile, the server... It also holds the corresponding share of generated images. Returned to the same user.

[0139] Client-side local reconstruction: After receiving the shares from the two servers, the client utilizes the homomorphic property of additive secret ASS to perform a simple addition operation on the shares locally, thus accurately reconstructing the final plaintext result. For example, for plaintext generated images... The reconstruction process is as follows: .

[0140] This process ensures that during the entire end-to-end computation and transmission process, no single server or attacker with eavesdropping channels can obtain any valid information about sensitive data (such as training loss, generated images), and only the user holding all shares can recover the final result, thus guaranteeing the integrity and security of the process.

[0141] Compared with existing technologies, especially the state-of-the-art LP-GAN schemes, the privacy-preserving GAN training method proposed in this invention achieves significant technical results without sacrificing model accuracy by introducing a series of innovative FSS-based protocols. These results are specifically reflected in the following aspects:

[0142] 1. Fundamental improvements in core protocol performance

[0143] This core security protocol far surpasses existing solutions in terms of computational and communication efficiency, fundamentally solving the performance bottleneck of nonlinear function processing.

[0144] Secure LeakyReLU Protocol: Compared to the protocol that LP-GAN relies on, which requires multiple rounds of interaction, the secure LeakyReLU protocol proposed in this invention significantly reduces communication overhead and significantly improves computation speed.

[0145] Secure BCELoss Protocol: Compared with the solution that directly uses existing high-level functions, the secure BCELoss protocol proposed in this invention achieves significant optimization in both communication and computation efficiency.

[0146] 2. End-to-end training efficiency achieved by orders of magnitude

[0147] By integrating the aforementioned efficient protocols into an end-to-end framework, this invention achieves a breakthrough improvement in overall training efficiency, making privacy-preserving training, which was previously impractical due to its excessive time consumption, feasible.

[0148] (1) Training time is greatly reduced: This solution shortens the training process that originally required tens of hours or even days to a more practical and acceptable time range, achieving an order-of-magnitude improvement in computational efficiency.

[0149] (2) Sharply reduced communication overhead: This solution reduces the total communication volume by an order of magnitude, greatly alleviating the network bandwidth pressure between servers, and is especially suitable for application scenarios with limited network resources, such as edge computing.

[0150] 3. Maintaining training accuracy

[0151] This invention is based entirely on lossless cryptography, without introducing any noise or approximations that could lead to a decrease in accuracy. Experimental results show that the model securely trained using this method achieves training accuracy comparable to that of a model trained directly on plaintext data, reaching a practical level.

[0152] In summary, this method successfully addresses the bottleneck of inefficiency in existing privacy-preserving GAN schemes, achieving orders-of-magnitude improvements in training speed and communication overhead while ensuring model performance. It provides a feasible technical path for deploying practical privacy-preserving data synthesis applications in resource-constrained environments such as edge computing.

[0153] Regarding privacy protection, this solution achieves end-to-end privacy protection for the entire GAN training process without sacrificing model accuracy through the following core technological innovations: A highly efficient and secure LeakyReLU protocol, a core technology, is built upon DCF and requires only a single DCF call and one round of online communication to complete computation, fundamentally solving the performance bottleneck caused by existing solutions relying on multiple rounds of secure comparison. Another core technology is a highly efficient and numerically stable secure BCELoss protocol, which incorporates a secure input truncation mechanism to effectively prevent gradient explosion / vanishing problems while maintaining privacy. It innovatively combines two secure computation primitives, utilizing ASS to efficiently handle linear operations such as convolution, and FSS to efficiently handle nonlinear operations such as LeakyReLU and BCELoss. This solution achieves orders-of-magnitude improvements in training speed and communication overhead, overcoming the performance bottlenecks of existing technologies.

[0154] The technical terms are explained below:

[0155] MPC: Secure Multi-Party Computation. A cryptographic system that allows multiple untrusted parties to collaboratively compute a function without revealing their private inputs.

[0156] SS: Secret Sharing. A core technology used in MPC that splits a secret value into multiple shares and distributes them to different participants.

[0157] ASS: Additive Secret Sharing. A basic secret sharing scheme, using a secret value. Split into multiple shares This makes the sum of all shares equal to the original secret value (e.g., in a two-party scenario). = + It forms the basis of the LP-GAN scheme and the linear operation part of this invention.

[0158] FSS: Function Secret Sharing. An advanced secret-sharing technique that shares a function's secrets. It is itself split into multiple function shares , ..., each party uses its own function share and the public input x to calculate the result share. The final function evaluation result can be obtained by adding these results together. .

[0159] DCF: Distributed Comparison Function. A cryptographic primitive based on Function Secret Sharing (FSS) for securely determining whether a secretly shared input value is greater than a publicly disclosed threshold without exposing the input value itself. It forms the basis for constructing the efficient and secure LeakyReLU protocol in this invention.

[0160] ibDCF: Incremental Boolean DCF. A one-dimensional, key-size-optimized variant of the distributed comparison function (DCF) used to determine whether an input value is greater than 0.

[0161] GAN: Generative Adversarial Network. A machine learning model consisting of a generator and a discriminator, which generate highly realistic data through adversarial training between the two.

[0162] LeakyReLU: Leaky Rectified Linear Unit. An activation function that, unlike the standard ReLU function, allows a small non-zero gradient when the input is negative, to avoid the "death" problem of neurons. Its definition is as follows:

[0163] (1)

[0164] BCELoss: Binary Cross-Entropy Loss. A commonly used loss function in binary classification tasks, used to calculate adversarial training loss. The definition of BCELoss is as follows:

[0165] (2).

Claims

1. A privacy-preserving training method for generative adversarial networks under outsourced computation, characterized in that, The specific steps are as follows: Step 1: Set up two servers that do not collude with each other. and And edge communication computing scenarios on the user end; Step 2: Set up the secure LeakyReLU protocol and the secure BCELoss protocol used for the training server; Each protocol is divided into an offline phase and an online phase; During the offline phase, the user generates the key and Beaver triple random material required for secure computation and inputs them into the server; During the online phase, the server utilizes offline resources to perform hierarchical secure convolution, transposed convolution, batch normalization, and nonlinear activation operations, and ensures that all intermediate values ​​throughout the forward and backward propagation are always in a state of secret sharing. The specific process of the offline phase is as follows: The user first generates a random number r as a mask and constructs a key parameter vector. , where s is the negative slope of LeakyReLU; Then, the user calls the key generation function. , using parameters and Generate a pair of function keys ; Finally, the user terminal will share the DCF key. Share of random mask and the share of the parameter vector Packaged into a single key packet Distribute to the corresponding server j is 0 or 1; The specific process of the online phase is as follows: Two servers and Upon receiving the input share After a round of collaborative communication, the input value hidden by the random mask r is reconstructed. ; Subsequently, each server Call the evaluation function independently on the local machine. Use your own key and public value Determine whether The judgment result is encoded in the output share in the form of a secret sharing mechanism. middle; Finally, each server Use the result from the previous step Vector shares, calculating the final share expression The above expression utilizes As a flag: if The value of the expression converges to the share of x; if Then converges to The share; Step 3: The user sends the split input share and random noise share to the server to obtain the noise vector of the secret sharing. and the actual share of data The two servers collaborate to execute a secure generation process and calculate the share of the synthesized data. ; Step 4: Both servers, based on the secure LeakyReLU protocol, determine the actual data share. and synthetic data share The security discrimination process is executed separately to obtain the discriminator's output share for the two types of data. ; Step 5: Based on the secret sharing state, the server calls the secure BCELoss protocol to calculate the share of discriminator loss and generator loss respectively; Step Six: Both servers perform secure backpropagation, using additive secret sharing (ASS) and locally calculated Jacobian matrices or partial derivatives to calculate the gradient share of each layer's parameters in the neural network. And perform safety gradient pruning and normalization; Step 7: Apply safe gradient descent to the post-processed gradient shares to synchronously update the generator holdings of both parties. and discriminator The model parameter share, the server will generate the data share The data is returned to the user's device, where it is reconstructed into plaintext locally.

2. The privacy-preserving training method for generative adversarial networks under outsourced computation as described in claim 1, characterized in that, In step one, the user is a trusted entity that inputs a plaintext dataset and random noise, then secretly shares the dataset to obtain two input shares. and Send to the server respectively and ; The two servers use neural network models, and each initializes its generator share. Share with the discriminator The neural network model is jointly trained under a secure multi-party computation protocol. The server returns its respective data share to the user, who then reconstructs it into plaintext locally.

3. The privacy-preserving training method for generative adversarial networks under outsourced computation as described in claim 1, characterized in that, In step two, convolution, transposed convolution, and batch normalization operations are evaluated using additive secret sharing (ASS); while nonlinear operations such as ReLU and LeakyReLU are handled by function secret sharing (FSS). The function secret sharing (FSS) includes a key generation function. and evaluation function .

4. The privacy-preserving training method for generative adversarial networks under outsourced computation as described in claim 1, characterized in that, The formula for the loss share of the discriminator in step five is: The formula for the generator's loss share is: 。 5. The privacy-preserving training method for generative adversarial networks under outsourced computation as described in claim 1, characterized in that, In step six, the safe gradient clipping refers to: calculating the L2 norm of the gradient using the safe multiplication protocol. and with the preset maximum norm threshold Perform a safety comparison; if the norm exceeds the threshold, reduce the gradient share proportionally to prevent gradient explosion. Safe gradient normalization refers to the process by which a safe multiplication protocol normalizes gradient fractions to unit vectors, keeping their direction unchanged while maintaining uniform scale.

Images