Nth power neural network training method based on shamir secret sharing over galois ring

Abstract

A kind of square neural network training method based on Shamir secret sharing on Galois ring, in preparatory stage, secret data is embedded into a ring element in one group by one party, secret sharing is carried out, and random shared ciphertext needed by protocol and the sharing ciphertext of initialization target parameter are calculated in advance;In the training stage, the model privacy training is carried out through local calculation and square cooperation alternately, and after the training is completed, one party discloses the sharing ciphertext of target parameter.The present application realizes the parallel computation of part of training process by embedding multiple secrets into the same Galois ring element, and can further improve the efficiency.

Description

Technical Field

[0001] This invention relates to a technology in the field of information security, specifically a technology based on Shamir secret sharing on the Galois ring. Square neural network training method. Background Technology

[0002] Secure multi-party computation is a key primitive in cryptography, where multiple participants jointly compute a public function using their respective data while protecting the privacy of their private data. This can be used to implement privacy-preserving machine learning. However, existing privacy-preserving machine learning techniques suffer from the following main problems: 1) Most techniques only consider computational security and cannot resist adversaries with unlimited computing resources; 2) Most techniques only support a specific number of participants and cannot effectively scale to accommodate more participants. These issues limit the application of secure multi-party computation techniques in privacy-preserving machine learning. Summary of the Invention

[0003] This invention addresses the shortcomings of existing technologies in achieving information-theoretic security through computation, proposing a method based on Shamir secret sharing on a Galois ring. A method for training square neural networks. By embedding multiple secrets into the same Galois ring element, parallel computation of part of the training process can be achieved, further improving efficiency.

[0004] This invention is achieved through the following technical solution:

[0005] This invention relates to a method for Shamir secret sharing on a Galois ring. Square neural network training method, in the preparatory stage Each participating party will use secret data Each element is embedded into a ring element for secret sharing, and the random sharing ciphertext needed for the protocol and the sharing ciphertext for initializing the target parameters are pre-computed; during the training phase, local computation and... The parties collaborate and interleave to train the model for privacy, and after training is complete... Each participating party publicly shares encrypted target parameters.

[0006] The aforementioned secret sharing refers to: for those located in the ring In Input data after fixed-point encoding First, it is embedded into a Galois ring. elements in In the middle, satisfy ,in: or Then generate Shamir's Secret Sharing of Cipher Text When according to The way in Embedded in A secret When sharing encrypted text Represented as ,in: belong .

[0007] When a secret is shared, the secret holder generates a Galois ring. polynomial of degree ,in: The secret value; other parameters To generate random numbers, Distributed to the Each participant shares their share, among which: For one of the Galois rings containing The exception set contains non-zero and distinct elements. When it is necessary to reconstruct the secret, first collect the shares of all participants, and then calculate using the Lagrange interpolation formula. Obtain the secret value; when the secret value is... At that time, the corresponding sharing is .

[0008] The protocol requires the use of randomly shared ciphertext, which is used to initialize the weights of the neural network. and bias terms ,in: This represents the current network layer number.

[0009] The aforementioned collaborative interleaved model privacy training specifically includes:

[0010] Step 1, Forward Propagation: ,in: This represents the activation function, which yields the output of the last layer. Then, calculate the objective function. Its derivative Based on Calculate the derivative of the target parameters for each layer. Then calculate the objective function. Corresponding to parameters gradient .

[0011] Step 2, for those containing The gradients corresponding to all samples in the training batch of each sample are aggregated to obtain the global gradient prediction value. (Share ciphertext) and Specifically, it includes:

[0012] 2.1 Weight Aggregation: For the calculated weights... Each sample corresponds to the target parameter. Gradient sharing ciphertext Calculate ciphertext That is, , ;

[0013] 2.2 Bias Aggregation: For the calculated Each sample corresponds to the target parameter. Gradient sharing ciphertext Calculate ciphertext That is .

[0014] The weight aggregation specifically includes:

[0015] 1) All participants generate a sharing pair offline. ,in: ;

[0016] 2) Calculations for all participants ;

[0017] 3) Public ;

[0018] 4) Calculations for all participants .

[0019] The aforementioned bias aggregation specifically includes:

[0020] 1) All participants are aggregated according to the aforementioned weights. ;

[0021] 2) Local calculation by all participants .

[0022] Step 3: Based on the aggregated gradient For target parameters According to the formula Update the target parameters. According to the formula The update is performed, using truncation to calculate and Multiplication;

[0023] Step 4: Publicly share the secret ciphertext of the final calculated target parameters to obtain the result in plaintext form.

[0024] The computations involved in the privacy training of the model all adopt the computations shared by Shamir, including: addition, multiplication, plaintext multiplication, standard multiplication and truncation.

[0025] The aforementioned addition calculation refers to: based on the secret shared ciphertext input by the participants. , , and the secret shared by the participants All participants will directly use their local networks. The corresponding share is added together to generate .

[0026] The multiplication calculation mentioned refers to: based on the secret shared ciphertext input by the participants. , ,and , ,generate .

[0027] The plaintext multiplication calculation mentioned above refers to: based on the secret shared ciphertext input by the participants. , ,and ,generate .

[0028] The aforementioned multiplication calculations and plaintext multiplication calculations specifically include:

[0029] 1) All participants generate a sharing pair offline. ,in: , ;

[0030] 2) Calculations for all participants For plaintext multiplication calculations, each participating party sets its sharing method to... ;

[0031] 3) Public ;

[0032] 4) Settings ;

[0033] 5) Calculations for all participants .

[0034] The aforementioned standard multiplication calculation refers to: based on the secret shared ciphertext input by the participants. , ,and All participants will directly use their local networks. Multiplication generation Make .

[0035] The truncation calculation refers to: based on the secret shared ciphertext input by the participants. , truncated digits ,generate Specifically, it includes:

[0036] 1) All participants generated offline. A random share ,in: , Let be random bits. ;

[0037] 2) All participants calculate and disclose. ;

[0038] 3) Settings ,in: ,set up ,in: The most significant bit;

[0039] 4) Calculations for all participants ;

[0040] 5) Calculations for all participants ,in: when otherwise ;

[0041] 6) Calculations for all participants .

[0042] This invention, based on Shamir's secret sharing over a Galois ring, enables parallel computation of samples within a training batch. After parallel computation of the gradient for each sample in the batch, the samples are aggregated. During aggregation, different sharing methods are used for weights and biases, as weights are applied to multiplication and biases to addition. Compared to existing technologies, this invention, through an improved truncation method, requires only one round of online comparison operations, thus effectively accelerating the efficiency of existing neural network training protocols based on Shamir's secret sharing. Attached Figure Description

[0043] Figure 1This is a schematic diagram of the system of the present invention;

[0044] Figure 2 This is a flowchart of the present invention. Detailed Implementation

[0045] like Figure 1 As shown, this embodiment involves a Shamir secret sharing method on a Galois ring. The square neural network training system includes: a sharing unit, a reconstruction unit, an addition unit, a multiplication unit, a plaintext multiplication unit, a standard multiplication unit, and a truncation unit, wherein: the sharing unit calculates the input from the participants... Generate secret sharing ciphertext The reconstruction unit shares ciphertext based on the secrets input by the participants. Generate reconstructed plaintext The addition unit, multiplication unit, plaintext multiplication unit, standard multiplication unit, and truncation unit are calculated accordingly.

[0046] like Figure 2 As shown in this embodiment, a neural network training method based on Shamir secret sharing on a Galois ring is involved. In the preparatory stage, the three participants embed the secret data into the ring in groups of two. The three parties secretly share the random shared ciphertext required by the protocol and the shared ciphertext for initializing the target parameters through interactive computation. During the training phase, the three parties perform model privacy training computation in a collaborative and interleaved manner. After training is completed, the three parties publish the shared ciphertext of the target parameters.

[0047] The initialization target parameters mentioned above refer to: the target parameters set by the participating parties. .

[0048] The aforementioned method of performing model privacy training computation using a collaborative computation interleaved execution approach specifically includes:

[0049] Step 1: Gradient Calculation: According to the formula The forward propagation process is calculated, where: The ReLU activation function is used to obtain the output of the last layer. Then, calculate the objective function. Its derivative Based on Calculate the derivative of the target parameters for each layer. Calculate the objective function Corresponding to parameters gradient .

[0050] Step 2: Gradient aggregation: Aggregate the gradients corresponding to all samples in a single training batch, and obtain the global gradient prediction value and share the encrypted text through weight aggregation and bias aggregation;

[0051] Step 3: Parameter Update: Based on the Aggregated Gradient Update the target parameters.

[0052] This embodiment employs, but is not limited to, the protocol described in the literature (Hengcheng Zhou. Information-theoretically secure neural network training with flexible deployment. In 2023 Artificial Neural Networks and Machine Learning (ICANN), pages 324-336.). Perform Shamir secret sharing calculations.

[0053] The protocol Prefix multiplication in the code is implemented by continuously calling the multiplication unit.

[0054] Through specific experiments, on a server with two 24-core CPUs and 128GB of memory, in a local area network environment, using the MNIST dataset, a 3-layer fully connected neural network was trained using ReLU as the activation function and Support Vector Machine as the loss function. The runtime and communication volume during the online phase were measured, and the experimental data obtained are as follows:

[0055] When embedding two secrets in a ring element, using mini-batch stochastic gradient descent with a batch size of 128, the training time for 15 rounds is 0.59 hours and the communication volume is 0.013TB.

[0056] When embedding one secret in a ring element, using mini-batch stochastic gradient descent with a batch size of 128, the training time for 15 rounds is 0.98 hours and the communication volume is 0.024TB.

[0057] When using Shamir secret sharing over a finite field with mini-batch stochastic gradient descent with a batch size of 128, the training time for 15 rounds is 20.87 hours and the communication volume is 0.529TB.

[0058] Compared to using Shamir secret sharing over finite fields, this method avoids modulo prime operations and does not rely on security parameters, thus achieving higher efficiency.

[0059] The above-described specific implementations can be partially adjusted by those skilled in the art in different ways without departing from the principles and purpose of the present invention. The scope of protection of the present invention is defined by the claims and is not limited to the above-described specific implementations. All implementation schemes within the scope of the claims are bound by the present invention.

Claims

1. A method for Shamir secret sharing on a Galois ring The square neural network training method is characterized by, In the preparatory stage Each participating party will use secret data Each element is embedded into a ring element for secret sharing, and the random sharing ciphertext and the sharing ciphertext for initializing the target parameters are calculated in advance. In the training phase, the model privacy training is performed by local computation and collaborative interleaving, and after the training is completed, a participant discloses the sharing ciphertext of the target parameter; The aforementioned secret sharing refers to: for those located in the ring In Input data after fixed-point encoding First, it is embedded into the Galois ring. elements in In the middle, satisfy ,in: or Then generate Shamir's Secret Sharing When according to The way in Embedded in A secret hour, Represented as ,in: belong When a secret is shared, the secret holder generates a Galois ring. polynomial of degree ,in: The secret value; other parameters To generate random numbers, Distributed to the Each participant shares their share, among which: For one of the Galois rings containing The set of exceptions contains non-zero and distinct elements. When reconstructing the secret, first collect the shares of all participants, then calculate using the Lagrange interpolation formula. Obtain the secret value; when the secret value is... At that time, the corresponding sharing is ; The protocol requires the use of randomly shared ciphertext, which is used to initialize the weights of the neural network. and bias terms ,in: Represents the current network layer number; The aforementioned collaborative interleaved model privacy training specifically includes: Step 1, Forward Propagation: , ,in: This represents the activation function, which yields the output of the last layer. Then, calculate the objective function. Its derivative Based on Calculate the derivative of the target parameters for each layer. Then calculate the objective function. Corresponding to parameters gradient ; Step 2, aggregating the gradients corresponding to all samples in the training batch containing the sample to obtain a global gradient estimate sharing ciphertext and and , specifically comprising: 2.1 Weight aggregation: for the computed sample corresponding to the gradient share ciphertext of the target parameter , the ciphertext is calculated, that is , ; 2.2 Bias aggregation: for each computed gradient share ciphertext corresponding to a target parameter , compute the ciphertext that is , ; Step 3: Based on the aggregated gradient and For target parameters According to the formula Update the target parameters. According to the formula The update is performed, using truncation to calculate and Multiplication.

2. The Galois ring based Shamir secret sharing of claim 1, A neural network training method, characterized by The weight aggregation specifically includes: 1) All participants offline generate a pair of shares and wherein: ; 2) All participants compute ; 3) Public ; 4) Calculations for all participants .

3. The method based on Shamir secret sharing on a Galois ring as described in claim 2 The square neural network training method is characterized by, The aforementioned bias aggregation specifically includes: 1) All participants are calculated using a weighted aggregation method. ; 2) Local calculation by all participants ).

4. The method based on Shamir secret sharing on a Galois ring according to any one of claims 1-3 The square neural network training method is characterized by, The computations involved in the privacy training of the model all adopt the computations shared by Shamir, including: addition, multiplication, plaintext multiplication, standard multiplication and truncation. The aforementioned addition calculation refers to: based on the secret shared ciphertext input by the participants. , , and the secret shared by the participants , All participants will directly use their local networks. and The corresponding share is added together to generate Make ; The multiplication calculation mentioned refers to: based on the secret shared ciphertext input by the participants. , ,and , ,generate Make ; The plaintext multiplication calculation mentioned above refers to: based on the secret shared ciphertext input by the participants. , ,and ,generate Make ; The aforementioned standard multiplication calculation refers to: based on the secret shared ciphertext input by the participants. , ,and All participants will directly use their local networks. and Multiplication generation Make ; The truncation calculation refers to: based on the secret shared ciphertext input by the participants. , truncated digits ,generate Make .

5. The method for Shamir secret sharing on a Galois ring as described in claim 4. The square neural network training method is characterized by, The aforementioned multiplication calculations and plaintext multiplication calculations specifically include: 1) All participants generate a sharing pair offline. and ,in: ; 2) Calculations for all participants in: For plaintext multiplication calculations, each participating party sets its sharing method to... Get ; 3) Public ; 4) Settings ; 5) Calculations for all participants .

6. The method for Shamir secret sharing on a Galois ring as described in claim 4 The square neural network training method is characterized by, The truncation calculation specifically includes: 1) All participants generated offline. A random share ,in: , Let be random bits, let , ; 2) All participants calculate and disclose. , ; 3) Settings ,in: ,set up ,in: for The most significant bit; 4) Calculations for all participants ; 5) Calculations for all participants Make ,in: when otherwise ; 6) Calculations for all participants .

7. A method for implementing the method of any one of claims 1-6 based on Shamir secret sharing on a Galois ring. A square neural network training system includes: The system includes a sharing unit, a reconstruction unit, an addition unit, a multiplication unit, a plaintext multiplication unit, a standard multiplication unit, and a truncation unit. The sharing unit is based on the input from the participants. Generate secret sharing ciphertext The reconstruction unit shares ciphertext based on the secrets input by the participants. Generate reconstructed plaintext The addition unit, multiplication unit, plaintext multiplication unit, standard multiplication unit, and truncation unit are calculated accordingly.

Images