A Neural Network Model Inference System Based on Secure Multi-Party Computation

Abstract

This invention relates to a neural network model inference system based on secure multi-party computation. The system includes: a secure multi-party computation module, used to construct a ciphertext computation graph based on model information from a plaintext model, involving multiple participants; and to invoke a GPU computation module and an RDMA communication module to execute the ciphertext computation graph based on acquired ciphertext, obtain a ciphertext result, and restore the executed ciphertext result to the user. The ciphertext computation graph is obtained by transforming a model computation graph. The GPU computation module is used to execute the ciphertext computation graph using 32-bit integer matrix operations based on the acquired ciphertext sent by the user. The RDMA communication module is used for data communication during the GPU computation process using a GPU-RDMA communication scheme. The beneficial effects of this invention are: ensuring the accuracy of neural network model inference while improving inference efficiency.

Description

Technical Field

[0001] This invention relates to the field of network security technology, and more specifically, to a neural network model inference system based on secure multi-party computation. Background Technology

[0002] Thanks to the success of neural networks in various machine learning tasks, neural network inference services have been widely applied in daily life. Users of these services need to upload their own data for inference, making the inference service itself a potential data collection interface. Much of this uploaded data contains personal privacy information, posing a risk of data leakage and misuse. Secure multi-party computation (MPC), as a privacy-preserving computation solution, can perform computations on data in encrypted form, providing theoretical security guarantees and ensuring data is usable but not visible. However, the biggest problem in implementing neural network model inference based on MPC lies in inference latency. Because MPC involves a large amount of computation and communication, performance has always been a core issue.

[0003] Currently, solutions for secure multi-party computation using GPUs generally employ 64-bit floating-point numbers to simulate 64-bit integers for integer matrix multiplication. However, this approach has several drawbacks. First, the throughput of 64-bit floating-point multiplication on GPUs is relatively low, typically only half that of 32-bit floating-point multiplication. The aforementioned solution requires nine 64-bit floating-point multiplications to perform one integer multiplication, leading to a significant increase in GPU computational load. Second, during matrix multiplication, parameters need to be converted from integer to floating-point representation, and then back to integer representation after computation. Since linear or convolutional layers in neural networks are often followed by activation layers, this switching between representations is very frequent and consumes considerable time, impacting the efficiency of neural network inference. Third, the memory usage for storing the model using 64-bit floating-point numbers during computation is substantial. Combined with the storage of random variables in the offline phases of secure multi-party computation, such as triples, the memory overhead is extremely high. Executing the inference task of the LeNet network requires approximately 2GB of GPU memory, making it impossible to execute large networks on a typical GPU. Summary of the Invention

[0004] The problem addressed by this invention is how to improve the inference efficiency of neural network models.

[0005] To address the above problems, this invention provides a neural network model inference system and method based on secure multi-party computation.

[0006] This invention provides a neural network model inference system based on secure multi-party computation, comprising:

[0007] The secure multi-party computation module is used to construct a ciphertext computation graph based on the model information of the plaintext model, based on multiple participants, and call the GPU computing module and RDMA communication module to execute the ciphertext computation graph according to the acquired ciphertext, obtain the ciphertext result, and restore the executed ciphertext result to the user.

[0008] The encrypted computation graph is obtained by transforming the model computation graph. The GPU computing module is used to execute the encrypted computation graph using 32-bit integer matrix operations based on the encrypted text sent by the user. The RDMA communication module is used to perform data communication during the computation process of the GPU computing module using the GPU-RDMA communication scheme.

[0009] Optionally, the secure multi-party computation module is specifically used for:

[0010] Based on the FALOCN protocol, a secure multi-party computation protocol is constructed using three parties. The secure multi-party computation protocol is executed according to the ciphertext to execute the ciphertext computation graph. The three parties include two computation parties and one auxiliary party. The auxiliary party is used to determine whether there is a zero element in the matrix to execute the comparison protocol in the secure multi-party computation protocol.

[0011] Optionally, the 32-bit integer matrix operation includes 32-bit integer matrix multiplication, and the GPU computing module is specifically used for:

[0012] Based on the Cutlass open-source floating-point matrix multiplication library, this paper optimizes the implementation of 32-bit integer matrix multiplication using optimization methods for 32-bit floating-point matrix multiplication. These optimization methods include matrix sharding, shared memory caching, and computation time masking memory access time.

[0013] Optionally, the 32-bit integer matrix operation includes 32-bit integer matrix element operations, and the GPU computing module is specifically used for:

[0014] The transform operation in the Thrust parallel computing library is used to implement the element operations of the 32-bit integer matrix using a kernel fusion method. The element operations of the 32-bit integer matrix include integer matrix dot product operation and integer matrix addition operation.

[0015] Optionally, the GPU computing module is further configured to:

[0016] The multiplication protocol is executed using the 32-bit integer matrix multiplication to infer the linear and convolutional layers in the plaintext model.

[0017] The comparison protocol is executed using the element-wise operations of the 32-bit integer matrix to infer the activation layer and pooling layer in the plaintext model.

[0018] Optionally, the secure multi-party computation module is further used for:

[0019] Using different decimal places for different fixed-point matrix variables can be expressed using the first formula as follows:

[0020]

[0021] Where l represents the fixed-point matrix variable x f The number of bits used, x d Represents a fixed-point matrix variable x f The number of decimal places used, x max With x min These represent the maximum and minimum values ​​in matrix variable x, respectively.

[0022] When calculating the product of two fixed-point matrix variables, the number of bits to be truncated is expressed by the second formula as follows:

[0023] x d +y d -z d ,

[0024] Where, x d Represents a fixed-point matrix variable x f The number of decimal places used, y d Represents the fixed-point matrix variable y f The number of decimal places used, z d Represents the fixed-point matrix variable z f The number of decimal places used.

[0025] Optionally, the secure multi-party computation module is further configured to matrix the secure multi-party computation protocol.

[0026] Optionally, the RDMA communication module is specifically used for:

[0027] Enables data transfer between GPUs via RDMA network cards.

[0028] Optionally, the neural network model inference system based on secure multi-party computation further includes:

[0029] The neural network inference module is used to construct a model computation graph based on the acquired plaintext model, and to extract information from the model computation graph to obtain model information, wherein the model information includes model nodes and computation functions.

[0030] Optionally, the method for obtaining the plaintext model includes:

[0031] The neural network model is trained using the distillation defense method and the quantization perception training method to obtain the trained plaintext model.

[0032] The beneficial effects of this invention are:

[0033] By setting up a secure multi-party computation module, based on multiple participants, a encrypted computation graph is constructed according to the model information. The GPU computation module is then invoked to execute the encrypted computation graph using 32-bit integer matrix operations based on the encrypted text sent by the user. This effectively reduces the computational and storage requirements of the GPU, accelerating computation and thus reducing the inference time of the neural network model, increasing efficiency, alleviating GPU computational pressure, and extending GPU lifespan and usage time. Simultaneously, the RDMA communication module is invoked, utilizing a GPU-RDMA communication scheme for data communication during the GPU computation process. This avoids memory copying during data transmission and enables fast data transfer between GPUs through the RDMA network card, reducing latency caused by data transmission. By utilizing multiple participants, the encrypted computation graph is executed based on the encrypted text sent by the user, obtaining the encrypted result, which is then restored to the user. This multi-participant approach accelerates the inference process while ensuring model inference accuracy, effectively reducing model inference latency and improving model inference efficiency. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the structure of a neural network model inference system based on secure multi-party computation according to an embodiment of the present invention;

[0035] Figure 2 This is a schematic diagram of the inference method for a neural network model based on secure multi-party computation according to an embodiment of the present invention;

[0036] Figure 3 This is a schematic diagram of matrix partitioning according to an embodiment of the present invention;

[0037] Figure 4 This is the core calculation process of the comparison protocol in this embodiment of the invention;

[0038] Figure 5 This is a schematic diagram illustrating the communication principle between the GPU and the RDMA network card in an embodiment of the present invention.

[0039] Figure 6 This is a schematic diagram of the connection structure of three RDMA network cards according to an embodiment of the present invention;

[0040] Figure 7 This is a schematic diagram illustrating the quantitative perception training principle of an embodiment of the present invention. Detailed Implementation

[0041] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0042] Currently, the FALCON scheme for neural network inference suffers from excessive inference latency. Table 1 shows the inference latency of the FALCON protocol in three neural networks. Dense3 refers to a three-layer fully connected network, while LeNet and Vgg16 both use the architecture proposed in their respective papers. All three servers use Intel Xeon CPUs, with a network bandwidth of 1Gbps and an RTT latency of approximately 0.2ms. For all three neural networks, 50 inferences were performed using a batch size of 128, and the average value was taken. Dense3 and LeNet used the MNIST dataset, while Vgg16 used the Cifar10 dataset.

[0043] Table 1. Inference time of three neural networks under the FALCON scheme

[0044]

[0045] In all three types of neural networks, computation accounted for more than 60% of the time, indicating that, unlike traditional theoretical research, computation and communication will be the bottlenecks in secure multi-party computation processes in high-speed network environments.

[0046] To solve the above problems, refer to Figure 1 As shown, this embodiment of the invention provides a neural network model inference system based on secure multi-party computation, comprising:

[0047] A secure multi-party computation module is used to construct a ciphertext computation graph based on model information of a plaintext model from multiple participants, and to call a GPU computation module and an RDMA communication module to execute the ciphertext computation graph based on the acquired ciphertext, obtain the ciphertext result, and restore the executed ciphertext result to the user. The ciphertext computation graph is obtained by transforming the model computation graph. The GPU computation module is used to execute the ciphertext computation graph using 32-bit integer matrix operations based on the acquired ciphertext sent by the user. The RDMA communication module is used to perform data communication during the computation process of the GPU computation module using a GPU-RDMA communication scheme.

[0048] Specifically, before the secure multi-party computation module performs model inference, it also includes, for example, a neural network module. This module can communicate with the model provider to obtain the plaintext model transmitted by the model provider, such as the ONNX model, and obtain the model computation graph of the plaintext model (a computation graph is a representation of the computational logic of a neural network model, representing the relationships between some computational operations). It can parse the plaintext model within the computation graph to obtain model information, including the number of nodes, node types, node parameter matrices, model computation functions, etc., which are used for subsequent model inference calculations. The secure multi-party computation module constructs multiple participants, such as two computation participants and one auxiliary participant. Based on the model information extracted and sent by the aforementioned neural network module, it constructs a ciphertext computation graph, obtains the ciphertext sent by the end user, and calls the GPU computation module. Utilizing multiple participants, such as two computation participants and one auxiliary participant, it performs 32-bit integer matrix operations on the ciphertext, executes the computation of the ciphertext computation graph, obtains the ciphertext result of the ciphertext computation graph, restores the ciphertext result to the user, and completes the model inference. This involves simultaneously invoking the GPU computing module for computation and the RDMA communication module, employing a GPU-RDMA scheme. This allows data to be transferred between GPUs via the RDMA network card, enabling communication between multiple participants in computation. Figure 2 The diagram shown is a flowchart of a method for using a neural network model inference system based on secure multi-party computation according to an embodiment of the present invention.

[0049] In this embodiment, a secure multi-party computation module is set up. Based on multiple participants, a ciphertext computation graph is constructed according to the model information. A GPU computation module is then invoked. Based on the ciphertext sent by the user, the GPU computation module executes the ciphertext computation graph using 32-bit integer matrix operations. This effectively reduces the computational and storage requirements of the GPU, accelerating computation and reducing the inference time of the neural network model, increasing efficiency, alleviating GPU computational pressure, and extending GPU lifespan and usage time. Simultaneously, an RDMA communication module is invoked, utilizing a GPU-RDMA communication scheme for data communication during the GPU computation process. This avoids memory copying during data transmission and enables fast data transfer between GPUs through the RDMA network card, reducing latency caused by data transmission. By utilizing multiple participants, the ciphertext computation graph is executed based on the ciphertext sent by the user to obtain the ciphertext result, which is then restored to the user. This multi-participant approach accelerates the inference process while ensuring model inference accuracy, effectively reducing model inference latency and improving model inference efficiency.

[0050] Optionally, the secure multi-party computation module is specifically used for:

[0051] Based on the FALOCN protocol, a secure multi-party computation protocol is constructed using three parties. The secure multi-party computation protocol is executed according to the ciphertext to execute the ciphertext computation graph. The three parties include two computation parties and one auxiliary party. The auxiliary party is used to determine whether there is a zero element in the matrix to execute the comparison protocol in the secure multi-party computation protocol.

[0052] Specifically, the secure multi-party computation module in this embodiment calls the three-party computation protocol FALOCN. Based on this protocol, the three computation participants are adjusted to two computation participants and one auxiliary participant. A secure multi-party computation protocol including sharing protocol, addition protocol, multiplication protocol, comparison protocol, truncation protocol and recovery protocol is constructed. The auxiliary participant assists the computation participants in executing each protocol in the secure multi-party computation protocol to realize the execution of the encrypted computation graph.

[0053] The FALCON protocol is a three-party computation protocol, using P1, P2, and P3 to represent the three parties involved in the computation, and P... i-1 and P i+1 Relative to participant P i The previous participant and the next participant. For a secret value x, its corresponding ciphertext can be represented as [x]. m = ((x1,x2),(x2,x3),(x3,x1)), where participant P1 owns (x1,x2), participant P2 owns (x2,x3), and participant P3 owns (x3,x1). Here, m represents the order of the congruence ring on which the ciphertext is based, that is, x1,x2,x3 are all random values ​​in the congruence ring modulo m, and have the property x1 + x2 + x3 mod m = x mod m.

[0054] Addition operation [z] m =[x] m +[y] m The calculation method is shown in the following formula:

[0055] Participant P i Calculate: z i =x i +y i ,z i+1 =x i+1 +y i+1 ,

[0056] Addition operation [z] m =[x] m +c, the calculation method is shown in the following formula:

[0057] Participant P1 calculates: z i =x i +c,zi+1 =x i+1 +c,

[0058] Calculation of participants P2 and P3: z i =x i ,z i+1 =x i+1 ,

[0059] Multiplication operation [z] m =[x] m ×[y] m The calculation method is shown in the following formula:

[0060] Participant P i Calculate: z i =x i ×y i +x i ×y i+1 +x i+1 ×y i ,

[0061] Participant P i Send z i Give to participant P i-1 :z i+1 =recv_from(P i+1 ).

[0062] Multiplication operation [z] m =[x] m ×c is calculated using the following formula:

[0063] Participant P i Calculate: z i =x i ×c,z i+1 =x i+1 ×c,

[0064] The process of restoring the secret value: z = reveal([z]) m The calculation process is as follows:

[0065] Participant P i Send z i+1 Give to participant P i-1 ,

[0066] Comparison operation [x] m >[y] m This is equivalent to determining the secret value [z]. m =[x] m -[y] m The sign of z is determined by its highest bit, which can be calculated using the following formula:

[0067]

[0068] The function wrap(2z1+2z2+2z3,m) takes the value 1 if 2z1+2z2+2z3>m, and 0 otherwise. MSB(z1), MSB(z2), and MSB(z3) are known to P1, P2, and P3, respectively. The main task is to calculate wrap(2z1+2z2+2z3,m), and the calculation process is shown in the following formula, where [a] m It is a secret sharing of a random value, [α] 2 =wrap(a1+a2+a3,m).

[0069] [t] m =2×[z] m ,

[0070] [r] m =[t] m +[a] m ,

[0071] β i =wrap(t i +a i ,m),

[0072] r = reveal([r]) m ),

[0073] δ = wrap(r,m),

[0074] η = (a ≥ r + 1),

[0075] wrap(2z1+2z2+2z3,m)=β1+β2+β3+δ-α-η,

[0076] Only the calculation process of η=(a≥r+1) in the above formula is not obvious, and its calculation process is as follows:

[0077] [u i ] p =(-1) β ([a i ] p -r i ),

[0078]

[0079]

[0080]

[0081] Where a iLet be the individual bits of 'a', totaling l bits, where p is a small prime number, β is a random value, and [m] is the total number of bits. p It is a random secret value. If d equals 0, then a ≥ r + 1; otherwise, a <r+1。

[0082] Based on the FALCON protocol described above, the roles of the three participating parties in the computation process are changed. The three participants are adjusted to two computational participants and one auxiliary participant, forming a secure multi-party computation protocol. The three participants do not collude and adhere to the protocol's execution process. In the original FALCON protocol, all three participants, P1, P2, and P3, hold the secret text: participant P1 holds (x1, x2), participant P2 holds (x2, x3), and participant P3 holds (x3, x1). In our improved secure multi-party computation protocol, only two participants, P1 and P2, hold the secret text, in the form [x]. m = (x1, x2), participant P1 owns x1, participant P2 owns x2, and has the property x1 + x2 mod m = x mod m. Participant P3 exists as an auxiliary participant, mainly to help complete the comparison protocol calculation process.

[0083] When the form of the ciphertext changes, the corresponding multiplication and addition calculation processes also change; addition operations [z] m =[x] m +[y] m The calculation method is shown in the following formula:

[0084] Participant P i Calculate: z i =x i +y i ,

[0085] Addition operation [z] m =[x] m +c, the calculation method is shown in the following formula:

[0086] Participant P1 calculates: z i =x i +c,

[0087] Participant P2 calculates: z i =x i ,

[0088] Multiplication operation [z] m =[x] m ×[y] m The calculation method is shown in the following formula:

[0089] [α] m =[x] m -[a] m,[β] m =[y] m -[b] m ,

[0090] α = reveal([α]) m ),β=reveal([β] m ),

[0091] [z] m =[c] m +α×[y] m +β×[x] m +α×β,

[0092] [a] m [b] m [c] m Satisfying relation [c] m =[a] m ×[b] m ,

[0093] Multiplication operation [z] m =[x] m ×c is calculated using the following formula:

[0094] Participant P i Calculate: z i =x i ×c,

[0095] The process of restoring the secret value: z = reveal([z]) m The calculation process is as follows:

[0096] Participant P i Send z i Give to participant P i-1 ,

[0097] Participant P i Calculate z = z1 + z2 mod m.

[0098] The original FALCON protocol, which used chain multiplication to determine the presence of a zero element among n elements, now uses an auxiliary participant for judgment. In the adjusted comparison protocol, all calculations except for η = (a ≥ r + 1) remain the same as the original FALCON protocol; however, the specific addition and multiplication implementations are converted to our improved addition and multiplication methods. In calculating η = (a ≥ r + 1), only the last step of the comparison protocol is improved. In the original calculation process, the first three steps did not require communication and could be completed locally, while the last step, which uses multiplication, requires communication, at least... Round-robin communication. Our improved solution will use l [c i ] p The data is restored to the assisting participant, who then determines the value of c. i Does 0 exist in the equation, thus obtaining η = (a ≥ r + 1)?

[0099] In this embodiment, the three computational participants in the FALOCN protocol are adjusted to two computational participants and one auxiliary participant. A secure multi-party computation protocol is constructed and executed. While the computational participants satisfy the model inference computation, the auxiliary participant is used to compare whether there is a 0 element in the comparison protocol. Only one round of communication is needed in the comparison protocol, which greatly reduces the number of communication rounds required. This effectively solves the problem of a large number of communication rounds in the comparison protocol and improves the efficiency of the comparison protocol computation.

[0100] Optionally, the 32-bit integer matrix operation includes 32-bit integer matrix multiplication, and the GPU computing module is specifically used for:

[0101] Based on the Cutlass open-source floating-point matrix multiplication library, this paper optimizes the implementation of 32-bit integer matrix multiplication using optimization methods for 32-bit floating-point matrix multiplication. These optimization methods include matrix sharding, shared memory caching, and computation time masking memory access time.

[0102] Specifically, matrix operations are divided into matrix multiplication and matrix element operations. This embodiment uses 32-bit integer matrix operations, namely 32-bit integer matrix multiplication and 32-bit integer matrix element operations. Since the number of 32-bit integer computing units in a GPU is the same as the number of 32-bit floating-point computing units, this embodiment utilizes the optimization method used in 32-bit floating-point matrix multiplication to implement 32-bit integer matrix multiplication. Figure 3As shown, the input matrix is ​​divided into blocks, and each thread block is only responsible for computing its assigned block. This ensures that the size of the matrix processed by each thread block remains constant regardless of changes in the data matrix. Each thread block then further divides the matrix into blocks, with each thread processing a portion of the block. The core idea behind this acceleration is to utilize the shared memory of the SM (Multiplexer Memory) as a high-speed cache for these blocks, using the computation time of block matrix multiplication to mask the data transfer time from global memory to shared memory. Note that both 32-bit integer and 32-bit floating-point operations occupy 4 bytes, and each SM in the GPU has the same number of integer and floating-point computation units. This means that 32-bit integer matrix multiplication can reuse existing matrix block strategies and shared memory caching strategies, and the computation time of block integer matrix multiplication (which is slightly longer than floating-point multiplication) can also mask the data transfer time from global memory to shared memory. This implies that existing 32-bit floating-point matrix multiplication optimization methods can be reused when performing 32-bit integer matrix multiplication on existing NVIDIA GPUs. In terms of implementation, Cutlass is a template matrix computation library. Although Cutlass's declaration states that it only supports floating-point matrix operations, some modifications to the Cutlass source code, primarily to remove some limitations on computation types, allow the library to also support 32-bit integer matrix multiplication. The overall computational approach is the same as floating-point matrix multiplication. Convolution operations, after being processed by `Img2col`, are ultimately transformed into matrix multiplication. Cutlass also provides a 32-bit floating-point implementation of convolution operations, which, after modification, also supports 32-bit integer convolution operations. Based on the modified Cutlass open-source floating-point matrix multiplication library, 32-bit integer matrix multiplication and 32-bit integer convolution operations are implemented using optimized methods for 32-bit floating-point matrix multiplication.

[0103] In this embodiment, an optimization method for 32-bit floating-point matrix multiplication is used, which involves matrix sharding, shared memory caching, and computation time masking memory access time. Based on the improved Cutlass open-source floating-point matrix multiplication library, 32-bit integer matrix multiplication is implemented. This can effectively reduce GPU performance loss and computation time, thereby improving GPU lifespan and model inference efficiency, and effectively reducing inference latency.

[0104] Optionally, the 32-bit integer matrix operation includes 32-bit integer matrix element operations, and the GPU computing module is specifically used for:

[0105] The transform operation in the Thrust parallel computing library is used to implement the element operations of the 32-bit integer matrix using a kernel fusion method. The element operations of the 32-bit integer matrix include integer matrix dot product operation and integer matrix addition operation.

[0106] Specifically, 32-bit integer matrix element operations include matrix addition, matrix dot product, and matrix XOR. The `transform` operation in the Thrust open-source library is used to transform one or more input vectors into a result vector through a kernel function. Matrix addition is achieved when two matrices are used as input vectors and the kernel function is addition; similarly, matrix dot product is achieved when the kernel function is multiplication. The comparison protocol implementation involves numerous matrix operations, the most of which are... Figure 4 As shown. The first loop involves a large number of matrix addition and XOR operations. If we simply perform addition or XOR operations on two matrices each time, it will involve a lot of repeated memory accesses. Specifically, the matrix [u] is calculated... i ] p β and [x] need to be read in i ] p With r i Write into matrix [u i ] p Calculate matrix [w] i ] p It is necessary to read [x] again. i ] p With r i Write into matrix [w i ] p Calculate matrix [c] i ] p With [d] i ] p The process is similar. Therefore, this invention uses a kernel fusion method, treating the entire for loop calculation process as the kernel function of the transform operation, and using β, r, x, and m as inputs to the kernel function of the transform operation to directly calculate the output d. This avoids allocating storage space for intermediate variables w and u, and also reduces many unnecessary memory accesses.

[0107] In this embodiment, the transform operation in the Thrust parallel computing library is used to perform 32-bit integer matrix element operations using the kernel fusion method, which effectively reduces the use of storage space, avoids unnecessary memory accesses in many matrix operations, and reduces the number of memory accesses during the calculation process.

[0108] Optionally, the GPU computing module is further configured to:

[0109] The multiplication protocol is executed using the 32-bit integer matrix multiplication to infer the linear and convolutional layers in the plaintext model.

[0110] The comparison protocol is executed using the element-wise operations of the 32-bit integer matrix to infer the activation layer and pooling layer in the plaintext model.

[0111] Specifically, the optimized 32-bit integer matrix multiplication is used to calculate the multiplication protocol in the constructed secure multi-party computation protocol, enabling inference for linear layers in the neural network model. Similarly, the optimized 32-bit integer matrix convolution operation, using the same method as 32-bit integer matrix multiplication, is used to implement inference for convolutional layers. The optimized 32-bit integer matrix element-wise operations are used to calculate the comparison protocol in the constructed secure multi-party computation protocol, enabling inference for activation and pooling layers in the neural network model. Data communication is conducted via an RDMA communication module during the execution of each protocol. Inference calculations for the neural network model are achieved through 32-bit integer matrix multiplication and 32-bit integer matrix element-wise operations, realizing inference for each layer structure within the neural network model.

[0112] In this embodiment, 32-bit integer matrix multiplication and 32-bit integer matrix element operations are used to execute the multiplication and comparison protocols, respectively, to calculate the structure of each layer in the neural network model inference, effectively increasing the efficiency of neural network model inference.

[0113] Optionally, the secure multi-party computation module is further used for:

[0114] Using different decimal places for different fixed-point matrix variables can be expressed using the first formula as follows:

[0115]

[0116] Where l represents the fixed-point matrix variable x f The number of bits used, x d Represents a fixed-point matrix variable x f The number of decimal places used, x max With x min These represent the maximum and minimum values ​​in matrix variable x, respectively.

[0117] When calculating the product of two fixed-point matrix variables, the number of bits to be truncated is expressed by the second formula as follows:

[0118] x d +y d -z d ,

[0119] Where, x d Represents a fixed-point matrix variable x f The number of decimal places used, y d Represents the fixed-point matrix variable y f The number of decimal places used, z d Represents the fixed-point matrix variable z f The number of decimal places used.

[0120] In this embodiment, based on the fact that the parameters and outputs of each node in the neural network are distributed in a certain range, different decimal places are used for different fixed-point matrix variables in the calculation process to solve the problem of insufficient space for fixed-point representation caused by using 32-bit integers, and to alleviate the problems of insufficient precision and overflow of 32-bit representation compared to fixed-point numbers.

[0121] Optionally, the secure multi-party computation module is further configured to matrix the secure multi-party computation protocol.

[0122] Specifically, since the secure multi-party computation module calls the GPU computing module for computation, and the GPU computing module uses matrix form for all calculations, the computational operations required for secure multi-party computation are matrix operations, mainly of two types: matrix multiplication and matrix element operations. Linear layers and convolutional layers primarily use matrix multiplication, while activation layers and pooling layers primarily use matrix element operations, including matrix addition, matrix dot product, and matrix XOR. Therefore, before the secure multi-party computation module calls the GPU computing module for computation, the generated secure multi-party protocol needs to be matrixed.

[0123] In this embodiment, since the GPU computing module uses matrix form for calculation, the various protocols need to be matrixed before the secure multi-party computation module executes the protocol. This facilitates subsequent calculation and inference of the neural network model, thereby improving inference efficiency.

[0124] Optionally, the RDMA communication module is specifically used for:

[0125] Enables data transfer between GPUs via RDMA network cards.

[0126] Specifically, the GPU is used to accelerate the computation process, and all data involved in the computation resides in GPU memory. If a traditional socket communication process is used, data transmission requires copying data from GPU memory to CPU memory, processing it according to the CPU's protocol, and then sending the data to the network card. The same data copying and processing issues arise during data reception, resulting in slow data transmission speeds. This embodiment of the invention employs a GPU-RDMA communication module, such as... Figure 5 As shown, data is sent directly from the GPU to the RDMA network card via PCIe. The RDMA network card enables data transfer between GPUs, avoiding memory copying between the CPU and GPU. Figure 6 As shown, this embodiment illustrates the connection method between the RDMA network cards in the two computing participants and one auxiliary participant. The RDMA network card has two network ports, so the three network cards can be directly connected without the need for a switch.

[0127] In this embodiment, GPUs transmit data via RDMA network cards, avoiding the low communication efficiency caused by data copying in traditional communication processes, effectively increasing communication efficiency, and thus increasing the inference efficiency of neural network models.

[0128] Optionally, the neural network model inference system based on secure multi-party computation further includes:

[0129] The neural network inference module is used to construct a model computation graph based on the acquired plaintext model, and to extract information from the model computation graph to obtain model information, wherein the model information includes model nodes and computation functions.

[0130] Specifically, such as Figure 1 As shown, the neural network model inference system based on secure multi-party computation also includes a neural network inference module. This module is the top-level module of the system and can communicate with the model provider to obtain the plaintext model transmitted by the provider—that is, the plaintext model after processing by the neural network model, such as the ONNX model. The neural network module retrieves the model definition and parameter solution methods from the plaintext model. After abstracting the model definition and parameter solution methods, it obtains the unique computational logic and constructs a plaintext model computation graph (a computation graph is a representation of the computational logic of a neural network model, showing the relationships between computational operations). The plaintext model can be parsed using the protobuf protocol to obtain model information, including the number of nodes, node types, node parameter matrices, and model computation functions, which are used for subsequent model inference calculations.

[0131] In this embodiment, a neural network inference module is set up to construct a model computation graph based on the plaintext model and parse the obtained model information for subsequent construction of a ciphertext computation graph for model inference, ensuring the normal operation of the neural network model inference.

[0132] Optionally, the process of obtaining the plaintext model includes:

[0133] The neural network model is trained using the distillation defense method and the quantization perception training method to obtain the trained plaintext model.

[0134] Specifically, distillation defense is currently the best defense against member inference attacks. Models trained using knowledge distillation defense show a very low drop in accuracy, while the attacker's success probability is close to blind guessing, and it doesn't require a complex training process. Distillation defense requires an auxiliary reference dataset, which is unlabeled and relatively easy to obtain. In the defense process, a private model is first trained on the private dataset to be protected. This model is not used directly; its purpose is to use it to make predictions on the reference dataset, transferring the knowledge it possesses to the reference dataset. Then, a public model is trained using the reference dataset and the prediction output of the private model on the reference dataset. KL divergence is used as the loss function to assess the similarity between the private and public model predictions, as shown in the formula:

[0135]

[0136] Among them, X ref For the reference dataset used, The output of the private model in the reference dataset. Let KL divergence loss function be used. The quantization-aware training process is as follows: Figure 7 As shown, pseudo-quantization nodes exist during the quantization-aware training process. During forward propagation, the pseudo-quantization nodes first quantize the input:

[0137]

[0138] Then perform inverse quantization:

[0139]

[0140] Essentially, pseudo-quantization nodes first transform the input into a smaller representation space for inference, and then back into a larger representation space for training. If the input exceeds the size of the representation space, the larger portion is lost, similar to overflow during inference. If the input is within the representation space, only a portion of the output precision is lost, similar to precision loss during inference. The forward propagation process during training is exactly the same as the forward propagation process during inference, meaning the training phase's performance can be completely reproduced during inference without loss. During backpropagation, since the function `clamp` is almost everywhere non-differentiable, this embodiment uses the STE (Straight Through Estimator) method to directly propagate the gradient from the previous layer to the next. In pseudo-quantization, this embodiment uses a smaller bit representation, such as 14 bits, to further compress the representation space. However, in actual inference, the representation space is larger, at 16 bits, to tolerate larger intermediate computational results.

[0141] In this embodiment, the plaintext model is trained using the distillation defense method, which greatly reduces the risk of data leakage and minimizes the decline in model accuracy. The quantization perception training method is used to compress the range of variable representations of model parameters and outputs of each node in the model, preventing overflow problems caused by insufficient representation space during inference.

[0142] The neural network model inference system and method based on secure multi-party computation proposed in the embodiments of the present invention are tested and verified for feasibility as follows:

[0143] The model inference latency, model accuracy, and model training data leakage were tested using four neural networks: Dense3, Dense5, LeNet, and Vgg16, on the Location, Purchase 100, MNIST, and Cifar10 datasets, respectively.

[0144] The specific information for the four datasets is as follows:

[0145] The Location dataset is a collection of data from a social network called Foursquare, showing mobile phone users checking in at various locations. It contains 5,010 data entries, each with 446 binary features, and 30 categories, each representing a geographical community group.

[0146] The Purchase 100 dataset is a publicly available dataset from Kaggele containing purchase records from thousands of different users. The purpose of this dataset is to train accurate coupon promotion strategies. It contains 197,324 data points, each a 600-dimensional binary vector, with 100 categories, each representing a different purchasing habit.

[0147] The MNIST handwritten digit dataset, compiled by the National Institute of Standards and Technology (NIST), contains 70,000 28×28 grayscale images and is a classic dataset in the field of machine learning.

[0148] The CIFAR-10 dataset is a classic dataset in the field of image recognition. Each image is a 32×32 three-channel color image, containing 10 categories, including animals such as cats and dogs, as well as cars and airplanes, with 6,000 images in each category.

[0149] Of the four neural networks, Dense3 is a 3-layer fully connected network with weight matrices of (446×128), (128×128), and (128×30) dimensions, respectively. Dense5 is a 5-layer fully connected network with weight matrices of (600×1024), (1024×512), (512×256), (256×128), and (128×10) dimensions, respectively. Both LeNet and Vgg16 adopt the model structures used in their respective papers.

[0150] The experiment was conducted in a local environment, using a Supermicro server as the auxiliary participant machine. It was equipped with two Intel Xeon Silver 4210 CPUs, each with 20 physical cores, for a total of 40 cores, and two NVIDIA 3080Ti graphics cards; only one of them was used. Two Lenovo P520c workstations were used as computational participants, each equipped with an Intel Xeon W-2145 CPU with 16 physical cores and an NVIDIA P4000 graphics card. Each server was equipped with a Mellanox ConnectX-3 pro network card with a bandwidth of 56Gbps, using the Infiniband protocol. Since there were only two P520c workstations, the Supermicro server was used as the auxiliary participant machine. As mentioned above, the auxiliary participant is only responsible for the comparison protocol process and is not the computational bottleneck in the scheme; therefore, it has little impact on the overall experimental results. Table 2 shows a comparison between the system proposed in this invention and two existing secure multi-party computation inference systems. Due to the lack of a high-speed Ethernet network card, the communication time obtained from inference tests using the FALCON protocol on the auxiliary participant's machine in a localhost environment was combined with the computation time obtained from inference tests on three servers as a comparison similar to that of this invention. The crypto scheme was tested on a single 3080Ti graphics card, which is close to the computing power of the two P4000 graphics cards used in this invention, and can be considered a fair comparison. All test times are the average of 50 inferences, with each inference using a batch size of 128 samples. Due to the excessive memory consumption of the crypto scheme, testing on the Vgg16 network was not possible.

[0151] Table 2 Model Inference Latency Test

[0152]

[0153] In terms of communication, our proposed solution has the lowest communication volume among the three schemes. Because it uses 32-bit integers and employs a plaintext model for inference, our communication volume is less than half that of the FALCON protocol. The crypton scheme, due to its use of a sharing conversion protocol based on additive circuits and the surge in communication volume during convolution operations, has more than 10 times the communication volume of our proposed solution in LeNet. In both Dense3 and LeNet networks, our proposed solution achieves bandwidth close to that of FALCON localhost communication. In Vgg16, it achieves even higher communication bandwidth than localhost. Because the FALCON protocol uses multi-threaded communication, involving frequent thread creation and destruction, and with protocol processing handled by the CPU, the CPU load is high when the communication volume is very large.

[0154] In terms of computation, compared to the crypton scheme, which also uses GPU acceleration and employs floating-point numbers to simulate integers, this invention achieves approximately a 10x performance improvement. Compared to the FALCON protocol using a CPU, although it still only offers a 10x performance improvement in small networks like Dense3, thanks to the powerful parallel computing capabilities of GPUs, it can achieve approximately a 50x performance improvement in medium-sized networks like LeNet, and approximately a 120x performance improvement in large networks like Vgg16.

[0155] In terms of total inference time, this invention offers at least a 10x performance improvement, and nearly a 50x performance improvement compared to FALCON in Vgg16. Millisecond-level inference latency can be achieved in small networks like Dense3, and second-level inference can be achieved in large networks like Vgg16.

[0156] Because this invention uses a 32-bit specific point number, the representation space is smaller compared to the commonly used 64-bit specific point number, which affects the accuracy of model inference. Table 3 shows the inference accuracy of the 32-bit floating-point model in the plaintext, the dynamic 32-bit model used in this invention (with or without quantization-aware training), the 32-bit specific point number model (with or without quantization-aware training), and the 64-bit model used in the crypton framework. For each model, the corresponding neural network was tested on the four datasets described earlier.

[0157] Table 3 Model Accuracy Test

[0158]

[0159] For the Location and Purchase 100 datasets, almost all models achieved similar accuracy. Specifically, our model and the fixed-point model both showed a slight decrease in accuracy compared to the plaintext model, but the decrease was negligible. The parameters and outputs of each layer of both models were within their representation range, and there was no significant difference between our model and the fixed-point model. Whether or not quantization-aware training was used had almost no impact on the model's accuracy. Furthermore, we can see that the model trained using cryptography also achieved similar accuracy to the two models mentioned above; the cryptography scheme has a larger representation space, which is understandable. Overall, there was no significant difference between the various models in both types of fully connected networks.

[0160] For both the MNIST and Cifar 10 datasets, since both use convolutional networks, without quantization-aware methods, both our proposed model and the fixed-point model become unusable. The crypton approach, using a 64-bit representation space, achieves classification accuracy very close to the plaintext model on the MNIST dataset using normal training methods. After applying quantization-aware methods, both our proposed model and the fixed-point model achieve classification accuracy very close to the plaintext model on the MNIST dataset. However, on Cifar 10, even with quantization-aware methods, the fixed-point model remains unusable due to overflow caused by its small representation space. While our proposed model shows an approximately 10% decrease in accuracy compared to the plaintext model, it remains generally usable.

[0161] Because this invention uses a plaintext model, it faces a more severe risk of data leakage compared to traditional schemes that use encrypted models. We tested two membership inference attack schemes: NN attack and ML Privacy Meter, where NN attack is a black-box attack scheme and ML Privacy Meter is a white-box attack scheme. Table 4 shows the NN attack test results for the four models on four datasets. In the crypton model, the NN attack can achieve an accuracy of about 80% on the Location and Purchase 100 datasets, proving that the model output is a major cause of training data leakage, and that even with model encryption, a fairly high attack accuracy can still be achieved. In the two convolutional networks, the attack effect is not as good as that of fully connected networks. In the MNIST dataset, even without any defense measures, the attacker's success probability is about 50%, similar to random guessing. This indicates that membership inference attacks are highly dependent on the dataset and model used; on some simpler tasks, the probability of successful attack is very low because there is no obvious overfitting. The knowledge distillation model achieved good defense results on all datasets, with attackers succeeding nearly 50% of the time without obtaining any useful information. This demonstrates that the knowledge distillation defense method used in this paper can effectively prevent the leakage of training data during the inference process.

[0162] Table 4. NN attack accuracy

[0163]

[0164] Table 5 shows the ML Privacy Meter attack test results of the three models on four datasets. Since the attacker has more information in the white-box attack, the attack accuracy is significantly improved compared to NN attack. However, in the knowledge distillation model, the attacker's success rate is still close to 50%, and the knowledge distillation defense method is still effective.

[0165] Table 5. Accuracy of ML Privacy Meter Attack

[0166]

[0167] The above experimental results are the results of the test conducted on the neural network inference system based on secure multi-party computation in the embodiments of the present invention, which can directly reflect the inference capability of the neural network model of the system.

[0168] While the disclosure is as stated above, its scope of protection is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of this disclosure, and all such changes and modifications will fall within the protection scope of this invention.

Claims

1. A neural network model inference system based on secure multi-party computation, characterized in that, include: The secure multi-party computation module is used to construct a ciphertext computation graph based on the model information of the plaintext model, based on multiple participants, and call the GPU computing module and RDMA communication module to execute the ciphertext computation graph according to the acquired ciphertext, obtain the ciphertext result, and restore the executed ciphertext result to the user. Specifically, it is used based on the FALOCN protocol, employing three parties to construct a secure multi-party computation protocol, and executing the secure multi-party computation protocol according to the ciphertext to execute the ciphertext computation graph; wherein, the three parties include two computation parties and one auxiliary party, the auxiliary party being used to determine whether there are zero elements in the matrix elements to execute the comparison protocol in the secure multi-party computation protocol; The encrypted computation graph is obtained by transforming the model computation graph. The GPU computing module is used to execute the encrypted computation graph using 32-bit integer matrix operations based on the encrypted text sent by the user. The RDMA communication module is used to perform data communication during the computation process of the GPU computing module using the GPU-RDMA communication scheme.

2. The secure multi-party computation based neural network model inference system according to claim 1, wherein, The 32-bit integer matrix operation includes 32-bit integer matrix multiplication, and the GPU computing module is specifically used for: Based on the Cutlass open-source floating-point matrix multiplication library, this paper optimizes the implementation of 32-bit integer matrix multiplication using optimization methods for 32-bit floating-point matrix multiplication. These optimization methods include matrix sharding, shared memory caching, and computation time masking memory access time.

3. The secure multi-party computation based neural network model inference system according to claim 2, wherein, The 32-bit integer matrix operation includes 32-bit integer matrix element operations, and the GPU computing module is specifically used for: The transform operation in the Thrust parallel computing library is used to implement the element operations of the 32-bit integer matrix using a kernel fusion method. The element operations of the 32-bit integer matrix include integer matrix dot product operation and integer matrix addition operation.

4. The secure multi-party computation based neural network model inference system according to claim 3, wherein, The GPU computing module is also specifically used for: The multiplication protocol is executed using the 32-bit integer matrix multiplication to infer the linear and convolutional layers in the plaintext model; The comparison protocol is executed using the element-wise operations of the 32-bit integer matrix to infer the activation layer and pooling layer in the plaintext model.

5. The secure multi-party computation based neural network model inference system according to claim 1, wherein, The secure multi-party computation module is specifically used for: Using different decimal places for different fixed-point matrix variables can be expressed using the first formula as follows: ， wherein, denotes a fixed-point matrix variable the number of bits taken, denotes a fixed-point matrix variable the number of decimal places taken, and denotes the maximum and minimum values in the matrix variable respectively; When calculating the product of two fixed-point matrix variables, the number of bits to be truncated is expressed by the second formula as follows: ， wherein represents a fixed-point number matrix variable the number of decimal places taken, represents a fixed-point number matrix variable the number of decimal places taken, represents a fixed-point number matrix variable the number of decimal places taken.

6. The secure multi-party computation based neural network model inference system according to claim 1, wherein, The secure multi-party computation module is specifically used for: The secure multi-party computation protocol is matrixed.

7. The secure multi-party computation based neural network model inference system according to claim 1, wherein, The RDMA communication module is specifically used for: Enables data transfer between GPUs via RDMA network cards.

8. The secure multi-party computation based neural network model inference system according to claim 1, wherein, The neural network model inference system based on secure multi-party computation also includes: The neural network inference module is used to construct a model computation graph based on the acquired plaintext model, and to extract information from the model computation graph to obtain the model information, wherein the model information includes model nodes and computation functions.

9. The secure multi-party computation based neural network model inference system according to claim 8, wherein, The method for obtaining the plaintext model includes: The neural network model is trained by using a distillation defense method and a quantitative perception training method, and the plaintext model is obtained.

Images