An efficient method for converting a careless neural network
By combining segmented processing and encryption matrix techniques with a noisy generative adversarial network model, the privacy protection problem of complex neural network models is solved, achieving efficient privacy protection transformation, reducing computational and communication overhead, and meeting the needs of real-world application scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HUAWEI TECH CO LTD
- Filing Date
- 2022-04-01
- Publication Date
- 2026-06-26
AI Technical Summary
Existing privacy-preserving machine learning solutions cannot efficiently handle complex multi-layer neural network models such as ResNet101, resulting in high computational overhead and long latency, which cannot meet the needs of real-world use cases.
A segmented processing approach is adopted for multi-layer neural networks, dividing them into two parts. Data encryption is performed using encryption matrices and one-time encryption techniques. A noisy model is trained using generative adversarial networks. Privacy is protected through preprocessing interaction protocols and differential privacy techniques, enabling the computation of linear and nonlinear layers.
It significantly reduces computation time and communication volume, reducing prediction latency from several hours to within 10 seconds, meeting the needs of real-world use cases while protecting the privacy of user input and model parameters.
Smart Images
Figure CN115994559B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of privacy protection in machine learning, and in particular to a more efficient method for unintentional neural network transformation. Background Technology
[0002] Currently, several studies have proposed solutions to the problem of privacy-preserving machine learning. In 2016, Gilad-Bachrach et al. proposed CryptoNets, which for the first time applied fully homomorphic encryption to CNN neural networks for privacy protection, safeguarding data security. However, it requires the CNN model to use a special activation function ("square") and a special pooling operation ("average pooling"), and cannot solve the calculation of functions such as ReLU and Sigmoid. Therefore, this solution cannot be directly used on existing models, and the special activation function and pooling operation will limit the model's prediction performance. In addition, fully homomorphic encryption introduces a large amount of computation and communication overhead: CryptoNets requires 5 minutes and 372MB of bandwidth to process a simple three-layer CNN network request, which is unacceptable in practical applications.
[0003] In subsequent research, Faster CryptosNets used sparse polynomial multiplication to accelerate linear computation. Mohassel and Zhang proposed a new activation function in SecureML and proved that it could be efficiently computed using cryptographic techniques. This activation function was then used in the training phase of the corresponding model. However, the above schemes all have special requirements for the model training process and have significant limitations, making them unsuitable for application on existing models.
[0004] MiniONN (Unintentional Neural Network) utilizes homomorphic encryption, secure multi-square operations, and polynomial approximation to solve the problem of encrypted computation of common functions, achieving better performance through preprocessing. Because MiniONN can transform functions such as ReLU, Sigmoid, and Max pooling, it can be applied to many pre-trained models without incurring significant retraining costs and with almost no impact on model accuracy. However, its transformation methods are only suitable for neural networks with a small number of layers and relatively simple constructions. For more complex, multi-layered networks, its performance cannot meet the corresponding requirements.
[0005] Chiraag Juvekar et al. proposed the Gazelle framework based on MiniONN. Focusing on linear computation, it introduces a more efficient homomorphic encryption method that is more favorable for matrix multiplication. Furthermore, it improves the performance of the linear computation portion of the homomorphic encryption through operations such as packing and rotation, further enhancing efficiency. To date, this approach boasts the best performance, and some of its theoretical components serve as important references for the acceleration algorithm in this project. However, this framework omits the preprocessing component found in MiniONN, which is crucial for improving performance during prediction. Summary of the Invention
[0006] To address the limitations of existing technologies in handling privacy-preserving transformations of ResNet101, and the fact that current privacy-preserving machine learning solutions cannot support complex multi-layered network models like ResNet101 and are unsuitable for real-world applications, this invention provides an efficient, unintentional neural network transformation method. This method solves the privacy protection problem of the complex multi-layered ResNet101 network model when predicting large images (3*363*363). Without altering the existing model, it prevents the server from obtaining user input and the user from accessing cloud-based network model parameters, while still allowing the model to calculate reasonable prediction results based on user input. Compared to previous privacy-preserving transformation solutions, this method significantly improves computational efficiency, drastically reduces communication volume, and lowers prediction latency.
[0007] To achieve the above objectives, the technical solution of the present invention is: an efficient method for unintentional neural network transformation, characterized by comprising the following steps:
[0008] (1) The multilayer neural network is segmented to obtain two parts of the neural network;
[0009] (2) Define a preprocessing interaction protocol and use an accelerated calculation method based on an encryption matrix to generate an encrypted random number matrix between the client and the server;
[0010] (3) The client uses the encrypted random number matrix generated in step (2) as the key to encrypt the input data using one-time encryption technology, and sends the encrypted data to the server for linear layer calculation.
[0011] (4) The server adds Laplace noise to the linear layer calculation results obtained in step (3) and sends it back to the client. The client merges the data and has the linear layer calculation results with added noise.
[0012] (5) Train the noisy model Purifier using a generative adversarial network;
[0013] (6) The client performs nonlinear function calculation on the linear layer operation result after adding noise obtained in step (4), and passes the nonlinear function calculation result through the noise-adding model Purifier trained in step (5). Finally, the noise-adding result is returned to the server in plaintext form, thus completing the unintentional neural network transformation of the first half of the neural network.
[0014] (7) After the server receives the plaintext output in step (6), it completes the calculation of the second half of the multilayer neural network, that is, the prediction is completed.
[0015] Furthermore, step (2) specifically includes the following sub-steps:
[0016] (2.1) Define the preprocessing interaction protocol: The client first generates a protocol that corresponds to the input. Random number matrices of the same size And generate a public-private key pair for encryption. and Using public keys For random number matrix After encryption, an encrypted random number matrix is obtained. The client will encrypt the random number matrix. Send to the server; the server has parameters. and random number matrix The server will generate a file of size [size missing]. random number matrix The random number matrix Size and The results are the same;
[0017] (2.2) Based on the preprocessing interaction protocol defined in step (2.1), the encryption matrix is obtained by using the accelerated calculation method based on the encryption matrix;
[0018] Furthermore, the accelerated calculation method based on the encryption matrix in step (2) is specifically as follows:
[0019] (2.2.1) After arranging the random number matrix r in a special way, it is encrypted and sent using homomorphic encryption technology. The special way of arranging the random number matrix r is as follows: the random number matrix r is expanded and arranged into a matrix that is easy to calculate convolution according to the im2col method, and the data in different columns of the arranged matrix are arranged in a diagonal manner and encrypted in turn into the same ciphertext using the SIMD technology in the homomorphic encryption method.
[0020] (2.2.2) By using the linear parameters of the model Arranged into the corresponding special form, the plaintext matrix is obtained. In the received encrypted random number matrix The above directly completes the process with the plaintext matrix. Multiply, then subtract the generated random number matrix from the result of the multiplication. The result is sent back to the client, which decrypts it and can then directly obtain the matrix. The matrix satisfy:
[0021]
[0022] The linear parameters of the model Arranged into the corresponding special form, the plaintext matrix is obtained. The specific steps are as follows: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error. Based on the special arrangement of matrix r, the arrangement and filling can be made compatible with the encrypted random matrix. Complete the plaintext matrix by multiplying the corresponding positions. The purpose of its arrangement is to maintain the original The multiplication relationship between corresponding positions in the calculation process.
[0023] Furthermore, step (3) specifically includes the following sub-steps:
[0024] (3.1) The client encrypts the encrypted random number matrix r, u, v generated in step (1) using a one-time encryption method to obtain the encrypted data;
[0025] (3.2) Calculate the linear layer of the encrypted data generated in step (3.1), as shown in the following formula:
[0026]
[0027] in:
[0028] , , The client randomly generates a value similar to the input. Random matrices of equal size:
[0029]
[0030] The server randomly generates a value similar to the output. Random matrices of equal size:
[0031]
[0032] The client uses a random number matrix , The matrix is calculated as follows:
[0033]
[0034] satisfy:
[0035]
[0036] Right now:
[0037]
[0038] The client obtains the one-time key through preprocessing. And use a key to encrypt the input. Received the ciphertext ,satisfy:
[0039]
[0040]
[0041] Client sends encrypted text Give it to the server, and the server will use the encrypted text. The calculation is performed, and the result is obtained after passing through a linear layer. :
[0042]
[0043]
[0044] Client C has during preprocessing :
[0045]
[0046] The linear layer result is obtained by adding the data held by the client and the server respectively.
[0047]
[0048] Further, step (4) specifically involves: the server adding noise to the calculation results of the linear layer, the noise distribution conforming to the Laplace distribution formula:
[0049]
[0050] Where μ is the position parameter and b is the scale parameter;
[0051] The server obtains the calculation result of the linear layer with added Laplacian noise and sends the result to the client. The client merges the result with its own linear layer calculation result to obtain the linear layer result with added noise.
[0052] Furthermore, step (6) specifically includes the following sub-steps:
[0053] (6.1) Perform nonlinear function calculation on the noise addition result obtained in step (4);
[0054] (6.2) Input the result of the nonlinear function calculation obtained in step (6.1) into the denoising model Purifier trained in step (5) to obtain the result of the denoising completed;
[0055] (6.3) Return the noise addition result obtained in step (6.2) to the server in plaintext.
[0056] Overall architecture of the invention: The invention is designed with Figure 1 A more efficient unintentional neural network transformation scheme. It mainly consists of three parts:
[0057] (1) The transformation of the linear layer is completed by homomorphic encryption and secret sharing in the MiniONN framework, and the efficiency of the preprocessing process is improved by using the ideas of the improved Gazelle framework.
[0058] (2) A new interaction protocol is designed for the nonlinear layer transformation part, and differential privacy is adopted to prevent the leakage of model parameters.
[0059] (3) By training a good noise-adding model through GAN, the intermediate layer results are passed through the noise-adding model, thereby reducing the user input information exposed by the intermediate layer results.
[0060] The specific workflow is as follows: Client C and server S pre-run a preprocessing procedure to obtain a set of matrices for encryption. Client C uses the generated matrices to encrypt the data using one-time encryption technology and sends it to server S for linear layer calculation, ensuring that server S cannot see the plaintext input throughout the calculation process. After completing the linear layer calculation, server S adds Laplace noise to the result and sends it back to client C. Client C can then obtain the plaintext calculation result of the linear layer with added noise. Subsequently, client C completes the nonlinear function calculation and passes the result through the denoising model Purifier. Finally, the denoised result is returned to server S in plaintext form. After receiving the plaintext, S completes the subsequent calculations of the ResNet101 network, ultimately completing the prediction. It is ensured that the server cannot obtain the input data x and the client cannot obtain the model parameters during the prediction process.
[0061] The beneficial effects of this invention are as follows: By decomposing a multi-layered complex network and introducing GAN (Generative Adversarial Network) noise-adding technology into the traditional MiniONN framework, this invention solves the problem of huge overhead caused by the excessive number of network model layers in previous single-framework conversions. Based on this, MiniONN is optimized and improved in various ways. An accelerated computation algorithm from the Gazelle framework is applied during its preprocessing. The algorithm is further innovatively designed and optimized according to the specific project environment. The current version completes all operations in only 1 / 4 of the original method, while successfully reducing the communication volume to 1 / 40 of the original. Furthermore, the computation can be completed independently on the server side, greatly freeing up client resources. This invention also solves the privacy leakage problem generated in machine services. By utilizing differential privacy technology to construct a new interaction scheme to solve the performance bottleneck problem caused by the obfuscated circuits in the original framework, the time required for image prediction under privacy protection is reduced from several hours to less than 10 seconds, making it able to meet the needs of real-world usage scenarios. Attached Figure Description
[0062] Figure 1 This is a diagram illustrating the overall architecture of the efficient unintentional neural network proposed in this invention.
[0063] Figure 2 Preprocessing protocol diagram;
[0064] Figure 3 A schematic diagram of the improved algorithm for encrypted matrix multiplication;
[0065] Figure 4 Interaction diagram for the prediction process;
[0066] Figure 5 This is a schematic diagram illustrating the training of a GAN-based training model.
[0067] Figure 6 (A) in the figure is obtained using the method of the present invention. Figure 6 The restored image of (B) in the image. Figure 6 (B) in the image is the original input image;
[0068] Figure 7 (A) in the figure is obtained using the method of the present invention. Figure 7 The restored image of (B) in the image. Figure 7 (B) in the image is the original input image;
[0069] Figure 8 (A) in the figure is obtained using the method of the present invention. Figure 8 The restored image of (B) in the image. Figure 8 (B) in the image is the original input image. Detailed Implementation
[0070] This invention provides an efficient method for unintentional neural network transformation. The overall architecture of this invention is shown in the figure below. Figure 1 As shown.
[0071] The method provided in this application is mainly implemented in a system built on the client and server sides. The client can be a mobile phone, tablet, wearable device, in-vehicle device, laptop, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), desktop, laptop, handheld computer, or artificial intelligence device, etc.; the server can be an x86 server, an Advanced Reduced Instruction Set Machine (ARM) server, a Performance Optimized With Enhanced RISC (POWER) server, or a cloud server, etc. This application does not limit the specific types of client and server devices.
[0072] The unintentional neural network transformation method provided in this application mainly includes the following steps:
[0073] (1) Segmentation of multi-layer neural networks: This invention designs an unintentional transformation method for complex multi-layer neural networks like ResNet101, and proposes a segmentation processing scheme for multi-layer neural networks to solve the problem that existing transformation methods would generate huge computational overhead and delay if they were to transform complete complex multi-layer neural networks. Specifically:
[0074] A complete multilayer neural network is truncated in the middle and divided into two parts. The first half of the input is transformed using cryptographic methods through the specific transformation steps in steps (2) to (6). The truncated second half is the normal original neural network, which can obtain normal output results after normal computation. Through the above segmentation processing scheme, the information visible to the server S that owns the model is only the input at the truncated point, and the original input cannot be directly obtained, thus ensuring the privacy of the input data.
[0075] (2) Define a preprocessing interaction protocol. The client C and the server S use a preprocessing generation matrix based on an accelerated computation method using an encryption matrix as the key for one-time encryption. In the linear function encryption transformation scheme, a special set of matrices r, u, and v is used to hide the plaintext. After encryption in this way, the operations used by the model during the calculation process are the same as those used for the plaintext, without incurring additional overhead. However, the generation of this set of vectors itself requires a large number of ciphertext operations. Since it is independent of the model prediction process, it is necessary to choose a preprocessing method to complete this part of the operation. Specifically, it includes the following sub-steps:
[0076] (2.1) Define the preprocessing interaction protocol, specifically as follows:
[0077] The specific operation of the preprocessing protocol is as follows: Figure 2 As shown, client C first generates a value that matches the input. Random number matrices of the same size And generate a public-private key pair for encryption. (Public key) and (Private key), using public key For random number matrix After encryption, an encrypted random number matrix is obtained. Client C will send the encrypted random number matrix Send to server S.
[0078] Server S has parameters and random number matrix The server S will pre-generate a size of random number matrix Its size is The result was the same.
[0079] (2.2) Based on the preprocessing interaction protocol defined in step (2.1), the encryption matrix is obtained by using the accelerated calculation method based on the encryption matrix;
[0080] The accelerated calculation method based on the encryption matrix is as follows:
[0081] (2.2.1) The random number matrix r is arranged in a special way and then encrypted using homomorphic encryption technology before being sent. Specifically, arranging the random number matrix r in a special way involves: expanding the random number matrix r according to the im2col method to form a matrix that facilitates convolution calculation; and then, in the arranged matrix, encrypting the data in different columns sequentially using the SIMD technique in homomorphic encryption into the same ciphertext in a round-robin fashion. For example... Figure 3 For matrix r, data of the same color will be arranged sequentially and then encrypted into the same ciphertext using the SIMD instruction set in the homomorphic encryption method.
[0082] (2.2.2) By using the linear parameters of the model Arranged into the corresponding special form, the plaintext matrix is obtained. In the received encrypted random number matrix The above directly completes the plaintext matrix Multiply, then subtract the generated random number matrix from the result of the multiplication. The result is sent back to client C, which decrypts it and can directly obtain the matrix. , here satisfy:
[0083]
[0084] The model linear parameters Arranged into the corresponding special form, the plaintext matrix is obtained. Specifically: the linear parameters of the model Based on the special arrangement of r, the arrangement and filling can be made compatible with the encrypted random matrix. Complete the plaintext matrix by multiplying the corresponding positions. The purpose of its arrangement is to maintain the original The multiplication relationship between corresponding positions during the calculation process. Figure 3 The product of the ciphertext and the plaintext is the matrix after permutation and filling. .
[0085] plaintext matrix With the encrypted random number matrix The ciphertext result is obtained by multiplying corresponding positions. The problem of in-slot addition being impossible during the original matrix multiplication is solved by adding the ciphertext data. Finally, the ciphertext data to be sent is as follows: Figure 3 The results show that it is equivalent to the size of the encrypted plaintext calculation result, and the random number matrix v can be directly obtained after decryption.
[0086] The main purpose of the proposed accelerated computation method for encrypted matrices is to solve the problem that the slots cannot be added during the computation process using SIMD encryption technology. Compared with the most intuitive scheme of multiplying corresponding rows and columns, decrypting, and then adding, it can reduce the amount of returned data to 1 / n of the original. Compared with the Gazelle method, the proposed algorithm applies the idea of rearrangement and packing to the preprocessing protocol. Moreover, compared with the original Gazelle encrypted matrix multiplication algorithm, it does not use ciphertext rotation operations, which can reduce overhead.
[0087] When used for matrix multiplication, the plaintext needs to be specially arranged. On the server S, we can leverage the server's powerful resources to encode each specially arranged plaintext row and store it in memory. When multiplying the plaintext and ciphertext vectors, we directly read the required data from memory, reducing time complexity through space substitution. Furthermore, since it is the result of the specially arranged convolution parameters, it is universal for all clients C, allowing the server S to store it once and use it multiple times.
[0088] (3) Client C uses the encrypted random number matrix r, u, v generated in step (1) as the key and applies one-time pad (OTP) encryption to the input data. Encryption is performed to obtain the encrypted data, which is then sent to the server for linear layer calculation. Specifically, this includes the following sub-steps:
[0089] (3.1) Client C uses a one-time encryption method to encrypt the encrypted random number matrix r, u, v generated in step (1) to obtain the encrypted data.
[0090] (3.2) Calculate the linear layer of the encrypted data generated in step (2.1), as shown in the following formula:
[0091]
[0092] in:
[0093] , ,
[0094] The core idea of the linear layer computation is to allow the server S and client C to complete the linear computation through secret sharing. Specifically, after adding the input and output values of each layer of the neural network, server S and client C each hold a "share" of the input. The sum of these two shares equals the input of that layer after unintentional transformation by the neural network. The linear layer computation requires generating a set of dot product triples for secret sharing, as shown in the following formula:
[0095] Client C randomly generates a value similar to the input. Random matrices of equal size:
[0096]
[0097] Server S randomly generates a value similar to the output. Random matrices of equal size:
[0098]
[0099] Client C eventually obtained a pass , The calculated matrix:
[0100]
[0101] satisfy:
[0102]
[0103] Right now:
[0104]
[0105] Client C obtains the one-time key through preprocessing. And encrypt using a key Received the ciphertext ,satisfy:
[0106]
[0107]
[0108] Client C then sends Given a value to server S, server S can calculate the result, which is then processed through a linear layer. :
[0109]
[0110]
[0111] Meanwhile, client C has [the ability to] preprocess [the data]. :
[0112]
[0113] Clearly, the linear layer result is obtained by adding the data possessed by client C and server S respectively.
[0114]
[0115] Therefore, at the end of the interaction, after the linear transformation layer calculation is completed, the server S and client C share the output value generated by the addition. The server S did not know anything about the input. No one party knows the details individually. This achieves protection The purpose.
[0116] Many operations in machine learning model networks are essentially linear functions, and this method can be used to transform them into unintentional neural network models. Fully connected operations (Linear) can obviously be directly processed using the above method. Convolution operations (Conv) can be transformed into the result of multiplying two matrices using the im2col method. When normalization operations (BatchNormal) are used in the prediction stage, the parameters are fixed, but their essence can still be regarded as matrix operations. Average pooling can also be completed through this transformation method.
[0117] (4) The server S sends the result of the linear layer calculation in step (3) back to the client after adding Laplace noise using the differential privacy method. The client then merges the data and obtains the linear layer calculation result with added noise, specifically:
[0118] After step (2) is completed, the server S and the client C share the linear layer output. Only when the data they have are added together can the original result of the linear layer operation be obtained, that is, the output of the nonlinear layer. Noise is added to the output data of the nonlinear layer of the server S. The noise distribution conforms to the Laplace distribution formula: where μ is the location parameter and b is the scale parameter.
[0119]
[0120] Noise that conforms to a distribution is generated. By controlling the size of the parameters, the impact of the noise on the final result can be adjusted. The server S adds the generated noise to its data and sends the result to the client C. The client C merges the noise with its own data to obtain the linear layer result with noise.
[0121] In other solutions, secure multi-party computation is used to complete data merging and nonlinear layer operations. However, the delay and computational overhead caused by using secure multi-party computation for this part of the processing are unacceptable. To solve this problem, nonlinear function calculations need to be performed in plaintext. If any party directly obtains the linear layer output, it will cause leakage of model parameters or input data information. The solution described in step (4) can use differential privacy methods to add noise to alleviate the leakage problem, while making the nonlinear function calculated in plaintext.
[0122] (5) Train the noisy model Purifier based on GAN (Generative Adversarial Network) using the ImageNet dataset; specifically including the following sub-steps:
[0123] (5.1) The training is based on the generative adversarial network training method, where the generator G is the final noisy model. It is essentially an autoencoder that ensures that the output and input are exactly the same size. Its input is the result of the intermediate layer of the original ResNet101, and its output is the noisy result. Its training goal is to minimize the restoration ability of the restoration model H as much as possible, and to make the discriminator I unable to distinguish whether noise has been added.
[0124] (5.2) The goal of discriminator I is to determine whether noise has been added to the intermediate layer results. It outputs a Boolean value to indicate whether its input has passed through generator G. It can help train the noisy model. Its main implementation method is to map the input to a one-dimensional result, and use the one-dimensional result to indicate whether noise has been added.
[0125] (5.3) The goal of the restoration model H is to restore the intermediate layer results of ResNet101 to the size of the user input. Its input is the output of the generator G, and the output is the restored image. Its main structure consists of a combination of convolution and deconvolution, activation functions, etc., which can convert the intermediate layer results into the size of the user input.
[0126] During training, the generator G is used to compete against the discriminator model I and the reconstruction model H. While controlling the impact of the noise added by the generator G on the prediction accuracy of the original ResNet101 to within 10%, the maximum amount of noise is added as much as possible to reduce the attacker's ability to reconstruct the input from the intermediate results using H. The trained generator G will be used as the noise-adding model Purifier in the complete conversion system. Although the training operation is complex, it only needs to be performed once, and users can use it directly when making predictions.
[0127] In the training phase of machine learning models, the loss function has a significant impact on the model's performance. Since training a noisy model using GANs involves the coordination of multiple models, the loss function used is relatively complex, as follows:
[0128] The loss function of the noisy model G during training consists of four parts, which can be expressed as:
[0129]
[0130] The training objective is to minimize the value of L by adjusting... , The value of can adjust the proportion of each loss component, thereby controlling the noise addition effect of the noise-adding model G. (Figure 1) represent Norm, Represents cross-entropy loss, y The labels representing the predicted images are as follows:
[0131] The similarity between the autoencoder output and the input is represented by , where x represents the original input image, F(x) represents the output at the truncation point after passing through the ResNet101 network (i.e., the intermediate result), and G(F(x)) represents the output of the intermediate result after passing through the noisy model. To reconstruct the loss function, in the example:
[0132]
[0133] The difference between the model's prediction result after adding noise and the original result is represented by C(G(F(x))), where C(G(F(x))) represents the intermediate result after adding noise and inputting it back to ResNet101 for calculation, and y represents the original normal output of ResNet101. :
[0134]
[0135] The similarity between the image restored by the restoration model and the original image is represented by H(G(F(x))), where H(G(F(x)) represents the image obtained by the restoration model from the intermediate result after adding noise. For the reconstruction loss function:
[0136]
[0137] This represents the discriminant's ability to distinguish, where I(G(F(x))) represents the output of the discriminant after the intermediate results have passed through it.
[0138]
[0139] (6) Client C performs nonlinear function calculations on the linear layer operation results obtained in step (4) after adding noise, and then uses the noise-adding model Purifier trained in step (5) to return the noise-adding result to the server in plaintext. Specifically, this includes the following sub-steps:
[0140] (6.1) Perform nonlinear function calculation on the noise-added result obtained in step (4). Since this process is performed in plain text, it supports arbitrary nonlinear function calculation, including but not limited to the following nonlinear functions commonly used in machine learning: Max-Pooling, ReLU, Sigmoid, Square.
[0141] The similarity between the output and input results of the autoencoder is represented by the reconstruction loss function. The difference between the predicted result of the model after adding noise and the original result is represented by the reconstruction loss function. The similarity between the restored image and the original image is represented by the reconstruction loss function. The discriminant ability is represented by the reconstruction loss function. (6.2) Input the result of the nonlinear function calculated in step (6.1) into the noise-adding model Purifier trained in step (5) to obtain the result after noise addition.
[0142] (6.3) Return the noise addition result obtained in step (6.2) to the server S in plaintext.
[0143] (7) After the server obtains the plaintext output in step (6), this plaintext is the intermediate result of the segment after adding noise to the original resnet101. The server uses this result to perform the normal operation of the second half of the resnet101 network and finally completes the prediction work of the second half of the neural network.
[0144] The method of this invention can hide the input image, but because the acceleration process uses a segmented processing method, attackers may use intermediate results to reconstruct the input image. To solve this problem, this method designs and trains a noisy model. Figure 6 (A) Figure 7 (A) and Figure 8 In the image (A), the restored image is obtained after the intermediate result of the method of the present invention is processed by the noise-adding model. Figure 6 (B) Figure 7 (B) and Figure 8 In (B) of the image, the original input image is shown. The sensitive information is shown within the thick box. As can be seen from the comparison of the two images, the text in the restored image is blurry compared to the input image. The sensitive information in the text can no longer be recognized. Furthermore, the clarity of the output image is greatly reduced, and the details cannot be distinguished. This can protect privacy.
[0145] Table 1: Comparison of communication volume between the original algorithm and the method of the present invention during preprocessing operations.
[0146]
[0147] Table 2: Preprocessing Operation Completion Schedule
[0148]
[0149] As shown in Tables 1 and 2 above, the method of this invention makes various optimizations and improvements to MiniONN, attempting to apply the accelerated computation algorithm in the Gazelle framework during its preprocessing. Further innovative design and optimization of the algorithm are carried out based on the specific project environment. The current version completes all operations in only 1 / 4 of the original method, while successfully reducing the communication volume to 1 / 40 of the original. Furthermore, the computation operations can be completed independently on the server side, greatly freeing up client resources. This invention solves the privacy leakage problem generated in machine services. By utilizing differential privacy technology to construct a new interaction scheme to solve the performance bottleneck problem caused by the obfuscated circuit in the original framework, the time required for image prediction under privacy protection is reduced from several hours to less than 10 seconds, making it able to meet the needs of real-world usage scenarios.
[0150] In summary, the method of this invention can be applied to complex neural network models such as ResNet101 to achieve the purpose of privacy protection. Compared with previous solutions, it can greatly reduce the prediction time and reduce the additional communication and computation introduced by encryption schemes.
Claims
1. A highly efficient method for unintentional neural network transformation, characterized in that, Includes the following steps: (1) The multilayer neural network is segmented to obtain two parts of the neural network; (2) Define a preprocessing interaction protocol and use an accelerated calculation method based on an encryption matrix to generate an encrypted random number matrix between the client and the server; (3) The client uses the encrypted random number matrix generated in step (2) as the key to encrypt the input data using one-time encryption technology, and sends the encrypted data to the server for linear layer calculation. (4) The server adds Laplace noise to the linear layer calculation results obtained in step (3) and sends it back to the client. The client merges the data and has the linear layer calculation results with added noise. (5) Train the noisy model Purifier using a generative adversarial network; (6) The client performs nonlinear function calculation on the linear layer operation result after adding noise obtained in step (4), and passes the nonlinear function calculation result through the noise-adding model Purifier trained in step (5). Finally, the noise-adding result is returned to the server in plaintext form, thus completing the unintentional neural network transformation of the first half of the neural network. (7) After the server receives the plaintext output in step (6), it completes the calculation of the second half of the multilayer neural network, that is, the prediction is completed; The accelerated calculation method based on the encryption matrix in step (2) is as follows: (2.2.1) After arranging the random number matrix r in a special way, it is encrypted and sent using homomorphic encryption technology. The special way of arranging the random number matrix r is as follows: the random number matrix r is expanded and arranged into a matrix that is easy to calculate convolution according to the im2col method, and the data in different columns of the arranged matrix are arranged in a diagonal manner and encrypted in turn into the same ciphertext using the SIMD technology in the homomorphic encryption method. (2.2.2) By using the linear parameters of the model Arranged into the corresponding special form, the plaintext matrix is obtained. In the received encrypted random number matrix The above directly completes the process with the plaintext matrix. Multiply, then subtract the generated random number matrix from the result of the multiplication. The result is sent back to the client, which decrypts it and can then directly obtain the matrix. The matrix satisfy: The linear parameters of the model Arranged into the corresponding special form, the plaintext matrix is obtained. The specific steps are as follows: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error. Based on the special arrangement of matrix r, the arrangement and filling can be made compatible with the encrypted random matrix. Complete the plaintext matrix by multiplying the corresponding positions. The purpose of its arrangement is to maintain the original The multiplication relationship between corresponding positions in the calculation process.
2. The unintentional neural network conversion method according to claim 1, characterized in that, Step (2) specifically includes the following sub-steps: (2.1) Define the preprocessing interaction protocol: The client first generates a protocol that corresponds to the input. Random number matrices of the same size And generate a public-private key pair for encryption. and Using public keys For random number matrix After encryption, an encrypted random number matrix is obtained. The client will use the encrypted random number matrix Send to the server; the server has parameters. and random number matrix The server will generate a file of size [size missing]. random number matrix The random number matrix Size and The results are the same; (2.2) Based on the preprocessing interaction protocol defined in step (2.1), the encryption matrix is obtained by using the accelerated calculation method based on the encryption matrix.
3. The unintentional neural network transformation method according to claim 1, characterized in that, Step (3) specifically includes the following sub-steps: (3.1) The client encrypts the encrypted random number matrix r, u, v generated in step (1) using a one-time encryption method to obtain the encrypted data; (3.2) Calculate the linear layer of the encrypted data generated in step (3.1), as shown in the following formula: in: , , The client randomly generates a value similar to the input. Random matrices of equal size: The server randomly generates a value similar to the output. Random matrices of equal size: The client uses a random number matrix , The matrix is calculated as follows: satisfy: Right now: The client obtains the one-time key through preprocessing. And use a key to encrypt the input. Receive ciphertext ,satisfy: Client sends encrypted text Give it to the server, and the server will use the encrypted text. The calculation is performed, and the result is obtained after passing through a linear layer. : Client C has during preprocessing : The linear layer result is obtained by adding the data held by the client and the server respectively.
4. The unintentional neural network transformation method according to claim 1, characterized in that, Step (4) specifically involves: the server adding noise to the calculation results of the linear layer, the noise distribution conforming to the Laplace distribution formula: Where μ is the position parameter and b is the scale parameter; The server obtains the calculation result of the linear layer with added Laplacian noise and sends the result to the client. The client merges the result with its own linear layer calculation result to obtain the linear layer result with added noise.
5. The unintentional neural network conversion method according to claim 1, characterized in that, Step (6) specifically includes the following sub-steps: (6.1) Perform nonlinear function calculation on the noise addition result obtained in step (4); (6.2) Input the result of the nonlinear function calculation obtained in step (6.1) into the denoising model Purifier trained in step (5) to obtain the result of the denoising completed; (6.3) Return the noise addition result obtained in step (6.2) to the server in plaintext.