Neural network model reasoning method and computer equipment

Abstract

The invention provides a neural network model reasoning method and computer equipment. The data party firstly receives the encrypted neural network model sent by the model party. A first linear layer of the encrypted neural network model correspondingly has a first ciphertext linear layer and N-1 second ciphertext linear layers, the first ciphertext linear layer is obtained by directly encrypting the first linear layer by layer, and the second ciphertext linear layers are obtained by encrypting linear layers with all parameters being 0. The first ciphertext linear layer and the second ciphertext linear layer are respectively encrypted by using N fragmentation keys, and the N fragmentation keys are generated based on a distributed encryption protocol supporting homomorphic operation. And the data party performs inference of target data by using the encrypted neural network model to obtain intermediate results output by other layers before the first linear layer, and inputs the intermediate results into the first linear layer and N-1 second linear layers to obtain N outputs corresponding to the N ciphertext linear layers. And obtaining a reasoning result of the target data according to the sum of the N outputs.

Description

Technical Field

[0001] The embodiments of this specification belong to the field of computer application technologies, and particularly relate to a neural network model inference method and a computer device. Background Art

[0002] With the development of machine learning technologies, neural network models have been applied in more and more fields. In some cases, there is a need for two parties to jointly use a neural network model for inference. For example, there may be a model party and a data party. The model party has a neural network model, such as a neural network model for face recognition; the data party has data to be processed. The two parties need to complete data inference and obtain an inference result while ensuring that the data or models they own are not leaked. Summary of the Invention

[0003] The purpose of this specification is to provide a neural network model inference method and a computer device.

[0004] The first aspect of this specification provides a neural network model inference method, which is applied to the data party and includes:

[0005] Receiving an encrypted neural network model sent by the model party; the encrypted neural network model includes a first ciphertext linear layer and N - 1 second ciphertext linear layers, the first ciphertext linear layer and the N - 1 second ciphertext linear layers are respectively encrypted according to N shard keys, the N shard keys are generated based on a distributed encryption protocol that supports homomorphic operations; the first ciphertext linear layer is obtained by encrypting the first linear layer included in the neural network model, and the plaintext of the parameters of the second linear layer corresponding to any one of the second ciphertext linear layers is 0;

[0006] Using the encrypted neural network model to perform inference on target data to obtain intermediate results to be input into the first ciphertext linear layer and the second ciphertext linear layers;

[0007] Inputting the intermediate results into the first ciphertext linear layer and the N - 1 second ciphertext linear layers respectively to obtain N ciphertext outputs;

[0008] Using the sum of the N ciphertext outputs to obtain the inference result of the target data.

[0009] The second aspect of this specification provides a neural network model inference method, which is applied to the model party that has a neural network model and includes:

[0010] Generating N - 1 second linear layers with the parameter values of the first linear layer being 0;

[0011] Obtain N shard keys generated using the first encryption algorithm, and use the N shard keys to encrypt the first linear layer and N - 1 second linear layers respectively to obtain a first ciphertext linear layer and N - 1 second ciphertext linear layers; the first encryption algorithm is implemented based on a distributed encryption protocol that supports homomorphic operations;

[0012] Send the encrypted neural network model to the data party so that the data party can perform inference using the encrypted neural network model.

[0013] The third aspect of this specification provides a neural network model inference device, which is applied to the data party and includes:

[0014] A receiving module, configured to receive the encrypted neural network model sent by the model party; the encrypted neural network model includes a first ciphertext linear layer and N - 1 second ciphertext linear layers, and the first ciphertext linear layer and N - 1 second ciphertext linear layers are respectively encrypted according to N shard keys, and the N shard keys are generated based on a distributed encryption protocol that supports homomorphic operations; the first ciphertext linear layer is obtained by encrypting the first linear layer included in the neural network model, and the plaintext of the parameters of the second linear layer corresponding to any one of the second ciphertext linear layers is 0;

[0015] A first inference module, configured to perform inference on target data using the encrypted neural network model to obtain intermediate results to be input into the first ciphertext linear layer and the second ciphertext linear layers;

[0016] A second inference module, configured to input the intermediate results into the first ciphertext linear layer and N - 1 second ciphertext linear layers respectively to obtain N ciphertext outputs;

[0017] An output module, configured to use the sum of the N ciphertext outputs to obtain the inference result of the target data.

[0018] The fourth aspect of this specification provides a neural network model inference device, which is applied to the model party that owns the neural network model and includes:

[0019] A confusion module, configured to generate N - 1 second linear layers with parameter values of 0 corresponding to the first linear layer;

[0020] An encryption module, configured to obtain N shard keys generated using the first encryption algorithm, and use the N shard keys to encrypt the first linear layer and N - 1 second linear layers respectively to obtain a first ciphertext linear layer and N - 1 second ciphertext linear layers; the first encryption algorithm is implemented based on a distributed encryption protocol that supports homomorphic operations;

[0021] A sending module, configured to send the encrypted neural network model to the data party so that the data party can perform inference using the encrypted neural network model.

[0022] The fifth aspect of this specification provides a computer program product, including a computer program / instructions, which implement the steps of the above-mentioned neural network model inference method when executed by a processor.

[0023] The sixth aspect of this specification provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed in a computer, the computer is made to execute the above-mentioned neural network model inference method.

[0024] The seventh aspect of this specification provides a computing device, including a memory and a processor. An executable code is stored in the memory. When the processor executes the executable code, the above-mentioned neural network model inference method is implemented.

[0025] This specification provides a neural network model inference method. The data party first receives an encrypted neural network model sent by the model party. The first linear layer of the encrypted neural network model correspondingly has a first ciphertext linear layer and N - 1 second ciphertext linear layers. The first ciphertext linear layer is directly obtained by encrypting the first linear layer, and the second ciphertext linear layer is obtained by encrypting a linear layer with all parameters being 0. The first ciphertext linear layer and the second ciphertext linear layers are respectively encrypted with N shard keys, and the N shard keys are generated based on a distributed encryption protocol that supports homomorphic operations. The data party uses the encrypted neural network model to perform inference on target data, obtains intermediate results output by other layers before the first linear layer, inputs the intermediate results into the first linear layer and the N - 1 second linear layers respectively, and obtains N outputs corresponding to the N ciphertext linear layers respectively. According to the sum of the N outputs, the inference result of the target data is obtained.

[0026] The true and false linear layers are respectively encrypted with N shard keys, and the sum of the output results of the true and false linear layers can be decrypted with the aggregation key corresponding to the N shard keys, which improves the security of the encrypted model. Description of the Drawings

[0027] To more clearly illustrate the technical solutions of the embodiments of this specification, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments recorded in this specification. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0028] Figure 1 is a schematic diagram of the system architecture in an embodiment of this specification;

[0029] Figure 2 is a flowchart of the neural network model inference method in an embodiment of this specification;

[0030] Figure 3 is a flowchart of a neural network model inference method in another embodiment of this specification;

[0031] Figure 4 is a schematic diagram of matrix-vector multiplication in an embodiment of this specification;

[0032] Figure 5 is a block diagram of a neural network model inference device in an embodiment of this specification;

[0033] Figure 6 is a block diagram of a neural network model inference device in another embodiment of this specification. Detailed implementation manners

[0034] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings in the embodiments of this specification. Obviously, the described embodiments are only some of the embodiments of this specification, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this specification without making creative efforts shall fall within the protection scope of this specification.

[0035] First, the architecture of the neural network model inference method provided in this specification will be described.

[0036] As Figure 1 shown, the method in this specification involves a model party and a data party. The model party has a trained neural network model. The data party has data to be inferred. In the method of this specification, it is selected to deploy the model of the model party offline in ciphertext form to the data party, and the data party uses this model for inference. In this way, the model party cannot obtain the data of the data party, ensuring the security of the data of the data party. At the same time, the neural network model of the model party is encrypted and deployed to the data party, and the data party cannot obtain the plaintext of the parameters of the neural network model, ensuring the security of the model.

[0037] Among them, the neural network model can be any model including a linear layer, such as a Convolutional Neural Networks (CNN), etc. The neural network model can be a model used for image recognition, natural language processing, speech recognition, etc. The use of the neural network model in this specification is not limited. Optionally, the neural network model can be an image recognition model for face recognition, and the data owned by the data party is data to be face recognized.

[0038] Next, the method shown in this specification will be generally described in combination with Figure 1 the system architecture diagram. When executing, asFigure 1 As shown, the model party first obfuscates the first linear layer of the neural network model to generate N - 1 fake linear layers. The structure of the parameters of the fake linear layers is the same as that of the first linear layer, and the parameter values are 0. For the sake of convenient description, the fake linear layer for the first linear layer is called the second linear layer.

[0039] The model party then generates N shard keys based on a distributed encryption protocol that supports homomorphic operations. Data encrypted with N shard keys can support homomorphic operations. The shard keys have the following properties: The sum of the data encrypted by N shard keys respectively can be decrypted using the aggregation key corresponding to the N shard keys. However, it is impossible to decrypt the data encrypted with one shard key only using the aggregation key. Similarly, it is also impossible to decrypt the sum of the data encrypted by N shard keys respectively using any shard key. Homomorphic operations can include homomorphic addition operations and homomorphic multiplication operations. Here, one kind of homomorphic operation involved in this specification will be described. Suppose [[a]] represents the data obtained by encrypting a with a certain shard key, and b is plaintext data. Then the result of the homomorphic multiplication operation between [[a]] and b is [[a*b]]. In addition, the result of the homomorphic addition operation between [[a]] and [[b]] is [[a + b]].

[0040] As Figure 1 shown, the model party uses N shard keys to encrypt the first linear layer and N - 1 fake linear layers respectively, obtaining the first encrypted linear layer and N - 1 second encrypted linear layers, and thus obtaining the encrypted neural network model.

[0041] This neural network model uses the second linear layer to obfuscate the first linear layer, which makes it impossible for the data party to determine which encrypted linear layer is the first linear layer even when obtaining the encrypted model. Moreover, the data party has no idea that the first encrypted linear layer and the second encrypted linear layers are encrypted with N shard keys. Then, if the data party wants to crack the encrypted linear layers, it needs to crack N shard keys to obtain all the linear layers. It can be seen that encrypting with N shard keys can improve the security of the encrypted neural network model.

[0042] Among them, the value of N can be determined according to a preset obfuscation degree. A high obfuscation degree indicates that the model party has a higher requirement for confidentiality, so N can be taken a little higher. In addition, N cannot be infinitely high because increasing N will increase the number of homomorphic multiplication operations performed by the data party, and homomorphic multiplication operations are time-consuming. Therefore, the size of N can be reasonably determined according to the computational efficiency requirements of the data party and the obfuscation degree requirements of the model party.

[0043] Optionally, to ensure the security of other layers, the second encryption algorithm can also be used to further encrypt the neural network model, ensuring that all layers of the neural network model are in ciphertext and guaranteeing the security of the model.

[0044] In addition, for the inference program of the model, secure compilation, trusted execution environment (TEE), and fully homomorphic encryption can be used to ensure that the data party can perform inference on the model in a ciphertext state. This specification will take secure compilation as an example to illustrate how the data party can achieve ciphertext inference.

[0045] Secure compilation refers to ensuring the security of the compilation process through specific methods and technologies during the software development process to prevent malicious attacks and the introduction of potential vulnerabilities. Secure compilation aims to improve the reliability and security of software and prevent security defects from occurring during the construction, compilation, and deployment of application programs.

[0046] As Figure 1 shown, the inference program can be encrypted through secure compilation, so that the data party cannot obtain the plaintext inference program and can only obtain the ciphertext inference program. At the same time, during the process of the data party running the inference program, the data generated by the inference program cannot be obtained by the data party. This ensures the ciphertext inference of the data party.

[0047] Furthermore, as Figure 1 shown, the model party can send the encrypted neural network model and the encrypted inference program after secure compilation to the data party.

[0048] The data party can use the inference program after secure compilation to perform inference. The specific operations and generated data during the inference process cannot be obtained by the data party. For the specific inference process, in the case of encryption using the second encryption algorithm, the key of the second encryption algorithm written in the inference program can be used to decrypt the neural network model first.

[0049] To further ensure the data security of the neural network model, the neural network model can be decrypted in memory. Because if the decryption result is stored in a file on the hard disk, the data party can obtain the plaintext parameters of the neural network model by reading the file. Decrypting in memory can ensure that the plaintext does not fall to disk and guarantee the security of the model.

[0050] As Figure 1As shown in the figure, the data party can input the target data into the decrypted plaintext model to obtain the intermediate results output by other layers before the first linear layer in the model, and then input the intermediate results into the first ciphertext linear layer and N - 1 second ciphertext linear layers. Inside the ciphertext linear layer, according to the homomorphic multiplication between the plaintext of the intermediate results and the ciphertext of the linear layer parameters, N ciphertext outputs corresponding to the first linear layer and the second linear layer are calculated respectively. As Figure 1 shown, the N ciphertext outputs can be summed to obtain the ciphertext output, and then the inference result can be obtained based on this ciphertext output.

[0051] Since the parameter of the second ciphertext linear layer is 0, the plaintext corresponding to the ciphertext output obtained by the homomorphic multiplication of the parameter of the second ciphertext linear layer and the intermediate results is also 0. Furthermore, by adding the N ciphertext outputs, the ciphertext output corresponding to the first linear layer of the original neural network can be obtained.

[0052] For the specific method of obtaining the inference result based on the ciphertext output, for example, the encrypted inference program can decrypt the ciphertext output using the aggregation key corresponding to the N shard keys and input the decrypted data into the subsequent layers for further inference. If the subsequent layer is also a linear layer, then the inference can be directly performed using this ciphertext output. This specification does not make any limitations in this regard.

[0053] In the above method, through the ciphertext deployment of the neural network model, the inference of the ciphertext inference program, and the decryption using the aggregation key, the confidentiality and non - stealability of the entire inference process of the model are ensured, protecting the security of the model.

[0054] At the same time, this method adopts an efficient security compilation method, with less time consumption. Although homomorphic encryption is used, only one linear layer is encrypted using shard keys, and the homomorphic multiplication operation only needs to be calculated N times at least, which ensures the efficiency of the inference and can ensure that the end - to - end time consumption is less than 1.5 times that of the plaintext time consumption.

[0055] In addition, this method does not require the model to be modified, only an additional N - 1 second linear layers need to be generated.

[0056] Next, from the perspectives of the model party and the data party respectively, combined with Figure 2 and Figure 3 the flowcharts shown, the neural network model inference method provided in this specification will be described.

[0057] First, the method shown in this specification will be described from the perspective of the model party that owns the neural network model. As Figure 2 shown, it includes the following steps:

[0058] Step 201, generate N - 1 second linear layers with parameter values of 0 corresponding to the first linear layer.

[0059] Specifically, first, for the first linear layer in the neural network model, N - 1 fake linear layers corresponding to the first linear layer are generated. The fake linear layers are used for obfuscation. The size of the parameter matrix is the same as that of the first linear layer, but the parameter values are 0. The parameter values being 0 is to facilitate obtaining the ciphertext output that the original first linear layer should have by using the sum of N ciphertext outputs later.

[0060] Among them, a linear layer refers to a layer that performs a linear transformation on the input, generally expressed as y = Wx + b, where W is the weight matrix, b is the bias vector, x is the input, and y is the output. The reason for performing such processing on the linear layer in this specification is that the linear layer can perform linear operations. And the first encryption algorithm is implemented based on a distributed encryption protocol that supports homomorphic operations. Homomorphic operations generally only support linear operations and do not support non - linear operations.

[0061] Among them, the first linear layer can be any linear layer in the neural network model, such as a fully - connected layer, a convolutional layer, and an embedding layer, etc. This specification does not limit the specific form of the first linear layer.

[0062] In an alternative embodiment, the first linear layer is the last fully - connected layer of the neural network model. In the neural network model, the output of the last fully - connected layer is generally input into the activation function, and the output of the activation function is the output of the neural network model. In this way, during the inference process, after the last fully - connected layer, the sum of the N ciphertext outputs can be decrypted to obtain the output of the last fully - connected layer, and the plaintext output is input into the activation function to obtain the output of the neural network model. This can make the inference more flexible.

[0063] It should be noted that, in order to ensure the confidentiality of the inference result and the inference process, the above - mentioned decryption process can be implemented through a confidential inference program, so as to ensure that the inference process is unknown to the data party.

[0064] Step 203, obtain N shard keys generated by using the first encryption algorithm, and use the N shard keys to encrypt the first linear layer and N - 1 second linear layers respectively to obtain a first ciphertext linear layer and N - 1 second ciphertext linear layers.

[0065] Among them, the first encryption algorithm is implemented based on a distributed encryption protocol that supports homomorphic operations.

[0066] Specifically, first, N shard keys generated through a distributed encryption protocol based on homomorphic operation support are obtained, as well as the aggregation key corresponding to the N shard keys. The characteristics of the distributed encryption protocol are described in detail above and will not be elaborated here. The first linear layer and N - 1 fake second linear layers are encrypted respectively by the N shard keys to obtain the first encrypted linear layer and N - 1 second encrypted linear layers. Among them, different encrypted shard keys are used for different linear layers, and the N shard keys correspond to the N linear layers one by one.

[0067] In addition, although the values of the second linear layer are all 0, the data encrypted by different shard keys are different, and it is impossible to distinguish which is the original linear layer only from the first encrypted linear layer and the second encrypted linear layer. Although the first encrypted linear layer and the second encrypted linear layer are used here to distinguish the linear layers from two sources, for the data party, what it receives are N encrypted linear layers, and the N linear layers are not marked which is encrypted from the first linear layer and which is encrypted from the second linear layer, and the data party cannot distinguish the first encrypted linear layer and the second encrypted linear layer.

[0068] Among them, for the specific implementation method of the first encryption algorithm, it can be determined based on the distributed key generation protocol corresponding to homomorphic encryption. Two examples will be given here, and the two examples do not represent a limitation to this specification. One is the first encryption algorithm implemented based on semi - homomorphic encryption: the distributed key protocol based on Paillier. One is the first encryption algorithm implemented based on fully homomorphic encryption: the distributed key protocol implemented based on Cheon - Kim - Kim - Song (CKKS), where the polynomial coefficients corresponding to different shard key encryptions are the same.

[0069] The difference between fully homomorphic and semi - homomorphic lies in whether it supports: the homomorphic multiplication result of [[a]] and [[b]] is [[a*b]]. Both support the homomorphic multiplication operation of plaintext and ciphertext, as well as the homomorphic addition operation between ciphertext and ciphertext. That is, the homomorphic multiplication result of [[a]] and b is [[a*b]]. In addition, the homomorphic addition result of [[a]] and [[b]] is [[a + b]].

[0070] The specific implementation of these two encryption algorithms will be described in detail later and will not be elaborated here.

[0071] In an alternative embodiment, in addition to obfuscating and encrypting the first linear layer, other layers of the neural network model can also be encrypted. In other words, the neural network model is also encrypted based on a second encryption algorithm. The second encryption algorithm can be any encryption algorithm. In an alternative embodiment, the second encryption algorithm can be a symmetric encryption algorithm, such as the Advanced Encryption Standard (AES).

[0072] Regarding the specific encryption scope of the second encryption algorithm, in an alternative embodiment, it can be to encrypt other layers of the neural network model except the first linear layer. In this case, the aggregation key can be written into the encrypted inference program. Although the data party does not know that the first linear layer and the second linear layer are encrypted based on the distributed key, to ensure data security, by writing the aggregation key into the encrypted inference program, the data party cannot obtain the plaintext aggregation key. This can prevent the data party from performing homomorphic addition operations on the first linear layer and the N - 1 second linear layers and decrypting the operation result through the aggregation key to obtain the parameters of the first linear layer.

[0073] In another alternative embodiment, it can be to encrypt all layers of the neural network model, that is, to encrypt other layers except the first linear layer once through the second encryption algorithm, and for the first linear layer and the second linear layer, first encrypt them through the sharding key and then through the second encryption algorithm. In this way, the aggregation key can be written into the encrypted inference program or directly sent to the data party in plaintext. Since for the encrypted neural network model, the first linear layer and the second linear layer are encrypted twice, even if the data party obtains the plaintext of the aggregation key, it cannot decrypt the first linear layer.

[0074] In addition, in the above two cases, the decryption key of the second encryption algorithm can be written into the encrypted inference program to prevent the data party from obtaining the parameters of the plaintext neural network model.

[0075] In an alternative embodiment, as described above, the inference program can be processed so that the data party cannot obtain the plaintext inference program and the intermediate results of the inference program. Here, the security compilation algorithm will be used as an example to illustrate the specific processing process of the inference program.

[0076] The inference program can be obfuscated through secure compilation so that the inference program is in an unknown state to the data party. In addition, the inference program can be implemented based on the Triton Server framework. The core function of Triton Server is to achieve the efficient deployment and large-scale expansion of neural network inference services by uniformly managing models and optimizing resource scheduling, ensuring high-concurrency and low-latency inference performance.

[0077] In some cases, secure compilation can only process programs written in a specific programming language, such as only C++. However, the Triton Server framework is written in another programming language, such as Python. In this case, it is not convenient to directly perform secure compilation on the Triton Server framework. Therefore, the inference part of the neural network model can be separated and inference can be performed through an independent process. This process can be written in a specific programming language and can call the Triton Server framework written in another language. In other words, the inference program calls the unobfuscated Triton service framework during operation.

[0078] In this way, secure compilation can be performed on this to protect the security of the inference program (i.e., this process) and the keys written in the inference program.

[0079] In addition, since the inference program and the inference program are independent of the Triton Server framework, the two need to communicate. To ensure data security, the Triton service framework communicates through pipe communication during operation. Pipe communication is a communication method between processes. Through pipe communication, communication can be achieved through memory, ensuring that the communication content does not fall to disk and protecting the security of the intermediate results output by the inference program.

[0080] Through the above steps, an encrypted neural network model and an inference program of the encrypted neural network model obtained by obfuscation using the secure compilation method can be obtained.

[0081] Step 205: Send the encrypted neural network model to the data party so that the data party can perform inference using the encrypted neural network model.

[0082] After obtaining the above encrypted neural network model, the model can be sent to the data party for offline deployment of the encrypted neural network model by the data party. In this way, the data party can perform inference using the offline-deployed neural network model.

[0083] In an alternative embodiment, the inference program of the encrypted neural network model obtained by performing obfuscation processing using a secure compilation method may also be sent to the data party; wherein, the inference of the target data to obtain the inference result is implemented based on the inference program. This can protect the security of the inference program.

[0084] Next, the model inference method will be described from the perspective of the data party in combination with Figure 3 , as follows: Figure 3 As shown in

[0085] Step 301: Receive the encrypted neural network model sent by the model party.

[0086] The encrypted neural network model includes a first ciphertext linear layer and N - 1 second ciphertext linear layers, which are respectively encrypted according to N shard keys, and the N shard keys are generated based on a distributed encryption protocol that supports homomorphic operations; the first ciphertext linear layer is obtained by encrypting the first linear layer included in the neural network model, and the plaintext of the parameters of the second linear layer corresponding to any of the second ciphertext linear layers is 0.

[0087] This step corresponds to step 205. For the descriptions of the first ciphertext linear layer, the second ciphertext linear layer, the shard key, and the distributed key encryption protocol, please refer to the above, and will not be elaborated here.

[0088] In an alternative embodiment, the data party may also receive the inference program of the encrypted neural network model obtained by performing obfuscation processing using a secure compilation method. For the description of this inference program, please also refer to the above, and will not be elaborated here.

[0089] Step 303: Use the encrypted neural network model to perform inference on the target data to obtain the intermediate results to be input into the first ciphertext linear layer and the second ciphertext linear layers.

[0090] Specifically, the data party may use the layers before the first linear layer of the neural network model to perform inference to obtain intermediate results, and the intermediate results are the inputs that would originally be input into the first linear layer.

[0091] Next, the specific implementation manner of step 303 will be described in combination with the embodiments above.

[0092] As described above, the inference program for performing inference has been obfuscated and encrypted by a secure compilation method. The key of the second encryption algorithm is written in this inference program. Therefore, on the basis that the neural network model is also encrypted based on the second encryption algorithm, the inference program can use the key of the second encryption algorithm written in the inference program to decrypt the encrypted neural network model.

[0093] It should be noted that during decryption, in-memory decryption is used and the plaintext is not written to disk. This can prevent the data party from directly obtaining the decrypted result from the files stored on the hard disk. Moreover, the content stored in memory is often difficult to obtain. Also, the data party cannot obtain the decryption key of the second encryption algorithm written in the inference program.

[0094] After decrypting the encrypted neural network model, the target data can be input into the decrypted layer to obtain the intermediate result output by the layer before the first linear layer.

[0095] Step 305: Input the intermediate result into the first ciphertext linear layer and N - 1 second ciphertext linear layers respectively to obtain N ciphertext outputs.

[0096] Specifically, the plaintext of the intermediate result can be input into N ciphertext linear layers. Each ciphertext linear layer uses the homomorphic multiplication operation of the ciphertext weight matrix W and the input X, and the homomorphic addition operation result of the operation result and the bias vector b to obtain the ciphertext output of this ciphertext linear layer. Then N ciphertext outputs can be obtained.

[0097] It should be noted that although step 305 describes inputting the intermediate result into the first ciphertext linear layer and the second ciphertext linear layer respectively, for the data party's inference program, it cannot determine which is the first ciphertext linear layer. That is, the data party's inference program can input the intermediate result into N ciphertext linear layers, and it is impossible to distinguish which is the first ciphertext linear layer among the N ciphertext linear layers.

[0098] Step 307: Use the sum of the N ciphertext outputs to obtain the inference result of the target data.

[0099] After obtaining the N ciphertext outputs, the homomorphic addition operation result of the N ciphertext outputs can be used to obtain the sum of the N ciphertext outputs. Since the plaintext of the second ciphertext linear layer parameter is 0, correspondingly, the plaintext corresponding to the ciphertext output by the second ciphertext linear layer is also 0. Correspondingly, the plaintext corresponding to the sum of the N ciphertext outputs is actually equivalent to the output of the first linear layer. Furthermore, use the sum of the N ciphertext outputs to obtain the inference result of the target data.

[0100] When there are other linear layers after the first linear layer, the specific implementation process of step 307 can be to input the sum of the N ciphertext outputs into the subsequent linear layer to obtain the output of the subsequent linear layer, and then decrypt the output and input the decrypted result into the non-linear layer subsequent to this linear layer to obtain the inference result.

[0101] In addition, the specific implementation process of step 307 may be that the inference program outputs the inference result of the target data according to the sum of the N ciphertext outputs. Among them, the inference program executes: determining the sum of the N ciphertext outputs; using the aggregation key corresponding to the N shard keys written in the inference program to decrypt the sum of the N ciphertext outputs to obtain the inference result of the target data.

[0102] Specifically, after obtaining the sum of the N ciphertext outputs, it can be decrypted using the aggregation key written in the inference program. Then, the decrypted data is used for inference in the subsequent layers of the first linear layer to obtain the final inference result.

[0103] For example, if the first linear layer is the last fully connected layer of the neural network model, the output of this fully connected layer can be input into an activation function for classification, such as the softmax function, to obtain the final inference result.

[0104] In this way, the number of relatively time-consuming homomorphic operations can be minimized, reducing the time consumed by inference.

[0105] Next, the implementation method of the Paillier distributed key will be described.

[0106] As mentioned above, the model party needs to obfuscate the first linear layer of the neural network model to obtain m - 1 second linear layers. It should be noted that N - 1 second linear layers were mentioned previously. Since N will be used to represent other data later, m here is actually N in the previous text. The number of parameters in the second linear layer is the same as that in the first linear layer, but the parameters in W and b of the second linear layer are all 0. Encrypt the first linear layer and N - 1 second linear layers. Here, taking the encryption process of the W matrix as an example, the encryption processes of the first linear layer and the second linear layer will be described. It can be understood that the encryption process of the bias vector b is the same as that of the weight matrix W and will not be elaborated here. The encryption process includes the following steps:

[0107] 1. Key generation: Given the security parameter π, the model party first generates the public and private keys of the Paillier encryption algorithm, where the private key SK p =(μ,λ), and the public key PK p =(N,g), the order of the public key parameter g is a multiple of N, that is, ord(g)=kN (if k = 1, g = 1 + N, g N =(1 + N) N mod N 2 =1, g kN mod N 2 =1). Then, a large random integer γ is selected such that γ ∈ {0,1} π / 2 and gcd(k,γ)=1, and calculate h = gγ mod N 2 , the public system parameters PP = <π, N, g, h> are publicly disclosed.

[0108] 2. Key splitting: Split the public key PK p The N in the parameters into m random numbers {n1, n2, …, n m} such that m is the total number of the first linear layer and the second linear layer. Then, select a random number as the task ID for each model obfuscation task, and generate a shard key and an aggregated key SK data = <λ, μ, γ> for each linear layer.

[0109] 3. Model encryption: Encrypt the first linear layer and the second linear layer using the m shard keys obtained above and the public system parameters. Specifically, given the linear layer parameter w i ∈Z N , encrypt the model to obtain:

[0110] Correspondingly, the data party performs inference to obtain m ciphertext outputs [[y i . Aggregating the m ciphertext outputs can obtain

[0111] Furthermore, using the aggregated key SK data to decrypt [[y]] can obtain where L(x) = (x - 1) / N.

[0112] Next, the correctness of the above process will be explained.

[0113] According to the encryption formula, the ciphertext inference result can be transformed as shown in formula (1):

[0114]

[0115] Correspondingly, the decrypted result is as shown in formula (2):

[0116]

[0117] It can be seen that the plaintext corresponding to [[y]] is the sum of y (i) .

[0118] Next, the implementation method of the MPCKKS distributed key will be explained. This scheme is based on the CKKS distributed key implementation. CKKS is a fully homomorphic encryption algorithm.

[0119] Before using CKKS encryption, it is first necessary to define R q =Z q [X] / (X N +1), the residue class ring of a polynomial. Simply put, R q is a set of polynomials, where each element is a polynomial with coefficients in {0, 1, …, q - 1} and degree not exceeding N.

[0120] The solution in this specification involves the following operations in the CKKS algorithm:

[0121] 1. KeyGen: The key generation algorithm, generating a private key s ← R q ;

[0122] 2. Ecd: The encoding algorithm, which takes a vector of length less than N and converts it into a polynomial in R q ;

[0123] 3. Enc: The encryption algorithm, which takes a polynomial and a private key and encrypts the polynomial into a ciphertext. Specifically, for a polynomial m ∈ R q , the encryption algorithm first randomly selects a polynomial a ← R q from R q , and a noise polynomial e ← R q (the coefficients of e are relatively small), and then calculates and outputs it;

[0124] 4. HomMulPt: Plaintext-ciphertext multiplication, which takes a plaintext polynomial m′ ∈ R q and a cipher to calculate The new ciphertext encrypts m · m′;

[0125] 5. Dec: The decryption function, which takes a ciphertext (c0, c1) ∈ R q , and calculates c0 + c1 · s;

[0126] 6. Dcd: The decoding function, which takes a polynomial and outputs a vector (the inverse process of the encoding function).

[0127] The reasoning process of the data party is to calculate matrix-vector multiplication, and matrix-vector multiplication can be achieved by using coefficient encoding and calling the plaintext-ciphertext multiplication HomMulPt once. For example, as Figure 4 shown, to calculate the inner product of two vectors , can be mapped to the coefficients of the polynomial in sequence, while the vector is mapped in a certain reverse order, so as to ensure that the constant term after multiplying the two polynomials is the inner product of the two vectors.

[0128] In addition, it should be noted that this specification also improves CKKS. Specifically, during encryption, different ciphertexts share the same random polynomial a ∈ R q . Since the distributed key requires that the aggregated ciphertext of the data encrypted in shards can be decrypted with the aggregated key, by making different ciphertexts share the same polynomial coefficients, CKKS can implement distributed keys.

[0129] During encryption, obfuscated encryption is performed on the first linear layer (which can be a fully connected layer, for example). Regarding the parameters of the first linear layer and the second linear layer, assuming there are two second linear layers and the total number of the first linear layer and the first linear layer is 3, the encryption methods include:

[0130] The model party encodes the parameter matrices of the 3 linear layers into 3 polynomials m0, m1, m2 ∈ R q . Then the model party encrypts these three polynomials with 3 shard keys s0, s1, s2 respectively to obtain three ciphertexts as shown in formula (3).

[0131]

[0132] During the inference process, the data party encodes its own sample vector (i.e., the target data) into a polynomial m′ ∈ R q . Then the data party calls HomMulPt to calculate 3 plaintext-ciphertext multiplications to obtain the ciphertext outputs corresponding to the 3 linear layers, as shown in formula (4).

[0133]

[0134] The data party calculates the homomorphic addition operation of the three ciphertext outputs to obtain the aggregated result of the three ciphertext outputs, (d, a′) = (c0 + c1 + c2, a′). The data party decrypts using the aggregated key, where the aggregated key s = s0 + s1 + s2. The decryption process is as shown in formula (5).

[0135]

[0136] It can be seen that the data party can directly decrypt m′·m, m = m0 + m1 + m2, where m′ is the sample vector and m is the sum of the parameter matrices of the 3 linear layers. The results of these two polynomial multiplications contain the result of the inner product. Therefore, the data party can call the decoding function to recover the result.

[0137] In addition, it should be noted that the CKKS homomorphic encryption algorithm may contain noise, and the noise can be eliminated by selecting appropriate parameters.

[0138] Regarding correctness, it can be directly obtained through the above formula derivation.

[0139] This specification also provides a neural network model inference device, which is applied to a data party, such as Figure 5 including:

[0140] A receiving module 510, configured to receive the encrypted neural network model sent by the model party; the encrypted neural network model includes a first ciphertext linear layer and N - 1 second ciphertext linear layers, and the first ciphertext linear layer and the N - 1 second ciphertext linear layers are respectively encrypted according to N shard keys, and the N shard keys are generated based on a distributed encryption protocol that supports homomorphic operations; the first ciphertext linear layer is obtained by encrypting the first linear layer included in the neural network model, and the plaintext of the parameters of the second linear layer corresponding to any one of the second ciphertext linear layers is 0;

[0141] A first inference module 520, configured to perform inference on target data by using the encrypted neural network model to obtain intermediate results to be input to the first ciphertext linear layer and the second ciphertext linear layers;

[0142] A second inference module 530, configured to respectively input the intermediate results into the first ciphertext linear layer and the N - 1 second ciphertext linear layers to obtain N ciphertext outputs;

[0143] An output module 540, configured to use the sum of the N ciphertext outputs to obtain the inference result of the target data.

[0144] In an alternative embodiment, the neural network model is also encrypted based on a second encryption algorithm.

[0145] In an alternative embodiment, the receiving module 510 is further configured to receive the inference program of the encrypted neural network model obtained by performing obfuscation processing using a secure compilation method; wherein, the inference on the target data to obtain the inference result is implemented based on the inference program.

[0146] In an alternative embodiment, the output module 540 is specifically configured to: the inference program outputs the inference result of the target data according to the sum of the N ciphertext outputs; the inference program performs: determining the sum of the N ciphertext outputs; using the aggregation key corresponding to the N shard keys written in the inference program to decrypt the sum of the N ciphertext outputs to obtain the inference result of the target data.

[0147] In an alternative embodiment, the neural network model is also encrypted based on a second encryption algorithm; the first inference module 520 is further configured to decrypt the encrypted neural network model by using the key of the second encryption algorithm written in the inference program.

[0148] In an alternative embodiment, the inference program calls an unobfuscated Triton service framework during operation; the inference program and the Triton service framework communicate through a pipe during operation.

[0149] In an alternative embodiment, the first linear layer is the last fully connected layer of the neural network model.

[0150] In an alternative embodiment, the distributed encryption protocol supporting homomorphic operations is a distributed key protocol based on Paillier; or, the distributed encryption protocol supporting homomorphic operations is a distributed key protocol implemented based on CKKS, where the polynomial coefficients corresponding to different shard key encryptions are the same.

[0151] As Figure 6 shown, this specification also provides a neural network model inference device, which is applied to a model party having a neural network model, and includes:

[0152] An obfuscation module 610, configured to generate a second linear layer with N - 1 parameter values of 0 corresponding to the first linear layer;

[0153] An encryption module 620, configured to obtain N shard keys generated by using a first encryption algorithm, and use the N shard keys to encrypt the first linear layer and N - 1 second linear layers respectively, to obtain a first ciphertext linear layer and N - 1 second ciphertext linear layers; the first encryption algorithm is implemented based on a distributed encryption protocol supporting homomorphic operations;

[0154] A sending module 630, configured to send the encrypted neural network model to a data party, so that the data party performs inference by using the encrypted neural network model.

[0155] In the 1990s, it was obvious to distinguish whether an improvement to a technology was a hardware improvement (e.g., improvement to circuit structures such as diodes, transistors, switches, etc.) or a software improvement (improvement to method flows). However, with the development of technology, many method flow improvements today can be regarded as direct improvements to hardware circuit structures. Almost all designers obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement to a method flow cannot be implemented with a hardware entity module. For example, a Programmable Logic Device (PLD) (e.g., a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. The designer can program by themselves to "integrate" a digital system on a PLD, without having to ask a chip manufacturer to design and fabricate a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly implemented using "logic compiler" software, which is similar to the software compiler used in program development and writing. The original code before compilation also has to be written in a specific programming language, which is called a Hardware Description Language (HDL), and there is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones currently are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also be clear that as long as the method flow is slightly logically programmed with the above-mentioned several hardware description languages and programmed into the integrated circuit, it is easy to obtain the hardware circuit that implements the logical method flow.

[0156] The controller can be implemented in any suitable manner. For example, the controller can take the form of, for example, a microprocessor or a processor and a computer-readable medium storing computer-readable program code (such as software or firmware) executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of the controller include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in the form of pure computer-readable program code, it is entirely possible to logically program the method steps to enable the controller to be implemented in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. to achieve the same functions. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be regarded as the structures within the hardware component. Or even, the devices for implementing various functions can be regarded as either software modules for implementing the method or structures within the hardware component.

[0157] The systems, devices, modules, or units illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a server system. Of course, this specification does not exclude that with the development of future computer technologies, the computers implementing the functions of the above embodiments can be, for example, personal computers, laptop computers, in-vehicle human-machine interaction devices, cellular phones, camera phones, smart phones, personal digital assistants, media players, navigation devices, email devices, game consoles, tablet computers, wearable devices, or any combination of these devices.

[0158] Although one or more embodiments of this specification provide method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-creative means. The order of steps listed in the embodiments is only one way among many execution orders of steps and does not represent the only execution order. When an actual device or terminal product is executed, it may be executed in the order of the method shown in the embodiments or the drawings or executed in parallel (for example, in an environment of parallel processors or multi-threaded processing, or even in a distributed data processing environment). The terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, product or device comprising a series of elements not only includes those elements but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, product or device. Without further limitation, there is no exclusion of additional identical or equivalent elements in the process, method, product or device comprising the said elements. For example, if terms such as first and second are used to denote names, they do not denote any particular order.

[0159] For the convenience of description, the above device is described by dividing it into various modules according to functions. Of course, when implementing one or more of this specification, the functions of each module can be implemented in the same or multiple software and / or hardware, or the modules implementing the same function can be realized by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces, and the indirect coupling or communication connection of the device or unit can be in electrical, mechanical or other forms.

[0160] This specification is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for realizing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0161] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction device that implements the functions specified in one or more processes and / or blocks Figure 1 in one or more processes and / or blocks Figure 1 specified in the one or more processes and / or blocks.

[0162] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing steps for implementing the functions specified in one or more processes and / or blocks Figure 1 in one or more processes and / or blocks Figure 1 specified in the one or more processes and / or blocks.

[0163] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0164] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM) and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

[0165] Computer-readable media includes permanent and non-permanent, removable and non-removable media and can be implemented by any method or technology for information storage. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage, graphene storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0166] Those skilled in the art should understand that one or more embodiments of this specification can be provided as a method, a system, or a computer program product. Therefore, one or more embodiments of this specification can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, one or more embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0167] One or more embodiments of this specification can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of this specification can also be practiced in a distributed computing environment where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0168] The various embodiments in this specification are described in a progressive manner. For the same or similar parts among the various embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments. In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of this specification. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0169] The above description is only for the embodiments of one or more embodiments of this specification and is not used to limit one or more embodiments of this specification. For those skilled in the art, one or more embodiments of this specification can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification shall be included within the scope of the claims.

Claims

1. A neural network model inference method, applied to a data party, includes: Receiving an encrypted neural network model sent by a model party; the encrypted neural network model includes a first ciphertext linear layer and N - 1 second ciphertext linear layers, and the first ciphertext linear layer and the N - 1 second ciphertext linear layers are respectively encrypted according to N shard keys, and the N shard keys are generated based on a distributed encryption protocol supporting homomorphic operations; the first ciphertext linear layer is obtained by encrypting the first linear layer included in the neural network model, and the plaintext of the parameters of the second linear layer corresponding to any one of the second ciphertext linear layers is 0; Using the encrypted neural network model to perform inference on target data to obtain intermediate results to be input into the first ciphertext linear layer and the second ciphertext linear layers; Inputting the intermediate results into the first ciphertext linear layer and the N - 1 second ciphertext linear layers respectively to obtain N ciphertext outputs; Using the sum of the N ciphertext outputs to obtain the inference result of the target data.

2. The method according to claim 1, wherein the neural network model is further encrypted based on a second encryption algorithm.

3. The method according to claim 1, further including: Receiving an inference program of the encrypted neural network model obtained by performing obfuscation processing using a secure compilation method; wherein the inference to obtain the inference result for the target data is implemented based on the inference program.

4. The method according to claim 3, wherein using the sum of the N ciphertext outputs to obtain the inference result of the target data includes: The inference program outputs the inference result of the target data according to the sum of the N ciphertext outputs; The inference program executes: Determining the sum of the N ciphertext outputs; Using the aggregation key corresponding to the N shard keys written in the inference program to decrypt the sum of the N ciphertext outputs to obtain the inference result of the target data.

5. The method according to claim 3, wherein the neural network model is further encrypted based on a second encryption algorithm; The inference program is further configured to execute: Using the key of the second encryption algorithm written in the inference program to decrypt the encrypted neural network model.

6. The method according to claim 3, wherein the inference program calls an unobfuscated triton service framework during operation; the inference program and the triton service framework communicate through a pipe communication method during operation.

7. The method according to claim 1, wherein the first linear layer is the last fully connected layer of the neural network model.

8. The method according to claim 1, wherein the distributed encryption protocol supporting homomorphic operations is a distributed key protocol based on Paillier; Or, the distributed encryption protocol supporting homomorphic operations is a distributed key protocol implemented based on CKKS, where The polynomial coefficients corresponding to different shard keys during encryption are the same.

9. A neural network model inference method, applied to a model party having a neural network model, includes: Generating N - 1 second linear layers with parameter values of 0 corresponding to the first linear layer; Obtain N shard keys generated using a first encryption algorithm, and use the N shard keys to encrypt the first linear layer and N - 1 second linear layers respectively, to obtain a first encrypted linear layer and N - 1 second encrypted linear layers; the first encryption algorithm is implemented based on a distributed encryption protocol that supports homomorphic operations; Send the encrypted neural network model to the data party, so that the data party uses the encrypted neural network model for inference.

10. A computing device, comprising a memory and a processor, wherein executable code is stored in the memory, and when the processor executes the executable code, the method according to any one of claims 1 - 9 is implemented.

Images