Operation system and method of fully homomorphic encryption neural network model

Abstract

An operating method of a fully homomorphic encryption neural network model, where the fully homomorphic encryption neural network model includes a plurality of layers, includes encrypting a plaintext input with a first encryption algorithm to generate a ciphertext vector for one of the plurality of layers, performing a convolution operation according to the ciphertext vector to generate a result vector, the method further includes converting the result vector into a plurality of result ciphertexts employing a second encryption algorithm, inputting the plurality of result ciphertexts to an incentive function to generate a plurality of encrypted incentive values, and converting the plurality of encrypted incentive values into an output vector employing the first encryption algorithm.

Description

Technical Field

[0001] The present application relates to fully homomorphic encryption and neural networks, and in particular to an operating system and method for a fully homomorphic encryption neural network model. Background Art

[0002] Machine Learning as a Service (MLaaS) allows users to upload data to a neural network model running on a cloud platform for inference. Although MLaaS is extremely convenient, since users' private data is submitted or stored in an external environment, how to maintain data privacy becomes an important issue.

[0003] Fully Homomorphic Encryption (FHE) allows calculations to be performed directly on encrypted data. In other words, the neural network model does not need to use the original data during the calculation process, so the user's data privacy can be ensured. Currently, there are two commonly used FHE schemes for encrypted neural network reasoning: CKKS and FHEW / TFHE. The CKKS scheme supports floating-point operations, so linear operations can be performed efficiently. However, it does not support non-polynomial operations. FHEW / TFHE supports lightweight bit or integer operations, and includes functional bootstrapping technology to enable non-polynomial operations. Therefore, if the above two schemes are combined, the linear operation advantages of CKKS and the functional bootstrapping advantages of FHEW / TFHE seem to be applicable to encrypted neural network models. However, existing experimental data show that the reasoning accuracy of the neural network model established by simply combining these two encryption schemes is significantly reduced. Summary of the invention

[0004] In view of this, the present application proposes an operating system and method for a fully homomorphic encrypted neural network model, thereby solving the problem of decreased reasoning accuracy of the neural network model caused by simply combining the two solutions of CKKS and FHEW / TFHE.

[0005] According to an operation method of a fully homomorphic encrypted neural network model according to one embodiment of the present application, wherein the fully homomorphic encrypted neural network model includes multiple layers, the method includes executing with a processor: for one of the multiple layers, encrypting the plaintext input with a first encryption algorithm to generate a ciphertext vector, performing a convolution operation based on the ciphertext vector to generate a result vector, converting the result vector into multiple result ciphertexts using a second encryption algorithm, inputting the multiple result ciphertexts into an excitation function to generate multiple encrypted excitation values, and converting the multiple encrypted excitation values ​​to generate an output vector using the first encryption algorithm.

[0006] According to an embodiment of the present application, an operating system of a fully homomorphic encrypted neural network model includes a memory and a processor. The memory is used to store multiple instructions. The processor is electrically connected to the memory to execute the multiple instructions, which are used to perform the following operations on one of the multiple layers of the fully homomorphic encrypted neural network model: encrypting plaintext input with a first encryption algorithm to generate a ciphertext vector, performing a convolution operation based on the ciphertext vector to generate a result vector, converting the result vector into multiple result ciphertexts using a second encryption algorithm, inputting the multiple result ciphertexts into an excitation function to generate multiple encrypted excitation values, and converting the multiple encrypted excitation values ​​to generate an output vector using the first encryption algorithm.

[0007] The above description of the disclosed contents and the following description of the implementation modes are used to demonstrate and explain the spirit and principle of the present application, and to provide a further explanation of the scope of the patent application of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 Shown is a block diagram of the operation system of the fully homomorphic encryption neural network model described in one embodiment of the present application.

[0009] Figure 2 Shown is a flowchart of the operation method of the fully homomorphic encryption neural network model described in one embodiment of the present application.

[0010] Figure 3 Shown is a schematic diagram of an operation method of a fully homomorphic encrypted neural network model described in an embodiment of the present application.

[0011] Figure 4 (a) and Figure 4 (b) is a schematic diagram showing an example of range alignment described in an embodiment of the present application.

[0012] Figure 5 Shown is a flowchart of a lookup table perception model fine-tuning method described in an embodiment of the present application.

[0013] Figure 6 Display as Figure 5 A detailed flow chart of a training procedure described in one embodiment of the present application.

[0014] Component number description

[0015] 10 System

[0016] 1 Memory

[0017] 3 Processor

[0018] Steps S1-S5, P1-P5, P11-P12 DETAILED DESCRIPTION

[0019] The detailed features and characteristics of the present application are described in detail in the following embodiments, and the content is sufficient to enable any person skilled in the relevant art to understand the technical content of the present application and implement it accordingly, and according to the content disclosed in this specification, the scope of the patent application and the drawings, any person skilled in the relevant art can easily understand the relevant concepts and characteristics of the present application. The following examples further illustrate the viewpoints of the present application in detail, but do not limit the scope of the present application in any way.

[0020] Figure 1 is a block diagram of an operating system of a fully homomorphic encrypted neural network model according to an embodiment of the present application. Figure 1 As shown, the operating system 10 includes a memory 1 and a processor 3 electrically connected to the memory 1 .

[0021] The memory 1 is used to store a plurality of instructions. In one embodiment, the memory 1 can be implemented by one or more of the following examples: dynamic random access memory (DRAM), static random access memory (SRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), flash memory, and hard disk. The present application does not limit the type and quantity of the memory 1.

[0022] The processor 3 is used to execute multiple instructions stored in the memory 1. In one embodiment, the processor 3 can be implemented by one or more of the following examples: a personal computer, a network server, a microcontroller (MCU), an application processor (AP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system-on-a-chip (SOC), and a deep learning accelerator. The present application does not limit the type and quantity of the processor 3.

[0023] The fully homomorphic encrypted neural network model includes multiple layers, and the multiple instructions are used to perform Figure 2 The process shown. Figure 2 It is a flowchart of an operation method of a fully homomorphic encrypted neural network model according to an embodiment of the present application, including steps S1 to S5. Figure 3is a schematic diagram of an operation method of a fully homomorphic encrypted neural network model according to an embodiment of the present application, showing Figure 2 The input, output and corresponding operations of each step.

[0024] Step S1, encrypting plain text input using a first encryption algorithm to generate a ciphertext vector.

[0025] In one embodiment, the first encryption algorithm is the CKKS (Cheon-Kim-Kim-Song) algorithm, which can be found in "Cheon, JH, Kim, A., Kim, M., Song, Y.: Homomorphic encryption for arithmetic of approximate numbers. In: ASIACRYPT. pp. 409–437. Springer (2017)". CKKS supports arithmetic operations on floating-point vectors. During the encryption process, the plaintext input (floating-point vector) is first encoded into a plaintext polynomial and then encrypted into a ciphertext vector. Therefore, the operation of step S1 can be expressed as CKKS(m) = Enc(Ecd(m)), where m∈Rn represents plaintext input, Ecd(·) represents encoding operation, and Enc(·) represents encryption operation.

[0026] Please note that step S1 is applicable to the first convolutional layer executed in the neural network model. If it is a convolutional layer in the middle of the neural network model, since the result from the previous layer is already ciphertext, step S1 can be omitted and the process can be started directly from step S2.

[0027] Step S2, performing a convolution operation on the ciphertext vector to generate a result vector.

[0028] In one embodiment, the convolution operation can be expressed as fconv'(x)=W'(x)+b', where W' represents the convolution weight and b' represents the bias. The operation of step S2 can be expressed as CKKS.eval(fconv',{ctx0,ctx1,...})={ctx'0,ctx'1,...}, where {ctx0,ctx1,...} represents the ciphertext vector and {ctx'0,ctx'1,...} represents the result vector.

[0029] The evaluation function CKKS.eval(f,·) supported by CKKS takes as input the plaintext function f and the parameters of f following f, such as the plaintext input m or the ciphertext ctx∈CKKS(m). This function operates by executing the function in CKKS with these parameters and returning ctx∈CKKS(f(·)). For example: CKKS.eval(+,ctx,v) returns ctx'∈CKKS(m+v).

[0030] Step S3: Convert the result vector into multiple result ciphertexts using a second encryption algorithm. In one embodiment, the second encryption algorithm is associated with Learning with Error (LWE). The encryption operation of LWE can be expressed as LWE「m」, where「·」represents rounding and encoding operations, and m ∈ R is a scalar.

[0031] There are two implementation methods for LWE: The first is FHEW, please refer to Ducas, L., Micciancio, D.: Fhew: bootstrapping homomorphic encryption in less than a second. In: EUROCRYPT.pp. 617–640. Springer (2015); The second is TFHE, please refer to Chillotti, I., Gama, N., Georgieva, M., Izabachène, M.: TFHE: fast fully homomorphic encryption over the torus. Journal of Cryptology 33(1), 34–91 (2020).

[0032] The conversion operation mentioned in Step S3 is applied to PEGASUS, which is a framework for converting between CKKS ciphertexts and LWE ciphertexts without decryption. Please refer to jie Lu, W., Huang, Z., Hong, C., Ma, Y., Qu, H.: PEGASUS: Bridging polynomial and non-polynomial evaluations in homomorphic encryption. In: 2021 IEEE Symposium on Security and Privacy.pp. 1057–1073. IEEE Computer Society Press (May 2021). Therefore, Step S3 can be expressed as pegasus.extract(ctx) = {ctx’i}. ctx ∈ CKKS(m) represents the result vector, where m ∈ Rn. {ctx’i} ∈ LWE(「mi」) represents the multiple result ciphertexts, where mi ∈ m and 0 ≤ i < n. Step S4: Input the multiple result ciphertexts into an activation function to generate multiple encrypted activation values.

[0033] In one embodiment, the activation function is a rectified linear unit (ReLU). PEGASUS uses a discrete (fine-grained) look-up table (LUT) approximation to evaluate non-polynomial functions such as ReLU. Step S4 can be expressed as pegasus.eval(f ACT ,ctx i )=ctx' i , where f ACT represents the activation function, ctx i ∈LWE(「m i ") represents the multiple result ciphertexts, ctx' i ∈LWE(「f ACT’ (m i )”) represents the multiple encrypted incentive values.

[0034] Step S5: convert the multiple encrypted incentive values ​​into output vectors using a first encryption algorithm.

[0035] In one embodiment, step S5 can be represented as pegasus.repack({cxt' i}=ctx,where ctx' i ∈LWE(「m i ”) represents the multiple encrypted incentive values, and ctx∈CKKS(m) represents the output vector.

[0036] Please note: If the fully homomorphic encrypted neural network model is simply implemented using the CKKS-FHEW / TFHE hybrid encryption inference framework and PEGASUS, its accuracy will be severely reduced.

[0037] Accuracy degradation is caused by accumulated numerical errors during encrypted neural network inference, which are generated by the scale-down procedure in the PEGASUS framework, which limits the numerical precision when applying functional bootstrapping to large-domain ciphertexts.

[0038] FHEW supports functional bootstrapping. Functional bootstrapping is a type of bootstrapping used to reduce error accumulation in homomorphic encryption operations. It involves building a lookup table of a specific function to efficiently calculate the result. Although it requires fewer bootstrapping iterations and less computation, it is only effective for small message domains. If a large message domain is to be supported, it will incur a lot of computational cost.

[0039] Conventional functional guidance methods can only be applied to LWE ciphertexts whose message domain is limited to the size of the lookup table, usually about 210. However, when processing LWE ciphertexts converted from CKKS ciphertexts, a larger message domain is usually required. To solve this problem, PEGASUS introduces a technique for converting between CKKS and FHEW. The technique involves reducing the input ciphertext to a smaller message domain during the conversion from CKKS to FHEW, thereby enabling the expansion of conventional functional guidance to support larger message domains. However, this expansion comes at the cost of reduced accuracy.

[0040] If the input value range guided by the LWE functionality does not align with its intended input domain, the input domain will not be able to satisfy the input value range, or only a small number of lookup table entries will be used. Figure 4 (a) and Figure 4 (b). Figure 4 (a) shows an example where the predetermined input domain [-4, 4) of LWE functional guidance is larger than the input value range [-0.3, 1.7). Therefore, only three lookup table entries are used to map the input value x to the output y. When applying functional guidance, the range mismatch will produce large numerical errors, which will ultimately affect the accuracy of the neural network model.

[0041] Therefore, this application proposes a lookup table-aware (LUT-aware) model fine-tuning method to align the range of input values ​​with the message domain of the ciphertext. By aligning the input value with the lookup table item, such as Figure 4 (b) can alleviate the mismatch. Specifically, after establishing a fully homomorphic encrypted neural network model based on the training data set, the model weights and activation functions need to be adjusted. Please refer to Figure 5 . Figure 5 is a flow chart of a LUT-aware model fine-tuning method according to an embodiment of the present application. Figure 5 The execution process is at least before step S2 "performing a convolution operation based on the ciphertext vector to generate a result vector".

[0042] In step P1, for each layer of the model, the training procedure is performed multiple times to generate multiple plaintext stimulus values. Figure 6 is a flow chart of a training procedure, comprising step P11, performing a convolution operation with a plaintext input to generate a plaintext vector; and step P12, inputting the plaintext vector to an excitation function to generate one of the plurality of plaintext excitation values.

[0043] Step P2, setting a linear mapping range according to the ranges of the plurality of plaintext stimulus values.

[0044] Since the input of the activation function (i.e., the resulting ciphertext) is encrypted in the actual application of the fully homomorphic encryption neural network model, the range of the input of the activation function cannot be predetermined, but varies according to the input value input to the neural network. This application assumes that the distribution of the training data and the test data is similar. Therefore, in order to estimate the range of these inputs, in step P1, the plaintext input in the training data set is inferred using the trained model. In this process, the minimum and maximum values ​​of the activation function output at each layer are observed and recorded, denoted as a and b respectively. i and b i If [a i ,b i ] represents the output range of the activation function of the i-th layer, then the linear image range is set to [-B, +B], where B = max(|a i |,|b i |), where i = 1, 2, ..., n. In other words, the value of +B is greater than the maximum output of the excitation function of each layer, and the value of -B is less than the minimum output of the excitation function.

[0045] Step P3, for each layer, determine the linear mapping function according to the range of the plurality of plaintext stimulus values ​​and the linear mapping range. A simple example is given below to illustrate the method of determining the linear mapping function, but the numerical values ​​in the example are not limiting. Assume a i =2,b i =102, B = 1000. Then the linear mapping function is: f(x) = (xz) * s', where z represents the zero point and s' represents the adjustment factor. The calculation method of the parameters in the linear mapping function is as follows:

[0046] Output range: s = ba = 100

[0047] Zero point: z=(a+b) / 2=52.

[0048] Linear imaging range: S = B-(-B) = 2000

[0049] Adjustment factor: s' = S / s = 20

[0050] Simple verification based on the above obtained linear mapping function f(x) = (x-52) * 20 is as follows:

[0051] f(a)=f(2)=(2-52)*20=-1000=-B

[0052] f(b)=f(102)=(102-52) / 20=1000=B

[0053] Step P4, updating the weight of the convolution operation according to the linear mapping function.

[0054] Linear mapping the input can ensure that the input interval is aligned with the information domain. Incorporating the linear mapping function into the convolution weights and offset values ​​can avoid additional computational costs or reduce memory usage during operation. In one embodiment, step P4 can be expressed as where f CONV '(x) represents the convolution operation after the weight update used in step S2. Represents a linear mapping function that can transform an input in the range [a, b] Linearly mapped to the range [-B,B]. CONV (x) represents the original convolution operation of the model built based on the training data set. The following is a simple example to illustrate how to update the weights:

[0055] Assuming that the original convolution operation is represented by y=Wx+b, and the linear mapping function is represented by f(x)=(xz)*s', then

[0056] f(y)=(Wx+b)*s'-z*s'

[0057] =Wx*s'+b*s'-z*s'

[0058] =(W*s')*x+(b*s'-z*s')

[0059] =W'x+b', where the updated weight W'=W*s', and the updated offset value b'=b*s'-z*s'.

[0060] Step P5, updating the activation function according to the inverse function of the linear mapping function.

[0061] Step P5 can be expressed as f ACT '=f ACT ×f LM -1 , where f ACT ' represents the updated activation function, f ACT represents the original activation function established based on the training data set, f LM -1 represents the inverse function of the linear mapping function. Since the input of the activation function has been adjusted by the linear mapping function, the activation function itself also needs to be adjusted accordingly so that the output includes the same result, that is, f ACT '(f LM (x)) = f ACT (x). In this way, the error caused by the limited accuracy of functional guidance can be greatly reduced.

[0062] Table 1, where D represents the network depth and L represents the number of fully connected layers.

[0063]

[0064] Please refer to Table 1. The inventors conducted a comprehensive experiment to verify the effectiveness of the LUT perception model fine-tuning method proposed in this application on CIFAR-10, a color dataset that is more complex than the MNIST dataset. The experimental results show that compared with the neural network model that does not use the method of this application, the accuracy improved by applying this application can be from 7.5% (such as D7L3) to 15% (such as D8L1). In addition, the accuracy of the fine-tuned neural network model can reach the accuracy of the original neural network model (such as D7L3).

[0065] In summary, the operating system and method of the fully homomorphic encrypted neural network model proposed in this application inherit the advantages of linear operations in CKKS and the advantages of functional guidance in FHEW / TFHE. For the fully homomorphic encrypted neural network model, the CKKS scheme can be used for convolution operations, and the FHEW-TFHE functional guidance scheme can be used for non-polynomial excitation functions and guidance. This application uses PEGASUS to convert between CKKS ciphertext and LWE ciphertext, and proposes a lookup table-aware fine-tuning method to adjust the model weights and excitation functions, thereby improving the accuracy of the fully homomorphic encrypted neural network model.

[0066] Although the present application is disclosed in the above embodiments, it is not intended to limit the present application. Any changes and modifications made without departing from the spirit and scope of the present application are within the scope of patent protection of the present application. Please refer to the attached claims for the scope of protection defined by the present application.

Claims

1. A method for operating a fully homomorphic encrypted neural network model, characterized in that: The fully homomorphic encrypted neural network model includes multiple layers, and the method includes executing with a processor: For one of the plurality of layers, encrypting a plaintext input using a first encryption algorithm to generate a ciphertext vector; Performing a convolution operation according to the ciphertext vector to generate a result vector; Converting the result vector into a plurality of result ciphertexts using a second encryption algorithm; Inputting the plurality of result ciphertexts into an excitation function to generate a plurality of encrypted excitation values; as well as The plurality of encrypted stimulus values ​​are converted into an output vector using the first encryption algorithm.

2. The method for operating a fully homomorphic encrypted neural network model according to claim 1, characterized in that: Also includes: Before performing the convolution operation according to the ciphertext vector to generate the result vector, for each of the plurality of layers, a training procedure is performed multiple times to generate a plurality of plaintext excitation values, wherein the training procedure comprises: performing the convolution operation on the plaintext input to generate a plaintext vector; and Inputting the plaintext vector to the excitation function to generate one of the plurality of plaintext excitation values; Setting a linear mapping range according to the range of the plurality of plaintext stimulus values; For each of the plurality of layers, determining a linear mapping function according to the ranges of the plurality of plaintext stimulus values ​​and the linear mapping range; updating the weight of the convolution operation according to the linear mapping function; and The activation function is updated according to the inverse function of the linear mapping function.

3. The method for operating a fully homomorphic encrypted neural network model according to claim 1, characterized in that: The first encryption algorithm is a CKKS algorithm, and the second encryption algorithm is associated with fault-tolerant learning.

4. The method for operating a fully homomorphic encrypted neural network model according to claim 1, characterized in that: The activation function is a rectified linear unit.

5. An operating system for a fully homomorphic encrypted neural network model, characterized in that: include: a memory for storing a plurality of instructions; as well as a processor electrically connected to the memory to execute the plurality of instructions, the plurality of instructions being used to perform the following operations on one of the plurality of layers of the fully homomorphic encrypted neural network model: Encrypting a plaintext input using a first encryption algorithm to generate a ciphertext vector; Performing a convolution operation according to the ciphertext vector to generate a result vector; Converting the result vector into a plurality of result ciphertexts using a second encryption algorithm; Inputting the plurality of result ciphertexts into an excitation function to generate a plurality of encrypted excitation values; as well as The plurality of encrypted stimulus values ​​are converted into an output vector using the first encryption algorithm.

6. The operating system of the fully homomorphic encrypted neural network model according to claim 5, characterized in that: The plurality of instructions further include the following operations: Before performing the convolution operation according to the ciphertext vector to generate the result vector, for each of the plurality of layers, a training procedure is performed multiple times to generate a plurality of plaintext excitation values, wherein the training procedure comprises: performing the convolution operation on the plaintext input to generate a plaintext vector; and Inputting the plaintext vector to the excitation function to generate one of the plurality of plaintext excitation values; Setting a linear mapping range according to the range of the plurality of plaintext stimulus values; For each of the plurality of layers, determining a linear mapping function according to the ranges of the plurality of plaintext stimulus values ​​and the linear mapping range; updating the weight of the convolution operation according to the linear mapping function; and The activation function is updated according to the inverse function of the linear mapping function.

7. The operating system of the fully homomorphic encrypted neural network model according to claim 5, characterized in that: The first encryption algorithm is a CKKS algorithm, and the second encryption algorithm is associated with fault-tolerant learning.

8. The operating system of the fully homomorphic encrypted neural network model according to claim 5, characterized in that: The activation function is a rectified linear unit.

Images