Cipher state reasoning method of neural network model, computer equipment and program product

Abstract

On the basis, the invention provides a secret state reasoning method of a neural network model. The method is performed through acceleration hardware. A model for which the method aims is a quantized neural network model, and the neural network model comprises a first linear layer and a first nonlinear layer behind the first linear layer. The method comprises the following steps of: firstly, calculating first intermediate data after first input data subjected to fully homomorphic encryption processing is processed by a first linear layer by utilizing a plurality of basic arithmetic units, used for executing basic homomorphic operators, of acceleration hardware; and searching ciphertext input corresponding to the first intermediate data in a first lookup table stored in the storage unit to obtain an operation result of the first nonlinear layer. Wherein the mapping relation between the ciphertext input and the ciphertext output of the first nonlinear layer is recorded in the first lookup table. In the specification, a quantitative model is adopted for reasoning, and the requirement for the ciphertext modulus Q and the polynomial order is lower. In addition, the non-linear function is realized based on the lookup table, so that the reasoning precision is also improved.

Description

Technical Field

[0001] This specification relates to the field of privacy computing technology, and more particularly to a method for dense inference of a neural network model, a computer device, and a program product. Background Technology

[0002] Homomorphic encryption is an encryption algorithm that first encrypts plaintext data, then performs addition and multiplication operations on the encrypted ciphertext. The decrypted result is equivalent to the result of directly performing addition and multiplication operations on the plaintext data. Homomorphic encryption algorithms allow data providers to analyze and process data without revealing the original data.

[0003] The properties of homomorphic encryption algorithms make them widely used in scenarios requiring privacy protection. For example, ... Figure 1 As shown, in the method of joint inference of a neural network model by two parties, the data party possesses the data, and the model party possesses the model. The data party sends the data to the model party after homomorphic encryption. The model party then performs calculations on the encrypted data to obtain the encrypted inference result, and sends the encrypted inference result back to the data party so that the data party can decrypt it to obtain the inference result.

[0004] However, existing neural network models with dense-state inference methods suffer from poor usability. Summary of the Invention

[0005] In view of the above, this specification provides a method for dense-state inference of a neural network model, a computer device, and a program product through one or more embodiments.

[0006] According to a first aspect of one or more embodiments of this specification, a method for encrypted inference of a neural network model is proposed, which accelerates hardware execution. The neural network model is a quantized neural network, including a first linear layer and a subsequent first nonlinear layer. The accelerating hardware includes a storage unit and several basic operation units for executing several basic homomorphic operators. A first lookup table in the storage unit records a mapping between the encrypted input and corresponding encrypted output of the first nonlinear layer. The method includes:

[0007] Using the aforementioned basic computing units, the first intermediate data after the first input data has been processed by the first linear layer is calculated; the first input data is ciphertext data that has undergone fully homomorphic encryption.

[0008] The first intermediate data is used to search the first lookup table to obtain the corresponding ciphertext output as the output result of the first nonlinear layer.

[0009] According to a second aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the dense-state inference method of the neural network model as described in the first aspect of the embodiments of this specification.

[0010] According to a third aspect of the embodiments of this specification, a computer device is provided, the computer device comprising:

[0011] processor;

[0012] Memory used to store processor-executable instructions;

[0013] The processor implements the dense-state inference method for a neural network model as described in the first aspect of the embodiments of this specification by running the executable instructions.

[0014] According to a fourth aspect of the embodiments of this specification, a computer program product is provided, which, when executed by a processor, implements the dense-state inference method of the neural network model described in the first aspect of the embodiments of this specification.

[0015] Based on this, this specification provides a method for encrypted inference of a neural network model. This method accelerates hardware execution. The model targeted by this method is a quantized neural network model, which includes a first linear layer and a first nonlinear layer following the first linear layer. First, using several basic operational units of the accelerated hardware for executing basic homomorphic operators, the first intermediate data, after processing the first input data (which has undergone fully homomorphic encryption) through the first linear layer, is calculated. Then, the ciphertext input corresponding to the first intermediate data is searched in a first lookup table stored in the storage unit to obtain the operation result of the first nonlinear layer. The first lookup table records the mapping relationship between the ciphertext input and ciphertext output of the first nonlinear layer.

[0016] This specification employs a quantized model for inference. Compared to floating-point parameters in related technologies, it eliminates the need for expansion factors to guarantee inference accuracy. Homomorphic operations introduce less noise, resulting in lower requirements for the ciphertext modulus Q and polynomial order in dense-state inference. This makes the method presented in this specification more hardware-friendly and easier to use.

[0017] Furthermore, because dense-state inference is performed using a quantized model, the inputs and outputs of the nonlinear layers in the quantized model are discrete numerical values, unlike models with floating-point parameters. This makes it possible to perform nonlinear function calculations using lookup tables. Compared to nonlinear functions implemented through series expansion, nonlinear functions implemented using lookup tables have higher accuracy and improved ease of use.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this specification. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this specification and, together with the description, serve to explain the principles of this specification.

[0020] Figure 1 This is a schematic diagram of a type of dense state inference in related technologies.

[0021] Figure 2 This is a flowchart illustrating a dense-state inference method for a neural network model as shown in this specification.

[0022] Figure 3 This is a schematic diagram illustrating a linear layer operation as shown in this specification.

[0023] Figure 4 This is a schematic diagram illustrating another linear layer operation as shown in this specification.

[0024] Figure 5 This is a schematic diagram illustrating another type of linear layer operation as shown in this specification.

[0025] Figure 6 This is a schematic diagram illustrating analog-to-digital switching, encrypted format conversion, and lookup table as shown in this specification.

[0026] Figure 7 This is a schematic diagram illustrating a dense-state inference method for a neural network model provided in this specification.

[0027] Figure 8 This is a schematic diagram of the processing procedure of a sample extraction unit as shown in this specification.

[0028] Figure 9 This is a hardware structure diagram of a function multiplexing unit shown in this specification.

[0029] Figure 10 This is a structural diagram of an acceleration hardware shown in this specification.

[0030] Figure 11 This is a schematic diagram of a fully automated production line design shown in this specification.

[0031] Figure 12 This is a hardware structure diagram of a computer device shown in this specification. Detailed Implementation

[0032] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0033] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0034] To facilitate understanding, we will first explain homomorphic encryption algorithms.

[0035] Given plaintexts a and b, and their respective homomorphic ciphertexts E(a) and E(b) obtained through homomorphic encryption algorithms, a homomorphic encryption algorithm supporting addition homomorphic operations can achieve: E(a) + E(b) = E(a + b), and a homomorphic encryption algorithm supporting multiplication homomorphic operations can achieve: E(a) × E(b) = E(a * b). Here, the × and + operations in the ciphertext operations represent homomorphic multiplication and homomorphic addition, respectively. Homomorphic encryption algorithms ensure that data is "usable but not visible," making them widely used in scenarios requiring privacy protection.

[0036] Homomorphic encryption algorithms that only support addition or multiplication homomorphic operations are called semi-homomorphic encryption. Fully homomorphic encryption (FHE) is a special type of homomorphic encryption algorithm that supports both addition and multiplication homomorphic operations.

[0037] The parameters of models in related technologies are generally floating-point numbers. For example, in convolutional neural networks (CNNs), inference is typically performed based on floating-point numbers. However, performing dense-state inference in neural networks based on floating-point numbers presents the following problems:

[0038] First, the parameter settings are not hardware-friendly.

[0039] For FHE, each homomorphic operation performed on the ciphertext increases the noise it contains. When the noise exceeds a certain threshold (typically Q / 2, where Q is the ciphertext modulus), decryption will fail. To control the noise growth of the homomorphic ciphertext, during the model's encrypted inference process, after performing a certain number of homomorphic operations on the ciphertext, a bootstrapping operation is performed to reset or reduce (not completely eliminate) the noise, allowing the homomorphic ciphertext to continue undergoing homomorphic operations. It should be noted that while the bootstrapping operation can reduce ciphertext noise, it often introduces some noise itself.

[0040] In related technologies, for models such as CNNs, since model parameters are generally floating-point numbers, high precision is required during secret inference to ensure the accuracy of the inference results. Specifically, CKKS (Cheon-Kim-Kim-Song) is generally used to perform encryption and homomorphic operations on floating-point numbers. In the CKKS-based FHE method, the precision of the ciphertext depends on the precision of the binary decimal, and noise increases with the increase of homomorphic operations. This noise gradually accumulates and erodes the precision of the result, especially in deep CNNs. To ensure the accuracy of the secret inference results and the precision of the ciphertext, the parameter settings need to use a large expansion factor (Δ) and sufficient noise space to tolerate the precision loss during the inference process.

[0041] The specific function of the expansion factor is that, during the ciphertext encoding stage, the plaintext to be encrypted needs to be multiplied by the expansion factor, then rounded to the nearest integer, and then the rounded result is encrypted. This makes it easier to embed the plaintext into the polynomial ring for encryption.

[0042] Therefore, the higher the required ciphertext precision, the larger the required expansion factor. A larger expansion factor leads to a greater increase in noise introduced by homomorphic operations, thus requiring a larger noise space to tolerate the noise. This necessitates higher ciphertext modulus and polynomial order. Consequently, to support larger CNN models (larger CNNs involve more homomorphic operations, further increasing noise), existing inference schemes often use larger polynomial orders (e.g., 2^25). 16 The ciphertext modulus Q is used to meet the requirements of multiplication depth and bootstrap operation.

[0043] To accelerate the homomorphic operation process, a dedicated hardware accelerator (hereinafter referred to as accelerated hardware) can be introduced. However, a large polynomial order and ciphertext modulus Q will result in large ciphertext and key values, which poses a significant challenge to the design and implementation of accelerated hardware for homomorphic operations.

[0044] Specifically, to support these large parameter settings, accelerated hardware requires significant on-chip storage resources and bandwidth to store the ciphertext or key generated during computation. This, in turn, introduces high area overhead. Furthermore, homomorphic operations on large ciphertexts also introduce significant power consumption. This makes it difficult to accelerate encrypted inference using dedicated accelerated hardware in related technologies, thus making the encrypted inference methods of neural network models in related technologies less usable and difficult to accelerate using dedicated hardware.

[0045] Second, the accuracy introduced by nonlinear functions is uncontrollable.

[0046] Homomorphic ciphertext only supports homomorphic addition and homomorphic multiplication, making it difficult to support operations on nonlinear functions in the model. Related techniques typically use series expansion (such as Taylor expansion) to expand the nonlinear function into a computable polynomial, thus approximating the actual value of the nonlinear function.

[0047] For nonlinear functions, higher-order expansions can improve the accuracy of the results. However, higher-order expansions often introduce more homomorphic multiplication operations, further increasing the noise in the ciphertext. When the expansion factor is not large enough, the order of the series expansion often fails to further improve the accuracy of the nonlinear function results after exceeding a certain value.

[0048] Therefore, to meet the accuracy requirements of inference, related technologies often require adjusting the choice of activation function and setting reasonable FHE parameters. Moreover, even with reasonable FHE parameter settings, the nonlinear function calculation results obtained through series expansion of nonlinear functions still differ significantly in accuracy from those obtained from the calculation of nonlinear functions in plaintext.

[0049] It is evident that the reduced accuracy caused by nonlinear functions diminishes the usability of dense-state inference methods.

[0050] Based on this, this specification provides a method for encrypted inference of a neural network model. This method accelerates hardware execution. The model targeted by this method is a quantized neural network model, which includes a first linear layer and a first nonlinear layer following the first linear layer. First, using several basic operational units of the accelerated hardware for executing basic homomorphic operators, the first intermediate data, after processing the first input data (which has undergone fully homomorphic encryption) through the first linear layer, is calculated. Then, the ciphertext input corresponding to the first intermediate data is searched in a first lookup table stored in the storage unit to obtain the operation result of the first nonlinear layer. The first lookup table records the mapping relationship between the ciphertext input and ciphertext output of the first nonlinear layer.

[0051] This specification employs a quantized model for inference. Compared to floating-point parameters in related technologies, it eliminates the need for expansion factors to guarantee inference accuracy. Homomorphic operations introduce less noise, resulting in lower requirements for the ciphertext modulus Q and polynomial order in dense-state inference. This makes the method presented in this specification more hardware-friendly and easier to use.

[0052] Furthermore, because dense-state inference is performed using a quantized model, the inputs and outputs of the nonlinear layers in the quantized model are discrete numerical values, unlike models with floating-point parameters. This makes it possible to perform nonlinear function calculations using lookup tables. Compared to nonlinear functions implemented through series expansion, nonlinear functions implemented using lookup tables have higher accuracy and improved ease of use.

[0053] The following section describes a dense-state inference method for a neural network model provided in this specification. This method accelerates hardware execution. In addition to improving the software-executed dense-state inference method, this specification also includes adaptive improvements to the acceleration hardware itself for software use; these hardware improvements will be explained later.

[0054] The acceleration hardware includes storage units and basic operational units for executing several basic homomorphic operators. The storage units contain the first lookup table required in later steps. The basic operational units may include automorphic units, (I)NTT units, and modular multiply-accumulate (MAC) units, etc. NTT refers to the Number Theoretic Transform (NTT), and INTT refers to the inverse of the NTT. Through the cooperation of these basic operational units, various homomorphic operations can be performed, such as homomorphic multiplication, homomorphic addition, and bootstrapping operations.

[0055] The neural network model discussed in this specification is a quantized neural network model, which includes a first linear layer and a subsequent first nonlinear layer. It is understood that a neural network model may include multiple linear and nonlinear layers. This specification describes the processing methods for the first linear layer and the first nonlinear layer following the first linear layer; the processing methods for other linear and nonlinear layers are the same and will not be repeated here.

[0056] Quantization refers to converting decimal parameters into integer parameters. Any quantization method can be used, and this specification does not impose any restrictions. It should be noted that the input value is also quantized to integer precision during the quantization process. Quantization aims to perform model (e.g., CNN) inference with integer precision while achieving negligible precision loss. Unlike relying on the expansion factor in CKKS to control decimal precision and precision loss due to noise growth, quantization is equivalent to having lower "integer" precision (typically 8 bits), as there are no decimals in the plaintext. Therefore, it is not necessary to set an appropriate expansion factor based on the parameter size; it is only necessary to ensure that the 8-bit integer remains accurate after dequantization and decryption, without maintaining approximately 30 bits of decimal precision. Based on the low precision requirements of the quantized model, parameters such as the "polynomial degree" and the ciphertext modulus Q can be set smaller without affecting precision.

[0057] Moreover, compared to models with floating-point parameters, quantized models can use a wider variety of encryption algorithms.

[0058] Furthermore, regarding the application scenarios of the method described in this specification, the method can be applied to the computer equipment of the model-owning party. The model-owning party is the party that owns the model. This method can also involve the data-owning party, who possesses user data. For example... Figure 1 As shown, the data provider sends the user data to the model provider after homomorphic encryption. The model provider uses the homomorphic ciphertext of the user data to perform inference and sends the homomorphic ciphertext of the inference result back to the data provider. The data provider decrypts the homomorphic ciphertext of the inference result to obtain the final inference result.

[0059] The steps included in the method provided in this manual will be described in detail below. Figure 2 As shown, this specification provides a dense-state inference method for a neural network model, which includes the following steps:

[0060] Step 201: Using the aforementioned basic computing units, calculate the first intermediate data after the first input data has been processed by the first linear layer.

[0061] The first input data is ciphertext data that has undergone fully homomorphic encryption.

[0062] The first input data is the input data for the first linear layer. The linear layer can include linear operation layers such as convolutional layers and fully connected layers. The operations of the linear layer include addition and multiplication operations between the quantized weight parameters and the input activation values ​​(i.e., the first input data). The first intermediate data is the output data of the first linear layer, and also the input data for the first non-linear layer.

[0063] For details regarding the basic arithmetic unit, please refer to the hardware method description below; it will not be elaborated upon here.

[0064] To facilitate understanding, the types of fully homomorphic encryption algorithms will be explained here. Fully homomorphic encryption generally has two implementation methods. One type is encryption algorithms based on the Learning With Errors (LWE) problem. These include algorithms such as GSW, FHEW, and TFHE. The other type involves high-throughput batch computation on integer or floating-point vectors, generally referred to as encryption algorithms based on Ring Learning With Errors (RLWE). These fully homomorphic encryption methods typically include CKKS (Cheon-Kim-Kim-Song), BGV (Brakerski-Gentry-Vaikuntanathan), and BFV (Brakerski-Fan-Vercauteren).

[0065] Unlike LWE-based schemes, RLWE-based schemes can encrypt multiple plaintexts into a single basic ciphertext unit (hereinafter referred to as a ciphertext or a single ciphertext). For example, in CKKS, a basic ciphertext unit consists of a pair of integer polynomials.

[0066] Since the RLWE scheme contains multiple plaintexts within a single ciphertext, it is convenient to perform parallel operations on these plaintexts simultaneously using methods such as Single Instruction Multiple Data (SIMD), thus accelerating execution. Therefore, related technologies generally use RLWE-like methods to perform matrix multiplication operations in linear layers.

[0067] In an alternative implementation, in step 201, linear operations can be performed using a method similar to that in related technologies. That is, multiple plaintext data are encoded using a slot-based encoding method, and linear layer operations are then performed on this basis.

[0068] Slot-based encoding is a specific encoding approach for RLWE-type methods. In this method, the vector of plaintext data to be encoded is first transformed into a polynomial function f(x) using a specific method (different RLWE-type methods choose different transformation methods). Then, multiple encoding points (the values ​​of x) are selected, and the values ​​of f(x) corresponding to these multiple encoding points are encrypted, thereby achieving encryption of the polynomial f(x).

[0069] For matrix multiplication or convolution operations in linear layers, if a common classical encoding scheme, i.e., a slot-based encoding scheme, is used, homomorphic operations are at the polynomial level, meaning only multiplication or addition of two polynomials can be performed. Correspondingly, only bitwise multiplication or addition of the data in each slot of the two ciphertexts can be performed. However, during inference, an accumulation operation is required, which necessitates multiple rotations (also known as cyclic shifts) of the ciphertext, followed by homomorphic operations on the rotated ciphertext to obtain the calculation result. The ciphertext rotation operation consumes significant computational resources, involving processes such as automorphism and key switching, which slows down the linear layer operations during encrypted inference.

[0070] In another alternative implementation, the first input data in this specification is fully homomorphic encrypted using coefficient-based encoding, and the first input data is an RLWE ciphertext.

[0071] Specifically, coefficient-based coding is a coding method in RLWE-type methods that contrasts with slot-based coding schemes. In this type of method, if the activation value (i.e., the first input data) is represented as: M = [m0, m1, ..., m N-1 The plaintext polynomial is then: Where Δ = Q / t, Q and t are the ciphertext modulus and plaintext modulus, respectively. Encryption is represented as... Where e < Δ / 2.

[0072] Furthermore, existing RLWE-type encoding schemes, such as BFV / BGV and CKKS, all support both coefficient-based and slot-based encoding.

[0073] In other words, this type of method uses multiple plaintext data points as coefficients of a plaintext polynomial, which is then encrypted. Specifically, the product of the plaintext data and the expansion factor (Δ) can be used as the polynomial coefficients. It should be noted that although the expansion factor is used for encryption here, it is only a necessary parameter for RLWE-type schemes, and its size does not affect the accuracy of the encrypted inference process. In contrast, related technologies require multiplying decimals by the expansion factor before rounding and encoding, thus the size of the expansion factor affects the accuracy of the encrypted inference.

[0074] In coefficient-based coding schemes, homomorphic operations are also based on polynomials. Multiplying two polynomials is equivalent to convolving the coefficients of those two polynomials. For example, f1(x) = a1 + b1x + c1x 2 f2(x) = a2 + b2x + c2x 2Then f1(x)·f2(x)=a1·a2+(b1·a2+a1·b2)x+(c1·a2+c2·a1+b2·b1)x 2 +(b1·c2+b2·c1)x 3 +c1·c2x 4 It should be noted that this example illustrates the operation method in plaintext. However, it is understandable that in homomorphic ciphertext based on coefficient encoding, one homomorphic multiplication is equivalent to multiplying two plaintext polynomials and then encrypting them. Therefore, one homomorphic multiplication is equivalent to including multiplication and accumulation operations, thus speeding up the reasoning.

[0075] Furthermore, when the ciphertext uses a coefficient-based encoding scheme, in order to ensure that the product of the two ciphertexts (i.e., the two polynomials) includes the required linear operation result, the weight parameters of the linear layer (hereinafter referred to as parameters) need to be rearranged.

[0076] Specifically, step 201 includes: rearranging the parameters of the first linear layer to obtain rearranged parameters; and using the first arithmetic unit, performing a single homomorphic multiplication operation on the first input data and the rearranged parameters to obtain the first intermediate data. The first arithmetic unit is one of several basic arithmetic units, and the first arithmetic unit is capable of performing homomorphic multiplication operations.

[0077] Next, we will combine Figure 3 This will explain the process described above. Figure 3 This illustrates the operational pattern where the linear layer is a convolutional layer, where M is the first input data, K is the convolution kernel, and R is the convolution result. M is encoded according to a coefficient-based encoding scheme as follows: Figure 3 The results shown show that the convolution kernel parameters have been rearranged as follows: Figure 3 The form shown is (rearrangement parameters).

[0078] Thus, the product of the polynomial represented by the rearranged parameter K and the polynomial corresponding to M can be obtained as follows: Figure 3 The resulting polynomial R is shown, where each bit of R represents a coefficient raised to the power of x. For example, the 0th bit of R represents the coefficient of x raised to the power of 0 in the resulting polynomial. For instance, the convolution at the position indicated by the red box M in the figure corresponds to r0 in R, i.e., the coefficient of x raised to the power of 0 in the polynomial. 4 The coefficient.

[0079] It should be noted that, Figure 3 This is just one example of a rearrangement method. K can be rearranged more efficiently, so that the values ​​in the resulting vector R are arranged more compactly in the ciphertext.

[0080] Furthermore, padding convolutions can also be implemented using the methods described above. Padding convolution refers to the process of adding extra values ​​(usually zero-value pixels) around the edges of the input data (such as feature maps) before performing the convolution operation. For example... Figure 4 As shown, Figure 4 The blue position in the image represents the original first input data, and the surrounding circle of zeros represents the added value of 0.

[0081] To achieve convolution with padding, during the encoding of the first input data, 0s can also be encoded in the ciphertext (i.e., the first input data). Figure 4 (As shown). Accordingly, change Figure 3 The rearrangement of the convolution kernel yields new rearrangement parameters (specific form as follows). Figure 4 (As shown). The product of the polynomial corresponding to the rearranged parameters and the polynomial corresponding to the first input data yields the convolution result with padding.

[0082] Convolutions with different strides can also be performed in the same way. The stride represents the horizontal / vertical distance moved when traversing the convolution range on the input feature map. Figure 3 The result shown is the convolution result with stride=1. In a specific implementation, the convolution result with stride=1 can be calculated. Since the convolution result with stride=1 includes the convolution results for other stride values, the corresponding values ​​can be extracted from the final result to obtain the convolution results for different strides.

[0083] for example Figure 4 The diagram illustrates the operation when stride=2. In the convolution result, light green and dark green areas together represent the convolution result when stride=1. The dark green areas represent the convolution result when stride=2. Therefore, it is possible to extract the convolution results for other stride values ​​from the convolution result when stride=1.

[0084] In some cases, the data size of the convolutional layer is large, and the first input data or the convolution result is difficult to represent using a single ciphertext. In such cases, a batching strategy can be employed. The first input data and the convolutional kernel are divided into multiple parts according to the channel dimension, and the operation result of each part is calculated separately. Then, by accumulating multiple batches of ciphertext, the final convolution result is obtained.

[0085] For example, Figure 5 As shown, the first input data (i.e., Input in the figure) can be arranged according to the dimension of the input channel (C). inDivide the data into b. i The convolutional kernel (i.e., the kernel in the diagram) is also divided into multiple parts according to the input channel dimension, and each part includes the output channel (C). out (Several sub-parts (each sub-part is a yellow block in the diagram). Then each b) i The ciphertext is multiplied by its corresponding convolution kernel portion, and the multiplication results are summed to obtain the final calculation result, output. Here, N in the diagram represents the polynomial order of the ciphertext.

[0086] Furthermore, the above examples illustrate the implementation process of convolution. The linear operation of a fully connected layer is similar to the above process, except that the parameters are rearranged differently.

[0087] As mentioned earlier, homomorphic operations in linear layers introduce noise into the ciphertext, thus requiring noise control. Noise control can employ methods from related technologies, such as bootstrapping. In an optional implementation, to accelerate computation, the first intermediate data can be multiplied by a preset reduction factor to obtain a new first intermediate data with reduced ciphertext modulus.

[0088] The above operation is equivalent to discarding the least significant bits of the first intermediate data, resulting in a new first intermediate data. Because the plaintext is scaled using a spreading factor during coefficient-based encoding, the effective values ​​of the plaintext are located in the most significant bits of the ciphertext. For example... Figure 6 As shown, the noise is located in the lower bits of the original first intermediate data, where 'e' represents the noise introduced by the homomorphic operation of the linear layer. Therefore, discarding the lower bits of the first intermediate data can remove the noise introduced by the linear operation and improve accuracy.

[0089] The aforementioned low-order truncation is achieved by multiplying the ciphertext data by a preset reduction factor. It's important to note that this multiplication is not a homomorphic operation, but rather a regular multiplication operation directly based on the ciphertext data. This process can also be called modulus switching, where the first data with the larger ciphertext modulus is replaced with the first intermediate data with a smaller modulus. Although this process changes the size of the ciphertext and the corresponding plaintext size, the changed ciphertext can be remapped back to its original size using the lookup table in step 203.

[0090] In one optional implementation, the reduction factor can specifically be t / Q, where Q is the ciphertext modulus and t is the plaintext modulus. For inference in quantized models, the inner product result often has greater precision than the parameters and activation values; therefore, an appropriate t needs to be set to prevent plaintext result overflow. Optionally, t = 2. 17 Here, the ciphertext modulus is switched from a larger modulus Q to a smaller modulus t by modulus switching.

[0091] Modulus-to-digital switching can be specifically represented as By switching between analog and digital, large noise in the Q / t range can be eliminated, and the noise increment space in the ciphertext can be refreshed.

[0092] Furthermore, since t / Q is a decimal, its direct multiplication with the ciphertext may introduce the following errors: rounding error e during the multiplication process. ms The error introduced by rounding is as follows: Figure 6 e ms As shown.

[0093] Although rounding may introduce errors, the errors introduced by rounding are smaller than the large noise in the lower bits that are discarded. Moreover, since t is large, the noise only contaminates the lower bits of the plaintext modulus. In addition, the quantization model will remap the inner product result to the original low precision (usually 8 bits of plaintext) in subsequent processes, which will further reduce the impact of the errors introduced by rounding.

[0094] Furthermore, when subsequent conversions between RLWE ciphertext and LWE ciphertext are required, reducing the modulus of the first intermediate data (RLWE ciphertext) will also reduce the time spent on subsequent RLWE ciphertext and LWE ciphertext conversions, as ciphertext conversion is relatively time-consuming.

[0095] It should also be noted that, as mentioned earlier, when the first input data uses a coefficient-based encoding method, the values ​​in the ciphertext of the linear layer's computation result are not arranged compactly (e.g., Figure 3 (As shown). Therefore, in order to obtain ciphertext that meets the requirements, it is also possible to: use several basic operation units to extract preset bits from the first intermediate data, and convert the extracted preset bits into LWE ciphertext; package the LWE ciphertext into the first intermediate data of the new RLWE ciphertext; the preset bits are determined according to the rearrangement of the first linear layer parameters; the plaintext data corresponding to the LWE ciphertext corresponds to continuous coefficients in the ciphertext polynomial represented by the first intermediate data of the new RLWE ciphertext.

[0096] Specifically, in the first intermediate data, the coefficients of multiple valid values ​​in the polynomial are not consecutive, which is detrimental to subsequent calculations. By converting the first intermediate data of RLWE ciphertext into the first intermediate data of LWE ciphertext, multiple plaintexts encoded together are extracted from an RLWE ciphertext and each plaintext is individually encoded into an LWE ciphertext. Then, the extracted multiple LWE ciphertexts are converted into RLWE ciphertext. This allows the first intermediate data to be encoded in the desired way, that is, multiple plaintexts serve as consecutive coefficients of multiple powers of a polynomial. For example, Figure 3 The significant bits of R are located in the coefficients of x to the powers of 4, 5, 7 and 8. Through the above transformation, the significant bits of R can be located in the coefficients of x to the powers of 0, 1, 2 and 3.

[0097] The extracted preset position is the valid value in the first intermediate result, which is determined according to the parameter rearrangement method of the first linear layer.

[0098] like Figure 6 As shown, firstly, preset bits are extracted from the first intermediate result. Each preset bit corresponds to an LWE ciphertext, and each LWE ciphertext contains the rounding error e. ms Then, the extracted LWE ciphertexts are repackaged into RLWE ciphertext. In the repackaged RLWE ciphertext, e represents the noise introduced during the ciphertext conversion process. ms The noise introduced by the rounding process in the aforementioned steps.

[0099] Step 203: Search the first lookup table based on the first intermediate data to obtain the corresponding ciphertext output as the output result of the first nonlinear layer.

[0100] The first lookup table records the mapping between the ciphertext input and the corresponding ciphertext output of the first nonlinear layer.

[0101] This process is also known as Functional Bootstrapping (FBS). The lookup table (LUT) records the correspondence between the ciphertext input and output of the nonlinear function. The lookup table achieves the mapping between ciphertext input and output. Furthermore, compared to the continuous input and output of the nonlinear function in the non-quantized model, the input and output of the nonlinear function in the quantized model are discrete points; therefore, the lookup table method can be used to handle the computation of the nonlinear layer.

[0102] Since the ciphertext output mapped by the lookup table is essentially the ciphertext data obtained through re-encryption, it is equivalent to completing the bootstrapping of the ciphertext through the lookup table, thus refreshing the noise space. Moreover, since the first intermediate data is RLWE ciphertext, which encodes multiple plaintexts, multiple data can be looked up in batches.

[0103] The lookup table can be pre-computed and encrypted based on the function of the nonlinear layer. To prevent the model from deducing user data from the output of the nonlinear function, the lookup table can be pre-computed by the data provider, encrypted with their own public key, and sent to the model provider. Since different data providers use different public keys, they will use different lookup tables.

[0104] Furthermore, for different nonlinear layers, if their nonlinear functions are different, their lookup tables will also be different.

[0105] In an alternative implementation, a lookup table can also be used to remap the first intermediate data, which will also be referred to as remapping below. Specifically, during the inference process of the quantization model, the computation result of the linear layer may exceed the quantization precision of the quantization model (e.g., 8 bits), so remapping is required to remap the output data of the nonlinear layer from the original plaintext modulus to the quantization precision (e.g., 8 bits).

[0106] In other words, the ciphertext output in the first lookup table is the ciphertext output remapped to the quantization range of the corresponding plaintext. The quantization range is also known as the quantization precision, and can be, for example, 8 bits. This process can also be described as... Where LUT represents the lookup table mentioned above, and scale is the parameter used in the quantization process.

[0107] In this way, the functionality of the nonlinear function and the remapping process are integrated into a lookup table, making it easier to complete the various calculations required in the inference process. Furthermore, during the remapping process, because the data is remapped to the quantization range, such as... Figure 6 As shown, the remapping result also reduces the noise e introduced by rounding mentioned earlier. ms .

[0108] In another alternative implementation, in addition to enabling remapping, nonlinear functions and / or bootstrapping, lookup tables can also remove the effects of analog-to-digital switching in the case of analog-to-digital switching mentioned above.

[0109] The implementation steps of the method provided in this manual can be as follows: Figure 7 As shown in the diagram, the linear layers of the quantized model (including fully connected layers and convolutional layers) are first processed using coefficient-based encoding, denoted as aBFV in the diagram. Here, Q in the lower right corner of the ciphertext m represents the ciphertext modulus.

[0110] Then, the second step is performed: modulus switching, converting the ciphertext with modulus Q to ciphertext with modulus t. Next, steps three and four are performed to convert between RLWE and LWE ciphertexts. The modulus of the ciphertext obtained in the fourth step can be restored to Q.

[0111] Finally, in step five, the activation function of the nonlinear layer is completed using a lookup table, and the ciphertext is remapped into the quantization range using the lookup table as well.

[0112] In response to the algorithm improvements mentioned above, this manual also describes corresponding improvements to the acceleration hardware used to implement homomorphic operations. The specific improvements to the acceleration hardware will be explained below.

[0113] As mentioned earlier, the acceleration hardware may include storage units and several basic arithmetic units. The storage units contain a first lookup table.

[0114] A basic operational unit refers to a unit used to execute several basic homomorphic operators. The basic operational units of typical accelerated hardware may include: automorphism units, discrete number theory transform (NTT) and its inverse transform (INTT) units (hereinafter referred to as (I)NTT units or NTT units), pseudo random number generators (PRNGs), MAC units, etc.

[0115] The functions of each basic arithmetic unit in this manual and its possible improvements will be explained next.

[0116] The automorphic unit is used to efficiently perform automorphic operations on ciphertext, and the (I)NTT unit is used to efficiently perform discrete number theory transformations or their inverse transformations. Both basic operational units are involved in various homomorphic operations on ciphertext.

[0117] For (I)NTT units, their architecture can be determined based on the predefined polynomial order of the ciphertext. As mentioned above, the method in this specification reduces the integer precision of the ciphertext, which correspondingly reduces the polynomial order of the ciphertext. Therefore, an NTT unit more suitable for the current scenario can be selected based on the polynomial order of the ciphertext.

[0118] For example, in the method described in this specification, the polynomial order of the ciphertext is only 2. 15 To improve resource utilization, a Radix-8 NTT unit design can be used, where each core of the NTT unit can complete 3 NTT iterations. For an order of 2... 15 The polynomial can be completed in just 5 iterations (log22). 15 / 3=5), maximizing the advantages of Radix-8 NTT. However, if Radix-4 NTT is used, each core of this architecture can complete two NTT iterations, which is beneficial for an order of 2. 15 For a polynomial of order 2, 8 iterations are required to complete the corresponding calculation, and one of these iterations results in resource waste. If Radix-16NTT is used, each NTT unit in this architecture can complete 4 NTT iterations. Therefore, for a polynomial of order 2... 15 Although the polynomial can be computed in just 4 iterations, there is still a waste of resources in one of the iterations.

[0119] In the accelerated hardware design, 256 NTT units can be deployed to process 2048 data points in parallel, and a fully pipelined approach is used to improve the throughput of the NTT core. Specifically, Radix-8 NTT reduces the overhead of analog-to-digital arithmetic units by incorporating iterative steps and enhancing throughput. For example, for polynomials of order N=2... 15 NTT requires only 5 iterations, with each iteration requiring 2048 modular arithmetic units, while the traditional Radix-2 method requires 3072 arithmetic units (1024×3).

[0120] For automorphic units, in the algorithm described in this specification, when using coefficient-based encoding, the linear layer operations no longer require rotation, reducing the automorphic operations required for rotation during inference. Therefore, the automorphic units in this specification mainly involve the overhead of index mapping, with low computational requirements. Based on the above requirements, automorphism can be implemented using the data read / write control method in Poseidon. Furthermore, when the polynomial order of the ciphertext is 2... 15 In this case, the parallelism of a single automorphic unit can be set to 256 to minimize computational latency. Accelerated hardware can deploy eight automorphic units to achieve a total parallelism of 2048.

[0121] The PRNG unit is used to reduce the storage and bandwidth requirements for key switching. Several steps in the algorithm provided in this specification may involve key switching. For example, during the process of obtaining the output of the nonlinear layer using a lookup table, key switching is required due to the ciphertext function interpolation operation (detailed below). Another example is that after modulus switching, the ciphertext modulus is changed, which correspondingly changes the ciphertext polynomial order, and this change in the ciphertext polynomial order is accompanied by a key switch.

[0122] The MAC unit is used to perform modular multiplication and modular addition operations. For example, it can multiply two ciphertexts (homomorphic multiplication) and then add them to another ciphertext (homomorphic addition). It can be used in linear layer operations and in retrieving the output of nonlinear layers using lookup tables.

[0123] In addition to the units mentioned above, based on the algorithm characteristics mentioned above, this specification may also include the following basic operational units: Sample Extract (SE) and Functional Reuse Unit (FRU).

[0124] Specifically, the sample extraction unit is used to extract preset bits from the RLWE ciphertext to complete the ciphertext format conversion, that is, the conversion between RLWE ciphertext and LWE ciphertext. As mentioned above, because the method in this specification uses a coefficient-based encoding method, and the weight parameters are rearranged to facilitate the operation of the linear layer, the data in the RLWE ciphertext in the first intermediate result output by the linear layer are not continuous. Therefore, ciphertext format conversion is required to obtain RLWE ciphertext with continuously encoded data. Thus, the acceleration hardware in this specification adds a sample extraction unit for ciphertext format conversion to complete the conversion between RLWE ciphertext and LWE ciphertext.

[0125] For the sample extraction unit, the computational requirements for the conversion between RLWE ciphertext and LWE ciphertext are relatively low, but it involves a large number of data position transformations. To extract LWE ciphertext (ct... i (LWE) requires cyclic shifting of the polynomial coefficients of the RLWE ciphertext (different from the ciphertext rotation mentioned earlier) and taking negative values ​​for some elements to obtain the corresponding LWE ciphertext. In related technologies, cyclic shifting is generally implemented using a barrel shifter. The barrel shifter can shift data by any number of bits, but the execution efficiency of each cyclic shift is relatively slow.

[0126] The cyclic shift described in this specification can be implemented using a register shifter. Specifically, the process of extracting a preset bit from the first intermediate data during the RLWE ciphertext and LWE ciphertext conversion described above can specifically include: for each bit of the first intermediate data, performing a one-bit cyclic shift on the first intermediate data according to the register shifter of the basic arithmetic unit, and extracting the first bit if the first bit after the cyclic shift is the preset bit.

[0127] Here, each bit of the first intermediate data refers to each bit of the polynomial coefficient corresponding to the first intermediate data.

[0128] Unlike bucket shifters, register shifters can only shift one position at a time. Register shifters are also more efficient. Such register shifters are better suited to the algorithms provided in this specification. The encoded plaintext positions in the first intermediate data are relatively compact, so a bucket shifter capable of rotating a large number of bits is unnecessary; a register shifter can achieve better results.

[0129] The hardware microarchitecture of the SE unit is as follows: Figure 8 As shown. Where a i These represent the polynomial coefficients of the ciphertext. Each polynomial coefficient shifts one position per period, such as... Figure 8As shown, the polynomial coefficients before and after one period shift are shown on either side of the red arrow on the left. The part below the letter 'q' refers to the subtractor, used to match the input data to the corresponding modulus. The MUX is used to select the positive or negative of the elements based on a flag signal. The flag signal can be determined based on the number of rotations and which bits are currently being processed. The data to the right of the red arrow on the right is the data obtained after positive or negative selection.

[0130] In the algorithm provided in this manual, the more time-consuming operation is the operation of obtaining the output result of the nonlinear layer using a lookup table.

[0131] Since this specification processes homomorphic ciphertext data, homomorphic ciphertext data can only undergo homomorphic multiplication and homomorphic addition operations, and cannot be directly looked up in a table like plaintext. The ciphertext input recorded in the first lookup table is the value of the corresponding ciphertext polynomial function under a preset input value. Therefore, the process of obtaining the output result (FBS) of the nonlinear layer using the lookup table in step 203 above specifically includes: using several basic operational units to perform function interpolation on the first intermediate data to obtain second intermediate data; and searching the first lookup table for the ciphertext output corresponding to the second intermediate data as the output result of the first nonlinear layer.

[0132] The process of function interpolation is to evaluate the values ​​of several specific x (i.e., the preset input values ​​mentioned above) after substituting them into the ciphertext polynomial f(x) represented by the first intermediate data. For example, if the ciphertext polynomial represented by the first intermediate data is f(x), and the specific x are 0, 1, 2, and 3, then the function interpolation process needs to calculate f(0), f(1), f(2), and f(3).

[0133] The ciphertext input recorded in the lookup table is also a specific x value under different ciphertext polynomials (each ciphertext polynomial represents an RLWE ciphertext). In the above process, the size of f(x) of the first intermediate data under multiple x values ​​is first evaluated by function interpolation to obtain the second intermediate data. Then, the second intermediate data is matched with the ciphertext input in the first lookup table to obtain the corresponding ciphertext output.

[0134] Since the aforementioned function interpolation process involves calculating multiple values ​​of f(x), this process involves numerous scalar multiplication (SMult) and homomorphic addition (HAdd) operations, which are essentially modular multiplication (MM) and modular addition (MA) operations, making it quite time-consuming. In related technologies, the bottleneck operations during computation using fully homomorphic encryption algorithms are NTT and Residue Number System (RNS) base conversion. The design focus of acceleration hardware in these technologies is on optimizing the NTT unit and using highly parallel RNS base conversion units, with limited hardware support for MA and MM.

[0135] To better suit the methods described in this specification, an FRU is used to implement MM, MA, and RNS basis conversion operations. The function multiplexing unit can be switched between modular addition, modular multiplication, and basis conversion operations via a selection signal.

[0136] Because the method described in this specification requires numerous MM, MA, and base conversion operations, implementing these operations separately using three different hardware chips would consume a significant amount of chip (acceleration hardware) area. Therefore, this specification integrates all three functions onto a single chip, achieving the same computational effect while reducing the chip size.

[0137] Regarding the specific implementation of the FRU unit, it can be as follows: Figure 9 As shown. For FBS, the FRU unit requires two input data to perform modular addition or modular multiplication. One input comes from an independent register, and the other input is selected by a selector for either FBS or RNS base conversion operation. For RNS base conversion, the broadcast data (brown line in the diagram) is selected as one input to the FRU module, and the other input is selected as the modular inverse (q) of the corresponding modulus. -1 Each FRU module contains 2048 MM and MA cores. Furthermore, MA and MM cores are cascaded, allowing the output of the MM core to be directly used as the input of the MA core, thus completing the MAC calculation.

[0138] Furthermore, to maximize execution efficiency, the basic computing units of the acceleration hardware are configured in two regions: a first region and a second region. The basic computing units in both regions execute in parallel. If all computing units were configured in one region, only one unit could be used at any given time. Configuring the basic computing units in two regions maximizes computational efficiency.

[0139] Since FBS is a relatively time-consuming operation in this specification, the second region may include multiple second operation units capable of performing modular multiplication and modular addition operations. In an optional embodiment, the second operation unit may be the FRU unit mentioned above. The first region may include at least a unit for performing modular multiplication and addition, such as a MAC unit. In addition, the first region may also include other basic operation units.

[0140] In an alternative implementation, such as Figure 10 As shown, the first region may include PRNG units, automorphic units, SE units, NTT units, FRU units, and MAC units. The second region may include an array of multiple FRU units, such as 16 FRU units. The two regions are interconnected via an inter-region buffer. Furthermore, each basic arithmetic unit has its own local cache. The storage unit mentioned earlier for storing the first lookup table can be a local cache of either the first or second region. Additionally, the acceleration hardware may include on-chip networks (NoC), high-bandwidth memory (HBM), and scratchpads (not shown in the figure), etc., which are not limited in this specification.

[0141] By dividing the data into two regions, the data flow between the regions can be designed to maximize computational efficiency.

[0142] The specific implementation of the function interpolation process can be as follows: segment the first intermediate data based on coefficient encoding, use multiple second operation units in the second region to calculate the polynomial segmentation result corresponding to each segment; use the MAC unit in the first region to obtain the second intermediate data based on the polynomial result corresponding to each segment.

[0143] The main computation in FBS is the evaluation of polynomials, that is, calculating the value of the ciphertext polynomial under a specific x. This process can be accomplished by multiplying the polynomial piecewise by different powers of x and summing the results.

[0144] For example, f(x) = a0 + a1x + a2x 2 +a3x 3 +a4x 4 +a5x 5 If the polynomial is piecewise divided into segments, each segment contains 3 elements, then f(x) can be divided into two segments: a0 + a1x + a2x. 2 and a3x 3 +a4x 4 +a5x 5 And a3x 3 +a4x 4 +a5x 5It can be viewed as a3 + a4x + a5x 2 With x 3 The product is obtained by multiplying.

[0145] Therefore, the process of calculating the value of f(x) under a specific x can include: the first step is polynomial piecewise computation, used to calculate a0 + a1x + a2x. 2 and a3+a4x+a5x 2 Poly mainly consists of SMult and HAdd operations. The second step is the multiplication of x powers, calculating x. 0 and x 3 And calculate a0 + a1x + a2x 2 With x 0 The result of the multiplication (denoted as A), and a3 + a4x + a5x 2 Calculate x 3 The result of multiplication (denoted as B). The third step is to sum them up, calculate the homomorphic sum of A and B, and obtain the value of f(x).

[0146] The pipeline of this method is as follows Figure 11 As shown, the first step, Poly, is implemented in parallel by multiple FRUs in the second region, while the second and third steps are implemented by the basic computation units in the first region. The second step is labeled CMult in the diagram, and the third step is HAdd. The second and third steps can be implemented by either MAC units or FRU units.

[0147] By distributing the Poly segmented computation and the CMult computation across two regions of computation units and setting a reasonable number of computation units, the latency of the two units can be balanced. For example, 2048 parallel modules in one FRU unit or MAC unit in the first region, and 16 FRU units in the second region, can achieve this latency balance. Figure 11 As shown in the pipeline diagram, each unit executes its processes continuously in the vertical time dimension, while multiple units can complete them in parallel in the horizontal dimension. Except for the first region not completing parallelism during the first Poly computation, a high degree of parallelism is achieved in the remaining time, realizing a fully pipelined design.

[0148] like Figure 12 As shown, Figure 12 A hardware structure diagram of a computer device is shown. The device may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are internally connected to each other via the bus 1050. This computer device can be used to perform the methods described above. The computer device may also include... Figure 10The acceleration hardware shown.

[0149] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification. The processor implements the above-described methods by running executable instructions.

[0150] The memory 1020 for storing processor-executable instructions can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1020.

[0151] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0152] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0153] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0154] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0155] This specification also provides a computer program product that, when executed by a processor, implements the dense-state inference method of the neural network model described above.

[0156] This specification also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the dense-state inference method of the neural network model described above.

[0157] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0158] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0159] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0160] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

Claims

1. A method for encrypted inference of a neural network model, achieved by accelerating hardware execution, wherein the neural network model is a quantized neural network, comprising a first linear layer and a subsequent first nonlinear layer; the accelerating hardware includes a storage unit and a plurality of basic operation units for executing a plurality of basic homomorphic operators, wherein a first lookup table in the storage unit records a mapping between the encrypted input and the corresponding encrypted output of the first nonlinear layer; the method includes: Using the aforementioned basic computing units, calculate the first intermediate data after the first input data has been processed by the first linear layer; The first input data is ciphertext data that has undergone fully homomorphic encryption. The first intermediate data is used to search the first lookup table to obtain the corresponding ciphertext output as the output result of the first nonlinear layer.

2. The method according to claim 1, wherein, The first input data uses coefficient-based encoding; the plurality of basic operational units include a first operational unit capable of performing homomorphic multiplication; the step of using the plurality of basic operational units to calculate the first intermediate data after the first input data has been processed by the first linear layer includes: The parameters of the first linear layer are rearranged to obtain rearranged parameters; Using the first processing unit, the first intermediate data is obtained by performing a single homomorphic multiplication operation on the first input data and the rearrangement parameters.

3. The method according to claim 2, wherein, The first lookup table records the encrypted input as the value of the corresponding encrypted polynomial function under a preset input value; the step of searching in the first lookup table based on the first intermediate data includes: By using several basic computing units, the first intermediate data is interpolated to obtain the second intermediate data; From the first lookup table, find the ciphertext output corresponding to the second intermediate data, and use it as the output result of the first nonlinear layer.

4. The method according to claim 3, wherein, The plurality of basic operation units are configured in a first region and a second region, and the basic operation units in the first region and the second region are executed in parallel; the first region includes at least a modular multiplication-addition (MAC) unit, and the second region includes a plurality of second operation units, which are capable of performing modular multiplication and modular addition operations; The step of using several basic computing units to perform function interpolation on the first intermediate data to obtain the second intermediate data includes: The first intermediate data based on coefficient encoding is segmented, and the polynomial segmentation result corresponding to each segment is calculated using multiple second operation units in the second region. Using the MAC unit of the first region, the second intermediate data is obtained based on the Poly results corresponding to each segment.

5. The method according to claim 4, wherein, The second arithmetic unit is a function multiplexing unit, which switches between modular addition, modular multiplication and base conversion operations via a selection signal.

6. The method according to claim 2, wherein, The first intermediate data is the ciphertext of the fault-tolerant learning problem RLWE on the ring; Before performing a lookup in the first lookup table based on the first intermediate data, the method further includes: Using several basic computing units, a preset bit is extracted from the first intermediate data, and the extracted preset bit is converted into fault-tolerant learning problem LWE ciphertext; The LWE ciphertext is packaged into the first intermediate data of the new RLWE ciphertext; the preset bit is determined according to the rearrangement of the first linear layer parameters; the plaintext data corresponding to the LWE ciphertext corresponds to continuous coefficients in the ciphertext polynomial represented by the first intermediate data of the new RLWE ciphertext.

7. The method according to claim 6, wherein, The acceleration hardware also includes a sample extraction unit, which is used to extract a preset bit from the RLWE ciphertext; the conversion between the RLWE ciphertext and the LWE ciphertext is achieved through the sample extraction unit.

8. The method according to claim 6, wherein, The step of extracting the preset bit from the first intermediate data includes: For each bit of the first intermediate data, the first intermediate data is cyclically shifted by one bit according to the register shifter of the basic arithmetic unit, and if the first bit after the cyclic shift is a preset bit, the first bit is extracted.

9. The method according to claim 1, further comprising, before performing a search in the first lookup table based on the first intermediate data: Also includes: Multiply the first intermediate data by a preset reduction factor to obtain a new first intermediate data after the ciphertext modulus is reduced.

10. The method according to claim 1, wherein, The ciphertext output in the first lookup table is the ciphertext output remapped to the quantization range of the corresponding plaintext.

11. The method according to claim 1, wherein, The basic operational unit includes a Discrete Number Theory Transform (NTT) unit, the architecture of which is determined based on the polynomial order of the preset ciphertext.

12. A computer program product that, when executed by a processor, implements the method as described in any one of claims 1-12.

13. A computer device, comprising: processor; Memory used to store processor-executable instructions; The processor implements the method as described in any one of claims 1-11 by executing the executable instructions.

Images