Homomorphic Evaluation of Tensor Programs
By generating and selecting optimized FHE adaptive tensor circuits and specifications, the problems of high cost and low efficiency of FHE in cloud computing are solved, efficient homomorphic computing and security are achieved, and FHE applications in cloud computing are optimized.
Patent Information
- Application Number
- CN201980056845.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2018-10-31
- Filing Date
- 2019-06-28
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2039-06-28
AI Technical Summary
The existing fully homomorphic encryption (FHE) technology is costly and inefficient when integrated in cloud computing. The process of selecting the optimal encryption mode is time-consuming and easy to introduce noise, resulting in low application efficiency and inability to efficiently evaluate encryption mode, limiting the application. operability and safety.
By generating and selecting optimized FHE adaptive tensor circuits and corresponding specifications, use computing devices to obtain input tensor circuits and modes, map to equivalent tensor circuits, perform symbolic execution, monitor data flow, track cost, and determine optimal Specification, reduce overhead costs, and choose efficient tensor circuit specifications.
It realizes efficient homomorphic computing of encrypted data, reduces computing resources and time overhead, improves security and performance, reduces noise, and optimizes the efficiency and security of FHE applications.
Smart Images

Figure CN112640357B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure generally relates to the field of data processing technology, and more particularly to systems, methods, and computer storage media for optimizing the generation, evaluation, and selection of tensor circuit specifications for tensor circuits to perform homomorphic encryption operations on encrypted data. Background Art
[0002] With the advent and mass adoption of cloud storage and computing, data security and privacy have become increasingly important for both cloud service providers and consumers. Various consumer groups, especially those with strict confidentiality and privacy standards, may be unable to use cloud services due to concerns about data breaches of sensitive data. Cloud service providers must develop functional solutions to reassure their consumers that their data is secure without causing inefficiencies in the development process. Fully homomorphic encryption (FHE) is a technology that enables computations to be performed directly on encrypted data, which is an attractive solution for cloud service providers. However, due to the high costs typically associated with the operational noise introduced when using conventional fully homomorphic encryption modes, the process of developing fully homomorphic encryption applications may be inefficient. The process of selecting the optimal fully homomorphic encryption mode for various operations can be extremely expensive in terms of both computing resources and the development process. Summary of the invention
[0003] Embodiments of the present invention relate to fully homomorphic encryption. More specifically, embodiments describe systems and techniques for optimizing the generation, evaluation, and selection of FHE adaptive tensor circuits and corresponding specifications (e.g., encryption parameters, data layout) to perform homomorphic encryption operations on encrypted data.
[0004] In various embodiments, an input tensor circuit and an associated pattern (such as inputs and weights of the circuit) are acquired by a computing device for evaluation. The acquired input tensor circuit is mapped to an equivalent tensor circuit (more specifically, a tensor circuit that can be used to perform a fully homomorphic encryption (FHE) operation). An equivalent tensor circuit is generated for the acquired input tensor circuit based on the acquired pattern associated with the acquired input tensor circuit. The computing device may also acquire encrypted data, as well as unencrypted weights and (multiple) public keys in a private key-public key pair to evaluate the acquired input tensor circuit. The generated equivalent tensor circuit (hereinafter referred to as a "FHE adaptive tensor circuit") with specific specifications (e.g., data layout, encryption parameters) can then be symbolically executed to perform multiple homomorphic calculations on the encrypted data. During the symbolic execution of the generated equivalent FHE adaptive tensor circuit, the data flow (e.g., library calls) through the generated equivalent FHE adaptive tensor circuit can be monitored by the computing device, so that the characteristics of the data flow of the generated equivalent FHE adaptive tensor circuit with a specific specification (e.g., the count of library calls) can be tracked and / or stored. A set of results can then be generated based on the symbolic execution of the generated equivalent FHE adaptive tensor circuit. The computing device can then generate another equivalent FHE adaptive tensor circuit with a different corresponding specification based on the generated set of results. The process of generating various specifications for the equivalent FHE adaptive tensor circuit and tracking the data flow through it can be repeated to enable the computing device to determine the cost (e.g., efficiency, noise, asymptotic complexity) of each generated equivalent FHE adaptive tensor circuit with a corresponding specification. The determined cost can be used to sort the various specifications and / or narrow the various specifications to a smaller subset of more efficient or most efficient tensor circuit specifications. At this point, the computing device may select a smaller subset of tensor circuit specifications based on their determined corresponding costs, thereby reducing overhead costs (e.g., energy, computational resources, time) associated with conventional techniques for evaluating tensor programs on homomorphically encrypted data.
[0005] This Summary is provided to introduce some concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS
[0006] The present invention is described in detail below with reference to the accompanying drawings, in which:
[0007] Figure 1 is a schematic diagram of an exemplary system depicted according to one embodiment of the present invention;
[0008] Figure 2is a data flow diagram depicting inputs and outputs of an exemplary compiler component according to some embodiments of the present invention;
[0009] Figure 3 is a data flow diagram depicting inputs and outputs of exemplary runtime components according to some embodiments of the present invention;
[0010] Figure 4 is a process flow diagram depicting a process flow for transforming a tensor circuit that may be used to perform a fully homomorphic encryption operation according to some embodiments of the present invention;
[0011] Figure 5 is a block diagram depicting an exemplary computing device with exemplary compiler and runtime components according to some embodiments of the present invention;
[0012] Figure 6 is a flow chart depicting a method for generating an optimized FHE adaptive tensor circuit according to some embodiments of the present invention;
[0013] Figure 7 is a block diagram of an exemplary computing environment suitable for implementing some embodiments of the invention. DETAILED DESCRIPTION
[0014] The subject matter of the present invention is specifically described herein to meet statutory requirements. However, the description itself is not intended to limit the scope of this patent. On the contrary, the inventors have anticipated that the claimed subject matter may also be embodied in other ways in conjunction with other current or future technologies to include steps or combinations of steps different from those described in this document. In addition, although the terms "step" and / or "box" may be used herein to refer to different elements of the method employed, unless the order of the various steps is explicitly described, these terms should not be interpreted as implying any particular order among or between the various steps disclosed herein.
[0015] Many computing applications benefit from the ability to store and process data in the cloud (i.e., on a remote computing system accessible over a network). Cloud providers centralize and manage computing and storage resources, which allows developers to focus on the logic of their applications, thereby providing developers with a way to avoid deploying and managing infrastructure around applications. However, a large class of applications has strict data confidentiality and privacy requirements. In such cases, data confidentiality and privacy issues ultimately limit the adoption of otherwise convenient cloud-based solutions.
[0016] Fully homomorphic encryption (FHE) provides a simple trust model: the owner of the data does not need to trust the cloud software provider or hardware vendor. FHE allows computations to be applied directly to encrypted data, thus making a wider variety of cloud applications available. A cloud-based FHE application can encrypt its plaintext input "a" and store it in encrypted form (e.g., in cloud storage) as "α". Later, the application can request the cloud to perform an FHE operation "f" on the encrypted data "α" and store the still encrypted result "β=f(α)". The application can then download "β" from the cloud storage at any time and decrypt the result to obtain the plaintext (i.e., decrypted) result "b=f(a)".
[0017] Although various aspects of FHE can provide breakthroughs in cloud computing technology, there are many costs associated with the integration of FHE in cloud computing. For example, FHE operations on encrypted data can be significantly slower than those performed on plaintext copies. Despite these setbacks, some entities are willing to address these inefficiencies in exchange for more secure cloud-based solutions. In order to build efficient FHE applications, conventional techniques require manual determination of FHE encryption modes, which introduces a significant factor of human error. The process of identifying the optimal encryption mode can be very time-consuming, and adopting a specific encryption mode can introduce undesirable noise, rendering the FHE application useless or extremely inefficient. Naturally, the development of FHE applications will waste a lot of time, effort, and computing resources because conventional techniques cannot provide a means to evaluate FHE encryption modes in an efficient manner and with high accuracy. It is also noted that some FHE modes support vectorized operations, also known as "batching", where the vector size is very large. At this point, application efficiency depends crucially on maximizing the parallelism available in the application to exploit vector capabilities, and the ability to maximize the availability of parallelism can be a difficult task. Finally, FHE modes may restrict the operations allowed in applications (i.e., no branching or control flow), thus requiring developers to creatively structure their programs to work around these limitations. In summary, it is clear that there is a need to provide efficient techniques to ensure optimal performance, security, and noise management when performing operations on encrypted data.
[0018] The embodiments described herein generally relate to applications that apply deep neural network inference calculations to the FHE space. There are many benefits to using deep neural network inference calculations in the FHE space. For example, deep neural network inference calculations do not involve discrete control flow, which is an ideal choice in FHE applications. In addition, deep neural network inference involves non-polynomial activation functions, which can be effectively replaced by polynomial approximations. The fact that deep neural networks can tolerate errors provides a certain degree of flexibility in quantization and reducing the computational precision for performance evaluation, adjustment, and optimization. In this way, the embodiments described herein more specifically relate to optimizing the generation, evaluation, and selection of FHE adaptive tensor circuits and corresponding tensor circuit specifications (e.g., parameters, data layout) to perform homomorphic encryption operations on encrypted data. More particularly, the embodiments described herein can automatically generate and select optimal (i.e., cost-effective) encryption parameters and data layouts to configure tensor circuits (hereinafter referred to as "FHE adaptive" tensor circuits) with maximized performance, accuracy, and improved security when used to perform deep neural network inference calculations.
[0019] Now turn to Figure 1 , a schematic diagram showing an exemplary homomorphic assessment system 100 in which some embodiments of the present invention may be adopted is provided. It should be understood that this arrangement and other arrangements described herein are only set forth as examples. Other arrangements and elements (e.g., machines, interfaces, functions, commands, groupings of functions, etc.) may be used as supplements or substitutes for the arrangements and elements shown, and some elements may be omitted completely. In addition, many elements described herein are functional entities that may be implemented as discrete or distributed components or implemented in combination with other components and implemented in any suitable combination and position. The various functions described herein as being performed by one or more entities may be performed by hardware, firmware, and / or software. For example, various functions may be performed by a processor executing instructions stored in a memory.
[0020] Figure 1The homomorphic evaluation system 100 shown includes a computing device, such as a homomorphic evaluation server 110, having one or more processors and a memory. In various embodiments, the homomorphic evaluation server 110 can obtain input tensor circuits and associated patterns for evaluation. The homomorphic evaluation server 110 can obtain the input tensor circuits and associated patterns by retrieving the input tensor circuits and associated patterns from a coupled storage device such as a storage device 120 or receiving the input tensor circuits and associated patterns from a client device such as a client device 140 via a network such as a network 130. The storage device 120 may include a database or any other storage medium or another computing device associated with a storage medium for storing data generated or retrievable by the homomorphic evaluation server 110. The network 130 may include any type of networking medium. Such as the Internet, a LAN, a WAN, a telecommunications network, a peer-to-peer network, near field technology, RF, etc. In addition, the communication transmitted through the network 130 may be wired or wireless or any combination thereof. In various embodiments, any combination or format of results generated by homomorphic evaluation server 110 based on evaluation of various specifications of input tensor circuits may be stored in storage device 120, provided for display by homomorphic evaluation server 110, or transmitted from homomorphic evaluation server 110 to client device 140 for display.
[0021] In some embodiments, the input tensor circuit and the associated pattern, such as the input and weight of the input tensor circuit, are received from the client device 140 or obtained from the storage device 120 by the homomorphic evaluation server 110 for evaluation. The obtained input tensor circuit can be mapped to an equivalent FHE adaptive tensor circuit generated for the input tensor circuit based on the pattern associated with the obtained input tensor circuit. The homomorphic evaluation server 110 can also receive from the client device 140 or obtain from the storage device 120 encrypted data (e.g., encrypted images, files, or other digital assets), as well as unencrypted weights and (multiple) public keys in the private key public key pair for evaluating the input tensor circuit. The homomorphic evaluation server 110 can then symbolically execute the generated equivalent FHE adaptive tensor circuit to perform multiple homomorphic calculations on the encrypted data. During the symbolic execution of the generated equivalent FHE adaptive tensor circuit, the homomorphic evaluation server 110 can monitor the data flow (e.g., operations, calls to libraries) generated or processed by the generated equivalent FHE adaptive tensor circuit. The monitored data flow can be used by the homomorphic evaluation server 110 to determine the efficiency cost associated with the current specification (e.g., encryption parameters, data layout) of the generated equivalent FHE adaptive tensor circuit. The homomorphic evaluation server 110 can generate a set of results based on the symbolic execution of the equivalent FHE adaptive tensor circuit generated under the current specification, so that the generated set of results can be stored in the storage device 120 or transmitted to the client device 140. Then, the homomorphic evaluation server 110 can specify new parameters based on the generated set of results to generate another equivalent FHE adaptive tensor circuit with a different corresponding specification. This process may be referred to as "transforming" the equivalent FHE adaptive tensor circuit in this article, which includes determining a set of parameters for the input tensor circuit and directly mapping the input tensor circuit to an equivalent FHE adaptive tensor circuit with unspecified parameters. The process of generating (i.e., transforming) the FHE adaptive tensor circuit and tracking the data flow through it can be repeated so that the homomorphic evaluation server 110 can determine the cost (e.g., efficiency, noise, asymptotic complexity) of each generated equivalent FHE adaptive tensor circuit with a corresponding specification. The determined costs may be used to rank and / or narrow down the various specifications to a smaller subset of more efficient or most efficient (e.g., fast, accurate, secure) tensor circuit specifications. In this regard, the homomorphic evaluation server 110 may select a smaller subset of tensor circuit specifications for equivalent FHE adaptive tensor circuits based on their corresponding costs, thereby reducing overhead costs (e.g., energy, computational resources, time) associated with conventional techniques for evaluating tensor programs on homomorphically encrypted data.
[0022] At a high level, the homomorphic evaluation server 110 may include a compiler and runtime components (also referred to herein as "CHET") that are particularly suitable for performing homomorphic evaluation of tensor programs or "circuits". In various embodiments, the homomorphic evaluation server 110 may employ encryption modes that promote approximate arithmetic to emulate fixed-point arithmetic, which is often required to perform deep neural network inference computations. More information about exemplary encryption modes (also referred to herein as "CKKS" encryption modes) that the homomorphic evaluation server 110 may employ can be found in "Homomorphic encryption for arithmetic of approximate numbers" by Jung Hee Cheon, Andrey Kim, Miran Kim, and Yongsoo Song, "Cryptology-ASIACRYPT 2017, LNCS10624, pages 409-437", which also describes an accessible encryption mode library related to homomorphic encryption for approximate digital arithmetic (referred to herein as "HEAAN" encryption modes). By employing compiler and runtime components, the homomorphic evaluation server 110 can explore various optimization options, such as various ways of packing data into ciphertext, and various ways of implementing computational kernels for each layout of a given tensor circuit. Similar to linear algebra libraries commonly used for unencrypted evaluation of neural networks, the homomorphic evaluation server 110 runtime component can include a set of layouts and a unified metadata representation for the layouts. In this regard, each layout can be associated with a corresponding computational kernel that implements common operations found in convolutional neural networks ("CNNs").
[0023] Watch now Figure 2 , a data flow diagram 200 is provided, which depicts a data flow diagram such as Figure 1 The data flow diagram 200 illustrates an exemplary compiler component 210 (“CHET compiler”) that can be used to perform evaluation of tensor programs (i.e., circuits) on homomorphically encrypted data. At a high level, the compiler component 210 is responsible for generating optimized FHE adaptive tensor circuits that not only produce accurate results, but also greatly improve the security of the data being computed. As shown in FIG. Figure 1 Briefly described, homomorphic evaluation server 110 may obtain tensor circuit 220 and schema 230 corresponding to the inputs and weights of tensor circuit 220. Schema 230 specifies the dimensions of the tensors and the required floating point precision of the values in these tensors. Homomorphic evaluation server 110 may also obtain the desired floating point precision 240 of the output of the circuit, which may be obtained from, for example, Figure 1The compiler component 210 may be received by a client device such as the client device 140 of the embodiment of the present invention, or retrieved from the memory or storage device 120. Using the mode and the desired floating point precision, the compiler component 210 may generate an equivalent optimized FHE adaptive tensor circuit 250, as well as an encryptor and decryptor 260 executable file, which may be used by the server device 110 and / or the client device 140 to generate a private key public key pair, encrypt data, and / or decrypt data, etc. Both executable files may be used to encode the choices made by the compiler component 210 to make homomorphic computation more secure and efficient.
[0024] Now cross reference Figure 1 Come and see Figure 3 , a data flow diagram 300 is provided, which depicts a data flow diagram such as Figure 1 The input and output of the homomorphic evaluation system 100 and the like. In various embodiments, the homomorphic evaluation system 100 can be used to evaluate input tensor circuits on data (e.g., images, documents, digital assets). As an example, and at a higher level, as a non-limiting example, the client device 140 can employ (multiple) private keys 320 to encrypt data such as an image 310. The (multiple) private keys 320 used to encrypt the image 310 can be obtained from an encryptor, which can be generated and / or provided to the client by the homomorphic evaluation server 110. The encrypted image 330, as well as the unencrypted weights 230 and public keys 340 (i.e., the (multiple) public keys associated with the (multiple) private keys) required to evaluate homomorphic operations on the encrypted data can be transmitted to the homomorphic evaluation server 110. The homomorphic evaluation server 110 may receive the encrypted image 330, the unencrypted weights 230, and the public key 340, which may be stored in a memory of the homomorphic evaluation server 110 or in a storage device 120 coupled to the homomorphic evaluation server 110. The homomorphic evaluation server 110 may employ a method such as Figure 2A compiler component such as the compiler component 210 of the embodiment of the present invention generates an optimized FHE adaptive tensor circuit 250 based on the received encrypted image 330, the unencrypted weights 230, and the public key 340. The homomorphic evaluation server 110 may also employ a runtime component 345 (“CHET runtime”) to symbolically perform homomorphic tensor operations with the generated optimized FHE adaptive tensor circuit 240. In various embodiments, the base FHE mode 350 may be employed to symbolically perform homomorphic computations on the encrypted image 330 and generate an encrypted prediction 360, which may then be transmitted to the client device 140 via the network 130. The client device 140 may receive the encrypted prediction 360 and decrypt the encrypted prediction 360 using its (multiple) private keys 320 using the decryptor 260, which may be generated and / or provided to the client device 140 by the homomorphic evaluation server 110. In this manner, client device 140 may run a tensor program similar to a neural network on homomorphic evaluation server 110 without homomorphic evaluation server 110 having access to the data (e.g., unencrypted image 310), the output (i.e., prediction 370), or any intermediate states.
[0025] Reference now Figure 4 , a process flow diagram 400 is provided, which depicts a method for (e.g., via Figure 2 The compiler component 210 of FIG. 2 transforms the processing flow of the FHE adaptive tensor circuit. As briefly noted, Figure 2 The compiler component 210 can generate an optimized FHE adaptive tensor circuit that not only produces accurate results but also greatly improves data security and computational performance. The compiler component 210 can efficiently analyze and transform various FHE adaptive tensor circuit specifications to generate new optimized FHE adaptive tensor circuits. Compared to most traditional optimizing compilers, the input tensor circuit has two key properties that the compiler component 210 can exploit: the data flow graph is a directed acyclic graph (DAG), and the tensor size is known at compile time based on a provided pattern associated with the input tensor circuit. The compiler component 210 can also exhaustively analyze multiple data flow graphs corresponding to different specifications one at a time, which is certainly feasible due to the limited design space.
[0026] In some embodiments, the compiler component 210 only needs to determine a strategy for performing homomorphic computations, while Figure 3The runtime component 345 of the same evaluation server 110 can handle the efficient execution mechanism. In various embodiments, the homomorphic evaluation server 110 may include a homomorphic instruction set architecture (also referred to herein as "HISA") that decouples compiler optimizations and runtimes used to support FHE applications from progress in the underlying FHE mode. The compiler component 210 can generate high-level homomorphic tensor operations, but requires analysis of the low-level HISA instructions executed. In this way, the homomorphic evaluation server 110 can utilize the runtime component 345 to directly perform such analysis.
[0027] Figure 4 The process flow diagram 400 of FIG. 4 depicts the process flow of a single analysis and transformation process in the compiler component 210. The compiler component 210 may include a transformer component 410 that specifies encryption parameters to be used with an input tensor circuit. In various embodiments, the input tensor circuit may be directly mapped to an equivalent FHE adaptive tensor circuit with unspecified parameters, thereby producing or generating a "transformed" FHE adaptive tensor circuit 420. The transformed FHE adaptive tensor circuit 420 may be generated via a processor such as Figure 3 The symbolic execution of the transformed FHE adaptive tensor circuit 420 includes the execution of the same instructions (e.g., HISA instructions) that would be executed at runtime because the tensor size is known. Instead of executing these instructions using FHE mode, these instructions call a symbolic analyzer 440, which can monitor data (e.g., library calls, operations) flowing through the transformed FHE adaptive tensor circuit 420. In some embodiments, the transformed FHE adaptive tensor circuit 420 is a directed acyclic graph ("DAG"), and therefore, iteration to a fixed point is not required. The directed acyclic graph can be dynamically expanded to dynamically track the flow of data flowing through the transformed FHE adaptive tensor circuit 420 without having to construct an explicit directed acyclic graph. In various embodiments, the symbolic analyzer 440 can employ multiple analyzer operations to track different information flows through the transformed FHE adaptive tensor circuit 420. In some other embodiments, the symbolic analyzer 440 may return one or more results obtained from the trace data flow as output of the symbolic execution. The transformer component 410 may then utilize the result(s) to generate a new transformed specification of the FHE adaptive tensor circuit.
[0028] Now turn to Figure 5 , a block diagram 500 is provided, which depicts an example system such as a processor having an exemplary compiler 510 and a runtime 560 according to some embodiments of the present invention. Figure 1As described, the homomorphic evaluation server 110 may include a compiler component 510 (e.g., Figure 2 The compiler component 210 ("CHET compiler") and the runtime component 560 (e.g., Figure 3 A runtime component 345 (“CHET runtime”) of CHET is used to perform homomorphic evaluation of tensor circuits on homomorphically encrypted data.
[0029] The homomorphic evaluation server 110 may also include an FHE library repository 570, which may include a homomorphic instruction set architecture ("HISA") that cleanly abstracts and exposes various features of the FHE library. In various embodiments, the HISA may be used by the compiler component 510 to interact with the FHE mode employed by the homomorphic evaluation server 110 to facilitate the various features described herein. More specifically, the HISA of the FHE library repository 570 is designed to abstract the details of the homomorphic encryption mode, such as the use of evaluation keys and the management of moduli, to provide a common interface for shared functionality in the FHE library, while exposing unique features that, rather than including the complexity associated with plain text evaluation, are embedded in the host language.
[0030] HISA may be divided into multiple profiles such that all libraries are expected to implement at least the cryptographic profile. For example, each FHE implementing HISA provides two types: "pt" for plaintext and "ct" for ciphertext, and provides other types depending on the various profiles implemented by the FHE library repository 570. Among other things, the cryptographic profile may provide for encryption, decryption, and memory management functions. The use of encryption and evaluation keys may be the responsibility of the FHE library so that they are not disclosed in HISA.
[0031] In various embodiments, the HISA may include integer profiles that may provide operations for encoding and computing integers. The integer profiles may cover both FHE modes with batch encoding and FHE modes without batch encoding, as they have a configurable number of slots "s" that may be provided as an additional parameter during FHE library initialization.
[0032] In various embodiments, as a non-limiting example, HISA may also include a partition profile that may expose additional functionality provided by a family of encryption modes, such as the HEAAN family of encryption modes. For libraries implementing the basic CKKS encryption mode, the function "maxScalarDiv( c,ub )" can return the largest power of 2 d such that d ≤ ub and d ≤ q, where q is the modulus of c. For the variant of CKKS based on the residue number system, "maxScalarDiv( c,ub)" can return the largest coprime modulus of c that is less than ub, or "1" if none. In this way, the division profile can clearly disclose the division functionality of the two variants of the CKKS encryption mode.
[0033] The HISA may also include a Relin profile that may provide the ability to separate multiplication from relinearization. Although relinearization is semantically a no-op, exposing linearization calls to the compiler component 510 may be useful since their correct placement is a highly nontrivial (e.g., NP-complete) problem.
[0034] HISA may also include a bootstrap profile that may expose bootstrapping in FHE libraries that support it. Although bootstrapping is semantically a no-op, it should be exposed in HISA because the choice of cryptographic parameters depends on when bootstrapping is performed. Furthermore, the optimal location for bootstrapping operations may depend on the tensor program ("circuit") being evaluated. For example, a "wide" program with a large number of concurrent operations may be bootstrapped at a lower depth than a "narrow" program that is mostly sequential.
[0035] In various embodiments, HISA can be implemented in accordance with exact or approximate semantics. Using approximate semantics, all operations that return "pt" or "ct" may introduce error terms. It can be expected that the error comes from a random distribution that may be limited at a given confidence level. However, HISA does not provide a description of the estimated error because these estimates may be required when selecting fixed-point precision and encryption parameters. In contrast, in some embodiments, the approximate FHE library can provide an implementation of HISA without actual encryption, and can be a method of querying a secure estimate of the error accumulated in each "pt" and "ct", or a sampling of errors from the same distribution that can produce encrypted evaluations. The compiler component 510 can use two methods to select fixed-point precision. For applications where it is difficult to quantify the acceptable error range (such as neural network classification where only the order of results is important), the sampling method can be more flexible.
[0036] Initialization of an FHE library is typically specific to the encryption and encoding modes used. Nevertheless, initialization may include at least the generation or import of encryption and evaluation keys. FHE libraries implementing hierarchical encryption modes may also need to select encryption parameters that are large enough for the computation to succeed. Such FHE libraries should provide analysis procedures as other implementations of HISA to help with parameter selection, etc.
[0037] As background technology, it is well known that Intel's Math Kernel Library (MKL) can provide efficient implementations of Basic Linear Algebra Subroutines (BLAS) operations. In a similar manner, the runtime component 510 can be designed to provide efficient implementations for homomorphic tensor primitives. Although the interfaces for BLAS and tensor operations are well defined, the interfaces for tensor operations on encrypted data are not necessarily obvious, because encrypted data can be encryption of vectors, while tensor operations are located on high-dimensional tensors. At this point, types need to be coordinated to define clean interfaces. For illustrative purposes, 4-dimensional tensors (batch, channel, height and width dimensions) are considered in various discussions to provide exemplary implementations, but it can be considered that the concepts can be similarly applied to other high-dimensional tensors, and sensors of any size are included within the scope of this disclosure.
[0038] Continuing, in various embodiments, a cryptographic tensor data type may be defined by the runtime component 510. Each tensor operation on an unencrypted tensor may correspond to an equivalent homomorphic tensor operation on (multiple) cryptographic tensors and (multiple) ordinary tensors (if any). In various embodiments, the cryptographic tensor data type needs to allow padding or logical reorganization without changing elements of the tensor circuit. To meet these requirements, the runtime component 510 may define a cryptographic tensor data type, referred to herein as a "CipherTensor," which includes a ciphertext vector (i.e., a vector) and associated metadata that captures the manner in which the ciphertext vector is interpreted as a corresponding unencrypted tensor. The metadata may include the physical dimensions of the (external) vector and the (internal) ciphertext, the logical dimensions of the equivalent unencrypted tensor, and the physical stride of each dimension of the (internal) ciphertext.
[0039] The runtime component 510 may store the metadata as a pure integer in a memory or storage device so that it can be easily modified. However, the metadata does not leak any information about the data because it is independent of the value of the tensor and depends only on the architecture or size of the tensor. The runtime component 510 can use the metadata to satisfy all required constraints on the data type. The metadata may include parameters that the compiler component 510 may choose to instantiate and adopt a specific data layout. For example, a process referred to as "HW tiling" herein may include blocking or tiling the inner dimensions (e.g., height and width) of a 4-d tensor, which may correspond to selecting two dimensions of the (external) vector and two dimensions of the (internal) ciphertext, where the logical dimensions match the physical dimensions. Another option, referred to as "CHW tiling" herein, is to also block the channel dimensions so that multiple but not all channels can be in the cipher. This may correspond to selecting two dimensions for the (external) vector, three dimensions for the (internal) ciphertext, and only four logical dimensions.
[0040] In various embodiments, the physical stride of each dimension may be specified to include sufficient padding between elements of that dimension. For example, for an image with a height (rows) and width (columns) of 28, a stride of "1" for the width dimension and a stride of "30" for the height dimension allows padding of two (e.g., zero or invalid) elements between rows. Thus, reshaping a cryptographic tensor may only include updating metadata to change the logical dimensions, without performing any homomorphic operations.
[0041] The interface of homomorphic tensor operations is mostly similar to that of non-encrypted tensor operations, except that its type is replaced by a cryptographic tensor or a plain tensor, respectively referred to herein as a "cryptographic tensor" and a "plain tensor". Homomorphic tensor operations may also expose the output CipherTensor as a pass-by-reference parameter in the interface so that the compiler component 510 can specify the data layout of the input and output CipherTensors. In addition, the interface may expose parameters to specify the scaling factors for the (multiple) input CipherTensors and (multiple) PlainTensors (if any). When the compiler component 510 specifies all of these parameters, it may select an implementation dedicated to the specification. For a specific FHE library (e.g., for a library with divScalar and a library without divScalar), multiple implementations may be regulated for the same specification. In this case, the compiler component 510 may also select the implementation to use. However, several algorithm choices may be provided for a given specification, and these choices have a significant impact on the performance of the operation.
[0042] In various embodiments, the compiler component 510 may generally be responsible for generating optimized FHE adaptive tensor circuits that not only produce accurate results, but also greatly improve the security of the computed data. An analysis and transformation framework used by the compiler component 510 may be provided to perform transformations of the tensor circuits required for accuracy and security, and also to perform optimizations that significantly improve the performance of the tensor circuits.
[0043] As noted, compared to most conventional optimizing compilers, the input tensor circuit includes two key properties that the compiler component 510 can exploit: the data flow graph is a directed acyclic graph ("DAG"), and the tensor size is known at compile time according to a pattern provided by the user. The compiler component 510 can analyze multiple data flow graphs corresponding to different implementations. Since the design space is not so large, the compiler component 510 can explore the design space exhaustively one at a time. In some embodiments, the compiler component 510 can determine the strategy for performing homomorphic computations, while the runtime component 560 can handle the efficient execution mechanism. This separation of concerns can simplify the code generation task of the compiler component 510, but in some cases can complicate the required analysis. Therefore, the compiler component 510 can generate high-level homomorphic tensor operations, but will need to analyze the low-level HISA instructions being executed. To solve this problem, the runtime component 560 can be directly used to perform the analysis.
[0044] In various embodiments, compiler component 510 may include transformer component 520, which may define parameters used in a FHE adaptive tensor circuit (e.g., directly mapping an input tensor circuit to an equivalent FHE adaptive tensor circuit with unspecified parameters). Such a transformed FHE adaptive tensor circuit may be symbolically executed using runtime component 560. This will execute the same HISA instructions that would be executed at runtime, since the tensor dimensions are known. Instead of executing these HISA instructions using FHE mode, the HISA instructions may call symbolic analyzer 530.
[0045] The symbolic analyzer 530 may include, among other things, a data tracking component 532, a parameter selection component 534, a padding selection component 536, a rotation key selection component 538, a cost determination component 540, and a result generation component 542. The data tracking component 532 may monitor the data flowing through a given tensor circuit and record various characteristics of the tensor circuit, such as the number of library calls, the number of operations being performed, etc. In various embodiments, since the tensor circuit is a directed acyclic graph, it is not necessary to repeatedly iterate to a fixed point. The directed acyclic graph is expanded on the fly by the symbolic analyzer 530 to dynamically track the data flow via the data tracking component 532 without having to construct an explicit directed acyclic graph. In various embodiments, the data tracking component 532 may include one or more subcomponents or analyzers that may track different information flows. The symbolic analyzer 530 may return the results generated based on the symbolic execution of the tensor circuit as the output of the symbolic execution. The transformer component 520 may then utilize the results generated by the symbolic analyzer 530 to instantiate a new transformed specification of the FHE adaptive tensor circuit.
[0046] The compiler component 510 can use the parameter selection component 534 to select encryption parameters that provide accurate output data when executing the tensor circuit. To this end, the parameter selection component 534 can analyze the depth of the tensor circuit. As an example only, in the HEAAN encryption mode family, divScalar consumes the modulus Q from the ciphertext that calls it. The output should leave enough modulus Q to capture accurate results. The symbol analyzer 530 can therefore be designed to have a pseudo-ciphertext data type that increases the modulus Q of the input ciphertext and copies it to the output ciphertext whenever divScalar is called. This can be distinguished from other HISA instructions, which only copy the modulus from the input to the output. Here, the modulus Q in the pseudo-output ciphertext corresponds to the depth of the circuit. The input ciphertext can then be encrypted using a modulus Q that is at least the sum of the circuit depth and the desired output precision to ensure that the result is accurate. For the input modulus Q, a sufficiently large N can be selected to improve or guarantee the security of the data in some cases. In this way, this can be a deterministic mapping from Q to N. In some cases, Q may require a very large N, in which case the compiler component 510 may introduce a bootstrap between the two.
[0047] The padding selection component 536 does not utilize the traced data flow in the HISA instruction. Instead, the padding selection component 536 can analyze the metadata in the CipherTensor flowing through the FHE adaptive tensor circuit. If a tensor operation such as a convolution requires padding, the padding selection component 536 can determine that the previous operation until the input should maintain the padding. In some aspects, some tensor operations can change strides, in which case the padding selection component 536 can determine that the required padding should be scaled by that factor. The input padding selected by the padding selection component 536 (e.g., in each ciphertext size) can be the maximum padding required by any homomorphic tensor operation in the circuit.
[0048] The rotation key selection component 538 can track the different slots used to rotate the FHE adaptive tensor circuit. By default, as a non-limiting example, the HEAAN encryption mode family usually inserts a public evaluation key of a power-of-two left and right rotation, and all rotations are performed using a combination of power-of-two rotations. The ciphertext size is N=2, so CKKS stores 2log(N)-2 rotation keys by default. Since the rotation keys usually take up a lot of memory, a trade-off between space and performance is required. By storing only the rotation key selection component 538 of these keys, any rotation can be performed. However, in a given FHE adaptive tensor circuit, the different slots to be rotated will not be of the order of N. Therefore, the rotation key selection component 538 can track and determine the different rotation slots used in the FHE adaptive tensor circuit. All rotation HISA instructions store constant slots to be rotated in the symbol analyzer component 530 (for example, converting a right rotation to a left rotation), and other HISA instructions can be ignored. The symbol analyzer component 530 can then return the different slots utilized to rotate. In this manner, the compiler component 510 can generate an encryptor so that it can utilize the private key on the client to generate evaluation keys for these rotations, as described above with reference to Figures 2 to 3 The rotation key selection component 538 need not rotate the key by a power of two, and the rotation key selected by the compiler component 510 is a constant factor of log(N).
[0049] The cost determination component 540 can determine the cost of a particular FHE adaptive tensor circuit under a particular specification based on a variety of factors, such as the duration to complete the execution of the FHE adaptive tensor circuit configured with the particular specification, the number of operations performed by the FHE adaptive tensor circuit configured with the particular specification, and / or the number of library calls made by the FHE adaptive tensor circuit configured with the particular specification. As a non-limiting example, by utilizing the full RNS variant of the CKKS encryption mode and storing the ciphertext and plaintext in the number theoretic transform (NTT) domain, the complexity of mulPlain* can be reduced from O(nlog(n)) to O(n). The compiler component 510 can encode the cost of each operation according to the asymptotic complexity or according to micro-benchmarking (e.g., monitoring and logging) of each operation. In various embodiments, an increase in asymptotic complexity or an increase in the number of operations results in a higher cost associated with the operation and its corresponding tensor circuit specification. These costs can then be analyzed by the cost determination component 540 to decide which specification or data layout to select for a given FHE adaptive tensor circuit. The cost determination component 540 can count the number of occurrences of each operation and then determine the asymptotic complexity and calculate the total cost of the circuit. At this point, the transformer component 520 can generate or instantiate a new FHE adaptive tensor circuit corresponding to a particular tensor circuit specification or data layout, and can determine the cost of the new FHE adaptive tensor circuit via the cost determination component 540. This process can be repeated for different data layout options, among other things.
[0050] The result generation component 542 can generate any combination of results associated with the execution of the FHE adaptive tensor circuit configured with a particular specification or data layout. In various embodiments, the result generation component 542 can generate results including, but not limited to, the cost of the particular specification or data layout determined via the cost determination component 540. The results generated for the FHE adaptive tensor circuit configured with the particular specification or data layout can be stored by the homomorphic evaluation server 110 to a storage medium such as Figure 1 or may be transferred to a storage device such as the data storage device 120. Figure 1 The client device 140 and the like are connected to the client device 140, so that the client device 140 provides the generated results for display.
[0051] The data layout selection component 550 can obtain the determined cost of each tensor circuit specification or data layout for the FHE adaptive tensor circuit to select the FHE adaptive tensor circuit with the lowest determined cost, or to reduce the multiple tensor circuit specifications or data layouts to a smaller subset based on the corresponding determined cost. In more detail, the number of tensor circuit specifications or data layouts can be reduced by comparing the cost to a defined threshold cost and only selecting the tensor circuit specifications or data layouts determined to be below the defined threshold cost. In some other embodiments, the number of possible tensor circuit specifications or data layouts can be reduced by sorting each tensor circuit specification or data layout based on the determined cost (e.g., from lowest cost to highest cost) and only selecting a predefined number of tensor circuit specifications or data layouts at the preferred lower cost end of the ordered list.
[0052] In various aspects, different homomorphic operations may have different costs even within the same FHE mode. The cost of homomorphic operations may also vary greatly based on slight changes in the FHE mode. For example, by utilizing the full RNS variant of the CKKS mode and storing the ciphertext and plaintext in the number theoretic transform (NTT) domain, the data layout selection component 550 can reduce the complexity of mulPlain* from O(nlog(n)) to O(n). The compiler component 510 using the data layout selection component 550 can encode the cost of each operation based on the asymptotic complexity or micro-benchmark each operation. The data layout selection component 550 can then use these costs to decide which implementation or data layout to select for a given FHE adaptive tensor circuit. In this case, the symbol analyzer component 530 can count the number of occurrences of each operation and then determine the total cost of the circuit using the determined asymptotic complexity. The transformer component 520 can then create a FHE adaptive tensor circuit corresponding to the data layout and obtain its cost via the symbol analyzer component 530. This loop is repeated for different possible data layout options, such that data layout selection component 550 can select one or more data layout options associated with the lowest determined costs, as described herein.
[0053] Now turn to Figure 6 , a flowchart 600 is provided, which illustrates a method for generating an optimized FHE adaptive tensor circuit according to some embodiments of the present disclosure. In various embodiments, such as Figure 1 The computing device such as the homomorphic evaluation server 110 can obtain the tensor circuit and the pattern associated with the tensor circuit in step 610. The tensor circuit and the associated pattern can be obtained from, for example, Figure 1 The client computing device 140 or the like may be received by the computing device from a local memory or such as Figure 1The tensor circuit and the associated pattern may be retrieved from a storage device such as storage device 120. The tensor circuit and the associated pattern may be taken as input to evaluate the tensor circuit and determine one or more optimal (e.g., efficient) specifications (e.g., data layout) for the tensor circuit. In various embodiments, the efficiency measure of the tensor circuit specification may be determined by the computing device during execution of the tensor circuit configured according to the tensor circuit specification based on a variety of factors, such as the use of an FHE library (such as Figure 5 The number of calls of the FHE library repository 570), or the determined asymptotic complexity of the execution operation, etc.
[0054] At step 520, the computing device may map the obtained tensor circuit to an equivalent FHE adaptive tensor circuit that is instantiated or generated for the obtained tensor circuit based at least in part on the obtained associated pattern. In other words, the computing device may employ a method such as Figure 5 The compiler component 510 and other compiler components can be used to specify a set of parameters used in the obtained tensor circuit. The compiler component can use the obtained input to define the ciphertext tensor data type as a vector of ciphertext with associated metadata (e.g., vector), which metadata can capture the way the vector of ciphertext is interpreted as a corresponding unencrypted tensor. The metadata may include the physical size of the (external) vector and the (internal) ciphertext, the logical size of the equivalent unencrypted tensor, and the physical stride of each size of the (internal) ciphertext, etc. Since the size of the obtained tensor circuit is known based on the obtained pattern, the compiler component can generate an FHE adaptive tensor circuit equivalent to the obtained tensor circuit, and further determine a strategy for performing homomorphic computations with it. At this point, the compiler component can map the obtained tensor circuit to the generated equivalent FHE adaptive tensor circuit.
[0055] At step 630, the computing device may use a method such as Figure 5 The symbol analyzer component 530 of the embodiment of the present invention is used to analyze the generated equivalent FHE adaptive tensor circuit and other tensor specifications generated for the generated equivalent FHE adaptive tensor circuit. The symbol analyzer component can use a symbol analyzer component such as Figure 5 The runtime component 560 and other runtime components are used to symbolically execute the generated equivalent FHE adaptive tensor circuit and other tensor specifications generated for the generated equivalent FHE adaptive tensor circuit.
[0056] In various embodiments, the compiler component may be used to Figure 5The transformer component 520 and other transformer components generate other tensor circuit specifications for the generated equivalent FHE adaptive tensor circuit. In various embodiments, the obtained tensor circuit can be mapped to the generated equivalent FHE adaptive tensor circuit with unspecified parameters. In this way, the computing device can use the transformer component to determine the parameters to be used with the generated equivalent FHE adaptive tensor circuit.
[0057] In various embodiments, the compiler component may employ a method such as Figure 5 The parameter selection component 536 and other parameter selection components are used to analyze the depth of the generated equivalent FHE adaptive tensor circuit and select a set of encryption parameters based on the analyzed depth. As a non-limiting example, in the HEAAN encryption mode, divScalar consumes the modulus Q from the ciphertext from which it is called. The output should leave enough modulus Q to capture accurate results. The symbol analyzer component can adopt a pseudo-ciphertext data type that increases the modulus Q of the input ciphertext and copies it to the output ciphertext when divScalar is called. In some other embodiments, other HISA instructions only copy the modulus from the input to the output. The modulus Q in the virtual output ciphertext corresponds to the depth of the circuit. Therefore, the input ciphertext can be encrypted using a modulus Q that is at least the sum of the circuit depth and the desired output precision.
[0058] In some other embodiments, the compiler component may also use Figure 5 A padding selection component such as padding selection component 536 of the FHE adaptive tensor circuit is used to analyze metadata in the CipherTensor flowing through the FHE adaptive tensor circuit. If a tensor operation such as a convolution requires padding, the padding selection component can determine that the padding should be maintained until the input is received. In some embodiments, the tensor operation can change stride, in which case the padding selection component can determine the required padding ratio as a factor corresponding to the change. The input padding (in each ciphertext size) selected by the padding selection component can correspond to the maximum padding required by any homomorphic tensor operation performed by the circuit.
[0059] In some other embodiments, the compiler component may also use Figure 5A rotation key selection component 538 and the like of the rotation key selection component can track the different slots used to rotate the generated equivalent FHE adaptive tensor circuit. As an example, the HEAAN encryption mode family typically inserts a public evaluation key of a power-of-two left and right rotation, and all rotations are performed using a combination of power-of-two rotations. The ciphertext size is N=2, so CKKS stores 2log(N)-2 rotation keys by default. Since rotation keys typically take up a lot of memory, a trade-off between space and performance is required. By storing only the rotation key selection component of these secret keys, any rotation can be performed. However, in a given FHE adaptive tensor circuit, the different slots to be rotated will not be of the order of N. Therefore, the rotation key selection component can track the different slots used for rotation in the generated equivalent FHE adaptive tensor circuit. All rotation HISA instructions can store constant slots to be rotated in the symbol analyzer component (e.g., converting a right rotation to a left rotation), while other HISA instructions can be ignored. The symbol analyzer component can then return the different slots utilized for rotation. In this manner, the compiler component can generate a cryptographic key such that it can be utilized to utilize a private key on a computing device or client device to generate evaluation keys for these rotations, as described above with reference to Figures 2 to 3 In this regard, the rotation key selection component need not rotate the key by a power of two, and the rotation key selected by the compiler component may maintain a constant factor of log(N).
[0060] Since the symbolic analyzer component uses the runtime component to symbolically execute each change (e.g., specification, data layout) of the generated equivalent FHE adaptive tensor circuit, the computing device can use such as Figure 5A cost determination component such as a cost determination component 540 of the generated equivalent FHE adaptive tensor circuit determines the cost of each specification or data layout of the generated equivalent FHE adaptive tensor circuit. In various embodiments, the cost of the FHE adaptive tensor circuit specification can be determined based on a variety of factors, such as a determined duration to complete the execution of the FHE adaptive tensor circuit under the specification or data layout, a determined number of operations performed by the FHE adaptive tensor circuit under the specification or data layout, and / or a determined number of library calls made by the FHE adaptive tensor circuit under the specification or data layout. In various embodiments, the determined increase in asymptotic complexity or an increase in the number of operations may result in a higher cost associated with the operation and its corresponding tensor circuit specification. These costs can then be analyzed by the cost determination component to determine which specification or data layout to select for a given FHE adaptive tensor circuit. In some embodiments, the cost determination component can count the number of occurrences of each operation and determine the asymptotic complexity to calculate the total cost of the circuit. At this point, the transformer component can generate or instantiate a new FHE adaptive tensor circuit corresponding to a particular tensor circuit specification or data layout, and the cost of the FHE adaptive tensor circuit can be determined via the cost determination component. This process can be repeated for different possible specifications or data layout options.
[0061] At step 650, the computing device may employ a method such as Figure 5 A data layout selection component such as a data layout selection component 550 of the FHE adaptive tensor circuit may obtain the cost of each tensor circuit specification or data layout of the FHE adaptive tensor circuit determined by the cost determination component. The data layout selection component may select a specification or data layout with the lowest cost, or may reduce multiple tensor circuit specifications or data layouts to a smaller subset. In some embodiments, the number of tensor circuit specifications or data layouts may be reduced by comparing the cost with a defined threshold cost, so that the data layout selection component selects a smaller subset of tensor circuit specifications or data layouts determined to be below the defined threshold cost. In some other embodiments, the number of tensor circuit specifications or data layouts may be reduced by sorting each tensor circuit specification or data layout based on its determined cost (e.g., from the lowest cost to the highest cost), so that the data layout selection component selects only a predefined number of tensor circuit specifications or data layouts at the lowest cost end of the ordered list of tensor circuit specifications or data layouts.
[0062] In some other embodiments, the computing device may store the selected set of tensor circuit specifications or data layouts to a storage medium or device, provide the selected set of tensor circuit specifications or data layouts for display, and / or transmit the selected set of tensor circuit specifications or data layouts to a client device, such as Figure 1The client device 140 of the transmitted set may cause the client device to provide for display a selected set of tensor circuit specifications or data layouts determined to be most economical (e.g., optimal) for configuring the FHE adaptive tensor circuit associated with the acquired tensor circuit and mode. The provided set of tensor circuit specifications or data layouts may be employed by any computing device, such as a client device, to optimize the specifications or data layouts of the FHE adaptive tensor circuits for performing homomorphic encryption operations on encrypted data.
[0063] Having described embodiments of the present invention, an exemplary operating environment in which embodiments of the present invention may be implemented is described below to provide a general background for various aspects of the present invention. Figure 7 , an exemplary operating environment for implementing embodiments of the present invention is shown, and is generally represented as computing device 700. Computing device 700 is only one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the present invention. Neither should computing device 700 be interpreted as having any dependency or requirement relating to any one component or combination of components illustrated.
[0064] The present invention may be described in the general context of computer code or machine usable instructions, including computer executable instructions (such as program modules) executed by a computer or other machine (such as a personal data assistant or other handheld device). In general, program modules, including routines, programs, objects, components, data structures, etc., refer to code that performs a specific task or implements a specific abstract data type. The present invention may be practiced in a variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, more specialized computing devices, etc. The present invention may also be practiced in a distributed computer environment in which tasks are performed by remote processing devices linked through a communications network.
[0065] refer to Figure 7 , computing device 700 includes bus 710 that directly or indirectly couples: memory 712, one or more processors 714, one or more presentation components 716, input / output (I / O) ports 718, input / output components 720, and an illustrative power supply 722. Bus 710 represents what may be one or more buses, such as an address bus, a data bus, or a combination thereof. Although Figure 7 The various boxes of FIG. 1 are shown with lines for clarity, but in reality, it is not very clear to delineate the various components, and metaphorically, the lines would more accurately be gray and fuzzy. For example, one could consider presentation components such as a display device to be I / O components. In addition, the processor has memory. The inventors recognize that this is the nature of the prior art, and reiterate that Figure 7The figures are merely illustrations of exemplary computing devices that may be used in conjunction with one or more embodiments of the present invention. No distinction is made between categories such as "workstation," "server," "notebook," "handheld device," etc., as all of these are described in the same manner as those in the examples of the present invention. Figure 7 within the scope of and with reference to a “computing device”.
[0066] The computing device 700 typically includes various computer-readable media. Computer-readable media can be any available media that can be accessed by the computing device 700, and includes volatile and non-volatile media and removable and non-removable media. As an example and not limitation, computer-readable media can include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other storage technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by the computing device 700. Computer storage media do not include signals themselves. Communication media typically embody computer-readable instructions, data structures, program modules or other data in modulated data signals such as carrier waves or other transmission mechanisms, and include any information delivery media. The term "modulated data signal" refers to a signal that has one or more of its characteristics set or changed in a manner that encodes information as a signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of any of the above should also be included within the scope of computer-readable media.
[0067] Memory 712 includes computer storage media in the form of volatile and / or non-volatile memory. Memory may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. Computing device 700 includes one or more processors that read data from various entities such as memory 712 or I / O components 720. Presentation component(s) 716 present data indications to a user or other device. Exemplary presentation components include display devices, speakers, printing components, vibration components, etc.
[0068] I / O port 718 allows computing device 700 to be logically coupled to other devices including I / O components 720, some of which may be built-in. Illustrative components include microphones, joysticks, game controllers, satellite dishes, scanners, printers, wireless devices, etc. I / O components 720 may provide a natural user interface (NUI) that processes air gestures, voices, or other physiological inputs generated by a user. In some cases, the input may be transmitted to an appropriate network element for further processing. NUI may implement speech recognition, stylus recognition, facial recognition, biometric recognition, gesture recognition on and adjacent to the screen, air gestures, head and eye tracking, and touch recognition associated with the display of computing device 700 (described in more detail below). Computing device 700 may be equipped with a depth camera for gesture detection and recognition, such as a stereo camera system, an infrared camera system, an RGB camera system, touch screen technology, and combinations thereof. In addition, computing device 700 may be equipped with an accelerometer or gyroscope capable of detecting motion. The output of the accelerometer or gyroscope may be provided to the display of computing device 700 to render an immersive augmented reality or virtual reality.
[0069] It will be appreciated that embodiments of the present invention provide, among other things, generation, evaluation, and selection of optimized FHE adaptive tensor circuits and corresponding specifications (e.g., encryption parameters, data layout) to perform homomorphic encryption operations on encrypted data. The present invention has been described with respect to specific embodiments, which are intended in all respects to be illustrative rather than restrictive. Alternative embodiments will become apparent to those of ordinary skill in the art to which the present invention pertains without departing from the scope of the present invention.
[0070] From the foregoing it can be seen that the present invention is well suited to achieve all of the above-mentioned aims and objectives, as well as other advantages which are clearly inherent to the system and method. It should be understood that certain features and sub-combinations are useful and can be employed without reference to other features and sub-combinations. This is contemplated by the claims and is within the scope of the claims.
Claims
1. A computer-implemented method for generating an optimized fully homomorphic encrypted (FHE) adaptive tensor circuit, the method comprising: Retrieving, by a computing device, a tensor circuit, a mode associated with the tensor circuit, and a defined output precision; mapping, by the computing device, the obtained tensor circuit to an equivalent FHE adaptive tensor circuit generated based at least in part on the associated mode and the defined output precision; monitoring, by the computing device, calls to a library by the generated equivalent FHE adaptive tensor circuit for each corresponding determined specification of a plurality of determined specifications utilized by the generated equivalent FHE adaptive tensor circuit; determining, by the computing device, a cost for each of the plurality of determined specifications based at least in part on the corresponding monitored library calls; as well as A set of specifications is selected, by the computing device, from the plurality of determined specifications based at least in part on the determined costs, wherein the selected set of specifications corresponds to a set of specifications for the obtained tensor circuit to perform homomorphic encryption operations on encrypted data.
2. The computer-implemented method of claim 1 , wherein each specification is determined based at least in part on the monitored library calls corresponding to previously determined specifications utilized by the generated equivalent FHE adaptive tensor circuit.
3. The computer-implemented method of claim 1 , wherein a set of encryption parameters is selected for each of the determined plurality of specifications based at least in part on the determined depth of the obtained tensor circuit.
4. The computer-implemented method of claim 1 , wherein the generated equivalent FHE adaptive tensor circuit utilizing each of the plurality of determined specifications is symbolically executed via runtime.
5. The computer-implemented method of claim 1 , wherein a data object is generated based at least in part on the acquired tensor circuit and associated pattern, the data object being employable to determine padding for homomorphic tensor operations symbolically executed via the generated equivalent FHE adaptive tensor circuit. 6 . The computer-implemented method of claim 1 , wherein the cost of each determined specification is further determined based in part on a corresponding asymptotic complexity.
7. The computer implemented method of claim 1, wherein a number of calls to the library are monitored and the cost of each specification is determined based at least in part on the corresponding number of monitored library calls.
8. The computer-implemented method of claim 1, wherein the defined output precision corresponds to floating point precision.
9. The computer-implemented method of claim 1, wherein the pattern comprises inputs and weights.
10. A non-transitory computer storage medium storing computer usable instructions that, when used by one or more computing devices, cause the one or more computing devices to perform operations comprising: Retrieving a tensor circuit, a mode associated with the tensor circuit, and a defined output precision; mapping the obtained tensor circuit to an equivalent fully homomorphically encrypted (FHE) adaptive tensor circuit generated based at least in part on the associated mode and the defined output precision; monitoring calls to a library by the generated equivalent FHE adaptive tensor circuit for each corresponding determined specification of a plurality of determined specifications utilized by the generated equivalent FHE adaptive tensor circuit; determining a cost for each of the plurality of determined specifications based at least in part on the corresponding monitored library calls; as well as A set of specifications is selected from the plurality of determined specifications based at least in part on the determined costs, wherein the selected set of specifications corresponds to a set of specifications for the obtained tensor circuit to perform homomorphic encryption operations on encrypted data.
11. The non-transitory computer storage medium of claim 10, wherein each specification is determined based at least in part on the monitored library calls corresponding to previously determined specifications utilized by the generated equivalent FHE adaptive tensor circuit.
12. The non-transitory computer storage medium of claim 10, wherein a set of encryption parameters is selected for each of the determined plurality of specifications based at least in part on the determined depth of the obtained tensor circuit.
13. The non-transitory computer storage medium of claim 10, wherein the generated equivalent FHE adaptive tensor circuit utilizing each of the plurality of determined specifications is symbolically executed via runtime.
14. The non-transitory computer storage medium of claim 10, wherein a data object is generated based at least in part on the acquired tensor circuit and associated pattern, the data object being employable to determine padding for a homomorphic tensor operation symbolically executed using the generated equivalent FHE adaptive tensor circuit.
15. The non-transitory computer storage medium of claim 10, wherein the cost of each determined specification is further determined based in part on a corresponding asymptotic complexity.
16. A system for generating optimized fully homomorphic encrypted (FHE) adaptive tensor circuits, comprising: comprising at least one hardware processor; input receiving means for a tensor circuit, a mode associated with said tensor circuit, and a defined output precision; transforming means for mapping the received tensor circuit to a fully homomorphically encrypted FHE adaptive tensor circuit generated based at least in part on the associated mode and the defined output precision, wherein the FHE adaptive tensor circuit having a first specification is based at least in part on the associated mode and the defined output precision; symbolic analysis means for providing a result generated for symbolic execution of the FHE adaptive tensor circuit; the result generated to instantiate the FHE adaptive tensor circuit having a second specification equivalent to the received tensor circuit; cost determining means for determining a cost of the symbolically executed FHE adaptive tensor circuit based at least in part on monitored calls to a library by the FHE adaptive tensor circuit; and Data layout selection means for selecting one or more efficient specifications for the FHE adaptive tensor circuit based at least in part on the determined cost of the symbolic execution of the FHE adaptive tensor circuit using at least the first specification and the second specification.
17. The system of claim 16, further comprising: Parameter selection means for selecting encryption parameters for symbolically executing the FHE adaptive tensor circuit, the encryption parameters being selected for the FHE adaptive tensor circuit based at least in part on the received determined depth of the tensor circuit.
18. The system of claim 16, further comprising: Compiling means for generating a data object corresponding to the received tensor circuit based at least in part on the associated pattern, the data object being employable to determine padding for homomorphic tensor operations performed in symbolic execution of the FHE adaptive tensor circuit.
19. The system of claim 16, further comprising: The FHE adaptive tensor circuit transformation device is used to instantiate the FHE adaptive tensor circuit with the second specification based on the generated result.
