Articles and methods for automated parallelization of zero knowledge protocols
The automatic partitioning and parallelization of zero-knowledge proof protocols address scalability and flexibility issues by optimizing task distribution across multiple cores, enhancing the efficiency and accuracy of ZKP implementations for complex statements.
Patent Information
- Application Number
- PCT/US2024/026025
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-04-25
- Filing Date
- 2024-04-24
- Publication Date
- 2024-12-05
AI Technical Summary
Current zero-knowledge proof (ZKP) technologies face challenges in scalability and flexibility, as they require manual partitioning of computations for distributed processing, leading to suboptimal solutions and errors, especially for complex real-world statements, and lack a general-purpose framework for supporting multiple computational models.
A method and framework for automatically partitioning and parallelizing ZKP protocols by calculating local and communication costs, distributing tasks across multiple computing cores, and using shallow and deep simulation to optimize chunking and variable selection, ensuring consistency and scalability.
This approach enables efficient and scalable implementation of ZKPs, reducing computational overhead and allowing for complex statements to be processed in parallel, improving the practicality and accuracy of zero-knowledge proofs in real-world applications.
Smart Images

Figure US2024026025_05122024_PF_FP_ABST
Abstract
Description
[0001] TITLE OF THE INVENTION ARTICLES AND METHODS FOR AUTOMATED PARALLELIZATION OF ZERO KNOWLEDGE PROTOCOLS CROSS-REFERENCE TO RELATED APPLICATIONS The present application claims priority under 35 U.S.C. § 119(e) to U.S. Provisional Patent Application No.63 / 498,230, filed April 25, 2023, which application is incorporated herein by reference in its entirety. STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT This invention was made with government support under CCF-2106845, CCF-2131476, CCF-1763399, 2016240, and 2236819 awarded by National Science Foundation (NSF) and HRO002230C0087 awarded by Defense Advanced Research Projects Agency (DARPA). The government has certain rights in the invention. BACKGROUND OF THE INVENTION Zero-knowledge proofs (ZKPs) enable a prover to convince a verifier about the truth of some statement without revealing why. Recent advances by the cryptographic research community have brought a tremendous improvement in the efficiency of general-purpose zero- knowledge (ZK) proving, as well as numerous creative applications of ZKPs. Despite this progress, there are still significant challenges that prevent the widespread adoption of ZKP in real-world scenarios. For example, when applying ZKPs to new problems, a major challenge is to describe the statement to be proven in a way that is both natural for programmers and suitable for compilation. Most ZK languages and compilers currently available only support the circuit model, which relies on data-oblivious computation and requires that control flow be independent of all private variables. However, this model doesn't fully capture the flexibility and power of modern ZKP constructions. For instance, non-determinism can accelerate many statements where the prover provides extra hints to the computation. Other statements, such as 3-coloring and Hamiltonian cycles in graphs, can be verified with high efficiency under a probabilistic guarantee, where the verifier provides extra random challenges and the prover replies with challenge-dependent hints. Unfortunately, there's currently no general-purpose ZK programming framework that can support proofs across multiple computational models. Another practical challenge of ZKP is the demand for massive hardware resources, which limits scaling ZKPs to large statements and creates bottlenecks in computation and communication resources. To address this, recent research has explored "scaling out" ZK protocols by distributing the task among a cluster of machines. However, this requires modifying the ZKP protocols to be amenable to distributed computation and manually partitioning the computation into smaller chunks to be proven in parallel. These manual processes may introduce errors and lead to suboptimal solutions, especially for statements of real-world complexity. Accordingly, there is a need in the art for articles and methods that improve on existing articles and methods for partitioning, parallelizing, and solving ZKPs. The present invention addresses this need. SUMMARY OF THE INVENTION In one aspect, a method of parallelizing zero-knowledge (ZK) protocols includes providing a ZK framework; calculating local and communication costs for each program instruction of a ZK statement; automatically partitioning the ZK statement according to the calculation; and distributing the partitioned ZK statement to a corresponding number of computing cores for parallel proving or verification. In some embodiments, the distributed ZK statement includes randomized verification. In some embodiments, the calculating includes determining sums of the maximum per- machine cost and cumulative communication costs. In some embodiments, the partitioning of the ZK statement includes chunking the ZK statement to provide a lowest or substantially lowest sum of the maximum per-machine cost and cumulative communication costs based upon the calculation. In some embodiments, the partitioning of the ZK statement is arbitrary. In some embodiments, the ZK framework permits proving multiple statements of a single witness while ensuring consistency. In some embodiments, the framework is configured to execute a commit- and-prove paradigm. In some embodiments, the method further includes a prover committing private inputs; and constructing a proof establishing some public relationship among the committed values; wherein the public relationship among the committed values is established without revealing the committed values to a verifier. In some embodiments, the method further includes establishing multiple statements on the private inputs. In some embodiments, the partitioning comprises forming at least two substatements. In some embodiments, the method further includes precomputing the input / output of each substatement. In some embodiments, precomputing comprises forming an extended witness, the extended witness resolving data dependency. In some embodiments, the partitioning includes shallow simulation and deep simulation. In some embodiments, the shallow simulation includes the automatic partitioning of the ZK statement; and selecting of variables as witnesses for each substatement. In some embodiments, the shallow simulation includes computing the costs of each instruction in the ZK statement using live variable analysis; and modeling the computed costs as a directed acyclic graph for partitioning. In some embodiments, the partitioning includes pseudo-Boolean optimization (PBO) or integer linear programming (ILP). In some embodiments, the deep simulation includes computing extended witness values using a prover’s private inputs; and computing public variable values. In some embodiments, the computing of the extended witness values and public variable values includes propagating input-dependent values through the ZK statement. In some embodiments, the framework includes annotation tools configured to identify knowledge levels. In some embodiments, the knowledge levels include private input-independent values known at compile time, private input-dependent values that are known at distribution time, and variables that are only known at runtime. In some embodiments, the private input- independent values and the private input-dependent values are allowed to flow into the variables that are only known at runtime, but the variables that are only known at runtime are not allowed to flow into the private input-independent values and the private input-dependent values. In some embodiments, the partitioning step includes treating the variables that are only known at runtime as a single contiguous block to estimate worth-case cost and enforce sequential ordering. Also provided is an apparatus for parallelizing zero-knowledge (ZK) protocols, the apparatus including a processor; a memory unit; and a communication interface; wherein the processor is connected to the memory unit and the communication interface; and wherein the processor and memory are configured to implement the method of any of the embodiments disclosed herein. Also provided herein is an apparatus for proving zero-knowledge (ZK) protocols, the apparatus including at least two processors; a memory unit; and a communication interface; wherein the processors are connected to the memory unit and the communication interface; and wherein a ZK statement parallelized according to the method of any of the embodiments disclosed herein is distributed among the at least two processors. Further provided herein is a non-transitory computer readable storage medium storing computer-executable instructions for performing the method of any of the embodiments disclosed herein. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 shows an image illustrating an example of an Ou implementation of Freivalds’ algorithm for randomized verification of 100x100 matrix multiplication. FIG. 2 shows a schematic illustrating the DZKP framework, beginning with an Ou program and ending in binaries for an implementation of a ZKP protocol. Square blocks with light text identify the compiler components: the operations (blue) and their underlying methods (green). Dark text indicates a program or protocol: the Ou program (purple semi-rounded rectangle), compiled substatement binaries (gold rounded rectangles), and the ZKP protocol itself (light blue diamond). Corresponding knowledge levels are displayed as well. FIG. 3 shows a schematic illustrating a security lattice. FIG. 4 shows an image illustrating the syntax of the Ou language compiling a reconciliation step into the binaries themselves, which requires no additional expertise on behalf of the verifier. FIG. 5 shows an image illustrating typing rules. FIG. 6 shows an image illustrating shallow semantics. FIG. 7 shows a schematic illustrating shallow semantics refinement. FIG. 8 shows a schematic illustrating deep semantics refinement. FIG. 9 shows a schematic illustrating execution in a chunk. FIGS. 10A-10B show the effective ratio for the distribution scheme found by DZKP for the gradient descent (GD) programs with (A) different numbers of iterations and (B) various numbers of machines (κ). With the increase in κ, the effective ratio will reach the limit. A small- size program with a large κ will make the effective ratio of the distribution scheme much smaller than κ. FIGS. 11A-B show graphs illustrating that the effective ratio for the distribution scheme found by DZKP for the MT programs is close to (A) the numbers of machines (κ) with (B) different numbers of input blocks and κ. FIG. 12 shows graphs illustrating that compiling takes significantly greater time for GD programs when the function update_all() is not annotated with the atomic keyword. On the other hand, atomic annotation rules out some feasible solutions, making the distribution scheme slightly suboptimal. FIG. 13 shows graphs illustrating that adding annotation improves both the effectiveness of both compilation and the effective ratio of the MT program for 32-block input. FIG. 14 shows an image illustrating a merkle tree (MT) example. FIG. 15 shows an image illustrating MT after shallow simulation. FIG. 16 shows a schematic illustrating a MT dependency graph. FIG. 17 shows an image illustrating MT generated programs. FIG. 18 shows an image illustrating dynamtic semantics. FIG. 19 shows schematics illustrating conversion of symbolic values and expressions. FIG. 20 shows graphs illustrating the effect of adding atomic annotations to the MT program for 16-block inputs. FIG. 21 shows graphs illustrating the effect of adding atomic annotations to the MT program for 64-block inputs. FIG. 22 shows graphs illustrating the effect of adding atomic annotations to the MT program for 8-block inputs. FIGS. 23A-D show program structures of GD and MT under different conditions. (A) Program structure of gradient descent (GD) with update_all() annotated with atomic keyword. (B) Program structure of GD without update_all() being atomic. (C) Program structure of merkle tree (MT) with sha256_node() annotated with atomic keyword. (D) Program structure of MT without sha256_node() being atomic. DETAILED DESCRIPTION OF THE INVENTION Definitions Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Although any methods and materials similar or equivalent to those described herein can be used in the practice or testing of the present invention, the preferred methods and materials are described. The articles “a” and “an” are used herein to refer to one or to more than one (i.e., to at least one) of the grammatical object of the article. By way of example, “an element” means one element or more than one element. “About” as used herein when referring to a measurable value such as an amount, a temporal duration, and the like, is meant to encompass variations of ±20% or ±10%, more preferably ±5%, even more preferably ±1%, and still more preferably ±0.1% from the specified value, as such variations are appropriate to perform the disclosed methods. Ranges: throughout this disclosure, various aspects of the invention can be presented in a range format. It should be understood that the description in range format is merely for convenience and brevity and should not be construed as an inflexible limitation on the scope of the invention. Accordingly, the description of a range should be considered to have specifically disclosed all the possible subranges as well as individual numerical values within that range. For example, description of a range such as from 1 to 6 should be considered to have specifically disclosed subranges such as from 1 to 3, from 1 to 4, from 1 to 5, from 2 to 4, from 2 to 6, from 3 to 6 etc., as well as individual numbers within that range, for example, 1, 2, 2.7, 3, 4, 5, 5.3, and 6. This applies regardless of the breadth of the range. Detailed Description Provided herein are methods of parallelizing zero-knowledge (ZK) protocols. In some embodiments, the method includes providing a ZK framework, calculating local and communication costs for each program instruction of a ZK statement, automatically partitioning the ZK statement according to the calculation, and distributing the partitioned ZK statement to a corresponding number of computing cores for parallel proving or verification. In some embodiments, the programming framework permits programmers to focus on writing ZK applications without worrying about the complexities of distributed computation. Additionally or alternatively, in some embodiments, the ZK framework includes a programming language with annotations that make it easy for developers, even those without extensive cryptography knowledge, to write ZK applications using advanced protocols. Suitable programming languages include, but are not limited to, C-like programming languages. In some embodiments, the framework includes annotations for identifying the public or private nature of information and / or the knowledge level of the information. For example, in one embodiment, the knowledge levels include private input-independent values known at compile time (K0), private input-dependent values that are known at distribution time (K1), variables that are only known at runtime (K2), and / or any other suitable knowledge level for variables / values of the ZK statement. Accordingly, in some embodiments, the method includes specifying non-interference along two dimensions, such as, but not limited to, the public or private nature of the information and the knowledge level of the information. In some embodiments, the framework includes a compiler and a static analysis pipeline. In some embodiments, the framework focuses on a commit-and-prove paradigm, where the prover first commits to their private inputs, and then constructs a proof establishing some (public) relationship among the committed values without revealing them to the verifier. The framework calculates the local and communication costs of each program instruction, then automatically chunks, or partitions, the corresponding ZK statement according to the calculations. In some embodiments, the framework performs the calculating and partitioning steps using two compilation phases – shallow simulation and deep simulation. In such embodiments, for example, the shallow simulation automatically partitions the ZK statement and then selects variables as witnesses for each of κ substatements. In some embodiments, the parameter κ is user-chosen, and equals the number of available compute cores in the cluster. Following shallow simulation, the deep simulation then uses a prover’s private inputs to concretely compute the values of extended witnesses, as well as of any public variables. In some embodiments, prior to the shallow simulation, the method includes parsing and / or typechecking the program. The shallow simulation proceeds by using live variable analysis to compute the costs of each instruction in the ZK statement, and then models them as a directed acyclic graph for partitioning. A non-limiting example of live variable analysis is discussed in Section 5.1 of Example 1 herein. The live variable analysis tracks dependencies among independent blocks formed from the instructions, and the results of the live variable analysis are used to decorate the graph with both edge and node labels. Each edge and node label indicates a form of cost to the partition. For example, in some embodiments, the nodes are labeled with their computation cost while edges are labeled with the cutting cost. The computation cost models the time required by the prover and verifier to execute the block’s proving or verification operations in the chosen ZKP protocol backend. The cutting cost models the communication cost required to reconcile the state of the blocks across compute nodes in the cluster. After labeling the edges and nodes, the ZK statement is partitioned by any suitable method, such as, but not limited to, pseudo-Boolean optimization (PBO) or integer linear programming (ILP). The partitioning cuts, or chunks, the ZK statement to provide the lowest or substantially lowest sum of the maximum per-machine cost and cumulative communication costs, as calculated from the costs associated with the labels. A non-limiting example of the cutting is discussed in Section 5.2 of Example 1 herein. Following partitioning, the partitioned program is compiled into binaries for any suitable chosen ZKP protocol backend. Suitable backends include, but are not limited to, public-coin zero-knowledge proofs, private-coin zero-knowledge proofs, NIZK protocols obtained via the Fiat-Shamir (FS) transformation, NIZK protocols obtained without FS transformation, and / or any other suitable backend. In some embodiments, all computation through shallow simulation is based only on public information. Accordingly, in such embodiments, the binaries may be freely distributed and reused without divulging private information. In some embodiments, the shallow simulation produces a sequential program including a flattened sequence of operations through loop unrolling and function inlining. In some embodiments, every instruction in the original program, with the exception of constituent instructions of functions decorated by the atomic keyword and all K2-annotated variables, forms its own independent block in the sequential program. The deep simulation then computes dependent values between partitions to facilitate running of the programs in parallel. In some embodiments, deep simulation includes a prover simulating execution of each substatement program using their private information, then saving the output values of each one that then becomes the input to others. These output values are referred to herein as the extended witnesses. After all the chunks have been checked in parallel, their inputs and outputs are reconciled to check consistency of variables passed across the cuts. In some embodiments, a proof with different inputs may include recomputing only the deep simulation, and not the shallow simulation or partitioning (which is likely to be the most computationally intensive compilation task). In some embodiments, the methods disclosed include randomized verification, which in turn supports general-purpose ZK. In some embodiments, the methods disclosed herein provide arbitrary partitioning of the ZK statement. Additionally or alternatively, in some embodiments, the method permits proving multiple statements of a single witness while ensuring consistency. In some embodiments, the method includes allowing the private input-independent values and the private input-dependent values to flow into the variables that are only known at runtime, but not allowing the variables that are only known at runtime to flow into the private input- independent values and the private input-dependent values. This flow of values enforces non- interference with respect to a security lattice, ensuring the security and correctness of resulting ZK programs when randomized verification are introduced. In some embodiments, the security lattice specifies non-interference along two dimensions: (1) the public or private nature of the information, and (2) the knowledge level of the information (e.g., if the information is available at compile time, distribution time, or run time). In such embodiments, the lattice defines seven security domains, as discussed in detail in the Examples section below. In some embodiments, the method includes sublinear representation through a user- provided annotation to indicate that a function should be condensed, rather than flattened, during the shallow simulation. The user-provided annotation is also referred to herein as atomic. By condensing, rather than flattening, during the shallow simulation, the component of the cost graph corresponding to the function is contracted into a single node of cost cumulative of its constituent instructions. This reduces the size of the graph and facilitates optimization of large statements. In some embodiments, even if the atomic function is called multiple times, only one copy of the declaration exists. This enables the partitioning of large ZK statements for both PBO and ILP solving, despite its inherent hardness. Accordingly, in some embodiments, the method may be scaled to large ZK statements. Also provided herein, in some embodiments, is an apparatus for parallelizing zero- knowledge (ZK) protocols, the apparatus including a processor, a memory unit, and a communication interface. The processor is connected to the memory unit and the communication interface, and the processor and memory are configured to implement the method according to any of the embodiments disclosed herein. Additionally or alternatively, in some embodiments, the apparatus includes an apparatus for proving zero-knowledge (ZK) protocols. In such embodiments, the apparatus includes at least two processors, a memory unit, and a communication interface. The processors are connected to the memory unit and the communication interface, and a ZK statement parallelized according to any of the methods disclosed herein is distributed among the at least two processors. Further provided herein, in some embodiments, is a computer readable storage medium storing computer-executable instructions for performing the method according to any of the embodiments disclosed herein. Without wishing to be bound by theory, it is believed that the articles and methods disclosed herein facilitate the practical implementation of Zero-Knowledge Proofs (ZKP) in real- world software, overcoming the limitations of existing ZKP implementations. Additionally, the framework disclosed herein permits non-expert programmers to effortlessly write efficient and intuitive ZKP programs. Furthermore, the articles and methods disclosed herein facilitate the scalability and cost savings offered by distributed computing, which is especially advantageous for computationally intensive ZKP programs or those involving large amounts of data. By distributing the workload across multiple machines, computations can be completed faster, and the burden on programmers is reduced through automatic parallelization. This simplifies the creation of complex ZKP applications that leverage distributed computing, without requiring specialized knowledge of distributed systems or parallel programming. Accordingly, the articles and methods disclosed herein may be applied to various applications such as, but not limited to, blockchain privacy and scalability, legal systems, and anonymous networks. Those skilled in the art will recognize, or be able to ascertain using no more than routine experimentation, numerous equivalents to the specific procedures, embodiments, claims, and examples described herein. Such equivalents are considered to be within the scope of this invention and covered by the claims appended hereto. It is to be understood that wherever values and ranges are provided herein, all values and ranges encompassed by these values and ranges, are meant to be encompassed within the scope of the present invention. Moreover, all values that fall within these ranges, as well as the upper or lower limits of a range of values, are also contemplated by the present application. The following examples further illustrate aspects of the present invention. However, they are in no way a limitation of the teachings or disclosure of the present invention as set forth herein. EXAMPLES EXAMPLE 1 Abstract A zero-knowledge proof (ZKP) is a powerful cryptographic primitive used in many decentralized or privacy-focused applications. However, the high overhead of ZKPs can restrict their practical applicability. This Example describes the design of a programming language, Ou, aimed at easing the programmer’s burden when writing efficient ZKPs, and a compiler framework, DZKP, that automates the analysis and distribution of statements to a computing cluster. DZKP uses programming language semantics, formal methods, and combinatorial optimization to automatically partition an Ou program into optimally sized chunks for parallel ZK-proving and / or verification. Discussed herein are: (1) A front-end language where users can write proof statements as imperative programs in a familiar syntax; (2) A compiler architecture and implementation that automatically analyzes the program and compiles it into an optimized IR that can be lifted to a variety of ZKP constructions; and (3) A cutting algorithm, based on Pseudo-Boolean optimization and Integer Linear Programming, that reorders instructions and then partitions the program into optimally sized chunks for parallel evaluation and efficient state reconciliation. 1. Introduction Zero-knowledge proofs (ZKPs) (Oded Goldreich, Silvio Micali, and Avi Wigderson. 1986. Proofs that Yield Nothing But their Validity and a Methodology of Cryptographic Protocol Design (Extended Abstract). In 27th FOCS. IEEE Computer Society Press.) enable a prover to convince a verifier about the truth of some statement without revealing why. Recent advances by the cryptographic research community have brought a tremendous improvement in the efficiency of general-purpose zero-knowledge (ZK) proving, as well as numerous creative applications of ZKPs (e.g., reputation systems of anonymous networks, decentralized cryptocurrency, decentralized private computation). However, various practical challenges make the adoption of ZK unfeasible for tasks of realistic sizes. A particular problem is the lack of a comprehensive toolkit of programming languages and compilers providing a full suite of effective, intuitive, parallel-aware, and general methods for writing complex statements intended for efficient ZKPs. Challenge 1: Simple and scalable development. One significant challenge for applying ZKPs to new problems is describing the statement to be proven in a way that is at the same time natural to the programmer and conducive to compilation. Most existing languages and compilers for ZK (and for the closely related problem of secure multiparty computation, or MPC) support only the ‘circuit model’ based on data- oblivious computation, where control flow must be independent of all private variables. They encode the statement as a monolithic program and emit an appropriate circuit representation for their target protocol. Although this model is theoretically complete, it does not capture the full flexibility and power of modern ZKP constructions. For example, many statements can be accelerated by non-determinism, where the prover provides extra hints to the computation. Other statements (e.g., 3-coloring and Hamiltonian cycles in graphs) can be verified with high concrete efficiency under a probabilistic guarantee: the verifier provides extra random challenges, to which the prover replies with challenge-dependent hints. Provided in this Example is a more general-purpose ZK programming framework able to support proofs over multiple computational models. The aim is to ease development for new domains – especially so for ZK statements that are more naturally expressed in those alternative models, or for which those models provide superior performance. Challenge 2: Scalability in hardware resources. All sufficiently advanced technology may be indistinguishable from magic, but the ‘magic’ of ZK often depends as much in practice on massive hardware resources as on mathematical ingenuity. These demands lead to practical bottlenecks in computation and / or communication resources when scaling ZKPs to large statements. For example, most zkSNARK protocols require temporary storage linear in the running time of the computation being proved. Although VOLE-based ZKPs use less memory, they only do so at the expense of a significant increase in required bandwidth. To aid practicality, a line of recent work has started to “scale out” ZK protocols: instead of assuming the prover and / or verifier have access to one giant machine, they assume access to a cluster of machines and the capability to distribute the task amongst these resources. For example DIZK distributes the prover computation of Groth’s ZK protocol by manually partitioning the R1CS constraints into equal-sized chunks. zkBridge can distribute the prover computation of Virgo efficiently to multiple servers when the statement already has a high degree of parallelism. Meanwhile, EZEE distributes among multiple machines the prover-verifier computation and communication of an interactive, garbled-circuit-based zero-knowledge proof protocol. Giraffe addresses the challenge of automatically distributing computations in the verifiable outsourcing context by breaking down overly large computations that cannot be outsourced as a whole. Though efficient in deployment, in order to distribute their chosen ZK protocol over a cluster of servers each of these works require the developer to first both i) modify the ZKP protocols to be amenable to distributed computation; and ii) to manually partition the computation into smaller chunks to be proven in parallel. Such manual processes may be error- prone and lead to suboptimal solutions, especially for statements of real-world complexity. 1.1 Summary of Example This Example describes the design of a programming framework – a language Ou and a supporting compiler architecture DZKP – enabling programmers to write zero-knowledge statements without thinking about distributed computation. The compiler automatically and optimally chunks this program for efficient parallel proving or verification. (1) The framework provides a C-like programming language with annotations so that developers, including those without deep knowledge of cryptography, can easily write ZK applications even when using protocols with advanced features. (2) Developed herein is a compiler and static analysis pipeline, written in OCaml, that calculates the local and communication costs of each program instruction, then automatically finds an efficient way to chunk the corresponding ZK statement. The optimality measure minimizes the sum of the maximum per-machine cost and cumulative communication costs. This analysis can distribute ZK statements incorporating randomized verification. (3) The system is proved sound, i.e., it is shown that the semantics of the distributed verification of substatements is equivalent to verifying the whole statement centrally. (4) Empirically it is shown that for many statements, including gradient descent and Merkle Tree, the framework described herein can automatically find almost optimal partitioning of the statements, thus achieving speedup proportional to the number of machines available. It is assumed that a program written in the language described herein is sound and secure for its intended ZK application, and is suitably constructed for the target protocol: e.g., it is assumed that the program is data-oblivious if the target is circuit-based. 1.2 Technical Challenges and Solutions Ou and DZKP address a series of challenges arising from each of language design, cryptography, and combinatorial optimization. Cryptographic insight. To enable arbitrary partitioning of a statement, the underlying ZK protocol needs to be ‘flexible’: it must allow proving multiple statements of the same witness while ensuring consistency. To this end, the present framework focuses on the commit-and-prove paradigm. The prover first commits to their private inputs, and then constructs a proof establishing some (public) relationship among the committed values without revealing them to the verifier. Because the inputs are committed the prover can establish multiple statements on them, all while the verifier is certain their values remain consistent. Any ZKP system can support this capability by proving the consistency of commitments in ZK, but certain protocols have direct support which incurs less cost. Given such a ZK backend secure under parallel composition, regardless of its internals, e.g., the use of Fiat-Shamir or random oracle, the proof of any ZK statement can then be distributed in parallel. During compilation, the statement is first partitioned into multiple substatements, before the prover precomputes the input / output of each substatement locally. The prover can then in parallel prove to the verifier the correctness of each substatement and then establish the consistency of the input / output between substatements. Although this works in principle, designing a scalable system that automatically distributes the computation with minimum runtime- and developer-overhead is still challenging. For example, in ZK data dependency rarely constrains statement partitioning, unlike in normal computation or MPC where it does so always. So in ZK even inherently sequential statements can often be parallelized. Automatic statement partitioning. In ZK the prover knows both the public and private inputs, and can thus ‘predict’ all intermediate values. As a result, data dependencies can be resolved before proving: computation of can be partitioned into two parallel verification tasks, . The prover, with inputs x1and x2, just appends t to form an extended witness. For ordinary computation this parallelization would be infeasible, because depends on . The prover’s ability to ‘locally precompute’ creates new opportunities to maximize parallelism. To partition effectively, after parsing and typechecking the present architecture undertakes two distinct special compilation phases: shallow simulation and deep simulation. The first automatically partitions the ZK statement and then selects variables as witnesses (the appendices to the input, like ^) for each of the κ substatements. The parameter κ is user-chosen, and will equal the number of available compute cores in the cluster. Deep simulation then uses the prover’s private inputs to concretely compute the values of those extended witnesses, as well as of any public variables. Shallow simulation proceeds by using live variable analysis to compute the costs of each instruction in the statement, and then models them as a directed acyclic graph for partitioning by either pseudo-Boolean optimization (PBO) or integer linear programming (ILP). This requires loop unrolling and function inlining as flattening techniques. The resultant ZK substatements are then compiled to the chosen backend protocol implementation for distribution as programs to the participating servers: the evaluation herein uses the VOLE-based EMP-toolkit (Xiao Wang, Alex J. Malozemoff, and Jonathan Katz. 2016. EMP-toolkit: Efficient MultiParty Computation Toolkit.). DZKP’s deep simulation then propagates input-dependent values through the statement to concretely compute both public values and the prover’s witnesses for each substatement. To distinguish which values are available during different simulation phases, knowledge levels: K0for (private input-independent) values that are known at compile time, and K1for (private input- dependent) values that are known at distribution time are used. This way, reusing a proof with different inputs only requires recomputing the deep simulation, and not the shallow simulation or partitioning, where the latter is likely to be the most computationally intensive compilation task. Supporting randomized verification. As noted, randomized verification can be much more efficient than deterministic verification. However, supporting it during shallow simulation in particular is a challenge. With deterministic verification an oblivious program trace is known at compile time, which allows DZKP to analyze the cost of each substatement easily. With randomized verification the program might branch on the at-the-time unknown randomness, preventing the cost from being calculated precisely as whether an expensive statement is entered may depend on a coin flip. Also, privacy imparts an ordering requirement on randomized verification: the verifier’s randomness used in checking a ZK statement can only be revealed after the prover commits to its inputs. This means that the partitioning needs to obey sequential ordering for values that are dependent on dynamic choices of the verifier. Ou uses a third knowledge level, K2, to indicate that the value of a variable can only be known at runtime. The type system enforces non-interference with respect to a security lattice, allowing K0- and K1-leveled variables to flow into K2-leveled variables, but not the other way around. The partitioning then treats the K2-leveled variables as one contiguous block in order to estimate worst-case cost and enforce sequential ordering. Handling large statements. Naively implementing the aforementioned techniques does not scale to large ZK statements. If the cost graph contains too many nodes then finding optimal or close-to- optimal chunks will be computationally infeasible. Classical graph partitioning and many related variants are NP-hard with efficient (approximate) optimization algorithms known only in certain special cases, and even then often only theoretically. More subtly, a naive representation of each substatement, e.g., as a circuit or R1CS, is necessarily linear in its running time. For large statements, especially those containing unrolled loops or inlined function calls, it is therefore crucial to work with ‘compact representations’ of substatements which are sublinear in the running time. Ou and DZKP support sublinear representations through atomic, a user-provided annotation to indicate that a function should be condensed, rather than flattened, during shallow simulation. By doing so, we contract the component of the cost graph corresponding to the function into a single node of cost cumulative of its constituent instructions. This reduces the size of the graph and makes optimization more practical. Even if the atomic function is called multiple times, only one copy of the declaration exists. This brings the partitioning problem into the realm of practicality for both PBO and ILP solving, despite its inherent hardness. 1.3 Roadmap Sec.2 is an overview of the DZKP framework’s workflow. Sec.3 presents the language’s syntax, typing rules, and dynamic semantics. Sec.4 shows how the shallow simulation unfolds the program into a sequential program, and formally proves the two programs have the same behavior. Sec. 5 then describes how the compiler partitions the sequential program into multiple chunks and generates a distributed program. Sec.6 shows how the deep simulation evaluates all dependent data between chunks. It also proves that with these data, the distributed program has the same behavior as if it was run without distribution. Finally, Sec.7 studies the effectiveness of the framework for distributing the computations of various benchmarks. 2. DZKP Design Overview FIG. 2 shows the architecture and workflow of the DZKP framework. The user first writes a program in Ou. Upon invocation, the compiler parses and typechecks the program. The Ou type system enforces non-interference over the security lattice given in FIG. 3. This lattice specifies non-interference along two dimensions: i the public or private nature of the information; and ii the knowledge level of the information (i.e., if the information is available at compile time, distribution time or run time) Accordingly, the lattice defines seven security domains: public , committed and authenticated and prover-local . As syntactic sugar, we write pub0 for (pub, K0) and similarly for the rest of the lattice. The variables in the public domain are known by both the prover and the verifier, the variables in the prover-local domain are only known by the prover, and the variables in the authenticated domain are only known by the prover but the verifier holds a commitment to them. Since values known at compile time can only be public, (pvt, K0) and (plc, K0) do not exist. The authenticated and prover-local variables contain information computed from the prover’s secret inputs. Public variables are usually used to guide the program’s execution, either deterministically as iterators or fixed constants, or as verifier-provided randomness. In a valid Ou program every variable is annotated by the security label of a position in the lattice.1For example, in FIG. 1 pub0 annotates the public matrix size constant S that is known at compile time. In comparison, further up the lattice pvt2 = (pvt, K2) annotates w, which depends on both the prover’s committed private values (M2, necessitating pvt) and on the verifier’s public randomness (s, necessitating K2). The security lattice enforces that the private and prover-local domains’ information never flow into the public domain except by explicit lowering through a reveal operation. This non- interference property (Joseph A. Goguen and José Meseguer.1982. Security Policies and Security Models. In 1982 IEEE Symposium on Security and Privacy, Oakland, CA, USA, April 26-28, 1982. IEEE Computer Society, 11–20; Andrei Sabelfeld and Andrew C. Myers. 2003. Language-Based Information-Flow Security. IEEE Journal on Selected Areas in Communications 21, 1 (2003), 5–19.) allows the safe partial execution of the program during compile time (the shallow simulation), as the control flow can be pre-determined. The inclusion of the knowledge levels in the security lattice is essential to guaranteeing that reveal operations and public verifier-randomness are safely handled under the non-interference guarantee. Once typechecking completes shallow simulation commences. Shallow simulation yields1The parser accepts unannotated declarations, which are interpreted as pub0 a sequential program consisting of a flattened sequence of operations through loop unrolling and function inlining. With two exceptions, every instruction in the original Ou program forms its own independent block in this sequential program: the constituent instructions of functions decorated by the atomic keyword are combined into a single block, as are all K2-annotated variables. In both cases, this mapping enforces they be treated as one cumulative ‘instruction’ for the purposes of partitioning. In FIG.1 for example, the suffix of the program beginning with the first mmult_pubvec call on L19 will be condensed, thereby guaranteeing the security of the parallel composition in the presence of public randomness. If the user sets κ = 1, so that only a single partition is desired, then this sequential program will be directly compiled into the ZK statement as specified by the backend. However, usually κ > 1 as the developer desires parallelization, and this is generally assumed to hold through the remainder of this discussion. The DZKP compiler next conducts live variable analysis to track dependencies among the blocks and builds a corresponding directed acyclic graph G = (V , E). An edge (i, j ) ∈ E captures that j-th instruction block depends on some information computed in the i-th instruction block. For example, in FIG. 1 the commit(M) on L37 depends on the mmult_plcmx block on L36. In order to partition optimally, the results of the live variable analysis are used to decorate G with both edge and node labels, each indicating a form of cost to the optimality of any partition. Nodes in ^ are labeled with their computation cost, which models the time required by the prover and verifier to execute the block’s proving or verification operations in the chosen ZKP protocol backend. Edges are labeled with the cutting cost, which models the communication cost required to reconcile the state of the blocks across compute nodes in the cluster. The latter is linear to the operation’s inbound dependent data. It is assumed that cutting and computation costs are directly comparable through some scaling factor (α), but that detail is otherwise omitted as it is irrelevant to the function of the compiler. DZKP next uses combinatorial optimization to find the optimal cuts to partition G. This optimization problem is most easily understood as an ILP, though as the constituent variables are all Booleans it is amenable to representation as a PBO problem as well. The function is formally given for this problem, as well as all of its constraints, in Sec.5.2. Informally, the objective can be understood as the maximum of the cumulative computation costs of each partition, plus the sum of all cutting costs on edges bisected by the κ - 1 cuts (needed to create the user-specified κ chunks). Users can adjust the parameters of the ILP / PBO instances or objective functions to align with different ZK backends and relevant metrics. The partitioned program is compiled into binaries for the chosen ZKP protocol backend. As DZKP is designed to use commit-and-prove ZK protocols as a black box without any intern changes of them, our highly adaptable framework can support a wide range of backends, including the NIZK protocols obtained via the Fiat-Shamir transform (Mihir Bellare and Phillip Rogaway. 1993. Random Oracles are Practical: A Paradigm for Designing Efficient Protocols. In ACM CCS 93, Dorothy E. Denning, Raymond Pyle, Ravi Ganesan, Ravi S. Sandhu, and Victoria Ashby (Eds.). ACM Press.). These binaries are then sent to any and all individual provers and verifiers. All computation up to this point is based only on public K0information, and so the binaries may be freely distributed and reused. However, for the programs to run in parallel, the dependent values between partitions must be computed ahead of time. This is the deep simulation. The prover uses their private information to simulate execution of each substatement program, and saves the output values of each one that then become the inputs to others. These output values are the ‘extended witnesses’ of FIG. 2. For example, in FIG.1 a cut at L12 would make each entry in M an extended witness. After deep simulation completes, the prover can, with any verifier, execute the ZKP protocol by running their binaries in parallel and connecting each binary to the other party’s running copy over the network (i.e., part0 to part0, part1 to part1, and so on up to partκ). Finally, after all the chunks have been checked in parallel, their inputs and outputs must be reconciled to check consistency of variables passed across the cuts. This step prevents the prover from violating soundness by inconsistently computing their extended witnesses, so that, e.g., some variable t = 0 in one partition but t = 1 in another. Therefore the runtime of the proving is not only a function of how large each of the partitioned substatements are, but also of how much state must be communicated amongst the compute cores during this consistency checking step – justifying its inclusion in the optimization objective. DZKP automatically compiles this reconciliation step into the binaries themselves, so it requires no additional expertise on behalf of the verifier. 3. Ou Language 3.1 Language Syntax FIG. 4 demonstrates Ou’s syntax. Note that is used as short-hand for one or more non- terminals separated by commas like and is used to represent zero or more non- terminals likewise. A program consists of a collection of struct declaration (sd), function declaration and external function declaration statements. There must exist a main function acting as the entry point to the entire program. The syntax is very similar to many imperative languages, except for the addition of security levels ℓ to label primitive types and annotations η to label certain functions. Each atomic type τ is annotated with a security level ℓ. Arrays have both a τ indicating their elements’ type, as well as an ℓ indicating their access level. The access level can be viewed as the security level of indexes. A struct’s type only mentions its name, and not its fields, so that we can support recursive struct definitions. There are two different kinds of expressions: L-expressions and R-expressions (e). An L-expression refers to a memory location like a variable ^, an array index , a struct field or a dereferenced pointer . An R-expression represents a value like an integer, float, or Boolean constant; the calling of a builtin function, array constructor, or struct constructor; or the loading of an L-expression or a pointer to an L-expression. When constructing an array the user has to specify its access level. The user can also write normal binary and unary operators, including typecasting following the arrows in FIG.3, and all typical arithmetic operations on integers, floats, and Booleans, but DZKP will automatically convert them into builtin function calls. For example, given a variable x of type pvt1 int and a variable y of type pub0 int, x + y will be automatically rewritten into the expression pvt1_int_add(x, pvt1_int_of_pub0(y)). A command c can be a variable declaration, an assignment, the calling of a user-defined function, conditional branching, a while loop, a sequence of commands c1; c2, a block of commands, returning from a function, or break-ing out of a loop. Note the syntax treats builtin functions and user-defined functions differently. A builtin function can be called in expressions freely, while a user-defined function has to be received immediately by a variable. This distinction is needed because all builtin functions are pure while user-defined functions can have side-effects. As shallow simulation will reorder computations, having side-effects in expressions can alter behavior. Calling user-defined functions freely in any expression is still allowed, but just like in C the programmer cannot assume any execution order as DZKP will automatically rewrite the code to extract these calls. Each function also has an annotation η which enforces restrictions on its body. Most common are normal functions (the parser interprets unannotated functions as normal) which only permit branching on pub0 Boolean expressions. An atomic function is similar, except that shallow simulation will not unfold its body. To branch on a pub1 value, the user must write the code inside a box1 function, while box2 is similar except for additionally allowing branching on pub2 variables. Functions annotated by plocal1 or plocal2 are prover-local functions, which only manipulate plc1 or plc2 values respectively. Functions annotated by box or plocal are called sandboxed functions, as the type system enforces restrictions to isolate their effects. This is further elaborated in Sec. 3.2. Before typechecking, DZKP first scans the program and collects some top-level definitions: maps a struct name s to its definition maps a user-defined function name to its definition or declaration. When is defined, is like internal (η, (τ1x1, ... ,τ1xn) → τ , c) consisting of its annotation, parameter list, return type, and function body; When is external, records only external (η, (τ1, ... , τ1) → τ) ,Note the language does not support global variables, so the prover can only use external functions to communicate her secret with the program. In addition, the language has builtin functions’ typing information Λb, which map a builtin function name to its signature (τ1, ... , τ1) → τ. Builtins do not have an explicit function body c, but in Sec.3.3 we will assign each a math interpretation In order to ensure the security of ZKP programs, it is crucial for users to label witness variables as pvt accurately. Failing to do so can lead to information leakage. DZKP offers provably-sound typing rules that enable users to implement secure Ou programs. By implementing a type checker for these rules, DZKP can identify and report any errors resulting from incorrect usage of security levels or annotations that violate the typing rules. 3.2 Typing Rules FIG. 5 demonstrates the language’s typing rules. Each typing rule relies on the defined struct definitions and functu b ion definitions Λ and Λ . As these are all fixed they are omitted from the written rules for concision. Also relied on is a typing environment Γ, which is a list of scopes γ1... γn. Each scope γimaps variable names to their types. All newly defined variables reside in the first scope γ1. Note that though the compiler allows the same name to be defined in different scopes, for clarity our discussion assumes that names never conflict. Typing orders. We say ℓ1≤ ℓ2if there’s a path from ℓ1to ℓ2in FIG.3. We say a type τ is wellformed if: · it is an atomic type like ℓ int or ℓ bool; · it is struct s such that s is defined in and the types of all its fields are wellformed; or · it is an array of type τ [ℓ n] such that τ is wellformed and ℓ is no greater than any security level in τ. For the last condition the relevant security levels include those of the atomic type as well as the array’s access levels. Wellformedness guarantees the values loaded from arrays are never less secure than the access needed to load them. This restriction is important for both shallow simulation and deep simulation as they need to fully evaluate all K0expressions and K1expressions, respectively. For example, if loads a pub0 integer from array with pvt1 access, then shallow simulation does not have enough knowledge to evaluate e, thus it cannot know which value is loaded. So although type is pub0, the shallow simulation cannot evaluate it. To make shallow and deep simulation behave as expected, it is assumed all types in this Example are wellformed. Typing expressions. For L-expressions, means in the typing environment an L-expression points to a value of type τ, and the L-expression itself has access level ℓ. The access level can help determine whether an L-expression can be fully evaluated during shallow simulation (pub0) or deep simulation (pub1, pvt1 and plc1) or neither (pub2, pvt2 and plc2). Similarly, for R-expressions, means an R-expression e is of type τ in the typing environment Note that in practice the DZKP compiler is more forgiving than the language’s strict typing rules, as it performs implicit type conversion along the security lattice’s arrows. For example, an expression e of type pub0 int can be used as a pvt1 int, and DZKP will automatically wrap it inside a casting function pvt1_int_of_pub0(e). All pointers are also required to have pub0 access level. This restriction is important to the live variable analysis needed for partitioning. Suppose a pointer ^ were allowed to have a pvt1 access level: then shallow simulation could not evaluate the pointer itself in order to determine which location is mutated after running This would mean all locations in the stack could potentially be the address. Similarly, loading a value from would mean any location could potentially be read. Therefore this restriction on pointer access levels is needed to make the live variable analysis yield meaningful results, rather than just concluding by labeling all variables as live. Typing commands. When typechecking a command c which constitutes a function’s body, its return type τris needed to guarantee that each return command in c yields a valid value. A set of branching security levels Lbare also needed to guard all branching expressions inside c. In normal and atomic functions, all branching expressions have to be pub0, but prover- local and boxed functions enforce fewer restrictions. Typing a program. The type checker first checks all struct definitions, and makes sure there are no name collisions between struct names and field names, and then checks each function’s body. For a function definition ^ , the command’s typing rule is applied on its body: and Lbis determined by η: There are additional restrictions placed on sandboxed functions. It is required that plocal1 and box1 functions only modify external data that is K1or K2, and only return K1or K2results. Similarly plocal2 and box2 functions can only modify external data that is K2, and only produce K2results. The reason is similar to the restrictions on arrays’ access levels: shallow simulation cannot execute plocal1 or box1 functions, so it is infeasible to allow them to modify or return any K0data. If they could, simulation of any other computations that rely on such data could not proceed after the function call. It is also restricted that plocal functions can only manipulate plc data. Since all plocal functions only run on the prover’s side they cannot include any pub or pvt values, as their use would require the verifier’s participation. To enforce the first restriction, only the function’s signature needs to be examined: all pointers passed into plocal1 and box1 functions should only point to K1or K2data, while all pointers passed into plocal2 and box2 functions should only point to K2data. The return types have similar restrictions. Since the language does not support global variables, sandboxed functions can only update external data via the pointers in the arguments. To enforce the second restriction, the entire function body needs to be examined to make sure all expressions’ types only have plc annotations, and the body only calls other plocal functions. 3.3 Dynamic Semantics This section defines the dynamic semantics that apply if all secrets are known. The semantics correspond to the program’s behavior in the ideal world where it has enough knowledge to fully evaluate all expressions into values and check if all assertions are satisfied. This semantics is used as a baseline to prove the simulation and distribution are correct. The semantics rules are not surprising. The detailed definitions are listed in Section 9 and shown in FIG.18. R-values and L-values. is used to denote R-values (or “values” for short) and μ to denote L-values. R-values can be atomic values like Vunit and Vint ℓ n, an empty pointer Vnull, a pointer Vref ^, an array Varray where ℓ is the array’s access level and is a mapping from indexes to R-values like , or a struct Vstruct m where m is a mapping from label to R-values like . An L-value is a reference to a memory location: it can be Vvar α that points to an alias, or Vindex μ ℓ i that refers to an array μ’s index i, or Vfield μ l that refers to a struct μ’s field l . An alias is used by the stack to disambiguate during recursion, and can be viewed as a memory address. The program runs on a stack Ω, i.e., a list of frames ω1· ω2···ωn. The last frame ωnstores main’s variables. The stack grows with nested function calls, and the top frame ω1stores the current function call’s local variables. Each frame ω is a map from aliases to values. is used to denote adding a mapping to the top frame. Whenever a new variable x is defined, DZKP allocates a unique alias new(x) for it. This is useful when recursion occurs, as the same variable x might have different instances in the stack, so their values cannot be bound to the same name x. A function find(Ω, x) is also used to find the latest alias allocated for x in the stack. Note that when x is defined for the first time, new(x) returns x directly instead of allocating a new alias for it. Given a stack Ω, a function load(Ω, μ) is used to load the value referred by μ in Ω, and a function store(Ω, μ, ν) is used to compute the updated stack after replacing the value at μ with ν. They both search the entire stack from top frame to bottom instead of working only with the top frame. Semantics. The semantics look up the sets of user-defined functions Λuand builtin functions Λbto interpret function calls. As they are immutable, for concision they are omitted from the rules. Expressions: computes L-expression in stack Ω and yields an L-value, evaluates R-expression e and yields an R-value. Note DZKP assigns a mathematical interpretation for each builtin function which maps a tuple of R-values to an R-value. For example, Commands: (c,Ω1) →c(r,Ω2, β) means that running a command c from a stack Ω1yields a sequence of assertion results β, a result r which is either cont or retν, and a changed stack Ω2. Since the command constitutes a function’s body, retνis used to mean the function returns with ν and cont to mean it has not yet returned. 4. Shallow Simulation Shallow simulation executes and unfolds the program with K0knowledge to obtain a simplified sequential program. DZKP will attempt as much simplification as possible, and output all unknown computations as a sequence of commands for later analysis and partitioning. All normal and atomic functions will be executed, but only the former’s execution history will be output, while the latter will be output as indivisible calls. As sandboxed functions rely on K1or K2knowledge, they will not be executed in shallow simulation and will be kept in the generated sequential program. 4.1 Shallow Semantics This section formalizes the shallow simulation’s behavior as a shallow semantics. FIG.6 shows its details. The actual implementation uses several optimizations to speed up the simulation. Symbolic values. To define the semantics of both shallow simulation and deep simulation, symbolic values are defined to represent R-values not known during the simulation. This is done by redefining the R-value constructor from Sec.3.3 to support SVsym variants, and also by using symbolic stacks, which store symbolic values. Builtin functions. During shallow simulation only a subset of the builtin functions can be interpreted as the rest rely on either K1or K2knowledge. is used to denote a shallow simulateable interpretation of a builtin function. It takes in a list of expressions and yields an expression as the result. For example, we have the interpretation as it relies on K 1knowledge. Semantics. means simplifying an L-expression in a symbolic stack Ω with K0knowledge results in an L-expression . These semantics simplify an L-expression instead of evaluating it directly, as the stack may not contain enough information to fully evaluate it to an L-value. Similarly, simplifies an R-expression. When storing a simplified expression into the stack, a partial function is used to evaluate an L-expression down to an L-value, and a total function is used to evaluate an R- expression down to a symbolic R-value. is able to be total because it replaces all unknown subexpressions by SVsym. When loading a value from the stack, a total function is used to lift a symbolic L-value to an L-expression, and a total function is used to lift a symbolic R-value to an R-expression and concretize any SVsym variables using the appropriate L-value. See FIG.19. Load and store are reused to load and store symbolic values to and from the symbolic stack. When is not completely known, type wellformedness guarantees that the symbolic value referred to by it must also be unknown, i.e., SVsym or an array or struct of unknown values. When so the value to be stored must also be unknown, thus sstore behaves correctly even when is not fully evaluated. Commands: means simplifying a command c in a symbolic stackΩ1results in: a return status r which is either cont or ret e, a new stack Ω2, and a sequence history is used to denote an empty history. Shallow simulation does not have enough knowledge to simulate a sandboxed function, thus it returns a symbolic value generated from the type τ. Also, the typing rule guarantees even if a pointer is passed into the function, the pointed data must be filled with SVsym. So no matter whether the function has side effect or not, the symbolic stack remains unchanged. Note that two pseudo-commands push and pop have been added into the generated program. They help mark the points where a function is called and returned, and their dynamtic semantics are just pushing and poping stacks. Though helpful for the compiler, ignoring them does not change the program’s behavior as the alias mechanism guarantees there will never be name conflicts. They are introduced mainly as machinery to help prove refinement. Atomic function calls. The size of a generated program can be very large as it is linear in the execution time. To reduce the challenge for partitioning, users are allowed to annotate some functions as atomic in order to contract the function call into one command in the generated program. Atomic annotations are optional for programs and are intended solely for optimization purposes. The effectiveness of an atomic annotation depends on the user’s understanding of the program structure. If an atomic annotation is inappropriate, it may increase the compilation time, but it will not compromise the program’s security. Their semantics are very similar to normal function calls, but instead of generating a sequence of history, an atomic function call only yields one command in the history: . Here, a new kind of command is added to encode atomic function calls, but its dynamic semantics are the same as for normal function calls. The two extra sets SRand SWare only used to help perform live variable analysis. SRrecords a set of L-values defined before calling f that may be read during the function call, while SWrecords a set of L-values defined before calling f that be updated during the function call. They are collected during the simulation using functions from Sec.5.1: REF(c) computes all L-values that may be read in command c; DEF(c) computes all L-values that may be updated in c. One convenient fact about shallow simulation is that no pointer dereference will appear in a generated program. It is a corollary of a more essential property: shallow simulation fully evaluates all expressions that only need K0knowledge, including all pointers. In Section 10, normalized expressions are defined to formally describe this property to help prove the correctness of shallow simulation. 4.2 Correctness of Shallow Simulation A refinement function mapping R-values to symbolic R-values is first defined. It replaces all non-pub0 atomic values with SVsym. is overloaded to also represent the function that performs the transformations to convert a stack into a symbolic stack. See FIG.7. A semantics is also defined in order to evaluate return statuses. FIG.7 demonstrates this theorem. Although shallow simulation reorders some computations by simplifying K0expressions and unfolding the program, this theorem guarantees such reordering does not change the behavior. The theorem is proved in Section 10.2. 5. Statement Partitioning The shallow simulation unfolds the original program into a sequential program h = c1; ...; cn. The compiler then tries to find an optimal way to partition these commands into κ chunks so that the distributed running time and the communication time are minimized. Sec.5.1 describes the live variable analysis algorithm that finds data dependencies in the sequential program. Sec.5.2 then describes how DZKP encodes the cut search problem and uses a solver to find the efficient partition. Sec.5.3 then explains how we generate the distributed program from the sequential program, the partition, and the dependencies. 5.1 Live Variable Analysis DZKP performs live variable analysis on a sequential program h = c1; ... ; cn. The goal is to generate a dependency graph DG = (DV, DE) where DV = {1... n} and each edge (i, j) ∈ DE is decorated with a set of L-values DEP(i, j) that may be read in cjand may be manipulated in ci. The term ‘may’ is used rather than ‘will’ because shallow simulation cannot determine all L- expressions’ values. Nonetheless, an over-approximate analysis can still be conducted using some approximation functions. Note DEP(i, j) is only meaningful when i < j , so DEP(i, j) = when i ≥ j. Section 11 defines two utility functions: REF(c) computes the set of L-values that may be read in c, while DEF(c) computes the set of L-values that are newly defined or may be updated in c. Recall that these functions are used in Sec. 4.1. Analysis Algorithm. The program is a sequence of commands h = c1; ... ;cn. Live variable analysis starts from its end and LIVE(n) := It is then scanned from the end to the beginning and unresolved dependent variables are computed for each command ci. LIVE(i) is a mapping from L-values to the commands that may read them in the future. If ciuses an L-value,2then it can safely be deleted from LIVE(i) and that L-value can be added as a dependency between ciand cj. However, cimay read other L-values, so all L-values from REF(ci) need to be marked as unresolved and dependent on ci. Such dependencies will eventually be resolved because every L-value is part of a variable defined in the program, and the command that defines the variable cannot read it. Now the graph DG can be constructed by defining the dependency function: In order to capture all dependencies of and from a command, and are denoted. It follows from the definition that DG is a directed acyclic graph, i.e., i ≥ j ⇒ DEP(i, j) = 0. A notion of similarity between stacks need to be defined to reason about the correctness of live variable analysis and program distribution. Given two stacks Ω1and Ω2and a set of L- values S, it is said Ω1and Ω2are similar up to S, denoted as Ω1≈sΩ2, if (1) ∀μ ∈ S, load(Ω1, μ) = load(Ω2, μ), and (2) the two stacks have similar structures in terms of S, i.e., every alias appearing in S resides in the same frame in both stacks.2In traditional live variable analysis, a live variable is only resolved when it is modified or defined, but here we also resolve it when it is read. This is a compromise to simplify the optimal cut search and to reduce cutting cost in most cases. See Appendix D. Theorem 2 shows the live variable analysis is correct. It says the dependent L-values of cjare enough to safely execute the command and yield all values that are dependent later. It can be proven by observing DEP( j, −) ⊆ DEF(cj) ∪ REF(cj) and DEP(−, j ) = REF(cj), then proving the values in REF(cj) are enough for cjto execute and produce the desired behavior. Theorem 2. If both (c1; … ; cj-1, Ω) →c(cont, Ωj-1, βj-1) and (c1, Ωj-1) →c(cont, Ωj, βj), then 5.2 Optimal Cut Search The final intermediary pass of the compilation partitions the program into κ chunks for parallel execution. Recall that after the flattening during shallow simulation (Sec. 4) and the live variable analysis (Sec.5.1), DZKP obtains a sequential program h = c1; … ; cnand a dependency graph DG = (DV , DE) where each edge (i, j) in DE is decorated with a set of dependent L-values DEP(i, j). The compiler then extracts a directed acyclic graph G = (V , E) from DG capturing the dependency relation and the costs. Each node i in V is decorated with ci’s computation cost PCi. An edge (i, j) ∈ E indicates cjreads at least one L-value used last by ci, and is decorated with the communication cost Cijand a cuttable bit Bij. If there is a K2L-value in DEP(i, j), then ciand cjmust reside in the same chunk, thus Cij= ∞ and Bij= 0; Otherwise, Cijequals to the accumulated cutting cost of all K1L-values in DEP(i, j) and Bij= 1. Combinatorial optimization constrained by ^ is used to find the optimal cutting of h into κ chunks This optimization problem is natively written as an integer linear program (ILP). However, the variables (though not the constants) it uses are all Booleans, and therefore the objective can be interpreted as a pseudo-Boolean function and optimized using pseudo-Boolean optimization (PBO) as well. Initially, the length of the cost of the sequential computation, is defined. Next a pair of matrices is defined. Y ∈ {0, 1}n×n, indicates the location of cuts: Yij= 1 means that a cut is made between the i-th and j-th instruction block. X ∈ {0, 1}n×κindicates the inclusion of instruction blocks into partitions: Xit= 1 indicates the i-th instruction is in the t-th partition for t ∈ [κ]. Finding the cheapest partitioning then reduces to solving the optimization objective of subject to some consistency constraints on X and Y. In order to soundly model this problem for constrained optimization without a min-max objective requires, for PBO in particular, a bit of due care. The present approach is described below: The first constraint enforces that every instruction appears in exactly one partition, the second that only cuttable edges are cut, and the third and fourth together that if the ^-th and ^-th instruction blocks are in different partitions, then any edge (i, j ) must be represented in the appropriate cut by The final four constraints simply enforce the Boolean character of the variables. The fifth constraint is the most nuanced. It enforces that so that the computation in the objective function is correct. This relationship can be encoded much more simply in a general ILP, but is necessary for PBO compatibility. Notice that no constraints force the partitions to be contiguous: even though ℎ is a sequential program, the can have a much more complex structure, at the cost of additional extended witnesses and so more communication during the consistency checking. With this encoding, DZKP hands this optimization problem to either a PBO or ILP solver (the Gurobi PBO solver is used in the present benchmarks). The solver’s output may not always be optimal in practice, as it is limited by the available computational resources. Consequently, the corresponding cut scheme that is employed may also be suboptimal. The compiler then converts the result into a function chunk(-) such that chunk(i) = t if Xit= 1 in the solver’s result. FACT 1. chunk(i) ≠ chunk(j) ⇒ no K2L-value in DEP(i,j) This follows directly from the PBO constraints. 5.3 Distributed Program Given a sequential program h = c1; ... ; cnfrom the shallow simulation, a dependency graph DEP(−, −) from live variable analysis, and a partition chunk(−) from optimal cut search, the compiler generates ^ programs for distributed execution among κ pairs of prover and verifier computation cores. The t -th chunk is a sequential program such that t = chunk(t1) = chunk(t2) = … and t1< t2< …. A communication between the prover and the verifier is denoted by synctj. These communication rounds are used to share K0or authenticate K1data that are needed by ctj, i.e., data pointed by L-values in ∪chunk(i)≠tDEP(i ,tj) as necessary. Deep simulation (Sec.6) computes all this dependent data and distribute it to all of the prover cores. So each pair of prover core and verifier core can run their chunk without the others’ involvement. At the end, all verifier cores will work together to do a consistency check. The goal is to make sure all data provided by the prover cores in synctjare indeed the same as produced by their dependent commands in other chunks. Note the deep simulation only computes K0and K1values, but Fact 1 guarantees all data needed for multipole chunks does not contain K2information, thus the deep simulation is enough. 6. Deep Simulation The deep simulation executes the program with K0and K1knowledge and evaluates all K0and K1values. The goal is to compute all data that are depended between different chunks in the distributed program. It is similar to shallow simulation, except that (i) the definitions of all external functions are known, (ii) the knowledge level is K1instead of K0, and (iii) the purpose is to evaluate all K0and all K1values instead of unfolding the program. 6.1 Deep Semantics This section formalizes deep simulation as a deep semantics. The same symbolic values and stacks are used as in Sec.4.1. and are still used to lift R-values and L-values to R-expressions and L-expressions. But when converting in the reversed direction, and are used instead of and . The only difference is when converting atomic expressions. For example, . dload and dstore are also defined based on these operations just like sload and sstore. The semantics for L-expressions and R-expressions are also similar except for using and to interact with the symbolic stack, allowing dereference expression to be not fully evaluated, and use to interpret builtin functions. is a superset of by adding more functions that can be computed with K1knowledge. For example Section 12 has their detailed definitions. Commands’ semantics is also similar. means running command c from stack Ω1results in a return status r which is either cont or ret e and a new stack Ω2. Note it no longer generates a history as the purpose of the deep simulation is merely computing K0and K1values. Also, it executes normal, atomic, plocal1 and box1 functions while ignores plocal2 and box2. The detailed rules are in Section 12. 6.2 Correctness of Deep Simulation Similar to shallow simulation, a refinement function that converts R-values to symbolic R-values is defined. This time only all values with pub2, pvt2 and plc2 security levels are turned into SVsym. This function is also overridden to apply on stacks. FIG.8 demonstrates this theorem. The proof is very similar to theorem 1 as their semantics are similar except for the deep semantics is a bit simpler. 6.3 Correctness of Distribution Sec.5.3 shows the distributed program, and Sec.6.1 evaluates all dependent data that are needed in the sync steps. It is now proved that these data are sufficient for the distributed program to terminate with the same behavior as if running without distribution. Given a sequential program c1;…; cngenerated by the shallow simulation and an initial stack Ω0, suppose the program runs like (c1, Ωi-1) →c(cont, Ωi, βi) without cutting. The focus is on the t-th chunk and aligning its execution like in FIG.9. Each command cjcorresponds to two stacks If chunk(j) = t, syncjis first run to add external dependent values into then cjis run; Otherwise, the stack remains unchanged. It needs to be proven that the following theorem holds, then theorem 2 ensures that cjterminates with the same behavior. In Section 13, this theorem is proven by first defining a set of L-values that are internally dependent in the t-th chunk after cj, then proving the stack in the uncut execution is always similar to the t-th chunk’s local stack up to this set. So the internal dependency is handled, theorem 3 is then used to handle the external dependency happening in sync, combining them together proves the theorem. 7. Evaluation 7.1 Evaluation Setup and Metrics Three metrics are used to evaluate the effectiveness of the present tool: · Compilation Time is the time used for shallow simulation, data dependency analysis, and optimal cut search. PBO solving during the compilation is done via the Gurobi (L Gurobi Optimization. 2020. Gurobi Optimizer Reference Manual (2020).) PBO solver. The time cost of deep simulation is not included in the compilation time, but deep simulation takes less than 0.5% of the compilation time for all examples. · Effective Ratio of a distributed zero knowledge program is defined as the quotient of the cost of the sequential execution (total costs of all instructions) by the cost of the distributed execution (as the objective function in Sec. 5). When a program is distributed to κ chunks, the closer the effective ratio is to κ, the better the distribution scheme is. It is noted that the optimal effective ratio may be smaller than κ, depending on the statement. · Execution Time refers to the end-to-end performance. Ou code is compiled to a VOLE-based ZK implementation (EMP (Xiao Wang, Alex J. Malozemoff, and Jonathan Katz. 2016. EMP-toolkit: Efficient MultiParty Computation Toolkit.)) and measure the execution time of the resulting distributed programs. VOLE-based ZK backend was selected as one of the motivations is to work with large-scale applications. This choice allows handling of large statements without requiring costly machines with large memory. In addition, the present implementation can be easily extended to support a rich set of ZK protocols, since EMP can be connected to SIEVE-IR (the DARPA SIEVE Program.2022. SIEVE Intermediate Representation). To evaluate the present tool, two programs with different features were focused on. All implementations in the present language Ou can be found in the supplementary material. As the effectiveness of DZKP also depends on the structure of the input program, further provided is a beautiful visualization for the program structure of gradient descent and merkle tree in FIGS. 23A-D. (1) Gradient descent (GD) is an optimization algorithm for finding the optimal model in machine learning. It works by iteratively updating the parameters of a model in the direction of the negative gradient of the cost function concerning those parameters. Gradient decent is implemented for logistic regression with 10 features. (2) Merkle tree (MT) is a type of data structure used to verify the integrity of large amounts of data. It breaks the data into smaller blocks, then recursively hashes each pair of blocks to create a new set of parent nodes using a cryptographic hash function. By proving the correct computation of the Merkle tree, the prover can prove they have the original data without revealing the actual data to the verifier (Ralph C. Merkle. 1987. A Digital Signature Based on a Conventional Encryption Function. In CRYPTO (Lecture Notes in Computer Science, Vol. 293). Springer, 369–378.). 7.2 Evaluation of the Partition Quality First, it is demonstrated that the changes in the effective ratio of the distribution scheme found by DZKP for GD and MT program changes with the size of programs and the number of cuts. GD takes a set {(x, y)} as input, where x is a vector of size M. Throughout this section, |{(x, y)}| is fixed to be 20 and |x| is fixed to be 10. The effective ratio of DZKP is affected by the length of the program being distributed. The length of GD and MT programs are decided by the number of iterations and the number of input blocks, respectively. As shown in FIGS. 10B and 11B, DZKP can find the distribution scheme that achieves the almost optimal effective ratio for large programs for both GD and MT cases. FIGS.10A and 11A demonstrate that the effective ratio will increase with the number of machines rising when the number of partitions is relatively small. Moreover, the results present an upper bound of the effective ratio of the distribution scheme for a given program. For example, the effective ratio for the GD programs involving 80 iterations reaches 40 and stays unchanged regardless of the increasing number of machines. Nevertheless, the results show that our tool can utilize a high degree of parallelization. 7.3 Effectiveness of Atomic Annotation In this section, the impact of annotation to the compilation time and effective ratio is studied. For all examples, the time limit of the PBO solver is set to 10 minutes. In the program for GD, there is a function called update_all(). This function updates the model’s parameter using all data points and is called for each iteration. There is a function called sha256_node() that computes the hash of internal blocks in the MT program. The compilation time and effective ratio of GD programs ( or MT programs) are compared with and without update_all()(or sha256_node()) being atomic while changing ^ and the number of iterations (FIGS.12-13 and 20-22). The results of the evaluation show that the effects of adding atomic annotations depend on the structure of the programs. FIGS. 12 and 13 show a significant rise in the compilation efficiency when adding annotation for both GD and MT programs. The compilation time decreases by 1X to 60X with κ and the number of iterations varying. The price of efficient compilation is a reduction in the effective ratio. The effective ratio for the annotated program is smaller than the unannotated one for GD programs: adding atomic annotation rules out a part of feasible solutions which may contain the optimum. Sometimes, adding annotation could even improve the effective ratio, especially when the constrained time is far less than what is needed to search for an optimal partition. FIG.13 demonstrates an example of this case. The effective ratio is measured while setting the time limit of the PBO solver to 600 seconds. The effective ratio of the MT program for 32-block input with annotation is even better than the one without annotation when the number of cuts increases. One reason behind this result is the MT program structure. The solution found for the annotated program is also optimum for the unannotated one. Furthermore, with a part of the functions being atomic, the search space of the PBO solving essentially shrinks. For additional information, the results of MT programs for various input sizes are included in the sections below. 7.4 End-to-End Running Time The present implementation is end-to-end: it can compile an Ou program to ^ pieces of EMP circuits. Most of the compiler’s implementation is not tied to EMP, so it is flexible to connect with other backends. The running time of the distributed circuits generated from GD and MT is tested herein. The testing machine is an AWS m5.large instance with 2vCPU and 8GB memory. For GD, the number of dataset is fixed to be 10, the size of each dataset is fixed to be 100 and iterate 200 rounds. For MT, the data length is fixed to be 256. Table 1 shows with different number of machines, the slowest part’s running time and the overall speedup comparing to uncut execution time. The effective ratio estimated by the solver is also listed. For GD, the effective ratio approximates the real speedup very well; For MT, the effective ratio becomes inaccurate as each chunk’s runtime reduces. Without wishing to be bound by theory, it is believed that the reason is mainly because the underlying circuits need a constant setup time (around 4 seconds). After subtracting it, the speedup becomes close to the effective ratio. Table 1: End-to-end running time of the framework. 8. Appendix A - Example FIG. 14 shows a code snippet of the Merkle tree used in the benchmark. The prover uses a binary Merkle tree to compute the hash of ^ private data blocks. The merkle function computes the hash of a segment of data blocks. Then the verifier calls verify to verify the hash. When the tree is a leaf node, merkle calls sha256_leaf to fetch a data block and hash it; When the tree is a non-leaf node, merkle recursively computes its two subtrees’ hashes, then use sha256_node to compute the merged hash. Suppose the user wants to cut it into 2 fragments, and there are 4 data blocks. Here sha256_node and sha256_leaf are annotated as atomic, and load_block is annotated as plocal1. So during the shallow simulation, sha256_node and sha256_leaf will not be unfolded, and load_block will be ignored during the shallow simulation and executed during the deep simulation. Shallow simulation executes the program with K0knowledge, yielding a sequence as FIG.15. The recursive function merkle is unfolded. Each variable is annotated with a unique number to distinguish it from others with the same name. Line 1-5 computes the hash of data block 0 and 1, line 6-11 computes the hash of data block 2 and 3, line 13-14 merges them, then line 15 verifies it. Notice in line 4, the recursion condition is a pub0 expression. This is necessary for the unfolding to proceed during the shallow simulation phase. If there is any non-pub0 expression involved in deciding the control flow, then 1) the shallow simulation does not have enough knowledge to evaluate that expression so it cannot decide which branch to take, and 2) even if it uses the prover’s knowledge to evaluate that expression, the information will be leaked to the verifier via the generated programs. Then live variable analysis generates a dependency graph as in FIG.16. The ILP / PBO solver decides to cut it between lines 5 and 6, so the only dependency is hash5. Now the compiler can already generate two programs as in FIG.17. The produce and sync functions are only shown here. Note produce_5 in chunk 1 records a digest of all variables that are produced by FIG. 15’s line 5 for consistency check in the end. Here it is hash5. Similarly, sync_6 records a digest of all variables dependent by FIG.15’s line 6 after assigning values to them. But hash5’s value is not known yet, so it is marked using a question mark. The code can already be distributed to individual verifiers, who will count on the provers to supply encrypted hash5 during the runtime. But the provers need to know these values before interacting with the verifiers. So they perform deep simulation on FIG. 15’s program based on the secret input and find out hash5’s value when reaching line 5. Here the type system again guarantees the deep simulation can proceed. For example, suppose there is any K2value that is dependent between two chunks, and it relies on the verifier’s randomness, then the prover at the compile time cannot guess this value, thus it cannot proceed. In the end, chunk 1 and chunk 2 can run in parallel without any communication. When both chunks’ execution finish, they will run a consistency check together to make sure the hash5 used in chunk 2 is indeed produced by chunk 1’s execution. Here the live variable analysis can be seen and the cut-searching should find the minimal dependency between the two chunks that guarantees the safe execution of both. Suppose the live variable analysis fails to determine that hash5 is the dependency between chunk 1 and chunk 2, then chunk 2’s line 4 will fail. Suppose the cut-searching algorithm considers hash1, hash3 and hash5 are all needed by chunk2, then more data needs to be committed in sync_6, which will slow down the runtime performance. 9. Appendix B - Detailed Semantics 10. Appendix C - Detailed Shallow Simulation 10.1 Shallow Semantics Here are some rules not listed in the main content. Here’s the dynamic semantics of the two new commands: As with the dynamic semantics, it can be shown that command concatenation is associative. The proof is similar to Fact 2. An R-expression e is said to be normalized in a symbolic stack Ω, denoted as rnorm(Ω,e) (or lnorm ), if all its sub-expressions cannot be further simplified.
[0002] These definitions establish another helpful consequence. FACT 5. No pointer dereference will appear in a generated program. 10.2 Proof of Theorem 1 Before the proof itself, some simple lemmas about interacting with the actual stack and the symbolic stack are first presented, and then two further lemmas about the correctness of simplifying R-expressions and L-expressions are shown. The proof follows from structural induction on L-expressions. The two lemmas can be proven together by structural induction on R-expressions and L- expressions. We discuss a few tricky cases here. . Lemma 14 guarantees
[0003] L-expression Suppose . Lemma 14 shows there exists e' such that ۤ Depending on whether e' can be simplified or not, there are two cases. Theorem 1 can finally be proven by structural induction on commands. assert e. Suppose Using lemma 13 we know there exists e' such that The goal is to prove . Both goals can be proven using semantics definitions.
[0004] (1) If , then also μ has non-pub0 information, so the type wellformedness guarantees the value pointed by it can not contain any pub0. So we know is full of SVsym, so Termination. Technically, we cannot do structural induction on while loops as they may not terminate. But since in the hypothesis the command terminates with r and β, we can always find a ranking function on stacks that reduces as the program executes. 11. Appendix D - Detailed Live Variable Analysis Recall that is a partial function as it may fail when the L-expression refers to an array index with a non-K0access level. We can, however, define a total function to compute the smallest L-value with K0access level that includes the result of . Due to Fact 5 we can ignore dereferencing pointers. Let the set prims(^) contain all smallest L-values with K0access level that reside in μ. We use prims(μ) to define two functions and Rread(e) that collect all of the smallest K0L-values from and e’s subexpressions, respectively. We also define Pointed(e) to collect all of the smallest L-values with K0access level that are pointed to by e’s subexpressions. It is used to analyze a sandboxed function’s arguments, as every pointer passed into a sandboxed function is potentially read. We omit its definition as it is near identical to Rread(e) itself.
[0005] Some simple facts about the ≈ relation. In Theorem 2’s setting, Some dependencies may go over c j , i.e., a command after c j may depend on a command before c j . This theorem shows c j does not touch such values. Theorem 15. Using Theorem 2’s setting, if μ ∈ DEP(i, k) where i < j < k, then 11.1 Compromise in analysis Note this section’s discussion assumes the reader has read Sec. 5.2. One part of the optimal cutting problem’s goal is to minimize the sum of all chunk’s external dependencies. However, in many cases, the cutting cost could be smaller. Consider Suppose the cutting cost of one pvt int value is N. If DZKP decides to partition L1 into one chunk, L2 and L3 into another chunk, then the OPB’s encoding considers the cutting cost to be 2N. However, since x is not updated between L2 and L3, L3 can use x’s local value without synchronization. We could have used a more accurate encoding, but that will significantly increase the complexity and slow down the solver. So here we make a compromise by moving the dependency between L1 and L3 to L2 and L3. Now the dependency becomes local, so it no longer contributes to the total cutting cost. We can rigorously prove in most cases, this approach does not reduce the total cutting cost in the encoding: If the program is cut into the total cost does not change (even if there are other commands involved). There is only one case that results in bigger cutting cost: In other words, we will overestimate the total cost. This may not be a bad thing as now our encoding is prone to cut the program without breaking its original structure. Also, if the program has better performance by cutting 2 out, then usually the user can rewrite the program by moving commands that have close data dependencies together. 12. Appendix E - Detailed Deep Simulation Here are some unlisted semantics rules.
[0006] 13. Appendix F - Distribution Proofs We denote chunk-1(t) := {i │ chunk(i) =t}. We let be the set of L-values that are used in [1, j] ∩ chunk-1(t) and needed in [j + 1,n] ∩ chunk-1(t). Theorem 4 can be derived from the following lemma. We prove it together with the following lemma by induction on the sequential program. Subgoal 1. Suppose chunk(j) = t and we want to derive We know as syncjonly updates external dependencies, so fact 7 ensures Also, theorem 3 guarantees the correctness of syncj, i.e., Finally, we can prove then use fact 6 and 8 to derive Subgoal 2. Suppose chunk(j) = t and we want to derive We divide into two parts S1and S2where S1=∪chunk(k)=tDEP(j, k) contains all internal dependencies coming out of j, and contains all internal depen dencies coming before j. Theorem 2 ensures and theorem 15 ensures Ωjand . Notice so we can combine with the hypothesis to derive Finally, we can use fact 6 to derive the goal. The disclosures of each and every patent, patent application, and publication cited herein are hereby incorporated herein by reference in their entirety. While this invention has been disclosed with reference to specific embodiments, it is apparent that other embodiments and variations of this invention may be devised by others skilled in the art without departing from the true spirit and scope of the invention. The appended claims are intended to be construed to include all such embodiments and equivalent variations.
Claims
CLAIMS What is claimed is:
1. A method of parallelizing zero-knowledge (ZK) protocols, the method comprising: providing a ZK framework; calculating local and communication costs for each program instruction of a ZK statement; automatically partitioning the ZK statement according to the calculation; and distributing the partitioned ZK statement to a corresponding number of computing cores for parallel proving or verification.
2. The method of claim 1, wherein the calculating includes determining sums of the maximum per-machine cost and cumulative communication costs.
3. The method of claim 2, wherein the partitioning of the ZK statement includes chunking the ZK statement to provide a lowest or substantially lowest sum of the maximum per-machine cost and cumulative communication costs based upon the calculation.
4. The method of claim 1, wherein the partitioning of the ZK statement is arbitrary.
5. The method of claim 4, wherein the ZK framework permits proving multiple statements of a single witness while ensuring consistency.
6. The method of claim 5, wherein the framework is configured to execute a commit-and- prove paradigm.
7. The method of claim 6, further comprising: a prover committing private inputs; and constructing a proof establishing some public relationship among the committed values; wherein the public relationship among the committed values is established without revealing the committed values to a verifier.
8. The method of claim 7, further comprising establishing multiple statements on the private inputs.
9. The method of claim 1, wherein the partitioning comprises forming at least two substatements.
10. The method of claim 9, further comprising precomputing the input / output of each substatement.
11. The method of claim 10, wherein precomputing comprises forming an extended witness, the extended witness resolving data dependency.
12. The method of claim 1, wherein the partitioning comprises shallow simulation and deep simulation.
13. The method of claim 12, wherein the shallow simulation comprises: the automatic partitioning of the ZK statement; and selecting of variables as witnesses for each substatement.
14. The method of claim 13, wherein the shallow simulation comprises: computing the costs of each instruction in the ZK statement using live variable analysis; and modeling the computed costs as a directed acyclic graph for partitioning.
15. The method of claim 14, wherein the partitioning comprises pseudo-Boolean optimization (PBO) or integer linear programming (ILP).
16. The method of claim 13, wherein the deep simulation comprises: computing extended witness values using a prover’s private inputs; and computing public variable values.
17. The method of claim 16, wherein the computing of the extended witness values and public variable values comprises propagating input-dependent values through the ZK statement.
18. The method of claim 1, wherein the framework comprises annotation tools configured to identify knowledge levels.
19. The method of claim 18, wherein the knowledge levels comprises private input-independent values known at compile time, private input-dependent values that are known at distribution time, and variables that are only known at runtime.
20. The method of claim 19, wherein the private input-independent values and the private input-dependent values are allowed to flow into the variables that are only known at runtime, but the variables that are only known at runtime are not allowed to flow into the private input- independent values and the private input-dependent values.
21. The method of claim 20, wherein the partitioning step includes treating the variables that are only known at runtime as a single contiguous block to estimate worth-case cost and enforce sequential ordering.
22. The method of claim 1, wherein the distributed ZK statement includes randomized verification.
23. An apparatus for parallelizing zero-knowledge (ZK) protocols, the apparatus comprising: a processor; a memory unit; and a communication interface; wherein the processor is connected to the memory unit and the communication interface; and wherein the processor and memory are configured to implement the method of any one of the previous claims.
24. An apparatus for proving zero-knowledge (ZK) protocols, the apparatus comprising: at least two processors; a memory unit; and a communication interface; wherein the processors are connected to the memory unit and the communication interface; and wherein a ZK statement parallelized according to the method of any one of claims 1-22 is distributed among the at least two processors.
25. A non-transitory computer readable storage medium storing computer-executableinstructions for performing the method of any one of claims 1-22.
