Method for implementing a hardware device for performing operations defined by high-level software code

Abstract

The present disclosure provides a method for implementing a hardware device, particularly a hardware device configured on an FPGA or fabricated as an ASIC, which is configured to meet the maximum performance achievable by certain algorithms defined by high-level software code. The method is based on the following steps: translating the high-level software code into a corresponding low-level software code that defines low-level operations to perform the same operations defined by the high-level software code; then estimating certain parameters to calculate the peak performance value P and the memory implementation transfer performance Pm of the hardware device; and finally, implementing the hardware device using hardware resources having performance within the peak performance value P and the memory transfer performance Pm.

Description

Technical Field

[0001] The present disclosure relates to high-level synthesis tools for hardware implementation of algorithms and, more particularly, to a method for implementing a hardware device for performing operations defined by high-level software code. Background Art

[0002] The growing demand for computing performance and energy-efficient solutions in the context of high-performance computing (HPC) and cloud has led to the adoption of heterogeneous systems. Among the hardware accelerators available on the market, field-programmable gate arrays (FPGAs) offer an attractive trade-off between performance and power efficiency on the one hand and, on the other hand, a flexible solution due to their reconfigurable nature. Despite their advantages, one of the main limiting factors for the widespread adoption of FPGA technology is the complexity of their programmability (see [1] D.F. Bacon, R.M. Rabbah, S. Shukla et al., “Fpga programming for the masses.” Commun. ACM, vol. 56, no. 4, pp. 56–63, 2013.). On the other hand, application-specific integrated circuits (ASICs) offer the best power-performance profile at the cost of longer development and design times and the lack of the reconfigurable nature typical of FPGAs. Nowadays, high-level synthesis (HLS) tools are widely adopted to continuously reduce the hardware design time compared to more traditional developments using hardware description languages (HDLs). Starting from a high-level representation of the algorithm (e.g., C / C++), designers can specify multiple optimization instructions to refer to specific parts of the code. These instructions are usually given in the form of pragmas, allowing the definition of high-level architectural choices for implementing the corresponding kernels. Although this approach is very effective in terms of the obtained performance / design-time ratio, HLS tools still require a consistent amount of time to generate the final design. Moreover, the performance of the final design mainly depends on the user's experience in choosing the best set of optimization instructions.

[0003] For this reason, some work has been proposed to automate the design space exploration (DSE) of HLS optimization instructions (see [2] J. Zhao, L. Feng, S. Sinha, W. Zhang, Y. Liang, and B. He, “Comba: A comprehensive model-based analysis framework for high level synthesis of real applications,” in Proceedings of the 36th International Conference on Computer-Aided Design. IEEE Press, 2017, pp. 430–437; [3] G. Zhong, A. Prakash, Y. Liang, T. Mitra, and S. Niar, “Lin-analyzer: a high-level performance analysis tool for fpga-based accelerators,” in Proceedings of the 53rd Annual Design Automation Conference. ACM, 2016, p. 136; [4] G. Zhong, V. Venkataramani, Y. Liang, T. Mitra, and S. Niar, “Design space exploration of multiple loops on fpgas using high level synthesis,” in 2014 IEEE 32nd International Conference on Computer Design (ICCD), Oct 2014, pp. 456–463; [5] J. Cong, P. Wei, C. H. Yu, and P. Zhang, “Automated accelerator generation and optimization with composable, parallel and pipeline architecture,” in 2018 55th ACM / ESDA / IEEE Design Automation Conference (DAC). IEEE, 2018, pp. 1–6.), and multiple methods have been proposed to achieve the best kernel design in a relatively short time.Despite the high quality of the search methods used, such work either does not explicitly consider memory transfers to off-chip memory or assumes that kernel inputs and outputs can be fully cached on-chip. Therefore, when the kernel communicates with off-chip DDR memory, for example through the AXI interface in the SDAccel design flow by Xilinx (see [6] L. Wirbel, “Xilinx sdaccel: a unified development environment for tomorrow’s data center,” Technical Report, The Linley Group Inc, Tech. Rep., 2014.), the inefficiencies in memory transfers can easily jeopardize the effectiveness of optimized instructions. Therefore, it is important to determine where the current performance bottleneck lies before optimizing the kernel itself. The directives provided by HLS tools allow exploring multiple hardware implementations of the software description of an algorithm. Although HLS directives allow improving the hardware design from a computational perspective, in most cases, optimizing memory transfers requires the designer to restructure the code. This limits the effectiveness of DSE methods that target only HLS directives.

[0004] In this context, the roofline model (see [7] S. Williams, A. Waterman, and D. Patterson, “Roofline: An insightful visual performance model for multicore architectures,” Commun. ACM, vol. 52, no. 4, pp. 65–76, Apr. 2009. [Online]. Available: http: / / doi.acm.org / 10.1145 / 1498765.1498785; [8] B. da Silva, A. Braeken, E. H. D’Hollander, and A. Touhafi, “Performance modeling for fpgas: Extending the roofline model with high-level synthesis tools,” Int. J. Reconfig. Comput., vol. 2013, pp. 7:7–7:7, Jan. 2013. [Online]. Available: http: / / dx.doi.org / 10.1155 / 2013 / 428078) can be used to analyze the performance bounds of a particular application running on a specific architecture. Through this analysis model, it can be quickly verified whether the algorithm is memory-bound and whether designers should restructure the code and apply memory transfer optimizations before investigating compute-specific optimization instructions.

[0005] The following surveys related work in terms of the automatic exploration of HLS instructions and roofline model analysis.

[0006] A. Automatic HLS Code Optimization

[0007] Several works in the literature have proposed methods for effectively exploring HLS optimization directives in different settings (see [2]-[5] mentioned above). Specifically, [4] performed several initial HLS runs on a set of sample designs derived from the target code. The HLS resource and performance reports were used as the basis for analyzing a linear model to drive DSE. The analysis model was parameterized with respect to the loop hierarchy of the algorithm, providing an estimate of the application classification rather than the algorithm itself. Additionally, the tool relied on the execution of the underlying HLS tool, which might take a significant amount of time to run depending on the target code. Other methods such as COMBA [2] and Lin-Analyzer [3] do not require an external HLS tool. In particular, Lin-Analyzer utilizes the dynamic data dependence graph obtained from program traces to derive the resource and performance estimates required for DSE. On the other hand, COMBA directly performs static code analysis on the LLVM (see [9] C. Lattner and V. Adve, “Llvm: A compilation framework for lifelong program analysis & transformation,” in Proceedings of the international symposium on Code generation and optimization: feedback-directed and runtime optimization. IEEE Computer Society, 2004, p. 75.) intermediate representation (IR) generated from the Clang front-end (see

[20] C. Lattner, “LLVM and Clang: Next Generation Compiler Technology,” In The BSD Conference, May 2008.). It constructs a hierarchical data flow graph (DFG) of the top-level functions to accelerate and recursively compute resource and performance estimates using the analysis model. Both [2] and [3] considered DSP consumption, but they did not estimate any LUT usage, which might be the most limiting resource in several cases.In addition, neither of the two methods considered the impact of code optimizations applied after loop unrolling, such as instcombine and gvn (see

[10] J. Oppermann, A. Koch, T. Yu, and O. Sinnen, “Domain-specific optimisation for the high-level synthesis of cellml-based simulation accelerators,” in 2015 25th International Conference on Field Programmable Logic and Applications (FPL). IEEE, 2015, pp. 1–7.), which may significantly reduce the code complexity. Thus, although these methods provide reasonably accurate results in estimating the kernel execution time, they tend to overestimate the resource consumption. Finally, different from previous work, [5] considered the impact of off-chip memory transfers. However, it is limited to applications with a load-compute-store paradigm whose working sets can be cached on-chip.

[0008] In our work, we adopt the same model-based COMBA method and add preprocessing optimization steps, including in-built and custom LLVM compiler passes. In this way, we are able to explore the design space more accurately and identify solutions that COMBA considered infeasible due to overestimated resource requirements.

[0009] B. Generalized Roofline Model

[0010] The first work on the FPGA-based architecture adaptation for the Roofline model [7] was [8], where the authors proposed a method for constructing the Roofline model relying on the use of HLS tools. Leveraging the increased operation intensity brought by loop unrolling, users can evaluate the performance and resource usage of multiple unrolled versions through HLS tools and obtain the Roofline model through interpolation. However, this work lacks automation in both calculating the operation intensity and peak performance. These limitations were addressed by (see

[11] S. Muralidharan, K. O’Brien, and C. Lalanne, “A semi-automated tool flow for roofline anaylsis of opencl kernels on accelerators,” in First International Workshop on Heterogeneous High-performance Reconfigurable Computing (H2RC’15), 2015.), which provides a tool for semi-automatically constructing the Roofline model of OpenCL kernels. The peak bandwidth and peak performance are calculated through benchmarking on the target architecture without considering the computation to be accelerated. However, FPGAs and ASICs allow defining hardware accelerators according to the computation to be performed, so the assumption of computation-independent peak performance makes this method less reliable. On the other hand, in (see

[12] J. de Fine Licht, K. F. Larsen, T. Hoefler, and S. Ramos, “Modeling and implementing high performance programs on fpga,” Master’s thesis, University of Copenhagen, Department of Computer Science, 2016.), the calculation of peak performance also takes into account the input algorithm. Combining the data provided in the data sheet, they calculate several peak performance values based on two factors: the ratio of floating-point additions and multiplications required by the kernel and the ratio of DSP and LUT resources used for its implementation. Although this work partially considers the acceleration of the algorithm, these two factors only provide a coarse-grained classification of the algorithm. In addition, the user has to manually specify the usage ratio of DSP and LUT. Summary of the Invention

[0011] To overcome the discussed limitations, we propose a novel method to automatically calculate the maximum performance achievable by a certain algorithm defined by high-level software code, so as to enable a hardware device, especially a hardware device configured on an FPGA or manufactured as an ASIC, to achieve the maximum performance.

[0012] The method according to the present disclosure is defined in the appended claims and is based on the following steps: translating high-level software code into corresponding low-level software code that defines low-level operations to perform the same operations defined by the high-level software code; then estimating certain parameters to calculate at least a peak performance value P of a hardware device that can be fabricated or configured to execute the high-level software code; selecting the hardware device to be fabricated or configured according to the corresponding peak performance value P; and finally, fabricating or configuring the selected hardware device to execute the high-level software code.

[0013] According to one aspect of the method according to the present disclosure, the total number of bytes of data to be transferred to and from the memory of the hardware device when executing the software code and the maximum bandwidth for transferring data to and from the memory are estimated, and then the operation intensity of the low-level software code is calculated. Finally, the memory transfer performance value of the hardware device is calculated as the product of the operation intensity and the maximum bandwidth, and the hardware device to be fabricated or configured is also selected according to the memory transfer performance value.

[0014] According to one aspect, the high-level software code is transformed into low-level software code by using an LLVM front-end compiler, such as the Clang front-end compiler or the Flang front-end compiler (https: / / en.wikipedia.org / wiki / LLVM,https: / / llvm.org / ProjectsWithLLVM / ), and the low-level software code is processed by using an LLVM compiler to estimate the number of runtime occurrences of each operation and the usage of hardware resources, which can be, by way of non-limiting example, block RAM, DSP, flip-flop, look-up table, and region, to generate the low-level software code.

[0015] According to one aspect, the hardware device is configured on an FPGA or fabricated as an ASIC, and is implemented by: performing a design space exploration algorithm on the low-level software code based on the peak performance value and the memory transfer performance value to be satisfied to identify a suitable FPGA configuration or ASIC. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a block diagram illustrating one aspect of the method according to the present disclosure.

[0017] Figure 2 shows Pareto charts of designs explored by DSE on two PolyBench test cases: a) Gesummv test case; b) Mm test case.

[0018] Figure 3 is a roofline model diagram representing the optimization process for the N-body algorithm. Detailed implementation manners

[0019] The present disclosure provides a method for implementing a hardware device, particularly a hardware device configured on an FPGA or fabricated as an ASIC, to achieve the maximum performance achievable by a specific algorithm defined by high-level software code.

[0020] A comprehensive method is also provided to support designers in generating an optimal HLS-based hardware implementation.

[0021] First, we propose to automatically generate a roofline model starting from the high-level software code description of the target algorithm. This method can quickly evaluate the operation intensity of the target function and visualize the main bottlenecks of the current HLS implementation, providing guidance on how to improve it. Second, we propose a DSE method for quickly evaluating different HLS instructions to determine the optimal implementation. We evaluated the DSE in multiple test benchmarks from the PolyBench test suite and obtained a performance improvement up to 14.36 times compared to previous automated solutions in the literature. Finally, we validated the overall methodology for the N-body physics simulation algorithm and obtained results comparable to state-of-the-art custom implementations.

[0022] Given these motivations, we propose a comprehensive method that combines roofline model analysis with automatic DSE to select a set of optimal optimization instructions. The novel contributions of this work are:

[0023] - An automatic method for generating roofline model analysis for a given target hardware device (i.e., FPGA or ASIC technology) and the high-level software code of an algorithm to help designers evaluate performance bottlenecks and off-chip memory transfer optimizations,

[0024] and allowing the selection of a suitable set of hardware devices within the hardware device according to the given target function,

[0025] - An analysis model for predicting the application peak performance boundary while considering the application to be accelerated and the underlying target hardware device,

[0026] - An automatic DSE of HLS optimization instructions driven by fine-grained estimation, allowing full utilization of the available hardware resources.

[0027] Part I outlines the proposed method and shows how application designers can benefit from it. Part II illustrates our method for obtaining performance and resource estimates from a given high-level software implementation. Part III reviews the roofline model analysis and discusses how we automatically generate and adapt it to an FPGA-based architecture or a given ASIC technology. Part IV presents the DSE for identifying the optimal HLS instructions. Finally, Part V evaluates the proposed method.

[0028] I. Overview of the proposed method

[0029]

[0030] List 1: Optimization directives aimed at specifying a particular design configuration

[0031] The different steps of our method are detailed in Figure 1 This method takes as input the target hardware and the target high-level software code (such as C / C++) which is rich in optimization directives similar to those in Vivado HLS (see

[13] Xilinx Inc., “Vivado HLS.” [Online]. Available: https: / / www.xilinx.com / products / design-tools / vivado / integration / esl-design.html). We support specifying pragmas for loop unrolling, loop pipelining, loop flattening, array partitioning, and memory transfer through the AXI protocol interface using the format reported in List 1. Similar to other methods in the literature [2], we require that all loop bounds in the target function are known. This is needed by DSE to estimate the kernel latency and by the roofline model generation to automatically derive the peak performance and the operational intensity.

[0032] The first step performed by our method involves generating LLVM IR from the high-level software input code (such as C and C++). Subsequently, we analyze the LLVM IR to provide initial performance and resource estimates. Performance is estimated by scheduling the kernel function and computing the latency of the critical path, while resource usage is estimated considering the resource utilization of the operations within the schedule. To support this phase, our method provides a performance and resource modeling library, abbreviated as the analysis library in the rest of this description, which contains operations reported by synthesis tools for performance (latency, timing) and resource usage (such as BRAM, DSP, FF, LUT, and area) for implementing the target hardware for different frequency syntheses.

[0033] Examples of synthesis tools include Vivado HLS / Vivado provided by Xilinx for implementing FPGA configurations, Catapult HLS provided by Mentor, and the design compiler adopted by Synopsys for ASICs for a given technology.

[0034] In the next step, the workflow automatically generates a roofline model analysis and visualizes it to a web-based user interface, where the performance bounds, operation intensity, and estimated performance of the initial design are shown in the same graph. At this point, the user can restructure the code and attempt to overcome any memory bottlenecks identified by the roofline model analysis, or continue with the automatic DSE of the optimization directives. In the case of performing DSE, the estimated performance of the best feasible design produced by DSE will be directly plotted into the roofline model graph. In this way, the user can have a comprehensive understanding of the current design configuration, whether it is the potentially achievable performance limited by peak performance or bandwidth, or the actual performance of the solution derived from DSE. Finally, the user can also revise the code or specify manual optimization directives. The generated code can be used as a baseline version for further iteration, where the roofline model analysis and DSE are performed. Overall, the proposed method allows the user to iteratively converge to the best design while taking into account the performance and memory bottlenecks of the analytical solution. In the following sections, we will discuss the various stages of our method in detail.

[0035] II. Performance and Resource Estimation

[0036] In this section, we will present the lightweight resource and performance estimation performed on the LLVM IR of the target code. The whole process relies on both the built-in LLVM passes and the custom passes required to improve the accuracy of the estimation. In particular, we first apply the standard O1-level optimization from LLVM, which is also common in commercial high-level synthesis tools. Subsequently, we apply the following custom passes: loop unrolling marked by the loop unrolling optimization directive, tree balancing of associative operators, analysis and propagation of constant memory accesses, range analysis of type operations, instruction redundancy elimination, and multiply-add resource mapping operations. In addition, we apply additional built-in LLVM passes (such as instcombine and gvn) that can further improve the code quality after the custom passes.

[0037] After code optimization, the function is scheduled to estimate its overall latency. This method calculates the arrival time of each basic block, recursively considering the arrival time of its predecessor tasks and the latency of the basic block itself. If the predecessor task is a complex structure, such as a loop, its latency is calculated by recursively applying the same method to its basic blocks. The latency of the kernel function is obtained from the arrival time of the basic block that returns from the function call. Additionally, the latency of a loop is obtained from the latency of the loop body according to specific optimization instructions. Since loop unrolling is directly applied as a code transformation, we only need to distinguish between pipelined and non-pipelined loops. If pipelining is not applied, the loop latency is calculated as the product of the iteration delay IL and the number of iterations N. For a pipelined loop, the loop latency is calculated as IL + II * (N - 1), where II is the initiation interval of the loop. To speed up the search, we approximate II as the minimum initiation interval (see

[14] J.M. Codina, J.Llosa, and A. Gonzalez, “A comparative study of modulo scheduling techniques,” in Proceedings of the 16th international conference on Supercomputing. ACM, 2002, pp. 97–106.), without running a time-consuming modulo scheduling algorithm (see

[14] ;

[15] L. de Souza Rosa, C. Bouganis, and V. Bonato, “Scaling up modulo scheduling for high-level synthesis,” IEEE Transactions on Computer Aided Design of Integrated Circuits and Systems, pp. 1–1, 2018;

[16] J. Cong and Z. Zhang, “An efficient and versatile scheduling algorithm based on sdc formulation,” in 2006 43rd ACM / IEEE Design Automation Conference. IEEE, 2006, pp. 433–438.). We calculate the minimum initiation interval

[14] as MII = max(RecMII, ResMII), where RecMII and ResMII are the contributions of the minimum initiation interval with respect to recurrence constraints and resource-constrained minimum initiation interval. RecMII is the constraint on the initiation interval given by dependencies carried by the loop, such as accumulation. It is calculated as RecMII = max i[delay i / distance i , where delay i is the clock cycle delay along the dependence path of the i-th dependence, and distance i is the distance through the i-th repetition of the loop iteration. The path considered for each dependence is the path from the target operation to the source operation through the back edge. In contrast, ResMII is a constraint on the resource usage of a finite functional unit. Typically, it is calculated as ResMII = max i [numOps i / numFUs i , where numOps i is the number of operands of type i in the loop, and numFUs i is the number of available functional units of type i. Since in our use case, the restricted operation is the operation of accessing memory, and since the memory can be partitioned, the formula can be rewritten as ResMII = max i [numAccs i / numPorts i , where numAccs i is the number of accesses on the i-th partition, and numPorts i is the number of ports available for accessing the i-th partition. The number of iterations N of a particular loop execution is equal to its trip count, which is obtained from the LLVM scalar evolution analysis applied to the loop induction variable. However, to account for the loop flattening optimization of perfectly nested loops containing pipelined loops, we corrected the distribution of loop iterations. In particular, for any perfect loop nest with a pipelined innermost loop, the number of iterations executed by the pipelined loop is equal to the product of the trip counts of all loops in the loop nest, while the other loops in the loop nest are assigned an iteration count equal to 1. By this method, we simulated the collapse of the loop nest into a single loop, which occurs when performing loop flattening.

[0038] In addition to the optimizations applied to the loop, the single-iteration latency is obtained from the arrival time of its latch block, considering that the head block has no preceding tasks. Finally, the latency of a single basic block is calculated using an adapted ALAP version of the resource-constrained list scheduling algorithm, which includes instruction chaining. This algorithm imposes a restriction on instructions accessing the same memory port in the same clock cycle. It takes into account the access to the partitioned array, modeled by instantiating a queue for each partition, and the access to the external AXI interface. Each queue has an index that identifies the partition it models as equal to (baseAddr, id1, id2,..., id n-1 ), where baseAddr is the base address of the n-dimensional array being accessed, and idi Is the partition index associated with each dimension i. Given the index index for each dimension i i In the case of a memory access, the corresponding queue being accessed is determined as follows:

[0039] -id i =[index i / [size i / f i ]], in the case of block partitioning

[0040] -id i =(index i )mod(f i ), in the case of loop partitioning

[0041] -id i =index i , in the case of complete partition

[0042] Where size i is the size of dimension i, f i is the partitioning factor for dimension i. Once the kernel is scheduled, the total resource consumption of each resource type is estimated specific to the target hardware device. In our model, we consider the number of BRAM banks used to allocate a specific array to be equal to 2 [log2(partitionSize / capacity)] *partitions where partitionSize is the size of each partition in bits, capacity is the capacity of the BRAM bank, and partitions is the total number of partitions that make up the array. The resource consumption given by the operator is taken from the analysis library, while taking into account resource sharing between instances of the same operator that start computation at different clock cycles. Finally, due to the multiplexing and routing logic introduced by resource sharing, we also model the use of LUTs and FFs. The following is a more detailed explanation of the resource estimation:

[0043] 1) Expressions: Expression instances that form a certain algorithm consume resources such as DSPs, FFs, LUTs, etc. Shared operators executed in different clock cycles can be reused, saving resources but increasing routing complexity. For this reason, we only allow sharing of two expressions using at least one DSP (usually a very critical resource) and complex operations with a latency greater than or equal to 1. Since we assume that shareable operators are fully pipelined, the number of instances required for a single operator in a particular basic block depends on the maximum number of instances of that particular operator that start computing in the same clock cycle. Without applying reuse, combining this data with the number of operators to be instantiated enables us to calculate the reuse rate of a particular operator. In contrast, the number of instances required for non-shareable operators depends only on the number of operators of that type used in the application. Once we have obtained the number of instances required for the application, we multiply this number by the estimated resource usage per single instance given by the analysis library.

[0044] 2) Partial result storage: Partial results computed across clock cycles are stored in registers (FFs). Therefore, considering that instructions cannot be chained, we estimate the usage of FFs since results must be stored for further computation. The FF usage for a single instruction can be calculated by multiplying the number of these instructions by the bit-width of the operation and weighting it with the reuse factor of that operator.

[0045] 3) Shared resource routing: Shared resources require the use of multiplexers to provide the correct operands for the instances. Since multiplexers are implemented with LUTs, we calculate the number of LUTs required to implement a multiplexer with a given number of inputs, and thus we can calculate the LUT usage per instance, assuming that for each operand, a multiplexer with the number of inputs given by the reuse factor is used.

[0046] III. Automatic Roofline Model Analysis

[0047] In this section, we will discuss how to apply the classical roofline model [7] to FPGA / ASIC, and how we can automatically calculate the roofline model starting from the given code and target device. In the classical roofline model for CPUs, both peak bandwidth and peak performance depend on the architecture, while the operational intensity is a characteristic of the algorithm. However, when considering FPGAs or ASICs, it is inaccurate to provide a peak performance independent of the target computation for acceleration. In fact, different operations implemented in hardware require different resource consumptions. Therefore, the number of processing elements that can be configured in parallel and the peak performance depend to a large extent on the operations to be executed. For this reason, we propose a method for calculating peak performance that takes into account the algorithm and the target FPGA or ASIC technology. Additionally, the roofline model can be automatically calculated through a custom LLVM pass that works on the IR of high-level software code.

[0048] Although fixed architectures such as CPUs have performance metrics associated with the architecture itself (e.g., GFLOP, giga floating-point operations per second), reconfigurable architectures such as FPGAs or ASICs better associate their performance metrics with the algorithm to be synthesized (since once the algorithm is synthesized, it itself defines the target architecture). For example, considering an image processing algorithm, a meaningful performance metric might be the number of images processed per second by the hardware implementation. Therefore, we allow users to specify preferred performance metrics in two formats:

[0049] - Define a set of operators to be measured (such as GFLOP);

[0050] - Insert custom instructions at the code location where the result is generated to define the throughput metric (results / second) in this way.

[0051] In fact, the throughput metric is specified by inserting the custom instruction _result_generation_() next to the instruction responsible for the calculation. We intend to use the string identifying a single LLVM instruction (add, fmul,...) as the operator (or simply op), although for a function call (llvm::CallInst instruction), it should be the name of the called function. Regardless of the performance metric format, we need to define the subset of operators that the user is interested in. Therefore, we start by defining the set OP of operators that make up the code, which we will name as the set of operators that the user is interested in. In the case of considering the first format, the user directly defines the set SOP of operators to be measured. In the case of considering the second format, the set SOP = {op r}, where op r is the single operator that identifies the function call _result_generation_() required to specify the throughput. During the implementation of the algorithm, we define the (mapping) data structure OP map whose index is an element of the set OP and the index entry is the number of executions of the relevant index operator during the execution of the algorithm calculated by static code analysis. We fill the OPmap of the analysis code through the LLVM framework. Specifically, the analysis starts from the LLVM IR of the kernel function, passes through each basic block bb, and then through each included instruction i in turn.

[0052] For each basic block bb ∈ kernelfunction, considering the loop nesting in which the basic block is located, the analysis calculates the number of times e bb that the basic block is executed. Since it is assumed that the loop boundaries are known at compile time, we can use List 2 to calculate e bb .

[0053]

[0054] List 2: C++ code of the computeExecutionTimes function required to calculate the execution times of a specific basic block bb

[0055] For each instruction i ∈ bb, analyze:

[0056] - Compute the relevant operator op i (as described above),

[0057] - Check whether the operator op i is already an index of a specific value in OP map or not.

[0058] - If so, increment the index value by e bb where bb is the basic block containing the instruction,

[0059] - Otherwise, insert op i into OP map and initialize the index value to e bb where bb is the basic block containing the instruction.

[0060] In a similar way, we also count the total number of memory accesses of the function arguments mapped to, but not limited to, the AXI master / slave interfaces, and obtain the total number of bytes tb accessed. For a specific argument arg of the user-specified function f mapped to the AXI master / slave interface, the instruction _axi_master_(arg, bundle) is inserted into the function f. We calculate the number of bytes transferred considering all llvm::LoadInst and llvm::StoreInst whose base addresses correspond to the specific argument mapped to the AXI master / slave interface. Specifically, initialize the variable tb to zero, and for each basic block bb ∈ kernelfunction, analyze and calculate the number of times e the basic block is executed bb considering the loop nesting where the basic block is located. Since we assume that the loop boundaries are known at compile time, we can use List 2 to calculate e bb . For each instruction i ∈ bb, analyze:

[0061] - Check whether the instruction i is an llvm::LoadInst or llvm::StoreInst operation

[0062] - If so, check whether the base address is the function argument arg1 of the function f

[0063] - If so, check whether there is a function call instruction in the form of _axi_master_(arg, bundle) inside f, regardless of the value of the bundle that is not relevant here.

[0064] - If so, retrieve the number of load or store bytes tb of instruction i, considering the type of load and store, and increment the value of tb by a factor of tb. i of tb i increment the value of tb.

[0065] The analyzed tb value represents the total number of bytes transferred in and out of off-chip memory during kernel execution. After calculating o op and tb, we can automatically calculate the operation intensity as follows:

[0066]

[0067] The estimation of peak performance is calculated by assuming an ideal hardware implementation of the algorithm on the target hardware, with no data dependencies between algorithm operations. For each operation (e.g., floating-point summation), we associate the corresponding operator physically implemented on the hardware (e.g., floating-point adder). We assume that these operators are fully pipelined and operate at the target clock frequency f. Thus, in each clock cycle, the operator can produce a valid result. We have collected in the analysis library the performance and resource usage estimations of several operators synthesized (offline) at a specific frequency on the target hardware, specifically for a given FPGA or ASIC technology. Specifically, for each target hardware and frequency, we have built an instance of the analysis library associated with several operators that appear in the code, so that the framework can load the correct library instance according to the target frequency and target hardware under consideration, providing for each operator the latency (the number of clock cycles required for the operator to complete), the timing (the maximum combinational time required for the slowest completion of the operator), and the amount of resources required to implement the operator (such as BRAM, DSP, FF, LUT, and area). We build the analysis library through the HLS tool (the same tool that finally generates the final design of our method), synthesizing each instruction to form the library. Specifically, in the case of a specific instruction in the form of a = f(op1, op2,..., op n )), we rely on template functions to build the kernel that is further synthesized by the HLS tool to provide the required estimations.

[0068] This template is the string kernel in List 3, representing a Python function for implementing an instance of the kernel function, given the instruction to be synthesized and the input and output types of the instruction itself (other parameters are not relevant here).

[0069]

[0070]

[0071] List 3: The writeCppKernelPython function for generating a simple C++ HLS kernel consisting of a single instruction to be documented

[0072] Once the kernel is synthesized, the tool reports the required resources, latency, and timing of the instructions of interest. Therefore, once the analysis library with the target frequency and architecture corresponding to the current settings is loaded, the tool can start calculating the peak performance. Ideally, the implementation with the highest performance on the target hardware requires instantiating each operator the same number of times as the corresponding operation count o op as many. This implementation requires cr resources of each resource type r ∈ R = {BRAM, DSP, FF, LUT, area}:

[0073]

[0074] where u op,r is the number of resources of type r ∈ R required by operator op ∈ OP, which is provided by the just-loaded analysis library. It is very likely that the required resources will far exceed the availability of the hardware resources. Therefore, we calculate the scaling factor to account for the reduction in resources required to fit within the target FPGA or given ASIC resource constraints (such as the maximum area):

[0075]

[0076] where av r is the amount of resources of type type r ∈ R available on the target hardware. Compared to the ideal implementation, the actual number of operators of type op ∈ OP implemented in the design i op is scaled by the factor SF:

[0077]

[0078] And the peak performance bound (in terms of the total number of operations per second) is calculated as follows:

[0079]

[0080] This formula takes into account the fact that each operator can produce useful results in each clock cycle (pipelined operator), and the operator is never idle (no data dependencies). Finally, in the roofline model diagram, we empirically consider 80% of P as the peak performance to account for routing congestion and timing convergence issues. Regarding the peak bandwidth boundary, we ran a set of microbenchmarks on the target FPGA or considered bandwidth estimation when targeting an ASIC with a given off-chip memory and memory interconnect. Considering different configurations of the communication interface between the accelerator and the off-chip memory, microbenchmarks were executed or estimations were made. In particular, we considered the following combinations: the number of DMA channels used, the bit width of the AXI interface, and the burst length. In this way, the appropriate peak bandwidth boundary can be derived according to the HLS interface configuration of the given code. Finally, based on the given peak bandwidth (bw) used to transfer data to and from the memory of the hardware device, we can calculate the memory transfer performance boundary as: Pm = bw * CI.

[0081] IV. Design Space Exploration

[0082] Several hardware design configurations are explored iteratively by evaluating the performance provided by a specific optimized instruction set using the method described in Part II, so as to correspondingly select the next instruction configuration. Throughout the iteration process, the instruction configurations are saved in a stack, so that if a specific configuration is infeasible due to resource limitations, the corresponding loop that exceeds the available resources is marked as fully optimized and not considered for subsequent optimizations. Then, the exploration backtracks to the previous feasible configuration and continues to look for other optimization opportunities in other loops within the code.

[0083] Perform each iteration:

[0084] - Insert optimized instructions into the LLVM IR to be analyzed;

[0085] - Estimate the performance and resource usage of the current configuration;

[0086] - Select the optimized instructions to be applied in the next iteration.

[0087] Selecting the next configuration to study is performed by a controller that alternates between loop optimizations (through loop unrolling, pipelining, and flattening) and local memory optimizations (through array partitioning). Regarding loop optimizations, at each step, we recursively visit loops with higher latencies contained within the function body or loop until the innermost loop l is identified. If loop l is not already pipelined, we first attempt to pipeline it. This choice relies on the fact that pipelining a loop does not increase the resource usage as much as unrolling, thus keeping the configuration feasible and the analysis faster. However, if l is already pipelined, the DSE attempts to achieve higher performance by further unrolling the loop. Each subsequent unroll optimization of loop l is done by selecting the next higher unroll factor, which divides the loop's trip count.

[0088] After each loop optimization, global memory optimizations are performed. In fact, after pipelining or unrolling the loop, the performance may now be limited by the number of available memory ports. In this step, for each array a in the function and each dimension d of array a, we collect the number k of different offsets used to access dimension d. Then, we divide dimension d by the factor k. The choice between loop partitioning and block partitioning is done by simulating the number of conflicts that occur within the partition. The partition type that minimizes the maximum number of conflicts per partition is selected. Note that if block partitioning or loop partitioning is applied and the factor k is equal to the number of elements in dimension d, a full partition is automatically performed. Finally, the exploration continues with loop optimization steps and terminates when there are no more loops to explore because they are all fully unrolled or marked as fully optimized.

[0089] V. Experimental Evaluation

[0090] We evaluate our DSE for HLS optimization instructions on the PolyBench test suite (see

[17] L.-N. Pouchet, “Polybench: The polyhedral benchmark suite,” URL: http: / / www.cs.ucla.edu / pouchet / software / polybench, 2012.) and validate the whole method in the N-body physics simulation case study. The experiments are performed on an Intel Xeon E5-2680 v2, while we use Vivado HLS 2018.2 as a reference for resource and performance estimation comparison.

[0091] A. PolyBench Test Suite

[0092] The PolyBench test suite

[17] is a set of benchmarks that includes multiple kernels with different loop nests. Through these benchmarks, we evaluate the DSE efficiency and estimation accuracy of our method and compare our results with [2]. To reproduce the settings of [2], we target the Xilinx Virtex-7 XC7V2000T-FLG1925 FPGA device running at 100 MHz. Additionally, the latency and area results of [2] were obtained by applying the best optimization directives reported in their work and running Vivado HLS 2018.2.

[0093] Table I summarizes the results obtained through DSE.

[0094]

[0095] Table I: Results of our DSE on the PolyBench test suite for HLS optimization directives.

[0096] Specifically, we compare the execution time of DSE on different test cases with the execution time required to run the same DSE method but relying on Vivado HLS for performance and resource estimation. This table reports the size of the solution space (the possible combinations of optimization directives for a given benchmark) and the number of designs required for our DSE to converge to the final solution. As we can see, compared to the HLS-based solution, our DSE requires 8 to 28 times less time while achieving the same final configuration. Additionally, DSE effectively explores a small number of interesting optimization configurations from a larger design space.

[0097] Table I also reports the improvements over the baseline unoptimized HLS implementation and the results obtained by [2] in reducing the overall kernel latency. Additionally, the table shows the area utilization of our final design and the design from [2] (considering the most constrained resource calculation). Overall, compared to the best configuration found by [2], we can observe an average latency reduction of 7.53 times, achieved in less than a minute. The main reason for the improved results comes from the additional code transformations applied during the resource and performance estimation phases and the strategy adopted by the DSE controller. Since neither [2] nor our DSE explores optimizations that lead to exceeding the available resources, our accurate prediction can prevent premature stopping of the search, thus driving the exploration of solutions and averaging 65.83% utilization of the device area, more than 2.03 times more than [2]. On the other hand, alternate loop optimization and array partitioning optimization allow the DSE controller to fine-tune the partitioning factor and type based on array accesses.

[0098] Table II reports the best configurations found by our DSE and illustrates how our method identifies coherent array partitions through applied loop optimizations.

[0099]

[0100] Table II: Best optimized instruction configurations achieved by the proposed DSE on the PolyBench test suite.

[0101] Figure 2 The DSE process for two test cases, Gesummv and Mm, in terms of latency and area usage is described. We focused on these test cases which require the lowest and highest number of DSE iterations respectively to converge. The first test case (Gesummv) explores the design space with a performance prediction error of 1.90% and a resource utilization rate of 1.24%, while the second test case (Mm) offers a performance prediction error of 4.08% and a resource utilization rate of 5.60%. For each graph, the curves obtained by inserting all the designs show how the DSE controller alternates between loop pipelining (steep feature) and loop unrolling (linear feature) and converges to a region utilization rate of 75.56% and 80% respectively.

[0102] B. N-body simulation test case

[0103] To validate the overall method, we considered the N-body physical simulation, which is a practical application in the field of astrophysics. We started the optimization process from the basic memory-bound implementation of the algorithm, and with our method, we achieved a final version with performance comparable to (see

[18] E. Del Sozzo, M. Rabozzi, L. Di Tucci, D. Sciuto, and M. D. Santambrogio, “Ascalable fpga design for cloud n-body simulation,” in 2018 IEEE 29th International Conference on Application-specific Systems, Architectures and Processors (ASAP). IEEE, 2018, pp. 1–8.) but following an automated approach. For this test case, we targeted the Xilinx Virtex UltraScale+ VU9P FPGA board on Amazon Web Service (AWS) with a target design frequency of 126 MHz.

[0104] Before starting the optimization process, we need to specify the throughput metric. This step is performed by inserting a custom instruction in the code, where the results of interest (in this case the body comparison) are produced. We now analyze the plain version of the algorithm with our method, reporting Figure 3 all the steps in. The plotted roofline analysis clearly visualizes that the initial design is memory-bound ( Figure 3 the green dashed line in). In fact, the operation intensity reported by the chart is 2.15e-2 pairs / B, far below the ridge point of 1.35 pairs / B. Moreover, our method estimates the performance of the initial implementation as 2.52e+6 pairs / second ( Figure 3 the red triangle on the green dashed line in). However, for the current operation intensity and available memory bandwidth (14.5 GB / s, 1 DDR port), due to the memory transfer limitation (the accuracy of the peak bandwidth estimation widely discussed in the benchmark [8]

[11]

[19] L. Di Tucci, K. O’Brien, M. Blott, and M. D. Santambrogio, “Architectural optimizations for highperformance and energy efficient smith-waterman implementation on fpgas usingopencl,” in Design,Automation&Test in Europe Conference&Exhibition (DATE), 2017. IEEE, 2017, pp. 716–721.), the roofline model reports an achievable performance of 3.12e+8 pairs / second. This means that the DSE can only find designs with performance lower than 3.12e+8 pairs / second, which in turn is definitely lower than the estimated peak performance of 1.57e+10 pairs / second that can be achieved within the computational bound region. For this reason, we try to increase the operation intensity to cross the ridge point. In particular, we consider caching data in local memory to increase the operation intensity. We rewrite the kernel, adding an initial phase to copy the data into local memory before the computation, and a final phase to copy the data back to the host after the computation.

[0105] Analyzing the cached version of the algorithm, we verify that the BRAM resource constraint is met and the operation intensity grows to 6.77e+2 pairs / B ( Figure 3 the blue dashed line in), reaching the computational bound region. According to the roofline model, with the newer implementation, we expect to find designs whose performance is limited by the peak performance (1.57e+10 pairs / second). By running the automated DSE, we obtain an estimated performance of 1.51e+10 pairs / second ( Figure 3The best design (the red triangle on the blue dashed line) has a region utilization rate of 75.57%. The estimated performance of this solution is only 3.7%, which is lower than the theoretical boundary. In addition, the estimated report of the final design shows a latency estimation error of 0.000298%. Finally, we tested the obtained design on the board and measured 1.207e+10 pairs / second, which is about 20% lower than the DSE and Vivado HLS estimates due to memory transfer overhead. Regarding the state-of-the-art implementation, the work in

[18] achieved 1.3441e+10 pairs / second through a tailored implementation, while the solution we provided comes from an automated exploration with a performance loss of only 8.98%.

[0106] The present disclosure provides a comprehensive method for optimizing high-level software code to achieve ASIC hardware implementation or FPGA configuration to meet target performance requirements. The disclosed method combines the automatic generation of the roofline model analysis for FPGA / ASIC, which allows identifying candidate FPGA or ASIC technologies in a given set and quickly identifying whether the application is memory- or compute-bound, and an automatic DSE to determine an optimal set of HLS optimization instructions. The DSE efficiency has been evaluated on the PolyBench test suite, and its performance is superior to the solutions in the literature, reducing the kernel latency by 14.36 times. In addition, the entire method has been verified in a practical application in the field of astrophysics and obtained a solution with performance comparable to the tailored implementation in the state-of-the-art.

Claims

1. A computer-implemented method for implementing a hardware device for performing operations defined by high-level software code, the method comprising the following steps: Translating the high-level software code into corresponding low-level software code to perform the same operations defined by the high-level software code, wherein the low-level software code defines low-level operations; For each hardware device in a set of hardware devices that can be fabricated or configured to execute the low-level software code, perform the following operations: For each low-level operation and for each type of available hardware resource of the hardware device, estimate the usage amount of the type of available hardware resource required to implement the low-level operation using the hardware device, wherein, assuming no data dependencies between the low-level operations, perform the estimating step; For each low-level operation in the low-level operations, estimate the runtime occurrence count of the low-level operation in the low-level software code; For each type of the available hardware resources, calculate the corresponding quantity of the required hardware resources of the type as the sum over all the low-level operations of the product of the runtime occurrence count of the low-level operation and the usage amount of the corresponding type of available hardware resource; For each type of the available hardware resources, calculate the corresponding ratio between the quantity of the required hardware resources of the type and the defined amount of the available hardware resources in the hardware device; Calculate a scaling factor for the hardware device as the maximum value of the ratios calculated in the previous step; Calculate the peak performance value of the hardware device as the sum over all the low-level operations of the product of the runtime occurrence count of the low-level operation divided by the scaling factor and a rate defining the number of low-level operations that can be executed per second by the hardware device; Select, from the set of hardware devices that can be fabricated or configured, the hardware device to be fabricated or configured according to the corresponding peak performance values calculated for each hardware device in the set; Fabricate or configure the hardware device selected in the previous step to execute the high-level software code.

2. The method according to claim 1, wherein, For each hardware device in the set, the method comprises the following steps: Estimate the total number of bytes of data to be transferred to the memory of the hardware device and to be transferred from the memory of the hardware device when executing the low-level software code; Calculate the operation intensity of the low-level software code as the sum over all the low-level operations of the ratio of the runtime occurrence count of the low-level operation and the total number of bytes of data to be transferred to and from the memory; Estimate the maximum bandwidth for transferring data to and from the memory of the hardware device; Calculate the memory transfer performance value of the hardware device as the product of the operation intensity and the maximum bandwidth; Also select the hardware device to be fabricated or configured in the set according to the corresponding memory transfer performance values calculated for each hardware device in the set.

3. The method according to claim 1, wherein the low-level software code is generated by the following means: Transform the high-level software code into the low-level software code using an LLVM front-end compiler; Process the low-level software code using an LLVM compiler to estimate the runtime occurrence count and the usage amount.

4. The method according to claim 1, further comprising the step of estimating the usage amount of each type of hardware resource of the hardware device for each low-level operation in the low-level operations by a resource estimation algorithm.

5. The method according to claim 1, wherein the type of hardware resource is block RAM, DSP, flip-flop, lookup table, and region.

6. The method according to claim 2, wherein the hardware device is configured on an FPGA or fabricated as an ASIC, and the step of selecting the hardware device to be fabricated or configured in the set comprises performing a design space exploration algorithm on the low-level software code based on the peak performance value and the memory transfer performance value to be satisfied to identify a suitable FPGA configuration or ASIC.

7. The method according to claim 1, wherein the peak performance value is calculated on a subset of the low-level operations in the low-level software code.

8. The method according to claim 2, wherein the operation intensity is calculated on a subset of the low-level operations in the low-level software code.

Images