Agile hardware implementation system with learning-based synthesis assistance

Abstract

To provide an agile prototyping system and method based on meta learning.SOLUTION: A hardware design flow includes: usually preparing system specifications 110 as a target and requirement such as clock frequency, latency, power consumption, accuracy, and footprints; designing (120) computing architecture to meet the specifications; describing (130) its functional design and logic design onto register-transistor logic (RTL); synthesizing (140) the RTL design to gate logics; physically designing (150) the gate logics so as to be placed onto a plane; conducting (160) design rule check (DRC) and other physical verification; fabricating a chiplet (170); packaging and testing (180) the chiplet; and deploying the chiplet as a custom computing chip 190.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] [Field of the Invention] The present invention relates to a hardware implementation system for low-power, high-performance computers, and in particular to an agile prototyping system based on meta-learning. [Background technology]

[0002] [Background and Prior Art] Traditionally, hardware description languages ​​(HDLs) such as Verilog have been widely used in the past decades to realize computing methods on hardware chips such as field programmable gate arrays (FPGAs). However, the use of HDLs is often not time-efficient because there is no inherent way to write HDL code from a specific computing method. In addition, the quality of the resulting hardware resources and performance depends heavily on how the computing method is described in the HDL code. Thus, the development of hardware chips takes a long time to meet specifications such as power consumption, processing throughput, computing latency, and hardware footprint. To improve the development timescale, high-level synthesis (HLS) design frameworks have been introduced recently. HLS design can use high-level languages ​​such as C / C++ and Python to describe the computational method. This allows for rapid prototyping of the computational method by bypassing the HDL coding step. However, HLS design involves a huge number of design options to consider, such as the target hardware device, resources, the required level of accuracy, and the time required for simulation, synthesis, co-simulation, etc. In addition, there may be many alternative realizations of a simple operation. To help hardware designers achieve their target designs, high-level languages ​​provide pragmas as guidelines for HLS systems. Proper use of pragmas can lead to high-quality, optimized, and efficient hardware designs. Because pragma installation affects other design features such as resource usage, latency, and target clock, finding a better Pareto front within a constrained development time is difficult in practice, especially due to the high flexibility in pragma installation and kernel translation.

[0003] Most design space exploration (DSE) tools use heuristic techniques to explore the optimal Pareto front with shorter development time. In recent years, learning-based DSE frameworks have been proposed. For example, Pyramid uses machine learning (ML) models to estimate the maximum achievable throughput. Recent studies have predicted resource usage for synthesizing various convolutional neural networks (CNNs). Another tool, Sherlock, finds Pareto-optimal solutions by using active learning with surrogate models. This presents the challenge of dealing with conflicting objectives in parameter optimization. However, a key element missing here is synthesizability within a given time budget. If the designs suggested by these tools do not synthesize within a given time frame, the applicability of such tools remains limited.

[0004] Therefore, there is a need for an improved hardware design methodology for agile prototyping that considers multi-objective functions including latency, power consumption, accuracy, footprint size, speed, and throughput under a limited development time budget. Summary of the Invention

[0005] We first address the shortcomings of traditional Bayesian Optimization (BO), specifically, it converges slowly if the HLS tool cannot synthesize the design within a given time budget. In addition, our method considers alternative realizations of the same kernel that use equivalent operations under a certain precision. With ML-based fault / resource predictions in action, AutoHLS becomes an efficient design space exploration (DSE) framework for high-level design. Key contributions are summarized below: · We propose an AutoHLS framework that accelerates DSE by using ML models. · The framework uses a learning-based optimization framework based on Bayesian optimization and machine learning methods. · Employing novel DNN and QNN models to accurately predict synthesis failures and resource usage. · A multi-objective optimization framework is used to simultaneously consider various hardware resources and performance metrics by achieving Pareto-optimal DSE. Accelerate DSE by using historical data exploration with reinforcement learning. · Fix your computing code by applying automatic installation of pragma directives. Apply automatic chord transpilation to explore different coding methods. · Performance specifications are adjusted by the user via a natural language interface and through the use of generative large-scale language models (LLMs). Under the automated HLS framework, energy-efficient artificial intelligence (AI) models can be achieved by considering quantization, pruning, and distillation. We achieve up to 70x speedup for HLS prototyping, which finds the Pareto front via kernel search.

[0006] We propose a machine learning framework that can accelerate DSE. We consider synthesis time as a major obstacle to DSE and overcome it by developing a time-budget-centric ML method. The proposed method can reach the optimal Pareto front of solving multi-objective resource optimization problems in FPGA / ASIC resource space. A unique feature of our method is that it also includes kernel transpilation for resource-efficient design. A real-world DNN design synthesis evaluation demonstrates the effectiveness of our framework in an HLS design flow.

[0007] Some embodiments use reinforcement learning to explore different high-level scripts from scratch and evaluate their feasibility / accuracy for a target operation. In some embodiments, green AI acceleration is achieved by considering speed, accuracy, and power consumption. By tuning the behavior in a meta-learning framework, the integration of DNN / QNN into existing optimization methods such as evolutionary optimization is also achieved.

[0008] Some embodiments of the present invention may provide a system or method for the need for a framework that combines software and hardware implementation level optimization to improve the energy efficiency of sparse quantized deep neural networks (DNNs). The proposed joint neural architecture optimization method explores the best design in each paradigm, from Python simulation to hardware-FPGA implementation. As a result, it reaches the best power and area requirements in the FPGA implementation. We evaluate our method on a real-time signal processing DNN model and find that it achieves 1.7x power and 40x area improvement compared to a baseline implementation of the same model. Our findings demonstrate the effectiveness of the proposed framework in optimizing power and area requirements for DNNs, which is important for IoT and edge devices where resource constraints are severe.

[0009] Another embodiment of the present invention is based on the recognition that High Level Synthesis (HLS) is a design flow that leverages modern language features and flexibility such as complex data structures, inheritance, templates, etc., to rapidly prototype hardware designs. Exploring various design space parameters can take a lot of time and effort for hardware engineers to meet certain design specifications. This embodiment provides a novel framework called AutoHLS that integrates Deep Neural Networks (DNNs) with Bayesian Optimization (BO) to accelerate HLS hardware design optimization. Our tool focuses on HLS pragma exploration and arithmetic transformation. It utilizes the integrated DNNs to predict synthesizability within a given FPGA resource budget. We also investigate the potential of the emerging Quantum Neural Networks (QNNs) as an alternative to classical DNNs for the AutoHLS pipeline. Our experimental results show up to 70x speedup in exploration time.

[0010] According to some embodiments of the present study, a system for electronic design automation is provided, the system including a memory storing instructions and a processor configured to execute steps of the instructions, the steps including: transforming application code according to a set of design specifications and a set of design parameters, synthesizing the transformed application code according to a high-level synthesis method to generate a set of profiling reports for implementation on a target hardware device, predicting the set of profiling reports and synthesizability under a time budget based on a set of machine learning models, exploring the set of design parameters according to an agent policy based on the set of profiling reports, and generating a set of optimized hardware implementations according to a Pareto front selection.

[0011] Yet another embodiment of the present invention provides a computer-implemented method for electronic design automation, the computer-implemented method including transforming application code according to a set of design specifications and a set of design parameters, synthesizing the transformed application code according to a high-level synthesis method to generate a set of profiling reports for implementation on a target hardware device, predicting the set of profiling reports and synthesizability under a time budget based on a set of machine learning models, exploring the set of design parameters according to an agent policy based on the set of profiling reports, and generating a set of optimized hardware implementations according to a Pareto front selection.

[0012] The present invention is based on a system for electronic design automation using a memory storing instructions and a processor configured to execute the steps of the instructions. The steps include transforming an application code according to a set of design specifications and a set of design parameters, synthesizing the transformed application code according to a high-level synthesis method to generate a set of profiling reports for realization on a target hardware device, predicting the set of profiling reports and synthesizability under a time budget based on a set of machine learning models, exploring the set of design parameters according to an agent policy based on the set of profiling reports, and generating a set of optimized hardware implementations according to a Pareto front selection. The system can accelerate the design flow to find optimized hardware implementations by exploring design parameters subject to the application specification and the design time budget. The application code is transformed by a combination of code analysis, kernel translation, pragma installation, etc. The application code is further modified by kernel translation based on a combination of quantization, sparsification, approximation, partitioning, pipelining, unrolling, inlining, distillation, etc. Pragma installation is further used by a combination of pragma type directives and pragma parameters to control HLS behavior. It explores a set of pragma types including inline, interface, dataflow, pipeline, unfold, array partition, latency, alias, protocol, stream, etc. The system uses a set of machine learning models based on a combination of support vector machines, logistic regression, ridge regression, deep neural networks, quantum neural networks, reinforcement learning, large-scale language models, etc.The set of machine learning models are trained with datasets relating to hardware implementations, software implementations, algorithmic implementations, artificial intelligence, digital signal processing, field programmable gate array prototyping, application specific integrated circuits, microprocessors, liquid state computing, quantum computing, molecular computing, etc. The agent policies make decisions based on a combination of multi-objective reinforcement learning, metaheuristic optimization, Bayesian optimization, the set of machine learning models, etc.

[0013] The target hardware devices include field programmable gate arrays, programmable logic arrays, application specific integrated circuits, graphic processor units, central processor units, microprocessors, liquid state computers, quantum computers, molecular computers, and variants. The system finds a multi-objective solution to the Pareto front selection based on a set of profiling reports. The set of profiling reports includes look-up tables, flip-flops, digital signal processing, latency, power consumption, clock frequency, mean squared error, and the like. In some embodiments, the system further uses a large-scale language model to tune agent policies controlled by a set of natural language prompts. This further increases the usefulness and flexibility of the design automation system without requiring domain-specific knowledge to realize high-performance, high-speed, and low-power hardware prototypes in shorter design pipeline times.

[0014] The accompanying drawings, which are included to provide a further understanding of the invention, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention. [Brief description of the drawings]

[0015] [Figure 1] FIG. 2 illustrates an exemplary design flow for a hardware implementation, according to some embodiments.

[0016] [Diagram 2] FIG. 1 illustrates an example design flow using ML-based forward model prediction, in accordance with some embodiments.

[0017] [Diagram 3] FIG. 2 illustrates an example design flow using ML-based forward model prediction, in accordance with some embodiments.

[0018] [Figure 4] FIG. 1 illustrates an example design flow using ML-based forward model prediction and active learning, in accordance with some embodiments.

[0019] [Diagram 5] FIG. 2 illustrates an exemplary hardware chip based on an FPGA framework, according to some embodiments.

[0020] [Figure 6] FIG. 2 illustrates an example pragma installation and kernel translation according to some embodiments.

[0021] [Figure 7] FIG. 1 illustrates example implementation results and Pareto fronts for processing time and look-up table (LUT) count using different pragma installations and kernel translations in accordance with some embodiments.

[0022] [Figure 8] FIG. 1 illustrates an exemplary design automation system, in accordance with some embodiments.

[0023] [Figure 9] FIG. 2 illustrates an example computing method using different precisions and quantizations in accordance with some embodiments.

[0024] [Figure 10]FIG. 1 illustrates an example method for using deep neural networks to assist in design automation, according to some embodiments.

[0025] [Figure 11] FIG. 1 illustrates an example method for using a quantum neural network to assist in design automation, according to some embodiments.

[0026] [Figure 12A] 5A-5C illustrate example performance using different precision and quantization methods in accordance with some embodiments.

[0027] [Figure 12B] 5A-5C are diagrams illustrating example performance using different time budgets and quantization methods in accordance with some embodiments.

[0028] [Figure 13] 1A-1C illustrate example results of predicting synthesis failures using different machine learning methods, according to some embodiments.

[0029] [Figure 14] 1A-1C illustrate example results of predicting LUT count using different machine learning methods, according to some embodiments.

[0030] [Figure 15] 1 illustrates an example result of using an agile prototyping system, according to some embodiments.

[0031] [Figure 16] FIG. 1 illustrates an example result of using an agile prototyping system, according to some embodiments.

[0032] [Figure 17] FIG. 1 illustrates an example method of using model distillation for power-efficient AI computation, according to some embodiments.

[0033] [Figure 18] FIG. 1 illustrates an example schematic diagram of agile design automation for power-efficient AI models, according to some embodiments.

[0034] [Figure 19A] FIG. 1 illustrates an example result of agile design automation for power-efficient AI models, according to some embodiments.

[0035] [Figure 19B] FIG. 1 illustrates an example result of agile design automation for power-efficient AI models, according to some embodiments.

[0036] [Figure 19C] FIG. 1 illustrates an example result of agile design automation for power-efficient AI models, according to some embodiments.

[0037] [Figure 19D] FIG. 1 illustrates an example result of agile design automation for power-efficient AI models, according to some embodiments.

[0038] [Figure 20] FIG. 1 illustrates an example design automation system using reinforcement learning and a language interface, according to some embodiments.

[0039] [Figure 21] FIG. 1 illustrates an exemplary schematic diagram of a system configured with a processor, memory, and interfaces in accordance with some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0040] Detailed Description of the Best Mode Various embodiments of the present invention will now be described with reference to the drawings. It should be noted that the drawings are not drawn to scale, and elements of similar structure or function are represented by similar reference numerals throughout the drawings. It should also be noted that the drawings are intended only to facilitate the description of certain embodiments of the present invention. They are not intended as an exhaustive description of the present invention or as limitations on the scope of the present invention. Moreover, aspects described in connection with a particular embodiment of the present invention are not necessarily limited to that embodiment, and may be implemented in any other embodiment of the present invention.

[0041] FIG. 1 illustrates an exemplary design flow for hardware implementation, according to some embodiments. Hardware design typically starts with a system specification 110 as goals and requirements such as clock frequency, latency, power consumption, accuracy, and footprint. A computing architecture is then designed to meet the specification (120). Its functional and logical design is then described (130) to register transistor logic (RTL). The RTL design is then synthesized (140) into gate logic. The gate logic is then physically designed (150) to be placed on a planar surface. Design rule checks (DRC) and other physical verifications are performed (160) prior to fabrication 170. The fabricated chiplets are then packaged and tested (180) before being deployed as custom computing chips 190. Target devices (or target hardware devices) include field programmable gate arrays, programmable logic arrays, application specific integrated circuits, graphic processor units, central processor units, microprocessors, liquid state computers, quantum computers, molecular computers, and the like. Implementing a specific application code on such a target hardware device is not trivial due to the existence of device-specific characteristics and constraints such as power consumption, logic resources, processing speed, and the like.

[0042] A typical workflow for hardware realization takes a long time because there are a large number of different designs to realize the system specification, and experienced designers use heuristic methods from experience. In addition, the designed architecture, logic, and fabrication do not always meet the required performance quality. Therefore, it typically requires many trial and error iterations over different steps of the design workflow. The present invention provides a method to reduce design time and effort by using AI-based assistance in different steps of the design workflow in an electronic design automation (EDA) system.

[0043] FIG. 2 illustrates an exemplary design flow using ML-based forward model prediction according to some embodiments. This embodiment provides a way to accelerate the design workflow by skipping the RLT design 130 through the use of high-level synthesis (HLS) 220, which employs a high-level language 210, such as C / C++ / SystemC / Python, to automatically generate a hardwired description language (HDL) 230 for realization 240. HLS is useful for maintaining the high-level language and the functional validity tested in simulation without violating the consistency of HDL. However, HDL synthesis takes a long time to generate high-performance HDL code, and the realization result 250 is often unsatisfactory compared to that of an experienced HDL designer due to the large number of hyper-parameters of HDL tools.

[0044] The present invention provides a method to improve design workflow by using AI-based performance prediction 260 that estimates actual performance results from HLS report 270 before realization. By using AI-based prediction, the system can pre-tune important hyperparameters (or design parameters), such as pragma factors that are inserted into source code (or application code) written in a high-level language. For example, those hyperparameters are optimized as agent policies by a metaheuristic optimizer, such as Bayesian optimization, by observing the landscape of predicted realization results generated by the AI ​​model, even without actual realization. Other examples of metaheuristic optimizers include differential evolution, evolutionary strategies, Nelder-Mead methods, genetic algorithms, simulated annealing, quantum annealing, swarm intelligence, and other variants.

[0045] 3 illustrates an example design flow using ML-based forward model prediction, according to some embodiments. An AI-based predictor 260 obtains additional information, such as application characteristics 320 and target FPGA specifications 330, to predict 340 performance on a target FPGA device. An application 310 is described by a high-level language 210. In addition to an HLS report 270, the application characteristics 320 and FPGA specifications 330 provide additional degrees of freedom to improve prediction accuracy across different target FPGA devices.

[0046] FIG. 4 illustrates an example design flow using ML-based forward model prediction and active learning, according to some embodiments. In this embodiment, active learning is used to improve the accuracy of prediction. The AI-based predictor 230 generates a performance estimate 410, from which several promising candidates for design parameters are sampled (420). The sampled parameters are then validated through an HLS tool to generate true results of performance in refined samples 430. Using the refined samples, the AI ​​model 230 is trained (440) to gradually adjust the behavior of the prediction towards the actual realization.

[0047] FIG. 5 illustrates an exemplary hardware chip based on an FPGA framework, according to some embodiments. An FPGA chip typically has a network interface, a computing engine, and adaptable hardware. The hardware design takes advantage of the high degree of flexibility in adaptable hardware, where computing function behavior is designed to take advantage of other built-in engines and interfaces. Hardware prototyping should consider computational efficiency and power efficiency as well as footprint efficiency simultaneously. The HLS tool needs to control various hyperparameters, such as interface mechanisms, parallelism, pipelining, and precision, so that the application specifications (or design specifications) in the quality of the results are met. One such control method is achieved by installing pragmas in the source code without destroying the computing functions. [Learning to Accelerate Design Space Exploration for HLS Design]

[0048] HLS is a rapid design and prototyping methodology that is widely used in industry and academia. Nevertheless, the rich features of modern programming languages ​​such as C / C++ / Python pose several challenges for source code optimization. Unthinking optimization can result in inefficient and resource-hungry designs with high latency, or in some cases, loss of synthesizability under reasonable FPGA resource budgets. HLS compilers such as Vitis offer optimization strategies such as pragma directives and timing / closure analysis to address these issues that have spurred an active research area in design space exploration (DSE) for HLS. Accelerated DSE is needed because downstream tools used for RTL generation such as Vitis can take significant time to compile and report synthesis results. This limits the number of designs evaluated during DSE, resulting in suboptimal solutions. Furthermore, the time required for RTL generation can increase DSE time from hours to days, depending on the complexity of the design. The demand for faster and more efficient DSE in HLS has led to the development of machine learning (ML), artificial intelligence (AI) and analytics methods. In this regard, one analytical approach utilizes a "quality of results" (QoR) estimator to accelerate the DSE process. By statically analyzing code blocks and modeling latency and resource utilization, the QoR estimator allows the DSE engine to efficiently explore the design space and converge faster to a Pareto front selection. Other methods use statistical, heuristic, ML, or meta-learning techniques to accelerate DSE. For example, an ML model estimates the maximum achievable throughput. At the same time, it can predict resource usage for compounding computing methods such as convolutional neural networks. To find the Pareto front, active learning is used with surrogate models to highlight the challenges in handling competing objectives in parameter optimization. Some embodiments use a Bayesian Optimization (BO) framework as a multi-objective optimization tool. BO is generally slow to find the Pareto front because the downstream HLS flow takes a lot of time to generate a QoR for each sample design point.Therefore, the present invention adds an early fault prediction network along with BO to accelerate DSE, focusing on reducing the search space based on synthesizability constraints such as FPGA footprint or synthesis time budget. FPGA footprint includes the block size of DSP engine, flip-flop (FF) gates, look-up table (LUT) size, etc.

[0049] The present invention, called AutoHLS, optimizes the design by considering the synthesizability constraints as a multi-objective optimization problem. AutoHLS efficiently determines the loop unrolling factor, pipeline depth, array partition, etc. for pragma installation to optimize the HLS design considering signal DSP, FF, LUT, power consumption, and latency. In addition, AutoHLS also includes a step of kernel operation transformation to further optimize the design. Pragma installation is realized by the selection of pragma type directives and their pragma parameters. Pragma types include inline, interface, dataflow, pipeline, unroll, array partition, latency, alias, protocol, stream, etc. to adjust the behavior of the HLS tool.

[0050] FIG. 6 illustrates an exemplary pragma installation and kernel translation according to some embodiments. In designing hardware in HLS, several important factors must be considered, such as the target hardware device, available resources, the required level of accuracy, and simulation, synthesis, and co-simulation time. In this regard, source code 610 in FIG. 6 presents a regular array doubling kernel in C++ as an example. This source code has a latency of 2032 ns when HLS tries to synthesize it in an FPGA device without any pragma directives. However, HLS provides several alternative implementations, such as codes 620, 630, and 640 in FIG. 6, in which different pragma directives are inserted. More specifically, to adjust the behavior of HLS, while keeping the computing function the same in the FPGA chip, code 620 uses interface and pipeline pragmas, code 630 uses unfolding pragmas, and code 640 uses array pragmas. Code 620 can improve the latency from 2032ns to 1683ns, code 630 can improve the latency to 556ns, and code 640 can further improve the latency to 399ns. Another code 660 uses a kernel transformation in which a functionally equivalent exponential addition replaces the multiplication operation. Since the computationally expensive multiplication can be eliminated, the latency can be dramatically improved to 69ns. However, determining which kind of pragma directives and their respective pragma parameters is not straightforward due to the large amount of design space. Kernel transpilation includes quantization, sparsification, approximation, partitioning, pipelining, inlining, unrolling, and distillation to modify the application code. The transformed application code is functionally equivalent to the original application code, but is modified to be efficiently synthesized by the HLS tool.

[0051] FIG. 7 shows exemplary realization results and Pareto fronts for processing time and look-up table (LUT) count using different pragma installations and kernel transpilations according to some embodiments. The synthesis profiling results (or a set of profiling reports) across different pragma factors and kernel operation transformations show trade-off behavior in multi-objective optimization, where reductions in LUT resources and runtime will compete as shown in the Pareto front curve 710 of FIG. 7. Nevertheless, due to the high degree of flexibility in pragma installations and kernel operation transformations, it remains challenging to find the optimal Pareto front in a constrained development time. Moreover, the Pareto front landscape becomes more complex when more QoR metrics are considered, such as power consumption and throughput. The present invention provides a method to solve this problem.

[0052] FIG. 8 illustrates an exemplary design automation system, according to some embodiments. This embodiment uses unoptimized kernels to efficiently explore different design alternatives so that the design objectives meet the specifications such as runtime, precision level, DSP, FF, LUT, etc. usage. It can discover the optimal set of pragma and kernel operation transformations with the help of an ML-based synthesizability prediction mechanism. The system utilizes source code 810 in addition to pragma directives and kernel transformations 860. An HLS 820 on the target FPGA device tries to synthesize the source code into an FPGA implementation. The profiling results 830 are analyzed with other verification codes 831 and stored in a database 832. The use of Bayesian Optimization (BO) tries to find the best candidates to improve the results and provides a sample of hyperparameters 840. Then, among the candidates suggested by BO, they are examined by an AI-based surrogate model 850, which predicts the QoR and synthesizability under the time budget. Candidate sampling by the BO will be rejected to reanalyze and resample candidates until the AI-based surrogate model finds the best synthesizable candidate with high QoR (855). After the best candidate is determined by the AI ​​surrogate model, pragma insertion and kernel transformations are specified to explore the next HLS synthesis (860). This design search exploration continues until the requested design time budget is reached.

[0053]

number

[0054] HLS synthesis tools often utilize high-cost resources such as DSP blocks to meet high throughput requirements, which may not be available for resource-constrained applications such as edge / embedded devices. Therefore, consider alternative operations that can save resources at the potential cost of throughput or precision. For example, the regular multiplication kernel in source code 610 of FIG. 6 may be functionally equivalent to the exponential addition kernel in code 650 of FIG. 6 for floating-point arithmetic when the multiplicand is a power of two (PoT) value. Additionally, simplifications can be achieved by reducing the bit-width precision and using fixed-point arithmetic with bit shifts.

number

[0055] FIG. 9 illustrates an exemplary computing method using different precisions and quantizations according to some embodiments. FIG. 9(a) uses a typical multiply-accumulate (MAC) 910 for floating-point precision, which generally requires high-speed and high-power DSP blocks. PoT quantization 920 can reduce computational resources by approximating multiplication operations through the use of shift and add operations, which can completely remove the high-power DSP blocks that perform multiplication. To reduce precision errors by PoT quantization, APoT quantization uses multiple shift and add operations in sequence, which can still remove the requirement of DSP multipliers while maintaining higher precision. Design space exploration selects different types of operations, including integer and fractional bit widths, as well as precision factors. Different combinations 940 of these operations can be utilized to adjust the QoR of realization resources for FF, LUT, DSP, latency, and MSE.

[0056] AutoHLS explores both kernel and parameter space. Given a set of kernels K, an objective function, and HLS design constraints, AutoHLS analyzes the kernels and returns a set of optimal synthesizable kernels for the given objective that satisfy the design constraints. For kernel transformation / transpilation, AutoHLS first analyzes the input C / C++ / Python kernels and constructs pragmas with a selected set P that includes pipelining, unrolling, latency, array partitions, etc. These kernels are then checked for feasibility before being synthesized.

[0057] For kernel profiling, after the synthesis step, the quality of results (QoR), kernel type, and pragma parameters are collected. Synthesis can be completed or failed for given constraints such as the HLS time budget. These data are utilized directly or indirectly in the objective function. To optimize hyperparameters, AutoHLS employs BO methods such as the Tree-Structured Parzen Estimator (TPE) for DSE, which can handle multi-objective optimization. The TPE-based optimizer proposes a set of optimized design parameters from the parameter space based on some acquisition function for efficient Pareto optimization.

[0058] To determine the next design space exploration candidates, the AutoHLS tool incorporates machine learning techniques to predict synthesis failures and estimate resource utilization of the designed kernels. Specifically, a deep neural network (DNN) and a quantum neural network (QNN) provide a failure prediction score for each sample set generated by BO. Based on the prediction results, the tool decides to synthesize or discard the kernel and moves on to the next kernel. This technique enables accelerated design space exploration and reduces the overall design time.

[0059] To analyze candidate samples, AutoHLS uses ML models to predict synthesis failures and estimate the resource profile of the design. These models, including classifiers and regression models, are trained on already explored samples and assign scores to new samples generated by BO. A decision is then made based on a threshold τ. Finally, a sample is sent for synthesis only if it passes the decision maker.

[0060] FIG. 10 illustrates an exemplary method of using deep neural networks to assist in design automation, according to some embodiments. This embodiment provides a DNN model to predict the synthesis score and resource usage of a design. This exemplary model takes design parameters as input and consists of three iterations of a batch normalization layer, a fully connected layer, and a rectified linear unit (ReLU) activation. A batch normalization layer, dropout, and sigmoid are applied at the end of this exemplary DNN model. DNN models typically have a relatively large number of trainable parameters, such as 3000, and are designed to learn with limited training samples, which is essential in DSE due to the long synthesis time of HLS tools.

[0061] FIG. 11 illustrates an exemplary method of using a quantum neural network to assist in design automation, according to some embodiments. In this embodiment, the ML-based prediction method for decision making uses a quantum neural network (QNN) model. Recent advances in quantum technology have led to the availability of high qubit processors, such as the 433 qubit processor released by IBM in 2022. This has led to a new paradigm of ML models known as QNNs, which have universal approximation properties and are more compact than modern DNNs. In this embodiment, QNNs are introduced for HLS acceleration. The exemplary QNN architecture shown in FIG. 12 has five qubits and only 54 trainable parameters, significantly fewer than typical DNN models. This QNN model consists of an angle embedding layer, two design entangling layers, and a measurement layer, as well as pre-scaling and post-scaling layers to predict composability and resource usage. In some embodiments, the QNN model includes different embeddings, such as amplitude embedding, basis embedding, Mottonen state preparation and other variants; different entangling layers, such as tree tensor network (TTN), matrix product state (MPS), multiscale entanglement renormalization ansatz (MERA) and other variants; and different measurement layers, such as Z-expectation, variance, state, probability, sample and other variants. The trainable variation parameters are trained to minimize the loss function through backpropagation, metaheuristic optimization, or reinforcement learning.

[0062] In yet another embodiment, classical ML algorithms are used for resource prediction and decision making. Classical ML models include Support Vector Machine (SVM), Logistic Regression (LR), Linear Regression, Lasso, Kernel Ridge Regression (KRR), Bayes Ridge Regression (BRR) for failure prediction and hardware profile prediction.

[0063] For the exemplary system, design automation is performed on a machine with an Intel Core i7-8700K CPU at 3.70 GHz and 64 GB of main memory running on Ubuntu 20.04.5 LTS to design on a Xilinx ZCU104 board as the target FPGA via Vitis HLS 2022.1 for kernel synthesis.

[0064] We verify the effectiveness of AutoHLS against DSE of a convolutional neural network (CNN) block. We consider the synthesis time t as the design resource budget or constraint. The CNN block is a CNN block with a window size L and input channels C in and output channel C out The convolution operation involves element-wise multiplication and accumulation of window elements and input channel elements. Figure 12A shows an example performance using different precision and quantization methods according to some embodiments. The table in Figure 12A shows the performance of the channel input C in = 100, filter length L = 7, channel output C out We provide the QoR results and area utilization of convolution kernel implementations for σ = 106, and float32. The table shows different types of kernels such as PoT<16,6> and APoT<16,6>, which are fixed-point data types that specify the total word bit width and fractional bit width in the first and last arguments in brackets, respectively. The results reveal that kernel transformation significantly affects the hardware footprint. However, kernel transformation may cause some loss in accuracy, as indicated by the mean squared error (MSE). Furthermore, fixed-point PoT has the lowest resource usage but has a higher MSE than APoT, and APoT has a better MSE but consumes more area than PoT. Finding the optimal kernel requires the DSE to determine the pragmas and appropriate pragma parameters. To find the best hyperparameters, the BO method is integrated with AutoHLS with an ML-based decision maker for further optimization.

[0065] Most ML models based on DNN use affine transformations that typically require high-power MAC operations. To create a hardware-friendly design for green ML models, some embodiments use PoT quantization and APoT quantization as kernel conversion schemes. The regular MAC with W being the weights of the DNN and b being the bias is represented as y = Wx + b; the PoT quantization of the weights W, u ∈ Z: W = ±2 u ; and the APoT quantization of the weights W: W = ±2 u ±2 v where u, v ∈ Z and v < u. Some embodiments further use additional integer factors greater than 2 to achieve higher accuracy while maintaining operations without MACs.

[0066] FIG. 12B shows exemplary performance using different time budgets and quantization methods according to some embodiments. This table in FIG. 12B shows the performance evaluation of BO in the exploration process. The "Time" column shows the synthetic time budget in minutes. The "Completed" and "Failed" columns show the number of samples for which kernel synthesis was successful and the number of samples for which it failed, respectively. The exploration includes 3302 designs and takes 4 to 6 minutes to complete regardless of the synthesis status. However, most of the parameters suggested by the BO method could not be synthesized, resulting in wasted attempts to synthesize incorrect designs. To address this problem, the present invention, known as AutoHLS, utilizes an early failure prediction mechanism with an ML-based decision maker.

[0067] In an exemplary validation, we generate 3302 convolutional design samples with BO, of which 961 are synthesizable within a given time budget. Each sample has five independent variables, one dependent variable, and a kernel identifier. We use all samples to train a classification model and use the synthesizable samples to train a regression model. The classification model predicts sample outcomes and is used as an early failure prediction model. The regression model predicts FPGA resource usage. Our model converges quickly on the training data. We demonstrate that our model can learn from a small number of training samples and achieve high accuracy on the test data.

[0068] FIG. 13 shows exemplary results of predicting synthesis failures using different machine learning methods according to some embodiments. The performance on the AI-assisted AutoHLS system is evaluated under various conditions. The results in FIG. 13 show a high true positive (TP) rate in the receiver operating characteristic (ROC) curve. The robustness and generalization ability of the AI ​​models are also confirmed. They still achieve a high true positive rate even when trained on only 5% of the samples. In particular, the QNN-based prediction of synthesis feasibility judgment achieves the highest area under the curve (AUC) score in the ROC curve of FIG. 13. FIG. 14 shows exemplary results of predicting LUT counts using different machine learning methods according to some embodiments. The AI-based prediction methods based on DNN and QNN models outperform classical regression methods such as SVM and LR. FIG. 15 shows exemplary results using an agile prototyping system according to some embodiments. In terms of the Pareto front, AutoHLS outperforms BO as highlighted by the dotted line in FIG. 15. The present invention can simultaneously achieve a lower LUT footprint and faster processing speed by taking into account the compositing time budget.

[0069] FIG. 16 shows exemplary results using an agile prototyping system, according to some embodiments. The table shows the effectiveness of an AI-based early failure prediction model by performing pragma parameter exploration on the APoT kernel. The estimated time for each design point synthesis is 10 minutes. The table in FIG. 16 shows results for different thresholds and demonstrates speedup of synthesizable design exploration time ranging from 15 to 74 times faster when using an AI-based decision maker to predict synthesis failure. As verified, the system of the present invention can provide an agile hardware prototyping method.

[0070] Through the use of AI-based HLS behavior prediction and decision-making mechanisms, AutoHLS can provide high versatility for unseen design spaces. Some embodiments improve versatility by leveraging the vast amount of open-source FPGA synthesis data available in DB4HLS, for example, including over 100,000 design samples. The low false positive (FP) rate achieved by AutoHLS indicates that the machine learning model can learn effectively. Some embodiments consider synthesizability within a given time budget, while other embodiments consider other metrics such as DSP and clock cycle counts. The nature of HLS synthesis data allows AutoHLS to learn from a small number of training data. Some embodiments integrate a multi-objective reinforcement learning (MORL) method with AutoHLS to increase the robustness of the framework. For example, a Pareto Q-learning method is used to simultaneously optimize multiple factors such as LUT, MSE, FF, latency, throughput, and power consumption.

[0071] As mentioned above, the present embodiment can provide AutoHLS, a framework for accelerating DSE for HLS using DNN / QNN-enabled multi-objective BO. It addresses the shortcomings of BO in HLS optimization. It further provides a resource prediction mechanism and faster exploration of the Pareto front. It demonstrates the effectiveness of this framework in achieving specific design goals through accelerated DSE and kernel operation transformation. Our experiments significantly speed up the discovery of optimal FPGA design parameters for CNN kernels. [Joint Software-Hardware Design for Green AI]

[0072] AI based on deep neural networks (DNNs) has been widely adopted in various domains including speech / audio, computer vision and signal processing. However, despite their high performance, DNN models are known to be energy intensive. For example, the energy required to train a large DNN model for natural language processing (NLP) can result in a significant carbon footprint, with an estimated CO2 emissions of 284 metric tons, equivalent to the lifetime emissions of five cars. This has led to the emergence of a new research direction called "Green AI", which aims to balance the trade-off between power efficiency and inference accuracy. Green AI models have shown promise in accelerating DNN models, especially on field programmable gate array (FPGA) platforms.

[0073] The energy consumption of AI models is mainly due to their architecture, especially the computationally expensive vector-matrix multiplication and bias-add operations. FPGAs often use highly customized digital signal processing (DSP) blocks to realize these operations as shown in FIG. 5. In addition, the size of training parameters such as weights and biases is another important factor that affects energy consumption, and larger parameter sets require more energy. Therefore, various green AI models, including knowledge distillation, pruning, and quantization techniques as shown in FIG. 17, which aim to reduce the size of DNN models while maintaining high inference accuracy, are believed to address this challenge. FIG. 17 illustrates an exemplary method of using model distillation for power-efficient AI computation, according to some embodiments. These models are designed to shrink DNN models from a teacher model 1710 to a student model 1720 while maintaining high inference accuracy. In addition to PoT / APoT quantization, model sizes such as kernel size / filter length, DNN layer depth, and neuron width size should be kept small for green AI. Furthermore, to realize sparse DNN1730 to reduce power consumption, it is often necessary to prune weights / neurons to remove redundant calculations. However, how to obtain those hyperparameters is not easy to realize in real hardware due to the huge design space. Furthermore, software-level optimization does not always lead to hardware-level green AI. The present invention based on AutoHLS provides a method to solve this problem by jointly optimizing software-level architecture design and hardware-level architecture design.

[0074] Although sparse DNNs have lower computational complexity from a software perspective, their efficient hardware implementations may require significant resources. Therefore, an optimized design must consider software and hardware requirements to minimize energy consumption. By jointly optimizing the DNN model from both domains, computational efficiency and energy consumption should be balanced, resulting in a fast and energy-efficient implementation. The present invention provides a framework for optimizing green AI models from Python software implementation to FPGA hardware deployment to achieve an energy-efficient design. Some key properties are: The present invention provides a software-hardware design methodology for generating machine learning models optimized for energy-efficient hardware implementation. · We find the optimal word size for fixed precision data that results in compact circuits. The methodology enables the efficient realization of sparse quantized DNNs, significantly reducing the energy consumption of the model.

[0075] One popular approach to optimize DNN models is to reduce the precision of weights and activations, reducing the amount of data that needs to be transferred and processed. This can be achieved by weight quantization techniques and activation quantization techniques, such as fixed-point quantization and dynamic fixed-point quantization. The Hardware Aware Automated Quantization (HAQ) framework leverages reinforcement learning to determine quantization policies for different neural networks and hardware architectures to effectively reduce latency and energy consumption with negligible accuracy loss. For SqueezeNet, a set of modifications made to the network architecture to achieve energy goals includes aggressive channel reduction, separable 3×3 convolutions, and element-wise add skip connections, as well as optimizing the architecture through simulation, although FPGA-targeted optimization has not been discussed in the prior art. Another approach is to compress the size of the DNN model through techniques such as pruning, knowledge distillation, and parameter sharing. Pruning removes unimportant connections or filters in the network. At the same time, knowledge distillation trains a smaller network to mimic the behavior of the broader network.

[0076] In addition, hardware-level optimizations have been widely studied to improve the power efficiency of DNNs. These optimizations include designing dedicated hardware accelerators and optimizing hardware architectures, which can significantly reduce power consumption. Fixed-point arithmetic is often used instead of floating-point arithmetic to represent weights and activations in DNNs, which requires less energy to compute. Binary weights can also be restricted to only two possible values, which significantly reduces power consumption and hardware complexity by replacing multiply-accumulate (MAC) operations with simple additions. To reduce power consumption in DNNs realized on FPGAs, optimization techniques can be applied, including software-level techniques such as quantization and hardware-level techniques such as pipelining, parallelization, etc. In the present invention, an agile design system can optimize both the software and hardware domains for power efficiency and computing precision.

[0077] FIG. 18 shows an exemplary schematic diagram of agile design automation for power-efficient AI models according to some embodiments. An overview of the proposed approach is shown in FIG. 18. The framework consists of Python simulation, C++ simulation, FPGA synthesis, and realization to find an optimized RTL or FPGA implementation from a basic ML model designed in Python. The general steps in this embodiment are as follows: Various ML models 1810 with different hyperparameters are evaluated 1820, architecture selection 1830 is performed in a Pareto sense, and various optimization techniques are applied to maximize performance for a specific problem. Two important techniques used in this step are weight quantization and pruning. Weight quantization reduces the precision of neural network weights to reduce memory and computation requirements during inference. At the same time, pruning removes weights that have little impact on the output, reducing computation during inference. An optimized model or weights is the result of this step.

[0078] The complexity of a DNN model is often related to the number of non-zero (nnz) parameters associated with each ML model. In some embodiments, different weight quantization schemes, such as no quantization, power of two (PoT) quantization, and additive PoT (APoT) quantization, are considered to optimize the DNN for efficient hardware implementation. The Pareto optimization approach selects the most power-efficient solution based on the tradeoff between accuracy and nnz. The Pareto front is identified, and the Pareto-optimal solution that provides the best performance for a given power budget is selected based on those tradeoffs between accuracy and nnz in two steps.

[0079] The first step is to generate Pareto solutions for each quantization scheme by plotting accuracy against nnz. The set of solutions with the best tradeoff between accuracy and complexity is selected.

[0080] In the second step, the most optimized solution is selected for RTL synthesis by comparing the Pareto fronts generated by different quantization schemes and refining them to obtain the final Pareto front.

[0081] Some solutions may vary only on one axis or even overlap in some cases. In that case, the following rule can be used to select the most promising solution: Let S be the set of Pareto solutions, nnz(s) be the number of non-zero parameters, nmse(s) be the normalized mean squared error, P(s) be the power consumption, C(s) be the number of channels used, and H(s) be the number of hidden layers in the solution set s. * is selected as follows:

number

[0082] Hereafter, we refer to the set of five rules mentioned above as the "exclusion rules", and in the following sections we will refer to them by their assigned rule numbers where appropriate.

[0083] Using an HLS tool 1850 such as Vitis to convert a machine learning model from a Python implementation to HDL code involves converting the DNN layers, activation functions, and multi-input multi-output convolutions to C++ code while preserving the optimizations made in Python. To improve the HLS behavior, in some embodiments, pragma installation and code transpilation / conversion 1840 are performed. To ensure accuracy, the converted C++ code must undergo c-simulation. Some device and FPGA specific tasks may be performed in this code conversion 1840. The convolution functions may also be optimized for performance and efficiency by exploring different quantization and sparsification.

[0084] In the Vitis HLS tool, the realization step plays a key role in transforming high-level C++ code into an optimized hardware realization design 1870 that meets the specific constraints and requirements of the target FPGA device. This process accurately estimates the resources required for the design, including the number of logic cells, DSP blocks, memory blocks, and other FPGA resources. This step is important to ensure that the design meets the power constraints. The realization step generates the RTL HDL that can program the FPGA to realize the hardware design. The HDL code can be written in VHDL or Verilog. This step allows for an approximate power estimate to be obtained.

[0085] Quality of Results (QoR) is an important metric for assessing the overall quality of an FPGA design, considering factors such as performance, power consumption, area utilization, and timing. To facilitate comparative analysis of QoR1860, design automation systems use various methods that can be optimized for low power, such as timing analysis to check if the design meets timing constraints, resource utilization analysis to ensure that the design fits within the capacity of the target FPGA device, and power analysis to measure the power consumption of the design. These analyses can be performed at different stages of the design flow, including simulation, synthesis, and realization.

[0086] In some embodiments, the method of the present invention can jointly optimize software and hardware for real-world low-latency and high-throughput CNN models. For example, validation is performed on a machine with an Intel Core i7-8700K CPU at 3.70GHz, 64GB main memory, and Ubuntu 20.04.5 LTS. The target FPGA synthesis board is Xilinx ZCU104, and kernels are synthesized using Vitis HLS2022.1.

[0087] We optimize a CNN-based digital predistortion (DPD) model to mitigate distortion caused by power amplifier (PA) nonlinearity in digital communication systems. DNNs have shown promising results in mitigating nonlinear distortion in PAs. We consider a 1D CNN-based DPD system with two input channels and two output channels that may have any number of hidden channels. Our approach involves training a neural network with a large input / output signal pair to learn the nonlinear behavior of the PA. The neural network then predistorts the input signal before amplification by the PA, countering the nonlinear distortion introduced by the amplifier. The CNN model has adjustable network configuration parameters such as kernel size, quantization type, number of hidden channels, number of hidden layers, and the percentage of weights to be pruned.

[0088] The optimization range includes kernel size (e.g., different values: 3, 5, 7, and 11), quantization type (e.g., no quantization, PoT, APoT), number of hidden layers (e.g., 2, 3, and 4), number of hidden channels (e.g., 2, 4, 6, 8, 10, 14, 16), and pruning percentage (e.g., 0, 30, 65, 83, 91, 95, 98, 99). A total of 840 samples are collected by sweeping these hyperparameters through all configurations with metrics including normalized mean squared error (nmse) and number of non-zero weights (nnz). The impact of each optimization technique on QoR, including hidden channels, pruning percentage, nmse, and nnz, is analyzed. Pareto solutions are identified for each quantization type based on the results. Pareto optimal solutions can be obtained after optimizing the Python code for the three types of quantization. FIG. 19A shows an example result of agile design automation for power-efficient AI models, according to some embodiments. Following the second step, the large search space reduced the number of solutions to 112 samples as potential Pareto front candidates. Further elimination rules are then applied to obtain the top few solutions. Eight of these top 50 solutions are shown in FIG. 19A. FIG. 19B shows an example result of agile design automation for power-efficient AI models, according to some embodiments.

[0089] The use of fixed precision representation in FPGAs has several advantages, including reduced power consumption, increased operating speed, and more compact area requirements. Fixed precision representation reduces the number of bits required to represent floating-point numbers, resulting in smaller circuit size and lower power consumption. It also allows for better control over precision levels and value ranges, which can be optimized to meet specific application requirements. Some embodiments compare various inputs with arbitrary precision points to determine the optimal word size for fixed precision representation in DNN implementations.

[0090] Pragmas can substantially benefit HLS by reducing power consumption and optimizing memory accesses. They are commonly used for loop unrolling, data pipelining, array partitioning, etc. to minimize the number of operations and data movement in the design. By inserting pragmas into the C++ source code of the CNN, DSP blocks and other resources can be saved.

[0091] After determining the optimal word size and quantization type for the neural network, certain parameters including kernel size, number of channels, number of layers, pruning percentage, and quantization type need to be identified from the Pareto solutions. The system then generates C++ code and synthesizes it in Vitis HLS to estimate the FPGA footprint. FIG. 19C illustrates an example result of agile design automation for power-efficient AI models, according to some embodiments. The impact of the number of hidden channels on power consumption and the number of non-zero values ​​(nnz) is analyzed as shown in FIG. 19C. Based on these observations, the hidden channel size can be optimized, resulting in Pareto solutions. The synthesis and realization profiles of these solutions are presented in the table of FIG. 19D. FIG. 19D illustrates an example result of agile design automation for power-efficient AI models, according to some embodiments.

[0092] The PoT implementation of the kernel is generally more energy efficient. However, APoT may have better nmse and power consumption for certain datasets and DNN models. The table in Figure 19D shows the design of the optimized hardware implementation against the non-optimized counterpart. In every row, the number of DSPs in the optimized CNN is reduced due to the careful use of pragmas and code transformations. The optimization technique performed better in APoT quantization, but not in LUT and FF optimization in non-quantization. The PoT kernel is not present in the top Pareto solutions because we prioritized nmse over nnz. PoT is more likely to be on the Pareto front if we prioritize nnz.

[0093] As described above, this embodiment can provide a framework for addressing the hardware-software optimization challenge of DNN models by presenting a design method for generating optimized hardware implementations by transpiling software code. By jointly optimizing hardware and software components, it is possible to achieve a balance between computational efficiency and energy consumption, resulting in a fast and energy-efficient system. The contributions of this framework include saving significant circuit resources and effort spent in discovering multiple designs for rapid hardware prototyping and enabling efficient realization of green AI models.

[0094] In some embodiments, the system uses a reinforcement learning framework as shown in FIG. 20. FIG. 20 illustrates an exemplary design automation system using reinforcement learning and a language interface, according to some embodiments. The agent 2010 adapts the DSE policy 2012 such that HLS synthesis is accelerated under a time budget using either a classical deep neural network (DNN) model or a quantum neural network (QNN) model. The DNN / QNN agent obtains observations 2050 of the environment 2020 to adjust the actions 2040 to maximize the reward 2060. For example, the environment 2020 includes HLS synthesis, the observations 2050 include a QoR report of hardware synthesis, and the actions 2040 include pragma installation and code transpilation to adjust pipelining, unrolling, quantization, etc. to optimize the latency, hardware footprint, power consumption, and throughput of the hardware chiplets. The reward 2060 includes, for example, a score of the hardware implementation's improvement in QoR. In some embodiments, a large-scale language model (LLM) 2080 such as ChatGPT and Llama fine-tuned with coding analysis is used as the user interface, and the translation policy is modified by natural language instructions by user prompts 2070. For example, the LLM gets a user prompt such as "Modify the code to reduce the latency of the synthesized hardware". The LLM 2080 attempts to suggest a translation policy to transform the original code based on the prompts 2070 so that the requested modifications are applied. The LLM can be fine-tuned in active learning or pre-trained with a large public dataset of hardware implementation samples, for example on GitHub.Such datasets include code samples and tricks for hardware implementation, software implementation, algorithm implementation, artificial intelligence (AI), digital signal processing (DSP), field programmable gate array (FPGA) prototyping, application specific integrated circuit (ASIC), microprocessor, liquid state computing, quantum computing, molecular computing, etc. It can explore various aspects of computing methods with pipelining, parallelization, quantization, nesting, etc. In some embodiments, the original code is also analyzed before being fed to the LLM. For example, C / C++ code analysis with Clang formatting can generate a unified extensible markup language (XML) format to be analyzed by the LLM. Transpilation and HLS automation allows the system to avoid further behavioral checks after hardware realization, resulting in agile prototyping.

[0095] The system can also be used in different synthesis steps such as logic synthesis, physical design, and fabrication as shown in Figure 1. This can be either forward modeling in Figure 2, reinforcement learning in Figure 3, or active learning in Figure 4. [Example System]

[0096] FIG. 21 illustrates an exemplary schematic diagram of a system configured with a processor, memory, and interfaces, according to some embodiments. Specifically, FIG. 21 illustrates a block diagram of an example system for agile hardware realization with AI-based synthesis assistance. The system 2100 includes a device 2101 having a set of interfaces and data links 2105 configured to transmit and receive signals, at least one processor 2120, a memory (or a set of memory banks) 2130, and a storage 2140. The processor 2120, in association with the memory 2130, executes computer-executable programs and algorithms stored in the memory 2130 and the storage 2140. The set of interfaces and data links 2105 includes a human machine interface (HMI) 2110 and a network interface controller 2150. The processor 2120 can execute computer-executable programs and algorithms in association with the memory 2130 that uploads computing instructions, computer-executable programs, and algorithms from the storage 2140. The instructions, computer executable programs, and algorithms stored in storage 2140 use deep neural networks (DNN) 2141, quantum neural networks (QNN) 2142, high level synthesis (HLS) 2143, reinforcement learning agent policies 2144, forward pass signal / backward pass gradient and other temporary caches 2145, optimizers 2146, large language models (LLM) 2147, code analysis and transpilation 2148, and pragma design methods 2149. The processor performs design space exploration (DSE) to optimize hardware hyperparameters such as pipelining, unrolling, bit width, quantization method, interface, etc. through analysis of a set of data sets by using the optimizers, agents, and DNN / QNN models.LLM is used to tune the code translation method by analyzing natural language and controlling the quality of the results through multi-objective optimization, e.g. lower LUT / FF / DSP footprint, lower power consumption, higher clock frequency, higher throughput, shorter latency, etc.

[0097] The processor 2120, in association with the interface and memory bank 2105, is configured to submit the signal and dataset 2195 to the DNN block 2141 and the QNN block 2143 to predict resource usage and composition feasibility under a time budget to support the HLS method 2143 via the agent policy 2144. The optimizer 2146 includes Nelder-Mead, stochastic gradient, and Bayesian optimization. The processor 2120 further performs the following: constructing the DNN 2141; calculating a loss function by forward propagating the dataset 2195; and backpropagating the gradient of the loss function with respect to trainable parameters over the DNN 2141 to update the trainable parameters with an optimization method 2146. The system uses several DNN models 2141 and QNN models 2142 to establish an agent model 2144, and an LLM model 2147 for the natural language interface. A code analysis and translation method 2148 is stored in the memory. An automated pragma installation and modification method is performed to optimize the hardware implementation through HLS method 2143. The system 2100 receives signals from a set of sensors 2111 via a network 2190, a set of interfaces and data links 2105, and other interface modules such as a pointing device / medium 2112.

[0098] The above-described embodiments of the present invention may be implemented in any of numerous ways. For example, the embodiments may be implemented using hardware, software, or a combination thereof. If implemented in software, the software code may be executed on any suitable processor or collection of processors, whether provided on a single computer or distributed across multiple computers. Such a processor may be implemented as an integrated circuit with one or more processors within an integrated circuit component. However, the processor may be implemented using any suitable form of circuitry.

[0099] Also, embodiments of the invention may be embodied as methods, of which examples are provided. The acts performed as part of the method may be ordered in any suitable manner. Thus, some acts shown in the example embodiments as sequential acts may be included to be performed simultaneously, and embodiments may be constructed in which acts are performed in an order different from that illustrated.

[0100] Moreover, the use of ordinal numbers such as "first," "second," etc. to modify claim elements in the claims does not, by itself, imply a priority, precedence, or ordering of a claim element over other claim elements, or the temporal order in which method operations are performed, but rather is merely used as a label to distinguish a claim element having a particular name from other elements of the same name (absent the use of the ordinal number).

[0101] Although the invention has been described by examples of preferred embodiments, it will be understood that various other adaptations and modifications can be made within the spirit and scope of the invention. Therefore, it is the object of the appended claims to cover all such variations and modifications which come within the true spirit and scope of the invention.

Claims

1. 1. A system for electronic design automation, comprising: a memory storing instructions; and a processor configured to execute steps of the instructions, the steps comprising: Transforming the application code according to a set of design specifications and a set of design parameters; synthesizing the translated application code according to a high level synthesis methodology to generate a set of profiling reports for implementation on a target hardware device; predicting the set of profiling reports and their composability under a time budget based on a set of machine learning models; exploring the set of design parameters according to an agent policy based on the set of profiling reports; generating a set of optimized hardware implementations according to a Pareto front selection.

2. The system of claim 1 , wherein the converting step further comprises code analysis, kernel translation, pragma installation, and combinations of variations thereof.

3. The system of claim 2 , wherein the kernel transpilation further comprises quantization, sparsification, approximation, partitioning, pipelining, unrolling, inlining, distillation, and combinations of variants thereof.

4. The pragma installation further includes a combination of a pragma type directive and a pragma parameter; The system of claim 2 , wherein the pragma types include inline, interface, dataflow, pipeline, unfold, array partition, latency, alias, protocol, stream, and variations thereof.

5. 2. The system of claim 1, wherein the set of machine learning models includes combinations of support vector machines, logistic regression, ridge regression, deep neural networks, quantum neural networks, reinforcement learning, large scale language models, and variations thereof.

6. 10. The system of claim 1, wherein the set of machine learning models is trained using datasets related to hardware implementations, software implementations, algorithmic implementations, artificial intelligence, digital signal processing, field programmable gate array prototyping, application specific integrated circuits, microprocessors, liquid state computing, quantum computing, molecular computing, and variations thereof.

7. 10. The system of claim 1, wherein the agent policies include decision making based on a combination of multi-objective reinforcement learning, metaheuristic optimization, Bayesian optimization, and the set of machine learning models.

8. 10. The system of claim 1, wherein the target hardware device is a field programmable gate array, a programmable logic array, an application specific integrated circuit, a graphic processor unit, a central processor unit, a microprocessor, a liquid state computer, a quantum computer, a molecular computer, and combinations thereof.

9. the Pareto-front selection being based on the set of profiling reports; The system of claim 1 , wherein the set of profiling reports further includes combinations of lookup tables, flip-flops, digital signal processing, latency, power consumption, clock frequency, mean squared error, and variations thereof.

10. The system of claim 5 , further comprising adjusting the agent policies controlled by a set of natural language prompts in the large-scale language model.

11. 1. A computer-implemented method for electronic design automation, comprising: Transforming the application code according to a set of design specifications and a set of design parameters; synthesizing the translated application code according to a high level synthesis methodology to generate a set of profiling reports for implementation on a target hardware device; predicting the set of profiling reports and their composability under a time budget based on a set of machine learning models; exploring the set of design parameters according to an agent policy based on the set of profiling reports; generating a set of optimized hardware implementations according to a Pareto-front selection.

12. 12. The computer-implemented method of claim 11, wherein the converting step further comprises code analysis, kernel translation, pragma installation, and combinations of variations thereof.

13. 13. The computer-implemented method of claim 12, wherein the kernel transpilation further comprises quantization, sparsification, approximation, partitioning, pipelining, unrolling, inlining, distillation, and combinations of variants thereof.

14. The pragma installation further includes a combination of a pragma type directive and a pragma parameter; 13. The computer-implemented method of claim 12, wherein the pragma types include inline, interface, dataflow, pipeline, unfold, array partition, latency, alias, protocol, stream, and variations thereof.

15. 12. The computer-implemented method of claim 11, wherein the set of machine learning models includes combinations of support vector machines, logistic regression, ridge regression, deep neural networks, quantum neural networks, reinforcement learning, large scale language models, and variations thereof.

16. 12. The computer-implemented method of claim 11, wherein the set of machine learning models is trained using datasets related to hardware implementations, software implementations, algorithmic implementations, artificial intelligence, digital signal processing, field programmable gate array prototyping, application specific integrated circuits, microprocessors, liquid state computing, quantum computing, molecular computing, and variations thereof.

17. 12. The computer-implemented method of claim 11, wherein the agent policies include decision making based on a combination of multi-objective reinforcement learning, metaheuristic optimization, Bayesian optimization, and the set of machine learning models.

18. 12. The computer-implemented method of claim 11, wherein the target hardware device is a combination of a field programmable gate array, a programmable logic array, an application specific integrated circuit, a graphic processor unit, a central processing unit, a microprocessor, a liquid state computer, a quantum computer, a molecular computer, and variations thereof.

19. 12. The computer-implemented method of claim 11, wherein the Pareto-front selection is based on the set of profiling reports, the set of profiling reports further including combinations of lookup tables, flip-flops, digital signal processing, latency, power consumption, clock frequency, mean squared error, and variations thereof.

20. 16. The computer-implemented method of claim 15, further comprising adjusting the agent policies controlled by a set of natural language prompts in the large scale language model.

Images