Adaptive and reconfigurable dataflow computing system and method

Abstract

The present invention relates to an adaptive and reconfigurable dataflow computing system and method. The dataflow computing includes a dataflow-driven core to dynamically reprogram itself based on computational needs during runtime. In this system, an application program is broken down into a dataflow graph with functional units by a compiler, and an instruction dispatcher allocates and runs available processing elements to execute all functional units in the dataflow graph to perform distributed or parallel processing. The dynamic allocation of processing units can be adapted during runtime.

Description

Adaptive and Reconfigurable Dataflow Computing System and MethodRelated Application

[0001] The present invention claims priority to Singapore patent application no. 10202402059U filed on 12 July 2024, the disclosure of which is incorporated in its entirety.Field of Invention

[0002] The present invention relates to adaptive and reconfigurable dataflow computing; this facilitates distributed or parallel computing according to computing loads, for example, by splitting a computing task into functional units or phases to achieve high efficiency and performance, yet balancing the frequency of re-programming during runtime.Background

[0003] Although modem processors have consistently improved with each new Central Processing Units (CPU) generation, the rate of progress has slowed compared to previous decades. To address the limitations of traditional designs, the use of specialized accelerators is increasingly common in high-performance systems. These state-of-the-art accelerators strive for high performance through cost-effective, energy-efficient designs. However, achieving these goals often involves compromises in flexibility, programmability, or both.

[0004] There are some specialized accelerators, such as Al Accelerators, Digital Signal Processors (DSPs) and Graphic Processing Units (GPUs) that deliver high performance and energy efficiency for specific tasks. However, these accelerators typically focus on single workload type and are not capable of speeding up a wide range of applications. To achieve both high performance and energy efficiency across a variety of applications, researchers have been exploring reconfigurable architectures like Field Programmable Gate Arrays (FPGAs) and Coarse-Grained Reconfigurable Arrays (CGRAs).

[0005] CPUs are general-purpose chips that can support a wide variety of applications. However, due to its complexity, their performance has been saturated, and they cannot cope with performance requirements of current workloads such as networking and artificial intelligence. This invention on adaptive and reconfigurable dataflow system and method overcome these limitations and is ready for more demanding future workloads.

[0006] FPGAs are specialized chips that provide higher performance while sacrificing its flexibility compared to CPUs. FGPAs are known to be difficult to program as profound hardware knowledge is required. Moreover, the complexity of the mapping process is extremely high, taking hours to days to complete without any guarantee of correctness. This invention on adaptive and reconfigurable dataflow system and method provides higher performance than both CPUs and FPGAs while having simpler software-level programmability. This allows users to use this invention without need for specialised hardware architecture knowledge.

[0007] Part of the prior art of this invention includes a dynamic data-driven reconfigurable architecture (3DRA), which targets both low-latency and high-throughput workloads. This 3DRA architecture implements a dynamic dataflow execution model that resolves data dependencies at run-time and utilizes non-blocking broadcast communication that reduces transmission latency to a single cycle to achieve high performance and energy efficiency. By employing a dynamic model, 3DRA eliminates costly mapping algorithms during compilation and improves the flexibility and compilation time of conventional CGRAs.

[0008] Within 3DRA, each computing element is programmed to perform a singular operation, which simplifies programming but can potentially underutilize the computing or processing elements. This underutilization results in missed opportunities to enhance performance and efficiency through more aggressive use of the computing or processing elements. Furthermore, 3DRA's applicability is confined to simpler tasks as the number of processing elements is limited due to 3DRA’s limited scalability.

[0009] Despite advances of these accelerators, there is a need to provide an adaptive and reconfigurable dataflow system and method that overcome the above limitations and is able to handle more demanding workloads, providing higher flexibility and performance and can be implemented in any network-on-chips.Summary

[0010] The following presents a simplified summary to provide a basic understanding of the present invention. This summary is not an extensive overview of the invention and is not intended to identify key features of the invention. Rather, it is to present some of the inventive concepts of this invention in a generalised form as a prelude to the detailed description that is to follow.

[0011] Preferably, the invention relates to a dynamic dataflow architecture comprising a dataflow-driven core configured to dynamically reprogram itself based on computationalneeds, and to facilitate adaptive reconfigurability during runtime, an instruction dispatcher mechanism that activates when a program requires reconfiguration, wherein the dynamic dataflow architecture facilitates distributed or parallel processing to execute multiple operations simultaneously.

[0012] Preferably, the invention relates to a method for dynamic re-programming of processing elements aimed at augmenting the performance and efficiency by optimising the use of processing elements. The invention method broadens the scope of applicability by overcoming the limitations imposed by a fixed number of available processing elements.

[0013] The invention is designed to surpass conventional CPUs in both performance and efficiency while also providing broader applicability than current spatial architectures such as FPGAs or CGRAs. Its design aims to meet the escalating demand for computational power in modern applications, where CPUs falter due to their linear (or sequential) execution paths and power constraints. The invention’s flexibility allows it to adaptively handle a variety of computational tasks, making it a versatile solution for a range of applications that would benefit from its enhanced capabilities.

[0014] The invention relates to a dataflow-driven architecture that dynamically re-programs itself based on computational needs. Its adaptive reconfigurability during run-time differs from the fixed execution schedules of existing spatial architectures. This Dynamically Orchestrated Dataflow Architecture (DODA) facilitates distributed or parallel processing, enabling it to execute multiple operations simultaneously at available processing elements, thus significantly boosting efficiency and performance.

[0015] A key part of the invention is its instruction dispatcher mechanism, which plays a critical role in the invention architecture's dynamic re-programmability. For example, this mechanism is activated when a program reaches a juncture requiring reconfiguration, such as transitioning between two loops that cannot concurrently occupy the available spatial processing elements. In such instances, the instruction dispatcher seamlessly manages the execution flow, ensuring that the operations for a subsequent loop (e.g., loop B) are initiated immediately after a preceding loop (e g., loop A) has concluded. This innovative approach ensures optimal utilization of spatial resources, enabling it to dynamically adjust to varying program demands without compromising on performance or efficiency.

[0016] As the instruction dispatcher can keep track of operands for each instruction, this allows the invention to simplify branch or loop management through the innovative use of predication outcomes and source indices, thus surpassing the optimisation strategies seen in a CGRA implementation that is dedicated to handle branches efficiently. The instructiondispatcher can keep track of branch / loop outcomes to ensure all the functional units of a program are executed.Brief Description of the Drawings

[0017] This invention will be described by way of non-limiting embodiments of the present invention, with reference to the accompanying drawings, in which:

[0018] FIG. 1 illustrates an adaptive and reconfigurable dataflow computing system according to an embodiment of the present invention.

[0019] FIG. 2 illustrates a schematic of the adaptive and reconfigurable dataflow system shown in FIG. 1 to comprise an instruction dispatcher to distribute component tasks to available processing elements (PEs) and a PE interconnection to feed results of computing tasks to the instruction dispatcher;

[0020] FIG. 3 illustrates a processing element executing a computing operation with operands as programmed by the instruction dispatcher; and

[0021] FIG. 4 illustrates an instruction table generated by the instruction dispatcher to track end of execution of each functional unit and to direct processing elements to execute the next functional unit.Detailed Description

[0022] One or more specific and alternative embodiments of the present invention will now be described with reference to the attached drawings. It shall be apparent to one skilled in the art, however, that this invention may be practised without such specific details. Some of the details may not be described at length so as not to obscure the present invention.

[0023] This invention relates to dataflow-driven re-programming depending on computation needs; thus this invention is referred to as an adaptive and reconfigurable dataflow computing system and method involving aDynamically Orchestrated Dataflow Architecture (DODA) 4. The dynamic and adaptive re-programming feature of the DODA 4 presents significant advantages over conventional spatial architectures like the Field Programmable Gate Arrays (FPGAs), Coarse-Grained Reconfigurable Architectures (CGRAs) or other accelerators. In these conventional systems, the re-programming process is static and manual, requiring a host CPU to pause its computational tasks to update the hardware configuration for new workloads. This not only diverts the CPU from its primary computational duties but also introduces considerable delays or latencies as it navigates through the CPU’s memory hierarchy to re-program the spatial architecture.

[0024] The DODA 4 invention revolutionises this approach by incorporating a built-in capability for self-reconfiguration based on the dataflow, thus effectively eliminating the need for external intervention from a host processor / CPU. This dynamic orchestration means that the DODA 4 invention can seamlessly transition between different computational tasks while minimizing the performance penalties associated with re-programming of conventional spatial architectures. The invention adaptable dataflow ability to re-program itself on the fly dramatically enhances its operational efficiency, significantly reducing the time required to execute new workloads.

[0025] Furthermore, by eliminating reliance on the host CPU for re-programming, the DODA 4 invention ensures that the CPU’s computational resources are fully dedicated or allocated to executing other critical tasks, thereby maximizing the overall system performance. This autonomous adaptability not only shortens processing times but also optimises resource utilization across the computing environment. The result is a highly efficient, flexible dataflow structure that can dynamically adjust to varying computational demands, thereby achieving superior performance levels unattainable by conventional FPGAs and CGRAs.

[0026] On-chip instruction memory and dispatcher: The DODA 4 introduces a transformative approach to spatial computing by integrating on-chip instruction memory and an instruction dispatcher 10. This innovation significantly enhances the flexibility, efficiency, and scalability of hardware programming, setting DODA 4 apart from conventional spatial architectures like the FPGAs and CGRAs.

[0027] Enhanced flexibility and resource utilization: Conventional spatial architectures often suffer from rigid programming structures, where hardware resources are statically allocated for specific functions. This static allocation limits the system's ability to adapt to varying computational demands. DODA 4, however, is designed to dynamically reconfigure itself to adapt itself to changing computational demands during computation runtime. For instance, upon the completion of a function A, DODA 4 automatically reconfigures to process another function B, thus optimising the use of available resources or processing elements and reducing idle times.

[0028] Advanced Programming for Complex Applications: The inclusion of on-chip instruction memory and the instruction dispatcher 10 enables the DODA 4 invention to support more complex or larger programs than its predecessors. By allowing instructions to be stored and dispatched based on dataflow and runtime outcomes, the DODA 4 invention can accommodate more complicated programs by splitting each program into multiplephases or functional units. This capability significantly diminishes the challenges posed by limited resources in mapping complex applications into the DODA 4 dataflow system.

[0029] Improved power efficiency: By programming functional units in large granularity, the DODA 4 invention minimizes the occurrence of re-programming and, consequently, the power consumption associated with context switching. This invention saves energy of frequent reconfiguration. As will be appreciated, after a compiler 2 breaks down an application program or code 1 into a dataflow graph 3 comprising functional units 6,8,9; these functional units are made up of relatively large granularities or functions; during runtime, these large granular functional units may be reprogrammed according to the number of available processing elements 11; by making available sufficient numbers of processing elements 11 to execute a computing task, occurrence of reprogramming or reconfiguration is controlled, thus this DODA 4 invention leads to saving energy by preventing frequent reconfiguration during runtime which occurs in every cycle in convention or traditional CGRAs

[0030] Extensible design: The DODA 4 invention’s flexible, dataflow reconfigurable architecture is designed to scale and support large and more complex programs by interconnecting multiple processing elements. The sole constraint for this interconnection is ensuring the order of packet delivery between a sender and a receiver, allowing the timing of packet delivery to vary as long as the sequence remains intact thanks to its dynamic instruction execution of the processing elements. Unlike conventional CGRAs, which necessitate strict timing adherence to maintain computation accuracy, this DODA 4 system offers more relaxed timing requirements. This flexibility significantly eases execution scheduling, which is a task that poses considerable challenges in conventional CGRA setups.

[0031] The DODA 4 invention represents a novel reconfigurable system that significantly enhances the functionality of spatial architectures by allowing for dynamic selfreconfiguration during computing runtime. For example, it works by converting C or C++ programs 1 into dataflow graphs 3, which is then used to structure the dataflow system to execute the given program. The core module that manages the operation of the DODA 4 invention is the instruction dispatcher 10, which is a component that holds the program's instructions and coordinates tasks delivery to various processing elements (PEs) 11. These PEs 11 are responsible for carrying out computational tasks and communicating results back to the instruction dispatcher 10. The DODA 4 invention's innovative approach allows it to modify its functions in response to the program's execution needs, thus dynamically managing the computational tasks without relying on external interventions. Thisautonomous and dynamic adaptability enables the DODA 4 invention to effectively tailor its operations for a wide range of applications, potentially leading to improvements in both computational performance and energy efficiency.

[0032] FIG. 1 illustrates the adaptive and reconfigurable dataflow of the DODA 4 system and method according to an embodiment of the present invention. As shown in FIG.l, the adaptive and reconfigurable dataflow or DODA 4 system and method is designed to process a variety of programs 1 which are compiled or broken down into dataflow graphs 3 showing constituent functional units 6,8,9. The DODA 4 system and method is designed to be dynamically self-re-programmable, meaning that the functionality of the dataflow system is executed depending on the run-time phase of the program as illustrated in FIG. 1. To execute a computing task, a program or an input code 1, for example, a generic C (or C++) program, is processed by a compiler 2 which converts or breaks down the input code or application program into a dataflow graph 3 at a functional level and inter-functional level. The resulting dataflow graph 3 of the input code is then stored in the instruction dispatcher 10 Depending on the dynamic results, the DODA 4 invention is able to self-re-programmed, being able to lead programming data from the instruction dispatcher 10 itself. For instance, depending on the outcome of a branch 5 after executing Function 1 (6) in the example dataflow graph, the PEs 11 are allocated to execute Function 2 (8), Function 3 (9), etc.. If Function 3 (9) requires additional computing needs, Function 3 is dynamically looped back to Function 1 (6) until all the functional units are executed.

[0033] FIG. 2 shows the functional architecture of the DODA 4 invention. The functional architecture comprises the instruction dispatcher 10 and multiple PEs (11-1 to 11-N) available to execute a functional unit. A PE is allocated to handle an operation using operands and if necessary from source PEs. For example, PE 3 (11-3) can be programmed to add the data from PE 1 (11-1) and PE 2 (11-2). In another example, if they are in a loop and until the loop is completed, PE 3 (11-3) iteratively adds the output data from PE 1 (11-1) and PE 2 (11-2). Then, when the loop is done, the PEI, PE2 and PE3 can be re-programmed to execute another operation. The re-programming is triggered when one of the PEs 11 sends a programming request 13 back to the instruction dispatcher 10 via a PE interconnection 12. Based on the reprogramming request 13, the instruction dispatcher 10 sends programming bits to the available PEs 11. It is designed to be flexible, and, thus, can be used alone or be replicated to connect with any network-on-chips such as mesh and torus topologies based on preferred performance and scalability. In other words, the selection in the module of PE interconnection 12 can be made without restrictions. The size of the instruction dispatcher10 and number of PEs 11 can vary depending on the types of target applications. The PEs (11,11-1,11-2,11-3, etc.) can be specialized or dedicated, for example, by removing unnecessary operations, when target workloads are known to increase performance and energy efficiency.

[0034] To compute a workload, available PEs 11 are programmed by configuring target operations and sources of operands depending on the target workload. The instruction dispatcher 10 contains such information for available PE 11 to be fetched to execute multiple functions. When the instruction dispatcher 10 receives a programming request 13, it fetches up to N instructions where N is the number of PEs 11 and sends them to the PEs. A programming request 13 is sent to the instruction dispatcher 10 when one of the PEs 11 triggers a re-programming event. For example, at the end of a loop, a branch operation (such as branch 5 checks if the loop is done or not. The PE handling the branch operation can then sends a programming request 13 when it detects that execution of the loop is not completed.

[0035] FIG. 3 demonstrates how a PE 11 is designed. An arithmetic logic unit (ALU) 15 is a unit in a processor that computes data based on the input instruction. Each ALU 15 comprises all the logic that perform the required operations. For example, in FIG. 3, the ALU 15 executes an operation 16 with operands 17, 18. The operation 16 and sources of the operands 17, 18 can be re-programmed at run-time by the instruction dispatcher 10 so as to optimize the use of all available operands to meet the computing loads. As seen from FIG. 3, the output of the computation from the ALU 15 is used to generate an outbound message 19 with an associated PE index 20. The outbound / output message or output data 19 and the PE index 20 are sent to the instruction dispatcher 10 via the PE interconnection 12, as illustrated in FIG. 2. Depending on configuration of the PE interconnection 12, it might be required to program the interconnection to make sure all the output messages / output data are delivered to destination PEs, converting the PE index 20 to an actual destination location for the PE interconnection module 12 The output message or output data 19 are used by other PEs as their operands. In addition, the output message / data can include a message indicating the status of whether the current function is done or whether the DODA 4 invention needs to be re-programmed. When PEs are required to be re-programed, the programming request 13 is included in the message with a base address to execute the next function, and subsequently the programming request 13 is sent to the instruction dispatcher 10 as shown in FIG. 2.

[0036] The instruction dispatcher 10 shown in FIG. 4 is responsible for, among others, storing operations, source indices of each processing elements 11, obtaining information required to decide if an operation reaches the end of the current functional unit and where to read thenext functional unit. When a programming request 13 is received with the base entry index, N instructions from the base entry index are sequentially fetched and sent to the PEs 11, where N is the number of PEs. For example, if there are 16 PEs and the base entry index is 0, entries from 0 to 15 are made available to all of the PEs 11.

[0037] For example, as shown in FIG 4, PE 0 (22) is programmed to add data from PE 4 (23) and PE 8 (24), and it is not the end of the currently mapped function. In another operation, PE 1 compares data from PE 5 (25) and PE 10 (26); depending on the output from PE 1 , the instruction dispatcher 10 determines whether the current functional unit reaches an end of operation, ie. whether the DODA 4 invention is to be re-programmed. When re-programming is needed, the DODA 4 invention sends a programming request 13 to the instruction dispatcher 10 with the base entry index 64 of the next functional unit, as shown in FIG. 4.

[0038] When a function is too large to fit in the limited number of PEs 11, the function can be split into multiple sub-functions. Then, at the end of each sub-function, an instruction can be added to mark the end of the sub-function and the base index of the next sub-function to re-program the DODA invention architecture and to continue the execution until all the functional units are completed.

[0039] From the above description, there are many advantages of the DODA 4 system and method; this DODA invention thus provides a new computing processing architecture or processor to handle demanding workloads with higher computing efficiency and performance, with higher energy-efficiency and area-efficiency (where one processing element or core is autonomously and dynamically reconfigurable to perform multiple functions according to computational demand during runtime).

[0040] While specific embodiments have been described and illustrated, it is understood that many changes, modifications, variations and combinations of variations disclosed in the text description and drawings thereof could be made to the present invention without departing from the scope of the present invention. For example, the processing elements or cores can be a mix of general processing elements / cores and dedicated processing elements / cores to optimise current and future computing workloads; such dedicated processing elements / cores may be directed to dedicated tasks like edge Al or graphic workloads, which may be located in wearable devices, general-purpose accelerators or processing-in-memory devices.

Claims

CLAIMS1. A dynamic reconfigurable dataflow computing system comprises: a compiler to break down a program into a dataflow graph showing functional units of the program; and an instruction dispatcher to allocate available processing elements (PEs) to execute all the functional units in distributed or parallel computing; wherein the instruction dispatcher dynamically allocates and runs the available processing elements (PEs) according to computing demands during runtime.

2. The system according to claim 1, further comprises a processing element interconnection to which outputs of the processing elements (PEs) are checked whether execution of a current functional unit is completed, and if not, the processing element interconnection sends a programming request to the instruction dispatcher to dynamically allocates and runs available processing elements (PEs) to continue executing the current functional unit, and until all the functional units of the program are completed.

3. The system according to claim 1 or 2, wherein execution in each processing element (PE) comprises performing an operation of each of the functional unit by an arithmetic logic unit (ALU) together with operands, and attaching an output data with an index associated with the PE to the processing element interconnection.

4. The system according to claim 3, wherein execution of the instruction dispatcher follows an instruction table, wherein each operation lays out source indexes of the processing elements (PEs) and indicates an end state of the current functional unit.

5. The system according to claim 4, wherein when the end state of the current functional unit is false, the processing element interconnection sends another programming request to the instruction dispatcher, until operation of the current functional unit reaches an end state at which a base index of the next functional unit is assigned, so that all the functional units of the program are executed.

6. The system according to any one of the preceding claims, wherein the program comprises an application program or a network-on chip program7. A method for dynamically orchestrating dataflow in a computing system to enhance performance and efficiency, the method comprising: dynamically reprogramming a dataflow-driven core based on computational needs, and allowing adaptive reconfigurability at runtime, to facilitate distributed or parallel processing to execute multiple computing operations simultaneously, thereby boosting efficiency and performance and high energy efficiency.

8. A computer processor implementing Dynamic Orchestrated Dataflow Architecture(DODA) according to the system recited in any one of claims 1-6 or to the method recited in claim 7.

Images