Coarse-grained reconfigurable array processor concurrently processing multiple graphs on a single mesh

Abstract

The processor (20) includes a computing architecture (24) and a controller (32). The computing architecture includes an array of computing nodes (28) and an interconnect (30) configurably connecting the computing nodes. The controller is configured to receive at least two different first and second data flow graphs (DFGs) (54), each specifying code instructions. The controller is also configured to configure at least some of the computing nodes in the computing architecture and the interconnect to concurrently execute code instructions specified in the first and second DFGs, and to send to the computing architecture a plurality of first threads executing code instructions specified in the first DFG and a plurality of second threads executing code instructions specified in the second DFG, such that the computing architecture executes both code instructions specified in the first DFG and code instructions specified in the second DFG at least during a given time interval.

Description

Invention Field

[0001] This invention relates generally to processor architecture, and more particularly to coarse-grained reconfigurable array (CGRA) processors. Background of the Invention

[0003] Coarse-grained reconfigurable arrays (CGRAs) are computing architectures in which interconnected arrays of computing elements can be reconfigured as needed to execute a given program. Various CGRA-based computing techniques are known in the art. For example, U.S. Patent Application Publication 2018 / 0101387 (the disclosure of which is incorporated herein by reference) describes a GPGPU-compatible architecture that combines a coarse-grained reconfigurable structure (CGRF) with a dynamic dataflow execution model to accelerate the execution throughput of massively threaded parallel code. CGRAs distribute computation across a structure of functional units. Computational operations are statically mapped to functional units, and interconnects are configured to pass values ​​between functional units. Invention Overview

[0005] Embodiments of the invention described herein provide a processor including a compute fabric and a controller. The compute fabric includes an array of compute nodes and an interconnect configurably connecting the compute nodes. The controller is configured to receive at least two distinct first and second data flow graphs (DFGs), each specifying code instructions. The controller is also configured to configure at least some of the compute nodes in the compute fabric and the interconnect to concurrently execute the code instructions specified in the first and second DFGs, and to send to the compute fabric a plurality of first threads executing the code instructions specified in the first DFG and a plurality of second threads executing the code instructions specified in the second DFG, such that the compute fabric executes both the code instructions specified in the first DFG and the code instructions specified in the second DFG at least during a given time interval.

[0006] In some embodiments, at least within a given clock cycle of the computing architecture, one or more computing nodes execute code instructions specified in a first DFG, and one or more other computing nodes execute code instructions specified in a second DFG. In some embodiments, at least during a given time interval, one or more computing nodes switch from executing code instructions specified in the first DFG to executing code instructions specified in the second DFG.

[0007] In the disclosed embodiments, a first compute node in the computing architecture is configured to switch from a first DFG to a second DFG at a first time, and a second compute node in the computing architecture is configured to switch from the first DFG to the second DFG at a second time different from the first time. In an example embodiment, a given compute node is configured to switch from the first DFG to the second DFG in response to completing processing of all threads associated with the first DFG. In an embodiment, a controller is configured to send a control signal to one or more compute nodes allowing the compute node to switch from the first DFG to the second DFG, and a given compute node in the computing architecture is configured not to switch from the first DFG to the second DFG before receiving the control signal.

[0008] In another embodiment, a given compute node that participates in executing code instructions specified in the first DFG but not in executing code instructions specified in the second DFG is configured to self-deactivate in response to completing processing of all threads associated with the first DFG. Additionally or alternatively, the controller may be configured to activate a given compute node that does not participate in executing code instructions specified in the first DFG but participates in executing code instructions specified in the second DFG.

[0009] In some embodiments, the controller is configured to instruct compute nodes to synchronously alternate between at least a first DFG and a second DFG. In other embodiments, a given compute node is configured to autonomously schedule the alternation between at least a first DFG and a second DFG. In an example embodiment, a given compute node is configured to receive a first input token according to a first DFG by communicating with one or more first compute nodes preceding the given compute node, is configured to receive a second input token according to a second DFG by communicating with one or more second compute nodes preceding the given compute node, and is further configured to execute the first and second input tokens according to the scheduled alternation between the first and second DFGs.

[0010] According to embodiments of the present invention, a computing method is also provided, the method comprising receiving at least different first and second data flow graphs (DFGs) in a processor including a computing structure, the computing structure including an array of computing nodes and interconnects configurably connecting the computing nodes. At least some of the computing nodes in the computing structure and the interconnects are configured to concurrently execute the code instructions specified in the first and second DFGs. A plurality of first threads executing the code instructions specified in the first DFG and a plurality of second threads executing the code instructions specified in the second DFG are sent to the computing structure, such that the computing structure executes both the code instructions specified in the first DFG and the code instructions specified in the second DFG at least during a given time interval.

[0011] The invention will be more fully understood from the following detailed description of embodiments thereof, taken in conjunction with the accompanying drawings, in which: Brief description of the attached diagram

[0013] Figure 1 This is a block diagram schematically illustrating a coarse-grained reconfigurable array (CGRA) processor according to an embodiment of the present invention;

[0014] Figure 2 This is a schematic illustration based on an embodiment of the present invention. Figure 1 A block diagram of the FPU / ALU node in the CGRA processor;

[0015] Figure 3 This is a schematic illustration based on an embodiment of the present invention. Figure 1 A block diagram of the LSU node in the CGRA processor;

[0016] Figure 4 This is a flowchart illustrating, schematically, a method for concurrent execution of multiple DFGs according to an embodiment of the present invention; and

[0017] Figure 5 This is a flowchart illustrating a method for concurrent execution of multiple DFGs, according to an alternative embodiment of the present invention. Detailed Implementation

[0018] Overview

[0019] The embodiments of the invention described herein provide improved methods and apparatus for coarse-grained reconfigurable array (CGRA) computing. An exemplary CGRA-based processor includes a coarse-grained computing architecture and a controller for configuring and controlling the computing architecture. The computing architecture includes (i) an array of computing nodes and (ii) an interconnect configurably connecting the computing nodes.

[0020] The controller receives compiled software code, also known as a software program, which is represented as a data flow graph (DFG). Based on the DFG, the controller configures the compute nodes and interconnections, thereby configuring the architecture to execute the program. The controller then sends multiple threads to the compute architecture, which executes these threads in a pipelined manner according to the DFG.

[0021] Multiple threads typically execute the same code, represented by a DFG (Device Execution Group), on different data. (Strictly speaking, different threads differ slightly from each other because they depend on their thread number, for example, in specifying different memory addresses to fetch different data from those addresses. However, in the current context, these threads are considered to be executing the same code. Although different threads execute the same static code, their dynamic execution may differ due to the differences in the data.)

[0022] However, in many practical situations, software programs are too large to be mapped onto a computational architecture as a whole. Therefore, in some embodiments of the invention, the software program provided to the controller is represented as a collection of interconnected Data Entry Groups (DFGs) rather than as a single DFG. Each DFG specifies the code instructions for a corresponding section of the executor program. Each DFG has a single input and one or more outputs. DFGs are interconnected according to a predefined interconnect topology that follows high-level flow control of the program. This architecture is typically provided by the compiler. Note that even if the computational architecture is large enough to map the entire code, there may be other reasons (e.g., due to branching) to divide the code into multiple DFGs.

[0023] In the disclosed embodiments, the controller includes a scheduler that schedules the execution of various Functional Graphs (DFGs) on a structure on an alternating time basis. When a particular DFG is scheduled, the scheduler “maps the DFG onto the structure,” that is, configures at least some compute nodes and interconnects in the structure to execute the code instructions specified in the DFG. The scheduler sends multiple threads to the structure for execution, and the structure executes these threads according to the DFG, typically starting by reading initial values ​​for each thread from memory. During the execution of threads on the currently scheduled DFG, the structure saves the results of each thread to memory, and the scheduler reconfigures the structure to execute the next DFG. Scheduling, reconfiguration, and execution are performed in this manner until all threads have finished executing the software program, or until the controller decides to suspend processing for any reason.

[0024] Other aspects of running multiple threads on a CGRA architecture are resolved in U.S. Patent Application Publication 2018 / 0101387 and U.S. Patent Application 16 / 860,070, which are cited above and incorporated herein by reference.

[0025] In some embodiments of the invention, a CGRA processor, including a computing architecture and a controller, is configured to concurrently execute instructions belonging to multiple different DFGs. The disclosed techniques relate to the concurrent processing of multiple different DFGs, rather than the concurrent processing of multiple copies of the same DFG that can run concurrently on different CN sets. Hybrid cases in which the architecture concurrently processes multiple different DFGs are also supported, where one or more DFGs have multiple copies. The different DFGs can belong to the same software program or different programs or tasks, such as different database queries. The different programs or tasks can be associated with the same user or with different users.

[0026] In the current context, the phrase "concurrent execution of code instructions specified in multiple different DFGs" refers to the computational structure as a whole, and not necessarily to each individual compute node. In some embodiments described herein, the structure progressively switches from one DFG to another. In these embodiments, during a specific transition interval, one or more compute nodes execute instructions specified in the current DFG, while at the same time (e.g., the same clock cycle), one or more other compute nodes execute instructions specified in another DFG with threads ready for execution. Gradual switch-over is highly effective in reducing the high latency overhead that would otherwise occur when switching from one DFG to another.

[0027] In other embodiments, one or more compute nodes in the architecture alternate between the execution of instructions specified in two or more different DFGs (e.g., from one clock cycle to the next). This technique provides improved processing efficiency and throughput because compute node resources are utilized more efficiently.

[0028] System Description

[0029] Figure 1 This is a block diagram schematically illustrating a coarse-grained reconfigurable array (CGRA) processor 20 according to an embodiment of the present invention; the processor 20 is based on a reconfigurable computing architecture 24, which includes a plurality of computing nodes 28 arranged in an array. For simplicity, the CNs 28 are also simply referred to as "nodes". In the example embodiment, the architecture 24 has a total of 256 CNs 28 arranged in a 16×16 array, although any other suitable number and arrangement of CNs can be used.

[0030] Interconnection 30 connects the CNs 28 in structure 24 to each other for transmitting data and control information (referred to as "tokens") between CNs. In this embodiment, each CN 28 includes a switch (in... Figure 1 Not seen in the middle, below Figure 2 and Figure 3 An example is shown below, which can be configured to connect a CN to one or more selected adjacent CNs. In some embodiments, CN28 can send and / or receive additional control information, such as configuration and / or exceptions. Such information can be transmitted via a separate interconnect (e.g., a control network separate from interconnect 30). See below for example. Figure 2 and Figure 3 Interconnections marked as "to / from controller".

[0031] Processor 20 also includes a Level 1 (L1) cache 36 (or, alternatively, a multi-level cache) connected to external memory 40 and structure 24. Cache 36 and external memory 40 are collectively referred to as the processor's memory system. At least some of the CNs 28 in structure 24 are configured to store data in and load data from the memory system.

[0032] In some embodiments, all CNs 28 in structure 24 are structurally similar. In other embodiments, structure 24 includes two or more different types of CNs. The following... Figure 2 and Figure 3 Examples are shown of floating-point unit / arithmetic logic unit (FPU / ALU) CNs configured to perform mathematical calculations and load memory unit (LSU) CNs configured to perform memory load and store operations. In some embodiments, the LSU CNs are located along the periphery of structure 24 to simplify their interconnection with the memory system. The FPU / ALU CNs can be located anywhere suitable within structure 24. Other types of CNs not discussed in detail herein may include control CNs and CNs that perform high-complexity calculations such as division or square root calculations.

[0033] Processor 20 also includes controller 32, which, among other functions, configures structure 24 to perform desired computational tasks. Controller 32 typically configures the functionality of each CN and the interconnect topology between CNs. Controller 32 typically configures structure 24 by sending control registers to CN 28 via a control network.

[0034] Controller 32 typically receives a compiled version of software code generated by an appropriate compiler. The compiled code comprises a software code representation as a structure 50 of interconnected data flow graphs (DFGs) 54. Each DFG 54 represents a corresponding part of the code and has a single entry point and one or more exit points. Each DFG is a directed graph, where vertices represent code instructions and arcs represent data flow between instructions. The exit points of a DFG (e.g., conditional or unconditional branches, function calls, or returns from functions) include instructions that lead to (different or the same) DFGs or terminate the program.

[0035] To execute a given DFG, controller 32 configures selected CN 28 and their interconnections to execute the code instructions and flow control specified in the DFG. This operation is also known as “mapping the DFG onto a structure”.

[0036] A typical DFG begins by reading an initial value (live value) and possible data from a memory system, then continues with a sequence of instructions that process and potentially store the data, ending with the live value being stored. However, loading values ​​(live value and data) from memory and / or storing values ​​into memory can typically be performed at any stage of the DFG, not necessarily at the start / end. In the example implementation, controller 32 configures the structure to perform such a DFG by: allocating one or more LSU CNs to read the initial value, allocating appropriate CNs (e.g., FPU / ALU CNs) to execute the instructions specified in the DFG, and allocating one or more LSU CNs to store the results. Controller 32 also configures a switch for one or more CNs to interconnect them according to the flow specified in the DFG. Examples of such configurations can be found in U.S. Patent Application Publication 2018 / 0101387 and U.S. Patent Application 16 / 860,070, both of which are cited above and incorporated herein by reference.

[0037] Once a given DFG is configured, structure 24 can pipeline multiple threads, each executing the instructions of the given DFG on different data. In such a pipelined operation, a given CN executes the same instructions in a DFG (with different parameters) for different threads. Once a CN has finished executing the instructions for one thread, it is ready to execute the instructions for the next thread. Typically, different threads process different data because the load instructions for loading data for a thread specify an address that depends on the thread ID.

[0038] Typically, the size of structure 24 is insufficient to execute the entire structure 50 (i.e., all DFGs) simultaneously. Therefore, in some embodiments, controller 32 includes a DFG scheduler 44 that schedules the execution of each DFG 54 alternately over time. Scheduler 44 uses a data structure called the Next DFG Table (NDT) 48 to schedule the DFGs. The operation of scheduler 44 and the various data structures used for scheduling are described in detail in U.S. Patent Application 16 / 860,070.

[0039] Figure 2 This is a block diagram schematically illustrating an FPU / ALU node 60 according to an embodiment of the present invention. The configuration of node 60 can be used to implement... Figure 1 At least some of CN 28 in the structure 24 of the CGRA processor 20.

[0040] FPU / ALU node 60 includes computing element 64, switch 68, and token buffer 72. Switch 68 is configured to connect node 60 to one or more selected neighbor CNs in structure 24, for receiving input tokens from neighbor CNs and sending output tokens to neighbor CNs. Input tokens may include, for example, data and metadata (control information) from neighbor CNs for use as operands in the computation of node 60. Output tokens may include, for example, the computational results of node 60 provided to neighbor CNs for subsequent processing. Input and output tokens may also include control information.

[0041] In this example, node 60 can connect to any of the eight nearest neighbors CN 28 in structure 24, denoted as CN0…CN7. Therefore, node 60 has eight interconnects 30 connected to switch 68. Alternatively, any other suitable interconnect topology can be used.

[0042] The token buffer 72 is configured to store input tokens received from the neighboring CN and awaiting processing, and / or output tokens processed and awaiting output to the neighboring CN. The computing element 64 includes circuitry for performing calculations on the node 60, such as an FPU, ALU, or any other suitable computing device.

[0043] Consider a scenario where controller 32 configures structure 24 to execute a specific DFG 54. As part of this configuration, FPU / ALU nodes 60 are assigned to execute specific code instructions within the DFG. To play this role, controller 32 configures computing element 64 to perform calculations specified in the instructions, such as addition or multiplication. Controller 32 also configures switch 68 to connect appropriate interconnects 30 to CNs that provide input to and consume the results of the code instructions. This switch can also be configured to transfer values ​​to other CNs without passing through computing element 64.

[0044] In this example, node 60 also includes auxiliary logic 76 and decoder 80. Auxiliary logic 76 includes a configuration register that holds configuration parameters for the switch 68 and computing element 64 configured by controller 32. Furthermore, auxiliary logic 76 can perform functions such as communication with controller 32, interrupt handling, and counter management.

[0045] In a typical configuration process, controller 32 sends one or more instructions to the configuration registers in configuration auxiliary logic 76 to FPU / ALU node 60. Decoder 80 decodes the instructions received from controller 32 and configures the configuration registers accordingly. As will be explained below, in some embodiments, node 60 includes dual configuration registers (also known as “shadow registers”) for enabling fast switching between DFGs.

[0046] Figure 3This is a block diagram schematically illustrating an LSU node 84 according to an embodiment of the present invention. The configuration of node 84 can be used to implement... Figure 1 Some CN28 in the structure 24 of the CGRA processor 20.

[0047] LSU node 84 is configured to execute load and store instructions, namely, to load data from memory 40 via cache 36 and to store data into memory 40 via cache 36. As described above, LSU nodes are typically located along the periphery of structure 24 to simplify their interconnection with the memory system.

[0048] LSU node 84 includes a switch 88, a token buffer 92, and a decoder 98, which are structurally and functionally similar to the switch 68, buffer 72, and decoder 80 of FPU / ALU node 60, respectively. Additionally, node 84 includes a computational element 94, for example, for memory address calculation. Node 84 also includes auxiliary logic 96, which performs functions such as communication with controller 32, communication with L1 cache 36, storage of configuration registers, interrupt handling, counter management, NDT updates, and initiator and termination functions (described below).

[0049] In an alternative embodiment, the initiator and terminator functions may be implemented in a separate CN, rather than in LSU node 84. Therefore, in this context, the terms "initiator" and "terminator" refer to any CN that performs the initiator and terminator functions, respectively. The initiator and terminator functions may be implemented in a dedicated CN or combined with other functions in the same CN, such as in an LSU node.

[0050] In some embodiments, when a given DFG is mapped onto structure 24, scheduler 44 may deactivate (e.g., power down) one or more CNs 28 that are not involved in executing the code instructions specified in the given DFG in order to reduce power consumption. The scheduler may deactivate only portions of the CNs, for example, only switches (e.g., switch 68 or switch 88) or only computing elements (e.g., element 64 or element 94).

[0051] In some embodiments, when configuring structure 24 to execute a particular DFG, scheduler 44 configures the structure with multiple replicas of the DFG that will run concurrently. This feature is useful, for example, for DFGs much smaller than the structure. In these embodiments, the DFG replicas execute the same parts of the program but are mapped to different subsets of CN 28. In particular, each replica of the DFG has its own initiator and terminator nodes.

[0052] like Figures 1-3The configuration of the CGRA processor 20, FPU / ALU node 60, and LSU node 84, and their various components, shown are illustrative configurations described purely for conceptual clarity. In alternative embodiments, any other suitable configuration may be used. Additional aspects of the multi-threaded CGRA processor are addressed in U.S. Patent Application Publication 2018 / 0101387 and U.S. Patent Application 16 / 860,070, cited above.

[0053] In various embodiments, processor 20 and its elements may be implemented using any suitable hardware such as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA). In some embodiments, some elements of processor 20, such as controller 32, may be implemented using a programmable processor programmed with software to perform the functions described herein. The software may be downloaded to the processor electronically via a network, or alternatively or additionally, the software may be provided and / or stored on a non-transitory tangible medium, such as magnetic storage, optical storage, or electronic storage.

[0054] Initiator and Terminator Operation

[0055] In some embodiments, the LSU node in structure 24 (e.g., Figure 3 Each of nodes 84 can be used as a starter or a terminator. In other embodiments, the starter and terminator are implemented as dedicated CNs. A starter is a CN that begins execution of a thread in the DFG. A terminator is a CN that completes execution of a thread in the DFG.

[0056] The initiator typically receives batches of threads from scheduler 44 for execution. The initiator then calculates a thread ID (tid) for each initiated thread and sends the tid (one after another) to its successor CN 28 in structure 24. Upon receiving a tid, each CN 28 performs its configured computation and forwards the result to the next CN according to the DFD. Thus, each tid travels through structure 24 as a data token.

[0057] Typically, thread batches are passed between scheduler 44 and the initiator as tuples of (Thread_batch_id, Thread_batch_bitmap, Thread_set_ID). When a thread batch arrives at the initiator, the initiator analyzes the bitmap to identify the thread ID it should initiate (e.g., by adding the index of the set bit in the bitmap to Thread_batch_ID). To avoid pauses, the initiator can use double buffering ("shadowing") of prefetching and thread batches. Whenever a thread batch is received, the initiator immediately requests the next thread batch from scheduler 44.

[0058] When the entire thread set has finished executing on the current DFG (or when the scheduler decides to terminate the thread set for some reason, such as to perform a context switch), i.e., when the scheduler has no remaining batches of threads to send, the initiator sends a "final token" to the structure. This token cannot be bypassed by any other token. The final token is sent to all copies of the DFG and used by the terminator node to verify that the thread set has been completed.

[0059] The terminator executes the final branch instruction that leaves the current DFG. In the disclosed embodiments, each exit point from the DFG is implemented as a branch instruction, referred to herein as an "exit branch" for brevity. The target address of the exit branch is the starting address of the next DFG. The exit branch can be a conditional or unconditional branch instruction. The exit branch can be, for example, a direct branch instruction, an indirect branch instruction, a predicted branch instruction, or a "thread halt" instruction. The DFG can include any suitable number of exit points, possibly a single exit point. The exit branch can be located anywhere in the DFG and is not necessarily the last instruction.

[0060] For exit branches implemented as direct branches, the identifier of the successor DFG is indicated to the terminator node when it is initially configured (as part of the configuration structure to execute the current DFG). At runtime, the successor DFG is determined by the input tokens (or tokens) to the terminator node or by the terminator node itself based on the branch conditions. The successor DFG is determined per thread based on the result (taken / not taken) of the branch conditions for each thread. In this embodiment, the terminator node holds four 64-bit bitmaps. As the batch of threads completes execution, the terminator node progressively constructs a pair of 64-bit bitmaps, one bitmap for each possible successor DFG and one bitmap for the result of each possible branch decision.

[0061] When all threads in a batch have finished executing, the terminator node holds two bitmaps. In one bitmap, a set bit indicates that the thread whose successor DFG has "taken" the result. In the other bitmap, a set bit indicates that the thread whose successor DFG has "not taken" the result.

[0062] When the next thread's batch begins execution, the terminator node uses another pair of 64-bit bitmaps. This mechanism allows the terminator to return to the first pair of bitmaps even if the thread batches are out of order. If another thread's batch arrives (i.e., at a new address), the terminator updates the least recently used 64-bit bitmap in the NDT.

[0063] For exit branches implemented as indirect branches, the terminator node typically uses the mechanism described above to calculate the target address of the successor DFG at runtime. The terminator node calculates the address of each successor DFG and updates the associated 64-bit bitmap. Once a new 64-bit bitmap is needed (e.g., when the current thread's batch is new, or when the successor DFG is different), the terminator node updates the least recently used 64-bit bitmap in the NDT.

[0064] In some embodiments, the terminator node may additionally receive a corresponding predicate for each thread, i.e., a Boolean value specifying whether the branch is an actual exit point of the DFG. The terminator may receive the predicate in the input token. Since the DFG may have several exit points, in the case of multiple exits, each exit point is typically a predicate. For each thread, only the actual exit point will receive a true predicate, and the other exit points will receive false predicates.

[0065] Upon receiving the final token, the terminator node updates the 64-bit bitmap of all its activities in the NDT (i.e., updates the NDT with the identifier of the successor DFG for each completed thread). Once the NDT update is known to be complete, the terminator node notifies the scheduler of completion by sending the final token. This reporting mechanism allows the scheduler to ensure that all terminators in all replicas of the current DFG have completed the execution of their respective thread batches.

[0066] In this embodiment, instead of using a final token, scheduler 44 counts the number of thread batches sent to the initiator (e.g., the number of bitmaps sent to the initiator) and compares this number with the number of completed thread batches reported by the terminator (e.g., the number of bitmaps reported by the terminator). When the number of sent batches (e.g., bitmaps) equals the number of received batches (e.g., bitmaps), the scheduler determines that the execution of the current DFG has been completed. At this stage, controller 32 can consider the program state as complete (fully defined), save the state, and switch to a new DFG (either the same program / user or a different program / user).

[0067] Concurrent execution of multiple DFGs

[0068] In some embodiments of the invention, the structure 24 of the processor 20 is configured to concurrently execute instructions belonging to multiple different DFGs. For clarity, the embodiments described below primarily relate to the concurrent processing of two DFGs. However, the disclosed techniques can be used in a similar manner to concurrently process a larger number of DFGs.

[0069] Different DFGs can belong to the same software program or to different programs or tasks, such as different database queries. Different programs or tasks can be associated with the same user or different users. When different DFGs belong to the same software program or task, they can share information such as NDTs, live values, and buffer and cache contents. When different DFGs belong to different software programs or tasks, such information is usually maintained separately by each program or task.

[0070] In this context, the phrase "concurrent execution of code instructions specified in multiple different DFGs" refers to structure 24 as a whole, and not necessarily to each individual CN 28 within the structure. In some embodiments, CN 28 progressively switches from one DFG to another. In other embodiments, one or more CN 28 alternate between the execution of instructions specified in two or more different DFGs. All these possibilities are considered herein as concurrent execution of code instructions specified in multiple different DFGs.

[0071] Progressive DFG switching

[0072] In some embodiments, CN 28 in structure 24 progressively switches from the current DFG to the next scheduled DFG. For example, in response to detecting that a given CN 28 has completed its portion of execution across all threads in the current thread set, the given CN 28 can switch to the next DFG. This CN-specific switching reduces the latency overhead caused by starting a new DFG.

[0073] In these embodiments, for different CNs 28, the switching from one DFG to the next DFG is performed at different times. Therefore, during the transition interval, some CNs will still execute the instructions specified in the current DFG, while other CNs have already executed the instructions specified in the next DFG.

[0074] In this context, the term "switch-over time" refers to the time required to reconfigure the CN hardware (e.g., compute components, switches) based on the next DFG. Typically, the configuration settings for the next DFG are provided to the CN in advance and stored in the CN's shadow configuration register.

[0075] In the example embodiment, as described above, the initiator generates a final token after the last batch of threads in the initiating thread set. CN 28 passes the final tokens from one to another through structure 24 according to the data flow specified by the DFG until it reaches the terminator. In some cases, when the data flow is split into multiple paths and subsequently merged, a given CN 28 will receive multiple final tokens. Each CN knows the number of final tokens that should pass through it on its way to the terminator. Therefore, each CN 28 is able to verify whether it has completed its portion of execution across all threads in the current thread set by counting the final tokens that have passed through the CN.

[0076] In some embodiments, given CN 28, it detects that it has forwarded the last final token and, in response, switches to the configuration of the next DFG. This detection and switching is typically independent of any other CN. When CN 28 operates in this manner, the switching to the next DFG typically occurs progressively through structure 24, where CNs closer to the initiator switch earlier and CNs farther from the initiator switch later.

[0077] In such an embodiment, the initiator can begin sending threads to run on the next DFG earlier via the structure, before all CNs have switched (i.e., while some CNs near the terminator are still busy executing the last thread of the previous DFG). This reduces the latency caused by starting a new DFG.

[0078] In one embodiment, once the final token has been verified, the CN's auxiliary logic reconfigures all CN components (e.g., both compute elements and switches) to the configuration of the next DFG. In other embodiments, when some of the final tokens have passed, only the CN is partially reconfigured; for example, only some of the compute elements and switches are reconfigured, rather than all of the switches. For example, once the final token reaches the terminator, the remaining CN components are still reconfigured simultaneously.

[0079] Figure 4 This is a flowchart illustrating a method for concurrent execution of multiple DFGs according to an embodiment of the present invention. The method is described from the perspective of a single CN 28. Typically, different CN 28s execute the method independently of each other.

[0080] At token reading step 100, the method begins with CN reading the next token from its token buffer. At checking step 104, CN checks whether the token is the final token for the current DFG. (If CN expects to receive N final tokens for the current DFG, then CN checks whether the token is the Nth final token.)

[0081] If the token is not the final token (or not the Nth final token, if applicable), then at step 108, CN executes the token. The method then loops back to step 100 to read the next token from the token buffer. (In this context, the phrase "CN executes the token" means that CN is configured to apply the code instructions, executed according to the applicable DFG, to the argument provided in the token.)

[0082] If the token is the final token (or the Nth final token, if applicable), then at token passing step 112, the CN passes the token to the next CN or multiple CNs along the data flow of the current DFG toward the terminator. Then, at CN reconfiguration step 116, the CN reads the configuration of the next DFG from the shadow register and applies the new configuration. The method then loops back to step 100 to read the next token from the token buffer. At this stage, the next token will belong to the newly configured DFG.

[0083] Figure 4 The process described is a simplified example process, depicted for clarity purposes. In alternative embodiments, a gradual switch from one DFG to another can be performed in any other suitable manner.

[0084] In some embodiments, the switching time is determined entirely by each CN 28. In other embodiments, the switching is regulated at least in part by the controller 32. For example, the controller 32 may determine the time to begin switching from the current DFG to the next DFG and send appropriate control signals to the CN 28 in response. In such embodiments, each CN 28 switches to the next DFG after receiving both (i) the last final token of the current DFG and (ii) the control signal from the controller 32.

[0085] Typically, switching CN 28 to a different DFG involves reconfiguring various CN elements, such as computing elements (64, 94) and switches (68, 88). Consider a CN configured to receive multiple final tokens via a switch according to the current DFG. In one embodiment, the CN waits until the last final token passes before reconfiguring the switch according to the next DFG. In an alternative embodiment, the CN attempts to reconfigure the switch as partially as possible after each final token (not just the last one).

[0086] In some embodiments, a CN that is inactive in the next DFG may deactivate itself in response to detection that it has forwarded the last final token (or otherwise detects that it has completed its portion of processing in the current DFG). When using this technique, power consumption is reduced because CNs are powered off independently of each other once they determine that they will not be used in the next DFG.

[0087] In some embodiments, controller 32 identifies CNs 28 that are inactive in the current DFG but need to be active in the next DFG, and activates them. For example, CNs should be treated differently because they do not receive any final tokens from the current DFG. In one embodiment, controller 32 powers on these DFGs once structure 24 begins switching to the next DFG (e.g., when a control signal allowing CNs 28 to switch when they are ready is sent). In another embodiment, controller 32 powers on these DFGs when the configuration for the next DFG is sent to them.

[0088] In any of the above embodiments, the activation and deactivation of CN can be implemented in any suitable manner, for example, by using clock gating or power supply voltage connection / disconnection.

[0089] In practice, two DFGs running concurrently on structure 24 may follow each other in the program's data flow. In this case, controller 32 typically ensures that no thread will begin execution on the next DFG until it has completed execution in the current DFG (e.g., reached the terminator). In one embodiment, as previously described, this condition is guaranteed by resetting the NDT bit of the thread executing on the current DFG. Alternatively, any other suitable means may be used for this purpose.

[0090] As described above, processor 32 can configure structure 24 with multiple copies of a specific DFG running concurrently. In some embodiments, processor 32 configures the structure with both multiple copies of a certain DFG and multiple different DFGs (all running concurrently). Typically, structure 24 can concurrently execute any number of different DFGs, each of which has a single copy or any number of copies.

[0091] Alternation between DFG configurations in CN

[0092] In some embodiments, at least some of CN 28 in structure 24 alternate between the execution of instructions specified in two or more different DFGs. For example, the alternation can be performed from one clock cycle to the next, or at any other suitable time granularity.

[0093] To alternate between N different DFGs, CN 28 typically holds N configurations for the N DFGs in its configuration register. Controller 32 sends a batch of threads associated with all N DFGs being executed concurrently to the initiator (or multiple initiators). The initiator typically sends each thread, along with information about the DFG on which it should be executed, to the relevant CN or multiple CNs. Alternatively, the information about each thread being associated with the appropriate DFG can be provided to the CN in any other suitable manner.

[0094] Typically, CN 28 can hold tokens belonging to various different DFGs in its token buffer. (As mentioned above, tokens typically include parameters provided by the preceding CN in the DFG, which will be used as operands in code instructions executed by the current CN.) The terms "thread" and "token" are sometimes used interchangeably. CNs can schedule threads according to any suitable scheduling criteria serving the various DFGs. In one example embodiment, the CN performs round-robin scheduling among all DFGs with threads ready to execute. Alternatively, the CN can apply appropriate priorities among DFGs with threads ready to execute.

[0095] In some embodiments, the alternation between DFGs is centralized and controlled by controller 32. In these embodiments, controller 32 typically synchronously instructs each CN to switch to a designated DFG in a specific clock cycle, then to another designated DFG in the next clock cycle, and so on. In these embodiments, all CNs execute the same DFG in a given cycle. This centralized control is relatively simple to implement, control, and debug, but it is not optimal in terms of performance.

[0096] In an alternative embodiment, the alternation between DFGs is distributed, allowing each CN 28 to autonomously select the DFG it will execute in each cycle. For example, each CN 28 may include a separate token buffer for each DFG, as well as a separate mechanism for coordinating the receipt of tokens from the preceding CN of each DCF. In this context, separate areas within the same buffer are also considered separate token buffers for different DFGs. The allocation of separate buffer spaces may be predefined or performed dynamically at runtime.

[0097] In one embodiment, for each DFG, the CN sends a "ready" signal to the preceding CN when it is ready to receive a new input token into the corresponding token buffer. (Note that for different DFGs, the same CN 28 can receive input tokens from different preceding CNs, and for different DFGs, different output tokens are sent to different CNs. Therefore, token transmission typically involves reconfiguring the switches of the CNs sending and receiving tokens. Thus, switch reconfiguration should be coordinated between two neighboring CNs, for example, using a "ready" signal.)

[0098] In other words, a CN, for example, uses multiple separate hardware units to carry multiple buffer population processes. Each buffer population process corresponds to a corresponding DFG. Each buffer population process interacts with the preceding CN or multiple CNs, for example, using a corresponding "ready" signal, and coordinates the transfer of input tokens from the preceding CN or multiple CNs to the corresponding token buffer assigned to the corresponding DFG.

[0099] Parallel to the buffer group processes, the CN autonomously schedules the alternation between DFGs using appropriate scheduling criteria. For example, the CN can perform round-robin scheduling among DFGs with non-empty token buffers. Alternatively, the CN can apply appropriate priorities among DFGs with non-empty token buffers.

[0100] Figure 5 This is a flowchart schematically illustrating a method for concurrent execution of multiple DFGs according to an alternative embodiment of the invention. The method is again described from the perspective of a single CN 28. In this simplified example, CN alternates between two DFGs denoted as "DFG X" and "DFG Y".

[0101] The method begins with the CN receiving new tokens from the preceding CNs (or multiple CNs) of DFG X and the preceding CNs (or multiple CNs) of DFG Y at token receiving steps 120 and 124, respectively. For each DFG, the new token is cached in the CN's token buffer.

[0102] At scheduling step 128, CN selects the DFG whose tokens will be executed in the next cycle. Any suitable scheduling criterion can be used for this purpose, such as round-robin scheduling among non-empty token buffers or some kind of priority.

[0103] At token reading step 132, CN reads the token from the token buffer of the selected DFG. At execution step 136, CN executes the read token. The method then loops back to steps 120 and 124 above. As mentioned above, the phrase "CN executes the token" means that CN is configured to apply code instructions, executed according to the applicable DFG, to the parameters provided in the token.

[0104] Figure 5 The process described is a simplified example process, depicted for clarity. In alternative embodiments, the alternation between DFGs can be performed in any other suitable manner.

[0105] In these embodiments, two DFGs running concurrently on structure 24 may also follow each other in the program's data flow. In this case, controller 32 typically ensures that no thread will begin execution on the next DFG until it has completed execution in the current DFG (e.g., reached the terminator). In one embodiment, as previously described, this condition is guaranteed by resetting the NDT bit of the thread executing on the current DFG. Alternatively, any other suitable means may be used for this purpose.

[0106] In some embodiments, processor 32 configures the structure with both multiple copies of a certain DFG and multiple different DFGs (all running concurrently). Typically, structure 24 can execute any number of different DFGs concurrently, each of which has a single copy or any number of copies.

[0107] It should be understood that the above embodiments are cited by way of example, and the invention is not limited to what has been specifically shown and described above. Rather, the scope of the invention includes combinations and sub-combinations of the various features described above, as well as variations and modifications of the invention that would be conceived by one of skill in the art upon reading the above description and that are not disclosed in the prior art. Documents incorporated herein by reference are considered part of this application, and the definitions in this specification should be considered only, unless any terms are defined in these incorporated documents in a manner that conflicts to some extent with the definitions expressly or implicitly made in this specification.

Claims

1. A processor, comprising: A computing architecture comprising an array of computing nodes and an interconnect configurable to connect the computing nodes; as well as The controller is configured to: Receive at least different first data flow graphs (DFGs) and second data flow graphs (DFGs) that each receive specified code instructions; and The computing nodes in the computing structure and at least some of the interconnects are configured to concurrently execute code instructions specified in the first DFG and the second DFG, and multiple first threads executing code instructions specified in the first DFG and multiple second threads executing code instructions specified in the second DFG are sent to the computing structure, such that the computing structure executes both the code instructions specified in the first DFG and the code instructions specified in the second DFG at least during a given time interval.

2. The processor according to claim 1, wherein, At least within a given clock cycle of the computing structure, one or more computing nodes execute code instructions specified in the first DFG, and one or more other computing nodes execute code instructions specified in the second DFG.

3. The processor according to claim 1, wherein, At least during the given time interval, one or more of the computing nodes switch from executing code instructions specified in the first DFG to executing code instructions specified in the second DFG.

4. The processor according to any one of claims 1-3, wherein, The first computing node in the computing structure is configured to switch from the first DFG to the second DFG at a first time, and the second computing node in the computing structure is configured to switch from the first DFG to the second DFG at a second time different from the first time.

5. The processor according to any one of claims 1-3, wherein, A given compute node is configured to switch from the first DFG to the second DFG in response to completing processing of all threads associated with the first DFG.

6. The processor according to any one of claims 1-3, wherein, The controller is configured to send a control signal to one or more of the computing nodes, allowing the computing node to switch from the first DFG to the second DFG, and wherein a given computing node in the computing architecture is configured not to switch from the first DFG to the second DFG before receiving the control signal.

7. The processor according to any one of claims 1-3, wherein, A given compute node that participates in executing the code instructions specified in the first DFG but not in executing the code instructions specified in the second DFG is configured to deactivate itself in response to completing processing of all threads associated with the first DFG.

8. The processor according to any one of claims 1-3, wherein, The controller is configured to activate a given compute node that does not participate in the execution of code instructions specified in the first DFG but participates in the execution of code instructions specified in the second DFG.

9. The processor according to any one of claims 1-3, wherein, The controller is configured to instruct the computing nodes to synchronously alternate between at least the first DFG and the second DFG.

10. The processor according to any one of claims 1-3, wherein, A given compute node is configured to autonomously schedule at least the alternation between the first DFG and the second DFG.

11. The processor according to claim 10, wherein, The given computing node is configured as follows: The first DFG receives the first input token by communicating with one or more first computing nodes in front of the given computing node; The second DFG receives the second input token by communicating with one or more second computing nodes in front of the given computing node; as well as The first input token and the second input token are executed according to the scheduled alternation between the first DFG and the second DFG.

12. A calculation method, comprising: A processor comprising a computing architecture receives at least two distinct first data flow graphs (DFGs) and second data flow graphs (DFGs) that each have specified code instructions, the computing architecture comprising an array of computing nodes and an interconnect configurably connecting the computing nodes; as well as The computing nodes in the computing structure and at least some of the interconnects are configured to concurrently execute code instructions specified in the first DFG and the second DFG, and multiple first threads executing code instructions specified in the first DFG and multiple second threads executing code instructions specified in the second DFG are sent to the computing structure, such that the computing structure executes both the code instructions specified in the first DFG and the code instructions specified in the second DFG at least during a given time interval.

13. The method according to claim 12, wherein, Configuring the computing structure includes: at least during a given clock cycle of the computing structure, causing one or more computing nodes to execute code instructions specified in the first DFG, and causing one or more other computing nodes to execute code instructions specified in the second DFG.

14. The method according to claim 12, wherein, Configuring the computing structure includes: at least during the given time interval, switching one or more of the computing nodes from executing code instructions specified in the first DFG to executing code instructions specified in the second DFG.

15. The method according to any one of claims 12-14, wherein, Configuring the computing structure includes: causing a first computing node in the computing structure to switch from the first DFG to the second DFG at a first time, and causing a second computing node in the computing structure to switch from the first DFG to the second DFG at a second time different from the first time.

16. The method according to any one of claims 12-14, wherein, Configuring the computing structure includes: in a given computing node, switching from the first DFG to the second DFG in response to completing the processing of all threads associated with the first DFG.

17. The method according to any one of claims 12-14, wherein, Configuring the computing structure includes: sending a control signal to one or more of the computing nodes, allowing the computing node to switch from the first DFG to the second DFG, and, for a given computing node in the computing structure, not switching from the first DFG to the second DFG before receiving the control signal.

18. The method of any one of claims 12-14, further comprising, in response to completing processing of all threads associated with the first DFG, deactivating a given compute node that participates in executing code instructions specified in the first DFG but does not participate in executing code instructions specified in the second DFG.

19. The method according to any one of claims 12-14, wherein, Configuring the computing structure includes activating a given computing node that does not participate in the execution of code instructions specified in the first DFG but participates in the execution of code instructions specified in the second DFG.

20. The method according to any one of claims 12-14, wherein, Configuring the computing structure includes instructing the computing nodes to synchronously alternate between at least the first DFG and the second DFG.

21. The method according to any one of claims 12-14, further comprising, autonomously scheduling at least the alternation between the first DFG and the second DFG in a given computing node.

22. The method according to claim 21, wherein, Scheduling the alternation includes performing the following operations on the given compute node: The first DFG receives the first input token by communicating with one or more first computing nodes in front of the given computing node; The second DFG receives the second input token by communicating with one or more second computing nodes in front of the given computing node; as well as The first input token and the second input token are executed according to the scheduled alternation between the first DFG and the second DFG.

Images