Processor and method for privileged configuration in a spatial array

Abstract

Methods and apparatus for privileged configurations in a spatial array are described. In one embodiment, a processor includes: processing elements; an interconnection network between the processing elements; and a configuration controller coupled to a first subset of a plurality of processing elements and a different second subset, the first subset having outputs coupled to inputs of the different second subset, wherein the configuration controller is configured to configure the interconnection network between the first subset of the plurality of processing elements and the different second subset such that communication on the interconnection network between the first subset and the different second subset is not allowed when a privilege bit is set to a first value; and communication on the interconnection network between the first subset of the plurality of processing elements and the different second subset is allowed when the privilege bit is set to a second value.

Description

Technical Field

[0001] The present disclosure is generally related to electronics, and more particularly, embodiments of the present disclosure relate to a configuration controller for privileged configurations in a spatial array. Background Art

[0002] A processor or a collection of processors executes instructions from an instruction set (e.g., an instruction set architecture (ISA)). An instruction set is part of a computer architecture related to programming and generally includes native data types, instructions, register architectures, addressing modes, memory architectures, interrupt and exception handling, and external input and output (I / O). It should be noted that the term "instruction" generally refers to a macro instruction (e.g., an instruction provided to the processor for execution) or a micro instruction (e.g., an instruction generated by decoding a macro instruction by a decoder of the processor) in this document. Brief Description of the Drawings

[0003] The present disclosure is illustrated by way of example and not limitation in the accompanying drawings, in which like reference numerals indicate like elements, wherein:

[0004] Figure 1 Illustrates an accelerator slice according to an embodiment of the present disclosure.

[0005] Figure 2 Illustrates a hardware processor coupled to a memory according to an embodiment of the present disclosure.

[0006] Figure 3 Illustrates a hardware processor slice including an accelerator according to an embodiment of the present disclosure.

[0007] Figure 4 Illustrates an in-flight configuration of an accelerator having multiple processing elements according to an embodiment of the present disclosure.

[0008] Figure 5A Illustrates a configurable data path network according to an embodiment of the present disclosure.

[0009] Figure 5B Illustrates a configurable flow control path network according to an embodiment of the present disclosure.

[0010] Figure 6 Illustrates a flowchart according to an embodiment of the present disclosure.

[0011] Figure 7 Illustrates a context switch in a spatial array of processing elements of a processor according to an embodiment of the present disclosure.

[0012] Figures 8A - 8D Illustrates an in-flight configuration for context switching of a spatial array of processing elements according to an embodiment of the present disclosure.

[0013] Figures 9A - 9J Illustrates the staged extraction of the context of a spatial array of processing elements configured to execute a data flow graph, according to an embodiment of the present disclosure.

[0014] Figure 10A Illustrates an extracted state, according to an embodiment of the present disclosure.

[0015] Figure 10B Illustrates the state at the start of extraction, according to an embodiment of the present disclosure.

[0016] Figure 11 Illustrates a state machine for a controller (e.g., configured), according to an embodiment of the present disclosure.

[0017] Figure 12A Illustrates the extraction of the context of a spatial array of processing elements, according to an embodiment of the present disclosure.

[0018] Figure 12B Illustrates the extraction of the context of a spatial array of processing elements, according to an embodiment of the present disclosure.

[0019] Figure 13 Illustrates the staged extraction of the context of a spatial array of processing elements that includes an (e.g., mezzanine or global) network therebetween, according to an embodiment of the present disclosure.

[0020] Figure 14 Illustrates the staged extraction of the context of a spatial array of processing elements that includes memory access, according to an embodiment of the present disclosure.

[0021] Figure 15A Illustrates the extraction of the context of a spatial array of processing elements, according to an embodiment of the present disclosure.

[0022] Figure 15B Illustrates the extraction of the context of a spatial array of processing elements, according to an embodiment of the present disclosure.

[0023] Figure 16 Illustrates a flow chart, according to an embodiment of the present disclosure.

[0024] Figure 17 Illustrates a flow chart, according to an embodiment of the present disclosure.

[0025] Figure 18A Illustrates a program source, according to an embodiment of the present disclosure.

[0026] Figure 18B Illustrates, according to an embodiment of the present disclosure, for Figure 18A the data flow graph of the program source.

[0027] Figure 18C The figure shows an accelerator according to an embodiment of the present disclosure, the accelerator having a plurality of processing elements configured to execute Figure 18B the data flow graph.

[0028] Figure 19 The figure shows an example execution of a data flow graph according to an embodiment of the present disclosure.

[0029] Figure 20 The figure shows a program source according to an embodiment of the present disclosure.

[0030] Figure 21 The figure shows an accelerator slice according to an embodiment of the present disclosure, the accelerator slice including an array of processing elements.

[0031] Figure 22A The figure shows a configurable data path network according to an embodiment of the present disclosure.

[0032] Figure 22B The figure shows a configurable flow control path network according to an embodiment of the present disclosure.

[0033] Figure 23 The figure shows a hardware processor slice including an accelerator according to an embodiment of the present disclosure.

[0034] Figure 24 The figure shows a processing element according to an embodiment of the present disclosure.

[0035] Figure 25 The figure shows a request address file (RAF) circuit according to an embodiment of the present disclosure.

[0036] Figure 26 The figure shows a plurality of request address file (RAF) circuits coupled between a plurality of accelerator slices and a plurality of cache blocks according to an embodiment of the present disclosure.

[0037] Figure 27 The figure shows a floating-point multiplier partitioned into three regions (a result region, three potential carry regions, and a gated region) according to an embodiment of the present disclosure.

[0038] Figure 28 The figure shows a runtime configuration of an accelerator having a plurality of processing elements according to an embodiment of the present disclosure.

[0039] Figure 29 The figure shows a snapshot of a runtime pipelined fetch according to an embodiment of the present disclosure.

[0040] Figure 30 The figure shows a compilation toolchain for an accelerator according to an embodiment of the present disclosure.

[0041] Figure 31Illustrates a compiler for an accelerator according to an embodiment of the present disclosure.

[0042] Figure 32A Illustrates a serialized assembly code according to an embodiment of the present disclosure.

[0043] Figure 32B Illustrates a data flow assembly code for the Figure 32A serialized assembly code according to an embodiment of the present disclosure.

[0044] Figure 32C Illustrates a data flow graph for the data flow assembly code for an accelerator according to an embodiment of the present disclosure. Figure 32B

[0045] Figure 33A Illustrates a C source code according to an embodiment of the present disclosure.

[0046] Figure 33B Illustrates a data flow assembly code for the Figure 33A C source code according to an embodiment of the present disclosure.

[0047] Figure 3 Illustrates a data flow graph for the data flow assembly code for an accelerator according to an embodiment of the present disclosure. ​

[0048] ​ Illustrates a C source code according to an embodiment of the present disclosure.

[0049] ​ Illustrates a data flow assembly code for the ​ C source code according to an embodiment of the present disclosure.

[0050] ​ Illustrates a data flow graph for the data flow assembly code for an accelerator according to an embodiment of the present disclosure. ​

[0051] ​ Illustrates a flowchart according to an embodiment of the present disclosure.

[0052] ​ Illustrates a flowchart according to an embodiment of the present disclosure.

[0053] ​ Illustrates a graph of throughput versus energy per operation according to an embodiment of the present disclosure.

[0054] ​ Illustrates an accelerator chip according to an embodiment of the present disclosure, the accelerator chip including an array of processing elements and a local configuration controller.

[0055] ​ ​​​Illustrates a local configuration controller for a configuration data path network according to an embodiment of the present disclosure.

[0056] ​ Illustrates a configuration controller according to an embodiment of the present disclosure.

[0057] ​ Illustrates an accelerator slice according to an embodiment of the present disclosure, the accelerator slice including an array of processing elements, a configuration cache, and a local configuration controller.

[0058] ​ Illustrates an accelerator slice according to an embodiment of the present disclosure, the accelerator slice including an array of processing elements and a configuration and exception handling controller having a reconfiguration circuit.

[0059] ​ Illustrates a reconfiguration circuit according to an embodiment of the present disclosure.

[0060] ​ Illustrates an accelerator slice according to an embodiment of the present disclosure, the accelerator slice including an array of processing elements and a configuration and exception handling controller having a reconfiguration circuit.

[0061] ​ Illustrates an accelerator slice according to an embodiment of the present disclosure, the accelerator slice including an array of processing elements and a mezzanine exception aggregator coupled to a chip-level exception aggregator.

[0062] ​ Illustrates a processing element having an exception generator according to an embodiment of the present disclosure.

[0063] ​ Illustrates an accelerator slice according to an embodiment of the present disclosure, the accelerator slice including an array of processing elements and a local extraction controller.

[0064] ​ Illustrates a local extraction controller for a configuration data path network according to an embodiment of the present disclosure.

[0065] ​ Illustrates an extraction controller according to an embodiment of the present disclosure.

[0066] ​ Illustrates a flowchart according to an embodiment of the present disclosure.

[0067] ​ Illustrates a flowchart according to an embodiment of the present disclosure.

[0068] ​ Is a block diagram illustrating a general vector friendly instruction format and its class A instruction template according to an embodiment of the present disclosure.

[0069] ​FIG. is a block diagram showing a general vector friendly instruction format and its B - type instruction template according to an embodiment of the present disclosure.

[0070] ​ FIG. is for showing according to an embodiment of the present disclosure ​ and ​ a block diagram of fields of the general vector friendly instruction format in

[0071] ​ FIG. is a block diagram showing fields of a dedicated vector friendly instruction format in ​ that constitute a complete opcode field according to an embodiment of the present disclosure.

[0072] ​ FIG. is a block diagram showing fields of a dedicated vector friendly instruction format in ​ that constitute a register index field according to an embodiment of the present disclosure.

[0073] ​ FIG. is a block diagram showing fields of a dedicated vector friendly instruction format in ​ that constitute an extended operation field 5150 according to an embodiment of the present disclosure.

[0074] ​ FIG. is a block diagram of a register architecture according to an embodiment of the present disclosure.

[0075] ​ FIG. is a block diagram showing both an exemplary in - order pipeline and an exemplary out - of - order issue / execution pipeline with register renaming according to an embodiment of the present disclosure.

[0076] ​ FIG. is a block diagram showing both an exemplary in - order architecture core to be included in a processor and an exemplary out - of - order issue / execution architecture core with register renaming according to an embodiment of the present disclosure.

[0077] ​ FIG. is a block diagram of a single processor core according to an embodiment of the present disclosure, its connection to an on - die interconnect network, and a local subset of its level 2 (L2) cache.

[0078] ​ FIG. is according to an embodiment of the present disclosure ​ an expanded view of a part of the processor core in

[0079] ​ FIG. is a block diagram of a processor that may have more than one core, may have an integrated memory controller, and may have an integrated graphics device according to an embodiment of the present disclosure.

[0080] ​ FIG. is a block diagram of a system according to an embodiment of the present disclosure.

[0081] ​ is a block diagram of a more specific exemplary system according to an embodiment of the present disclosure.

[0082] ​ Shown is a block diagram of a second, more specific exemplary system according to an embodiment of the present disclosure.

[0083] ​ Shown is a block diagram of a system on a chip (SoC) according to an embodiment of the present disclosure.

[0084] ​ is a block diagram illustrating converting binary instructions in a source instruction set into binary instructions in a target instruction set using a software instruction converter according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0085] In the following description, a number of specific details are set forth. However, it should be understood that embodiments of the present disclosure may be implemented without these specific details. In other instances, well-known circuits, structures, and technologies are not shown in detail to avoid obscuring the understanding of this description.

[0086] References in the specification to "one embodiment," "an embodiment," "an example embodiment," and the like indicate that the described embodiment may include a particular feature, structure, or characteristic, but not every embodiment necessarily includes that particular feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it is understood that it is within the knowledge of those skilled in the art to be able to affect such feature, structure, or characteristic in conjunction with other embodiments, whether or not explicitly described.

[0087] A processor (e.g., having one or more cores) may execute instructions (e.g., an instruction thread) to operate on data, for example, to perform arithmetic, logical, or other functions. For example, software may request an operation, and a hardware processor (e.g., one or more cores of the hardware processor) may perform the operation in response to the request. A non-limiting example of an operation is a blend operation that inputs multiple vector elements and outputs a vector having the blended multiple elements. In some embodiments, multiple operations are performed using the execution of a single instruction.

[0088] For example, exascale performance as defined by the U.S. Department of Energy may require system-level floating-point performance exceeding 10 Mbps within a given (e.g., 20 MW) power budget. 18Certain embodiments herein relate to spatial arrays of processing elements (e.g., configurable spatial accelerators (CSAs)) for high performance computing (HPC), such as processors. Certain embodiments herein of spatial arrays of processing elements (e.g., CSAs) are directed to direct execution of a dataflow graph (or multiple dataflow graphs) to achieve a computationally intensive yet energy-efficient spatial microarchitecture that far exceeds conventional roadmap architectures.

[0089] Certain embodiments of spatial architectures (e.g., the spatial arrays disclosed herein) are energy-efficient and high-performance methods for accelerating user applications. In some embodiments, a spatial array (e.g., a plurality of processing elements coupled together by a (e.g., circuit-switched) (e.g., interconnect) network) is used to accelerate an application, for example, to execute a region of a single stream program (e.g., faster than a core of a processor). In some embodiments, a measure of the effectiveness of a spatial architecture is the speed with which a region (e.g., to be accelerated) can be accelerated into the spatial architecture, e.g., the longer it takes to load the region, the larger the region can be in order to amortize the cost of loading the program. Conversely, where configuration time is short, smaller program regions can be accelerated, for example, expanding the applicability of a spatial architecture (e.g., an accelerator).

[0090] Certain embodiments herein provide hardware and techniques for providing privileged configurations in spatial arrays (eg, spatial structures).

[0091] Certain embodiments herein provide hardware and techniques for pipeline configuration in a spatial array (e.g., a spatial structure). Certain embodiments herein utilize (e.g., regional) control (e.g., a configuration controller) and (e.g., low-level) dataflow semantics of a spatial array (e.g., a configurable spatial array (CSA)) to produce a pipelined configuration effect that enables previously (e.g., early) configured (e.g., processing) elements of the spatial array to begin operation (e.g., immediately) before the entire (e.g., segment) of the spatial array is configured. Certain embodiments herein may reduce the effective latency of configuration to tens of nanoseconds. In one embodiment, configuration may be two (e.g., separate) operations: for example, the actual configuration and the (e.g., simultaneous) extraction of a previous configuration (e.g., the state of the previous configuration) loaded into the spatial array (e.g., the structure), which operations may occur during a context switch. Certain embodiments herein may allow these operations to occur simultaneously within the spatial array. Certain embodiments herein may use (multiple) microprotocols for configuration and extraction, for example, as described below with reference to ​ 、 ​ as well as ​ discussed.

[0092] Certain embodiments herein provide techniques and hardware (e.g., microarchitectural extensions and / or definitions) for pipelining the configuration and / or extraction operations of a spatial array. Certain embodiments herein utilize one or more controllers to schedule wavefronts across the configuration and extraction regions of the spatial array. Certain embodiments herein utilize a higher-level (e.g., configuration and / or extraction) controller to schedule local-level controllers, which implements wavefronts across the configuration and extraction regions of the spatial array. In one embodiment, the wavefront logically separates new and old (e.g., program) contexts, which enables, for example, the new context to be executed immediately. Certain embodiments herein transform an object that was previously a serial process (e.g., extraction, followed by configuration) into a pipelined process, thereby, for example, reducing the latency by an order of magnitude.

[0093] Certain embodiments herein reduce the amount of time required to configure (and / or extract) a spatial accelerator, thereby, for example, enabling beneficial acceleration of smaller code regions. As a result, the performance of more programs can be improved, and the performance of more programs can be improved to a greater extent.

[0094] Also included below is a description of the architectural concept of embodiments of a spatial array of processing elements (e.g., CSA) and certain of its features. As with any revolutionary architecture, programmability can be a risk. To mitigate this issue, embodiments of the CSA architecture have been co-designed with a compilation toolchain (which is also discussed below).

[0095] ​

[0096] Exascale computing goals may require huge system-level floating-point performance (e.g., 1 ExaFLOP) within a stringent power budget (e.g., 20 MW). However, it has become difficult to improve both the performance and energy efficiency of program execution using classical von Neumann architectures: out-of-order scheduling, simultaneous multithreading operations, complex register files, and other constructs provide performance but at a high energy cost. Certain embodiments herein achieve both the performance and energy requirements. Exascale computing power-performance goals may require both high throughput and low energy consumption per operation. Certain embodiments herein provide this by providing a large number of low-complexity, energy-efficient processing (e.g., computing) elements that greatly eliminate the control overhead of previous processor designs. Guided by this observation, certain embodiments herein include a spatial array of processing elements, e.g., a configurable spatial accelerator (CSA), which, for example, includes an array of processing elements (PEs) connected by a lightweight backpressure (e.g., communication) network. An example of a CSA tile is in ​Depicted in. Some embodiments of processing (e.g., computing) elements are data flow operators, e.g., multiple data flow operators that process input data only when both (i) the input data has arrived at the data flow operator and (ii) there is space available for storing the output data (e.g., otherwise no processing is occurring). Some embodiments (e.g., of an accelerator or CSA) do not utilize triggered instructions.

[0097] Such as ​ The coarse-grained spatial architecture of embodiments of a configurable spatial accelerator (CAS), such as shown in, is a composition of lightweight processing elements (PEs) connected by an interconnection network. A program, e.g., considered as a control flow graph, can be mapped onto the architecture by configuring the PEs and the network. In general, a PE can be configured as a data flow operator, e.g., once all input operands have arrived at the PE, an operation occurs and the result is forwarded downstream (to, e.g., (multiple) destination PEs) in a pipelined manner. A data flow operator can optionally consume incoming data on a per-operator basis.

[0098] Both the space architecture defense system program (e.g., operating system (OS)) (e.g., system-level privilege) and user programs (e.g., user-level privilege) can be required to be protected from malicious code. In one embodiment, a circuit-switched interconnection network multiplexes the space architecture among a number of user programs. Multiple portions of the space structure (e.g., circuit-switched interconnection network) can be given to different user-level programs for exclusive use by these user-level programs. However, supporting such multiplexing among multiple programs (e.g., multiple threads of each program) introduces system-level problems: for example, programs are protected from affecting each other, such that a program cannot observe the operations of other programs residing in the same structure (e.g., circuit-switched interconnection network), nor will it interfere with the operations of other programs residing in the same structure (e.g., circuit-switched interconnection network). Such operations can occur as part of a privileged program such as an operating system (e.g., system-level). Certain embodiments of the present disclosure provide direct user-level configuration of the structure (e.g., circuit-switched interconnection network), which can significantly reduce the configuration overhead. In certain embodiments, the space array (e.g., its circuit-switched interconnection network) of (e.g., CSA) includes a privileged configuration (e.g., privileged mode) to load, for example, configuration bits that are protected from user-level modification. Certain embodiments herein utilize, for example, privileged control circuits for implementing protection mechanisms in the space structure to augment the space structure. For example, a privileged configuration circuit can be used to load a process identification value (process ID) into the structure (e.g., memory interface). Certain embodiments herein allow partitioning of the structure (CSA) using privileged bits in the structure (e.g., in the interconnection itself) (e.g., using the privileged mode of the configuration controller). Certain embodiments herein provide support for privileges (e.g., privileged configuration). A configuration controller that is set or set to a privileged mode can enable a privileged program such as an operating system to set one or more special configuration bits in the structure (e.g., circuit-switched interconnection network), and these one or more special configuration bits control, for example, higher-level functions (such as protection). Certain embodiments herein also implement protection in a shared structure. For example, certain embodiments herein provide a privileged mode to set regions of logic (e.g., data does not cross) that are hardened as hardware (e.g., configuration controller and structure) to prevent, for example, signals from crossing between multiple user-level program regions, which is, for example, in contrast to soft protection configured as a field programmable gate array (FPGA). Certain embodiments herein relate to a space architecture that uses special hardware (e.g., configuration (e.g., controller) circuits) to provide protection. Certain embodiments herein do not add significant performance overhead to the configuration time. Certain embodiments herein do not require programming to occur in a privileged context even, for example, when the configuration has direct access to physical memory.Some embodiments herein separate the user-visible architecture from the privileged architecture in order to, for example, minimize the trusted code base and permit true user-level reconfiguration. Some embodiments herein provide an architecture for separating these configurations. Another benefit of this separation may be to enable direct user-level fabric configuration, which, for example, reduces configuration overhead. Some embodiments herein relate to CSA hardware that defends itself and benign user programs against malicious user programs. Some embodiments herein allow a single configuration step to set up the fabric (e.g., a circuit-switched interconnect network) for subsequent configurations (e.g., all), thereby greatly reducing the protection overhead in terms of configuration time. Some embodiments herein distinguish between user-level configuration and privileged configuration. Some embodiments herein provide a hardware-protected privileged configuration that is not visible to user programs at any given time. Some embodiments herein allow a spatial array (e.g., a CSA) to safely participate within a shared memory system using a privileged mode. Some embodiments herein allow a spatial array (e.g., a CSA) to use a privileged mode to protect multiple portions of the configuration of the spatial array (e.g., configured according to a data flow graph) and / or enable user-level configuration of other portions of the spatial array, thereby, for example, reducing the latency of these operations.

[0099] Additionally or alternatively, some embodiments herein extend the capabilities of a spatial array (e.g., a CSA) to include pipelined configuration(s) and extraction(s), which, for example, contrasts with these operations being serialized. Some operators (such as those that handle unconditional evaluation of arithmetic expressions) may consume all incoming data. However, it is sometimes useful for an operator to maintain a state (e.g., an execution context) in an accumulative operation. In one embodiment, the current state is saved prior to an execution context switch.

[0100] A context switch generally may refer to the process of storing and restoring the state (more specifically, the execution context) of an operation (e.g., a process or a thread) such that execution can be resumed at the same point later. Some embodiments herein extend the capabilities of a spatial array (e.g., a CSA) to include pipelined configuration(s) and extraction(s), which, for example, contrasts with these operations being serialized. Some embodiments herein reduce the perceived latency of context switches (e.g., from hundreds of cycles to an effective latency of dozens of cycles, e.g., an order-of-magnitude reduction)

[0101] ​ An accelerator slice 100 embodiment of a spatial array of processing elements according to an embodiment of the present disclosure is illustrated. The accelerator slice 100 may be part of a larger slice. The accelerator slice 100 executes one or more data flow graphs. The data flow ​Generally, it can refer to an explicitly parallel program description that occurs during the compilation of serialized code. Some embodiments herein (e.g., CSA) allow a data flow graph to be directly configured onto a CSA array, e.g., rather than being transformed into a serialized instruction stream. Some embodiments herein allow a first (e.g., type of) data flow operation to be performed by one or more processing elements (PEs) of a spatial array, and additionally or alternatively, allow a second (e.g., different type of) data flow operation to be performed by one or more of the network communication circuits (e.g., endpoints) of the spatial array.

[0102] The deviation of the data flow graph from the serialized compilation stream allows embodiments of CSA to support familiar programming models and directly execute existing high-performance computing (HPC) code (e.g., without using work sheets). The CSA processing element (PE) can be energy-efficient. In ​ it, the memory interface 102 can be coupled to a memory (e.g., ​ the memory 202 in

[0103] to allow the accelerator die 100 to access (e.g., load and / or store) data from / to a (e.g., off-die) memory. The depicted accelerator die 100 is a heterogeneous array composed of several types of PEs coupled together via an interconnect network 105. The accelerator die 100 can include, for example, as part of a spatial array of processing elements 101, one or more of the following: integer arithmetic PEs, floating-point arithmetic PEs, communication circuits (e.g., network data flow endpoint circuits), and structural storage. A data flow graph (e.g., a compiled data flow graph) can be overlaid on the accelerator die 100 for execution. In one embodiment, for a particular data flow graph, each PE only handles one or two (e.g., data flow) operations in the graph. The PE array can be heterogeneous, e.g., such that no PE supports the full CSA data flow architecture and / or one or more PEs are programmed (e.g., customized) to perform only some but highly efficient operations. Some embodiments herein thus implement a processor or accelerator with an array of processing elements that are more computationally intensive than roadmap architectures and achieve approximately an order-of-magnitude gain in energy efficiency and performance relative to existing HPC offerings.Certain embodiments herein provide performance improvements from parallel execution within a (e.g., dense) spatial array of processing elements (e.g., CSA), where, for example, if input data is available, each PE utilized can perform its operation simultaneously. The efficiency improvements can stem from the efficiency of each PE, e.g., where the operation (e.g., behavior) of each PE is fixed once for each configuration (e.g., mapping) step and execution occurs when local data arrives at the PE (e.g., regardless of other structural activity). In certain embodiments, the PE is a data flow operator (e.g., each PE is a single data flow operator), e.g., a data flow operator that operates on input data only when both (i) the input data has arrived at the data flow operator and (ii) there is space available for storing the output data (e.g., otherwise no processing is occurring).

[0104] Certain embodiments herein include a spatial array of processing elements as an energy - efficient and high - performance method for accelerating user applications. In one embodiment, the (multiple) spatial arrays are configured via a serial process in which the configuration latency is fully exposed via a global reset. Some of this can stem from the register transfer level (RTL) semantics of the array (e.g., field - programmable gate array (FPGA)). Programs for execution on the array (e.g., FPGA) can assume a fundamental reset concept according to which every part of the desired design is to operate out of a configuration reset. Certain embodiments herein provide a data - streaming array in which the PEs (e.g., all PEs) conform to a stream - controller micro - protocol. This micro - protocol can have the effect of distributed initialization. The micro - protocol can, for example, allow pipelined configuration and extraction mechanisms using a regional (e.g., non - whole - column) arrangement. Certain embodiments herein provide context switching in a data - flow architecture.

[0105] The depicted accelerator tile 100 includes (e.g., tile-level) configuration controller 104, which (e.g., tile-level) controller 104 is used, for example, to configure one or more of the processing elements (PEs) and / or the network (e.g., interconnect network 105) between multiple PEs according to an input data flow graph. Additionally or alternatively, accelerator tile 100 includes one or more (e.g., local) configuration controllers 106, 108. For example, each local configuration controller may configure a (e.g., corresponding) subset of the processing elements and / or the network (e.g., input and / or output to that subset of the processing elements). Each local (e.g., configuration) controller may operate independently. In one embodiment, the configuration controller includes the ability to set the privilege level for (e.g., for some PEs (or all PEs) in the PEs) and the network between the PEs. In one embodiment, the configuration controller includes the ability to extract, e.g., an extraction controller. In one embodiment, a configuration controller and a separate extraction controller are used. In one embodiment, the local controllers are located on a network through which these local controllers communicate, for example, with a higher-level control hierarchy, memory, and / or with each other via ​ the network in the dashed box in

[0106] In one embodiment, the local (e.g., configuration) controller is responsible for setting the privilege level of components (e.g., a circuit-switched interconnect network) and / or for the (e.g., low-level) operation of runtime services, such as (e.g., PE) configuration and extraction. Each local controller may be responsible for a subset of the PEs (e.g., a small number of PEs), as ​ and ​ illustrated in ​ The PEs can be divided into two subsets, e.g., as managed by configuration controller 106 and configuration controller 108 respectively.

[0107] As shown below, the execution plan for pipelined services may include three steps: configuration (e.g., and setting privilege levels), buffering, and extraction. Similarly, the control hardware (e.g., (multiple) controllers) needs to have knowledge of and coordinate these three steps. An example control flow is as follows: Each local controller may contain a list of those controllers physically adjacent to it. When a local controller receives a message from each of those local controllers that precede it, a context (e.g., state) transition may begin. Subsequently, that local controller can start its current operation. When the operation is complete, the local controller may transition its context (e.g., state) and send a message to each succeeding controller. In one embodiment, the local controller follows four states: Run, Extract, Inactive, and Configure. Inactive may be obtained, for example, by starting a Configure microprotocol, which deactivates the PE but may not immediately supply configuration information, e.g., thereby keeping the PE in a deactivated state.

[0108] Pipelined runtime services may include coordination between a higher-level (e.g., chip-level) controller and local controllers responsible for configuration. To reduce this communication time and improve pipeline behavior, some embodiments herein include a microarchitecture for supporting direct forwarding of commands (e.g., configuration, extraction, and / or completion) between local controllers. This may allow the (multiple) higher-level controllers to overlay a coordinated configuration and extraction graph on top of the local controllers, which may be used to dynamically construct a wavefront.

[0109] Some embodiments herein provide a paradigm-shifting level of performance and a significant improvement in energy efficiency across a broad class of existing single-stream and parallel programs (e.g., all programs), while maintaining a familiar HPC programming model. Some embodiments herein may be targeted at HPC such that floating-point energy efficiency is extremely important. Some embodiments herein not only achieve remarkable performance improvements and energy reductions, but also pass these gains on to existing HPC programs written in mainstream HPC languages and for mainstream HPC frameworks. Some embodiments of the architecture herein (e.g., with thoughtful compilation) provide several extensions in direct support of the control data flow internal representation generated by modern compilers. Some embodiments herein relate to a CSA data flow compiler (e.g., which may accept C, C++, and Fortran programming languages) for a CSA architecture.

[0110] ​FIG. illustrates a hardware processor 200 coupled to (e.g., connected to) a memory 202 according to an embodiment of the present disclosure. In one embodiment, the hardware processor 200 and the memory 202 are a computing system 201. In certain embodiments, one or more of the accelerators are CSAs according to the present disclosure. In certain embodiments, one or more of the cores in the processor are the cores disclosed herein. The hardware processor 200 (e.g., each of its cores) may include a hardware decoder (e.g., decoding unit) and a hardware execution unit. The hardware processor 200 may include registers. Note that the drawings herein may not depict all data communication couplings (e.g., connections). Those skilled in the art will recognize that this is done so as not to obscure certain details in the drawings. Note that the two-way arrows in the drawings may not require two-way communication, e.g., it may indicate one-way communication (e.g., to or from that component or device). Any path or all combinations of paths in the communication paths may be used in certain embodiments herein. The depicted hardware processor 200 includes a plurality of cores (0 to N, where N may be 1 or greater) and hardware accelerators (0 to M, where M may be 1 or greater) according to the present disclosure. The hardware processor 200 (e.g., its (multiple) accelerators and / or (multiple) cores) may be coupled to the memory 202 (e.g., data storage device). The hardware decoder (e.g., of the core) may receive (e.g., a single) instruction (e.g., a macro instruction) and decode the instruction into, e.g., microinstructions and / or micro-operations. The hardware execution unit (e.g., of the core) may execute the decoded instruction (e.g., a macro instruction) to perform one or more operations.

[0111] Section 1 below discusses privilege configuration and the configuration and (e.g., simultaneous) extraction of previous configurations (e.g., their states). Section 2 below discloses embodiments of the CSA architecture. Specifically, novel embodiments of integrating memory within a data flow execution model are disclosed. Section 3 explores the microarchitecture details of embodiments of the CSA. In one embodiment, the main purpose of the CSA is to support compiler-generated programs. Section 4 below examines embodiments of the CSA compilation toolchain. In Section 5, the advantages of embodiments of the CSA are compared with other architectures in the execution of compiled code. Finally, the performance of embodiments of the CSA microarchitecture is discussed in Section 6, further CSA details are discussed in Section 7, and a summary is provided in Section 8.

[0112] ​

[0113] In some embodiments, as discussed herein, processing elements (PEs) communicate using dedicated virtual circuits formed, for example, by statically configuring (e.g., circuit-switched) communication networks. These virtual circuits can be flow-controlled and fully backpressureable, such that, for example, if the source has no data or the destination of the PE is full, the PE will stop. At runtime, data can flow through PEs that implement a mapped dataflow graph (e.g., a mapped algorithm). For example, data can flow into the processing element from memory through a spatial array (e.g., a structural region thereof), and then flow back out to memory.

[0114] Such architectures can achieve excellent performance efficiency relative to traditional multi-core processors: for example, computing in the form of PEs can be simpler and more numerous than cores, and, for example, in contrast to the scaling of memory systems, communication can be direct. In some embodiments, the spatial array (e.g., a CSA) includes privileged configuration modes (e.g., setting components to a first mode or a second mode). These modes can be invoked by a privileged program such as an operating system to modify the security mode within the spatial array (e.g., the circuit-switched interconnect network of the spatial array), thereby, for example, setting dynamic partitions in the spatial array in a multi-programming scenario. To configure the spatial array to the first mode or the second mode, the privileged program can use a privileged interface, e.g., a memory-mapped interface (e.g., a configuration space register (e.g., CSR)); or decode an instruction into a decoded instruction and execute the decoded instruction to cause the spatial array to enter or leave the privileged mode (e.g., decode and execute the instruction on a processor core coupled to the CSA). The privileged interface can be protected, for example, using a virtual memory translation mechanism or through the privileged mode of the processor (e.g., a core). When invoking the privileged mode, the configuration controller (e.g., the chip-level configuration controller 104) can send an appropriate configuration message to local configuration controllers (e.g., configuration controller 106 and configuration controller 108), which can then invoke privileged configuration on their sub-regions. Depending on the choice of microprotocol (e.g., described below), the memory interface 102 can use an additional virtual memory translation mechanism to support any privileged address space used by the calling program. In one embodiment, to support privileged configuration, the configuration microprotocol is extended to include a control channel: CFG_START_PRIVILEGE, as discussed below with reference to ​ discussed.

[0115] The signal on the CFG_START_PRIVILEGE channel can indicate (e.g., to the PE and / or the interconnect network (e.g., a circuit-switched network)) the start of privileged mode (e.g., for a privileged configuration) and / or indicate to privileged bit stores (e.g., in privileged registers) that they will accept incoming values according to the configuration micro-protocol. In other respects, it can be like the signal on the CFG_START channel. Given these concepts, it is possible to implement several types of privileged configurations.

[0116] ​

[0117] In some embodiments, privileged configuration (e.g., privileged mode) is separate from user mode configuration. Configured control bits are used in the configuration microprotocol to, for example, indicate privileged mode or non-privileged mode. Some embodiments assert a value on the CFG_START_PRIVILEGE channel to indicate that it is one or more privileged bits (e.g., one or more privileged configuration bits) that are being modified. ​ , may instruct (e.g., via a privileged interface, e.g., as discussed herein) one or more of the configuration controllers (e.g., storage therein) to set the accelerator slice 100 (e.g., the PEs and / or the interconnect network 105) to a privileged mode or to a non-privileged mode, e.g., to update one or more privileged bits (e.g., one or more privileged configuration bits) when in privileged mode.

[0118] In one embodiment, privileged configuration is non-destructive to existing configuration bits (e.g., configuration for a data flow graph) other than the privilege bit(s) used to indicate the first mode or the second mode. In one embodiment, configuration to a new mode (e.g., between a first mode that uses one or more privileged bits to block communications on a network and a second mode that does not use one or more privileged bits to block communications on a network) is called at runtime. In one embodiment, when configuration mode is entered, a local network multiplexer (e.g., such as ​ In some embodiments, no additional configuration is required if the local multiplexer state is preserved, e.g., once the multiplexer is used in configuration (e.g., between the configuration controller and (e.g., storing operational configuration values in, e.g., ​Once the operations (e.g., providing a path between PEs in the operation configuration register 2419) are completed, they return to their original configuration. In some embodiments, it is possible to set privilege bits without reconfiguring and / or stopping a running (e.g., user) program. In another embodiment, privilege configuration (e.g., setting privilege mode) includes placing the CSA (e.g., accelerator tile) in a no-operation (e.g., no-op) state before attempting to configure it. For example, a processor (e.g., running an OS) may request that the CSA place some or all of the CSA (e.g., PEs and / or interconnect) in privilege mode before passing a privilege task out to the CSA (e.g., accelerator tile) for execution by the CSA.

[0119] ​

[0120] In one embodiment, privilege configuration occurs separately from user configuration. In this case, the user part of the privilege configuration can be a no-operation (no-op) configuration. During subsequent user configuration, the privilege configuration (e.g., set to privilege mode) is not used, and the configuration associated with the privilege bits is either ignored or not included in the (e.g., binary) user configuration format. In the latter case, the privilege configuration state may include a status bit for automatically bypassing its configuration.

[0121] ​ FIG. illustrates a hardware processor tile 300 including an accelerator 302 according to an embodiment of the present disclosure. The accelerator 302 may be a CSA according to the present disclosure. The tile 300 includes a plurality of cache blocks (e.g., cache block 308). A Request Address File (RAF) circuit 310 may be included, for example, as discussed in Section 3.2 below. ODI may refer to on-die interconnect, e.g., an interconnect that stretches across the entire die and connects all tiles. OTI may refer to on-tile interconnect (e.g., stretching across the tile, e.g., connecting the cache blocks on the tile together). The CSA may allow privilege configuration, e.g., in which (multiple) configuration bits (e.g., (multiple) privilege bits) are used to set a boundary where data does not cross between two user applications (e.g., Process ID zero (PID 0) and Process ID one (PID1)) within the accelerator (e.g., CSA) 302 and / or RAF 310. A wavy line is included to generally indicate the boundary, and the actual boundary between regions (e.g., applications) may not follow the exact pattern.

[0122] ​

[0123] In some embodiments, the protection (e.g., in privileged mode) includes defense against potentially shared resources. In a processor, the protection can be restricted to a state, and more specifically, to the memory subsystem. The protection can be implemented by a virtual memory translation mechanism. However, in addition to memory protection, a spatial array (e.g., a CSA) may also require protection of communications within the structure itself (e.g., a circuit-switched interconnect network). In ​ and ​ , by way of example, a local circuit-switched communication network (e.g., ​ the first network including switching device 2110 and the second network including switching device 2111 therein) can span a die or even an entire chip. If two (e.g., user) programs share the same die or chip, either of the two programs (e.g., running on a spatial array) can observe the communication of the other, for example, by sampling a shared communication channel or shared resources.

[0124] To implement protection, some embodiments herein provide privilege (e.g., privileged state) bits at certain points in the structure (e.g., a circuit-switched interconnect network) that correspond to partitionable regions. The partitionable regions can be coarse or fine, depending, for example, on the overhead level included in a particular design. For example, in ​ , a two-way partition is shown by placing the privilege (e.g., privileged state) bits along a wavy line at (e.g., some or each) local network intersections (e.g., placed in configuration storage (e.g., a register) 410 together with other configuration bits, or placed in a separate privilege storage (e.g., a privilege state register) 412). The wavy line is included to generally indicate the boundary, and the actual boundary between regions (e.g., applications) may not follow that exact pattern. In one embodiment, the privilege bits are set to a certain value (e.g., logic one or logic zero) to indicate that no communication will cross that point (e.g., ​ the intersection 414 therein). The blocking of communication can be blocking the data flow to and / or from one or more (e.g., each) pins of the intersection 414. A switching device (e.g., a switching circuit) (e.g., as further discussed below with reference to ​ ) can be controlled by the privilege bit field.

[0125] ​Shows the in - operation configuration of an accelerator 400 having multiple processing elements (e.g., 402, 404, 406, 408) according to an embodiment of the present disclosure. The accelerator 400 includes privileged - mode and non - privileged - mode capabilities, e.g., allowing setting of (a plurality of) privilege (e.g., privilege status) bits (e.g., setting (a plurality of) privilege (e.g., privilege status) bits in a configuration store (e.g., register) 410 and / or setting other configuration bits, or setting (a plurality of) privilege (e.g., privilege status) bits in a separate privilege store (e.g., register 412)) when in privileged mode at (e.g., some or each) local network intersection. Although the in - operation configuration of the PEs is depicted, the setting of, e.g., privilege bits can be synchronized with this or separate from this. A switching device (e.g., switching circuit) (e.g., at the intersection 414) can be controlled (e.g., only) by its associated privilege store (e.g., privilege store 412). In one embodiment, the switching device (e.g., switching circuit) at the intersection 414 in the first network 401 (e.g., further discussed below with reference to ​ and the switching device (e.g., switching circuit) at the intersection 416 in the second network 403 (e.g., further discussed below with reference to ​ are controlled (e.g., only) by a single privilege store 412 (e.g., privilege status register). In one embodiment, the switching device (e.g., switching circuit) at the intersection 414 in the first network 401 (e.g., further discussed below with reference to ​ and the switching device (e.g., switching circuit) at the intersection 416 in the second network 403 (e.g., further discussed below with reference to ​ are controlled (e.g., only) by their respective privilege stores (e.g., privilege store 412 (e.g., privilege status register) for intersection 414 and privilege store 418 (e.g., privilege status register) for intersection 416). As depicted, each intersection can have its own privilege store associated with that intersection (e.g., physically adjacent to a structure (e.g., circuit - switched interconnect network) and controlling one or more switching devices of that structure). Once configured, the PEs can execute subject to data - flow constraints and / or privilege constraints (e.g., separate from data - flow constraints).

[0126] In one embodiment, if deactivated, then (e.g., by a configuration controller (e.g., ​One or more of the privilege bits set in the configuration controller (in the example of the configuration controller in the CSA) will inhibit (e.g., stop) the communication between these two partitions. In one embodiment, all signals attempting to cross the partition are pulled low (e.g., to zero). This can ensure that the non - relevant partitions are completely isolated, so as to create, for example, hard - circuit partitions, not only in the form of software. Although two partitions are discussed, three or more partitions can be formed, for example, by setting the (multiple) privilege bits accordingly. For example, each column of the accelerator 400 can be set to be its own partition. To partition a spatial structure (e.g., CSA), the operating system or other privileged users can determine the appropriate settings for the required (e.g., user) workload and build a privilege profile based on the required partition operations. The operating system or other privileged users can cause the privilege profile (e.g., via the (multiple) configuration controllers) to be dispatched to the CSA, for example, based on the techniques discussed herein. As further discussed below, for example, with reference to ​ As further discussed, the interconnect can include configurable data paths (e.g., networks) and configurable flow - control paths (e.g., networks). In addition to, or instead of, setting the configuration of the data path and the flow - control (e.g., back - pressure) path, some embodiments herein allow privilege configuration, for example, to set one or more privilege bits to allow or disallow (e.g., block) communication on the data path (e.g., network) and / or the flow - control (e.g., back - pressure) path (e.g., network).

[0127] ​ Illustrates a configurable data - path network 500 according to an embodiment of the present disclosure (e.g., in network one or network two as discussed with reference to ​ Network 500 includes a plurality of multiplexers (e.g., multiplexers 502, 504, 506), which can be configured (e.g., via their respective control signals) to connect one or more data paths (e.g., from PEs) together. ​ Illustrates a configurable flow - control path network 501 according to an embodiment of the present disclosure (e.g., in network one or network two as discussed with reference to ​ The network can be a lightweight PE - to - PE network. Some embodiments of the network can be regarded as a set of constituent primitives for building a distributed point - to - point data channel. ​ Illustrates a network in which two channels (bold black line and dotted black line) are enabled. The bold black - line channel is multicast. For example, a single input is sent to two outputs. Note that even if dedicated circuit - switched paths are formed between the channel endpoints, the channels can cross at some points within a single network. In addition, this crossing does not introduce a structural hazard between the two channels, such that each channel operates independently and at full bandwidth.

[0128] Implementing a distributed data channel may include ​ the two paths illustrated in ​ . The forward path, i.e., the data path, carries data from the producer to the consumer. The multiplexer may be configured to direct data and valid bits from the producer to the consumer, e.g., as shown in ​ ​ . In the case of multicast, the data will be directed to multiple consumer endpoints. The second part of this embodiment of the network is flow control, i.e., the backpressure path, which flows in the opposite direction to the forward data path, e.g., as shown in ​ ​ . The consumer endpoints may assert when they are ready to accept new data. Subsequently, these signals may be directed back to the producer using configurable logic knots (labeled, e.g., as backflow, flow control functions in ​ ​ ). In one embodiment, each flow control function circuit may be a plurality of switching devices (e.g., a plurality of muxes), e.g., similar to ​ ​ . The flow control path may handle the return of control data from the consumer to the producer. The knot may enable multicast, e.g., where each consumer is ready to receive data before the producer assumes the data has been received. In one embodiment, the PE is a PE having a data flow operator as its architectural interface. Additionally or alternatively, in one embodiment, the PE may be any kind of PE (e.g., in a fabric), such as but not limited to a PE having an instruction pointer, triggered instructions, or a state machine-based architectural interface.

[0129] In addition to, e.g., the PE being statically configured, the network may also be statically configured. During this configuration step, configuration bits may be set at each network component. These bits control, e.g., mux selection and flow control functions. The network may include multiple networks, e.g., a data path network and a flow control path network. One or more networks may utilize paths of different widths (e.g., a first width and a narrower or wider width). In one embodiment, the data path network has a wider (e.g., bit transfer) width than the flow control path network. In one embodiment, each of the first network and the second network includes its own data path network and flow control path network, e.g., data path network A and flow control path network A, and a wider data path network B and flow control path network B.

[0130] Some embodiments of the network are bufferless, and data is used to move between the producer and the consumer in a single cycle. Some embodiments of the network are also borderless, i.e., the network spans the entire fabric. In one embodiment, one PE is used to communicate with any other PE in a single cycle. In one embodiment, to improve routing bandwidth, several networks may be arranged in parallel between the rows of PEs.

[0131] Certain embodiments of the networks in this document have three advantages over FPGAs: area, frequency, and program expressiveness. Certain embodiments of the networks in this document operate at a coarse-grained level, which for example reduces the number of configuration bits and thus decreases the area of the network. Certain embodiments of the network also achieve area reduction by directly implementing flow control logic in the circuit (e.g., silicon). Certain embodiments of the enhanced network implementation also enjoy a frequency advantage over FPGAs. Due to the area and frequency advantages, there may be a power advantage when using a lower voltage at throughput parity. Finally, certain embodiments of the network provide better high-level semantics than FPGA wires, especially with respect to variable timing aspects, and thus those embodiments are more amenable to targeting by compilers. Certain embodiments of the networks in this document can be viewed as a collection of composable primitives for building distributed point-to-point data channels.

[0132] In certain embodiments, a multicast source may not be able to assert its data valid unless the multicast source receives a ready signal from each sink. Thus, in the multicast case, additional knots and control bits may be utilized.

[0133] Like certain PEs, the network can be statically configured. During this step, configuration bits are set at each network component. These bits control, for example, mux selection and flow control functions. The forward path of the network of this application requires some bits to swing the mux of the forward path. In the example shown in ​ , four bits are required per hop: one bit is utilized for each of the east mux and the west mux, while two bits are utilized for the south mux. In this embodiment, four bits can be used for the data path, but seven bits can be used for the flow control function (e.g., in the flow control path network). If, for example, the CSA further utilizes the north-south direction, other embodiments may utilize more bits. The flow control function can use control bits for each direction from which flow control can come. This can achieve a static setting of the sensitivity of the flow control function. Table 1 below summarizes the Boolean algebra implementation of the flow control function for the network in ​ , with the configuration bits in uppercase. In this example, seven bits are utilized.

[0134] Table 1 Flow implementation

[0135]

[0136] For the third flow control box from the left in ​ , EAST_WEST_SENSITIVE and NORTH_SOUTH_SENSITIVE are depicted as being set to implement flow control for the bold line channel and the dotted line channel respectively.

[0137] The depicted configurable data path network 500 and configurable flow control path network 501 may include one or more privilege bit stores (e.g., (multiple) registers) for storing privilege bits to control one or more circuits (e.g., AND logic gates) to block or allow communication on one or more data paths. Each of the depicted AND logic gates includes a privilege bit store (e.g., shared or exclusive), e.g., a privilege bit store 508 for the privilege bits of the positive input to AND logic gate 510, a privilege bit store 512 for the privilege bits of the positive input to AND logic gate 516, and a privilege bit store 522 for the privilege bits of the positive input to AND logic gate 524. In one embodiment, each intersection (e.g., intersection 514 of path multiplexers 502, 504, 506, which path multiplexers cumulatively form a switching device, e.g., a switching device in a circuit-switched interconnect network as in ​ may include a single privilege bit store (e.g., privilege bit store 508, which is for the privilege bits of the positive input to AND logic gate 510 and for the same privilege bits of the positive input to AND logic gate 516 and / or for the same privilege bits of the positive input to AND logic gate 524) (e.g., privilege bit store 522, which is for the privilege bits of the positive input to AND logic gate 524 and for the same privilege bits of the positive input to (e.g., between adjacent intersections) AND logic gate 516). A single privilege (e.g., one or more bits) value (e.g., privilege bit store) may be used for a single AND logic gate or distributed to any combination of multiple AND logic gates. As an example, when privilege bit store 512 has a first value (e.g., logical zero) stored therein (e.g., stored in a privilege configuration level), it causes AND logic gate 516 not to output the value that AND logic gate 516 receives from multiplexer 506. For example, in the depicted embodiment, the privilege bit value may be logical zero to cause AND logic gate 516 to block (e.g., not output) the value of input 518 that AND logic gate 516 receives from multiplexer 506 from proceeding to the next intersection 520 of the multiplexer (e.g., which cumulatively forms a switching device, e.g., a switching device in a circuit-switched interconnect network as in ​ ). As another example, when privilege bit store 512 has a second value (e.g., logical one) (e.g., different from the first value) stored therein (e.g., stored in a privilege configuration level), it causes AND logic gate 516 to output the value that AND logic gate 516 receives from multiplexer 506. For example, in the depicted embodiment, the privilege bit value may be logical one to cause AND logic gate 516 to output the value of input 518 that AND logic gate 516 receives from multiplexer 506 to the next intersection 520 of the multiplexer (e.g., which cumulatively forms a switching device, e.g., a switching device in a circuit-switched interconnect network as in ​in a switching device in a circuit-switched interconnect network as in ). Privilege bit storage may be updated along data paths within configurable data path network 500 and / or configurable control path network 501. For example, a configuration controller (e.g., only when invoking privilege mode) may cause an update of the privilege bit storage. In one embodiment, the configuration controller (e.g., local configuration controller) (e.g., on the CFG_START_PRIVILEGE channel) asserts the CFG_START_PRIVILEGE signal to cause some of the switching devices (e.g., multiplexers) in the (multiple) networks to swing, thereby forming a path in the (multiple) (e.g., local) networks to the (multiple) privilege bit storage locations. In one embodiment, a configuration controller in user mode is not permitted to cause an update of the privilege bit storage.

[0138] Each AND logic gate depicted includes a privilege bit storage, e.g., privilege bit storage 505 for the privilege bit input to AND logic gate 507, privilege bit storage 509 for the privilege bit input to AND logic gate 511, and privilege bit storage 513 for the privilege bit input to AND logic gate 515. In one embodiment, each cross point (e.g., the cross point at flow control function storage 503 or flow control function storage 517, e.g., which cumulatively forms a switching device, e.g., as ​ in a switching device in a circuit-switched interconnect network as in ) may include a single privilege bit storage (e.g., privilege bit storage 509, which is for the privilege bit input to AND logic gate 511 and for the same privilege bit input to AND logic gate 515), e.g., a single privilege bit may control multiple AND logic gates (e.g., may be an input to multiple AND logic gates).

[0139] ​

[0140] All communication is implicitly flow-controlled, such that a spatial array (e.g., CSA) can be a channel-based architecture. In some embodiments of a CSA, partitioning is created in the CSA by manipulating (e.g., only) the ready / enable signals of the channel communication protocol. In one embodiment, it is possible to simply pull these signals to an inactive value. In one embodiment of a CSA, the configuration of a local circuit-switched network includes a separately configurable forward data path and a backward control path. In some embodiments, if malicious code configures its data path in one direction and its control path in the other direction, this creates a security vulnerability because the data path can originate from different partitions (e.g., different programs), while the control path can originate from a local partition. To prevent this, some embodiments herein utilize a single privileged configuration state (e.g., one or more privileged bits) to control the switching of both the data path and the control path. In this way, the data path and the control path are physically paired by the hardware, enabling, for example, the privileged partition to be implemented by manipulating the data path and the control path.

[0141] In one embodiment, the forward data path (e.g., of configurable data path network 500) and the flow control path of the backward flow (e.g., of configurable flow control path network 510) are paired together (e.g., providing inputs and / or outputs to a PE in a configuration where the forward data path and the flow control path of the backward flow are coupled together), and a single control value (e.g., a control bit) can be used to control, for example, via one or more AND logic gates or other blocking circuits, the forward data path (e.g., of configurable data path network 500) and the paired flow control path of the backward flow (e.g., of configurable flow control path network 501). A single privilege (e.g., one or more bits) value (e.g., in privilege bit store 522) can be used for at least one AND logic gate in the forward data path and for at least one AND logic gate in the flow control path of the backward flow, e.g., any combination of AND logic gates. For example, a single privilege (e.g., one or more bits) value (e.g., in privilege bit store 522) can be used to allow or block communication, based on that value, by allowing or blocking communication through AND logic gates (e.g., AND logic gates 511, 515, 516, and 524) (e.g., where an AND logic gate only outputs a non-zero value (e.g., logic one) when all inputs to the AND logic gate are non-zero values (e.g., logic one)). For example, a single privilege (e.g., one or more bits) value (e.g., in privilege bit store 508) can be used to allow or block communication, based on that value, by allowing or blocking communication through AND logic gates 510, 517. In some embodiments, turning off a circuit switched path (e.g., via an AND logic gate) is different from turning off a buffer (e.g., a register) (e.g., clocking the buffer (e.g., the register)) in a PE because it is ensured that data is not observable by any adversary. A spatial array (e.g., a CSA), a processor, or a system can include any of the disclosures herein, e.g., a spatial array according to any of the architectures disclosed herein.

[0142] ​A flowchart according to an embodiment of the present disclosure is illustrated. The depicted process 600 includes: 602: When an incoming set of operands arrives at a plurality of processing elements of a processor, performing operations of a data flow graph using the interconnection network and the plurality of processing elements of the processor, where the processor includes a plurality of processing elements and an interconnection network between the plurality of processing elements and has a data flow graph that includes a plurality of nodes, the plurality of nodes being overlaid onto the plurality of processing elements of the processor and the interconnection network between the plurality of processing elements of the processor, and each node being represented as a data flow operator in the interconnection network and the plurality of processing elements; and 604: Configuring the interconnection network between a first subset of the plurality of processing elements and a different second subset using a configuration controller of the processor such that when a privilege bit is set to a first value, communication on the interconnection network between the first subset of the plurality of processing elements and the different second subset of the plurality of processing elements is not allowed; and 604: When the privilege bit is set to a second value, allowing communication on the interconnection network between the first subset of the plurality of processing elements and the different second subset of the plurality of processing elements. In one embodiment, one or more privilege bits may be stored, together with other configuration bits, in one or more of the configuration stores 822A - 822O such as ​ one or more of

[0143] ​ A context switch in a spatial array 701 of processing elements of a processor 700 according to an embodiment of the present disclosure is illustrated. The spatial array 701 is depicted as an accelerator coupled to a processor core 702 and / or a vector processing unit (VPU) 704, for example, an accelerator for performing tasks rather than a core and / or a VPU. The depicted processor 700 includes a cache home agent 706, for example, for acting as a local coherence and cache controller (e.g., cache agent) and / or also acting as a global coherence and memory controller interface (e.g., home agent).

[0144] The spatial array 701 may be any of the spatial arrays discussed herein, such as in ​ or ​ Specifically, ​ A conceptual view of context switching in the spatial array is provided. (e.g., centralized) Control arranges wavefronts radiating outward, for example, from a cache (e.g., L2 cache 708). The new configuration 710 region and the old configuration 716 region may be active simultaneously, for example, processing data. ​ Illustrated is the concept that through the coordination of the spatial array 701 (e.g., architecture), the spatial array can achieve pipelined, wavefront-oriented runtime services. In ​shows a coordinated implementation of context switching. Here, the extraction region 714 (e.g., which is saving the state of the first spatial context) and the configuration region 712 (e.g., the loading of a second (new or prior) different context into the spatial array) are both active simultaneously. In this embodiment, both the new configuration and the old configuration can be active simultaneously, for example, to limit the degradation of the spatial array throughput caused by runtime operations. In one embodiment, to achieve pipelined configuration and extraction, the main property to be ensured is that the new configuration 712 (e.g., the configuration region) and the old extraction 714 (e.g., the extraction region) do not communicate. To achieve this assurance, some embodiments herein utilize the architectural properties of the spatial array in terms of communication. For example, the PEs follow a full / empty (e.g., backpressure) microprotocol that can be manipulated in the microarchitecture to prevent communication. In ​ shows an example of this mechanism. Some embodiments herein provide a coordination mechanism (e.g., a controller) for the spatial array to ensure that the new region and the old region do not communicate, for example, to ensure program correctness.

[0145] [[ID=e5]] ​ illustrates a runtime configuration for context switching (e.g., configuration and extraction) of a spatial array 800 for processing elements 802A - 802O according to an embodiment of the present disclosure. In one embodiment, the spatial array is an accelerator of a processor (e.g., having cores). Once configured, the PEs can be subject to data flow constraints for execution. However, channels involving unconfigured PEs can be disabled by the microarchitecture, thus preventing any undefined operations from occurring, for example. These properties allow the embodiments herein to be initialized and executed in a distributed manner, for example, without any centralized execution control at all. From an unconfigured state, the configuration can occur completely in parallel (e.g., perhaps in as short as 200 nanoseconds). However, due to the distributed initialization of the embodiments of the spatial array (e.g., CSA), the PEs may become active and send requests to the memory, for example, long before the entire structure is configured. Extraction can proceed in a manner almost identical to configuration. The local network (e.g., 804 or 806) can be adhered to (e.g., its circuitry is switched) to extract data from one target at a time and extract the status bits for distributed coordination. The spatial array (e.g., CSA) can arrange the extraction to be non-destructive, i.e., at the completion of the extraction, each extractable target has returned to its starting state. In this implementation, all the states in the target can be propagated to the output registers connected to the local network in a scan-like manner. However, in-place extraction can be achieved by introducing new paths at the register transfer level (RTL) or using existing wires to provide the same function with lower overhead. Similar configuration and hierarchical extraction are implemented in parallel.

[0146] In ​In this example, multiple local (e.g., configuration) controllers 808A - 808E are included, e.g., connected to a network (e.g., local network 804 or 806). In one embodiment, the local (e.g., configuration) controllers are used to control configuration and / or extraction. The local controllers may further be controlled by a higher-level controller (e.g., the controller 104 in ​ ). The (e.g., configuration) controller may manage the configuration and / or extraction of a subset of processing elements. The local (e.g., configuration) controller 808A may manage (e.g., cause) the configuration and / or extraction of processing elements 802A - 802C. The local (e.g., configuration) controller 808B may manage (e.g., cause) the configuration and / or extraction of processing elements 802D - 802O. The local (e.g., configuration) controller 808C may manage (e.g., cause) the configuration and / or extraction of processing elements 802G - 802I. The local (e.g., configuration) controller 808D may manage (e.g., cause) the configuration and / or extraction of processing elements 802J - 802L. The local (e.g., configuration) controller 808E may manage (e.g., cause) the configuration and / or extraction of processing elements 802M - 802O. Although each subset of the managed processing elements (e.g., 802A - 802C) is shown on the same line as its local (e.g., configuration) controller (e.g., 808A), other orientations are possible. Although three processing elements (e.g., 802A - 802C) are shown as having a single local (e.g., configuration) controller (e.g., 808A), a (e.g., local) controller may be used for one processing element or any number of processing elements. The processing elements may be as disclosed herein, e.g., as in ​ . The processing elements may be configured, for example, by writing to configuration registers. The local (e.g., configuration) controller may be a configuration controller (e.g., as in ​ ) and / or an extraction controller (e.g., as in ​ ). The state of the processing element (e.g., configuration information) may include data in any (input or output) queue or buffer, backpressure data (e.g., signals), operational configuration, and / or any other data. The state may include information stored in any register(s) of the processing element. The state may include where (e.g., from one or more PEs or memories) data is sourced (input) for the PE and where (e.g., which or which PEs or memories) the (output) data is sent. The state may include the data path network and / or the flow control (e.g., backpressure) path network (e.g., switching settings thereof), see, e.g., ​ . The state may include data related to memory access, e.g., including the address and the returned data.

[0147] In the depicted embodiment, each processing element may be in an indicated state, e.g., configured with a particular configuration, actively configuring (e.g., loading and enabling a configuration to be executed), unconfigured, actively unconfiguring, or extracting a configuration (e.g., state). Configurations in a spatial array (e.g., (multiple) processing elements) may be used for the same data flow graph, e.g., where one or more processing elements are not reconfigured. For example, for each configuration, the one or more processing elements may perform the same operation, but with different (multiple) input sources and / or (multiple) output destinations (e.g., values). Configurations may be used for different data flow graphs, e.g., where one or more processing elements are reconfigured to perform different operations. A configuration may be where to configure (e.g., a subset of) the processing elements (e.g., program them) such that each node of the data flow graph is represented in a spatial array (e.g., having processing elements as data flow operators).

[0148] In ​ , the local controller 808A may have previously received (e.g., from a higher-level controller) a command to apply a new configuration to a subset 802A - 802C of the processing elements coupled to the local controller 808A, and in ​ , the new configuration has been fully applied (e.g., loaded). The local controller 808B may have received (e.g., from a higher-level controller) a command to apply a new configuration to a subset 802D - 802F of the processing elements coupled to the local controller 808B. In ​ , the processing element 802D is unconfigured, the processing element 802E is being actively configured (e.g., configured with the new configuration), and the processing element 802F has been configured with the new configuration. The local controller 808B may send configuration information (e.g., data) 810 (e.g., including state, etc.) to the processing element 802E to effect the configuration of the processing element 802E accordingly. Line 820 schematically illustrates the configuration control sent from the configuration store 822E to the network (e.g., 804 and / or 806) to effect the required (multiple) configuration data paths. The configuration control signal data on line 820 may come from the local controller 808B.

[0149] The configuration store 822A - 822O schematically illustrates the configuration and extraction of control data (signals) for circuit switching in the network, e.g., as contrasted with the configuration and extraction of the data payload itself. In one embodiment, the configuration store is a register in a local (e.g., configuration and / or extraction) controller. Lines 818, 820, 824 schematically illustrate the configuration control sent from the configuration store 822D - 822F to the network (e.g., 804 and / or 806) to effect the desired configuration data path(s). The lines from processing element 802B to processing element 802F may represent an active channel in network 804 and / or 806, which is set (e.g., the switching device of a circuit - switched network is set) to couple the output (e.g., of a buffer) of processing element 802B to the input (e.g., of a buffer) of processing element 802F. The channel may be set according to the new configuration. Dotted lines 812, 814, 816 may indicate inactive channels of the network as being active when both the (multiple) input and output processing elements are configured accordingly.

[0150] The local (e.g., configuration) controller 808D is depicted as sending and / or receiving un - configuration (e.g., extraction) data 830 (e.g., including status, etc.), and the processing element 802J accordingly causes the un - configuration (e.g., extraction of status) of the processing element 802J. Line 826 schematically illustrates the un - configuration (e.g., extraction) control sent from the configuration (e.g., un - configuration) store 822J to the network (e.g., 804 and / or 806) to effect the desired un - configuration (e.g., extraction) data path(s). The un - configuration (e.g., extraction) control signal data on line 826 may come from the local controller 808D.

[0151] The local controllers 808A - 808E may each include a store 828A - 828E (e.g., (multiple) registers) to store information describing the coordination between the local controllers, e.g., what operation each controller is performing (e.g., active with the new configuration, active with the old configuration, unconfigured, un - configuring (extracting) or configuring).

[0152] Go to ​ , e.g., the processing element 802J that was previously un - configured in ​ is now unconfigured, and the status data for that context is now saved in any of the stores discussed herein, for example. Subsequently, the processing element 802J is unconfigured, e.g., it may assert backpressure to any upstream processing element, etc. so as not to allow data to be input to the processing element 802J. In one embodiment, when saving the status data for the processing element 802J (e.g., as managed by the local controller 808D), it may be disabled, for example, by clearing the data in the configuration (e.g., un - configuration) store 822J. ​The path for sending and / or receiving unconfigured (e.g., fetched) data 830.

[0153] Go to ​ , after the completion of the unconfiguration (e.g., unconfiguration operation) of processing element 802J (and processing elements 802K and 802L), local controller 809D may send (completion) messages 832, 834 to neighboring controllers (sent to controllers 808C and 808E respectively). In one embodiment, such messages (e.g., completion of fetching) may cause one or more in a set of neighboring processing elements to be in their next batch of operations.

[0154] Now go to ​ , the reception of the completion (e.g., unconfiguration) message 832 by local controller 808C may trigger local controller 808C to start its next operation, e.g., as indicated in ​ the configuration of processing element 802I by using configuration information (e.g., data) 840 sent along a path (e.g., in network 804 and / or 806) to processing element 802I, which may accordingly result in the configuration of processing element 802I (and e.g., processing elements 802G and 802H). In one embodiment, configuration (e.g., fetching) control is sent from configuration stores 822G - 822I to the network (e.g., 804 and / or 806) to achieve the desired configuration data path. The configuration control signal 842 data may come from local controller 808C.

[0155] Additionally or alternatively, the reception of the completion (e.g., unconfiguration) message 834 by local controller 808E may trigger local controller 808E to start its next operation, e.g., as indicated in ​ the unconfiguration (e.g., fetching) of processing element 802O by using unconfigured (e.g., fetched) data 850 (e.g., including status, etc.) sent and / or received by processing element 802O along a path (e.g., in network 804 and / or 806), which may accordingly result in the unconfiguration (e.g., fetching) of processing element 802O (and e.g., processing elements 802M and 802N). In one embodiment, unconfiguration (e.g., fetching) control is sent from configuration (e.g., unconfiguration) stores 822O to the network (e.g., 804 and / or 806) to achieve the desired (multiple) unconfiguration data paths. The unconfiguration (e.g., fetching) control signal 852 data may come from local controller 808D. Note that the term "old" as referred to in reference to these figures may refer to an existing configuration. Note that the term "new" as used in reference to these figures may refer to a previous configuration, but it is replacing the currently configured PE or is being configured into an unconfigured PE.

[0156] In one embodiment, a network (e.g., a circuit-switched network) includes a plurality of channels (e.g., as shown in ​ ). Channel semantics (e.g., the dashed and solid lines overlaid into the network in ​ ) can enable the natural activation of a pipeline, e.g., where unconfigured PEs clamp control values and / or when the PEs are configured, they begin to compute. The spatial array can thus become active in very few cycles (e.g., within about dozens of nanoseconds) (e.g., in contrast to the cycle-level semantics of an FPGA where backpressure is not implicit and the entire design (FPGA) must be configured, similar to "coming out of reset").

[0157] Although the above discussion of ​ refers to multiple local controllers, in another embodiment, a single (e.g., configuration) controller can implement the above scheme.

[0158] ​ Further illustrated is the communication microprotocol during extraction, e.g., via the manipulation of full / empty (e.g., backpressure) bits in the communication microprotocol to prevent fine-grained data flow during runtime service events.

[0159] ​ Illustrated is the staged extraction of (e.g., a first) context of a spatial array 900 of processing elements 902A - 902P configured to execute a data flow graph according to an embodiment of the present disclosure. In ​ , a single data flow graph is depicted as being overlaid, e.g., into the spatial array 900 of processing elements 902A - 902P (and e.g., into the (multiple) (e.g., interconnect) network therebetween), such that each node of the data flow graph is represented as a data flow operator in the spatial array of processing elements. In one embodiment, one or more of the processing elements in the spatial array of processing elements are used to communicate via a memory interface (e.g., ​access the memory via the memory interface 1802). In one embodiment, the select node of the data flow graph thus corresponds to the select operator 904 (e.g., is represented by the select operator 904), the switch node of the data flow graph thus corresponds to the switch operator 906 (e.g., is represented by the switch operator 906), the multiplier node of the data flow graph thus corresponds to the multiplier operator 908 (e.g., is represented by the multiplier operator 908), the "==" node of the data flow graph thus corresponds to the equality test operator 910 (e.g., is represented by the equality test operator 910), and the "<" node of the data flow graph corresponds to the "<" operator 912 (e.g., is represented by the "<" operator 912). Another processing element and / or flow control path network may provide control signals (e.g., control tokens) to the select operator 904A and the switch operator 906 to perform operations. In one embodiment, the spatial array 900 of processing elements is configured (to execute the data flow graph) prior to the start of execution. In one embodiment, the compiler performs the conversion from the program and / or data flow graph to ​ the configuration in. In one embodiment, the input of the data flow graph nodes into the spatial array of processing elements logically embeds the data flow graph into the array of processing elements (e.g., as further discussed below), such that the input / output paths are configured to produce the desired result. For example, see the discussion below of ​ below.

[0160] In ​ the spatial array 900 is depicted as having local (e.g., configuration and / or extraction) controllers 908A - 908D. (e.g., configuration) controllers may manage the configuration and / or extraction of a subset of the processing elements. The local (e.g., configuration) controller 908A may manage (e.g., cause) the configuration and / or extraction of the processing elements 902A - 902D. The local (e.g., configuration) controller 908B may manage (e.g., cause) the configuration and / or extraction of the processing elements 902E - 902H. The local (e.g., configuration) controller 908C may manage (e.g., cause) the configuration and / or extraction of the processing elements 902I - 922L. The local (e.g., configuration) controller 908D may manage (e.g., cause) the configuration and / or extraction of the processing elements 902M - 902P. The local controllers may be further controlled by a higher-level controller (e.g., controller 914).

[0161] In ​In [the context], the local controllers receive their signals from the (chip) controller 914 to obtain, for example, a configuration (e.g., according to a first context). These signals (e.g., commands) control the behavior of the local controllers to configure a corresponding subset of their processing elements. For example, an extraction of the current operation state (e.g., operands, etc.) may be desired during a context switch from a first context to a different second context. The constants 1 and 2 in the processing elements 902G and 902K may be used as inputs respectively, but the present disclosure is not limited thereto.

[0162] In ​ [the context], for example, when the input data and / or output data space (e.g., no backpressure is asserted) is available, the local (e.g., configuration and / or extraction) controllers 908A - 908D have completed their configuration (e.g., according to a first context) and can now operate. Next, assume a request is made to extract the context currently in ​ [the context]. This extraction can be performed in stages, for example, top - to - bottom (in the program / operation flow).

[0163] In ​ [the context], for example, the local controller 908A starts the extraction of the context of the (e.g., configured and / or unconfigured) processing elements 902A - 902D. For example, any input data, output data, and the corresponding backpressure signals are extracted (and saved) from the configured processing elements 902A and 902B. The remaining processing elements (902E - 902P, e.g., only 902G, 902K, and 902O) can continue to operate, for example, assuming they have input data and / or output data space (e.g., no backpressure is asserted). In ​ [the context], the processing element 902G has executed as the pick operator 904 and has output data 916. The hollow circles in these figures may represent the input data values and / or output data space according to the first context (e.g., no backpressure is asserted), and the solid circles in these figures may represent the input data values and / or output data space according to the first context (e.g., no backpressure is asserted).

[0164] In ​ [the context], the extraction of the context of the processing elements 902A - 902D is still occurring, and the output data 916 has been consumed by the processing element 902K executed as the multiplier operator 908 and has output data 918.

[0165] In ​Among them, the extraction of the context of processing elements 902A - 902D is completed, and the output data 918 has been consumed by the processing element 902O acting as the switching operator 906 and has the output data 918. For example, the local controller 908B starts the context of the next subset 902E - 902H of processing elements (e.g., configured and / or unconfigured). For example, any input data, output data, and corresponding backpressure signals are extracted (and saved) from the configured processing element 902G. The remaining processing elements (902I - 902P, e.g., only 902I and 902O) can continue to operate, e.g., assuming they have input data and / or output data space (e.g., no backpressure is asserted). In some embodiments, the data may not move through the extraction region, e.g., where the extraction region is the processing elements 902E - 902H. Thus, in embodiments where the extraction region will not accept new data (e.g., that region of the processing element asserts its backpressure signal), the output data 920 is stopped (e.g., it would originally go to the processing element 902G acting as the selection operator 904). In one embodiment, new data may be generated, e.g., by the processing element 902A acting as the "equality test" operator 910 and / or by the processing element 902B acting as the "less than" operator 912 for the region (e.g., in program flow order) above the extraction region.

[0166] In ​Among them, the extraction of the context of processing elements 902E - 902H is completed. For example, the local controller 908C starts to process the context of the next subset 902I - 902L of processing elements (e.g., configured and / or unconfigured). For example, any input data, output data, and corresponding backpressure signals are extracted (and saved) from the configured processing element 902K. The remaining processing elements (902M - 902P, e.g., only 902O) can continue to operate, e.g., assuming they have input data and / or output data space (e.g., no backpressure is asserted). In some embodiments, data may not move through the extraction area, e.g., where the extraction area is processing elements 902I - 902L. Thus, in embodiments where the extraction area (and any area logically above it) will not accept new data (e.g., that area of the processing element asserts its backpressure signal to prevent data from crossing that area), the output data 920 is stopped (e.g., it was originally going to processing element 902G which is a selection operator 904). In one embodiment, new data can be generated for the area above the extraction area (e.g., in program flow order), e.g., where the output data 922 from processing element 902A which is an "equality test" operator 910 is stopped from going to processing element 902O (e.g., across the extraction area) and / or where the output data 924 from processing element 902B which is a "less than" operator 912 is not stopped from going to processing element 902G (e.g., not across the (processing) extraction area).

[0167] In ​Among them, the extraction of the context of processing elements 902I - 902L is completed. For example, the local controller 908D starts to process the context of the next subset 902M - 902P of processing elements (e.g., configured and / or unconfigured). For example, any input data, output data (the stopped output data 920), and the corresponding backpressure signal are extracted (and saved) from the configured processing element 902O. Here, there are no further downstream processing elements for which their context needs to be saved (e.g., this is the end of this part of the data flow diagram), so the extraction is almost complete. The above - mentioned processing elements (902A - 902L, e.g., only 902A, 902B, 902G, and 902K) can continue to operate, e.g., assuming they have input data and / or output data space (e.g., no backpressure is asserted). In some embodiments, data may not move through the extraction region, e.g., where the extraction region is processing elements 902M - 902P. Thus, in embodiments where the extraction region (and any region logically above it) will not accept new data (e.g., that region of the processing element asserts its backpressure signal to prevent data from crossing that region), the output data from processing element 902G is stopped (e.g., processing element 902G is waiting for data from processing element 902O to continue). In one embodiment, new data can be generated for the region above the extraction region (e.g., in program - flow order), e.g., where the output data 922 from processing element 902G, which is a pick operator 904, is stopped together with the output data from processing element 902O, which is a pick operator 906, but not stopped across the (in - process) extraction region.

[0168] In ​ Among them, for example, when the input data and / or output data space (e.g., no backpressure is asserted) is available, the local (e.g., configuration and / or extraction) controllers 908A - 908D have completed their extraction of the first context (and configuration of the second context) and can now operate. For example, after extraction, the space array 900 (e.g., the data flow diagram loaded therein) can be reused for different sets of (input) operands, e.g., as the second context. In ​ Among them, the data output 926 (e.g., from the previous operation for part two below) can be available (e.g., configured into the space array 900) and thus flow upstream to the processing element 902G, which is a pick operator 904, as ​ shown in

[0169] In ​Among them, the processing element 902G that selects the operator 904 may have all its operands and no backpressure signal, so the output data 928 can be generated. In one embodiment, in order to access the local controller, a higher-level controller may have access to the address of the local controller that the higher-level controller is managing. The local controller may have access to the addresses of the processing elements it is managing (for example, it may calculate the addresses of the processing elements it is managing). The addresses of the PEs may be sent in by a higher-level (e.g., regional) controller (e.g., which knows the memory format of the spatial array context).

[0170] ​ FIG. illustrates the extracted state 1000 according to an embodiment of the present disclosure. ​ FIG. illustrates the state 1000 at the start of extraction according to an embodiment of the present disclosure. ​ and ​ FIG. illustrates that when the extraction is staged, the extraction may take ​ the form of the data flow diagram shown in (e.g., where the input and output data from that saved state are indicated by hollow circles), but that exact form of the state never exists at any one time. Thus, it can be said that ​ the staged extraction output in captures a legal view (e.g., state) of the spatial array (e.g., the data flow diagram represented in the spatial array), while ​ FIG. illustrates the state at the start of a staged extraction that includes multiple stages (e.g., multiple stages of a single part of the data flow diagram). ​ The state in may be, for example, several steps before the initial state (when the extraction first begins) because several more operations have been performed, e.g., consuming the input data 1002 and 1004 and generating the output data 1006.

[0171] ​FIG. 1100 shows a state machine for configuring, e.g., a controller according to an embodiment of the present disclosure. The controller may include, e.g., a hardware state machine in each local (e.g., configuration) controller. The state machine may be a Mealy machine or a Moore machine. State 1102 may be a state in which a (e.g., higher-level) controller sends a control message to initiate a context switch for one or more previously configured PEs of another (e.g., local) controller. In state 1104, one or more previously configured PEs may operate according to a first (e.g., old) context of a (e.g., data flow graph), e.g., until all the data is consumed, when the input data operands and output space are available. Subsequently, state 1106 may be entered, e.g., based on the requested context switch, and the context (e.g., old configuration) of one or more previously configured PEs may start to be extracted. After the extraction is completed (e.g., after state 1106 or after entering state 1112), the local (e.g., configuration) controller may send a message to neighboring controllers to form a pipelined context switch. In one embodiment, the local (e.g., configuration) controller sends a message to one or more subsequent local (e.g., configuration) controllers (e.g., 2, 3, 4, 5 controllers, etc.) as an indication that they may start to extract the first context of a subset of their processing elements. Additionally or alternatively, the local (e.g., configuration) controller sends a message to one or more previous local (e.g., configuration) controllers (e.g., 2, 3, 4, 5 controllers, etc.) as an indication that they may start to configure (e.g., load) a second (e.g., new) context into a subset of their processing elements. In state 1112, the most recently extracted processing elements may then be held in an unconfigured state. In state 1114, e.g., after receiving a message from one or more subsequent local (e.g., configuration) controllers (e.g., 2, 3, 4, 5 controllers, etc.) indicating completion of their extraction of the first state, those unconfigured processing elements may start to configure (e.g., load) the second context into a subset of their processing elements. In state 1116, e.g., when a (e.g., higher-level) controller sends a control message to initiate another context switch, that new state (e.g., configuration) may be executed on the configured processing elements and then return to state 1104. In one embodiment, the context may be from different data flow graphs or different parts of the same data flow graph.

[0172] ​Illustrated is the extraction of context for a spatial array 1202 of processing elements according to an embodiment of the present disclosure. The spatial array 1202 is schematically illustrated to show an initial configuration message for a barrier (e.g., an extraction region) that forms the extent of the spatial array (e.g., the structural extent), which sweeps across the entire spatial array (e.g., structure) under the control and direction of a higher-level (e.g., chip-level) controller 1204. The controller 1204 may arrange local controllers to coordinate the pipeline. The ACI network and RAF may be as discussed below. Although ​ A stage ordering is illustrated herein, but other topologies are possible.

[0173] ​ Illustrated is the extraction of context for a spatial array 1202 of processing elements according to an embodiment of the present disclosure. The spatial array 1202 is schematically illustrated to show a barrier (e.g., an extraction region in a "waiting for configuration" state) that forms the extent of the spatial array (e.g., the structural extent), which sweeps across the entire spatial array (e.g., structure) under the control and direction of a higher-level (e.g., chip-level) controller 1204. The controller 1204 may arrange local controllers to coordinate the pipeline. In one embodiment, ​ the state of the spatial array 1202 in [the figure] corresponds to those in the foregoing figures. The ACI network and RAF may be as discussed below. Although ​ A stage ordering is illustrated herein, but other topologies are possible.

[0174] ​ Illustrated is the staged extraction of context for a spatial array of processing elements according to an embodiment of the present disclosure, the spatial array of processing elements including a network (e.g., a mezzanine or global network) therebetween. The network may be any network discussed herein. ​ A block diagram illustration includes an extraction region 1302, a POST (extraction) (after (extraction)) region 1304, and a PRIOR (to extraction) (before (extraction)) region 1306, which includes network messages across these regions.

[0175] Messages in the network can cross phase boundaries (e.g., extraction regions). In one embodiment, hardware (e.g., a network controller as discussed herein) records the state of network tokens. In one embodiment, the network controller can inject POST (e.g., extraction region) transition messages on each channel and release PRIOR (e.g., extraction region) messages only until a POST message is received. When POST messages are received for all active channels, the network controller can record the state of the network controller. In one embodiment, early consumption of messages is legal, but the state is retained. Matching messages (e.g., messages from the same context) can be forwarded. For example, PRIOR can consume PRIOR messages, and POST can consume POST messages. A mismatch can mean that the data is from the wrong epoch and needs to wait. PRIOR messages can be promoted. PRIOR endpoints can wait for extraction. PRIOR messages can wait for POST transitions.

[0176] ​ FIG. illustrates staged extraction of contexts for a spatial array 1400 for a processing element 1402 according to an embodiment of the present disclosure, the staged extraction including memory access. In some embodiments, communication through memory can be utilized. In the depicted embodiment, only processing elements in the POST region (e.g., after the extraction region) contact memory (e.g., cache), e.g., alleviated by first extracting near-memory (e.g., near the request address file (RAF) circuit, which can be as described herein) PEs to quickly re-enable them. In one embodiment, the spatial array (e.g., controller) is used to treat the PRIOR region of the PE and the POST region of the PE as conflicting channel groups and roll back to a snapshot of PRIOR in response to conflict detection between the two regions. In one embodiment, the spatial array (e.g., controller) is used to keep old and new values alive in the cache, e.g., allowing each epoch to access its value. In one embodiment, extraction of processing elements physically close to the cache (e.g., ​ the L2 cache therein) can help quickly resume access, e.g., where in ​ the embodiment of, extraction proceeds outward (e.g., radiates). The spatial array 1400 can be coupled to a vector processing unit 1404 and / or a processor core 1410.

[0177] Disposing of Memory Operations: It may be the case that during the extraction period, the context has pending memory operations. In such a case, the cache interface (e.g., CHA) will reserve the resources (e.g., slots in the reorder buffer) allocated to the pending requests until those requests are completed by the memory system. At that moment, these requests can be written into the memory representation of the evicted process, and the allocated resources are returned to the memory interface for use by the newly configured context. In one embodiment, when the requests are pending, the associated resources are not used by the new context.

[0178] ​ Illustrated is the extraction of a context for a spatial array 1500A of processing elements according to an embodiment of the present disclosure. In one embodiment, a second (e.g., next) extraction region (e.g., where no data can span the region) may be maintained to prevent inter-epoch communication in addition to, for example, preventing the current extraction region. In one embodiment, phased extraction can be achieved without modification to the (multiple) processing elements, for example, when the local network is disabled during extraction (e.g., via a backpressure signal). This can provide an insurmountable barrier (e.g., due to the backpressure signal being active). The two-tiered nature of extraction can ensure that the PEs in the first tier do not interfere with the PRIOR region when they transition to POST, e.g., as ​ illustrated. In certain embodiments, elaborate high-level arrangements and coordination are used for the functions of, for example, regional and / or global extraction controllers. POST may refer to a newly configured region configured with a new configuration (e.g., context). PRIOR may refer to a previously configured region configured with a previous configuration (e.g., context).

[0179] ​ Illustrated is the extraction of a context for a spatial array 1500B of processing elements according to an embodiment of the present disclosure. ​ Depicted is a detailed time slice of a pipelined context switch. In ​ five regions are depicted: the execution context, the current extraction region, the current configuration region, and the buffer regions therebetween.

[0180] ​FIG. 1600 is a flowchart according to an embodiment of the present disclosure. The depicted process 1600 includes: 1602: providing a processor that includes a plurality of processing elements and an interconnection network between the plurality of processing elements and has a data flow graph that includes a plurality of nodes that are mapped to the plurality of processing elements of the processor and the interconnection network between the plurality of processing elements of the processor, and each node is represented as a data flow operator in the interconnection network and the plurality of processing elements; 1604: when an incoming set of operands (e.g., input data and / or output data space (e.g., no backpressure for output from the destination is asserted)) arrives at the plurality of processing elements, performing operations of the data flow graph using the interconnection network and the plurality of processing elements; 1606: configuring a first subset and a different second subset of the plurality of processing elements using a configuration controller of the processor according to configuration information for a first context of the data flow graph; and 1608: when (e.g., all) pending operations of the first context are completed in the first subset, for a requested context switch, configuring the first subset of the plurality of processing elements using the configuration controller of the processor according to configuration information for a second context of the data flow graph, and blocking the second context data flow from the output of the first subset from entering the input of the different second subset until the pending operations of the first context are completed in the different second subset.

[0181] ​ FIG. is a flowchart according to an embodiment of the present disclosure. The depicted process 1700 includes: 1702: providing a processor that includes a plurality of processing elements and an interconnection network between the plurality of processing elements and has a data flow graph that includes a plurality of nodes that are mapped to the plurality of processing elements of the processor and the interconnection network between the plurality of processing elements of the processor, and each node is represented as a data flow operator in the interconnection network and the plurality of processing elements; 1704: when an incoming set of operands arrives at the plurality of processing elements, performing operations of the data flow graph using the interconnection network and the plurality of processing elements; 1706: configuring a first subset and a different second subset of the plurality of processing elements using a first configuration controller and a second configuration controller of the processor according to corresponding configuration information for a first context of the data flow graph; and 1708: when (e.g., all) pending operations of the first context are completed in the first subset, for a requested context switch, configuring the first subset of the plurality of processing elements using the first configuration controller of the processor according to configuration information for a second context of the data flow graph, and blocking the second context data flow from the output of the first subset from entering the input of the different second subset until the pending operations of the first context are completed in the different second subset.

[0182] ​

[0183] Some embodiments of CSA are designed to execute programs (e.g., programs generated by compilers) quickly and efficiently. Some embodiments of the CSA architecture provide programming abstractions that support the requirements of compiler technologies and programming paradigms. Embodiments of CSA execute data flow graphs, e.g., a program manifestation very much like the internal representation (IR) of a compiler for a compiled program. In this model, a program is represented as a data flow graph consisting of nodes (e.g., vertices) and edges, where the nodes are drawn from a set of data flow operators defined by the architecture (e.g., covering both computational and control operations), and the edges represent the transfer of data between data flow operators. Execution can proceed by injecting data flow tokens (e.g., as data values or data flow tokens representing data values) into the data flow graph. Tokens can flow between them and can be transformed at each node (e.g., vertex), thus forming a complete computation, for example. In ​ a sample data flow graph and its deviation from high-level source code are shown, and ​ an example of the execution of the data flow graph is shown.

[0184] By providing exactly the data flow graph execution support required by the compiler, embodiments of CSA are configured for data flow graph execution. In one embodiment, CSA is an accelerator (e.g., the accelerator in ​ ), and it does not seek to provide some of the necessary but infrequently used mechanisms (such as system calls) available on a general-purpose processing core (e.g., the core in ​ ). Thus, in this embodiment, CSA can execute a lot of code but cannot execute all code. In exchange, CSA gains significant performance and energy advantages. To achieve the acceleration of code written in a commonly used serialized language, the embodiments herein also introduce several novel architectural features to assist the compiler. A specific novelty is the handling of memory by CSA, which is a topic that has been previously ignored or not well addressed. Embodiments of CSA are also unique in using data flow operators (e.g., as opposed to using look-up tables (LUTs)) as its basic architectural interface.

[0185] Turning back to embodiments of CSA, data flow operators are discussed below.

[0186] ​

[0187] A key architectural interface of an embodiment of an accelerator (e.g., CSA) is a data flow operator, e.g., a direct representation as a node in a data flow graph. From an operational perspective, the data flow operator behaves streamingly or in a data-driven manner. As soon as the number of incoming operations of a data flow operator is available, the data flow operator can execute. CSA data flow execution can (e.g., only) rely on highly localized state, thus, e.g., resulting in a highly scalable architecture along with a distributed asynchronous execution model. The data flow operator can include arithmetic data flow operators, e.g., one or more of the following: floating-point addition and multiplication, integer addition, subtraction, and multiplication, various forms of comparison, logical operators, and shifts. However, embodiments of CSA can also include a rich set of control operators for managing data flow tokens in an auxiliary program graph. Examples of these control operators include a "pick" operator (e.g., which multiplexes two or more logical input channels into a single output channel) and a "switch" operator (e.g., which operates as a channel demultiplexer) (e.g., outputs a single channel from two or more logical input channels). These operators can enable a compiler to implement control paradigms such as conditional expressions. Some embodiments of CSA can include a limited set of data flow operators (e.g., relative to a smaller number of operations) to implement a dense and energy-efficient PE microarchitecture. Some embodiments can include data flow operators for complex operations common in HPC code. The CSA data flow operator architecture is highly adaptable for deploying specific extensions. For example, more complex mathematical data flow operators (e.g., trigonometric functions) can be included in some embodiments to accelerate certain math-intensive HPC workloads. Similarly, extensions for neural network tuning can include data flow operators for vectorization, low-precision arithmetic.

[0188] ​ Illustrates a program source according to an embodiment of the present disclosure. The program source code includes a multiplication function (func). ​ Illustrates according to an embodiment of the present disclosure for ​ the program source of the data flow graph 1800. The data flow graph 1800 includes a pick node 1804, a switch node 1806, and a multiplication node 1808. Along one or more of the communication paths, buffers can optionally be included. The depicted data flow graph 1800 can perform the following operations: select the input X using the pick node 1804, multiply X by Y (e.g., multiplication node 1808), and then output the result from the left side of the switch node 1806. ​ Illustrates an accelerator (e.g., CSA) according to an embodiment of the present disclosure, the accelerator having a configuration for performing ​Multiple processing elements 1801 of the data flow diagram. More specifically, the data flow diagram 1800 is overlaid into an array of processing elements 1801 (and, for example, an (optional) (e.g., interconnect) network therebetween) such that, for example, each node in the data flow diagram 1800 is represented as a data flow operator in the array of processing elements 1801. For example, certain data flow operations may be implemented using processing elements, and / or certain data flow operations may be implemented using a communication network (e.g., its network data flow endpoint circuitry). For example, Pick, PickSingleLeg, PickAny, Switch, and / or SwitchAny operations may be implemented using one or more components of a communication network (e.g., its network data flow endpoint circuitry), e.g., as opposed to processing elements.

[0189] In one embodiment, one or more of the processing elements in the array of processing elements 1801 are used to access memory via a memory interface 1802. In one embodiment, the pick node 1804 of the data flow diagram 1800 thus corresponds to a pick operator 1804A (e.g., represented by the pick operator 1804A), the switch node 1806 of the data flow diagram 1800 thus corresponds to a switch operator 1806A (e.g., represented by the switch operator 1806A), and the multiplier node 1808 of the data flow diagram 1800 thus corresponds to a multiplier operator 1808A (e.g., represented by the multiplier operator 1808A). Another processing element and / or flow control path network may provide control signals (e.g., control tokens) to the pick operator 1804A and the switch operator 1806A to perform ​ the operations therein. In one embodiment, the array of processing elements 1801 is configured to perform ​ the data flow diagram 1800 prior to the start of execution. In one embodiment, a compiler performs the transformation from ​ thereof. In one embodiment, the input of the data flow diagram node entering the array of processing elements logically embeds the data flow diagram into the array of processing elements (e.g., as further discussed below) such that the input / output paths are configured to produce the desired result.

[0190] ​

[0191] A communication arc is the second primary component of a data flow graph. Some embodiments of the CSA describe these arcs as latency-insensitive channels, e.g., ordered, backpressure-based (e.g., output is not generated or sent until there is space to store it), point-to-point communication channels. Like data flow operators, latency-insensitive channels are fundamentally asynchronous, giving the freedom to combine many types of networks to implement the channels of a particular graph. Latency-insensitive channels can have arbitrarily long latencies and still faithfully implement the CSA architecture. However, in some embodiments, there is a strong motivation in terms of performance and energy to keep the latency as small as possible. Section 3.2 herein discloses a network microarchitecture in which data flow graph channels are implemented in a pipelined manner with a latency of no more than one cycle. Embodiments of latency-insensitive channels provide a key abstraction layer that can be leveraged with the CSA architecture to provide many runtime services to application programmers. For example, the CSA can utilize latency-insensitive channels when implementing a CSA configuration (loading a program onto the CSA array).

[0192] ​ An example execution of data flow graph 1900 in accordance with an embodiment of the present disclosure is illustrated. At step 1, input values (e.g., 1 for X in ​ and 2 for Y in ​ ) can be loaded into data flow graph 1900 to perform a 1*2 multiplication operation. One or more of the data input values can be static (e.g., constant) in the operation (e.g., referring to ​ , X is 1 and Y is 2) or updated during the operation. At step 2, a processing element or other circuitry (e.g., on a flow control path network) outputs 0 to the control input of selection node 1904 (e.g., a mux control signal) (e.g., obtaining "1" as a source from a port and sending it to its output), and outputs 0 to control the input of switching node 1906 (e.g., a mux control signal) (e.g., so that its input departs from port "0" and is provided outward to a destination (e.g., a downstream processing element)). At step 3, the data value 1 is output from selection node 1904 (and, e.g., consumes its control signal "0" at selection node 1904) to multiplier node 1908 to be multiplied by data value 2 at step 4. At step 4, the output of multiplier node 1908 arrives at switching node 1906, e.g., which causes switching node 1906 to consume control signal "0" to output value 2 from port "0" of switching node 1906 at step 5. Subsequently, the operation is complete. Thus, the CSA can be programmed accordingly such that the corresponding data flow operator of each node performs the operation in ​ . Although the execution is serialized in this example, in principle all data flow operations can be performed in parallel. In ​Use steps in order to distinguish data flow execution from any physical microarchitecture manifestation. In one embodiment, a downstream processing element is used to (e.g., on a flow control path network) send a signal (or a not-ready signal) to the switching device 1906 to stop the output from the switching device 1906 until the downstream processing element is ready for the output (e.g., has storage space).

[0193] ​

[0194] Data flow architectures generally focus on communication and data manipulation and are less concerned with state. However, enabling actual software, especially programs written in traditional sequential languages, requires a great deal of attention to the interface with memory. Some embodiments of CSA use architectural memory operations as their primary interface to (e.g., large) stateful storage. From the perspective of a data flow graph, memory operations are similar to other data flow operations, except that memory operations have the side effect of updating shared storage. Specifically, memory operations in some embodiments herein have the same semantics as every other data flow operator, e.g., these memory operations "execute" when their operands (e.g., addresses) are available and a response is generated after some latency. Some embodiments herein explicitly decouple operand inputs from result outputs, such that memory operators are essentially pipelined and have the potential to generate many simultaneous outstanding requests, thus, for example, making memory operators very adaptable to the latency and bandwidth characteristics of the memory subsystem. Embodiments of CSA provide basic memory operations, such as load and store, where the load takes an address channel and fills a response channel with the value corresponding to that address. Embodiments of CSA also provide more advanced operations (such as in-memory atomic and coherence operators). These operations may have semantics similar to their von Neumann counterparts. Embodiments of CSA may accelerate existing programs described using sequential languages such as C and Fortran. The result of supporting these language models is to address program memory order, e.g., the serial ordering of memory operations typically prescribed by these languages.

[0195] ​ FIG. 2000 illustrates a program source (e.g., C code) according to an embodiment of the present disclosure. According to the memory semantics of the C programming language, a memory copy (memcpy) should be serialized. However, if it is known that array A and array B do not intersect, the memcpy can be parallelized using an embodiment of CSA. ​Further illustrate the problem of program order. Generally speaking, the compiler cannot prove that array A is different from array B. For example, whether for the same index value or for different index values across loop bodies. This is called pointer or memory aliasing. Since compilers are used to generate statically correct code, they are often forced to serialize memory accesses. Typically, compilers for serialized von Neumann architectures use instruction ordering as a natural means of implementing program order. However, embodiments of the CSA do not have the concept of instruction ordering as defined by the program counter or instruction-based program ordering. In some embodiments, incoming dependency tokens (e.g., which do not contain architecturally visible information) are like all other data flow tokens, and memory operations cannot be executed until they receive a dependency token. In some embodiments, once the operations of a memory operation are logically visible to subsequent dependent memory operations, these memory operations generate an outgoing dependency token. In some embodiments, dependency tokens are similar to other data flow tokens in a data flow graph. For example, since memory operations occur in a conditional context, dependency tokens can also be manipulated using the control operators described in Section 2.1 (e.g., like any other token). Dependency tokens can have the effect of serializing memory accesses, thus providing, for example, a means for the compiler to define the order of memory accesses architecturally.

[0196] ​

[0197] The main architectural aspects considered for embodiments of CSA relate to the actual execution of user-level programs, but it is also desirable to provide several support mechanisms that reinforce this execution. The primary among these are configuration (where the data flow graph is loaded into the CSA), extraction (where the state of the execution graph is moved to memory), and exceptions (where mathematical, soft, and other types of errors in the structure may be detected and handled by external entities). Section 3.6 below discusses the latency-insensitive data flow architecture properties of embodiments of CSA that implement these functions in an efficient, highly pipelined manner. Conceptually, configuration can load the state of the data flow graph (e.g., generally from memory) into the interconnect (and / or communication network (e.g., its network data flow endpoint circuitry)) and processing elements (e.g., the fabric). During this step, all the fabric in the CSA can be loaded with a new data flow graph, and any data flow tokens survive in that graph, e.g., as a result of a context switch. The latency-insensitive semantics of the CSA can permit distributed asynchronous initialization of the fabric; for example, once the PEs are configured, they can immediately start execution. Unconfigured PEs can backpressure their channels until they are configured, thus preventing communication, for example, between configured and unconfigured elements. The CAS configuration can be partitioned into privileged and user-level states. Such a two-level partitioning can enable the primary configuration of the fabric to occur without invoking the operating system. In one embodiment of extraction, a logical view of the data flow graph is captured and submitted to memory, e.g., including all live control and data flow tokens and state in the graph.

[0198] Extraction can also play a role in providing reliability guarantees by creating fabric checkpoints. Exceptions in the CSA can generally be caused by the same events that cause exceptions in the processor, such events as illegal operator arguments or reliability, availability, and serviceability (RAS) events. In some embodiments, exceptions are detected at the level of the data flow operator (e.g., checking argument values) or through a modular arithmetic scheme. After an exception is detected, the data flow operator (e.g., circuitry) can stop and emit an exception message that contains, for example, both the operation identifier and some details about the nature of the problem that has occurred. In some embodiments, the data flow operator will remain stopped until it has been reconfigured. Subsequently, the exception message can be passed to the associated processor (e.g., core) for servicing (e.g., which may include extracting the graph for software analysis).

[0199] 2.5 Chip-Level Architecture

[0200] Embodiments of the CSA computer architecture (e.g., for HPC and data center use) are sharded. Figure 21 and Figure 23 illustrates the per-chip deployment of the CSA. Figure 23Shows a full-chip implementation of a CSA. For example, it can be an accelerator for a processor with a core. The main advantage of this architecture can be reduced design risk. For example, it enables the CSA to be fully decoupled from the core during manufacturing. In addition to allowing better component reuse, this also allows components (such as the CSA cache) to consider only the CSA, rather than, for example, needing to incorporate more stringent latency requirements for the core. Ultimately, a separate chip allows the CSA to be integrated with small or large cores. One embodiment of the CSA captures most vector-parallel workloads, such that most vector-type workloads run directly on the CSA. However, in some embodiments, vector-type instructions in the core may be included to support, for example, traditional binaries.

[0201] 3. Microarchitecture

[0202] In one embodiment, the purpose of the CSA microarchitecture is to provide a high-quality implementation of each data flow operator specified by the CAS architecture. Embodiments of the CSA microarchitecture provide that each processing element (and / or communication network (e.g., its network data flow endpoint circuit)) of the microarchitecture corresponds to approximately one node (e.g., entity) in the architecture data flow graph. In one embodiment, the nodes in the data flow graph are distributed among multiple network data flow endpoint circuits. In some embodiments, this results in an architecture element that is not only compact but also energy-efficient. Compactness leads to a dense computing array, and energy efficiency means that, for example, the processing element (PE) is both simple and highly underutilized (e.g., performing a single data flow operation for the configuration (e.g., programming) of the CSA). To further reduce energy and implementation area, the CSA may include a configurable heterogeneous architecture style, where each of its PEs implements only a subset of the data flow operators (e.g., using separate subsets of the data flow operators implemented in (multiple) network data flow endpoint circuits). Peripheral and support subsystems (such as the CSA cache) can be provided to support the current state of distributed parallelism in the main CSA processing structure itself. The implementation of the CSA microarchitecture can implement the data flow and latency-insensitive communication abstractions present in the architecture. In some embodiments, there is (substantially) a one-to-one correspondence between the nodes in the compiler-generated graph and the data flow operators (e.g., data flow operator computing elements) in the CSA.

[0203] The following is a discussion of an example CSA, followed by a more detailed discussion of the microarchitecture. Some embodiments herein provide a CSA that allows for easy compilation. For example, in contrast to existing FPGA compilers, which handle a small subset of programming languages (e.g., C or C++), and even compiling small programs can take many hours.

[0204] Some embodiments of the CSA architecture permit heterogeneous coarse-grained operations such as double-precision floating point. Programs can be expressed in less coarse-grained operations, such that, for example, the disclosed compiler runs faster than traditional spatial compilers. Some embodiments include a structure with new processing elements to support serialization concepts such as program-ordered memory access. Some embodiments implement hardware for supporting coarse-grained data-flow type communication channels. Such a communication model is abstract and very close to the control data-flow representation used by the compiler. Some embodiments herein include network implementations that support single-cycle latency communication, e.g., leveraging (e.g., small) PEs that support single control data-flow operations. In some embodiments, this not only improves energy efficiency and performance, but also simplifies compilation, since the compiler makes a one-to-one mapping between high-level data-flow constructs and the structure. Some embodiments herein thus simplify the task of compiling existing (e.g., C, C++, or Fortran) programs to CSA (e.g., the structure).

[0205] Energy efficiency can be a primary consideration in modern computer systems. Some embodiments herein provide a new mode of high-energy efficiency spatial architectures. In some embodiments, these architectures form a structure having a unique composition of a heterogeneous mix of small, energy-efficient, data-flow-oriented processing elements (PEs) (and / or packet-switched type communication networks (e.g., whose network data-flow endpoint circuits)) and a lightweight circuit-switched communication network (e.g., an interconnect), e.g., the structure having enhanced support for flow control. Due to the energy advantages of each, the combination of these components can form a spatial accelerator (e.g., as part of a computer) suitable for executing compiler-generated parallel programs in an extremely energy-efficient manner. Since the structure is heterogeneous, some embodiments can be customized for different application domains by introducing new domain-specific PEs. For example, a structure for high-performance computing can include some customization for double-precision, fused multiply-add, while a structure for deep neural networks can include low-precision floating point operations.

[0206] Spatial architecture patterns (e.g., as exemplified in Figure 21 are composed of lightweight PEs network-connected among processing elements (PEs). Generally, a PE can include a data-flow operator, e.g., where once (e.g., all) input operands arrive at the data-flow operator, an operation (e.g., a micro-instruction or a set of micro-instructions) is executed and the result is forwarded to downstream operators. Thus, control, scheduling, and data storage can be distributed among multiple PEs, thereby, for example, removing the overhead of the centralized structure that dominates classical processors.

[0207] The program can be transformed into a data flow graph by configuring the PEs and the network to represent the control data flow graph of the program, and this data flow graph is mapped onto the architecture. The communication channels can be flow-controlled and fully backpressureable, such that for example if the source communication channel has no data or the destination communication channel is full, the PE will stop. In one embodiment, at runtime, data flows through PEs and channels that have been configured to implement an operation (e.g., an accelerated algorithm). For example, data can flow into the fabric from memory and then back out to memory.

[0208] Embodiments of such architectures can achieve superior performance efficiency relative to traditional multi-core processors: the computation (e.g., in the form of PEs) can be simpler, more energy-efficient, and more plentiful compared to in larger cores, and communication can be direct and mainly short-range as opposed to being across a wide on-chip network as in a typical multi-core processor. Additionally, since embodiments of the architecture are highly parallel, many powerful circuit and device-level optimizations are possible without severely affecting throughput, such as low-leakage devices and low operating voltages. These lower-level optimizations can achieve even greater performance advantages relative to traditional cores. The combination of efficiency yielded at the architecture level, circuit level, and device level for these embodiments is compelling. As transistor density continues to increase, embodiments of this architecture can achieve a larger effective area.

[0209] Embodiments herein provide a unique combination of data flow support and circuit switching to enable a fabric to be smaller, more energy-efficient, and provide higher aggregate performance compared to prior architectures. FPGAs are generally tuned towards fine-grained bit manipulation, while embodiments herein are tuned towards double-precision floating-point operations found in HPC applications. Some embodiments herein may include FPGAs in addition to the CSA according to the present disclosure.

[0210] Some embodiments herein combine a lightweight network with energy-efficient data flow processing elements (and / or communication networks (e.g., their network data flow endpoint circuits)) to form a high-throughput, low-latency, energy-efficient HPC fabric. This low-latency network allows the establishment of processing elements (and / or communication networks (e.g., their network data flow endpoint circuits)) with fewer functions (e.g., only one or two instructions, and perhaps only one architecturally visible register, since it is efficient to group multiple PEs together to form a complete program).

[0211] Relative to a processor core, the CSA embodiments herein can provide more computational density and energy efficiency. For example, when the PEs (e.g., as compared to cores) are very small, the CSA can perform many more operations than a core and can have much more computational parallelism than a core, e.g., perhaps up to 16 times the number of FMAs of a vector processing unit (VPU). To utilize all these computational elements, in some embodiments, the energy per operation is very low.

[0212] There are many energy advantages to embodiments of the data flow architecture of this application. The parallelism is explicit in the data flow graph, and embodiments of the CSA architecture do not expend energy or expend minimal energy to extract this parallelism, e.g., unlike out-of-order processors which must rediscover parallelism every time an instruction is executed. In one embodiment, since each PE is responsible for a single operation, the register file and port count can be small, e.g., often only one, and thus use less energy than their counterparts in a core. Some CSAs include many PEs, each of which holds live program values, giving the aggregate effect of a huge register file in a traditional architecture, which significantly reduces memory accesses. In embodiments where the memory is multi-port and distributed, the CSA can maintain many more outstanding memory requests and utilize more bandwidth than a core. These advantages can be combined to achieve an energy per watt rating that is only a small percentage of the cost of a bare arithmetic circuit. For example, in the case of integer multiplication, the CSA can consume no more than 25% more energy than the underlying multiplication circuit. Relative to one embodiment of a core, the integer operations in that CSA structure consume less than 1 / 30 of the energy per integer operation.

[0213] From a programming perspective, the application-specific compliance implementation of embodiments of the CSA architecture achieves significant advantages over vector processing units (VPUs). In traditional inflexible architectures, the number of functional units such as floating-point division or various transcendental math functions must be selected at design time based on some expected use cases. In embodiments of the CSA architecture, such functions can be configured (e.g., by the user rather than the manufacturer) into the architecture based on the requirements of each application. The application throughput can thus be further increased. At the same time, by avoiding hardcoding such functions and instead supplying more instances of primitive functions such as floating-point multiplication, the computational density of embodiments of the CSA is improved. These advantages can be significant in HPC workloads, some of which spend 75% of their floating-point execution time in transcendental functions.

[0214] Some embodiments of the CSA represent significant progress as a dataflow-oriented spatial architecture. For example, the PEs of the present disclosure can be smaller but also more energy-efficient. These improvements can directly stem from the combination of dataflow-oriented PEs and lightweight, circuit-switched interconnects, such as those with a single-cycle latency, which, for example, contrast with packet-switched networks (such as those with a latency of at least 300% higher). Some embodiments of the PEs support 32-bit or 64-bit operations. Some embodiments herein allow for the introduction of new application-specific PEs, for example, for machine learning or security, and not just homogeneous combinations. Some embodiments herein combine lightweight, dataflow-oriented processing elements with lightweight, low-latency networks to form an energy-efficient computing fabric.

[0215] For some spatial architectures to succeed, programmers will spend relatively little effort configuring them, for example, while achieving significant power and performance advantages over serialized cores. Some embodiments herein provide a CSA (e.g., a spatial fabric) that is easy to program (e.g., by a compiler), highly power-efficient, and highly parallel. Some embodiments herein provide a network (e.g., an interconnect) that achieves these three goals. From the perspective of programmability, some embodiments of the network provide flow-controlled channels, which, for example, correspond to the control data flow graph (CDFG) model of execution used in compilers. Some network embodiments utilize dedicated circuit-switched links, making program performance more easily derivable by both humans and compilers because the performance is predictable. Some network embodiments provide both high bandwidth and low latency. Some network embodiments (e.g., static, circuit-switched) provide a latency of 0 to 1 cycle (e.g., depending on the transmission distance). Some network embodiments provide high bandwidth by arranging several networks in parallel (and, for example, in low-level metal). Some network embodiments communicate in low-level metal and over short distances, and thus are very power-efficient.

[0216] Some embodiments of the network include architectural support for flow control. For example, in a spatial accelerator composed of small processing elements (PEs), communication latency and bandwidth can be critical for overall program performance. Some embodiments herein provide a lightweight, circuit-switched network and the microarchitectural control features required to support the network, the lightweight, circuit-switched network facilitating a spatial processing array, such as Figure 21Communication between PEs in the spatial array (as shown in). Some embodiments of the network implement the construction of point-to-point, flow-controlled communication channels that support the communication of data flow-oriented processing elements (PEs). In addition to point-to-point communication, some networks herein also support multicast communication. The communication channels can be formed by statically configuring the network to form virtual circuits between PEs. The circuit-switching technology herein can reduce communication latency and accordingly minimize network buffering, thereby, for example, resulting in both high performance and high energy efficiency. In some embodiments of the network, the latency between PEs can be as low as zero cycles, which means that the downstream PE can operate on the data in that cycle after the data is generated. To achieve even higher bandwidth and to permit more programs, multiple networks can be arranged in parallel, e.g., as Figure 21 shown in.

[0217] A spatial architecture (such as, Figure 21 the spatial architecture shown in) can be composed of lightweight processing elements connected by a network between PEs (and / or a communication network (e.g., its network data flow endpoint circuit)). A program regarded as a data flow graph can be mapped onto the architecture by configuring the PEs and the network. Generally, a PE can be configured as a data flow operator, and once (e.g., all) input operands reach the PE, some subsequent operations can occur and the results are forwarded to the required downstream PEs. The PEs can communicate through dedicated virtual circuits that are formed by statically configuring a circuit-switching communication network. These virtual circuits can be flow-controlled and fully back pressure, such that, for example, if the source has no data or the destination is full, the PE will stop. At runtime, data can flow through the PEs implementing the mapped algorithm. For example, data can flow in from memory through the structure and then back out to memory. Embodiments of this architecture can achieve excellent performance efficiency compared to traditional multi-core processors: for example, where, in contrast to an extended memory system, the computing in the form of PEs is simpler and more numerous than larger cores, and the communication is direct.

[0218] Figure 21FIG. illustrates an accelerator tile 2100 according to an embodiment of the present disclosure. The accelerator tile 2100 includes an array of processing elements (PEs). The interconnect network is depicted as a circuit-switched, statically configured communication channel. For example, a set of channels are coupled together by switching devices (e.g., switching device 2110 in the first network and switching device 2120 in the second network). The first network and the second network may be separate or may be coupled together. For example, the switching device 2110 may couple one or more of four data paths 2112, 2114, 2116, 2118 together, e.g., configured to perform operations according to a data flow graph. In one embodiment, the number of data paths is any number. The processing element (e.g., processing element 2104) may be as disclosed herein, e.g., as Figure 24 as that in. The accelerator tile 2100 includes a memory / cache hierarchy interface 2102 to interface the accelerator tile 2100 with a memory and / or cache, for example. A data path (e.g., 2118) may extend to another tile or may terminate at, e.g., the edge of the tile. The processing element may include an input buffer (e.g., buffer 2106) and an output buffer (e.g., buffer 2108).

[0219] Operations may be performed based on the availability of inputs for these operations and the state of the PEs. The PEs may obtain operands from input channels and may write results to output channels, but may also use internal register states. Some embodiments herein include configurable data flow-friendly PEs. Figure 24 A detailed block diagram of one such PE is shown: an integer PE. The PE consists of several I / O buffers, an ALU, storage registers, some instruction registers, and a scheduler. In each cycle, the scheduler may select an instruction for execution based on the availability of the input and output buffers and the state of the PE. Subsequently, the result of the operation is written to an output register or to a register (e.g., local to the PE). The data written to the output buffer may be transferred to a downstream PE for further processing. This PE style can be extremely energy-efficient. For example, instead of reading data from a complex multi-port register file, the PE reads the data from a register. Similarly, instructions may be stored directly in registers rather than in a virtualized instruction cache.

[0220] The instruction registers may be set during a special configuration step. During this step, in addition to the PE-to-PE network, auxiliary control lines and status may also be used to flow configuration across several PEs of an encompassing structure. As a result of the parallelism, some embodiments of such networks may provide fast reconfiguration. For example, a structure of chip size may be configured in less than about 10 microseconds.

[0221] Figure 24Represents an example configuration of a processing element, e.g., where all architectural elements are sized to a minimum. In other embodiments, each of the multiple components of the processing element is scaled independently to produce a new PE. For example, to handle more complex programs, a greater number of instructions executable by the PE may be introduced. The second dimension of configurability is the functionality of the PE arithmetic logic unit (ALU). In Figure 24 , the integer PE is depicted as being able to support addition, subtraction, and various logical operations. By replacing different kinds of functional units into the PE, other kinds of PEs can be created. For example, an integer multiplication PE may not have registers, have a single instruction, and have a single output buffer. Some embodiments of the PE decompose fused multiply-add (FMA) into separate but tightly coupled floating-point multiplication and floating-point addition units to improve support for multiply-heavy workloads. The PE is further discussed below. <...< /

[0222] Figure 22A Illustrates a configurable data path network 2200 according to an embodiment of the present disclosure (e.g., in network one or network two as discussed with reference to Figure 21 . The network 2200 includes a plurality of multiplexers (e.g., multiplexers 2202, 2204, 2206), which can be configured (e.g., via their respective control signals) to connect one or more data paths (e.g., from a PE) together. Figure 22B Illustrates a configurable flow control path network 2201 according to an embodiment of the present disclosure (e.g., in network one or network two as discussed with reference to Figure 21 . The network can be a lightweight PE-to-PE network. Some embodiments of the network can be considered a collection of building blocks for constructing distributed point-to-point data channels. Figure 22A Shows a network in which two channels (bold black line and dotted black line) are enabled. The bold black line channel is multicast, e.g., a single input is sent to two outputs. Note that even though dedicated circuit-switched paths are formed between channel endpoints, the channels can cross at some points within a single network. Additionally, this crossing does not introduce structural hazards between the two channels, such that each operates independently and at full bandwidth.

[0223] Implementing a distributed data channel can include Figure 22A - Figure 22B the two paths shown in. The forward or data path carries data from a producer to a consumer. The multiplexer can be configured to direct data and valid bits from the producer to the consumer, e.g., as shown in Figure 22A . In the case of multicast, the data will be directed to multiple consumer endpoints. The second part of this embodiment of the network is the flow control or backpressure path, which flows in the opposite direction to the forward data path, e.g., as shown in Figure 22BAs shown. The consumer endpoints can assert when they are ready to receive new data. Subsequently, configurable logic knots (labeled as (e.g., backflow) flow control functions in Figure 22B can be used to direct these signals back to the producer. In one embodiment, each flow control function circuit can be a plurality of switching devices (e.g., a plurality of muxes), e.g., similar to Figure 22A . The flow control path can handle control data returning from the consumer to the producer. The knot can enable multicast, e.g., where each consumer is ready to receive data before the producer assumes the data has been received. In one embodiment, the PE is a PE with a data stream operator as its architectural interface. Additionally or alternatively, in one embodiment, the PE can be any kind of PE (e.g., in a fabric), such as but not limited to a PE with an instruction pointer, triggered instructions, or a state machine-based architectural interface.

[0224] In addition to, for example, the PE being statically configured, the network can also be statically configured. During this configuration step, configuration bits can be set at each network component. These bits control, for example, mux selection and flow control functions. The network can include multiple networks, e.g., a data path network and a flow control path network. The network or multiple networks can utilize paths of different widths (e.g., a first width and a narrower or wider width). In one embodiment, the data path network has a wider (e.g., bit transfer) width than the flow control path network. In one embodiment, each of the first network and the second network includes its own data path network and flow control path network, e.g., data path network A and flow control path network A and a wider data path network B and flow control path network B.

[0225] Some embodiments of the network are bufferless, and data is used to move between the producer and the consumer in a single cycle. Some embodiments of the network are also borderless, i.e., the network spans the entire fabric. In one embodiment, one PE is used to communicate with any other PE in a single cycle. In one embodiment, to improve routing bandwidth, several networks can be arranged in parallel between the rows of PEs.

[0226] Certain embodiments of the network herein have three advantages over FPGAs: area, frequency, and program expressiveness. Certain embodiments of the network herein operate at a coarse-grained level, which for example reduces the number of configuration bits and thus the area of the network. Certain embodiments of the network also achieve area reduction by directly implementing flow control logic in the circuit (e.g., silicon). Certain embodiments of the enhanced network implementation also enjoy a frequency advantage over FPGAs. Due to the area and frequency advantages, a power advantage may exist when using a lower voltage at throughput parity. Finally, certain embodiments of the network provide better high-level semantics than FPGA wires, especially with respect to variable timing aspects, and thus those embodiments are more amenable to being targeted by compilers. Certain embodiments of the network herein can be viewed as a collection of composable primitives for building distributed point-to-point data channels.

[0227] In certain embodiments, a multicast source may not be able to assert its data valid unless the multicast source receives a ready signal from each sink. Thus, in the multicast case, additional knots and control bits may be utilized.

[0228] Like certain PEs, the network can be statically configured. During this step, configuration bits are set at each network component. These bits control, for example, mux selection and flow control functions. The forward path of the network of this application requires some bits to swing the mux of the forward path. In the example shown in Figure 22A , four bits are required per hop: one bit is utilized for each of the east mux and the west mux, while two bits are utilized for the south mux. In this embodiment, four bits can be used for the data path, but seven bits can be used for flow control functions (e.g., in the flow control path network). If, for example, the CSA further utilizes the north-south direction, other embodiments may utilize more bits. The flow control function can use control bits for each direction from which flow control can come. This can achieve statically setting the sensitivity of the flow control function. Table 2 below summarizes the Boolean algebra implementation of the flow control function for the network in Figure 22B , with the configuration bits in uppercase. In this example, seven bits are utilized.

[0229] Table 2: Flow implementation

[0230]

[0231] For the third flow control box from the left in Figure 22B , EAST_WEST_SENSITIVE and NORTH_SOUTH_SENSITIVE are depicted as being set to implement flow control for the bold line channel and the dotted line channel, respectively.

[0232] Figure 23FIG. illustrates a hardware processor die 2300 including an accelerator 2302 according to an embodiment of the present disclosure. The accelerator 2302 may be a CSA according to the present disclosure. The die 2300 includes a plurality of cache blocks (e.g., cache block 2308). A Request Address File (RAF) circuit 2310 may be included, e.g., as discussed in Section 3.2 below. ODI may refer to on-die interconnect, e.g., an interconnect that stretches across the entire die and connects all dies. OTI may refer to on-chip interconnect (e.g., that stretches across the die, e.g., connecting cache blocks on the die together).

[0233] 3.1 Processing Element

[0234] In some embodiments, the CSA includes an array of heterogeneous PEs, where the fabric is composed of several types of PEs, each of which implements only a subset of the data flow operators. As an example, Figure 24 illustrates a tentative implementation of a PE capable of implementing a broad set of integer and control operations. Other PEs (including those that support floating-point addition, floating-point multiplication, buffering, and certain control operations) may also have a similar implementation style, e.g., replacing the ALU with appropriate (data flow operator) circuitry. Before execution begins, the PEs (e.g., data flow operators) of the CSA may be configured (e.g., programmed) to implement a specific data flow operation from the set supported by the PE. The configuration may include one or two control words that specify the opcode controlling the ALU, direct the various multiplexers within the PE, and drive data flow into and out of the PE channels. The data flow operator may be implemented by microcoding these configuration bits. Figure 24 The integer PE 2400 depicted in is organized as a single-stage logic pipeline that flows from the top to the bottom. Data enters the PE 2400 from one of the local networks in the local network set, where the data is stored in an input buffer for subsequent operations. Each PE may support multiple wide data-oriented channels and narrow control-oriented channels. The number of channels supplied may vary based on the function of the PE, but one embodiment of an integer-oriented PE has 2 wide and 1 - 2 narrow input and output channels. Although the integer PE is implemented as a single-cycle pipeline, other pipelined options may be utilized. For example, a multiplication PE may have multiple pipeline stages.

[0235] PE execution can continue in a data flow style. Based on the configured microcode, the scheduler can check the status of the PE input and output buffers and schedule the actual execution of an operation by a data operator (e.g., on an ALU) when all inputs for the configured operation have arrived and the output buffer for the operation is available. The resulting value can be placed in the configured output buffer. Transfer between the output buffer of one PE and the input buffer of another PE can occur asynchronously when the buffer becomes available. In some embodiments, PEs are provided such that at least one data flow operation is completed per cycle. Section 2 discusses data flow operators that cover primitive operations such as add, xor, or select. Some embodiments can provide advantages in terms of energy, area, performance, and latency. In one embodiment, more fused combinations can be enabled by an extension to the PE control path. In one embodiment, the width of the processing element is 64 bits, for example, for high utilization of double-precision floating-point calculations in HPC and for supporting 64-bit memory addressing.

[0236] 3.2 Communication Network

[0237] Embodiments of the CSA microarchitecture provide a hierarchy of multiple networks that together provide an implementation of an architectural abstraction of latency-insensitive channels across multiple communication scales. The lowest level of the CSA communication hierarchy can be the local network. The local network can be statically circuit-switched, e.g., using configuration registers to swing the (multiple) multiplexers in the local network data path to form a fixed circuit path between communicating PEs. In one embodiment, the configuration of the local network is set once for each data flow graph (e.g., while the PEs are being configured). In one embodiment, the static circuit switching is optimized for energy, e.g., where the vast majority (perhaps greater than 95%) of the CSA communication traffic will cross the local network. A program may include terms used in multiple expressions. To optimize for this situation, embodiments herein provide hardware support for multicast within the local network. A number of local networks can be aggregated together to form routing channels that are, e.g., spread between rows and columns of PEs (as a grid). As an optimization, a number of local networks can be included to carry control tokens. Compared to FPGA interconnects, the CSA local network can be routed at the granularity of the data path, and another difference can be the CSA's handling of control. One embodiment of the CSA local network is explicitly flow-controlled (e.g., backpressure). For example, for each forward data path and multiplexer set, the CSA provides a backward-flowing flow control path that is physically paired with the forward data path. The combination of the two microarchitecture paths can provide a low-latency, low-energy, small-area, point-to-point implementation of the latency-insensitive channel abstraction. In one embodiment, the flow control lines of the CSA are not visible to the user program, but these flow control lines can be manipulated by the architecture that maintains the user program. For example, the exception handling mechanism described in Section 2.2 can be implemented by pulling the flow control lines to the "non-existent" state after an abnormal condition is detected. This action can not only gently stop those parts of the pipeline involved in the offending computation, but also preserve the machine state prior to the exception for, e.g., diagnostic analysis. The second network layer (e.g., the mezzanine network) can be a shared packet-switched network. The mezzanine network can include multiple distributed network controllers, network data flow endpoint circuits. The mezzanine network (e.g., by Figure 37The network schematically indicated by the dashed box in [can provide more general long - distance communication at the cost of, for example, latency, bandwidth, and energy. In some programs, most communication can occur on the local network. Thus, comparatively, the mezzanine network provision will be significantly reduced. For example, each PE can be connected to multiple local networks, but the CSA will only provision one mezzanine endpoint for each logical neighborhood of the PE. Since the mezzanine is effectively a shared network, each mezzanine network can carry multiple logically independent channels and be provisioned, for example, with multiple virtual channels. In one embodiment, the primary function of the mezzanine network is to provide a wide - range communication between PEs and between the PE and the memory. In addition to this ability, the mezzanine can also include, for example, (multiple) network data - stream endpoint circuits for certain data - stream operations. In addition to this ability, the mezzanine can also operate as a runtime - support network. For example, through this runtime - support network, various services can access the complete fabric in a user - program - transparent manner. In this regard, the mezzanine endpoint can act as a controller for its local neighborhood during, for example, CSA configuration. To form a channel across the CSA tile, three sub - channels and two local - network channels (which carry traffic to and from a single channel in the mezzanine network) can be utilized. In one embodiment, one mezzanine channel is utilized, for example, one mezzanine and two locals = a total of 3 network hops.

[0238] The composability of channels across network layers can be extended to higher - level network layers at the tile - to - tile, die - to - die, and fabric granularity.

[0239] Figure 24 FIG. illustrates a processing element 2400 according to an embodiment of the present disclosure. In one embodiment, the operation - configuration register 2419 is loaded during configuration (e.g., mapping) and specifies the particular operation (or operations) that the processing (e.g., computing element) will perform. The activity of register 2420 can be controlled by that operation (the output of mux 2416, e.g., controlled by scheduler 2414). For example, when input data and control inputs arrive, scheduler 2414 can schedule one or more operations of processing element 2400. The control - input buffer 2422 is connected to the local network 2402 (e.g., and the local network 2402 can include, as Figure 22A the data - path network in [and as Figure 22Bin the flow control path network), and when a value arrives (e.g., the network has (multiple) data bits and (multiple) valid bits), load the control input buffer 2422 with that value. The control output buffer 2432, data output buffer 2434, and / or data output buffer 2436 may receive the output of the processing element 2400 (e.g., as controlled by the operation (output of mux 2416)). Whenever the ALU 2418 executes (also controlled by the output of mux 2416), the status register 2438 may be loaded. The data in the control input buffer 2422 and control output buffer 2432 may be single bits. Mux 2421 (e.g., operand A) and mux 2423 (e.g., operand B) may serve as sources of inputs.

[0240] For example, assume the operation of the processing (e.g., computing) element is (or includes) an operation called call pick in Figure 18B Then the processing element 2400 is used to select data from the data input buffer 2424 or data input buffer 2426, for example, to go to the data output buffer 2434 (e.g., default case) or data output buffer 2436. Thus, if selected from the data input buffer 2424, the control bit in 2422 may indicate 0, or if selected from the data input buffer 2426, the control bit in 2422 may indicate 1.

[0241] For example, assume the operation of the processing (e.g., computing) element is (or includes) an operation called call switch in Figure 18B The processing element 2400 is used to output data from, for example, the data input buffer 2424 (e.g., default case) or data input buffer 2426 to the data output buffer 2434 or data output buffer 2436. Thus, if output to the data output buffer 2434, the control bit in 2422 may indicate 0, or if output to the data output buffer 2436, the control bit in 2422 may indicate 1.

[0242] Multiple networks (e.g., interconnections) (e.g., (input) networks 2402, 2404, 2406 and (output) networks 2408, 2410, 2412) may be connected to the processing element. The connections may be, for example, with reference to Figure 22A and Figure 22B the switching devices discussed. In one embodiment, each network includes two sub-networks (or two channels on the network), e.g., one for Figure 22A the data path network in Figure 22BThe flow control (e.g., backpressure) path network in. As an example, the local network 2402 (e.g., established as a control interconnect) is depicted as being switched (e.g., connected) to the control input buffer 2422. In this embodiment, the data path (e.g., Figure 22A the network in) can carry control input values (e.g., one or more bits) (e.g., control tokens), and the flow control path (e.g., network) can carry a backpressure signal (e.g., backpressure or no-backpressure token) from the control input buffer 2422 to indicate to an upstream producer (e.g., PE), for example, that a new control input value will not be loaded into (e.g., sent to) the control input buffer 2422 until the backpressure signal indicates there is space in the control input buffer 2422 for the new control input value (e.g., from the control output buffer of the upstream producer). In one embodiment, the new control input value can enter the control input buffer 2422 only when both (i) the upstream producer receives a "space available" backpressure signal from the "control input" buffer 2422; and (ii) the new control input value is sent from the upstream producer, and this can cause the processing element 2400 to stop until that occurs (and there is space available in the (multiple) destination, output buffers).

[0243] The data input buffer 2424 and the data input buffer 2426 can operate in a similar manner. For example, the local network 2404 (e.g., established as a data (as opposed to control) interconnect) is depicted as being switched (e.g., connected) to the data input buffer 2424. In this embodiment, the data path (e.g., Figure 22A the network in) can carry data input values (e.g., one or more bits) (e.g., data stream tokens), and the flow control path (e.g., network) can carry a backpressure signal (e.g., backpressure or no-backpressure token) from the data input buffer 2424 to indicate to an upstream producer (e.g., PE), for example, that a new data input value will not be loaded into (e.g., sent to) the data input buffer 2424 until the backpressure signal indicates there is space in the data input buffer 2424 for the new data input value (e.g., from the data output buffer of the upstream producer). In one embodiment, the new data input value can enter the data input buffer 2424 only when both (i) the upstream producer receives a "space available" backpressure signal from the "data input" buffer 2424; and (ii) the new data input value is sent from the upstream producer, and this can cause the processing element 2400 to stop until that occurs (and there is space available in the (multiple) destination, output buffers). The control output values and / or data outputs can be stopped in their respective output buffers (e.g., 2432, 2434, 2436) until the backpressure signal indicates there is available space in the input buffer for the (multiple) downstream processing elements.

[0244] The processing element 2400 may cease execution until its operands (e.g., control input values and one or more corresponding data input values for the control input values) are received and / or until there is space in the output buffer(s) of the processing element 2400 for the data that will be produced by performing operations on those operands.

[0245] 3.3 Memory Interface

[0246] The request address file (RAF) circuit (at Figure 25(shown in its simplified version) may be responsible for performing memory operations and act as a mediator between the CSA structure and the memory hierarchy. Thus, the main microarchitecture task of the RAF can be to rationalize the out-of-order memory subsystem with the ordered semantics of the CSA structure. In this capacity, the RAF circuitry may be supplied with completion buffers (e.g., queue-like structures) that reorder memory responses and return these memory requests to the structure in the order of the requests. A second main function of the RAF circuitry can be to provide support in the form of address translation and page walkers. Incoming virtual addresses may be translated to physical addresses using channel-associated translation lookaside buffers (TLBs). To provide sufficient memory bandwidth, each CSA tile may include multiple RAF circuits. Like the various PEs of the structure, the RAF circuits may operate in a dataflow style by checking the availability of input arguments and output buffers (if needed) before selecting a memory operation to perform. However, unlike some PEs, the RAF circuits are multiplexed among several co-located memory operations. The multiplexed RAF circuits can be used to minimize the area overhead of their respective sub-components, thus sharing, for example, accelerator cache interface (ACI) ports (described in more detail in Section 3.4), shared virtual memory (SVM) support hardware, mezzanine network interfaces, and other hardware management facilities. However, there are some program characteristics that also facilitate this choice. In one embodiment, (e.g., efficient) dataflow graphs are used to round-robin memory in the shared virtual memory system. Memory latency-bound programs (like graph traversals) may utilize many separate memory operations to saturate the memory bandwidth due to memory-dependent control flow. Although each RAF can be multiplexed, the CAS may include multiple (e.g., between 8 and 32) RAFs at the tile granularity to ensure sufficient cache bandwidth. The RAF can communicate with the rest of the structure via both the local network and the mezzanine network. In the case where the RAF is multiplexed, each RAF may be supplied with several ports into the local network. These ports can act as the lowest latency, highly deterministic paths to memory for latency-sensitive or high-bandwidth memory operations. Additionally, the RAF may be supplied with mezzanine network endpoints, e.g., that provide memory access to runtime services as well as remote user-level memory accessors.

[0247] Figure 25FIG. illustrates a request address file (RAF) circuit 2500 according to an embodiment of the present disclosure. In one embodiment, upon configuration, memory load and store operations already in the data flow graph are specified in register 2510. Subsequently, arcs to those memory operations in the data flow graph may be connected to input queues 2522, 2524, and 2526. Arcs from those memory operations are thus used to leave completion buffers 2528, 2530, or 2532. Dependency tokens (which may be multiple individual bits) arrive at queues 2518 and 2520. Dependency tokens will leave from queue 2516. The dependency token counter 2514 may be a compact representation of the queue and may keep track of the number of dependency tokens for any given input queue. If the dependency token counter 2514 saturates, no additional dependency tokens can be generated for new memory operations. Accordingly, the memory ordering circuit (e.g., Figure 26 the RAF in

[0248] will stop scheduling new memory operations until the dependency token counter 2514 becomes unsaturated.

[0249] As an example of a load, an address arrives at queue 2522 and the scheduler 2512 matches queue 2522 with the load in 2510. The completion buffer slot for that load is assigned in the order in which the address arrives. Assuming no specified dependencies for that particular load in the figure, the address and the completion buffer slot are dispatched to the memory system by the scheduler (e.g., via memory command 2542). When the result returns to mux 2540 (shown schematically), the result is stored in its designated completion buffer slot (e.g., since the result carries the destination slot throughout the memory system). The completion buffer sends the result back to the local network (e.g., local networks 2502, 2504, 2506, or 2508) in the order in which the addresses arrived.

[0250] 3.4 Cache

[0251] The data flow graph may be able to generate a large number (e.g., word granularity) of requests in parallel. Accordingly, some embodiments of the CSA provide sufficient bandwidth to the cache subsystem to maintain the CSA. A highly block-structured cache microarchitecture (e.g., as shown in Figure 26 may be utilized. Figure 26FIG. illustrates a circuit 2600 according to an embodiment of the present disclosure, the circuit 2600 having a plurality of request address file (RAF) circuits (e.g., RAF circuit 1) coupled between a plurality of accelerator slices 2608, 2610, 2612, 2614 and a plurality of cache blocks (e.g., cache block 2602). In one embodiment, the number of RAFs and cache blocks may be in a ratio of 1:1 or 1:2. The cache block may include a complete cache line (e.g., as opposed to being word-sliced), and each line has exactly one home in the cache. The cache line may be mapped to the cache block via a pseudo-random function. The CSA may employ an SVM model to integrate with other sharding architectures. Some embodiments include an accelerator cache interconnect (ACI) network that connects the RAFs to the cache blocks. The network may carry addresses and data between the RAF and the cache. The topology of the ACI may be a cascaded crossbar, for example, as a trade-off between latency and implementation complexity.

[0252] 3.5 Floating-Point Support

[0253] Certain HPC applications are characterized by their need for significant floating-point bandwidth. To meet this need, embodiments of the CSA may be provided with multiple (e.g., each may be provided with between 128 and 256) floating-point addition and multiplication PEs, depending, for example, on the slice configuration. The CSA may provide some other extended precision modes, for example, to simplify the math library implementation. The CSA floating-point PEs may support both single-precision and double-precision, but the lower-precision PEs may support machine learning workloads. The CSA may provide an order of magnitude higher floating-point performance than a processing core. In one embodiment, in addition to increasing the floating-point bandwidth, the energy consumed in floating-point operations is reduced to drive all the floating-point units. For example, to reduce energy, the CSA may selectively gate the low-order bits of the floating-point multiplier array. When examining the behavior of floating-point arithmetic, the low-order bits of the multiplier array may not often affect the final rounded product. Figure 27 FIG. illustrates a floating-point multiplier 2700 partitioned into three regions (a result region, three potential carry regions 2702, 2704, 2706, and a gating region) according to an embodiment of the present disclosure. In some embodiments, the carry regions may affect the result region, while the gating region is less likely to affect the result region. Considering a g-bit gating region, the maximum carry can be:

[0254]

[0255] Given this maximum carry, if the result of the carry region is less than 2 c–g (where the carry region is c bits wide), the gating region can be ignored because it does not affect the result region. Increasing g means it is more likely that the gating region will be needed, while increasing c means that under random assumptions, the gating region will not be used and can be disabled to avoid power consumption. In an embodiment of the CSA floating-point multiplication PE, a two-stage pipelined approach is utilized, where first the carry region is determined, and subsequently, if it is found that the gating region affects the result, the gating region is determined. If more information about the context of the multiplication is known, the CSA adjusts the size of the gating region more aggressively. In an FMA, the multiplication result can be added to an accumulator, which is often much larger than either of the multiplicands. In this case, the addend exponent can be observed in advance before the multiplication, and the CSDA can adjust the gating region accordingly. One embodiment of the CSA includes a scheme where a context value (which bounds the minimum result of the computation) is provided to the relevant multiplier to select the lowest energy gating configuration.

[0256] 3.6 Runtime Services

[0257] In some embodiments, the CSA includes a heterogeneous distributed structure, and thus, runtime service implementations are used to accommodate several kinds of PEs in a parallel distributed manner. Although the runtime services in the CSA may be critical, they may be less frequent relative to user-level computing. Thus, some embodiments focus on overlaying services on the hardware resources. To meet these purposes, the CSA runtime services can be constructed as a hierarchy, for example, each layer corresponding to a CSA network. At the chip level, a single external-facing controller can accept service commands or can send service commands to the cores associated with the CSA chip. The chip-level controller can serve to (e.g., using the ACI network) coordinate the regional controllers at the RAF. The regional controllers can in turn coordinate the local controllers at certain mezzanine network stations (e.g., network data flow endpoint circuits). At the lowest level, service-specific microprotocols can be executed (e.g., during special modes controlled by the mezzanine controller) on the local network. The microprotocols can permit each PE (e.g., by type-partitioned PE classes) to interact with the runtime services according to its own needs. Thus, parallelism is implicit in this hierarchical organization, and operations at the lowest level can occur simultaneously. Depending, for example, on the configuration size of the CSA chip and the position of the CSA chip in the memory hierarchy, this parallelism can be achieved for the configuration of the CSA chip in between a few hundred nanoseconds to a few microseconds. Thus, embodiments of the CSA utilize the properties of the data flow graph to improve the implementation of each runtime service. A key observation is that the runtime services may only need to maintain a legal logical view of the data flow graph (e.g., a state that can be generated by a certain ordering of the data flow operators). The services generally may not need to guarantee the temporal view of the data flow graph (e.g., the state of the data flow graph in the CSA at a given moment). For example, assuming that the service is arranged to maintain a logical view of the data flow graph, this can permit the CSA to perform most runtime services in a distributed, pipelined parallel manner. The local configuration microprotocol can be a packet-based protocol overlaid on the local network. The configuration targets can be organized as a configuration chain, for example, which is fixed in the microarchitecture. The structural (e.g., PE) targets can be configured one at a time, for example, using a single extra register for each target to achieve distributed coordination. To start the configuration, the controller can drive an out-of-band signal that places all the structural targets in its neighborhood into an unconfigured, suspended state and swings the multiplexers in the local network to a predefined configuration. When the structural (e.g., PE) targets are configured (i.e., they have fully received their configuration packets), they can set their configuration microprotocol registers to notify the next target (e.g., PE) that the next target can continue with the configuration using subsequent packets. There is no limit on the size of the configuration packets, and the packets can have a dynamically variable length. For example, a PE that configures constant operands can have a length set to include a constant field (e.g., Figure 18B - Figure 18CConfiguration groups of X and Y in Figure 28 FIG. illustrates the runtime configuration of an accelerator 2800 having a plurality of processing elements (e.g., PEs 2802, 2804, 2806, 2808) according to an embodiment of the present disclosure. Once configured, the PEs can be subject to data flow constraints to execute. However, channels involving unconfigured PEs can be disabled by the microarchitecture, thereby preventing, for example, any undefined operations from occurring. These properties allow embodiments of the CSA to be initialized and executed in a distributed manner without any centralized control. From an unconfigured state, the configuration can occur completely in parallel (e.g., perhaps in as little as 200 nanoseconds). However, due to the distributed initialization of embodiments of the CSA, the PEs may become active, for example, to send requests to memory long before the entire fabric is configured. Fetching can continue in much the same way as configuration. The local network can be adhered to in order to fetch data from one target at a time and fetch status bits for distributed coordination. The CSA can arrange the fetch to be non-destructive, i.e., at the completion of the fetch, each fetchable target has returned to its starting state. In this implementation, all the states in the target can be propagated to the egress registers connected to the local network in a scan-like manner. However, in-place fetch can be achieved by introducing a new path at the register transfer level (RTL) or using existing wires to provide the same function with lower overhead. Similar configuration and hierarchical fetching are implemented in parallel.

[0258] Figure 29 FIG. illustrates a snapshot 2900 of runtime pipelined fetching according to an embodiment of the present disclosure. In some use cases of fetching (such as checkpoint operations), latency may not be a concern as long as the fabric throughput can be maintained. In these cases, the fetching can be arranged in a pipelined manner. Figure 29 This arrangement shown in permits most of the fabric to continue execution while a narrow region is disabled for fetching. Configuration and fetching can be coordinated and composed to achieve pipelined context switching. Qualitatively, exceptions can be different from configuration and fetching in that, unlike occurring at a specified time, exceptions can occur anywhere in the fabric at any time during runtime. Thus, in one embodiment, the exception microprotocol may not overlay the local network and utilizes its own network, which is occupied by the user program at runtime. However, by nature, exceptions are rare and are not sensitive to latency and bandwidth. Thus, some embodiments of the CSA utilize a packet-switched network to carry exceptions to a local mezzanine station where, for example, these exceptions are forwarded further up the service hierarchy (e.g., as Figure 40as shown). Packets in the local exception network can be extremely small. In many cases, a PE identifier (ID) of only 2 to 8 bits suffices as a complete packet, for example because the CSA can create a unique exception identifier as the packet traverses the exception service hierarchy. Such a scheme may be desirable as it reduces the area overhead of generating exceptions at each PE.

[0259] 4. Compilation

[0260] Compiling a program written in a high-level language onto the CSA may be necessary for industrial applications. This section gives a high-level overview of the compilation strategy for the CSA embodiments. First, a CSA software framework is proposed, which describes the desired properties of an ideal production-quality toolchain. Second, a prototype compiler framework is discussed. Then, the "control-data flow transformation" is discussed, which is used, for example, to transform ordinary serialized control flow code into CSA data flow assembly code.

[0261] 4.1 Example Production Framework

[0262] Figure 30 FIG. 3000 illustrates a compilation toolchain for an accelerator according to an embodiment of the present disclosure. The toolchain compiles high-level languages (such as C, C++, and Fortran) into a combination of main code (LLVM) intermediate representation (IR) for a specific region to be accelerated. The CSA-specific part of the compilation toolchain takes the LLVM IR as its input, optimizes and compiles the IR into CSA assembly, for example, adding appropriate buffering on latency-insensitive channels for performance. Subsequently, it places and routes the CSA assembly on the hardware fabric and configures the PEs and the network for execution. In one embodiment, the toolchain supports CSA-specific compilation as just-in-time (JIT) compilation, thereby incorporating potential runtime feedback from actual execution. One of the key design features of the framework is to compile (LLVM) IR to obtain CSA, rather than using a higher-level language as the input. Although programs written in a high-level programming language specifically designed for the CSA can achieve the highest performance and / or energy efficiency, adopting a new high-level language or programming framework may be slow and limited in practice due to the difficulty of converting the existing codebase. Using (LLVM) IR as the input enables a wide range of existing programs to potentially execute on the CSA, for example, without the need to create a new language or significantly modify the front-end of the new language that wants to run on the CSA.

[0263] 4.2 Prototype Compiler

[0264] Figure 31Illustrated is a compiler 3100 for an accelerator according to an embodiment of the present disclosure. The compiler 3100 initially focuses on ahead-of-time compilation of C or C++ via a front-end (e.g., Clang). To compile (LLVM) IR, the compiler implements a CSA backend target within LLVM using three main stages. First, the CSA backend lowers the LLVM IR to target-specific machine instructions for a serialization unit that implements most CSA operations as well as a traditional RISC-like control flow architecture (e.g., utilizing branches and program counters). The serialization unit in the toolchain can serve as an aid useful to both the compiler and the application developer, as the serialization unit allows for an incremental transformation from control flow (CF) to data flow (DF), e.g., transforming a code segment from control flow to data flow at a given moment and verifying program correctness. The serialization unit can also provide a model for handling code that does not fit within a spatial array. Subsequently, the compiler converts these control flow instructions into data flow operators (e.g., code) for the CSA. This stage is described later in Section 4.3. Subsequently, the CSA backend can run its own optimization passes on the data flow instructions. Finally, the compiler can dump the instructions in CSA assembly format. This assembly format is taken as input to a post-stage tool that places and routes the data flow instructions on the actual CSA hardware.

[0265] 4.3 Control-to-Dataflow Transformation

[0266] A key part of the compiler can be implemented in a control-data flow transformation pass (or simply data flow transformation pass). This pass takes a function represented in control flow form, such as a control flow graph (CFG) with serialized machine instructions operating on virtual registers, and converts the function into a data flow function, which conceptually is a graph of data flow operations (instructions) connected by latency-insensitive channels (LICs). This section gives a high-level description of this pass to describe how, in certain embodiments, this pass conceptually handles memory operations, branches, and loops.

[0267] Straight-line code

[0268] Figure 32A Illustrated is a serialized assembly code 3202 according to an embodiment of the present disclosure. Figure 32B Illustrated is an embodiment of the present disclosure for Figure 32A the serialized assembly code 3202 of the data flow assembly code 3204. Figure 32C Illustrated is an embodiment of the present disclosure for an accelerator Figure 32B the data flow assembly code 3204 of the data flow graph 3206.

[0269] First, consider the simple case of converting straight-line serialized code to data flow. The data flow conversion pass can convert a basic serialized code block (such as the code shown in Figure 32A ) to the CSA assembly code shown in Figure 32B . Conceptually, the CSA assembly representation in Figure 32B corresponds to the data flow graph in Figure 32C . In this example, each serialized instruction is converted to a matching CSA assembly. (For example, the.lic declaration for data claims a latency-insensitive channel corresponding to a virtual register (e.g., Rdata) in the serialized code. In practice, the input to the data flow conversion pass can be in numbered virtual registers. However, for clarity, this section uses descriptive register names. Note that in this embodiment, load and store operations are supported in the CSA architecture, allowing many more programs to run compared to architectures that only support pure data flow. Since the serialized code input to the compiler is in SSA (single static assignment) form, for a simple basic block, the control-data flow pass can convert each virtual register definition to the production of a single value on a latency-insensitive channel. The SSA form allows multiple uses of a single definition of a virtual register (such as in Rdata2). To support this model, the CSA assembly code supports multiple uses of the same LIC (e.g., data2), and the simulator implicitly creates the necessary copies of the LIC. A key difference between serialized code and data flow code lies in the handling of memory operations. Figure 32A The code in is conceptually serial, meaning that in the case where the addresses addr and addr3 overlap, the load32 (ld32) of addr3 should appear to occur after the st32 of addr.

[0270] Branch

[0271] To convert a program with multiple basic blocks and conditional statements to data flow, the compiler generates special data flow operators to replace branches. More specifically, the compiler uses a switch operator to direct the outgoing data at the end of the basic blocks in the original CFG, and a pick operator to select values from the appropriate incoming channels at the start of the basic blocks. As a specific example, consider the code in Figure 33A - Figure 33C and the corresponding data flow graph, which conditionally calculates the value of y based on several inputs: a, i, x, and n. After computing the branch condition test, the data flow code uses a switch operator (e.g., see Figure 18B - Figure 18C ) to: direct the value in channel x to channel xF if the test is 0, or direct the value in channel x to channel xT if the test is 1. Similarly, a pick operator (e.g., see Figure 18B - Figure 18C)For: if the test is 0, send channel yF to y, or if the test is 1, send channel yT to y. In this example, it is demonstrated that even though the value of a is only used in the true branch of the conditional statement, the CSA will include a switch operator that directs the value of a to channel aT when the test is 1 and consumes (swallows) the value when the test is 0. The latter case is expressed by setting the false output of the switch device to %ign. Simply connecting the channel directly to the true path may not be correct because in the case where the false path is actually taken, the value of "a" will be left in the graph, resulting in an incorrect value of a for the next execution of the function. This example highlights the property of control equivalence, which is a key property in embodiments of correct data flow transformations.

[0272] Control Equivalence : Consider a single-entry single-exit control flow graph G with two basic blocks A and B. A and B are control equivalent if all complete control flow paths through G visit A and B the same number of times.

[0273] LIC Replacement : In control flow graph G, assume that an operation in basic block A defines virtual register x and an operation in basic block B uses x. Then a correct control-data flow transformation is available to replace x with a latency-insensitive channel only if A and B are control equivalent. The control equivalence relation partitions the basic blocks of the CFG into strongly control-dependent regions. Figure 33A Illustrates C source code 3302 according to an embodiment of the present disclosure. Figure 33B Illustrates for Figure 33A the data flow assembly code 3304 of C source code 3302. Figure Illustrates for ​ the data flow graph 3306 of data flow assembly code 3304. In ​ the example, the basic blocks before and after the conditional statement are control equivalent to each other, but the basic blocks in the true path and the false path are each in their control-dependent regions. A correct algorithm for converting the CFG to a data flow is to enable the compiler to: (1) insert switch devices to compensate for mismatches in execution frequency for any values flowing between non-control-equivalent basic blocks; and (2) insert pick devices at the start of basic blocks to correctly select any incoming values into the basic blocks. Generating appropriate control signals for these pick devices and switch devices can be a key part of the data flow transformation. [[ID=!21]]

[0274] Loop

[0275] Another important CFG category in data flow transformation is the CFG for single-entry single-exit loops, which is a common form of loops generated in (LLVM) IR. These loops can be almost acyclic except for a single back edge from the end of the loop back to the loop header block. The data flow transformation pass can use the same high-level strategy to transform loops as it does for branches. For example, the data flow transformation pass inserts switching devices at the end of the loop to direct values out of the loop (either leaving the loop exit or going around the back edge to the start of the loop), and inserts picking devices at the start of the loop to select between the initial value entering the loop and the value coming in through the back edge. ​ Illustrates C source code 3402 according to an embodiment of the present disclosure. ​ Illustrates for ​ the data flow assembly code 3402 of C source code 3404 according to an embodiment of the present disclosure. ​ Illustrates for ​ the data flow graph 3400 of the data flow assembly code 3404 according to an embodiment of the present disclosure. ​ - ​ Shows C and CSA assembly code and the corresponding data flow graph for an example do-while loop that adds up the values of loop induction variable i. For each variable that conceptually loops around the loop (i and sum), the graph has a corresponding pair of pick / switch devices that control the flow of these values. Note that even though n is a loop invariant, this example uses a pair of pick / switch devices to loop the value of n around the loop. This duplication of n enables the virtual register for n to be transformed into the LIC because it matches the execution frequency between the conceptual definition of n outside the loop and one or more uses of n inside the loop. In general, to obtain correct data flow transformation, when a register is transformed into the LIC, for each iteration inside the loop body, the register that is live-in to the loop will be duplicated once. Similarly, a register that is updated inside the loop and live-out of the loop will be consumed (e.g., with a single final value sent out of the loop). Loops introduce wrinkles into the data flow transformation process, i.e., the control for the pick at the top of the loop and the switch at the bottom of the loop are offset. For example, if ​ the loop in

[0276] ​ Figure 3500 is a flowchart according to an embodiment of the present disclosure. The depicted process 3500 includes: 3502: decoding an instruction into a decoded instruction using a decoder of a core of a processor; 3504: executing the decoded instruction using an execution unit of the core of the processor to perform a first operation; 3506: receiving an input of a data flow graph including a plurality of nodes; 3508: overlaying the data flow graph onto a plurality of processing elements of the processor and an interconnection network between the plurality of processing elements, and each node is represented as a data flow operator in the plurality of processing elements; and 3510: when a corresponding set of incoming operands arrives at each of the data flow operators of the plurality of processing elements, performing a second operation of the data flow graph using the interconnection network and the plurality of processing elements.

[0277] ​ Figure 3501 is a flowchart according to an embodiment of the present disclosure. The depicted process includes: 3503: receiving an input of a data flow graph including a plurality of nodes; 3505: overlaying the data flow graph onto a plurality of processing elements of the processor, a data path network between the plurality of processing elements, and a flow control path network between the plurality of processing elements, and each node is represented as a data flow operator in the plurality of processing elements.

[0278] In one embodiment, the core writes a command to a memory queue, and the CSA (e.g., a plurality of processing elements) monitors the memory queue and starts execution when the command is read. In one embodiment, the core executes a first part of a program, and the CSA (e.g., a plurality of processing elements) executes a second part of the program. In one embodiment, while the CSA is performing an operation, the core performs other work.

[0279] ​

[0280] In certain embodiments, the CSA architecture and microarchitecture provide profound energy, performance, and availability advantages over roadmap processor architectures and FPGAs. In this section, these architectures are compared with embodiments of the CSA, and the superiority of the CSA in accelerating parallel data flow graphs over each of them is emphasized.

[0281] ​

[0282] ​ Figure 3600 is a graph showing the throughput according to an embodiment of the present disclosure versus the energy per operation. As ​As shown, small cores are generally more energy-efficient than large cores, and in some workloads, this advantage can be translated into absolute performance by a higher core count. The CSA microarchitecture follows these observations to their conclusion and removes (e.g., most of) the energy-hungry control structures associated with the von Neumann architecture (including most in the instruction-side microarchitecture). By removing these overheads and implementing a simple single-operation PE, embodiments of the CSA achieve a dense, efficient spatial array. Unlike small cores which are typically very serialized, the CSA can, for example, aggregate its PEs via a circuit-switched local network to form an explicitly parallel aggregated data flow graph. The result is performance not only in parallel applications but also in serial applications. Unlike cores that are costly in terms of area and energy, the CSA is already parallel in its native execution model. In some embodiments, the CSA neither requires speculation to boost performance nor needs to repeatedly re-extract parallelism from a serialized program representation, thus avoiding two of the major energy taxes in the von Neumann architecture. Most of the structures in embodiments of the CSA are distributed, small, and energy-efficient, in contrast to the centralized, large, energy-hungry structures found in cores. Consider the case of registers in the CSA: each PE can have some (e.g., 10 or fewer) storage registers. Individually, these registers can be more efficient than traditional register files. When aggregated, these registers can provide the effect of a register file in a large structure. As a result, embodiments of the CSA avoid most of the stack overflows and fills caused by classical architectures while using much less energy for each state access. Of course, applications can still access memory. In embodiments of the CSA, memory access requests and responses are architecturally decoupled, enabling the workload to maintain many more outstanding memory accesses per unit of area and energy. This property achieves significantly higher performance for cache-constrained workloads and reduces the area and energy required to saturate the main memory in memory-constrained workloads. Embodiments of the CSA expose new forms of energy efficiency unique to non-von Neumann architectures. One result of performing a single operation (e.g., instruction) at (e.g., most of) the PEs is reduced operand entropy. In the case of incremental operations, each execution results in a few circuit-level switches and very little energy consumption, a case examined in detail in Section 6.2. In contrast, von Neumanns are multiplexed, resulting in a large number of bit transitions. The asynchronous style of embodiments of the CSA also enables microarchitecture optimizations, such as the floating-point optimizations described in Section 3.5 that are difficult to implement in a strictly-scheduled core pipeline. Since the PEs can be relatively simple and the behavior of the PEs in a particular data flow graph can be statically known, clock gating and power gating techniques can be employed more efficiently than in coarser architectures.The figure execution style, small size, and extensibility of embodiments of CSA, PE, and the network together enable the expression of many kinds of parallelism: instruction, data, pipeline, vector, memory, thread, and task parallelism can all be implemented. For example, in an embodiment of CSA, one application can use arithmetic units to provide a high level of address bandwidth, while another application can use those same units for computation. In many cases, multiple parallelisms can be combined to achieve even higher performance. Many key HPC operations can be both replicated and pipelined, resulting in performance gains of multiple orders of magnitude. In contrast, von Neumann cores are typically optimized for one style of parallelism carefully chosen by the architect, resulting in the inability to capture all important application kernels. Because embodiments of CSA expose and facilitate many forms of parallelism, it does not mandate a particular form of parallelism, or worse, that a particular subroutine exist in the application to benefit from CSA. For example, many applications (including single-stream applications) can obtain both performance and energy benefits from embodiments of CSA even when compiled without modification. This is contrary to the long-term trend of requiring a great deal of programmer effort to obtain significant performance gains in single-stream applications. In fact, in some applications, embodiments of CSA obtain more performance from functionally equivalent but less "modern" code than from its complex contemporary counterpart that has been tortured to target vector instructions.

[0283] ​

[0284] The choice of the data flow operator as the basic architecture for an embodiment of the CSA differentiates those CSAs from FPGAs. Specifically, the CSA is a superior accelerator for HPC data flow graphs generated from traditional programming languages. The data flow operator is fundamentally asynchronous. This enables embodiments of the CSA to not only have freedom of implementation in the microarchitecture but also to adapt simply and concisely to abstract architectural concepts. For example, embodiments of the CSA naturally adapt to many memory microarchitectures with a simple load-store interface, which are largely asynchronous. One only needs to examine the FPGA DRAM controller to appreciate the difference in replication. Embodiments of the CSA also utilize asynchrony to provide faster and more full-featured runtime services such as configuration and extraction, which are believed to be 4-6 orders of magnitude faster than FPGA blocks. By narrowing the architectural interface, embodiments of the CSA provide control over most timing paths at the microarchitecture level. This allows embodiments of the CSA to operate at much higher frequencies than the more general control mechanisms available in FPGAs. Similarly, clock and reset, which may be fundamental to the architecture of FPGAs, are microarchitectural in the CSA, thus eliminating, for example, the need to support clock and reset as programmable entities. The data flow operator can be coarse-grained for most parts. By processing only in coarse operators, embodiments of the CSA improve both the density of the structure and its energy consumption. The CSA executes operations directly rather than emulating operations using lookup tables. A second result of coarseness is the simplification of placement and routing problems. The CSA data flow graph is many orders of magnitude smaller than the FPGA netlist, and in embodiments of the CSA, the placement and routing times are correspondingly reduced. The significant differences between embodiments of the CSA and FPGAs make the CSA superior as an accelerator for data flows generated from traditional programming languages, for example.

[0285] ​

[0286] CSA is a novel computer architecture that offers significant performance and energy advantages over roadmap processors. Consider the case of computing a single-stride address for a cross-array walk. This case can be important in HPC applications (e.g., which spend a large amount of integer work in computing address offsets). In address calculations, especially stride address calculations, for each calculation, one argument is constant and the other argument only varies slightly. Thus, in most cases, only a few bits switch per cycle. In fact, using a derivation similar to the constraints on floating-point carry bits described in Section 3.5, it can be shown that for stride calculations, on average, less than two input bits switch per calculation, resulting in a 50% reduction in energy for a random switching distribution. If a time-multiplexing approach is used, many of these energy savings are lost. In one embodiment, CSA achieves approximately 3x (3 times) energy efficiency relative to the core while achieving an 8x (8 times) performance gain. The parallelism gain obtained from embodiments of CSA leads to reduced program run times, thereby achieving a corresponding significant reduction in leakage energy. At the PE level, embodiments of CSA are extremely energy-efficient. A second important issue for CSA is whether it consumes a reasonable amount of energy at the chip level. Since embodiments of CSA can exercise every floating-point PE in the structure every cycle, it serves as a reasonable upper bound on energy and power consumption, e.g., such that most of the energy goes into floating-point multiplication and addition.

[0287] ​

[0288] This section discusses further details of configuration and exception handling.

[0289] ​

[0290] This section discloses examples of how to configure CSA (e.g., the structure), how to implement the configuration quickly, and how to minimize the resource overhead of the configuration. Quickly configuring the structure is extremely important for accelerating small parts of larger algorithms and thus for relaxing the applicability of CSA. This section further discusses features that allow embodiments of CSA to be programmed with configurations of different lengths.

[0291] Embodiments of a CSA (e.g., a fabric) can be different from traditional cores in that embodiments of a CSA can utilize a configuration step where a (e.g., large) portion of the fabric is loaded in program configuration ahead of program execution. The advantage of static configuration can be that very little energy is spent at runtime during configuration, which is in contrast to, for example, a serialized core that spends energy fetching configuration information (instructions) at almost every cycle. A previous disadvantage of configuration was that it was a coarse-grained step with potentially long latency, and the potentially long latency sets a lower bound on the size of programs that can be accelerated in the fabric due to the cost of context switching. This disclosure describes a scalable microarchitecture for rapidly configuring a spatial array in a distributed manner (which, for example, avoids the previous disadvantages).

[0292] As discussed above, a CSA can include lightweight processing elements connected by an inter-PE network. By configuring configurable fabric elements (CFEs) (e.g., PEs and the interconnect (fabric) network), a program, which is regarded as a control-data flow graph, is then mapped onto the architecture. Generally, a PE can be configured as a data flow operator, and once all input operands arrive at the PE, some operation occurs and the result is forwarded to one or more other PEs for consumption or output. PEs can communicate via dedicated virtual circuits that are formed by statically configuring a circuit-switched communication network. These virtual circuits can be flow-controlled and fully backpressureable, such that, for example, if the source has no data or the destination is full, the PE will stop. At runtime, data can flow through the PEs implementing the mapped algorithm. For example, data can flow into the fabric from memory and then back out to memory. Such a spatial architecture can achieve excellent performance efficiency compared to traditional multi-core processors: in contrast to scaling the memory system, computing in the form of PEs can be simpler and more numerous than larger cores, and communication can be direct.

[0293] Embodiments of a CSA can not utilize (e.g., software-controlled) packet switching (e.g., packet switching that requires a large amount of software assistance to implement), which slows down configuration. Embodiments of a CSA include out-of-band signaling in the network (e.g., out-of-band signaling of only 2 - 3 bits depending on the supported feature set) and a fixed configuration topology to avoid the need for a large amount of software support.

[0294] One key difference between embodiments of a CSA and the way used in an FPGA is that the CSA approach can use wide data words, is distributed, and includes a mechanism for directly fetching program data from memory. Embodiments of a CSA can not utilize JTAG-type single-bit communication for area efficiency, for example because that would require several milliseconds to fully configure a large FPGA fabric.

[0295] Embodiments of CSA include a distributed configuration protocol and a microarchitecture to support this protocol. Initially, the configuration state may reside in memory. Multiple (e.g., distributed) local configuration controllers (boxes) (LCCs) may, for example, use a combination of a small set of control signals and the network provided by the fabric to stream multiple parts of the entire program to their local regions in the spatial fabric. State elements may be used at each CFE to form a configuration chain, thus allowing, for example, individual CFEs to self-program without global addressing.

[0296] Embodiments of CSA include specific hardware support for forming configuration chains, e.g., not software that dynamically establishes these chains at the cost of increased configuration time. Embodiments of CSA are not purely packet-switching type and do include additional out-of-band control lines (e.g., control is not sent through the data path, thus requiring additional cycles to strobe this information and re-serialize the information). Embodiments of CSA reduce configuration latency (e.g., by at least half) by fixing the configuration ordering and by providing explicit out-of-band control, while not significantly increasing network complexity.

[0297] Embodiments of CSA do not use serial configuration for configuration where data is streamed bit by bit into the fabric using a JTAG-like protocol. Embodiments of CSA utilize a coarse-grained fabric approach. In some embodiments, adding some control lines or state elements to a 64-bit or 32-bit oriented CSA fabric has a lower cost than adding those same control mechanisms to a 4-bit or 6-bit fabric.

[0298] ​ FIG. illustrates an accelerator chip 3700 according to an embodiment of the present disclosure. The accelerator chip 3700 includes an array of processing elements (PEs) and local configuration controllers 3702, 3706. Each PE, each network controller (e.g., network data flow endpoint circuitry), and each switching device may be a configurable fabric element (CFE), e.g., the CFE is configured (e.g., programmed) by an embodiment of the CSA architecture.

[0299] Embodiments of CSA include hardware that provides efficient, distributed, low-latency configuration for a heterogeneous spatial fabric. This can be achieved according to four techniques. First, utilize hardware entities (local configuration controllers (LCCs)), e.g., as ​As shown in []. The LCC can fetch a stream of configuration information from (e.g., virtual) memory. Second, a configuration data path can be included, e.g., as wide as the native width of the PE structure and can be overlaid on top of the PE structure. Third, new control signals can be received into the PE structure that arranges the configuration process. Fourth, state elements can be located (e.g., in registers) at each configurable endpoint that tracks the state of adjacent CFEs, allowing each CFE to self-configure explicitly without additional control signals. These four microarchitecture features can allow the CSA to configure a chain of its CFEs. To obtain a low configuration latency, the configuration can be partitioned by establishing many LCC and CFE chains. At configuration time, these can operate independently to load the structure in parallel, thus dynamically reducing the latency. As a result of these combinations, a structure configured using an embodiment of the CSA architecture can be fully configured (e.g., within a few hundred nanoseconds). Below, the detailed operation of various components of an embodiment of the CSA configuration network is disclosed.

[0300] ​ FIG. illustrates a local configuration controller 3802 of a configuration data path network according to an embodiment of the present disclosure. The depicted network includes a plurality of multiplexers (e.g., multiplexers 3806, 3808, 3810), which can be configured (e.g., via their respective control signals) to connect one or more data paths (e.g., from a PE) together. ​ FIG. illustrates a network 3800 (e.g., a structure) configured for some previous operation or program (e.g., setup). ​ FIG. illustrates a local configuration controller 3802 that gates configuration signals (e.g., including network interface circuitry 3804 for transmitting and / or receiving signals), and the local network is set to allow the LCC to send configuration data to the default configuration of all configurable structure elements (CFEs) (e.g., muxes) as depicted in the figure. ​ FIG. illustrates an LCC that gates configuration information across the network to configure the CFEs in a predetermined (e.g., silicon-defined) sequence. In one embodiment, when the CFEs are configured, they can immediately start operating. In another embodiment, the CFEs wait to start operating until the structure has been fully configured (e.g., signaled by a configuration terminator (e.g., ​ configuration terminator 4004 and configuration terminator 4008 in []). In one embodiment, the LCC obtains control of the network structure by sending a special message or driving a signal. Subsequently, it gates the configuration data into the CFEs in the structure (e.g., over a period of many cycles). In these figures, the multiplexer network is similar to the "switching device" shown in some figures (e.g., ​ ).

[0301] Local Configuration Controller

[0302] ​ Illustrated is a (e.g., local) configuration controller 3902 according to an embodiment of the present disclosure. The Local Configuration Controller (LCC) can be a hardware entity responsible for: loading the local part of the fabric program (e.g., in a subset of the die or elsewhere); interpreting these program parts; and subsequently loading these program parts into the fabric by driving appropriate protocols on various configuration lines. In this capacity, the LCC can be a dedicated serialization microcontroller.

[0303] When the LCC operation receives a pointer to a code segment, it can start. Depending on the LCB microarchitecture, this pointer (e.g., stored in pointer register 3906) either comes to the LCC via a network (e.g., within the CSA (fabric) itself) or via a memory system access. When the LCC receives such a pointer, it can optionally drain the relevant state from its part of the fabric for context storage and then proceed to immediately reconfigure the part of the fabric that the LCC is responsible for. The program loaded by the LCC can be a combination of configuration data for the fabric and control commands for the LCC, e.g., the configuration data and the control commands are lightly encoded. When the LCC streams in a program part, it can interpret the program as a command stream and perform appropriate encoded actions to configure (e.g., load) the fabric.

[0304] In ​ are shown two different microarchitectures for the LCC, e.g., one or both of which are used in the CSA. The first microarchitecture places the LCC 3702 at the memory interface. In this case, the LCC can make a direct request to the memory system to load data. In the second case, the LCC3706 is placed on the memory network, in which the LCC3906 can only indirectly make a request to the memory. In both cases, the logical operation of the LCB does not change. In one embodiment, the LCC is notified of the program to be loaded, e.g., by a set of control status registers (e.g., visible to the OS), which will be used to notify individual LCCs of new program pointers, etc.

[0305] Additional out-of-band control channels (e.g., lines)

[0306] In some embodiments, the configuration relies on 2 - 8 additional out - of - band control channels to improve the configuration speed, as defined below. For example, the configuration controller 3902 may include the following control channels: for example, the CFG_START control channel 3908, the CFG_START_PRIVILEDGE control channel 3909, the CFG_VALID control channel 3910, and the CFG_DONE control channel 3912, examples of each are discussed in Table 3 below

[0307] Table 3: Control Channels

[0308]

[0309]

[0310] Generally, the handling of configuration information can be left to the implementer of a particular CFE. For example, an optional - function CFE may have provisions to use an existing data path to set registers, while a fixed - function CFE may simply set the configuration registers.

[0311] Due to the long wire delay when programming a large set of CFEs, the CFG_VALID signal can be considered as a clock / latch enable for CFE components. Since this signal is used as a clock, in one embodiment, the duty cycle of this wire is at most 50%. As a result, the configuration throughput is approximately halved. Optionally, a second CFG_VALID signal can be added to allow continuous programming.

[0312] In one embodiment, only CFG_START is strictly passed on an independent coupling device (e.g., a wire), for example, CFG_VALID and CFG_DONE can be overlaid on top of other network - coupled devices.

[0313] Reuse of Network Resources

[0314] To reduce the configuration overhead, some embodiments of the CSA utilize the existing network infrastructure to transfer configuration data. The LCC can utilize both the chip - level memory hierarchy and the fabric - level communication network to move data from storage to the fabric. As a result, in some embodiments of the CSA, the configuration infrastructure adds no more than 2% to the total fabric area and power.

[0315] The reuse of network resources in some embodiments of the CSA can give the network some hardware support for the configuration mechanism. The circuit-switched network of embodiments of the CSA causes the LCC to set the multiplexers of these circuit-switched networks in a specific manner when the 'CFG_START' signal is asserted for configuration. The packet-switched network does not require expansion, but the LCC endpoints (e.g., configuration terminators) use specific addresses in the packet-switched network. Network reuse is optional, and some embodiments may find a dedicated configuration bus more convenient.

[0316] Each CFE state

[0317] Each CFE can maintain a bit indicating whether it has been configured (e.g., see ​ ). When the configuration start signal is driven, this bit can be de-asserted, and subsequently, once a particular CFE has been configured, this bit is asserted. In a configuration protocol, the CFEs are arranged in a chain, and the CFE and the configuration status bits determine the topology of the chain. A CFE can read the configuration status bit of the adjacent CFE. If the adjacent CFE is configured and the current CFE is not configured, the CFE and determine any current configuration data pins are for the current CFE. When the 'CFG_DONE' signal is asserted, the CFE can set its configuration bit so that, for example, the upstream CFE can be configured. As a base case of the configuration process, a configured terminator (e.g., the configuration terminator 3704 for LCC3702 or the configuration terminator 3708 for LCC 3706 in ​ ) that asserts its being configured can be included at the end of the chain.

[0318] Inside the CFE, this bit can be used to drive the flow control ready signal. For example, when the configuration bit is de-asserted, the network control signals can be automatically clamped to values that prevent data flow, while inside the PE, no operations or other actions will be scheduled.

[0319] Handling high-latency configuration paths

[0320] An embodiment of the LCC can drive signals over long distances, for example, through many multiplexers and with many loads. Therefore, it may be difficult for the signal to reach the remote CFE within a short clock cycle. In some embodiments, the configuration signal is at a certain division (e.g., fractional) of the main (e.g., CSA) clock signal to ensure the digital timing rules during configuration. The clock division can be used in an out-of-band signaling protocol and does not require any modification to the main clock tree.

[0321] Ensuring consistent fabric behavior during configuration

[0322] Because some configuration schemes are distributed and have non-deterministic timing due to program and memory effects, different parts of the structure may be configured at different times. As a result, some embodiments of the CSA provide mechanisms for preventing inconsistent operation between configured and unconfigured CFEs. In general, consistency is regarded as a property required and maintained by the CFE itself, for example, using internal CFE states. For example, when the CFE is in an unconfigured state, it may claim that its input buffer is full and its output is invalid. When configured, these values will be set to the true state of the buffer. As sufficient parts of the structure come out of configuration, these techniques may allow the structure to start operating. For example, if long-latency memory requests are issued early, this has the effect of further reducing context-switch latency.

[0323] Variable-width configuration

[0324] Different CFEs may have different configuration word widths. For smaller CFE configuration words, the implementer may balance latency by fairly assigning CFE configuration loading across network lines. To balance the loading on the network lines, one option is to assign configuration bits to different parts of the network lines to limit the net latency on any one line. Wide data words may be handled using serialization / deserialization techniques. These decisions may be made on a per-structure basis to optimize the behavior of a particular CSA (e.g., the structure). A network controller (e.g., one or more of network controller 3710 and network controller 3712) may communicate with each domain (e.g., subset) of the CSA (e.g., the structure) to send configuration information to, for example, one or more LCCs. The network controller may be part of a communication network (e.g., separate from a circuit-switched network). The network controller may include network data flow endpoint circuitry.

[0325] ​

[0326] Embodiments of the CSA can be an energy-efficient and high-performance means of accelerating user applications. When considering whether a program (e.g., the data flow graph of a program) can be successfully accelerated by an accelerator, both the time for configuring the accelerator and the time for running the program may be considered. If the runtime is short, the configuration time will play a large role in determining successful acceleration. Therefore, to maximize the domain of accelerable programs, in some embodiments, the configuration time is made as short as possible. One or more configuration caches may be included in the CSA, for example, such that high-bandwidth, low-latency storage enables fast reconfiguration. A description of several embodiments of the configuration cache follows.

[0327] In one embodiment, during configuration, the configuration hardware (e.g., LCC) may optionally access the configuration cache to obtain new configuration information. The configuration cache may operate either as a traditional address-based cache or in an OS-managed mode, in which the configuration is stored in a local address space and addressed by reference to that address space. If the configuration state is in the cache, then in some embodiments, no request to the backing store will be made. In some embodiments, the configuration cache is separate from any (e.g., lower-level) shared cache in the memory hierarchy.

[0328] ​ FIG. illustrates an accelerator slice 4000 according to an embodiment of the present disclosure. The accelerator slice 4000 includes an array of processing elements, a configuration cache (e.g., 4018 or 4020), and a local configuration controller (e.g., 4002 or 4006). In one embodiment, the configuration cache 4014 is co-located with the local configuration controller 4002. In one embodiment, the configuration cache 4018 is in the configuration domain of the local configuration controller 4006. For example, the first domain ends at the configuration terminator 4004, and the second domain ends at the configuration terminator 4008. The configuration cache may allow the local configuration controller to reference the configuration cache during configuration so as to obtain the configuration state, for example, with a lower latency than referencing the memory. The configuration cache (store) may either be dedicated or accessed as a configuration mode of a storage element within the structure (e.g., the local cache 4016).

[0329] Cache Modes

[0330] 1. Demand Caching - In this mode, the configuration cache operates as a true cache. The configuration controller issues an address-based request that is verified against the tags in the cache. Misses may be loaded into the cache and subsequently re-referenced during future reprogramming.

[0331] 2. Storage Element (Scratchpad) Cache within the Structure - In this mode, the configuration cache receives references to configuration sequences in its own small address space rather than the larger address space of the host. This may improve memory density because the portion of the cache used to store tags may instead be used to store configuration.

[0332] In some embodiments, a configuration cache may have configuration data pre-loaded into it (e.g., via an external indication or an internal indication). This can allow for a reduction in the latency for loading programs. Some embodiments herein provide an interface to the configuration cache that permits the loading of a new configuration state into the cache even when a configuration is already running in the fabric. The initiation of this load can occur from an internal or external source. Embodiments of the pre-loading mechanism further reduce latency by removing the cache load latency from the configuration path.

[0333] Prefetch mode

[0334] 1. Explicit prefetch - Augment the configuration path with a new command, ConfigurationCachePrefetch. Different from programming the fabric, this command simply causes the relevant program configuration to be loaded into the configuration cache without programming the fabric. Since this mechanism rides on top of the existing configuration infrastructure, it is exposed both within the fabric and externally to cores and other entities that access the memory space, for example.

[0335] 2. Implicit prefetch - The global configuration controller may maintain a prefetch predictor and use it to initiate an explicit prefetch of the configuration cache (e.g., in an automated manner).

[0336] ​

[0337] Some embodiments of a CSA (e.g., a spatial fabric) include a large number of instructions and configuration states. For example, this configuration state is largely static during the operation of the CSA. Therefore, the configuration state may be vulnerable to soft errors. Quick and error-free recovery from these soft errors may be critical for the long-term reliability and performance of a space system.

[0338] Some embodiments herein provide a fast configuration recovery loop. For example, in this fast configuration recovery loop, configuration errors are detected and multiple parts of the fabric are immediately reconfigured. Some embodiments herein include a configuration controller with, for example, reliability, availability, and serviceability (RAS) reprogramming features. Some embodiments of a CSA include circuitry for high-speed configuration, error reporting, and parity checking within the spatial fabric. Using a combination of these three features and an optional configuration cache, the configuration / exception handling circuitry can recover from soft errors in the configuration. When detected, the soft error can be transmitted to the configuration cache, which initiates an immediate reconfiguration of the fabric (e.g., that part of the fabric). Some embodiments provide a dedicated reconfiguration circuit, for example, which is faster than any solution that would be implemented indirectly in the fabric. In some embodiments, the exception and configuration circuitry located together cooperate to reload the fabric upon detection of a configuration error.

[0339] ​ FIG. illustrates an accelerator slice 4100 according to an embodiment of the present disclosure. The accelerator slice 4100 includes an array of processing elements and a configuration and exception handling controller 4102, 4106 having reconfiguration circuits 4118, 4122. In one embodiment, when a PE detects a configuration error through its RAS feature, it sends a message (e.g., a configuration error or a reconfiguration error) to the configuration and exception handling controller (e.g., 4102 or 4106) through its exception generator. Upon receiving the message, the configuration and exception handling controller (e.g., 4102 or 4106) activates the co-located reconfiguration circuit (e.g., 4118 or 4122 respectively) to reload the configuration state. The configuration microarchitecture proceeds and (e.g., only) reloads the configuration state, and in some embodiments, only reloads the configuration state for the PE reporting the RAS error. After the reconfiguration is complete, the fabric can resume normal operation. To reduce latency, the configuration state used by the configuration and exception handling controller (e.g., 4102 or 4106) can be sourced from a configuration cache. As a base case of the configuration or reconfiguration process, a configuration terminator (e.g., ​ the configuration terminator 4104 for the configuration and exception handling controller 4102 or the configuration terminator 4108 for the configuration and exception handling controller 4106) used to assert that it is configured (or reconfigured) can be included at the end of the chain.

[0340] ​ FIG. illustrates a reconfiguration circuit 4218 according to an embodiment of the present disclosure. The reconfiguration circuit 4218 includes a configuration state register 4220 for storing a configuration state (or a pointer to the configuration state).

[0341] 7.4 ​

[0342] Some parts of an application for a CSA (e.g., a spatial array) may be run infrequently or may be mutually exclusive with other parts of the program. To save area, improve performance, and / or reduce power, it may be useful to time - multiplex multiple parts of a spatial structure among several different parts of a program data - flow graph. Some embodiments herein include an interface through which a CSA (e.g., via a spatial program) may request that that part of the structure be reprogrammed. This may enable the CSA to change itself dynamically according to a dynamic control flow. Some embodiments herein may allow structure - initiated reconfiguration (e.g., reprogramming). Some embodiments herein provide a set of interfaces for triggering configuration from within the structure. In some embodiments, a PE issues a reconfiguration request based on a certain decision in the program data - flow graph. The request may travel through a network to our new configuration interface where the request triggers the reconfiguration. Once the reconfiguration is complete, a message notifying of the completion may optionally be returned. Certain embodiments of the CSA thus provide program (e.g., data - flow graph) - guided reconfiguration capabilities.

[0343] ​ FIG. illustrates an accelerator slice 4300 according to an embodiment of the present disclosure. The accelerator slice 4300 includes an array of processing elements and a configuration and exception handling controller 4306 having a reconfiguration circuit 4318. Here, parts of the structure issue requests for (re)configuration to a configuration domain such as the configuration and exception handling controller 4306 and / or the reconfiguration circuit 4318. The domain (re)configures itself and, when the request has been satisfied, the configuration and exception handling controller 4306 and / or the reconfiguration circuit 4318 issues a response to the structure to notify the structure of the completion of the (re)configuration. In one embodiment, the configuration and exception handling controller 4306 and / or the reconfiguration circuit 4318 disables communication during the (re)configuration, so that during operation, the program has no consistency issues.

[0344] Configuration mode

[0345] Configuration by address - In this mode, the structure makes a direct request to load configuration data from a specific address.

[0346] Configuration by reference - In this mode, the structure makes a request to load a new configuration, for example, by a predefined reference ID. This may simplify the determination of the code to be loaded since the location of the code has been abstracted.

[0347] Configuring multiple domains

[0348] The CSA may include a higher-level configuration controller to support a multicast mechanism to broadcast configuration requests to multiple (e.g., distributed or local) configuration controllers (e.g., via the network indicated by the dashed box). This may enable a single configuration request to be replicated across multiple larger portions of the fabric, thus triggering, for example, a wide reconfiguration.

[0349] ​

[0350] Some embodiments of the CSA may also experience exceptions (e.g., exceptional conditions), such as, for example, floating-point underflow. When these conditions occur, special handlers may be invoked to either correct the program or terminate the program. Some embodiments herein provide a system-level architecture for handling exceptions in a spatial fabric. Since some spatial fabrics emphasize area efficiency, the embodiments herein minimize the total area while providing a general exception mechanism. Some embodiments herein provide a low-area means of signaling exceptional conditions occurring in the CSA (e.g., a spatial array). Some embodiments herein provide interfaces and signaling protocols for passing such exceptions as well as PE-level exception semantics. Some embodiments herein are dedicated exception handling capabilities and, for example, do not require explicit handling by the programmer.

[0351] One embodiment of the CSA exception architecture consists of four parts, as shown, for example, in ​ These parts may be arranged in a hierarchy in which exceptions flow out from the producer and ultimately flow up to a die-level exception aggregator (e.g., a handler) that may meet with an exception maintainer of, for example, a core. The four parts may be:

[0352] 1. PE exception generator

[0353] 2. Local exception network

[0354] 3. Mezzanine exception aggregator

[0355] 4. Die-level exception aggregator

[0356] ​ An accelerator die 4400 according to an embodiment of the present disclosure is illustrated, the accelerator die 4400 including an array of processing elements and a mezzanine exception aggregator 4402 coupled to a die-level exception aggregator 4404. ​ A processing element 4500 having an exception generator 4544 according to an embodiment of the present disclosure is shown.

[0357] PE exception generator

[0358] The processing element 4500 may include from ​processing element 2400, for example, similar numbers are similar components, for example, local network 2402 and local network 4502. An additional network 4513 (e.g., a channel) can be an exception network. The PE can implement an interface to the exception network (e.g., ​ the exception network 4513 (e.g., a channel) on ​ The microarchitecture illustrates such an interface, where the PE has an exception generator 4544 (e.g., initiating an exception finite state machine (FSM) 4540 to externally gate an exception packet (e.g., BOXID 4542) onto the exception network). The BOXID 4542 can be a unique identifier for an exception generation entity (e.g., a PE or a block) within the local exception network. When an exception is detected, the exception generator 4544 senses the exception network and gates out the BOXID when the network is found to be idle. Exceptions can be caused by many conditions, such as but not limited to arithmetic errors, failed ECC checks on the state, etc. However, it may also be the case that, using the idea of supporting constructs like breakpoints, an exception data stream operation is introduced.

[0359] The initiation of an exception can occur either explicitly through an instruction provided by the programmer or implicitly when a strengthened error condition (e.g., floating - point underflow) is detected. When an exception occurs, the PE 4500 can enter a waiting state, in which the PE 4500 waits to be serviced by a final exception handler, for example, external to the PE 4500. The content of the exception packet depends on how a particular PE is implemented, as described below.

[0360] Local exception network

[0361] (e.g., local) exception network guides exception packets from the PE 4500 to the mezzanine exception network. The exception network (e.g., 4513) can be a serial packet - switched network, which consists of, for example, (e.g., a single control line) and one or more data lines organized in a ring or tree topology, for example, for a subset of PEs. Each PE can have a (e.g., ring) station in the (e.g., local) exception network, where, for example, the PE can arbitrate to inject a message into the exception network.

[0362] The PE endpoints that need to inject exception packets can observe their local exception network exit points. If the control signal indicates busy, the PE will wait to start injecting the PE's packet. If the network is not busy, i.e., the downstream station has no packet to forward, the PE will proceed to start injecting.

[0363] Network packets can have variable or fixed lengths. Each packet can start with a fixed-length header field that identifies the source PE of the packet. This header field can then be followed by a variable number of PE-specific fields that contain information such as error codes, data values, or other useful status information.

[0364] Mezzanine exception aggregator

[0365] The mezzanine exception aggregator 4404 is responsible for assembling local exception networks into larger packets and sending these larger packets to the chip-level exception aggregator 4402. The mezzanine exception aggregator 4404 can anticipate the local exception packet using the unique ID of the local exception packet itself, for example, to ensure that the exception message is clear. The mezzanine exception aggregator 4404 can be docked to a special exception-only virtual channel in the mezzanine network, for example, to ensure that exceptions are deadlock-free.

[0366] The mezzanine exception aggregator 4404 may also be able to directly maintain certain categories of exceptions. For example, using a cache local to the mezzanine network station, configuration requests from the fabric can be distributed out of the mezzanine network.

[0367] Chip-level exception aggregator

[0368] The final stage of the exception system is the chip-level exception aggregator 4402. The chip-level exception aggregator 4402 is responsible for collecting exceptions from various mezzanine-level exception aggregators (e.g., 4404) and forwarding these exceptions to the appropriate maintenance hardware (e.g., cores). Thus, the chip-level exception aggregator 4402 can include some internal tables and controllers for associating specific messages with handler routines. These tables can be directly indexed or can be indexed with a small state machine to direct specific exceptions.

[0369] Like the mezzanine exception aggregator, the chip-level exception aggregator can maintain some exception requests. For example, it can initiate reprogramming of most of the PE fabric in response to a specific exception.

[0370] ​

[0371] Certain embodiments of the CSA include (a plurality of) extraction controllers for extracting data from the fabric. Embodiments for how to quickly implement this extraction and how to minimize the resource overhead of data extraction are discussed below. Data extraction can be used for critical tasks such as exception handling and context switching. Certain embodiments herein extract data from a heterogeneous spatial fabric by introducing features that allow extractable fabric elements (EFEs) (e.g., PEs, network controllers, and / or switching devices) with variable and dynamically variable numbers of states to be extracted.

[0372] Embodiments of the CSA include a distributed data extraction protocol and a microarchitecture to support this protocol. Some embodiments of the CSA include multiple local extraction controllers (LECs) that use a combination of a (e.g., small) set of control signals and the network provided by the structure to flow program data out of their local regions in the spatial structure. State elements may be used at each extractable structure element (EFE) to form an extraction chain, thus allowing, for example, individual EFEs to extract themselves without global addressing.

[0373] Embodiments of the CSA do not use a local network to extract program data. Embodiments of the CSA include, for example, specific hardware support (e.g., extraction controllers) for forming extraction chains and do not rely on software (e.g., at the cost of increased extraction time) to dynamically establish these chains. Embodiments of the CSA are not purely packet-switching type and do include additional out-of-band control lines (e.g., control is not sent through the data path, thus requiring additional cycles to strobe this information and re-serialize the information). Embodiments of the CSA reduce the extraction latency (e.g., by at least half) by fixing the extraction order and by providing explicit out-of-band control while not significantly increasing network complexity.

[0374] Embodiments of the CSA do not use a serial mechanism for data extraction where data is streamed bit-by-bit from the structure using a JTAG-like protocol. Embodiments of the CSA utilize a coarse-grained structural approach. In some embodiments, adding some control lines or state elements to a 64-bit or 32-bit oriented CSA structure has a lower cost than adding those same control mechanisms to a 4-bit or 6-bit structure.

[0375] ​ FIG. shows an accelerator chip 4600 according to an embodiment of the present disclosure. The accelerator chip 4600 includes an array of processing elements and local extraction controllers 4602, 4606. Each PE, each network controller, and each switching device may be an extractable structure element (EFE), e.g., the EFE is configured (e.g., programmed) by an embodiment of the CSA architecture.

[0376] Embodiments of the CSA include hardware that provides efficient, distributed, low-latency extraction of heterogeneous spatial structures. This can be achieved according to four techniques. First, utilize hardware entities (local extraction controllers (LECs)), e.g., as ​As shown. The LEC can accept commands from a host (e.g., a processor core), such as extracting a data stream from a spatial array and writing the data back to a virtual memory for inspection by the host. Second, an extraction data path can be included. For example, the extraction data path is as wide as the native width of the PE structure and can be overlaid on top of the PE structure. Third, new control signals can be received in the PE structure that arranges the extraction process. Fourth, state elements can be located (e.g., in registers) at each configurable endpoint that tracks the state of adjacent EFEs, allowing each EFE to explicitly output its state without additional control signals. These four microarchitecture features can allow the CSA to extract data from a chain of EFEs. To obtain low data extraction latency, some embodiments can partition the extraction problem by including multiple (e.g., many) LECs and EFE chains in the structure. During extraction, these chains can operate independently to extract data from the structure in parallel, thereby dynamically reducing latency, for example. As a result of these combinations, the CSA can perform a complete state dump (e.g., within a few hundred nanoseconds).

[0377] ​ Illustrates a local extraction controller 4702 that configures a data path network according to an embodiment of the present disclosure. The depicted network includes a plurality of multiplexers (e.g., multiplexers 4706, 4708, 4710) that can be configured (e.g., via their respective control signals) to connect one or more data paths (e.g., from PEs) together. ​ Illustrates a network 4700 (e.g., a structure) for some previous operating program configurations (e.g., settings). ​ Illustrates a local extraction controller 4702 that gates an extraction signal (e.g., including network interface circuitry 4704 for transmitting and / or receiving signals), and all PEs controlled by the LEC enter an extraction mode. The last PE in the extraction chain (or extraction terminator) can master the extraction channel (e.g., a bus) and send data according to (1) a signal from the LEC or (2) a signal generated internally (e.g., from the PE). Once completed, the PE can set its completion flag, thereby enabling the next PE to extract its data, for example. ​ Illustrates that the farthest PE has completed the extraction process, and as a result, the farthest PE has set one or more of its extraction status bits. For example, the one or more extraction status bits cause the mux to swing to an adjacent network to enable the next PE to start the extraction process. The extracted PE can resume normal operation. In some embodiments, the PE can remain disabled until other actions are taken. In these figures, the multiplexer network is similar to the "switching device" shown in certain figures (e.g., ​ ).

[0378] The next section describes the operation of the various components of the extraction network embodiments

[0379] Local Extraction Controller

[0380] ​ Illustrates an extraction controller 4802 according to an embodiment of the present disclosure. The Local Extraction Controller (LEC) can be a hardware entity responsible for accepting extraction commands, coordinating the extraction process of the EFE, and / or storing the extracted data into, for example, virtual memory. In this capacity, the LEC can be a dedicated serial microcontroller.

[0381] When the LEC receives a pointer to a buffer (e.g., in virtual memory) where the structure state will be written and optionally receives a command indicating how much of the control structure will be extracted, the operation of the LEC can begin. Depending on the LEC microarchitecture, this pointer (e.g., stored in pointer register 4804) can come to the LEC either via the network or via the memory system access. When the LEC receives such a pointer (e.g., command), it continues to extract the state from the part of the structure for which it is responsible. The LEC can stream the extracted data flow out of the structure and into a buffer provided by an external caller.

[0382] In ​ illustrates two different microarchitectures for the LEC. The first places the LEC 4602 at the memory interface. In this case, the LEC can make a direct request to the memory system to write the extracted data. In the second case, the LEC 4606 is placed on the memory network, where the LCC 4606 can only indirectly make requests to the memory. In both cases, the logical operation of the LEC can remain unchanged. In one embodiment, the expectation to extract data from the structure is notified to the LEC, for example, by a set of control status registers (visible to the OS, e.g.), which will be used to notify individual LECs of new commands.

[0383] Additional out-of-band control channels (e.g., wires)

[0384] In certain embodiments, the extraction relies on 2 - 8 additional out-of-band signals to improve the configuration speed, as defined below. Signals driven by the LEC can be labeled as LEC. Signals driven by the EFE (e.g., PE) can be labeled as EFE. The configuration controller 4802 can include the following control channels, e.g., the LEC_EXTRACT control channel 4806, the LEC_START control channel 4808, the LEC_STROBE control channel 4810, and the EFE_COMPLETE control channel 4812, examples of each being discussed in Table 4 below.

[0385] Table 4: Extraction Channels

[0386]

[0387] Generally, the disposition of the extraction can be left to the implementer of a particular EFE. For example, a selectable function EFE may have provisions to use an existing data path to dump registers, while a fixed function EFE may simply have a multiplexer.

[0388] Due to the long wire delays when programming a large collection of EFEs, the LEC_STROBE signal can be considered as the clock / latch enable for the EFE components. Since this signal is used as a clock, in one embodiment, the duty cycle of this wire is at most 50%. As a result, the extraction throughput is approximately halved. Optionally, a second LEC_STROBE signal can be added to achieve continuous extraction.

[0389] In one embodiment, only LEC_START is strictly passed on an independent coupling device (e.g., a wire), e.g., other control channels can be overlaid on an existing network (e.g., a wire).

[0390] Reuse of network resources

[0391] To reduce the overhead of data extraction, some embodiments of the CSA utilize the existing network infrastructure to transfer the extracted data. The LEC can utilize both the chip-level memory hierarchy and the fabric-level communication network to move data from the fabric to storage. As a result, in some embodiments of the CSA, the extraction infrastructure adds no more than 2% to the total fabric area and power.

[0392] The reuse of network resources in some embodiments of the CSA can enable the network to have some hardware support for the extraction protocol. Circuit-switched networks require some embodiments of the CSA to configure the multiplexers of these circuit-switched networks in a particular way when the "LEC_START" signal is asserted. Packet-switched networks do not require an extension, but the LEC endpoints (e.g., extraction terminators) use specific addresses in the packet-switched network. Network reuse is optional, and some embodiments may find a dedicated configuration bus to be more convenient.

[0393] Each EFE state

[0394] Each EFE maintains a bit indicating whether it has output its status. This bit can be de-asserted when the extraction start signal is driven, and subsequently asserted once a particular EFE has completed extraction. In an extraction protocol, EFEs are arranged to form a chain, and the EFE extraction status bits determine the topology of the chain. An EFE can read the extraction status bit of the adjacent EFE. If the adjacent EFE has its extraction bit set and the current EFE does not have its extraction bit set, the EFE can determine that it owns the extraction bus. When an EFE dumps its last data value, it can drive the "EFE_DONE" signal and set its extraction bit, enabling, for example, an upstream EFE to configure for extraction. The network adjacent to the EFE can observe this signal and also adjust its state to handle the transition. As a base case of the extraction process, an extraction terminator that asserts extraction complete (e.g., the extraction terminator 4604 for LEC 4602 or the extraction terminator 4608 for LEC 4606 in ​ can be included at the end of the chain.

[0395] Internally in the EFE, this bit can be used to drive a flow control ready signal. For example, when the extraction bit is de-asserted, network control signals can be automatically clamped to a value that prevents data flow, while within the PE, no operations or actions will be scheduled.

[0396] Handling high-latency paths

[0397] An embodiment of the LEC can drive signals over long distances, for example, through many multiplexers and with many loads. Thus, it may be difficult for signals to reach a distal EFE within a short clock cycle. In some embodiments, the extraction signal is at a certain division (e.g., fractional) of the master (e.g., CSA) clock signal to ensure digital timing rules during extraction. The clock division can be used in an out-of-band signaling protocol and does not require any modification to the master clock tree.

[0398] Ensuring consistent structural behavior during extraction

[0399] Since some extraction schemes are distributed and have non-deterministic timing due to program and memory effects, different members of the structure can be in the extraction state at different times. When LEC_EXTRACT is driven, all network flow control signals can be driven to logic low, thereby freezing the operation of a particular segment of the structure, for example.

[0400] The extraction process can be non-destructive. Thus, once extraction has been completed, the set of PEs can be considered to be running. An extension to the extraction protocol can allow the PEs to be optionally disabled after extraction. Alternatively, in an embodiment, starting configuration during the extraction process will have a similar effect.

[0401] Single PE extraction

[0402] In some cases, extracting a single PE may be expedient. In such a case, as part of the beginning of the extraction process, an optional address signal may be driven. This may enable the PE to be directly enabled for extraction. Once the PE has been extracted, the extraction process terminates with the lowering of the LEC_EXTRACT signal. In this way, a single PE may be selectively extracted, for example, by a local extraction controller.

[0403] Handling extraction backpressure

[0404] In embodiments where the LEC writes data to be extracted to memory (e.g., for post-processing, e.g., in software), it may be subject to limited memory bandwidth. In the case where the LEC has exhausted its buffer capacity or anticipates that it will exhaust its buffer capacity, the LEC may stop strobing the LEC_STROBE until the buffer issue has been resolved.

[0405] Note that in some of the figures (e.g., ​ , 40 , 41, 43, 44, and 46), communications are schematically illustrated. In some embodiments, those communications may occur via a network (e.g., an interconnect).

[0406] ​

[0407] ​ FIG. 4900 is a flowchart diagram illustrating an embodiment in accordance with the present disclosure. The depicted flow 4900 includes: 4902: decoding an instruction into a decoded instruction using a decoder of a core of a processor; 4904: executing the decoded instruction using an execution unit of the core of the processor to perform a first operation; 4906: receiving an input of a data flow graph including a plurality of nodes; 4908: overlaying the data flow graph onto an array of processing elements of the processor, and each node is represented as a data flow operator in the array of processing elements; and 4910: performing a second operation of the data flow graph using the array of processing elements when an incoming set of operands arrives at the array of processing elements.

[0408] ​FIG. 5000 is a flow chart according to an embodiment of the present disclosure. The depicted process 5000 includes: 5002: decoding an instruction into a decoded instruction using a decoder of a core of a processor; 5004: executing the decoded instruction using an execution unit of the core of the processor to perform a first operation; 5006: receiving an input of a data flow graph including a plurality of nodes; 5008: overlaying the data flow graph into a plurality of processing elements of the processor and an interconnection network between the plurality of processing elements of the processor, and each node is represented as a data flow operator among the plurality of processing elements; and 5010: when an incoming set of operands arrives at the plurality of processing elements, performing a second operation of the data flow graph using the interconnection network and the plurality of processing elements.

[0409] ​

[0410] ExaFLOP-scale supercomputing can be a challenge in high-performance computing, a challenge that may not be met by conventional von Neumann architectures. To achieve ExaFLOP, embodiments of CSA provide a heterogeneous spatial array that targets the direct execution of (e.g., compiler-generated) data flow graphs. In addition to laying out the architectural principles of embodiments of CSA, embodiments of CSA have been described and evaluated above, and these embodiments of CSA show 10x (10 times) higher performance and energy compared to existing products. Compiler-generated code can have significant performance and energy gains compared to the roadmap architecture. As a heterogeneous parametric architecture, embodiments of CSA can be easily adapted to all computing use cases. For example, a mobile version of CSA can be tuned to 32 bits, while the characteristics of an array focused on machine learning can be a significant number of vectorized 8-bit multiplication units. The main advantages of embodiments of CSA are high performance, extremely energy-efficient, and characteristic in relation to all forms of computing from supercomputing and data centers to the Internet of Things.

[0411] In one embodiment, a processor includes: a plurality of processing elements; an interconnection network between the plurality of processing elements, the interconnection network for receiving an input of a data flow graph including a plurality of nodes, wherein the data flow graph is for being overlaid onto the interconnection network and the plurality of processing elements, and each node is represented as a data flow operator in the interconnection network and the plurality of processing elements, and the plurality of processing elements are for performing operations when an incoming set of operands arrives at the plurality of processing elements; and a configuration controller, coupled to a first subset and a different second subset of the plurality of processing elements, the first subset of the plurality of processing elements having an output coupled to an input of the different second subset of the plurality of processing elements, wherein the configuration controller is for configuring the interconnection network between the first subset and the different second subset of the plurality of processing elements to: not permit communication on the interconnection network between the first subset and the different second subset of the plurality of processing elements when a privilege bit is set to a first value; and permit communication on the interconnection network between the first subset and the different second subset of the plurality of processing elements when the privilege bit is set to a second value. The interconnection network includes a privilege status register at a boundary between the first subset and the different second subset of the plurality of processing elements, the privilege status register for opening and closing a circuit at the boundary to: not permit communication on the interconnection network between the first subset and the different second subset of the plurality of processing elements when the privilege bit in the privilege status register is set to the first value by the configuration controller; and permit communication on the interconnection network between the first subset and the different second subset of the plurality of processing elements when the privilege bit in the privilege status register is set to the second value by the configuration controller. In a case where communication on the interconnection network between the first subset and the different second subset of the plurality of processing elements is not permitted when the privilege bit is set to the first value, a first context is for execution on the first subset of the plurality of processing elements, and a second context is for execution on the different second subset of the plurality of processing elements simultaneously. When the privilege bit is set to the first value, the configuration controller may not permit communication on either a forward data path or a backward flow control path of the interconnection network between the first subset and the different second subset of the plurality of processing elements, the backward flow control path being paired with the forward data path. When the privilege bit is set to the first value, the configuration controller may cause the interconnection network to pull all signals at a boundary between the first subset and the different second subset of the plurality of processing elements to zero.A configuration controller can be coupled to a first subset of a plurality of processing elements, a different second subset, and a different third subset, where the first subset of the plurality of processing elements has an output coupled to an input of the different second subset of the plurality of processing elements, and the different third subset of the plurality of processing elements has an output coupled to an input of the different second subset of the plurality of processing elements. The configuration controller can configure the interconnection network between the first subset of the plurality of processing elements, the different second subset, and the different third subset such that when a privilege bit is set to a first value, communication on the interconnection network between the first subset of the plurality of processing elements, the different second subset, and the different third subset is not allowed; and when the privilege bit is set to a second value, communication on the interconnection network between the first subset of the plurality of processing elements, the different second subset, and the different third subset is allowed. When the privilege bit is set to the second value, the configuration controller can configure the first subset of the plurality of processing elements and the different second subset according to configuration information for a first context of a data flow graph, and for a requested context switch, the configuration controller can configure the first subset of the plurality of processing elements according to configuration information for a second context of the data flow graph after pending operations for the first context are completed in the first subset, and block second context data flow from the output of the first subset of the plurality of processing elements to the input of the different second subset of the plurality of processing elements until pending operations for the first context are completed in the different second subset of the plurality of processing elements. The configuration controller can modify the privilege bit during the runtime of the operation.

[0412] In another embodiment, a method includes: when an incoming set of operands arrives at multiple processing elements of a processor, performing operations of a data flow graph using the processor's interconnection network and the multiple processing elements, wherein the processor includes multiple processing elements, an interconnection network between the multiple processing elements, and has a data flow graph that includes multiple nodes, the multiple nodes being mapped to the multiple processing elements of the processor and the interconnection network between the multiple processing elements of the processor, and each node being represented as a data flow operator in the interconnection network and the multiple processing elements; and configuring the interconnection network between a first subset of the multiple processing elements and a different second subset using a configuration controller of the processor such that: when a privilege bit is set to a first value, communication on the interconnection network between the first subset of the multiple processing elements and the different second subset of the multiple processing elements is not allowed; and when the privilege bit is set to a second value, communication on the interconnection network between the first subset of the multiple processing elements and the different second subset of the multiple processing elements is allowed. The configuration may include: the configuration controller updating the privilege bit in a privilege status register at the boundary between the first subset of the multiple processing elements and the different second subset of the multiple processing elements to turn on and off the circuit at the boundary so that: when the privilege bit in the privilege status register is set to the first value, communication on the interconnection network between the first subset of the multiple processing elements and the different second subset of the multiple processing elements is not allowed; and when the privilege bit in the privilege status register is set to the second value, communication on the interconnection network between the first subset of the multiple processing elements and the different second subset of the multiple processing elements is allowed. The method may further include: in the case where communication on the interconnection network between the first subset of the multiple processing elements and the different second subset of the multiple processing elements is not allowed when the privilege bit is set to the first value, executing a first context on the first subset of the multiple processing elements and simultaneously executing a second context on the different second subset of the multiple processing elements. The configuration may further include: when the privilege bit is set to the first value, preventing communication on either a forward data path or a backward flow control path of the interconnection network between the first subset of the multiple processing elements and the different second subset of the multiple processing elements, the backward flow control path being paired with the forward data path. The configuration may further include: when the privilege bit is set to the first value, causing the interconnection network to pull all signals at the boundary between the first subset of the multiple processing elements and the different second subset of the multiple processing elements to zero. The configuration may further include: configuring the interconnection network between the first subset of the multiple processing elements, the different second subset, and the different third subset such that: when the privilege bit is set to the first value, communication on the interconnection network between the first subset of the multiple processing elements, the different second subset, and the different third subset is not allowed; and when the privilege bit is set to the second value, communication on the interconnection network between the first subset of the multiple processing elements, the different second subset, and the different third subset is allowed.The method may further include: when the privilege bit is set to a second value, configuring a first subset and a different second subset of a plurality of processing elements according to configuration information for a first context of a data flow graph, and for a requested context switch, configuring the first subset of the plurality of processing elements according to configuration information for a second context of the data flow graph after pending operations for the first context have been completed in the first subset, and preventing data flow of the second context from entering an input of the different second subset of the plurality of processing elements from an output of the first subset of the plurality of processing elements until pending operations for the first context have been completed in the different second subset of the plurality of processing elements. The method may include: a configuration controller modifying the privilege bit during runtime of the operation.

[0413] In yet another embodiment, a non-transitory machine-readable medium storing code that, when executed by a machine, causes the machine to perform a method, the method comprising: when an incoming set of operands arrives at a plurality of processing elements of a processor, performing operations of a data flow graph using an interconnection network of the processor and the plurality of processing elements, wherein the processor includes a plurality of processing elements and an interconnection network between the plurality of processing elements and has a data flow graph that includes a plurality of nodes, the plurality of nodes being mapped to the plurality of processing elements of the processor and the interconnection network between the plurality of processing elements of the processor, and each node being represented as a data flow operator in the interconnection network and the plurality of processing elements; and configuring the interconnection network between a first subset of the plurality of processing elements and a different second subset using a configuration controller of the processor to: not allow communication on the interconnection network between the first subset of the plurality of processing elements and the different second subset of the plurality of processing elements when a privilege bit is set to a first value; and allow communication on the interconnection network between the first subset of the plurality of processing elements and the different second subset of the plurality of processing elements when the privilege bit is set to a second value. The configuration may include: the configuration controller updating the privilege bit in a privilege status register at a boundary between the first subset of the plurality of processing elements and the different second subset of the plurality of processing elements to open and close a circuit at the boundary so that: when the privilege bit in the privilege status register is set to the first value, not allow communication on the interconnection network between the first subset of the plurality of processing elements and the different second subset of the plurality of processing elements; and when the privilege bit in the privilege status register is set to the second value, allow communication on the interconnection network between the first subset of the plurality of processing elements and the different second subset of the plurality of processing elements. The method may further include: in a case where communication on the interconnection network between the first subset of the plurality of processing elements and the different second subset of the plurality of processing elements is not allowed when the privilege bit is set to the first value, performing a first context on the first subset of the plurality of processing elements and simultaneously performing a second context on the different second subset of the plurality of processing elements. The configuration may further include: when the privilege bit is set to the first value, preventing communication on either a forward data path or a backward flow control path of the interconnection network between the first subset of the plurality of processing elements and the different second subset of the plurality of processing elements, the backward flow control path being paired with the forward data path. The configuration may further include: when the privilege bit is set to the first value, causing the interconnection network to pull all signals at a boundary between the first subset of the plurality of processing elements and the different second subset of the plurality of processing elements to zero.The configuration may further include: configuring an interconnect network between a first subset of a plurality of processing elements, a different second subset, and a different third subset such that when a privilege bit is set to a first value, communication on the interconnect network between the first subset of the plurality of processing elements, the different second subset, and the different third subset is not allowed; and when the privilege bit is set to a second value, communication on the interconnect network between the first subset of the plurality of processing elements, the different second subset, and the different third subset is allowed. The method may further include: when the privilege bit is set to the second value, configuring the first subset of the plurality of processing elements and the different second subset according to configuration information for a first context of a data flow graph, and for a requested context switch, after pending operations of the first context are completed in the first subset, configuring the first subset of the plurality of processing elements according to configuration information for a second context of the data flow graph and blocking second context data flow from the output of the first subset of the plurality of processing elements from entering the input of the different second subset of the plurality of processing elements until pending operations of the first context are completed in the different second subset of the plurality of processing elements. The method may include: a configuration controller modifying the privilege bit during runtime of the operation.

[0414] In another embodiment, a processor includes: a plurality of processing elements; an interconnect network between the plurality of processing elements, the interconnect network for receiving an input of a data flow graph including a plurality of nodes, wherein the data flow graph is for being overlaid into the interconnect network and the plurality of processing elements, and each node is represented as a data flow operator in the interconnect network and the plurality of processing elements, and the plurality of processing elements for performing operations when an incoming set of operands arrives at the plurality of processing elements; and means coupled to a first subset of the plurality of processing elements and a different second subset, the first subset of the plurality of processing elements having an output coupled to an input of the different second subset of the plurality of processing elements, wherein the means is for configuring the interconnect network between the first subset of the plurality of processing elements and the different second subset such that when a privilege bit is set to a first value, communication on the interconnect network between the first subset of the plurality of processing elements and the different second subset of the plurality of processing elements is not allowed; and when the privilege bit is set to a second value, communication on the interconnect network between the first subset of the plurality of processing elements and the different second subset of the plurality of processing elements is allowed.

[0415] In one embodiment, a processor includes: a plurality of processing elements; an interconnection network between the plurality of processing elements for receiving an input of a data flow graph including a plurality of nodes, the data flow graph being for being overlaid onto the interconnection network and the plurality of processing elements, and each node being represented as a data flow operator in the interconnection network and the plurality of processing elements, the plurality of processing elements for performing operations when an incoming set of operands arrives at the plurality of processing elements; and a configuration controller coupled to a first subset and a different second subset of the plurality of processing elements, the first subset having an output coupled to an input of the different second subset, wherein the configuration controller is configured to: configure the first subset and the different second subset of the plurality of processing elements according to configuration information for a first context of the data flow graph; and after a pending operation for the first context has been completed in the first subset (e.g., until a backpressure signal is encountered and / or all input data has been consumed) (or the pending operation for the first context has not been completed, e.g., stop operating at a stopping point where the state can be extracted), for a requested context switch, configure the first subset of the plurality of processing elements according to configuration information for a second context of the data flow graph, and block the second context data flow from the output of the first subset into the input of the different second subset until the pending operation for the first context has been completed in the different second subset (e.g., until a backpressure signal is encountered and / or all input data has been consumed). The processor may include a first local configuration controller for the first subset and a second local configuration controller for the different second subset, wherein the configuration controller is configured to send corresponding configuration information to each of the first local configuration controller and the second local configuration controller. A pending operation may be an operation that is to (e.g., must) be completed to reach (e.g., fully) a savable state, e.g., see the discussion of one or more of the figures above. The configuration information may be stored somewhere and subsequently read in, and the extraction information may be written to somewhere in a storage, e.g., written to a virtual memory (e.g., via a RAF circuit). The configuration controller may include an extraction controller that is configured to cause state data from the first subset and the different second subset of the plurality of processing elements to be saved to a memory, and the extraction controller is configured to: when the pending operation for the first context has been completed in the first subset, for a requested context switch, extract first state data from the first subset. The plurality of processing elements may include a different third subset of the plurality of processing elements between the output of the first subset and the input of the different second subset, and the configuration controller is configured to: for a requested context switch, keep the different third subset of the plurality of processing elements in an unconfigured state to block the second context data flow from the output of the first subset to the input of the different second subset until the pending operation for the first context has been completed in the different second subset. The configuration controller may cause a backpressure signal for the different third subset to be output to the first subset of the plurality of processing elements for the unconfigured state.The configuration controller may allow operations on a first context in different second subsets to occur simultaneously (e.g., happen) with operations on a second context in a first subset.

[0416] In another embodiment, a method includes: receiving an input of a data flow graph including a plurality of nodes; overlaying the data flow graph onto a plurality of processing elements of a processor and an interconnection network among the plurality of processing elements of the processor, and each node is represented as a data flow operator in the interconnection network and the plurality of processing elements; when an incoming set of operands arrives at the plurality of processing elements, performing operations of the data flow graph using the interconnection network and the plurality of processing elements; configuring a first subset and a different second subset of the plurality of processing elements using a configuration controller of the processor according to configuration information for a first context of the data flow graph; and when a pending operation of the first context is completed in the first subset, for a requested context switch, configuring the first subset of the plurality of processing elements using the configuration controller of the processor according to configuration information for a second context of the data flow graph, and preventing second context data flow from an output of the first subset from entering an input of the different second subset until the pending operation of the first context is completed in the different second subset. The method may include: the configuration controller sending corresponding configuration information to each of a first local configuration controller of the first subset and a second local configuration controller of the different second subset. The method may include: extracting first state data from the first subset for a requested context switch when a pending operation of the first context is completed in the first subset. The method may include: for a requested context switch, keeping a different third subset of the plurality of processing elements between an output of the first subset and an input of the different second subset in an unconfigured state using the configuration controller to prevent second context data flow from the output of the first subset to the input of the different second subset until the pending operation of the first context is completed in the different second subset. The keeping may include: outputting backpressure signals of the different third subset to the first subset of the plurality of processing elements for the unconfigured state. The method may include: using the configuration controller to allow operations on a first context in different second subsets to occur simultaneously (e.g., happen) with operations on a second context in a first subset.

[0417] In yet another embodiment, a processor includes: a plurality of processing elements; an interconnect device between the plurality of processing elements for receiving an input of a data flow graph including a plurality of nodes, the data flow graph being for being overlaid onto the interconnect device and the plurality of processing elements, and each node being represented as a data flow operator in the interconnect device and the plurality of processing elements, the plurality of processing elements for performing operations when an incoming set of operands arrives at the plurality of processing elements; and a device coupled to a first subset and a different second subset of the plurality of processing elements, the first subset having an output coupled to an input of the different second subset, wherein the device is for: configuring the first subset and the different second subset of the plurality of processing elements according to configuration information for a first context of the data flow graph; and upon completion of a pending operation for the first context in the first subset, for a requested context switch, configuring the first subset of the plurality of processing elements according to configuration information for a second context of the data flow graph and preventing data flow for the second context from entering the input of the different second subset from the output of the first subset until a pending operation for the first context is completed in the different second subset.

[0418] In another embodiment, a processor includes: a plurality of processing elements; an interconnection network between the plurality of processing elements for receiving an input of a data flow graph including a plurality of nodes, the data flow graph being configured to be overlaid onto the interconnection network and the plurality of processing elements, and each node being represented as a data flow operator in the interconnection network and the plurality of processing elements, the plurality of processing elements being configured to perform operations when an incoming set of operands arrives at the plurality of processing elements; a first configuration controller coupled to a first subset of the plurality of processing elements; and a second configuration controller coupled to a different second subset of the plurality of processing elements, and the first subset having an output coupled to an input of the different second subset, wherein the first configuration controller and the second configuration controller are configured to: configure the first subset of the plurality of processing elements and the different second subset according to configuration information for a first context of the data flow graph; and upon completion of a pending operation for the first context in the first subset, for a requested context switch, the first configuration controller is configured to configure the first subset of the plurality of processing elements according to configuration information for a second context of the data flow graph and block second context data flow from the output of the first subset to the input of the different second subset until a pending operation for the first context is completed in the different second subset. The processor may include a higher-level configuration controller coupled to the first configuration controller and the second configuration controller, wherein the higher-level configuration controller is configured to send corresponding configuration information to each of the first configuration controller and the second configuration controller. The first configuration controller may include an extraction controller configured to cause state data from the first subset of the plurality of processing elements to be saved to a memory, and the extraction controller is configured to: upon completion of a pending operation for the first context in the first subset, for a requested context switch, extract first state data from the first subset. The plurality of processing elements may include a different third subset of the plurality of processing elements between the output of the first subset and the input of the different second subset, and a third configuration controller is coupled to the different third subset and is configured to: for a requested context switch, keep the different third subset of the plurality of processing elements in an unconfigured state to block second context data flow from the output of the first subset to the input of the different second subset until a pending operation for the first context is completed in the different second subset. The third configuration controller may cause a backpressure signal of the different third subset to be output to the first subset of the plurality of processing elements for the unconfigured state. The first configuration controller and the second configuration controller may allow operations for the first context in the different second subset and operations for the second context in the first subset to occur simultaneously (e.g., concurrently).

[0419] In yet another embodiment, a method includes: receiving an input of a data flow graph including a plurality of nodes; overlaying the data flow graph onto a plurality of processing elements of a processor and an interconnection network between the plurality of processing elements of the processor, and each node is represented as a data flow operator in the interconnection network and the plurality of processing elements; when an incoming set of operands arrives at the plurality of processing elements, performing operations of the data flow graph by using the interconnection network and the plurality of processing elements; configuring a first subset and a different second subset of the plurality of processing elements according to corresponding configuration information for a first context of the data flow graph by using a first configuration controller and a second configuration controller of the processor; and when pending operations of the first context are completed in the first subset, for a requested context switch, configuring the first subset of the plurality of processing elements according to configuration information for a second context of the data flow graph by using the first configuration controller of the processor, and preventing data flow of the second context from entering an input of the different second subset from an output of the first subset until pending operations of the first context are completed in the different second subset. The method may include: using a higher-level configuration controller of the processor to send the corresponding configuration information to each of the first configuration controller of the first subset and the second configuration controller of the different second subset. The method may include: extracting first state data from the first subset for a requested context switch when pending operations of the first context are completed in the first subset. The method may include: for a requested context switch, keeping a different third subset of the plurality of processing elements between an output of the first subset and an input of the different second subset in an unconfigured state by using a third configuration controller of a different third subset to prevent data flow of the second context from the output of the first subset to the input of the different second subset until pending operations of the first context are completed in the different second subset. The keeping may include: outputting backpressure signals of the different third subset to the first subset of the plurality of processing elements for the unconfigured state. The method may include: allowing operations of the first context in the different second subset and operations of the second context in the first subset to occur simultaneously (e.g., happen) by using the first configuration controller and the second configuration controller.

[0420] In another embodiment, a processor includes: a plurality of processing elements; an interconnection means between the plurality of processing elements for receiving an input of a data flow graph including a plurality of nodes, wherein the data flow graph is for being overlaid onto the interconnection means and the plurality of processing elements, and each node is represented as a data flow operator in the interconnection means and the plurality of processing elements, the plurality of processing elements for performing operations when an incoming operand set arrives at the plurality of processing elements; a first means coupled to a first subset of the plurality of processing elements; and a second means coupled to a different second subset of the plurality of processing elements, and the first subset has an output coupled to an input of the different second subset, wherein the first means and the second means are for: configuring the first subset of the plurality of processing elements and the different second subset according to configuration information for a first context of the data flow graph; and when a pending operation for the first context is completed in the first subset, for a requested context switch, the first means is for configuring the first subset of the plurality of processing elements according to configuration information for a second context of the data flow graph and preventin...

Claims

1. A processor, comprising: a plurality of processing elements; an interconnect network between the plurality of processing elements, the interconnect network being configured to receive an input of a data flow graph including a plurality of nodes, wherein the data flow graph is configured to be overlaid onto the interconnect network and the plurality of processing elements, and each node is represented as a data flow operator in the interconnect network and the plurality of processing elements, and the plurality of processing elements are configured to perform operations when an incoming set of operands arrives at the plurality of processing elements; and a configuration controller coupled to a first subset and a different second subset of the plurality of processing elements, the first subset of the plurality of processing elements having an output that is coupled to an input of the different second subset of the plurality of processing elements according to configuration bits, wherein the configuration controller is configured to configure the interconnect network between the first subset and the different second subset of the plurality of processing elements such that communication on the interconnect network between the first subset and the different second subset of the plurality of processing elements is not allowed when a privilege bit separate from the configuration bits is set to a first value; and communication on the interconnect network between the first subset and the different second subset of the plurality of processing elements is allowed when the privilege bit is set to a second value.

2. The processor according to claim 1, wherein, The interconnect network includes a privilege status register at a boundary between the first subset and the different second subset of the plurality of processing elements, the privilege status register being configured to turn on and off circuitry at the boundary to: not allow communication on the interconnect network between the first subset and the different second subset of the plurality of processing elements when the privilege bit in the privilege status register is set to the first value by the configuration controller; and allow communication on the interconnect network between the first subset and the different second subset of the plurality of processing elements when the privilege bit in the privilege status register is set to the second value by the configuration controller.

3. The processor according to claim 1, wherein, In a case where communication on the interconnect network between the first subset and the different second subset of the plurality of processing elements is not allowed when the privilege bit is set to the first value, a first context is configured to execute on the first subset of the plurality of processing elements, and a second context is configured to execute simultaneously on the different second subset of the plurality of processing elements.

4. The processor according to claim 1, wherein, The configuration controller is configured to: not allow communication on either a forward data path or a backward flow control path of the interconnect network between the first subset and the different second subset of the plurality of processing elements when the privilege bit is set to the first value, the backward flow control path being paired with the forward data path.

5. The processor according to claim 1, wherein, The configuration controller is configured to: when the privilege bit is set to the first value, cause the interconnect network to zero all signals at the boundary between the first subset of the plurality of processing elements and the different second subset of the plurality of processing elements.

6. The processor according to claim 1, wherein The configuration controller is coupled to the first subset, the different second subset, and the different third subset of the plurality of processing elements, the first subset of the plurality of processing elements having outputs coupled to inputs of the different second subset of the plurality of processing elements, and the different third subset of the plurality of processing elements having outputs coupled to inputs of the different second subset of the plurality of processing elements, wherein the configuration controller is configured to configure the interconnect network between the first subset, the different second subset, and the different third subset of the plurality of processing elements to: when the privilege bit is set to the first value, not allow communication on the interconnect network between the first subset, the different second subset, and the different third subset of the plurality of processing elements; and when the privilege bit is set to the second value, allow communication on the interconnect network between the first subset, the different second subset, and the different third subset of the plurality of processing elements.

7. The processor according to claim 1, wherein, The configuration controller is configured to: when the privilege bit is set to the second value, configure the first subset and the different second subset of the plurality of processing elements according to configuration information for a first context of the data flow graph, and for a requested context switch, configure the first subset of the plurality of processing elements according to configuration information for a second context of the data flow graph after pending operations for the first context are completed in the first subset, and block second context data flow from outputs of the first subset of the plurality of processing elements into inputs of the different second subset of the plurality of processing elements until pending operations for the first context are completed in the different second subset of the plurality of processing elements.

8. The processor according to any one of claims 1-7, wherein, The configuration controller is configured to modify the privilege bit during runtime of the operation.

9. A method, comprising: When an incoming set of operands arrives at a plurality of processing elements of a processor, performing operations of a data flow graph using the interconnect network of the processor and the plurality of processing elements, wherein the processor includes the plurality of processing elements and the interconnect network between the plurality of processing elements, and the data flow graph includes a plurality of nodes that are mapped into the plurality of processing elements of the processor and the interconnect network between the plurality of processing elements of the processor according to configuration bits, and each node is represented as a data flow operator in the interconnect network and the plurality of processing elements; and Using a configuration controller of the processor, configure the interconnect network between a first subset and a different second subset of the plurality of processing elements such that when a privilege bit separate from the configuration bits is set to a first value, communication on the interconnect network between the first subset and the different second subset of the plurality of processing elements is not allowed; and when the privilege bit is set to a second value, communication on the interconnect network between the first subset and the different second subset of the plurality of processing elements is allowed.

10. The method according to claim 9, wherein, The configuration includes: the configuration controller updates the privilege bit in the privilege status register at the boundary between the first subset and the different second subset of the plurality of processing elements to turn on and off the circuit at the boundary so that: when the privilege bit in the privilege status register is set to the first value, communication on the interconnect network between the first subset and the different second subset of the plurality of processing elements is not allowed; and when the privilege bit in the privilege status register is set to the second value, communication on the interconnect network between the first subset and the different second subset of the plurality of processing elements is allowed.

11. The method according to claim 9, further comprising: In the case where communication on the interconnect network between the first subset and the different second subset of the plurality of processing elements is not allowed when the privilege bit is set to the first value, a first context is executed on the first subset of the plurality of processing elements, and at the same time, a second context is executed on the different second subset of the plurality of processing elements.

12. The method according to claim 9, wherein The configuration further includes: when the privilege bit is set to the first value, prevent communication on either the forward data path or the backward flow control path of the interconnect network between the first subset and the different second subset of the plurality of processing elements, where the backward flow control path is paired with the forward data path.

13. The method according to claim 9, wherein, The configuration further includes: when the privilege bit is set to the first value, cause the interconnect network to pull all signals at the boundary between the first subset and the different second subset of the plurality of processing elements to zero.

14. The method according to claim 9, wherein The configuration further includes: configure the interconnect network between the first subset, the different second subset, and a different third subset of the plurality of processing elements such that when the privilege bit is set to the first value, communication on the interconnect network between the first subset, the different second subset, and the different third subset of the plurality of processing elements is not allowed; and when the privilege bit is set to the second value, communication on the interconnect network between the first subset, the different second subset, and the different third subset of the plurality of processing elements is allowed.

15. The method according to claim 9, further comprising: When the privilege bit is set to the second value, the first subset of the plurality of processing elements and the different second subset are configured according to the configuration information for the first context of the data flow graph, and for a requested context switch, when the pending operations of the first context are completed in the first subset, the first subset of the plurality of processing elements is configured according to the configuration information for the second context of the data flow graph, and the second context data stream is blocked from entering the input of the different second subset of the plurality of processing elements from the output of the first subset of the plurality of processing elements until the pending operations of the first context are completed in the different second subset of the plurality of processing elements.

16. The method according to any one of claims 9-15, further comprising: The configuration controller modifies the privilege bit during the runtime of the operation.

17. A non-transitory machine-readable medium storing code that, when executed by a machine, causes the machine to perform a method, the method comprising: When an incoming set of operands arrives at a plurality of processing elements of a processor, performing operations of a data flow graph using the interconnect network of the processor and the plurality of processing elements, wherein the processor includes the plurality of processing elements and the interconnect network between the plurality of processing elements, and the data flow graph includes a plurality of nodes that are mapped according to configuration bits into the plurality of processing elements of the processor and the interconnect network between the plurality of processing elements of the processor, and each node is represented as a data flow operator in the interconnect network and the plurality of processing elements; and Configuring the interconnect network between a first subset of the plurality of processing elements and a different second subset using a configuration controller of the processor such that: when a privilege bit separate from the configuration bits is set to a first value, communication on the interconnect network between the first subset of the plurality of processing elements and the different second subset of the plurality of processing elements is not allowed; and when the privilege bit is set to a second value, communication on the interconnect network between the first subset of the plurality of processing elements and the different second subset of the plurality of processing elements is allowed.

18. The non-transitory machine-readable medium according to claim 17, wherein, The configuration includes: the configuration controller updates the privilege bit in a privilege status register at the boundary between the first subset of the plurality of processing elements and the different second subset of the plurality of processing elements to open and close a circuit at the boundary so that: when the privilege bit in the privilege status register is set to the first value, communication on the interconnect network between the first subset of the plurality of processing elements and the different second subset of the plurality of processing elements is not allowed; and when the privilege bit in the privilege status register is set to the second value, communication on the interconnect network between the first subset of the plurality of processing elements and the different second subset of the plurality of processing elements is allowed.

19. The non-transitory machine-readable medium according to claim 17, wherein, The method further includes: when communication on the interconnect network between the first subset of the plurality of processing elements and the different second subset of the plurality of processing elements is not allowed while the privilege bit is set to the first value, executing a first context on the first subset of the plurality of processing elements and simultaneously executing a second context on the different second subset of the plurality of processing elements.

20. The non-transitory machine-readable medium according to claim 17, wherein, The configuration further includes: when the privilege bit is set to the first value, preventing communication on either the forward data path or the flow control path of the backward flow of the interconnect network between the first subset of the plurality of processing elements and the different second subset of the plurality of processing elements, where the flow control path of the backward flow is paired with the forward data path.

21. The non-transitory machine-readable medium according to claim 17, wherein, The configuration further includes: when the privilege bit is set to the first value, causing the interconnect network to pull all signals at the boundary between the first subset of the plurality of processing elements and the different second subset of the plurality of processing elements to zero.

22. The non-transitory machine-readable medium according to claim 17, wherein, The configuration further includes: configuring the interconnect network between the first subset, the different second subset, and the different third subset of the plurality of processing elements such that when the privilege bit is set to the first value, communication on the interconnect network between the first subset, the different second subset, and the different third subset of the plurality of processing elements is not allowed; and when the privilege bit is set to the second value, communication on the interconnect network between the first subset, the different second subset, and the different third subset of the plurality of processing elements is allowed.

23. The non-transitory machine-readable medium according to claim 17, wherein, The method further includes: when the privilege bit is set to the second value, configuring the first subset and the different second subset of the plurality of processing elements according to the configuration information for the first context of the data flow graph, and for a requested context switch, when the pending operations of the first context are completed in the first subset, configuring the first subset of the plurality of processing elements according to the configuration information for the second context of the data flow graph and blocking the second context data flow from the output of the first subset of the plurality of processing elements from entering the input of the different second subset of the plurality of processing elements until the pending operations of the first context are completed in the different second subset of the plurality of processing elements.

24. The non-transitory machine-readable medium according to any one of claims 17-23, wherein The method further includes: the configuration controller modifying the privilege bit during the runtime of the operation.

25. An apparatus comprising means for performing the method according to any one of claims 9 - 16.

Images