Processor and method for configurable clock gating in spatial arrays

Abstract

The present application provides a processor and method for configurable clock gating in a spatial array. Methods and apparatus for configurable clock gating in a spatial array are described. In one embodiment, a processor includes: processing elements; an interconnection network between the processing elements; and a configuration controller coupled to a first processing element and a second processing element of a plurality of processing elements, wherein the first processing element has an output coupled to an input of the second processing element, the configuration controller being configured to configure the second processing element to clock gate at least one clocked component of the second processing element, and to configure the first processing element to send a re-enable signal on the interconnection network to the second processing element to re-enable at least one clocked component of the second processing element when data is to be sent from the first processing element to the second processing element.

Description

Technical Field

[0001] The present disclosure relates generally to electronics, and more particularly, embodiments of the present disclosure relate to circuits for configurable time gating in spatial arrays. Background Art

[0002] A processor or set of processors executes instructions from an instruction set (e.g., an instruction set architecture (ISA)). An instruction set is the part of a computer's architecture that involves programming and generally includes native data types, instructions, register architecture, addressing modes, memory architecture, interrupt and exception handling, and external input and output (I / O). It should be noted that the term "instruction" is generally used herein to refer to either macroinstructions (e.g., instructions provided to a processor for execution) or microinstructions (e.g., instructions generated by a processor's decoder decoding a macroinstruction). BRIEF DESCRIPTION OF THE DRAWINGS

[0003] The present disclosure is illustrated by way of example and not limitation in the accompanying figures and in which like reference numerals indicate similar elements and in which:

[0004] Figure 1 An accelerator sheet according to an embodiment of the present disclosure is illustrated.

[0005] Figure 2 Illustrated is a hardware processor coupled to a memory according to an embodiment of the present disclosure.

[0006] Figure 3 An accelerator slice according to an embodiment of the present disclosure is shown, the accelerator slice including an array of processing elements.

[0007] Figure 4 Illustrated is a logical view of clock gating hardware in a processor element according to an embodiment of the present disclosure.

[0008] Figure 5 A processing element including clock gating circuitry is illustrated according to an embodiment of the present disclosure.

[0009] Figure 6 Illustrated are processing elements according to an embodiment of the present disclosure.

[0010] Figure 7 A flowchart according to an embodiment of the present disclosure is illustrated.

[0011] Figure 8 A flowchart according to an embodiment of the present disclosure is illustrated.

[0012] Figure 9 Illustrate context switching in a spatial array of processing elements of a processor according to an embodiment of the present disclosure.

[0013] Figures 10A-10DAn in-flight configuration of context switching for a spatial array of processing elements is illustrated according to an embodiment of the present disclosure.

[0014] Figures 11A-11J Illustrate the staged extraction of context for a spatial array of processing elements configured for executing a dataflow graph, according to an embodiment of the present disclosure.

[0015] Figure 12A The extracted states according to an embodiment of the present disclosure are illustrated.

[0016] Figure 12B FIG. 1 illustrates a state at the start of extraction according to an embodiment of the present disclosure.

[0017] Figure 13 A state machine for (eg, configuring) a controller according to an embodiment of the present disclosure is illustrated.

[0018] Figure 14A Illustrate the extraction of context for a spatial array of processing elements according to an embodiment of the present disclosure.

[0019] Figure 14B Illustrate the extraction of context for a spatial array of processing elements according to an embodiment of the present disclosure.

[0020] Figure 15 Illustrated is a staged extraction of context for a spatial array of processing elements including a (eg, mezzanine or global) network therebetween, in accordance with an embodiment of the present disclosure.

[0021] Figure 16 Illustrated is a staged fetch of context for a spatial array of processing elements, including memory accesses, according to an embodiment of the present disclosure.

[0022] Figure 17A Illustrate the extraction of context for a spatial array of processing elements according to an embodiment of the present disclosure.

[0023] Figure 17B Illustrate the extraction of context for a spatial array of processing elements according to an embodiment of the present disclosure.

[0024] Figure 18 A flowchart according to an embodiment of the present disclosure is illustrated.

[0025] Figure 19 A flowchart according to an embodiment of the present disclosure is illustrated.

[0026] Figure 20A A program source according to an embodiment of the present disclosure is illustrated.

[0027] Figure 20B FIG. 1 shows an embodiment of the present disclosure. Figure 20A The data flow diagram of the program source.

[0028] Figure 20C The accelerator according to an embodiment of the present disclosure is shown. Figure 20B A data flow graph of multiple processing elements.

[0029] Figure 21 Illustrate an example execution of a data flow graph according to an embodiment of the present disclosure.

[0030] Figure 22 A program source according to an embodiment of the present disclosure is illustrated.

[0031] Figure 23 An accelerator slice including an array of processing elements is illustrated according to an embodiment of the present disclosure.

[0032] Figure 24A Illustrated is a configurable data path network according to an embodiment of the present disclosure.

[0033] Figure 24B Illustrated is a configurable flow control path network according to an embodiment of the present disclosure.

[0034] Figure 25 Illustrated is a hardware processor slice including an accelerator according to an embodiment of the present disclosure.

[0035] Figure 26 Illustrated are processing elements according to an embodiment of the present disclosure.

[0036] Figure 27 Illustrated is a request address file (RAF) circuit according to an embodiment of the present disclosure.

[0037] Figure 28 Illustrated are multiple request address file (RAF) circuits coupled between multiple accelerator slices and multiple cache banks according to an embodiment of the present disclosure.

[0038] Figure 29 The diagram illustrates a floating-point multiplier partitioned into three regions (a result region, three potential carry regions, and a gate region) according to an embodiment of the present disclosure.

[0039] Figure 30 FIG. 1 illustrates a running configuration of an accelerator having multiple processing elements according to an embodiment of the present disclosure.

[0040] Figure 31 Illustrate a snapshot of a pipelined extraction in action, according to an embodiment of the present disclosure.

[0041] Figure 32FIG. 4 illustrates a compilation tool chain for an accelerator according to an embodiment of the present disclosure.

[0042] Figure 33 FIG. 1 illustrates a compiler for an accelerator according to an embodiment of the present disclosure.

[0043] Figure 34A Serialized assembly code according to an embodiment of the present disclosure is illustrated.

[0044] Figure 34B FIG. 1 shows an embodiment of the present disclosure. Figure 34A Data flow assembly code for serialization assembly code.

[0045] Figure 34C FIG. 1 illustrates an embodiment of the present disclosure for an accelerator. Figure 34B The data flow diagram of the data flow assembly code.

[0046] Figure 35A FIG. 1 illustrates C source code according to an embodiment of the present disclosure.

[0047] Figure 35B FIG. 1 shows an embodiment of the present disclosure. Figure 35A The data flow of C source code is compiled into assembly code.

[0048] Figure 35C FIG. 1 illustrates an embodiment of the present disclosure for an accelerator. Figure 35B The data flow diagram of the data flow assembly code.

[0049] Figure 36A FIG. 1 illustrates C source code according to an embodiment of the present disclosure.

[0050] Figure 36B FIG. 1 shows an embodiment of the present disclosure. Figure 36A The data flow of C source code is compiled into assembly code.

[0051] Figure 36C FIG. 1 illustrates an embodiment of the present disclosure for an accelerator. Figure 36B The data flow diagram of the data flow assembly code.

[0052] Figure 37A A flowchart according to an embodiment of the present disclosure is illustrated.

[0053] Figure 37B A flowchart according to an embodiment of the present disclosure is illustrated.

[0054] Figure 38 Graph illustrating throughput versus energy per operation according to an embodiment of the present disclosure.

[0055] Figure 39An accelerator slice according to an embodiment of the present disclosure is illustrated, the accelerator slice including an array of processing elements and a local configuration controller.

[0056] Figures 40A-40C Illustrated is a local configuration controller configuring a data path network according to an embodiment of the present disclosure.

[0057] Figure 41 A configuration controller according to an embodiment of the present disclosure is illustrated.

[0058] Figure 42 An accelerator slice is illustrated that includes an array of processing elements, a configuration cache, and a local configuration controller according to an embodiment of the present disclosure.

[0059] Figure 43 An accelerator slice including an array of processing elements and a configuration and exception handling controller with reconfiguration circuitry is illustrated according to an embodiment of the present disclosure.

[0060] Figure 44 A reconfiguration circuit according to an embodiment of the present disclosure is illustrated.

[0061] Figure 45 An accelerator slice including an array of processing elements and a configuration and exception handling controller with reconfiguration circuitry is illustrated according to an embodiment of the present disclosure.

[0062] Figure 46 An accelerator slice according to an embodiment of the present disclosure is illustrated, the accelerator slice including an array of processing elements and a mezzanine anomaly aggregator coupled to a chip-level anomaly aggregator.

[0063] Figure 47 Illustrated is a processing element with an exception generator according to an embodiment of the present disclosure.

[0064] Figure 48 An accelerator slice is illustrated, comprising an array of processing elements and a local fetch controller, according to an embodiment of the present disclosure.

[0065] Figures 49A-49C Illustrated is a local extraction controller configuring a data path network according to an embodiment of the present disclosure.

[0066] Figure 50 An extraction controller according to an embodiment of the present disclosure is shown.

[0067] Figure 51 A flowchart according to an embodiment of the present disclosure is shown.

[0068] Figure 52 A flowchart according to an embodiment of the present disclosure is shown.

[0069] Figure 53Ais a block diagram illustrating a general vector friendly instruction format and class A instruction templates thereof according to an embodiment of the present disclosure.

[0070] Figure 53B is a block diagram illustrating a generic vector friendly instruction format and its class B instruction template according to an embodiment of the present disclosure.

[0071] Figure 54A is a diagram illustrating a method for Figure 53A and Figure 53B Block diagram of the fields of the generic vector friendly instruction format in .

[0072] Figure 54B is a diagram illustrating a structure of a complete opcode field according to one embodiment of the present disclosure. Figure 54A Block diagram of the fields of the specialized vector friendly instruction format in .

[0073] Figure 54C FIG. 1 is a diagram illustrating a register index field according to an embodiment of the present disclosure. Figure 54A Block diagram of the fields of the specialized vector friendly instruction format in .

[0074] Figure 54D FIG. 5 is a diagram illustrating the structure of the extended operation field 5350 according to one embodiment of the present disclosure. Figure 54A Block diagram of the fields of the specialized vector friendly instruction format in .

[0075] Figure 55 is a block diagram of a register architecture according to one embodiment of the present disclosure.

[0076] Figure 56A is a block diagram illustrating both an exemplary in-order pipeline and an exemplary register-renaming, out-of-order issue / execution pipeline according to embodiments of the present disclosure.

[0077] Figure 56B is a block diagram illustrating both an exemplary embodiment of an in-order architecture core and an exemplary register renaming, out-of-order issue / execution architecture core to be included in a processor according to embodiments of the present disclosure.

[0078] Figure 57A is a block diagram of a single processor core and its connections to the on-die interconnect network and its local subset of the Level 2 (L2) cache according to an embodiment of the present disclosure.

[0079] Figure 57B According to an embodiment of the present disclosure Figure 57A An expanded view of a portion of a processor core in FIG.

[0080] Figure 58is a block diagram of a processor that may have more than one core, may have an integrated memory controller, and may have integrated graphics according to an embodiment of the present disclosure.

[0081] Figure 59 is a block diagram of a system according to one embodiment of the present disclosure.

[0082] Figure 60 is a block diagram of a more specific exemplary system according to an embodiment of the present disclosure.

[0083] Figure 61 Shown is a block diagram of a second, more specific exemplary system according to an embodiment of the present disclosure.

[0084] Figure 62 Shown is a block diagram of a system on a chip (SoC) according to an embodiment of the present disclosure.

[0085] Figure 63 is a block diagram illustrating converting binary instructions in a source instruction set into binary instructions in a target instruction set using a software instruction converter according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0086] In the following description, a number of specific details are set forth. However, it should be understood that embodiments of the present disclosure may be implemented without these specific details. In other instances, well-known circuits, structures, and technologies are not shown in detail to avoid obscuring the understanding of this description.

[0087] References in the specification to "one embodiment," "an embodiment," "an example embodiment," and the like indicate that the described embodiment may include a particular feature, structure, or characteristic, but not every embodiment necessarily includes that particular feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it is understood that it is within the knowledge of those skilled in the art to be able to affect such feature, structure, or characteristic in conjunction with other embodiments, whether or not explicitly described.

[0088] A processor (e.g., having one or more cores) may execute instructions (e.g., an instruction thread) to operate on data, for example, to perform arithmetic, logical, or other functions. For example, software may request an operation, and a hardware processor (e.g., one or more cores of the hardware processor) may perform the operation in response to the request. A non-limiting example of an operation is a blend operation that inputs multiple vector elements and outputs a vector having the blended multiple elements. In some embodiments, multiple operations are performed using the execution of a single instruction.

[0089] For example, exascale performance as defined by the U.S. Department of Energy may require system-level floating-point performance exceeding 10 Mbps within a given (e.g., 20 MW) power budget. 18 Certain embodiments herein relate to spatial arrays of processing elements (e.g., configurable spatial accelerators (CSAs)) for high-performance computing (HPC), such as processors. Certain embodiments herein of spatial arrays of processing elements (e.g., CSAs) are directed to direct execution of a dataflow graph (or multiple dataflow graphs) to achieve a computationally intensive yet energy-efficient spatial microarchitecture that far exceeds conventional roadmap architectures.

[0090] Certain embodiments of spatial architectures (e.g., the spatial arrays disclosed herein) are energy-efficient and high-performance methods for accelerating user applications. In some embodiments, a spatial array (e.g., a plurality of processing elements coupled together by a (e.g., circuit-switched) (e.g., interconnect) network) is used to accelerate an application, for example, to execute a region of a single stream program (e.g., faster than a core of a processor). In some embodiments, a measure of the effectiveness of a spatial architecture is the speed with which a region (e.g., to be accelerated) can be accelerated into the spatial architecture, e.g., the longer it takes to load the region, the larger the region can be in order to amortize the cost of loading the program. Conversely, where configuration time is short, smaller program regions can be accelerated, for example, expanding the applicability of a spatial architecture (e.g., an accelerator).

[0091] Certain embodiments herein provide hardware and techniques for providing clock gating in a spatial array (e.g., a spatial structure), for example, clock gating in one or more (e.g., each) processing elements of the spatial array. Clock gating can generally refer to turning off or preventing a clocking signal (e.g., a clock waveform) to a clocked component (e.g., a register or buffer), for example, turning off toggling / switching (and, for example, power consumption) caused by the clocking signal. For example, instead of turning off a clocked component and losing the data stored therein, the clocked component can maintain its current state (e.g., maintaining any data stored therein). The clocked component can be re-enabled to turn off clock gating, for example, to latch (e.g., new) data values into the clocked component. Certain embodiments herein provide (e.g., coarse-grained) programmable clock gating. Certain embodiments herein relate to a spatial architecture (e.g., a configurable spatial array (CSA)) that provides energy-efficient and high-performance acceleration of user applications. In some embodiments, the parallel nature of the spatial architecture enables multiple (e.g., many) processing elements (PEs) to execute simultaneously, making power (e.g., clock timing power) a primary consideration across the spatial array. In some embodiments, the programmatic manner in which the spatial array (e.g., spatial structure) is configured for a given dataflow graph allows for unique clock gating techniques to be used in hardware (e.g., in conjunction with software assistance). For example, by using (e.g., prior) knowledge of what will happen in the future within the spatial array, clock gating can be utilized globally and / or locally (e.g., this will change dynamically as the algorithm or dataflow graph progresses). A spatial array (e.g., as an accelerator) can be a (e.g., general-purpose) compute engine until the spatial array is programmed with a specific dataflow graph to execute a given code segment. The PEs and (e.g., interconnect) networks can be configured to complete specific tasks over and over again. By allowing the programmer / compiler / router to incorporate information known prior to execution about the specific operation(s) being called (e.g., hints) and / or the specific distances (e.g., lengths) of path connections in the circuit and / or their physical properties, the PE can be configured (e.g., via clock gating) to reduce clock switching to reduce power used. Certain embodiments herein utilize a priori knowledge of the size of data being clocked in and out to control the number of bits used and the number of bits that are clock gated in a clocked component. For example, in one embodiment, a clocked component (e.g., a data buffer) may have one or more elements clock gated (e.g., see the discussion of Figure 4), for example, clock gating a constant or portion of a data buffer that will not be used in an operation, such as clock gating the upper or lower halves (e.g., 32 bits) of a (e.g., 64-bit) buffer or register (e.g., when a smaller data value (e.g., 32 bits) will be used), or clock gating one or more portions of an address that do not pass through a certain address range to turn off those unused bits.

[0092] Certain embodiments herein provide hardware and techniques for pipeline configuration in a spatial array (e.g., a spatial structure). Certain embodiments herein utilize (e.g., regional) control (e.g., a configuration controller) and (e.g., low-level) dataflow semantics of a spatial array (e.g., a configurable spatial array (CSA)) to produce a pipelined configuration effect that enables previously (e.g., early) configured (e.g., processing) elements of the spatial array to begin operation (e.g., immediately) before the entire (e.g., segment) of the spatial array is configured. Certain embodiments herein may reduce the effective latency of configuration to tens of nanoseconds. In one embodiment, configuration may be two (e.g., separate) operations: for example, the actual configuration and the (e.g., simultaneous) extraction of a previous configuration (e.g., the state of the previous configuration) loaded into the spatial array (e.g., the structure), which operations may occur during a context switch. Certain embodiments herein may allow these operations to occur simultaneously within the spatial array. Certain embodiments herein may use (multiple) microprotocols for configuration and extraction, for example, as described below with reference to Figure 30 、 Figure 31 as well as Figures 39-46 discussed.

[0093] Certain embodiments herein provide techniques and hardware (e.g., microarchitecture extensions and / or definitions) for permitting pipelining of configuration and / or extraction operations for empty spatial arrays. Certain embodiments herein utilize one or more controllers to arrange wavefronts for configuration and extraction regions across spatial arrays. Certain embodiments herein utilize higher-level (e.g., configuration and / or extraction) controllers to arrange local-level controllers, which implement wavefronts for configuration and extraction regions across spatial arrays. In one embodiment, the wavefront logically separates new and old (e.g., program) contexts, which, for example, enables the new context to execute immediately. Certain embodiments herein convert what was previously a serial process (e.g., extraction, followed by configuration) into a pipelined process, thereby, for example, reducing latency by an order of magnitude.

[0094] Certain embodiments herein reduce the amount of time required to configure (and / or extract) a spatial accelerator, thereby enabling, for example, profitable acceleration of smaller code regions. As a result, the performance of more programs can be improved, and the performance of more programs can be improved to a greater extent.

[0095] The following also includes a description of the architectural concepts for embodiments of spatial arrays of processing elements (e.g., CSAs) and certain features thereof. As with any revolutionary architecture, programmability can be a risk. To mitigate this issue, embodiments of the CSA architecture have been co-designed with a compilation toolchain (also discussed below).

[0096] Introduction

[0097] Exascale computing goals may require massive system-level floating-point performance (e.g., 1 ExaFLOP) within a dramatic power budget (e.g., 20MW). However, it has become difficult to simultaneously improve the performance and energy efficiency of program execution using classical von Neumann architectures: out-of-order scheduling, simultaneous multi-threaded operation, complex register files, and other structures provide performance, but at a high energy cost. Certain embodiments herein achieve both performance and energy requirements. Exascale computing power-performance goals may require both high throughput and low energy consumption for each operation. Certain embodiments herein provide this by providing a large number of low-complexity, energy-efficient processing (e.g., compute) elements that significantly eliminate the control overhead of previous processor designs. Guided by this observation, certain embodiments herein include a spatial array of processing elements, such as a configurable spatial accelerator (CSA), which includes, for example, an array of processing elements (PEs) connected by a lightweight back-pressure (e.g., communication) network. An example of a CSA slice is in Figure 1 Some embodiments of processing (e.g., compute) elements are dataflow operators, e.g., multiple dataflow operators that only process input data when both (i) the input data has arrived at the dataflow operator and (ii) there is space available to store output data (e.g., otherwise no processing is occurring). Some embodiments (e.g., accelerators or CSAs) do not utilize triggered instructions.

[0098] Such as Figure 1The coarse-grained spatial architecture of an embodiment of a configurable spatial accelerator (CAS) shown in FIG is composed of lightweight processing elements (PEs) connected by an interconnection network. For example, a program viewed as a control flow graph can be mapped onto the architecture by configuring the PEs and the network. In general, the PEs can be configured as data flow operators, for example, once all input operands arrive at the PE, an operation occurs, and the results are forwarded downstream (for example, to (multiple) destination PEs) in a pipelined manner. The data flow operators can choose to consume the incoming data on an operator-by-operator basis.

[0099] Certain embodiments herein extend the capabilities of a spatial array (e.g., a CSA) to include clock gating within the spatial array (e.g., a spatial fabric), for example, clock gating within one or more (e.g., each) processing element of the spatial array. The spatial array may include one or more of the components or methods discussed below. As an example, in an HPC system, power can be one of the primary limiting factors in performance and / or design. Certain embodiments herein, for example, use knowledge from a programmer and / or compiler to reduce clock timing power for high-density processing elements.

[0100] Figure 1 An embodiment of an accelerator slice 100 illustrating a spatial array of processing elements according to an embodiment of the present disclosure. The accelerator slice 100 may be part of a larger slice. The accelerator slice 100 executes one or more data flow graphs. Data flow Figure 1 Generally, it may refer to an explicitly parallel program description that appears at compile time for serialized code. Certain embodiments herein (e.g., CSA) allow dataflow graphs to be configured directly onto a CSA array, e.g., rather than being transformed into a serialized instruction stream. Certain embodiments herein allow a first (e.g., type) of dataflow operations to be performed by one or more processing elements (PEs) of a spatial array, and additionally or alternatively, allow a second (e.g., different type) of dataflow operations to be performed by one or more of the network communication circuits (e.g., endpoints) of the spatial array.

[0101] The departure of the dataflow graph from the serialized compilation flow allows embodiments of the CSA to support familiar programming models and directly execute existing high performance computing (HPC) code (e.g., without using work tables). CSA processing elements (PEs) can be energy efficient. Figure 1 , the memory interface 102 may be coupled to a memory (eg, Figure 2Memory 202 in the accelerator slice 100 allows the accelerator slice 100 to access (e.g., load and / or store) data from (e.g., off-die) memory. The depicted accelerator slice 100 is composed of a heterogeneous array of several types of PEs coupled together via an interconnect network 105. The accelerator slice 100 may include one or more of the following as part of the spatial array of processing elements 101: integer arithmetic PEs, floating-point arithmetic PEs, communication circuits (e.g., network data flow endpoint circuits), and fabric storage. A dataflow graph (e.g., a compiled dataflow graph) may be overlaid on the accelerator slice 100 for execution. In one embodiment, for a particular dataflow graph, each PE only handles one or two (e.g., dataflow) operations in the graph. The PE array may be heterogeneous, for example, such that no PE supports the full CSA dataflow architecture and / or one or more PEs are programmed (e.g., customized) to perform only a few, but highly efficient, operations. Certain embodiments herein thus implement processors or accelerators having arrays of processing elements that are more computationally intensive than roadmap architectures, and achieve approximately order-of-magnitude gains in energy efficiency and performance relative to existing HPC offerings.

[0102] Certain embodiments herein provide performance gains from parallel execution within a (e.g., dense) spatial array of processing elements (e.g., CSA), where each utilized PE can execute its operations concurrently, e.g., if input data is available. Efficiency gains can arise from the efficiency of each PE, e.g., where the operations (e.g., behavior) of each PE are fixed once for each configuration (e.g., mapping) step, and execution occurs when local data arrives at the PE (e.g., without regard to other architectural activity). In certain embodiments, the PEs are dataflow operators (e.g., each PE is a single dataflow operator), e.g., a dataflow operator that only operates on input data when both (i) the input data has arrived at the dataflow operator and (ii) there is space available to store output data (e.g., otherwise no processing is occurring).

[0103] Certain embodiments herein include spatial arrays of processing elements as an energy-efficient and high-performance method for accelerating user applications. In one embodiment, the spatial array(s) are configured via a serial process in which the configuration latency is fully exposed via a global reset. Some of this may stem from the register transfer level (RTL) semantics of the array (e.g., a field programmable gate array (FPGA)). Programs executed on the array (e.g., an FPGA) may assume a fundamental reset concept, according to which each part of the design is expected to operate out of a configuration reset. Certain embodiments herein provide a dataflow array in which PEs (e.g., all PEs) conform to a flow controller microprotocol. This microprotocol may produce the effect of distributed initialization. This microprotocol may, for example, utilize a localized (e.g., non-entire column) arrangement to enable pipelined configuration and extraction mechanisms. Certain embodiments herein provide context switching in a dataflow architecture. Additionally or alternatively, certain embodiments herein provide clock gating in a spatial array (e.g., a spatial fabric), for example, in one or more (e.g., each) processing element of the spatial array.

[0104] The depicted accelerator slice 100 includes a (e.g., slice-level) configuration controller 104, which is used, for example, to configure one or more of the processing elements (PEs) and / or the network (e.g., interconnect network 105) between multiple PEs based on an input data flow graph. Additionally or alternatively, the accelerator slice 100 includes one or more (e.g., local) configuration controllers 106, 108. For example, each local configuration controller can configure a (e.g., a corresponding) subset of the processing elements and / or the networks (e.g., input and / or output to that subset of processing elements). Each local (e.g., configuration) controller can operate independently. In one embodiment, the configuration controller includes the ability to set clock gating (e.g., for certain of the PEs (or all of the PEs)) and / or the networks between the PEs. In one embodiment, the configuration controller includes the ability to set clock gating in clock gating circuits within the PEs. In one embodiment, the configuration controller includes the ability to extract, e.g., an extraction controller. In one embodiment, a configuration controller and a separate extraction controller are used. In one embodiment, the local controllers are located on a network through which these local controllers, for example, via Figure 39 The networks in the dashed boxes in FIG communicate with the upper level control hierarchy, memory and / or each other.

[0105] As shown below, the execution plan for pipelined services may include three steps:

[0106] Configuration (e.g., and setting clock gating functions), buffering, and extraction. Similarly, the control hardware (e.g., (multiple) controllers) may require knowledge and coordination of these three steps. An example control flow is as follows: Each local controller may contain a list of those controllers that are physically adjacent to it. When a local controller receives a message from each of those local controllers that precede it, a context (e.g., state) transition may begin. That local controller may then begin its current operation. When the operation is complete, the local controller may transition its context (e.g., state) and send a message to each subsequent controller. In one embodiment, local controllers follow four states: Run, Extract, Inactive, and Configure. Inactive can be achieved, for example, by starting the Configure microprotocol, which deactivates the PE, but may not be able to immediately supply configuration information, for example, thereby keeping the PE in a deactivated state.

[0107] Pipelining runtime services may include coordination between higher-level (e.g., chip-level) controllers and local controllers responsible for configuration. To reduce this communication time and improve pipeline behavior, certain embodiments herein include a microarchitecture for supporting direct forwarding of (e.g., configuration, fetch, and / or completion) commands between local controllers. This may allow (multiple) higher-level controllers to overlay a coordinated configuration and fetch graph on top of the local controllers, which may be used to dynamically construct wavefronts.

[0108] Certain embodiments herein provide paradigm-shifting levels of performance and dramatic energy efficiency improvements across a wide class of existing single-stream and parallel programs (e.g., all programs) while maintaining a familiar HPC programming model. Certain embodiments herein may be targeted at HPC where floating-point energy efficiency is paramount. Certain embodiments herein not only achieve dramatic performance improvements and energy reductions, but also pass these gains on to existing HPC programs written in mainstream HPC languages and for mainstream HPC frameworks. Certain embodiments of the architecture herein (e.g., with compilation in mind) provide several extensions in direct support for control dataflow internal representations generated by modern compilers. Certain embodiments herein relate to a CSA dataflow compiler (e.g., which can accept C, C++, and Fortran programming languages) to target the CSA architecture.

[0109] Figure 2The diagram illustrates a hardware processor 200 coupled to (e.g., connected to) memory 202 in accordance with an embodiment of the present disclosure. In one embodiment, the hardware processor 200 and memory 202 are a computing system 201. In some embodiments, one or more of the accelerators are CSAs in accordance with the present disclosure. In some embodiments, one or more of the cores in the processor are those disclosed herein. The hardware processor 200 (e.g., each of its cores) may include a hardware decoder (e.g., a decode unit) and a hardware execution unit. The hardware processor 200 may include registers. Note that the figures herein may not depict all data communication couplings (e.g., connections). Those skilled in the art will recognize that this is done to avoid obscuring certain details in the figures. Note that a bidirectional arrow in a figure may not require bidirectional communication; for example, it may indicate unidirectional communication (e.g., to or from that component or device). Any or all combinations of these communication paths may be used with certain embodiments herein. The depicted hardware processor 200 includes multiple cores (0 to N, where N may be 1 or greater) and hardware accelerators (0 to M, where M may be 1 or greater) in accordance with the present disclosure. The hardware processor 200 (e.g., its accelerator(s) and / or core(s)) may be coupled to a memory 202 (e.g., a data storage device). A hardware decoder (e.g., of a core) may receive (e.g., a single) instruction (e.g., a macroinstruction) and decode the instruction into, for example, microinstructions and / or micro-operations. A hardware execution unit (e.g., of a core) may execute the decoded instruction (e.g., a macroinstruction) to perform one or more operations.

[0110] Section 1 below discusses configurable clock gating in spatial arrays. Section 2 below discloses an embodiment of the CSA architecture. Specifically, novel embodiments of integrating memory within a dataflow execution model are disclosed. Section 3 explores the microarchitectural details of an embodiment of the CSA. In one embodiment, the primary purpose of the CSA is to support compiler-generated programs. Section 4 below examines an embodiment of a CSA compilation toolchain. In Section 5, the advantages of an embodiment of the CSA are compared to other architectures in the execution of compiled code. Finally, the performance of an embodiment of the CSA microarchitecture is discussed in Section 6, further CSA details are discussed in Section 7, and a summary is provided in Section 8.

[0111] 1. Configurable clock gating

[0112] In certain embodiments, as discussed herein, processing elements (PEs) communicate using dedicated virtual circuits, such as those formed by statically configuring (e.g., circuit-switched) communication networks. These virtual circuits can be flow-controlled and fully backpressured, such that, for example, if a source has no data or a PE's destination is full, the PE will stall. At runtime, data can flow through PEs that implement a mapped dataflow graph (e.g., a mapping algorithm). For example, data can flow from memory through a spatial array (e.g., a fabric region) of processing elements and then be returned to memory.

[0113] Such an architecture can achieve superior performance efficiencies relative to traditional multi-core processors: for example, in contrast to the expansion of the memory system, computations can be simpler and more numerous in the form of PEs than cores, and communication can be direct. In certain embodiments, a spatial array (e.g., a CSA according to any of the present disclosure) includes paths from one PE to another through the network, these paths being configurable based on programming of a dataflow graph (e.g., as discussed further below).

[0114] Figure 3 An accelerator slice 300 is illustrated according to an embodiment of the present disclosure, the accelerator slice 300 including an array of processing elements (PEs). The interconnection network is depicted as circuit-switched, statically configured communication channels. For example, a collection of channels are coupled together by switching devices (e.g., switching devices 310 in a first network and switching devices 320 in a second network). The first network and the second network can be separate or can be coupled together. For example, the switching device 310 can couple together one or more of the four data paths 312, 314, 316, 318, e.g., configured to perform operations according to a data flow graph. In one embodiment, the number of data paths is arbitrarily large. The processing elements (e.g., processing element 304) can be as disclosed herein, e.g., Figure 5 、 Figure 6 and / or Figure 26 As in [ 1 ], accelerator slice 300 includes a memory / cache hierarchy interface 302 to, for example, interface accelerator slice 300 with memory and / or cache. Data paths (e.g., 318) may extend to another slice or terminate, for example, at the edge of a slice. Processing elements (e.g., PE 304) may include input buffers (e.g., buffer 306) and / or output buffers (e.g., buffer 308).

[0115] Operations can be performed based on the availability of inputs to those operations and the state of the PE. The PE can fetch operands from input channels and write results to output channels, but can also use internal register state. Certain embodiments herein include configurable dataflow-friendly PEs. Figure 5 、 Figure 6 26 shows a detailed block diagram of such a PE (e.g., an integer PE). The PE consists of several I / O buffers, an ALU, storage registers, some instruction registers, and a scheduler. At each clock cycle, the scheduler can select instructions for execution based on the availability of input and output buffers and the state of the PE. The results of the operation can then be written to the output registers, or can be written to registers (e.g., local to the PE). The data written to the output buffers can be transferred to downstream PEs for further processing. This PE style can be extremely energy-efficient, for example, instead of reading data from a complex multi-port register file, the PE reads the data from the registers. Similarly, instructions can be stored directly in registers rather than in a virtualized instruction cache.

[0116] The instruction register can be set during a special configuration step. During this step, in addition to the inter-PE network, auxiliary control lines and status can also be used to flow the configuration across the several PEs comprising the fabric. As a result of the parallelism, certain embodiments of such a network can provide fast reconfiguration; for example, a chip-sized fabric can be configured in less than approximately 10 microseconds. Figure 5 、 Figure 6 and Figure 26 represents an example configuration of a processing element, for example, where all architectural elements are sized to their minimum. In other embodiments, each of the multiple components of a processing element is independently scaled to produce a new PE. For example, to handle more complex programs, a greater number of instructions that can be executed by a PE may be introduced. The second dimension of configurability is the functionality of the PE arithmetic logic unit (ALU). In Figure 5 、 Figure 6 and Figure 26 In [1], integer PEs are depicted as supporting addition, subtraction, and various logical operations. Other types of PEs can be created by substituting different types of functional units into PEs. For example, an integer multiplication PE may have no registers, a single operation, and a single output buffer. Certain embodiments of PEs deconstruct a fused multiply-add (FMA) into separate but tightly coupled floating-point multiplication and addition units to improve support for multiply-add-heavy workloads. PEs are discussed further below.

[0117] In one embodiment, each PE may perform clock gating to shut down and / or block the clock signal to the PE, and the PE may be re-enabled, for example, once valid (e.g., input) data is sent to the PE. Embodiments of clock gating may include clock gating one or more clocked components of the processing element (e.g., where data is latched based on a clock signal) (e.g., all clocked components, or all clocked components except for control input buffers). Embodiments of clock gating may include clock gating based on specific knowledge of the data flow graph to, for example, enable dynamic clock gating. Embodiments of clock gating may include clock gating based on the distance between a sending PE and a receiving PE (e.g., PEs on the same die or chip). In some embodiments, e.g., in contrast to embodiments in which clock gating is derived from circuit functionality and / or has no input from a compiler (e.g., or programmer) regarding how the program is implemented by hardware, information about the physical properties and flow of the dataflow graph is known at configuration time, such that configuration information (e.g., from a compiler / router tool) is used to cause hardware (e.g., clock gating circuitry) to implement clock gating.

[0118] Figure 4 4. A logical view of clock gating hardware 400 in a processor element according to an embodiment of the present disclosure is illustrated. The clock gating hardware may be a component in a processing element (PE) (e.g., any processing element disclosed herein). The depicted hardware includes: (e.g., Figure 5 data input buffer 524 or data input buffer 526 or data input buffer 624 or data input buffer 626 corresponding to) input data storage 404; operation circuit 418 (for example, which may include one or more clocked registers) (for example, which may include an ALU, such as, Figure 5 6 ) for performing one or more operations on (e.g., all) input data stored in input data store 404 and providing results of the one or more operations to output data store 406 (e.g., corresponding to data output buffer 534 or data output buffer 536 or data output buffer 634 or data output buffer 636 in FIG. 5 ). In one embodiment, the buffer may be a control input or control output buffer.

[0119] Input data storage 404 is shown as being divided into four elements (e.g., although in some embodiments it may be divided into a single element or any number of elements), and output data storage 406 is shown as being divided into the same number (e.g., four) of elements (e.g., although in some embodiments it may be divided into a single element or any number of elements). In one embodiment, input data storage 404 may be divided into a different number of elements than output data storage 406. Clock gating hardware 400 includes storage 402 for one or more configuration bits. In some embodiments, the one or more configuration bits are loaded (e.g., into storage 402 in a PE) by a configuration controller (e.g., one or more configuration controllers as discussed herein). In one embodiment, the configuration bits in configuration bit storage 402 include one bit for all clocked components, one bit for a subset of the clocked components, or one bit for each of the clocked components, e.g., multiple configuration bits with a single bit corresponding to a specific clocked (e.g., independently clock-gated) component or components (e.g., one bit for each independently clock-gated element in input data storage 404, one bit for each of operational circuitry 418 and output data storage 406, e.g., six bits total, as one example). In one embodiment, the configuration bits in configuration bit storage 402 are a single bit for all clocked components. The configuration bits can be input to clock gating circuitry 415 to clock-gate one or more of the clocked components (e.g., any combination of input data storage 404 (e.g., each element thereof), output data storage 406 (e.g., each element thereof), and operational circuitry 418). Clock gating circuitry can shut off or block a clock timing signal (e.g., a clock waveform) to a clocked component, e.g., shutting down toggling / switching (and, e.g., power consumption) caused by the clock timing signal. For example, rather than shutting down a clocked component and losing data stored therein, the clocked component can maintain its current state (e.g., maintaining any data stored therein). The clocked component can be re-enabled to shut down clock gating, e.g., to latch (e.g., a new) data value into the clocked component. In some embodiments, (e.g., each element of) input data store 404 (e.g., each element thereof) is a clocked component, such that new data values are latched (e.g., from an input of the interconnect network) during a clock cycle (e.g., on a falling and / or rising edge of a clock waveform). The depicted input data store 404 includes clock gates 408, 410, 412, 414, respectively, for each element of the input data store 404.In one embodiment, a clock gate controls whether its elements in input data store 404 are updated or clock-gated, e.g., whether controlled via a signal from clock gating circuitry 415. In one embodiment, a single clock gate controls the entire input data store 404. The depicted output data store 406 includes a shared clock gate 420 for all elements of output data store 404. In one embodiment, clock gate 420 controls whether the element(s) in output data store 406 are updated or clock-gated, e.g., whether controlled via a signal from clock gating circuitry 415. In one embodiment, output data store 406 includes a separate clock gate for each element of output data store 406. The clock gating circuitry can clock-gate one or more (e.g., all) of the clocked components based on dataflow graph inputs into a spatial array (e.g., having at least one PE) that includes clock gating hardware 400. Clock gating circuitry 415 can clock-gate operational circuitry 418 via clock gate 416. Thus, for example, based on configuration information (eg, in each cycle or in each dataflow graph) used to configure the dataflow graph entering the spatial array (eg, CSA), certain clocked components may not be utilized.

[0120] As an example, the following for loop code (1) traverses memory from address 0 to address N (e.g., 1 gigabit (1 Gb)) in increments of 8. This can be a fixed progression through the 64-bit address range required to represent 1 Gb. In one embodiment, the address range is 64 bits wide, and thus:

[0121] For address=0;address <N;address=address+8 (1)

[0122] In certain embodiments of spatial arrays, processing elements (e.g., Figure 5PE 500 in ( ) is used to generate a number from 1 to N, which is then sent (to, for example, another PE) to be added by 8 to obtain the correct address. During the execution of this for loop, the 64 bits of the address can be predictably progressive in increments of 8. One or more processing elements can therefore have their clocked components (e.g., flops) (e.g., input and / or output storage) clock-gated to avoid latching unused data or only logic low values (e.g., 0). In one embodiment, the PEs perform operations that define the number of bits used in the instructions (e.g., bytes, words, double words, etc.), and that number can be hard-coded as the maximum data width that the operation (e.g., algorithm) will use. In the for loop code (1) above and / or other operations in the data flow graph, the predictable variable use of the width of the data being used is known (e.g., before execution on the spatial array). During the configuration phase of the spatial array, configuration bits are sent to each PE (e.g., clock gating circuitry in that PE) to configure the PE for a specific function. Configuration bits can be set to inform a PE (e.g., a clock gating circuit of the PE) that data flow through the PE matches a specific (e.g., predetermined) pattern, thereby, for example, allowing (e.g., different) clock gating to be used for clocked components of a spatial array (e.g., different types of components in different PEs, e.g., clock gating the output of a first PE and clock gating the input of another PE connected to the output of the first PE) to, for example, save power. Configuration bits can be selected for specific one or more operations (e.g., an application) of a dataflow graph (e.g., unique for each of the specific one or more operations), wherein generation of the pattern is generated to enhance clock gating of the one or more PEs for the length of execution of the dataflow graph (e.g., its algorithm).

[0123] Figure 5The diagram illustrates a processing element 500 including a clock gating circuit 515 according to an embodiment of the present disclosure. The depicted clock gating circuit 515 is disposed within a scheduler 514. In other embodiments, the clock gating circuit is separate from the operation scheduler. The clock gating circuit 515 may include one or more of the depicted control lines to clock-gate components that are clocked. In one embodiment, the clock gating circuit 515 receives a configuration (e.g., in a configuration register or storage within the clock gating circuit) and performs clock gating actions based on the one or more configuration bits. In one embodiment, the (e.g., clock gating) configuration is transmitted and / or received over one of the (input) networks 502, 504, or 506 or the (output) networks 508, 510, or 512. For example, the configuration may be loaded into a configuration storage as discussed herein by a configuration controller as discussed herein. In certain embodiments, the state register 538 and / or register 520 may be clock-gated. In one embodiment, a configuration value (e.g., a single bit) is used to clock-gate all (e.g., data) input and output buffers (e.g., data input buffer 524, data input buffer 526, data output buffer 534, and data output buffer 536) for a first value (e.g., when the configuration value is set to logic one), and to enable all (e.g., data) input and output buffers (e.g., data input buffer 524, data input buffer 526, data output buffer 534, and data output buffer 536) for a second value (e.g., when the configuration value is set to logic zero). In one embodiment, control input buffer 522 is not clock-gated, such that, for example, a control input signal is received (e.g., from another PE) to re-enable one or more (e.g., not necessarily all) of the clocked components that are currently clock-gated, such as from (input) network 502, 504, or 506. In one embodiment, clock gating circuitry is used to change which components (eg, a subset of clocked components in a PE) are clock gated for each dataflow graph (eg, an execution cycle of the dataflow graph).In one embodiment, the configuration bits (e.g., stored in a configuration register and / or storage within the clock gating circuit 515) include one bit for all clocked components, a subset of the clocked components, or each of the clocked components, e.g., multiple configuration bits having a single bit of the multiple configuration bits corresponding to a particular clocked (e.g., independently clock-gated) component or components (e.g., one bit for each independently clock-gated element) (e.g., one bit for each of the control input buffer 522, the control output buffer 532, the data input buffer 524, the data input buffer 526, the data output buffer 534, the data output buffer 536, and the configuration register 519, e.g., for a total of seven bits as one example). In one embodiment, the configuration bits (e.g., stored in a configuration register and / or storage within the clock gating circuit 515) include one bit for all or a subset of clocked components (e.g., control input buffer 522, control output buffer 532, data input buffer 524, data input buffer 526, data output buffer 534, data output buffer 536, configuration register 519, or any combination of other clocked components). In one embodiment, the configuration value includes (e.g., is) a single bit (e.g., stored in a configuration register and / or storage within the clock gating circuit 515) for a subset of control input buffer 522, control output buffer 532, data input buffer 524, data input buffer 526, data output buffer 534, and data output buffer 536). In one embodiment, the configuration value includes (e.g., is) a single bit (e.g., stored in a configuration register and / or storage within the clock gating circuit 515) for a subset of control input buffer 522 and control output buffer 532. In one embodiment, the configuration value includes (e.g., is) a single bit for data input buffer 524, data input buffer 526, data output buffer 534, and a subset of data output buffer 536 (e.g., stored in a configuration register and / or storage within clock gating circuit 515).

[0124] In some embodiments, the clock gating circuit includes a state machine to perform one or more of the following operations:

[0125] Configuration register clock gating

[0126] In one embodiment, the configuration registers (e.g., Figure 5 The configuration registers 519 in the PE are only active during configuration and fetch. Therefore, they can be disabled at other times. Each PE can be equipped with a state that tracks these behaviors.

[0127] For example, a state machine in a clock gating circuit may operate as follows:

[0128] Config_clock_enable=! configured||! extracted;

[0129] The configuration register is caused to be clock-gated except when the configuration register is being configured or when the configuration register is being fetched (eg, as discussed below).

[0130] Input buffer clock gating

[0131] In one embodiment, a (e.g., data) input buffer is clock-gated when it is known that the input buffer will not be written to within a given time period (e.g., a cycle). For example, this may occur when the buffer (e.g., a queue) is full, unused, or in some cases, when no data is expected. The expectation of data may depend on the physical distance in the network (e.g., a circuit-switched network), and one or more configuration bits may be used to clock-gate or otherwise disable the buffer. Below, X represents an input buffer and i represents a specific slot in that input buffer. In another embodiment, fewer specific gates may be selected to simplify implementation.

[0132] For example, a state machine in a clock gating circuit may operate as follows:

[0133] Input_buffer_X_i_clock_enable=input_buffer_X_i_is_head&&input_X_buffer_used||(input_X_valid||!enable_input_X_network_gating)

[0134] This allows the input buffer to be clock-gated when it is full, unused, or when no input data is expected.

[0135] Output buffer clock gating

[0136] In one embodiment, a (e.g., data) output buffer is clock-gated when it is known that the output buffer will not be used in a given period of time (e.g., a cycle). This can occur when the buffer is unused, when the buffer slot is not at the head of a buffer (e.g., a queue), or when the processing element will not execute in that cycle. Y represents the output buffer, and j represents the specific slot of that buffer. In another embodiment, fewer specific gatings can be selected to simplify the implementation.

[0137] For example, a state machine in a clock gating circuit may operate as follows:

[0138] Output_buffer_Y_j_clock_enable=output_buffer_Y_j_is_head&&output_Y_buffer_used&&PE_executes

[0139] This causes the output buffer to be clock-gated when the buffer slot is not the head of the buffer (eg, queue), or when the processing element will not be executing in the cycle.

[0140] In one embodiment, the operation configuration register 519 is loaded during configuration (e.g., mapping) and specifies the particular operation (or operations) to be performed by the processing (e.g., compute) element and / or any clock gating to be performed. In one embodiment, the operation configuration register 519 is clock gated (e.g., when the data in the operation configuration register 519 is not being stored or loaded (e.g., fetched)). The activity of the register 520 may be controlled by that operation (e.g., the output of the mux 516, e.g., controlled by the scheduler 514). For example, as input data and control inputs arrive, the scheduler 514 may schedule one or more operations of the processing element 500. The control input buffer 522 is connected to the local network 502 (e.g., and the local network 502 may include, for example, Figure 24A The data path network in Figure 24B The control input buffer 522 is loaded with a value when a value arrives (e.g., the network has data bit(s) and valid bit(s). The control output buffer 532, data output buffer 534, and / or data output buffer 536 may receive the output of the processing element 500 (e.g., as controlled by the operation (output of mux 516)). The status register 538 may be loaded each time the ALU 518 executes (also controlled by the output of mux 516). The data in the control input buffer 522 and the control output buffer 532 may be a single bit. Mux 521 (e.g., operand A) and mux 523 (e.g., operand B) may serve as the source of the input. Multiple parallel input buffers (e.g., data input buffer 524 and data input buffer 526) may be utilized. Multiple parallel output buffers (e.g., data output buffer 534 and data output buffer 536) may be utilized.

[0141] For example, assuming that the operation of the processing (e.g., computing) element is (or includes) the operation referred to as a select in FIG14B , the processing element 500 is configured to select data from either the data input buffer 524 or the data input buffer 526 to be sent, for example, to the data output buffer 534 (e.g., by default) or the data output buffer 536. Thus, if selecting from the data input buffer 524, the control bit in 522 may indicate a 0, or if selecting from the data input buffer 526, the control bit in 522 may indicate a 1.

[0142] For example, assuming that the operation of the processing (e.g., computing) element is (or includes) the operation referred to as switching in Figure 14B, the processing element 500 is configured to output data from, for example, data input buffer 524 (e.g., by default) or data input buffer 526 to data output buffer 534 or data output buffer 536. Therefore, if the output is to data output buffer 534, the control bit in 522 may indicate 0, or if the output is to data output buffer 536, the control bit in 522 may indicate 1.

[0143] A plurality of networks (eg, interconnects) (eg, (input) networks 502, 504, 506 and (output) networks 508, 510, 512) may be connected to the processing element. The connections may be, for example, reference Figure 24A and Figure 24B In one embodiment, each network includes two subnetworks (or two channels on the network), for example, one for Figure 24A The data path network in Figure 24B As an example, local network 502 (e.g., established as a control interconnect) is depicted as being switched (e.g., connected) to control input buffer 522. In this embodiment, the data path (e.g., as Figure 24A522 (e.g., a network in the flow control path (e.g., a network in the flow control path (e.g., a network) may carry a control input value (e.g., one or more bits) (e.g., a control token), and the flow control path (e.g., a network) may carry a backpressure signal (e.g., a backpressure or no-backpressure token) from the control input buffer 522 to indicate, for example, to an upstream producer (e.g., a PE) that a new control input value (e.g., from a control output buffer of the upstream producer) will not be loaded into (e.g., sent to) the control input buffer 522 until the backpressure signal indicates that there is room in the control input buffer 522 for the new control input value. In one embodiment, a new control input value may not enter the control input buffer 522 until both (i) the upstream producer receives a "space available" backpressure signal from the "control input" buffer 522; and (ii) the new control input value is sent from the upstream producer, and this may stall the processing element 500 until that occurs (and space is available in the target, output buffer(s)).

[0144] Data input buffer 524 and data input buffer 526 can be implemented in a similar manner, for example, local network 504 (e.g., established as a data (as opposed to control) interconnect) is depicted as being switched (e.g., connected) to data input buffer 524. In this embodiment, the data path (e.g., as Figure 24A 524 (e.g., a network in the data input buffer 524) may carry a data input value (e.g., one or more bits) (e.g., a data flow token), and the flow control path (e.g., network) may carry a backpressure signal (e.g., a backpressure or no-backpressure token) from the data input buffer 524 to indicate, for example, to an upstream producer (e.g., a PE) that a new data input value (e.g., from a data output buffer of the upstream producer) will not be loaded into (e.g., sent to) the data input buffer 524 until the backpressure signal indicates that there is room in the data input buffer 524 for the new data input value. In one embodiment, the new data input value may not enter the data input buffer 524 until both (i) the upstream producer receives a “space available” backpressure signal from the “data input” buffer 524; and (ii) the new data input value is sent from the upstream producer, and this may stall the processing element 500 until that occurs (and space is available in the target, output buffer(s)). The control output values and / or data output values may be stalled in their respective output buffers (eg, 532 , 534 , 536 ) until the backpressure signal indicates that there is available space in the input buffer for the downstream processing element(s).

[0145] Processing element 500 may be stalled until its operands (e.g., a control input value and one or more corresponding data input values for the control input value) are received and / or until there is room in the output buffer(s) of processing element 500 for data to be produced by performing operations on those operands. During this stall in execution, clock gating circuitry 515 may stall one or more of the clocked components.

[0146] Figure 6 6. The processing element 600 is shown in accordance with an embodiment of the present disclosure. In one embodiment, operation configuration registers 619 are loaded during configuration (e.g., mapping) and specify a particular operation (or operations) to be performed by the processing (e.g., computation) element. The activity of registers 620 may be controlled by that operation (e.g., the output of mux 616, e.g., controlled by scheduler 614). For example, scheduler 614 may schedule one or more operations of processing element 600 as input data and control inputs arrive. Control input buffers 622 are connected to local network 602 (e.g., and local network 602 may include a data path network such as that shown in FIG. 24A and a control input buffer such as that shown in FIG. 24B ). Figure 24B 616). The control input buffer 522 is loaded with the value when a value arrives (e.g., the network has (a plurality of) data bits and (a plurality of) valid bits). The control output buffer 632, the data output buffer 634, and / or the data output buffer 636 may receive the output of the processing element 600 (e.g., as controlled by the operation (output of mux 616)). The status register 638 may be loaded each time the ALU 618 executes (also controlled by the output of mux 616). The data in the control input buffer 622 and the control output buffer 632 may be a single bit. The mux 621 (e.g., operand A) and the mux 623 (e.g., operand B) may serve as the source of the input.

[0147] For example, assuming that the operation of the processing (e.g., computing) element is (or includes) the operation referred to as a select in FIG14B , the processing element 600 is configured to select data from either the data input buffer 624 or the data input buffer 626 to be sent, for example, to the data output buffer 634 (e.g., by default) or the data output buffer 636. Thus, if selecting from the data input buffer 624, the control bit in 622 may indicate a 0, or if selecting from the data input buffer 626, the control bit in 622 may indicate a 1.

[0148] For example, assuming that the operation of the processing (e.g., computing) element is (or includes) the operation referred to as switching in Figure 14B, the processing element 600 is configured to output data from, for example, data input buffer 624 (e.g., by default) or data input buffer 626 to data output buffer 634 or data output buffer 636. Therefore, if the output is to data output buffer 634, the control bit in 622 may indicate 0, or if the output is to data output buffer 636, the control bit in 622 may indicate 1.

[0149] A plurality of networks (eg, interconnects) (eg, (input) networks 602, 604, 606 and (output) networks 608, 610, 612) may be connected to the processing element. The connections may be, for example, reference Figure 24A and Figure 24B In one embodiment, each network includes two subnetworks (or two channels on the network), for example, one for Figure 24A The data path network in Figure 24B As an example, local network 602 (e.g., established as a control interconnect) is depicted as being switched (e.g., connected) to control input buffer 622. In this embodiment, the data path (e.g., as Figure 24A , a network in the control input buffer 622 may carry a control input value (e.g., one or more bits) (e.g., a control token), and the flow control path (e.g., network) may carry a backpressure signal (e.g., a backpressure or no-backpressure token) from the control input buffer 622 to indicate, for example, to an upstream producer (e.g., a PE) that a new control input value (e.g., from a control output buffer of the upstream producer) will not be loaded into (e.g., sent to) the control input buffer 622 until the backpressure signal indicates that there is room in the control input buffer 622 for the new control input value. In one embodiment, a new control input value may not enter the control input buffer 622 until both (i) the upstream producer receives a “space available” backpressure signal from the “control input” buffer 622; and (ii) the new control input value is sent from the upstream producer, for example, and this may stall the processing element 600 until that occurs (and space is available in the target, output buffer(s)).

[0150] Data input buffer 624 and data input buffer 626 can be implemented in a similar manner, for example, local network 604 (e.g., established as a data (as opposed to control) interconnect) is depicted as being switched (e.g., connected) to data input buffer 624. In this embodiment, the data path (e.g., as Figure 24A, a network in the data input buffer 624) may carry a data input value (e.g., one or more bits) (e.g., a data flow token), and the flow control path (e.g., a network) may carry a backpressure signal (e.g., a backpressure or no-backpressure token) from the data input buffer 624 to indicate, for example, to an upstream producer (e.g., a PE) that a new data input value (e.g., from a data output buffer of the upstream producer) will not be loaded into (e.g., sent to) the data input buffer 624 until the backpressure signal indicates that there is room in the data input buffer 624 for the new data input value. In one embodiment, the new data input value may not enter the data input buffer 624 until both (i) the upstream producer receives a "space available" backpressure signal from the "data input" buffer 624; and (ii) the new data input value is sent from the upstream producer, for example, and this may stall the processing element 600 until that occurs (and space is available in the target, output buffer(s)). The control output values and / or data output values may be stalled in their respective output buffers (eg, 632 , 634 , 636 ) until the backpressure signal indicates that there is available space in the input buffer for the downstream processing element(s).

[0151] Processing element 600 may be stalled until its operands (e.g., a control input value and one or more corresponding data input values for the control input value) are received and / or until there is space in the output buffer(s) of processing element 600 for data to be produced by performing operations on those operands.

[0152] Certain embodiments herein allow for dynamic clock gating based on a dataflow graph to be executed.

[0153] Reference together Figure 3-Figure 6 Certain embodiments herein allow for distance-based clock gating between a producer PE and one or more of its consumer PEs. Figure 3 308, and the output may be coupled to the inputs of processing element 324 (e.g., the output data is initially stored in its input buffer and, e.g., subsequently consumed by PE 324) and processing element 334 (e.g., the output data is initially stored in its input buffer and, e.g., subsequently consumed by PE 324), or to one or more of processing element 324 and processing element 334. In one embodiment, one or more of (input) networks 502, 504, 506 are coupled to one or more of (output) networks 508, 510, 512, for example, to send data and / or re-enable signals from the outputs to the inputs.

[0154] In one embodiment, all (e.g., data, but not control) input and output PE buffers are clock gated until an element is enabled to receive or transmit. In one embodiment, data from PE 304 (e.g., a re-enable control signal) can be made to go to PE 324 at a given moment (e.g., within a clock signal) to trigger circuitry (e.g., Figure 4 The clock gating circuit 415 or Figure 5 515 in the clock gating circuit 515) to re-enable the clock-gated components (e.g., input buffers) to subsequently receive the transmitted data, for example, where the distance from PE 304 to PE 334 is too long to capture a valid signal and re-enable the clock-gated buffers (e.g., their clocks) (e.g., where PE 304 and PE 324 are adjacent to each other). For example, the transmitting PE and / or the receiving PE may include Figure 4 or Figure 5 In one embodiment, the sending PE includes one or more (e.g., all) of the components in Figure 4 、 Figure 5 or Figure 6 In one embodiment, PE 324 is from Figure 5 PE 500 of FIG304 is enabled, and the configuration received by clock gating circuit 515 causes one or more data input buffers 524, 526 to be clock gated until a re-enable control signal is received from PE 304 in control input buffer 522. In one embodiment, PE 304 is like Figure 5 or Figure 6 , and PE304 is used to send (e.g., from Figure 5 Control output buffer 532 in, or from Figure 6 ) re-enable signal of the control output buffer 632 is sent simultaneously (e.g., from Figure 5 One or more of the data output buffers 534, 536 in, or from Figure 6524) outputs data from one or more of the data output buffers 634, 636 in the transmitting PE (e.g., to be transmitted over the circuit-switched interconnect network currently coupling the PEs 304 and 324), e.g., the data is transmitted from the PE 304 with a corresponding re-enable signal and is received (and latched) into the PE 324 within the same time period (e.g., the same clock cycle) (e.g., a single clock cycle or a single subset of clock cycles). In one embodiment, all input buffers (e.g., one or more data input buffers 524, 526) are re-enabled or otherwise opened by receiving the re-enable signal via the clock gating circuit 515. In one embodiment, one or more bits configured (e.g., in the configuration register 519 and / or the clock gating circuit 515) are set in the transmitting PE to cause that PE to transmit (e.g., simultaneously) the (e.g., payload) data and the corresponding re-enable signal to the receiving PE. In one embodiment, one or more bits configured (e.g., in configuration register 519 and / or clock gating circuitry 515) are set in a receiving PE so that upon receiving a re-enable signal in control input buffer 522, that PE switches the indicated components (e.g., all data input buffers) from being clock-gated to being re-enabled (e.g., to receive data). In one embodiment, the receiving PE was previously clock-gated (e.g., the indicated components (e.g., input buffers) were clock-gated), for example, by the transmitting PE instead sending a clock timing mode value (e.g., zero) to the control input buffer of the receiving PE. In one embodiment, the configuration for both transmitting and receiving PEs in this clock-gated mode is set during the same configuration process. In some embodiments, a configuration bit is sent to the PE (e.g., by a configuration controller) to allow a programmer to use the incoming re-enable (e.g., valid) bit to enable clock gating for PE 324, but not for PE 334. In one embodiment, receiving PE 334 has a distance (e.g., a minimum distance) on the interconnect network that is, for example, longer than a threshold distance between PE 304 and PE 334 (e.g., a threshold distance for communication within the same time period (e.g., the same clock cycle) (e.g., sending data from PE 304 and receiving the data within 334)). In one embodiment, based on exceeding that threshold distance, one or more configuration bits are sent to PE 334 to disable clock gating functionality (e.g., disable clock gating circuitry in that PE). This allows a programmer to enable clock gating in a fine-grained manner. In one embodiment, PEs have clock gating functionality disabled (e.g., disable clock gating circuitry in that PE) by default. In some embodiments, the distance between PEs that are to be coupled together (e.g., as producers and consumers) is re-evaluated based on placement and routing for each new dataflow graph, for example, to save clock timing power.In one embodiment, the receiving PE uses a re-enable signal (e.g., a bit set high or low) to start the clock to capture the incoming data. In another embodiment, knowledge of a previous execution of the same dataflow graph is used to determine which components can be clock-gated for another execution of that dataflow graph (e.g., because those components were not updated or used in that cycle). Different clock gating can be used for the input and output buffers of a PE (e.g., different in cycle).

[0155] A spatial array (eg, a CSA) (eg, a PE of the spatial array), a processor, or a system may include any of the disclosures herein, eg, one or more PEs according to any of the architectures disclosed herein.

[0156] Figure 7 700 according to an embodiment of the present disclosure. The depicted flow 700 includes: 702: configuring, using a configuration controller of a processor, a plurality of processing elements of a processor based on configuration information of a dataflow graph, wherein the processor includes the plurality of processing elements and an interconnection network between the plurality of processing elements and has a dataflow graph including a plurality of nodes, the plurality of nodes being overlaid into the plurality of processing elements of the processor and the interconnection network between the plurality of processing elements of the processor, and each node being represented as a dataflow operator in the interconnection network and the plurality of processing elements; 704: clock-gating, using the configuration controller of the processor, at least one clocked component of the processing elements based on the configuration information of the dataflow graph; and 706: executing, using the interconnection network and the plurality of processing elements, operations of the dataflow graph when an incoming operand set arrives at the plurality of processing elements.

[0157] Figure 8800 according to an embodiment of the present disclosure is illustrated. The depicted process 800 includes: 802: configuring, using a configuration controller of a processor coupled to a first processing element and a second processing element of a plurality of processing elements, the second processing element to clock-gate at least one clocked component of the second processing element, the first processing element having an output coupled to an input of the second processing element, wherein the processor includes the plurality of processing elements and an interconnection network between the plurality of processing elements and has a dataflow graph including a plurality of nodes, the plurality of nodes being overlaid into the plurality of processing elements of the processor and the interconnection network between the plurality of processing elements of the processor, and each node being represented as a dataflow operator in the interconnection network and the plurality of processing elements; 804: configuring, using the configuration controller, the first processing element to clock-gate at least one clocked component of the second processing element, the first processing element having an output coupled to an input of the second processing element, wherein the processor includes the plurality of processing elements and an interconnection network between the plurality of processing elements and has a dataflow graph including a plurality of nodes, the plurality of nodes being overlaid into the plurality of processing elements of the processor and the interconnection network between the plurality of processing elements of the processor, and each node being represented as a dataflow operator in the interconnection network and the plurality of processing elements; The elements are configured to: when data is to be sent from the first processing element to the second processing element, send a re-enable signal on the interconnect network to the second processing element so as to re-enable at least one clocked component of the second processing element; 806: clock gate the at least one clocked component of the second processing element using a configuration controller of the processor; 808: when data is to be sent from the first processing element to the second processing element, send a re-enable signal on the interconnect network to the second processing element so as to re-enable at least one clocked component of the second processing element; and 810: when an incoming operand set including the data arrives at the second processing element, perform operations of the dataflow graph using the second processing element.

[0158] Figure 9 Context switching in a spatial array 901 of processing elements of a processor 900 according to an embodiment of the present disclosure is illustrated. Spatial array 901 is depicted as an accelerator coupled to a processor core 902 and / or a vector processing unit (VPU) 904, e.g., an accelerator for performing tasks other than the core and / or VPU. The depicted processor 900 includes a cache home agent 906, e.g., for acting as a local coherence and cache controller (e.g., a cache agent) and / or also acting as a global coherence and memory controller interface (e.g., a home agent).

[0159] The spatial array 901 may be one of the arrays discussed herein, for example, Figures 10A-10D Or any of the spatial arrays in Figures 11A-11J. Specifically, Figure 9 Provides a conceptual view of context switching in a spatial array. (E.g., centralized) control arranges wavefronts radiating outward from, for example, a cache (e.g., L2 cache 908). New configuration 910 regions and old configuration 916 regions can be active simultaneously, e.g., processing data. Figure 9The diagram illustrates the concept that through coordination of a spatial array 901 (eg, a fabric), the spatial array can implement a pipelined, wavefront-oriented runtime service. Figure 9 , a coordinated implementation of context switching is shown. Here, a fetch 914 region (e.g., which is saving the state of a first spatial context) is active at the same time as a configuration 912 region (e.g., a load of a second (new or prior) different context into the spatial array). In this embodiment, both the new configuration and the old configuration can be active at the same time in order to, for example, limit the degradation of spatial array throughput caused by runtime operations. In one embodiment, in order to achieve pipelined configuration and fetch, the main property to be guaranteed is that the new configuration 912 (e.g., the configuration region) and the old fetch 914 (e.g., the fetch region) do not communicate. To achieve this guarantee, certain embodiments herein take advantage of architectural properties of the spatial array with respect to communication, for example, the PEs follow a full / empty (e.g., backpressure) micro-protocol that can be manipulated in the micro-architecture to prevent communication. In Figures 10A-10D An example of this mechanism is shown in Certain embodiments herein provide a coordination mechanism (eg, a controller) of a spatial array to ensure that new regions and old regions do not communicate, for example, to ensure program correctness.

[0160] Figures 10A-10DThe diagram illustrates on-the-fly configuration of context switches (e.g., configuration and fetches) for a spatial array 1000 of processing elements 1002A-1002O, according to an embodiment of the present disclosure. In one embodiment, the spatial array is an accelerator for a processor (e.g., having a core). Once configured, the PEs can execute subject to data flow constraints. However, channels involving unconfigured PEs can be disabled by the microarchitecture, for example, to prevent any undefined operations from occurring. These properties allow embodiments herein to initialize and execute in a distributed manner, e.g., without any centralized execution control. From an unconfigured state, configuration can occur entirely in parallel (e.g., perhaps in as little as 200 nanoseconds). However, due to the distributed initialization of embodiments of the spatial array (e.g., CSA), PEs may become active, for example, sending requests to memory long before the entire structure is configured. Fetches can proceed in much the same manner as configuration. The local network (e.g., 1004 or 1006) can be observed (e.g., its circuitry switched) to fetch data from one target at a time, and state bits used to implement distributed coordination are fetched. A spatial array (e.g., CSA) can arrange for extraction to be non-destructive, i.e., upon completion of the extraction, each extractable target has been returned to its starting state. In this implementation, all states in the target can be propagated to exit registers connected to the local network in a scan-like manner. However, in-place extraction can be implemented by introducing new paths at the register transfer level (RTL) or using existing wires to provide the same functionality with lower overhead. Similar configurations, hierarchical extractions are implemented in parallel.

[0161] exist Figure 10A In the embodiment, a plurality of local (e.g., configuration) controllers 1008A-1008E are included, for example, connected to a network (e.g., local network 1004 or 1006). In one embodiment, the local (e.g., configuration) controllers are used to control configuration and / or extraction. The local controllers may be further controlled by higher level controllers (e.g., Figure 11002L). Local (e.g., configuration) controller 1008E may manage (e.g., cause) the configuration and / or extraction of processing elements 1002M-1002O. Although each subset of managed processing elements (e.g., 1002A-1002C) is shown on the same row as its local (e.g., configuration) controller (e.g., 1008A), other orientations are possible. Although three processing elements (e.g., 1002A-1002C) are shown with a single local (e.g., configuration) controller (e.g., 1008A), a (e.g., local) controller may be used for one processing element or any number of processing elements. The processing elements may be as disclosed herein, for example, Figure 26 The processing element may be configured, for example, by writing to a configuration register. The local (eg, configuration) controller may be a configuration controller (eg, Figure 41 ) and / or extracting the controller (e.g., as Figure 50 ). The state (e.g., configuration information) of a processing element may include data in any (input or output) queues or buffers, backpressure data (e.g., signals), operational configurations, and / or any other data. The state may include information stored in any (multiple) registers of the processing element. The state may include where data for a PE is sourced from (e.g., from one or more PEs or memories), and where (output) data is sent (e.g., to which PE(s) or memories). The state may include (e.g., switch settings of) a data path network and / or a flow control (e.g., backpressure) path network, see e.g. Figures 24A-24B The status may include data related to the memory access, including, for example, the address and the data returned.

[0162] In the depicted embodiment, each processing element may be in an indicated state, e.g., configured with a particular configuration, actively configured (e.g., a configuration loaded and enabled to be executed), unconfigured, actively not configured, or in an extracted configuration (e.g., state). The configurations in a spatial array (e.g., processing element(s)) may be used for the same dataflow graph, e.g., where one or more processing elements are not reconfigured. For example, for each configuration, the one or more processing elements may perform the same operation, but with different input source(s) and / or output destination(s) (e.g., values). The configurations may be used for different dataflow graphs, e.g., where one or more processing elements are reconfigured to perform different operations. A configuration may be where (e.g., a subset of) the processing elements are configured (e.g., programmed) such that each node of the dataflow graph is represented in the spatial array (e.g., having the processing elements as dataflow operators).

[0163] exist Figure 10A In , the local controller 1008A may have previously received a command (e.g., from a higher-level controller) to apply a new configuration to the subset of processing elements 1002A-1002C to which the local controller 1008A is coupled, and Figure 10A In the example embodiment, the new configuration has already been fully applied (e.g., loaded). The local controller 1008B may have received a command (e.g., from a higher-level controller) to apply the new configuration to the subset of processing elements 1002D-1002F to which the local controller 1008B is coupled. Figure 10A In FIG. 1 , processing element 1002D is unconfigured, processing element 1002E is actively being configured (e.g., configured with a new configuration), and processing element 1002F has already been configured with a new configuration. Local controller 1008B may send configuration information (e.g., data) 1010 (e.g., including state, etc.) to processing element 1002E to cause configuration of processing element 1002E accordingly. Line 1020 schematically illustrates configuration control being sent from configuration storage 1022E to a network (e.g., 1004 and / or 1006) to implement the required configuration data path(s). Configuration control signal data on line 1020 may originate from local controller 1008B.

[0164] Configuration stores 1022A-1022O schematically illustrate configuration and extraction control data (signals) for setting up circuit switching in a network (e.g., as opposed to the configuration and extraction data payloads themselves). In one embodiment, the configuration stores are registers in a local (e.g., configuration and / or extraction) controller. Lines 1018, 1020, 1024 schematically illustrate configuration control sent from configuration stores 1022D-1022F to a network (e.g., 1004 and / or 1006) to implement the desired configuration data path(s). A line from processing element 1002B to processing element 1002F may represent an active channel in network 1004 and / or 1006, which is set (e.g., a switch of a circuit-switched network is set to enable that data path) to couple an output of (e.g., a buffer of) processing element 1002B to an input of (e.g., a buffer of) processing element 1002F. This channel may be set according to the new configuration. Dashed lines 1012 , 1014 , 1016 may indicate inactive channels of the network as being active, for example, when both the input and output processing element(s) are configured accordingly.

[0165] Local (e.g., configuration) controller 1008D is depicted as sending and / or receiving unconfiguration (e.g., extraction) data 1030 (e.g., including state, etc.), and processing element 1002J accordingly causes unconfiguration (e.g., extraction of state) of processing element 1002J. Line 1026 schematically illustrates unconfiguration (e.g., extraction) control sent from configuration (e.g., unconfiguration) storage 1022J to the network (e.g., 1004 and / or 1006) to implement the desired unconfiguration data path(s). The unconfiguration (e.g., extraction) control signal data on line 1026 may originate from local controller 1008D.

[0166] The local controllers 1008A-1008E may each include storage 1028A-1028E (e.g., register(s)) to store information describing coordination between the local controllers, such as what operation each controller is performing (e.g., active with new configuration, active with old configuration, unconfigured, unconfigured (fetched), or configured).

[0167] Go to Figure 10B , for example, previously in Figure 10AThe processing element 1002J that is not configured in the context is now unconfigured, and the state data for that context is now saved in, for example, any storage discussed herein. Subsequently, the processing element 1002J is unconfigured, for example, it may assert backpressure to any upstream processing elements, etc. so as not to allow data to be input to the processing element 1002J. In one embodiment, when saving state data for the processing element 1002J (e.g., as managed by the local controller 1008D), the configuration (e.g., unconfigured) storage 1022J may be disabled, for example, by clearing the data in the configuration (e.g., unconfigured) storage 1022J. Figure 10A A path for sending and / or receiving unconfigured (eg, extracted) data 1030 .

[0168] Go to Figure 10C Upon completion of the unconfiguration (e.g., unconfiguration operation) of processing element 1002J (as well as processing elements 1002K and 1002L), local controller 1009D may send (completion) messages 1032, 1034 to neighboring controllers (to controllers 1008C and 1008E, respectively). In one embodiment, such messages (e.g., completion of extraction) may cause one or more of the set of neighboring processing elements to proceed to their next batch of operations.

[0169] Now go to Figure 10D , the receipt of the done (eg, not configured) message 1032 by the local controller 1008C may trigger the local controller 1008C to begin its next operation, e.g., Figure 10D 1006) to configure processing element 1002I, for example, using configuration information (e.g., data) 1040 sent to processing element 1002I along a path (e.g., in network 1004 and / or 1006) to accordingly configure processing element 1002I (and, for example, processing element 1002G and processing element 1002H). In one embodiment, configuration (e.g., fetch) control is sent from configuration storage 1022G-1022I to the network (e.g., 1004 and / or 1006) to implement the desired configuration data path. Configuration control signal 1042 data may originate from local controller 1008C.

[0170] Additionally or alternatively, receipt of the Done (eg, Unconfigured) message 1034 by the local controller 1008E may trigger the local controller 1008E to begin its next operation, e.g., Figure 10DIndicated in FIG. 1 is a method for initiating unconfiguration (e.g., extraction) of processing element 1002O, for example, using unconfiguration (e.g., extraction) data 1050 (e.g., including state, etc.) sent and / or received by processing element 1002O along a path (e.g., in network 1004 and / or 1006), thereby correspondingly causing unconfiguration (e.g., extraction) of processing element 1002O (and, for example, processing element 1002M and processing element 1002N). In one embodiment, unconfiguration (e.g., extraction) control is sent from configuration (e.g., unconfiguration) storage 1022O to network (e.g., 1004 and / or 1006) to implement the desired unconfiguration data path(s). Unconfiguration (e.g., extraction) control signal 1052 may originate from local controller 1008D. Note that the term "old" when referring to these figures may refer to an existing configuration. Note that the term "new" as used in reference to these figures may refer to a previous configuration, but which is replacing a configuration currently in a PE, or which is being configured into an unconfigured PE.

[0171] In one embodiment, a network (e.g., a circuit-switched network) includes multiple channels (e.g., Figures 10A-10D ). Channel semantics (e.g., Figures 10A-10D The dotted and solid lines (overlaid into the network in the figure) can enable natural activation of the pipeline, for example, where unconfigured PEs clamp control values and / or when PEs are configured, they begin computation. Spatial arrays can thus become active in very few cycles (e.g., on the order of tens of nanoseconds) (e.g., in contrast to the cycle-level semantics of FPGAs, where backpressure is not implicit and the entire design (FPGA) must be configured, similar to "coming out of reset").

[0172] Although the above Figures 10A-10D The discussion of

[0045] refers to multiple local controllers, but in another embodiment, a single (eg, configured) controller may implement the above scheme.

[0173] Figures 10A-10D The communication micro-protocol during extraction is further illustrated, for example, via manipulation of full / empty (e.g., backpressure) bits in the communication micro-protocol to prevent fine-grained data flow during runtime service events.

[0174] Figures 11A-11J 1 illustrates a phased extraction of a (e.g., first) context for a spatial array 1100 of processing elements 1102A-1102P configured to execute a dataflow graph, according to an embodiment of the present disclosure. Figures 11A-11J, a single dataflow graph is depicted as being overlaid, for example, into a spatial array 1100 of processing elements 1102A-1102P (and, for example, into a plurality of (e.g., interconnect) networks therebetween), such that each node of the dataflow graph is represented as a dataflow operator in the spatial array of processing elements. In one embodiment, one or more of the processing elements in the spatial array of processing elements are operable to communicate with each other via a memory interface (e.g., Figure 20C 2002 in the memory interface 2002) to access the memory. In one embodiment, the pick node of the dataflow graph thus corresponds to the pick operator 1104 (e.g., represented by the pick operator 1104), the switch node of the dataflow graph thus corresponds to the switch operator 1106 (e.g., represented by the switch operator 1106), the multiplier node of the dataflow graph thus corresponds to the multiplier operator 1108 (e.g., represented by the multiplier operator 1108), the "==" node of the dataflow graph thus corresponds to the equality check operator 1110 (e.g., represented by the equality check operator 1110), and the "<" node of the dataflow graph corresponds to the "<" operator 1112 (e.g., represented by the "<" operator 1112). Another processing element and / or the flow control path network can provide a control signal (e.g., a control token) to the pick operator 1104A and the switch operator 1106 to perform the operation. In one embodiment, the spatial array of processing elements 1100 is configured before execution begins (to execute the data flow graph). In one embodiment, the compiler performs the following operations: Figure 11A In one embodiment, the input of a dataflow graph node into the spatial array of processing elements logically embeds the dataflow graph into the array of processing elements (e.g., as discussed further below) such that the input / output paths are configured to produce the desired results. For example, see below for Figures 20A-20C Discussion.

[0175] exist Figures 11A-11J, spatial array 1100 is depicted as having local (e.g., configuration and / or extraction) controllers 1108A-1108D. The (e.g., configuration) controllers may manage the configuration and / or extraction of a subset of processing elements. Local (e.g., configuration) controller 1108A may manage (e.g., cause) the configuration and / or extraction of processing elements 1102A-1102D. Local (e.g., configuration) controller 1108B may manage (e.g., cause) the configuration and / or extraction of processing elements 1102E-1102H. Local (e.g., configuration) controller 1108C may manage (e.g., cause) the configuration and / or extraction of processing elements 1102I-1102L. Local (e.g., configuration) controller 1108D may manage (e.g., cause) the configuration and / or extraction of processing elements 1102M-1102P. The local controllers may be further controlled by a higher-level controller (e.g., controller 1114).

[0176] exist Figure 11A In the embodiment of the present invention, the local controllers receive their signals from the (slice) controller 1114, for example, to obtain a configuration (e.g., according to a first context). These signals (e.g., commands) control the behavior of the local controllers to configure a subset of their corresponding processing elements. For example, when switching from a first context to a different second context, it may be desirable to retrieve the current operating state (e.g., operands, etc.). Constants 1 and 2 in processing elements 1102G and 1102K, respectively, may be used as inputs, but the present disclosure is not limited thereto.

[0177] exist Figure 11B In the example, when input data and / or output data space is available (e.g., no back pressure is asserted), the local (e.g., configuration and / or extraction) controllers 1108A-1108D have completed their configuration (e.g., according to the first context) and are now operational. Next, assume that an extraction is currently in Figure 11B The extraction may be performed in stages, for example, from top to bottom (in a program / operation flow).

[0178] exist Figure 11C In

[0065] , extraction of context for (e.g., configured and / or unconfigured) processing elements 1102A-1102D is initiated, for example, by local controller 1108A. For example, any input data, output data, and corresponding backpressure signals are extracted (and saved) from configured processing elements 1102A and 1102B. The remaining processing elements (1102E-1102P, e.g., only 1102G, 1102K, and 1102O) may continue to operate, for example, assuming they have input data and / or output data space (e.g., no backpressure is asserted). In

[0066] , the context for processing elements 1102A-1102D is extracted, for example, by local controller 1108A. For example, any input data, output data, and corresponding backpressure signals are extracted (and saved) from configured processing elements 1102A and 1102B. The remaining processing elements (1102E-1102P, e.g., only 1102G, 1102K, and 1102O) may continue to operate, for example, assuming they have input data and / or output data space (e.g., no backpressure is asserted). Figure 11C, processing element 1102G has executed as pick operator 1104 and has output data 1116. The open circles in these figures may represent input data values and / or output data space according to the first context (e.g., no backpressure is asserted), and the filled circles in these figures may represent input data values and / or output data space according to the first context (e.g., no backpressure is asserted).

[0179] exist Figure 11D , the fetching of context for processing elements 1102A- 1102D is still occurring, and output data 1116 has been consumed by processing element 1102K, which is executing as multiplier operator 1108 and has output data 1118 .

[0180] exist Figure 11E , context extraction for processing elements 1102A-1102D is complete, and output data 1118 has been consumed by processing element 1102O, which is executing as switch operator 1106 and has output data 1118. Context extraction for the next subset of processing elements 1102E-1102H (e.g., configured and / or unconfigured) is initiated, for example, by local controller 1108B. For example, any input data, output data, and corresponding backpressure signals are extracted (and saved) from configured processing element 1102G. The remaining processing elements (1102I-1102P, e.g., only 1102I and 1102O) may continue to operate, e.g., assuming they have input data and / or output data space (e.g., no backpressure is asserted). In some embodiments, data may not be moved through the extraction region, e.g., in this case, processing elements 1102E-1102H. Thus, in embodiments where the fetch region will not accept new data (e.g., that region of processing elements asserts its backpressure signal), output data 1120 is stalled (e.g., it would otherwise go to processing element 1102G as pick operator 1104). In one embodiment, new data may be generated for the region above the fetch region (e.g., in program flow order), for example, by processing element 1102A as a "check for equality" operator 1110 and / or by processing element 1102B as a "less than" operator 1112.

[0181] exist Figure 11F, the context extraction for processing elements 1102E-1102H is complete. For example, the context for the next subset of processing elements (e.g., configured and / or unconfigured) 1102I-1102L is started by local controller 1108C. For example, any input data, output data, and corresponding backpressure signals are extracted (and saved) from the configured processing element 1102K. The remaining processing elements (1102M-1102P, e.g., only 1102O) may continue to operate, e.g., assuming they have input data and / or output data space (e.g., no backpressure is asserted). In some embodiments, data may not move through the extraction region, e.g., in this extraction region, processing elements 1102I-1102L. Thus, in embodiments where the extract region (and (logically) any regions above) will not accept new data (e.g., that region of processing elements asserts its backpressure signal to prevent data from crossing that region), output data 1120 is stalled (e.g., it would otherwise go to processing element 1102G as pick operator 1104). In one embodiment, new data may be generated for regions above the extract region (e.g., in program flow order), e.g., where output data 1122 from processing element 1102A as “check for equality” operator 1110 is stalled to processing element 1102O (e.g., crossing the extract region) and / or where output data 1124 from processing element 1102B as “less than” operator 1112 is not stalled to processing element 1102G (e.g., not crossing the (in-process) extract region).

[0182] exist Figure 11GContext extraction for processing elements 1102I-1102L is complete. For example, context extraction for the next subset of processing elements 1102M-1102P (e.g., configured and / or unconfigured) is initiated by local controller 1108D. For example, any input data, output data (stopped output data 1120), and corresponding backpressure signals are extracted (and saved) from configured processing element 1102O. At this point, no further downstream processing elements are having their contexts saved (e.g., this is the end of that portion of the dataflow graph), so the extraction is nearly complete. The aforementioned processing elements (1102A-1102L, e.g., only 1102A, 1102B, 1102G, and 1102K) may continue to operate, e.g., assuming they have input data and / or output data space (e.g., no backpressure is asserted). In some embodiments, data may not move through the extraction region, e.g., in this case, processing elements 1102M-1102P. Thus, in embodiments where the fetch region (and (logically) any regions above) will not accept new data (e.g., that region of processing elements asserts its backpressure signal to prevent data from crossing that region), output data from processing element 1102G is stalled (e.g., processing element 1102G is waiting for data from processing element 1102O to proceed). In one embodiment, new data may be generated for regions above the fetch region (e.g., in program flow order), e.g., where output data 1122 from processing element 1102G acting as pick operator 1104 is stalled along with output data from processing element 1102O acting as pick operator 1106, but is not stalled across the (in-process) fetch region.

[0183] exist Figure 11H In , for example, when input data and / or output data space is available (e.g., no back pressure is asserted), the local (e.g., configuration and / or extraction) controllers 1108A-1108D have completed their extraction of the first context (and configuration of the second context) and can now operate. For example, after extraction, the space array 1100 (e.g., the data flow graph loaded therein) can be reused for a different set of (input) operands, e.g., as the second context. Figure 11H In the example, data output 1126 (e.g., from a previous operation in the next pair) may be available (e.g., configured into the spatial array 1100) and thus flow upward to the processing element 1102G as the pick operator 1104, as shown in FIG. Figure 11I As shown in .

[0184] exist Figure 11J, processing element 1102G, acting as pick operator 1104, may have all its operands available and no backpressure signals, and thus may generate output data 1128. In one embodiment, to access a local controller, a higher-level controller may have access to the address of the local controller that the higher-level controller is managing. The local controller may have access to the address of the processing element it is managing (e.g., may calculate the address of the processing element it is managing). The address of the PE may be sent in by a higher-level (e.g., regional) controller (e.g., which understands the memory format of the spatial array context).

[0185] Figure 12A An extracted state 1200 is illustrated according to an embodiment of the present disclosure. Figure 12B Illustrated is state 1200 at the start of extraction according to an embodiment of the present disclosure. Figure 12A and Figure 12B When the extraction is staged, the extraction can take Figure 12A (e.g., where the input and output data from that saved state are indicated by open circles), but that state in that exact form never existed at any point in time. Figure 12A The staged extraction output in captures a legal view (e.g., state) of a spatial array (e.g., a data flow graph represented in the spatial array), while Figure 12B The diagram illustrates the state at the start of a phased extraction comprising multiple phases (eg, multiple phases of a single portion of a data flow graph). Figure 12A The state in may be several steps prior to the initial state (when extraction first begins), for example, because several more operations have been performed, for example, consuming input data 1202 and 1204, and producing output data 1206.

[0186] Figure 13The diagram illustrates a state machine 1300 for a (e.g., configuration) controller according to an embodiment of the present disclosure. The controller may include, for example, a hardware state machine in each local (e.g., configuration) controller. The state machine may be a Mealy machine or a Moore machine. State 1302 may be a state in which a (e.g., higher-level) controller sends a control message, for example, to initiate a context switch for one or more previously configured PEs of another (e.g., local) controller. In state 1304, (e.g., when input data operands and output space are available), the previously configured one or more PEs may operate according to a first (e.g., old) context (e.g., of a dataflow graph), for example, until all data is consumed. State 1306 may then be entered, for example, based on a requested context switch, and the context (e.g., old configuration) of the previously configured one or more PEs may begin to be extracted. After the extraction is complete (e.g., after state 1306 or after entering state 1312), the local (e.g., configuration) controller may send a message to a neighboring controller to form a pipelined context switch. In one embodiment, a local (e.g., configuration) controller sends a message to one or more subsequent local (e.g., configuration) controllers (e.g., two, three, four, five controllers, etc.) indicating that they may begin extracting a first context for their subset of processing elements. Additionally or alternatively, the local (e.g., configuration) controller sends a message to one or more preceding local (e.g., configuration) controllers (e.g., two, three, four, five controllers, etc.) indicating that they may begin configuring (e.g., loading) a second (e.g., new) context into their subset of processing elements. In state 1312, the most recently extracted processing element may then remain in an unconfigured state. In state 1314, upon receiving a message from one or more subsequent local (e.g., configuration) controllers (e.g., two, three, four, five controllers, etc.) indicating that extraction of the first context by these controllers is complete, those unconfigured processing elements may begin configuring (e.g., loading) the second context into their subset of processing elements. In state 1316, for example, when a (e.g., higher-level) controller sends a control message to initiate another context switch, that new state (e.g., configuration) may be executed on the configured processing element(s), and then return to state 1304. In one embodiment, the context may be from different dataflow graph(s) or (e.g., different portions of) the same dataflow graph.

[0187] Figure 14A1402 for processing elements in accordance with an embodiment of the present disclosure. The spatial array 1402 is schematically illustrated to show an initial configuration message that forms a spatial array-wide (e.g., fabric-wide) barrier (e.g., extraction region) that is swept across the entire spatial array (e.g., fabric) under the control and direction of a higher-level (e.g., chip-level) controller 1404, for example. The controller 1404 may arrange for local controllers to coordinate the pipeline. The ACI network and RAF may be discussed below. Although Figure 14A A phase ordering is illustrated here, but other topologies are possible.

[0188] Figure 14B 14. The extraction of context for a spatial array 1402 of processing elements according to an embodiment of the present disclosure is illustrated. The spatial array 1402 is schematically illustrated to show a spatial array-wide (e.g., fabric-wide) barrier (e.g., an extraction region in "waiting for configuration") that is swept across the entire spatial array (e.g., fabric) under the control and direction of, for example, a higher-level (e.g., chip-level) controller 1404. The controller 1404 may arrange for local controllers to coordinate the pipeline. In one embodiment, Figure 14B The states of the spatial array 1402 in FIG. 1 correspond to those in the preceding figures. The ACI network and RAF can be discussed below. Figure 14B A phase ordering is illustrated here, but other topologies are possible.

[0189] Figure 15 Illustrate a staged extraction of context for a spatial array of processing elements including (eg, mezzanine or global) networks therebetween, according to an embodiment of the present disclosure.The network may be any network discussed herein. Figure 15 Illustration of a block diagram including a PULL region 1502, a POST region 1504, and a PRIOR region 1506, including network messages traversing these regions.

[0190] Messages in the network may cross phase boundaries (e.g., fetch zones). In one embodiment, hardware (e.g., a network controller as discussed herein) records the state of the network token. In one embodiment, the network controller may inject a POST (e.g., fetch zone) transition message on each channel and not release a PRIOR (e.g., fetch zone) message until a POST message is received. When a POST message is received for all active channels, the network controller may record the state of the network controller. In one embodiment, early consumption of messages is legal, but the state is retained. Matching messages (e.g., messages from the same context) may be forwarded, e.g., a PRIOR may consume a PRIOR message, and a POST may consume a POST message. A mismatch may mean that the data is from the wrong epoch and needs to be waited for. The PRIOR message may be facilitated. The PRIOR endpoint may wait for the fetch. The PRIOR message may wait for the POST transition.

[0191] Figure 16 16. The diagram illustrates a phased fetch of context for a spatial array 1600 for a processing element 1602 in accordance with an embodiment of the present disclosure, the phased fetch including memory access. In some embodiments, communication through memory may be utilized. In the depicted embodiment, only processing elements in the POST region (e.g., after the fetch region) touch memory (e.g., cache), for example, by first fetching near memory (e.g., near request address file (RAF) circuitry, which may be as described herein) to the PEs to quickly re-enable them. In one embodiment, the spatial array (e.g., a controller) is configured to treat the PRIOR region of the PE and the POST region of the PE as conflicting lane groups, rolling back to a snapshot of the PRIOR upon detection of a conflict between the two regions. In one embodiment, the spatial array (e.g., a controller) is configured to keep old and new values alive in the cache, for example, allowing each epoch to access the value. In one embodiment, for processors physically close to the cache (e.g., Figure 16 The fetching of the processing element from the L2 cache in the memory can help to quickly restore the access, for example, where Figure 16 In an embodiment, extraction continues outward (eg, radiates). Spatial array 1600 can be coupled to vector processing unit 1604 and / or processor core 1610 .

[0192] Handling Memory Operations: It may be the case that during the fetch phase, a context has pending memory operations. In this case, the cache interface (e.g., CHA) will reserve resources (e.g., slots in the reorder buffer) that have been allocated to the pending requests until those requests are completed by the memory system. At that point, these requests can be written to the in-memory representation of the evicted process, and the allocated resources are returned to the memory interface for use by the newly configured context. In one embodiment, while a request is pending, the associated resources are not used by the new context.

[0193] Figure 17A 17. Illustrated is an extraction of context for a spatial array 1700A of processing elements in accordance with an embodiment of the present disclosure. In one embodiment, a second (e.g., next) extraction region may be maintained (e.g., where no data may traverse the region) to prevent inter-epoch communication in addition to, for example, the current extraction region. In one embodiment, a phased extraction may be implemented without requiring modifications to the processing element(s), for example, when the local network is disabled during the extraction (e.g., via a backpressure signal). This may provide an insurmountable barrier (e.g., due to the backpressure signal being active). The dual tiering of extraction may ensure that PEs in the first tier do not interfere with the PRIOR region as they transition to POST, e.g., as Figure 17A In some embodiments, detailed high-level orchestration and coordination is used for, for example, regional and / or global abstraction controller functionality. POST can refer to a newly configured region configured with a new configuration (e.g., context). PRIOR can refer to a previously configured region configured with a previous configuration (e.g., context).

[0194] Figure 17B Illustrated is the extraction of context for a spatial array 1700B of processing elements according to an embodiment of the present disclosure. Figure 17B Describes the detailed time slices of pipelined context switches. Figure 17B There are five regions depicted in : the execution context, the current fetch region, the current configuration region, and the buffer region in between.

[0195] Figure 18A flowchart 1800 is illustrated according to an embodiment of the present disclosure. The depicted flowchart 1800 includes: 1802: providing a processor, the processor including a plurality of processing elements and an interconnection network between the plurality of processing elements and having a dataflow graph, the dataflow graph including a plurality of nodes overlaid onto the plurality of processing elements of the processor and the interconnection network between the plurality of processing elements of the processor, and each node being represented as a dataflow operator in the interconnection network and the plurality of processing elements; 1804: executing operations of the dataflow graph using the interconnection network and the plurality of processing elements when an incoming operand set (e.g., input data and / or output data space (e.g., no backpressure from a destination on the output is asserted)) arrives at the plurality of processing elements ; 1806: configuring, using a configuration controller of a processor, a first subset and a different second subset of multiple processing elements according to configuration information for a first context of a data flow graph; and 1808: after (e.g., all) pending operations of the first context are completed in the first subset, for the requested context switch, configuring, using a configuration controller of the processor, the first subset of multiple processing elements according to configuration information for a second context of the data flow graph, and preventing second context data from flowing from an output of the first subset to an input of a different second subset until pending operations of the first context are completed in the different second subset.

[0196] Figure 19 A flowchart according to an embodiment of the present disclosure is illustrated. The depicted process 1900 includes: 1902: providing a processor, the processor comprising a plurality of processing elements and an interconnection network between the plurality of processing elements and having a dataflow graph, the dataflow graph comprising a plurality of nodes overlaid into the plurality of processing elements of the processor and the interconnection network between the plurality of processing elements of the processor, with each node represented as a dataflow operator in the interconnection network and the plurality of processing elements; 1904: performing operations of the dataflow graph using the interconnection network and the plurality of processing elements when an incoming operand set arrives at the plurality of processing elements; 1906: configuring, using a first configuration controller and a second configuration controller of the processor, a first subset of the plurality of processing elements and a different second subset according to corresponding configuration information for a first context of the dataflow graph; and 1908: after (e.g., all) pending operations of a first context are completed in the first subset, configuring, using the first configuration controller of the processor, the first subset of the plurality of processing elements according to configuration information for a second context of the dataflow graph for a requested context switch, and preventing the second context dataflow from an output of the first subset to an input of the different second subset until pending operations of the first context are completed in the different second subset.

[0197] 2.CSA Architecture

[0198] Certain embodiments of CSAs aim to execute programs (e.g., programs produced by a compiler) quickly and efficiently. Certain embodiments of the CSA architecture provide programming abstractions that support the needs of compiler technology and programming paradigms. Embodiments of CSAs implement dataflow graphs, e.g., a representation of a program that is much like the compiler's own internal representation (IR) of a compiled program. In this model, a program is represented as a dataflow graph consisting of nodes (e.g., vertices) drawn from a set of architecturally defined dataflow operators (e.g., covering both computational and control operations) and edges representing the transfer of data between the dataflow operators. Execution can proceed by injecting dataflow tokens (e.g., as data values or dataflow tokens representing data values) into the dataflow graph. Tokens can flow between them and can be transformed at each node (e.g., vertex) to form, for example, a complete computation. In Figure 20A - Figure 20C A sample data flow graph and its deviation from the high-level source code is shown in Figure 22 An example of the execution of a data flow graph is shown.

[0199] An embodiment of a CSA configures for data flow graph execution by providing exactly the data flow graph execution support required by the compiler. In one embodiment, the CSA is an accelerator (e.g., the accelerator in FIG. 2 ), and it does not seek to provide support for data flow graph execution on a general purpose processing core (e.g., Figure 2

[0014] The CSA utilizes some of the necessary but infrequently used mechanisms (such as system calls) available on the cores in the CSA. Thus, in this embodiment, the CSA can execute much code, but not all code. In exchange, the CSA gains significant performance and energy advantages. To achieve acceleration of code written in commonly used serializing languages, the embodiments herein also introduce several novel architectural features to assist the compiler. One particular novelty is the CSA's handling of memory, a topic that has previously been overlooked or poorly addressed. Embodiments of the CSA are also unique in using dataflow operators (e.g., as opposed to lookup tables (LUTs)) as their basic architectural interface.

[0200] Turning back to the embodiment of the CSA, the data flow operator is discussed below.

[0201] 2.1 Data Flow Operators

[0202] A key architectural interface of embodiments of an accelerator (e.g., a CSA) is dataflow operators, e.g., as direct representations of nodes in a dataflow graph. From an operational perspective, dataflow operators behave in a streaming or data-driven manner. A dataflow operator can be executed as soon as its incoming operands are available. CSA dataflow execution can rely (e.g., solely) on highly localized state, leading to a highly scalable architecture with, for example, a distributed asynchronous execution model. Dataflow operators can include arithmetic dataflow operators, e.g., one or more of the following: floating-point addition and multiplication; integer addition, subtraction, and multiplication; various forms of comparisons, logical operators, and shifts. However, embodiments of a CSA may also include a rich set of control operators that assist in the management of dataflow tokens in a program graph. Examples of these control operators include a "pick" operator (e.g., which multiplexes two or more logical input channels into a single output channel) and a "switch" operator (e.g., which operates as a channel demultiplexer) (e.g., outputting a single channel from two or more logical input channels). These operators enable the compiler to implement control paradigms (such as conditional expressions). Certain embodiments of the CSA may include a limited set of dataflow operators (e.g., relative to a small number of operations) to achieve a dense and energy-efficient PE microarchitecture. Certain embodiments may include dataflow operators for complex operations commonly found in HPC code. The CSA dataflow operator architecture is highly adaptable to deployment-specific extensions. For example, more complex mathematical dataflow operators (e.g., trigonometric functions) may be included in certain embodiments to accelerate certain math-intensive HPC workloads. Similarly, neural network-tuned extensions may include dataflow operators for vectorized, low-precision arithmetic.

[0203] Figure 20A The program source according to an embodiment of the present disclosure is shown in FIG. The program source code includes a multiplication function (func). Figure 20B FIG. 1 shows an embodiment of the present disclosure. Figure 20A 2000 for a program source. Dataflow graph 2000 includes a pick node 2004, a switch node 2006, and a multiplication node 2008. Buffers may optionally be included along one or more of the communication paths. The depicted dataflow graph 2000 may perform the following operations: select input X using pick node 2004, multiply X by Y (e.g., multiplication node 2008), and then output the output result from the left side of switch node 2006. Figure 20C An accelerator (eg, CSA) according to an embodiment of the present disclosure is shown, the accelerator having a configuration for performing Figure 20B2. A dataflow graph of a plurality of processing elements 2001 is shown. More specifically, dataflow graph 2000 is overlaid onto an array of processing elements 2001 (and, for example, a plurality of (e.g., interconnect) networks therebetween) such that, for example, each node in dataflow graph 2000 is represented as a dataflow operator in the array of processing elements 2001. For example, certain dataflow operations may be implemented using processing elements, and / or certain dataflow operations may be implemented using a communication network (e.g., its network dataflow endpoint circuitry). For example, Pick, PickSingleLeg, PickAny, Switch, and / or SwitchAny operations may be implemented using one or more components of a communication network (e.g., its network dataflow endpoint circuitry), e.g., in contrast to a processing element.

[0204] In one embodiment, one or more of the processing elements in the array of processing elements 2001 are configured to access memory via the memory interface 2002. In one embodiment, the pick node 2004 of the dataflow graph 2000 thus corresponds to the pick operator 2004A (e.g., represented by the pick operator 2004A), the switch node 2006 of the dataflow graph 2000 thus corresponds to the switch operator 2006A (e.g., represented by the switch operator 2006A), and the multiplier node 2008 of the dataflow graph 2000 thus corresponds to the multiplier operator 2008A (e.g., represented by the multiplier operator 2008A). Another processing element and / or the flow control path network may provide a control signal (e.g., a control token) to the pick operator 2004A and the switch operator 2006A to perform the operation. Figure 20A In one embodiment, the array of processing elements 2001 is configured to perform operations before execution begins. Figure 20B In one embodiment, the compiler executes the data flow diagram 2000 from FIG. 20A to FIG. 20B In one embodiment, the input of a dataflow graph node into the array of processing elements logically embeds the dataflow graph into the array of processing elements (e.g., as discussed further below) such that the input / output paths are configured to produce the desired results.

[0205] 2.2 Waiting time insensitive channels

[0206] Communication arcs (arcs) are the second main component of a dataflow graph. Certain embodiments of a CSA describe these arcs as latency-insensitive channels, such as ordered, back-pressured (e.g., output is not generated or sent until there is space to store it), point-to-point communication channels. Like dataflow operators, latency-insensitive channels are fundamentally asynchronous, giving the freedom to combine many types of networks to implement channels for a particular graph. Latency-insensitive channels can have arbitrarily long latencies and still faithfully implement the CSA architecture. However, in certain embodiments, there are strong performance and energy incentives to minimize latency. Section 3.2 herein discloses a network microarchitecture in which dataflow graph channels are implemented in a pipelined fashion with a latency of no more than one cycle. Embodiments of latency-insensitive channels provide a key abstraction layer that can be leveraged with the CSA architecture to provide many runtime services to application programmers. For example, a CSA can leverage latency-insensitive channels when implementing CSA configuration (loading a program onto a CSA array).

[0207] Figure 21 2100 according to an embodiment of the present disclosure. Figure 20B 1 of X in and for Figure 20B 2 of Y in ) can be loaded into the data flow graph 2100 to perform a 1*2 multiplication operation. One or more of the data input values can be static (e.g., constant) during the operation (e.g., referring to Figure 20B , X is 1 and Y is 2) or is updated during operation. In step 2, a processing element or other circuit (e.g., on a flow control path network) outputs a 0 to the control input (e.g., a mux control signal) of pick node 2104 (e.g., sourcing a "1" from a port to its output) and outputs a 0 to control the input (e.g., a mux control signal) of switch node 2106 (e.g., so that its input is provided outward from port "0" to a destination (e.g., a downstream processing element). In step 3, the data value 1 is output from pick node 2104 (and, for example, consumes its control signal "0" at pick node 2104) to multiplier node 2108 to be multiplied with the data value 2 in step 4. In step 4, the output of multiplier node 2108 reaches switch node 2106, for example, causing switch node 2106 to consume the control signal "0" to output the value 2 from port "0" of switch node 2106 in step 5. The operation is then complete. Therefore, the CSA can be programmed accordingly so that the corresponding data flow operator of each node performs Figure 21 Although the execution is serialized in this example, in principle all data flow operations can be performed in parallel. Figure 21In one embodiment, the downstream processing element is operable to send a signal (or not send a ready signal) to the switch 2106 (e.g., on a flow control path network) to stall output from the switch 2106 until the downstream processing element is ready for output (e.g., has storage space).

[0208] 2.3 Memory

[0209] Dataflow architectures generally focus on communication and data manipulation, with less attention paid to state. However, enabling real-world software, especially programs written in traditional sequential languages, requires significant attention to interfacing with memory. Certain embodiments of CSAs use architectural memory operations as their primary interface to (e.g., large) stateful stores. From a dataflow graph perspective, memory operations are similar to other dataflow operations, except that they have the side effect of updating shared storage. Specifically, the memory operations of certain embodiments herein have the same semantics as every other dataflow operator; for example, they "execute" when their operands (e.g., addresses) are available and a response is generated after some latency. Certain embodiments herein explicitly decouple operand inputs from result outputs, making memory operators inherently pipelined and capable of generating many simultaneous pending requests, thereby making them highly adaptable to the latency and bandwidth characteristics of a memory subsystem. Embodiments of CSAs provide basic memory operations such as loads and stores, which acquire an address channel and fill a response channel with the value corresponding to that address. Embodiments of CSAs also provide more advanced operations, such as in-memory atomics and consistency operators. These operations can have semantics similar to their von Neumann equivalents. Embodiments of CSA can accelerate existing programs written in sequential languages such as C and Fortran. Support for these language models results in addressing program memory order, e.g., serial ordering of memory operations typically specified by these languages.

[0210] Figure 22 2200 shows a program source (e.g., C code) according to an embodiment of the present disclosure. According to the memory semantics of the C programming language, memory copy (memcpy) should be serialized. However, if it is known that array A and array B are disjoint, memcpy can be parallelized using an embodiment of CSA. Figure 22The problem of program order is further illustrated. In general, compilers cannot prove that array A is different from array B, for example, whether for the same index value or for different index values across loop bodies. This is called pointer or memory aliasing. Because compilers are used to generate statically correct code, they are often forced to serialize memory accesses. Typically, compilers for serialized von Neumann architectures use instruction ordering as a natural means of enforcing program order. However, embodiments of CSA do not have the concept of instruction ordering defined by the program counter or instruction-based program ordering. In some embodiments, incoming dependency tokens (e.g., which do not contain architecturally visible information) are like all other dataflow tokens, and memory operations cannot be executed until they receive a dependency token. In some embodiments, once the operation of a memory operation is visible to logically subsequent dependent memory operations, these memory operations generate outgoing dependency tokens. In some embodiments, dependency tokens are similar to other dataflow tokens in a dataflow graph. For example, because memory operations occur in a conditional context, dependency tokens can also be manipulated using the control operators described in Section 2.1 (e.g., like any other token). Dependency tokens may have the effect of serializing memory accesses, thereby providing a compiler with a means to architecturally define the order of memory accesses, for example.

[0211] 2.4 Runtime Services

[0212] The primary architectural considerations for embodiments of the CSA concern the actual execution of user-level programs, but it is also desirable to provide several supporting mechanisms that underpin this execution. Chief among these are configuration (where the dataflow graph is loaded into the CSA), fetching (where the state of the execution graph is moved to memory), and exceptions (where mathematical, soft, and other types of errors in the structure can be detected and handled by external entities). Section 3.6 below discusses the latency-insensitive dataflow architecture properties of embodiments of the CSA for achieving efficient, highly pipelined implementations of these functions. Conceptually, configuration loads the state of the dataflow graph (e.g., typically from memory) into the interconnect (and / or communication network (e.g., its network dataflow endpoint circuits)) and processing elements (e.g., fabrics). During this step, all fabrics in the CSA can be loaded with the new dataflow graph, and any dataflow tokens that survive in that graph, for example, as a result of a context switch. The latency-insensitive semantics of the CSA permit distributed asynchronous initialization of fabrics, e.g., PEs can begin execution immediately upon configuration. Unconfigured PEs can backpressure their channels until they are configured, thereby, for example, preventing communication between configured and unconfigured elements. CAS configuration can be partitioned into privileged and user-level states. Such two-level partitioning allows the main configuration of the fabric to occur without invoking the operating system. In one embodiment of extraction, a logical diagram of a dataflow graph is captured and committed to memory, for example, including all live control and dataflow tokens and states in the graph.

[0213] Extraction can also play a role in providing reliability assurance by creating structural checkpoints. Exceptions in a CSA can generally be caused by the same events that cause exceptions in a processor, such as illegal operator arguments or reliability, availability, and durability (RAS) events. In some embodiments, exceptions are detected at the level of the dataflow operator (e.g., checking argument values) or through a modular arithmetic scheme. Upon detecting an exception, the dataflow operator (e.g., a circuit) can stop and emit an exception message that, for example, contains both an operation identifier and some details about the nature of the problem that occurred. In some embodiments, the dataflow operator will remain stopped until it has been reconfigured. Subsequently, the exception message can be passed to the associated processor (e.g., a core) for servicing (e.g., which may include an extraction graph for software analysis).

[0214] 2.5 Chip-Level Architecture

[0215] Embodiments of a CSA computer architecture (eg, for HPC and datacenter use) are sharded. Figure 23 and Figure 25 Shows the slice-level deployment of CSA. Figure 25A full-slice implementation of a CSA is shown, which can be, for example, an accelerator for a processor with a core. A major advantage of this architecture can be reduced design risk, for example, allowing the CSA to be completely decoupled from the core at fabrication time. In addition to allowing better component reuse, this can also allow components (like the CSA cache) to be considered CSA-only, rather than, for example, needing to incorporate more stringent latency requirements for the core. Ultimately, separate slices can allow the CSA to be integrated with either small or large cores. One embodiment of the CSA captures most vector-parallel workloads, allowing most vector-based workloads to run directly on the CSA, but in some embodiments, vector-based instructions in the core can be included, for example, to support legacy binaries.

[0216] 3. Microarchitecture

[0217] In one embodiment, the CSA microarchitecture aims to provide a high-quality implementation of each dataflow operator specified by the CAS architecture. Embodiments of the CSA microarchitecture provide that each processing element (and / or communication network (e.g., its network dataflow endpoint circuits)) of the microarchitecture corresponds to approximately one node (e.g., entity) in the architecture dataflow graph. In one embodiment, the nodes in the dataflow graph are distributed across multiple network dataflow endpoint circuits. In certain embodiments, this results in an architecture element that is not only compact, resulting in a dense array of computations, but also energy-efficient, e.g., processing elements (PEs) are both simple and highly unmultiplexed (e.g., configured (e.g., programmed) to perform a single dataflow operation). To further reduce energy and implementation area, the CSA may include a configurable heterogeneous fabric where each PE implements only a subset of the dataflow operators (e.g., utilizing a separate subset of the dataflow operators implemented as (multiple) network dataflow endpoint circuits). Peripheral and support subsystems (such as CSA caches) may be provided to support the distributed parallelism inherent in the main CSA processing structure itself. Implementations of the CSA microarchitecture can implement dataflow and latency-insensitive communication abstractions present in the architecture. In some embodiments, there is (e.g., substantially) a one-to-one correspondence between nodes in a compiler-generated graph and dataflow operators (e.g., dataflow operator compute elements) in the CSA.

[0218] The following is a discussion of an example CSA, followed by a more detailed discussion of the microarchitecture. Certain embodiments herein provide a CSA that allows for easy compilation, for example, in contrast to existing FPGA compilers, which handle a small subset of programming languages (e.g., C or C++) and can take many hours to compile even small programs.

[0219] Certain embodiments of the CSA architecture allow for heterogeneous coarse-grained operations such as double-precision floating point. Programs can be expressed in terms of less coarse-grained operations, for example, allowing the disclosed compiler to run faster than traditional spatial compilers. Certain embodiments include an architecture with new processing elements to support serialization concepts such as program-ordered memory access. Certain embodiments implement hardware to support coarse-grained dataflow-type communication channels. This communication model is abstract and closely resembles the control dataflow representation used by compilers. Certain embodiments herein include a network implementation that supports single-cycle latency communication, for example, utilizing (e.g., small) PEs that support a single control dataflow operation. In certain embodiments, this not only improves energy efficiency and performance, but also simplifies compilation because the compiler performs a one-to-one mapping between high-level dataflow constructs and structures. Certain embodiments herein therefore simplify the task of compiling existing (e.g., C, C++, or Fortran) programs to CSA (e.g., structures).

[0220] Energy efficiency can be a primary consideration in modern computer systems. Certain embodiments herein provide new models for energy-efficient spatial architectures. In certain embodiments, these architectures form a structure having a unique composition of a heterogeneous mix of small, energy-efficient, dataflow-oriented processing elements (PEs) (and / or packet-switched communication networks (e.g., their network dataflow endpoint circuits)) and lightweight circuit-switched communication networks (e.g., interconnects), for example, with enhanced support for flow control. Due to the energy advantages of each, the combination of these components can form a spatial accelerator (e.g., as part of a computer) suitable for executing compiler-generated parallel programs in an extremely energy-efficient manner. Because the structure is heterogeneous, certain embodiments can be customized for different application domains by introducing new domain-specific PEs. For example, a structure for high-performance computing may include some customization for double-precision, fused multiply-add, while a structure for deep neural networks may include low-precision floating-point operations.

[0221] Spatial architecture patterns (e.g. Figure 23 A processor (exemplified in FIG) is composed of lightweight processing elements (PEs) connected by a network of PEs. Generally speaking, a PE may include a dataflow operator, for example, where once (e.g., all) input operands arrive at the dataflow operator, an operation (e.g., a microinstruction or a set of microinstructions) is performed and the result is forwarded to a downstream operator. Thus, control, scheduling, and data storage can be distributed across multiple PEs, for example, removing the overhead of the centralized architecture that dominates classical processors.

[0222] A program can be converted into a data flow graph by configuring the PEs and the network to express the control data flow graph of the program, which is mapped onto the architecture. Communication channels can be flow controlled and fully backpressured, so that, for example, if the source communication channel has no data or the destination communication channel is full, the PE will stop. In one embodiment, at runtime, data flows through the PEs and channels that have been configured to implement the operation (e.g., the accelerated algorithm). For example, data can flow in from memory through the fabric and then out back to memory.

[0223] Embodiments of such an architecture can achieve remarkable performance efficiencies relative to traditional multi-core processors: computations (e.g., in the form of PEs) can be simpler, more energy-efficient, and richer than in larger cores, and communications can be direct and primarily short-range, as opposed to, for example, being conducted over a wide, full-chip network as in a typical multi-core processor. Furthermore, because embodiments of the architecture are extremely parallel, many powerful circuit and device-level optimizations are possible without severely impacting throughput, such as low-leakage devices and low operating voltages. These lower-level optimizations can achieve even greater performance advantages over traditional cores. The combination of efficiencies yielded by these embodiments at the architectural, circuit, and device levels is compelling. As transistor density continues to increase, embodiments of the architecture can achieve even larger effective areas.

[0224] The embodiments herein provide a unique combination of data flow support and circuit switching, enabling fabrics to be smaller, more energy efficient, and provide higher aggregate performance than previous architectures. FPGAs are generally tuned for fine-grained bit manipulation, while the embodiments herein are tuned for double-precision floating-point operations found in HPC applications. Certain embodiments herein may include an FPGA in addition to a CSA according to the present disclosure.

[0225] Certain embodiments herein combine a lightweight network with energy-efficient dataflow processing elements (and / or communication networks (e.g., their network dataflow endpoint circuits)) to form a high-throughput, low-latency, energy-efficient HPC fabric. The low-latency network allows the creation of processing elements (and / or communication networks (e.g., their network dataflow endpoint circuits)) with fewer functions (e.g., only one or two instructions, and perhaps only one architecturally visible register, because it is efficient to aggregate multiple PEs together to form a complete program).

[0226] Compared to processor cores, CSA embodiments herein can provide greater computational density and energy efficiency. For example, when the number of PEs (e.g., compared to cores) is very small, a CSA can perform many more operations than a core and can have much more computational parallelism than a core, for example, perhaps up to 16 times the number of FMAs as a vector processing unit (VPU). To utilize all of these computational elements, in some embodiments, the energy per operation is very low.

[0227] The energy advantages of embodiments of the dataflow architecture of the present application are numerous. Parallelism is explicit in the dataflow graph, and embodiments of the CSA architecture expend no or minimal energy to extract this parallelism, unlike, for example, out-of-order processors, which must rediscover parallelism each time an instruction is executed. In one embodiment, because each PE is responsible for a single operation, the register file and port count can be small, often just one, and thus use less energy than their counterparts in the core. Some CSAs include many PEs, each of which holds live program values, giving the aggregation effect of a large register file in traditional architectures, which significantly reduces memory accesses. In embodiments where memory is multi-ported and distributed, a CSA can maintain many more outstanding memory requests and utilize more bandwidth than a core. These advantages combine to achieve energy-per-watt levels that are only a small percentage of the cost of bare arithmetic circuitry. For example, in the case of integer multiplication, a CSA can consume no more than 25% more energy than the underlying multiplication circuitry. Relative to one embodiment of the core, integer operations in that CSA structure consume less than 1 / 30th the energy per integer operation.

[0228] From a programming perspective, the application-specific flexibility of embodiments of the CSA architecture provides significant advantages over vector processing units (VPUs). In traditional, inflexible architectures, the number of functional units like floating-point division or various transcendental math functions must be chosen at design time based on some desired use case. In embodiments of the CSA architecture, such functions can be configured into the architecture (e.g., by the user rather than the manufacturer) based on the requirements of each application. Application throughput can thereby be further increased. At the same time, by avoiding the need to harden such functions and instead providing more instances of primitive functions like floating-point multiplication, the computational density of embodiments of the CSA is improved. These advantages can be significant in HPC workloads, some of which spend 75% of their floating-point execution time in transcendental functions.

[0229] Certain embodiments of CSA represent significant advances in dataflow-oriented spatial architectures, e.g., the PEs of the present disclosure can be smaller and more energy efficient. These improvements can be directly derived from the combination of dataflow-oriented PEs with lightweight, circuit-switched interconnects, e.g., having single-cycle latency, as opposed to packet-switched networks (e.g., having at least 300% higher latency). Certain embodiments of the PEs support 32-bit or 64-bit operations. Certain embodiments herein allow the introduction of new application-specific PEs, e.g., for machine learning or security, and not just homogeneous combinations. Certain embodiments herein combine lightweight, dataflow-oriented processing elements with lightweight, low-latency networks to form energy-efficient computing structures.

[0230] To enable certain spatial architectures to succeed, programmers will have to spend relatively little effort to configure them, for example, while simultaneously achieving significant power and performance advantages over serialized cores. Certain embodiments herein provide CSAs (e.g., spatial structures) that are easy to program (e.g., by a compiler), highly power-efficient, and highly parallel. Certain embodiments herein provide (e.g., interconnected) networks that achieve these three goals. From a programmability perspective, certain network embodiments provide flow-controlled channels that correspond, for example, to the control data flow graph (CDFG) model of execution used in compilers. Certain network embodiments utilize dedicated circuit-switched links, making program performance easier to deduce by both humans and compilers because performance is predictable. Certain network embodiments provide both high bandwidth and low latency. Certain network embodiments (e.g., static, circuit-switched) provide latency of 0 to 1 cycle (e.g., depending on the transmission distance). Certain network embodiments provide high bandwidth by arranging several networks in parallel (and, for example, in low-level metal). Certain network embodiments communicate in low-level metal and over short distances, and therefore have very high power efficiency.

[0231] Certain embodiments of the network include architectural support for flow control. For example, in a spatial accelerator composed of small processing elements (PEs), communication latency and bandwidth can be critical to overall program performance. Certain embodiments herein provide a lightweight, circuit-switched network that facilitates spatial processing arrays (such as, Figure 23) in a spatial array shown in . Certain embodiments of the network implement the construction of point-to-point, flow-controlled communication channels that support communication of processing elements (PEs) oriented towards data flows. In addition to point-to-point communication, certain networks herein also support multicast communication. Communication channels can be formed by statically configuring the network to form virtual circuits between PEs. The circuit switching technology herein can reduce communication latency and correspondingly minimize network buffering, thereby, for example, producing both high performance and high energy efficiency. In certain embodiments of the network, inter-PE latency can be as low as zero cycles, which means that downstream PEs can operate on the data within the cycle after the data is generated. In order to obtain even higher bandwidth, and to allow more programs, multiple networks can be arranged in parallel, for example, as Figure 23 As shown in .

[0232] Spatial architecture (such as Figure 23 The spatial architecture shown in ) can be composed of lightweight processing elements connected by an inter-PE network (and / or a communication network (e.g., its network data flow endpoint circuits)). A program, viewed as a data flow graph, can be mapped onto the architecture by configuring the PEs and the network. In general, a PE can be configured as a data flow operator, and once (e.g., all) input operands arrive at the PE, some operation can then occur and the result is forwarded to the required downstream PE. The PEs can communicate through dedicated virtual circuits, which are formed by statically configuring a circuit-switched communication network. These virtual circuits can be flow controlled and fully back pressured, so that, for example, if the source has no data or the destination is full, the PE will stall. At runtime, data can flow through the PEs that implement the mapped algorithm. For example, data can flow from memory through the fabric and then out back to memory. Embodiments of this architecture can achieve superior performance efficiency relative to traditional multi-core processors: for example, computation in the form of PEs is simpler and more numerous than in larger cores, as opposed to an extended memory system, and communication is direct.

[0233] Figure 23An accelerator slice 2300 is illustrated according to an embodiment of the present disclosure, the accelerator slice 2300 including an array of processing elements (PEs). The interconnection network is depicted as circuit-switched, statically configured communication channels. For example, a collection of channels are coupled together by switching devices (e.g., switching devices 2310 in a first network and switching devices 2320 in a second network). The first network and the second network can be separate or can be coupled together. For example, the switching device 2310 can couple together one or more of the four data paths 2312, 2314, 2316, 2318, e.g., configured to perform operations according to a data flow graph. In one embodiment, the number of data paths is arbitrarily large. The processing elements (e.g., processing element 2304) can be as disclosed herein, e.g., as Figure 26 As in [ 15 ], the accelerator slice 2300 includes a memory / cache hierarchy interface 2302 to, for example, interface the accelerator slice 2300 with memory and / or cache. A data path (e.g., 2318) may extend to another slice or may terminate, for example, at the edge of a slice. A processing element may include an input buffer (e.g., buffer 2306) and an output buffer (e.g., buffer 2308).

[0234] Operations can be performed based on the availability of inputs to those operations and the state of the PE. The PE can fetch operands from input channels and write results to output channels, but can also use internal register state. Certain embodiments herein include configurable dataflow-friendly PEs. Figure 26 A detailed block diagram of one such PE is shown: an integer PE. This PE consists of several I / O buffers, an ALU, storage registers, some instruction registers, and a scheduler. At each cycle, the scheduler can select an instruction for execution based on the availability of input and output buffers and the state of the PE. The result of the operation is then written to the output register, or to a register (e.g., local to the PE). The data written to the output buffer can be transferred to a downstream PE for further processing. This PE style can be extremely energy efficient, for example, the PE reads data from registers instead of from a complex multi-port register set. Similarly, instructions can be stored directly in registers rather than in a virtualized instruction cache.

[0235] The instruction register can be set during a special configuration step. During this step, in addition to the inter-PE network, auxiliary control lines and status can also be used to flow the configuration across the several PEs comprising the fabric. As a result of the parallelism, certain embodiments of such a network can provide fast reconfiguration; for example, a chip-sized fabric can be configured in less than approximately 10 microseconds.

[0236] Figure 26represents an example configuration of a processing element, for example, where all architectural elements are sized to their minimum. In other embodiments, each of the multiple components of a processing element is independently scaled to produce a new PE. For example, to handle more complex programs, a greater number of instructions that can be executed by a PE may be introduced. The second dimension of configurability is the functionality of the PE arithmetic logic unit (ALU). In Figure 26 In [1], integer PEs are depicted as supporting addition, subtraction, and various logical operations. Other types of PEs can be created by substituting different types of functional units into PEs. For example, an integer multiplication PE may have no registers, a single instruction, and a single output buffer. Certain embodiments of PEs deconstruct fused multiply-add (FMA) into separate but tightly coupled floating-point multiplication and floating-point addition units to improve support for multiply-add-heavy workloads. PEs are discussed further below.

[0237] Figure 24A The embodiment of the present disclosure is shown in FIG. Figure 23 Configurable data path network 2400 (e.g., network 1 or network 2 discussed). Network 2400 includes a plurality of multiplexers (e.g., multiplexers 2402, 2404, 2406) that can be configured (e.g., via their respective control signals) to connect one or more data paths (e.g., from PEs) together. Figure 24B The embodiment of the present disclosure is shown in FIG. Figure 23 The network 1 or the network 2 discussed herein may be a configurable flow control path network 2401. The network may be a lightweight PE-to-PE network. Certain embodiments of the network may be viewed as a collection of primitives for constructing distributed point-to-point data channels. Figure 24A A network is shown with two channels (bold black line and dotted black line) enabled. The bold black channel is multicast, i.e., a single input is sent to two outputs. Note that even though dedicated circuit-switched paths are formed between the channel endpoints, the channels can cross at some points within a single network. Furthermore, this crossing does not introduce structural damage between the two channels, allowing each to operate independently and at full bandwidth.

[0238] Implementing a distributed data channel may include Figures 24A-24B The two paths shown in . The forward or data path carries data from the producer to the consumer. The multiplexer can be configured to direct data and valid bits from the producer to the consumer, for example, Figure 24A In the case of multicast, the data will be directed to multiple consumer endpoints. The second part of this embodiment of the network is the flow control or backpressure path, which flows in the opposite direction of the forward data path, such as Figure 24BAs shown in . Consumer endpoints can assert when they are ready to accept new data. Configurable logic can then be used (in Figure 24B 24A . The flow control paths may handle the return of control data from the consumer to the producer. The nodes may enable multicasting, for example, where each consumer is ready to receive data before the producer assumes that the data has been received. In one embodiment, the PE is a PE having a data flow operator as its architectural interface. Additionally or alternatively, in one embodiment, the PE may be any type of PE (e.g., in a fabric), such as, but not limited to, a PE having an instruction pointer, triggered instructions, or a state machine-based architectural interface.

[0239] In addition to, for example, PE being statically configured, the network may also be statically configured. During this configuration step, configuration bits may be set at each network component. These bits control, for example, mux selection and flow control functions. The network may include multiple networks, for example, a data path network and a flow control path network. A network or multiple networks may utilize paths of different widths (for example, a first width and a narrower or wider width). In one embodiment, the data path network has a width (for example, bit transmission) wider than the width of the flow control path network. In one embodiment, each of the first network and the second network includes its own data path network and flow control path network, for example, data path network A and flow control path network A and a wider data path network B and flow control path network B.

[0240] Certain embodiments of the network are unbuffered, and data is designed to move between producers and consumers in a single cycle. Certain embodiments of the network are also unbounded, that is, the network spans the entire fabric. In one embodiment, a PE is designed to communicate with any other PE in a single cycle. In one embodiment, to improve routing bandwidth, several networks can be arranged in parallel between rows of PEs.

[0241] Relative to FPGAs, certain embodiments of the networks herein have three advantages: area, frequency, and program expression. Certain embodiments of the networks herein operate at a coarse granularity, which, for example, reduces the number of configuration bits and thereby reduces the area of the network. Certain embodiments of the networks also achieve area reduction by directly implementing flow control logic in the circuit (e.g., silicon). Certain embodiments of the enhanced network implementation also enjoy frequency advantages relative to FPGAs. Due to the area and frequency advantages, power advantages may exist when lower voltages are used at throughput parity. Finally, certain embodiments of the networks provide better high-level semantics than FPGA lines, especially with respect to variable timing, and therefore, those embodiments are more easily targeted by compilers. Certain embodiments of the networks herein can be viewed as a collection of constituent primitives for building distributed point-to-point data channels.

[0242] In some embodiments, a multicast source may not be able to assert that its data is valid unless it receives a ready signal from each sink. Therefore, in the multicast case, additional binding and control bits may be utilized.

[0243] Like some PEs, the network can be statically configured. During this step, configuration bits are set at each network component. These bits control functions such as mux selection and flow control. The forward path of the network of this application requires some bits to enable the mux of the forward path to swing. Figure 24A In the example shown in FIG, four bits are required per hop: one bit is used for each of the east and west muxes, while two bits are used for the south mux. In this embodiment, four bits are available for the data path, but seven bits are available for the flow control function (e.g., in a flow control path network). Other embodiments may utilize more bits if, for example, the CSA further utilizes a north-south direction. The flow control function may use a control bit for each direction from which flow control may come. This allows for statically setting the sensitivity of the flow control function. Table 1 below summarizes the bits used for Figure 24B In the Boolean algebraic implementation of the network flow control function in

[0045] , the configuration bits are capitalized. In this example, seven bits are utilized.

[0244] Table 1: Stream implementation methods

[0245]

[0246] For from Figure 24B The third flow control box from the left in FIG, EAST_WEST_SENSITIVE and NORTH_SOUTH_SENSITIVE are depicted as being set to implement flow control for the bold line channel and the dotted line channel, respectively.

[0247] Figure 25

[00105] The diagram illustrates a hardware processor slice 2500 including an accelerator 2502 according to an embodiment of the present disclosure. Accelerator 2502 may be a CSA according to the present disclosure. Slice 2500 includes a plurality of cache blocks (e.g., cache block 2508). Request address file (RAF) circuitry 2510 may be included, for example, as discussed below in Section 3.2. ODI may refer to an on-die interconnect, e.g., an interconnect that extends across the entire die, connecting all slices. OTI may refer to an on-chip interconnect (e.g., that extends across the slice, e.g., connecting cache blocks on a slice together).

[0248] 3.1 Processing Elements

[0249] In some embodiments, a CSA comprises an array of heterogeneous PEs, where the structure consists of several types of PEs, each of which implements only a subset of the data flow operators. As an example, Figure 26 A tentative implementation of a PE capable of implementing a broad set of integer and control operations is shown. Other PEs (including those supporting floating-point addition, floating-point multiplication, buffering, and certain control operations) may also have similar implementation styles, for example, replacing the ALU with appropriate (dataflow operator) circuitry. Before execution begins, a CSA's PEs (e.g., dataflow operators) may be configured (e.g., programmed) to implement specific dataflow operations from the set supported by the PE. The configuration may include one or two control words that specify opcodes that control the ALU, direct various multiplexers within the PE, and drive dataflow into and out of the PE channels. The dataflow operators may be implemented by microcoding these configuration bits. Figure 26 The integer PE 2600 depicted in FIG is organized into a single-stage logic pipeline that flows from top to bottom. Data enters PE 2600 from one of a set of local networks, where it is stored in input buffers for subsequent operations. Each PE can support multiple wide data-oriented channels and narrow control-oriented channels. The number of channels provided can vary based on the functionality of the PE, but one embodiment of an integer-oriented PE has two wide and one or two narrow input and output channels. Although the integer PE is implemented as a single-cycle pipeline, other pipeline options are possible. For example, a multiplication PE can have multiple pipeline stages.

[0250] PE execution can proceed in a data flow style. Based on the configuration microcode, the scheduler can check the status of the PE entry and exit buffers and arrange for the actual execution of the operation by the data operator (e.g., on the ALU) when all inputs for the configured operation have arrived and the exit buffer of the operation is available. The resulting value can be placed in the configured exit buffer. When the buffer becomes available, the transfer between the exit buffer of one PE and the entry buffer of another PE can occur asynchronously. In some embodiments, the PE is provided so that at least one data flow operation is completed for each cycle. Section 2 discusses data flow operators covering primitive operations (such as add, exclusive OR (xor), or select). Certain embodiments can provide advantages in energy, area, performance, and latency. In one embodiment, more fusion combinations can be enabled by extending the PE control path. In one embodiment, the width of the processing element is 64 bits, for example, for high utilization of double-precision floating-point calculations in HPC and for supporting 64-bit memory addressing.

[0251] 3.2 Communication Network

[0252] Embodiments of the CSA microarchitecture provide a hierarchy of multiple networks that together provide an architectural abstraction for implementing latency-insensitive channels across multiple communication scales. The lowest level of the CSA communication hierarchy can be a local network. The local network can be statically circuit-switched, for example, using configuration registers to oscillate multiplexer(s) in the local network data path to form a fixed electrical path between communicating PEs. In one embodiment, the configuration of the local network is set once for each dataflow graph (e.g., at the same time as PE configuration). In one embodiment, static circuit switching is optimized for energy, for example, where the majority (perhaps greater than 95%) of CSA communication traffic will traverse the local network. Programs can include terms used in multiple expressions. To optimize for this situation, embodiments herein provide hardware support for multicast within the local network. Several local networks can be aggregated to form routing channels, which are, for example, spread across rows and columns of PEs (in a mesh). As an optimization, several local networks can be included to carry control tokens. Compared to FPGA interconnects, CSA local networks can be routed at the granularity of data paths, and another difference may be the CSA's handling of control. One embodiment of a CSA local network is explicitly flow-controlled (e.g., backpressure). For example, for each forward data path and multiplexer set, the CSA is used to provide a backward flow control path that is physically paired with the forward data path. The combination of the two microarchitectural paths can provide a low-latency, low-energy, small-area, point-to-point implementation of a latency-insensitive channel abstraction. In one embodiment, the CSA's flow control lines are not visible to user programs, but they can be manipulated by the architecture that maintains user programs. For example, the exception handling mechanism described in Section 2.2 can be implemented by pulling the flow control lines to the "absent" state after detecting an exception condition. This action not only gently stops the portion of the pipeline involved in the offending computation, but also preserves the machine state prior to the exception, for example, to facilitate diagnostic analysis. The second network layer (e.g., the mezzanine network) can be a shared packet-switched network. A mezzanine network may include multiple distributed network controllers, network data flow endpoint circuits. Figure 39The mezzanine network (schematically indicated by the dashed box in the figure) can provide more general long-distance communication at the expense of, for example, latency, bandwidth, and energy. In some applications, most communication can occur on the local network, so the mezzanine network provisioning will be significantly reduced in comparison. For example, each PE can be connected to multiple local networks, but the CSA will only provision one mezzanine endpoint for each logical neighborhood of the PE. Because the mezzanine is effectively a shared network, each mezzanine network can carry multiple logically independent channels and be provisioned, for example, as multiple virtual channels. In one embodiment, the primary function of the mezzanine network is to provide wide-range communication between PEs and between PEs and storage. In addition to this capability, the mezzanine may also include (multiple) network data flow endpoint circuits, for example, for certain data flow operations. In addition to this capability, the mezzanine can also operate as a runtime support network, for example, through which various services can access the complete structure in a user-program transparent manner. In this regard, the mezzanine endpoint can act as a controller for its local neighborhood, for example, during CSA configuration. To form a channel across a CSA slice, three subchannels and two local network channels (which carry traffic to and from a single channel in the mezzanine network) may be utilized. In one embodiment, one mezzanine channel is utilized, e.g., one mezzanine and two local = a total of three network hops.

[0253] The composability of channels across network layers is extended to higher-level network layers at inter-chip, inter-die, and fabric granularity.

[0254] Figure 26 26. The processing element 2600 is shown in accordance with an embodiment of the present disclosure. In one embodiment, operation configuration registers 2619 are loaded during configuration (e.g., mapping) and specify a particular operation (or operations) to be performed by the processing (e.g., computing element). The activity of registers 2620 may be controlled by that operation (e.g., the output of mux 2616 is controlled by scheduler 2614, for example). For example, scheduler 2614 may schedule one or more operations of processing element 2600 as input data and control inputs arrive. Control input buffer 2622 is connected to local network 2602 (e.g., and local network 2602 may include, for example, Figure 24A The data path network in Figure 24B2616). The control input buffer 2622 and the control output buffer 2632 may be loaded with a value when a value arrives (e.g., the network has data bit(s) and valid bit(s). The control output buffer 2632, data output buffer 2634, and / or data output buffer 2636 may receive the output of the processing element 2600 (e.g., as controlled by the operation (output of mux 2616)). The status register 2638 may be loaded each time the ALU 2618 executes (also controlled by the output of mux 2616). The data in the control input buffer 2622 and the control output buffer 2632 may be a single bit. Mux 2621 (e.g., operand A) and mux 2623 (e.g., operand B) may serve as sources of input.

[0255] For example, assuming that the operation of the processing (e.g., computing) element is (or includes) the operation referred to as a select in FIG20B , the processing element 2600 is configured to select data from either the data input buffer 2624 or the data input buffer 2626, for example, to be sent to the data output buffer 2634 (e.g., by default) or the data output buffer 2636. Thus, if selecting from the data input buffer 2624, the control bit in 2622 may indicate a 0, or if selecting from the data input buffer 2626, the control bit in 2622 may indicate a 1.

[0256] For example, assuming that the operation of the processing (e.g., computing) element is (or includes) the operation referred to as switching in Figure 20B, the processing element 2600 is configured to output data from, for example, data input buffer 2624 (e.g., by default) or data input buffer 2626 to data output buffer 2634 or data output buffer 2636. Therefore, if the output is to data output buffer 2634, the control bit in 2622 may indicate 0, or if the output is to data output buffer 2636, the control bit in 2622 may indicate 1.

[0257] A plurality of networks (eg, interconnects) (eg, (input) networks 2602, 2604, 2606 and (output) networks 2608, 2610, 2612) may be connected to the processing element. The connections may be, for example, reference Figure 24A and Figure 24B In one embodiment, each network includes two sub-networks (or two channels on the network), for example, one for Figure 24A The data path network in Figure 24B As an example, local network 2602 (e.g., established as a control interconnect) is depicted as being switched (e.g., connected) to control input buffer 2622. In this embodiment, the data path (e.g., Figure 24A 2622) may carry a control input value (e.g., one or more bits) (e.g., a control token), and the flow control path (e.g., network) may carry a backpressure signal (e.g., a backpressure or no-backpressure token) from the control input buffer 2622 to indicate, for example, to an upstream producer (e.g., a PE) that a new control input value (e.g., from a control output buffer of the upstream producer) will not be loaded into (e.g., sent to) the control input buffer 2622 until the backpressure signal indicates that there is room in the control input buffer 2622 for the new control input value. In one embodiment, a new control input value may not enter the control input buffer 2622 until both (i) the upstream producer receives a "space available" backpressure signal from the "control input" buffer 2622 and (ii) the new control input value is sent from the upstream producer, for example, and this may stall the processing element 2600 until that occurs (and space is available in the target, output buffer(s)).

[0258] Data input buffer 2624 and data input buffer 2626 can be implemented in a similar manner, for example, local network 2604 (e.g., established as a data (as opposed to control) interconnect) is depicted as being switched (e.g., connected) to data input buffer 2624. In this embodiment, the data path (e.g., Figure 24A 2624) may carry a data input value (e.g., one or more bits) (e.g., a data flow token), and the flow control path (e.g., network) may carry a backpressure signal (e.g., a backpressure or no-backpressure token) from the data input buffer 2624 to indicate, for example, to an upstream producer (e.g., a PE) that a new data input value (e.g., from a data output buffer of the upstream producer) will not be loaded into (e.g., sent to) the data input buffer 2624 until the backpressure signal indicates that there is room in the data input buffer 2624 for the new data input value. In one embodiment, the new data input value may not enter the data input buffer 2624 until both (i) the upstream producer receives a "space available" backpressure signal from the "data input" buffer 2624; and (ii) the new data input value is sent from the upstream producer, for example, and this may stall the processing element 2600 until that occurs (and space is available in the target, output buffer(s)). Control output values and / or data outputs may be stalled in their respective output buffers (eg, 2632 , 2634 , 2636 ) until a backpressure signal indicates that there is available space in the input buffer for the downstream processing element(s).

[0259] Processing element 2600 may stall execution until its operands (e.g., a control input value and one or more corresponding data input values for the control input value) are received and / or until there is space in the output buffer(s) of processing element 2600 for data to be produced by performing operations on those operands.

[0260] 3.3 Memory Interface

[0261] Request Address File (RAF) circuit (in Figure 27A RAF (a simplified version of which is shown in Figure 1) may be responsible for executing memory operations and acting as an intermediary between the CSA fabric and the memory hierarchy. Thus, the primary microarchitectural role of the RAF may be to rationalize the out-of-order memory subsystem with the in-order semantics of the CSA fabric. In this capacity, the RAF circuitry may be provided with completion buffers (e.g., queue-like structures) that reorder memory responses and return these memory requests to the fabric in the order in which they were requested. A second primary function of the RAF circuitry may be to provide support in the form of address translation and page walkers. Incoming virtual addresses are translated into physical addresses using channel-associated translation lookaside buffers (TLBs). To provide ample memory bandwidth, each CSA slice may include multiple RAF circuits. Like the fabric's various PEs, the RAF circuitry may operate in a dataflow fashion by checking the availability of input arguments and output buffers (if needed) before selecting a memory operation to execute. However, unlike some PEs, the RAF circuitry is multiplexed between several co-located memory operations. The reused RAF circuit can be used to minimize the area overhead of its various subcomponents, such as shared accelerator cache interface (ACI) ports (described in more detail in Section 3.4), shared virtual memory (SVM) support hardware, mezzanine network interfaces, and other hardware management facilities. However, there are some program characteristics that also promote this choice. In one embodiment, (e.g., efficient) data flow graphs are used to round-robin the memory in the shared virtual memory system. Memory latency-constrained programs (like graph traversals) can utilize many separate memory operations to saturate the memory bandwidth due to the control flow that depends on the memory. Although each RAF can be reused, CAS can include multiple (e.g., between 8 and 32) RAFs of slice granularity to ensure sufficient cache bandwidth. RAFs can communicate with the rest of the structure via both the local network and the mezzanine network. When RAFs are reused, each RAF can be supplied to the local network together with several ports. These ports can serve as the lowest latency, highly deterministic path to the memory for use by latency-sensitive or high-bandwidth memory operations. Additionally, the RAF may be provisioned with a mezzanine network endpoint that provides memory access to runtime services and remote user-level memory accessors, for example.

[0262] Figure 27The diagram illustrates a request address file (RAF) circuit 2700 according to an embodiment of the present disclosure. In one embodiment, at configuration time, memory load and store operations already in the data flow graph are specified in register 2710. Subsequently, arcs to those memory operations in the data flow graph may be connected to input queues 2722, 2724, and 2726. Arcs from those memory operations are therefore used to leave completion buffers 2728, 2730, or 2732. Dependency tokens (which may be multiple individual bits) arrive in queues 2718 and 2720. Dependency tokens will leave from queue 2716. Dependency token counter 2714 may be a compact representation of a queue and may track the number of dependency tokens for any given input queue. If dependency token counter 2714 is saturated, no additional dependency tokens may be generated for new memory operations. Accordingly, the memory sorting circuit (e.g., Figure 28 The RAF in will stop scheduling new memory operations until the dependency token counter 2714 becomes unsaturated.

[0263] As an example of a load, an address arrives in queue 2722, and scheduler 2712 matches queue 2722 with the load in 2710. The completion buffer slot for the load is assigned in the order the address arrives. Assuming that this particular load in the figure has no specified dependencies, the address and completion buffer slot are dispatched to the memory system by the scheduler (e.g., via memory command 2742). When the result returns to mux 2740 (shown schematically), the result is stored in its designated completion buffer slot (e.g., because the result carries the target slot all the way through the memory system). The completion buffer sends the results back to the local network (e.g., local network 2702, 2704, 2706, or 2708) in the order the address arrived.

[0264] Stores can be simpler, except that both the address and the data must arrive before any operation can be dispatched to the memory system.

[0265] 3.4 Cache

[0266] The data flow graph may be able to generate a large number of (e.g., word-granular) requests in parallel. Therefore, some embodiments of the CSA provide sufficient bandwidth to the cache subsystem to maintain the CSA. Figure 28 ). Figure 28A circuit 2800 according to an embodiment of the present disclosure is shown, having multiple request address file (RAF) circuits (e.g., RAF circuit 1) coupled between multiple accelerator slices 2808, 2810, 2812, 2814 and multiple cache blocks (e.g., cache block 2802). In one embodiment, the number of RAFs and cache blocks can be in a 1:1 or 1:2 ratio. A cache block can include a complete cache line (e.g., as opposed to word-by-word sharing), and each line has a specific home in the cache. Cache lines can be mapped to cache blocks via a pseudo-random function. The CSA can employ an SVM model for integration with other sharding architectures. Certain embodiments include an accelerator cache interconnect (ACI) network connecting the RAFs to the cache blocks. The network can carry addresses and data between the RAFs and the cache. The ACI topology can be a cascaded crossbar switch, for example, as a trade-off between latency and implementation complexity.

[0267] 3.5 floating point support

[0268] Certain HPC applications are characterized by their need for significant floating-point bandwidth. To meet this need, embodiments of the CSA can be provisioned with multiple (e.g., each with between 128 and 256) floating-point addition and multiplication processors, depending on the slice configuration. The CSA can offer several other extended-precision modes, for example to simplify math library implementations. CSA floating-point processors can support both single- and double-precision, but lower-precision processors can support machine learning workloads. The CSA can offer floating-point performance that is an order of magnitude higher than that of the processing cores. In one embodiment, in addition to increasing floating-point bandwidth, the energy consumed in floating-point operations is reduced to drive all floating-point units. For example, to reduce energy, the CSA can selectively gate the low-order bits of the floating-point multiplier array. When examining the behavior of floating-point arithmetic, the low-order bits of the multiplication array may not often affect the final rounded product. Figure 29 The diagram shows a floating-point multiplier 2900 partitioned into three regions (a result region, three potential carry regions 2902, 2904, 2906, and a gate region) according to an embodiment of the present disclosure. In some embodiments, the carry region may affect the result region, while the gate region is less likely to affect the result region. Considering a g-bit gate region, the maximum carry may be:

[0269]

[0270] Given this maximum carry, if the result of the carry area is less than 2 cIf the carry region is c bits wide (where the carry region is c bits wide), the gated region can be ignored because it does not affect the result region. Increasing g means it is more likely that the gated region will be needed, while increasing c means that, under random assumptions, the gated region will not be used and can be disabled to avoid energy consumption. In an embodiment of the CSA floating-point multiplication PE, a two-stage pipelined approach is utilized, where the carry region is first determined, and then, if the gated region is found to affect the result, the gated region is determined. If more information about the context of the multiplication is known, the CSA adjusts the size of the gated region more aggressively. In FMA, the multiplication result may be added to an accumulator, which is often much larger than either of the multiplicands. In this case, the addend exponent can be observed in advance of the multiplication, and the CSDA can adjust the gated region accordingly. One embodiment of the CSA includes a scheme in which a context value (which constrains the minimum result of the computation) is provided to the relevant multipliers to select the lowest-energy gating configuration.

[0271] 3.6 Runtime Services

[0272] In certain embodiments, the CSA comprises a heterogeneous distributed architecture, and therefore, the runtime service implementation is designed to accommodate several types of PEs in a parallel, distributed manner. While runtime services in the CSA may be critical, they may be less frequent than user-level computations. Therefore, certain embodiments focus on overlaying services onto hardware resources. To achieve these goals, CSA runtime services may be structured as a hierarchy, with each layer corresponding to a CSA network, for example. At the slice level, a single externally-facing controller may accept or send service commands to cores associated with a CSA slice. The slice-level controller may serve to coordinate regional controllers at the RAF (e.g., using ACI networking). The regional controllers may, in turn, coordinate local controllers at certain mezzanine network stations (e.g., network data flow endpoint circuits). At the lowest level, service-specific microprotocols may be executed on the local network (e.g., during special modes controlled by the mezzanine controller). The microprotocols may allow each PE (e.g., PE classes divided by type) to interact with runtime services according to its needs. Consequently, parallelism is implicit in this hierarchical organization, and operations at the lowest levels can occur simultaneously. This parallelism can enable configuration of CSA slices in a range of hundreds of nanoseconds to a few microseconds, depending, for example, on the configured size of the CSA slices and their location in the memory hierarchy. Thus, embodiments of the CSA exploit the properties of dataflow graphs to improve the implementation of each runtime service. A key observation is that runtime services may only need to maintain a valid logical view of the dataflow graph (e.g., the state that can be generated by a certain ordering of the execution of dataflow operators). Services generally may not need to guarantee a temporal view of the dataflow graph (e.g., the state of the dataflow graph in the CSA at a given moment). For example, assuming that services are arranged to maintain a logical view of the dataflow graph, this can allow the CSA to perform most runtime services in a distributed, pipelined, parallel manner. The local configuration microprotocol can be a packet-based protocol overlaid on a local network. Configuration targets can be organized into configuration chains, e.g., fixed in the microarchitecture. Structural (e.g., PE) targets can be configured one at a time, e.g., using a single additional register for each target to achieve distributed coordination. To begin configuration, the controller may drive an out-of-band signal that places all fabric targets within its neighborhood into an unconfigured, paused state and swings multiplexers in the local network to a predefined configuration. When fabric (e.g., PE) targets are configured (i.e., they have fully received their configuration packet), they may set their configuration micro-protocol registers, thereby notifying the next target (e.g., PE) that it may continue configuration using subsequent packets. There is no limit on the size of a configuration packet, and packets may have dynamically variable lengths. For example, a PE that configures a constant operand may have a length set to include a constant field (e.g., Figure 20B-Figure 20Cand Y in the configuration grouping. Figure 30 The diagram illustrates the on-the-fly configuration of an accelerator 3000 having multiple processing elements (e.g., PEs 3002, 3004, 3006, 3008) according to an embodiment of the present disclosure. Once configured, the PEs can execute subject to data flow constraints. However, channels involving unconfigured PEs can be disabled by the microarchitecture, for example, to prevent any undefined operations from occurring. These properties allow embodiments of the CSA to initialize and execute in a distributed manner, without any centralized control. From an unconfigured state, configuration can occur entirely in parallel (e.g., perhaps in as little as 200 nanoseconds). However, due to the distributed initialization of embodiments of the CSA, PEs can become active, for example, sending requests to memory long before the entire architecture is configured. Extraction can proceed in much the same manner as configuration. Local networks can be observed to extract data from one target at a time, and extract state bits to achieve distributed coordination. The CSA can arrange for extraction to be non-destructive, i.e., upon completion of the extraction, each extractable target has returned to its starting state. In this implementation, all states in the target are propagated to egress registers connected to the local network in a scan-like fashion. However, in-place extraction can be implemented by introducing new paths in the register transfer level (RTL) or using existing lines to provide the same functionality with lower overhead. Similar configurations and hierarchical extraction are implemented in parallel.

[0273] Figure 31 31 allows most of the fabric to continue executing while narrow regions are disabled for extraction. Configuration and extraction can be coordinated and composed to implement pipelined context switching. Qualitatively speaking, exceptions may differ from configuration and extraction in that, rather than occurring at a specified time, exceptions may occur anywhere in the fabric at any time during runtime. Thus, in one embodiment, the exception microprotocol may not be overlaid on the local network and utilize its own network, which is occupied by the user program at runtime. However, exceptions are inherently rare and are not sensitive to latency and bandwidth. Therefore, certain embodiments of the CSA utilize a packet-switched network to carry exceptions to a local mezzanine station, for example, where they are forwarded further up the service hierarchy (e.g., as Figure 42(as shown in ). Packets in a local anomaly network can be extremely small. In many cases, only 2 to 8 bits of PE identification (ID) are sufficient for a complete packet, for example because the CSA can create unique anomaly identifiers as the packet traverses the anomaly service hierarchy. Such a scheme can be desirable because it reduces the area overhead of generating anomalies at each PE.

[0274] 4. Compile

[0275] Compiling programs written in high-level languages onto CSAs may be necessary for industrial applications. This section provides a high-level overview of the compilation strategy for an embodiment of a CSA. First, a CSA software framework is proposed that illustrates the desired properties of an ideal production-quality toolchain. Next, a prototype compiler framework is discussed. This is followed by a discussion of "control-data flow conversion," which is used, for example, to convert ordinary serialized control flow code into CSA data flow assembly code.

[0276] 4.1 Example Production Framework

[0277] Figure 32 The diagram illustrates a compilation toolchain 3200 for an accelerator according to an embodiment of the present disclosure. The toolchain compiles high-level languages (such as C, C++, and Fortran) into a combination of (LLVM) intermediate representations (IR) of the main code for the specific area to be accelerated. The CSA-specific portion of the compilation toolchain takes LLVM IR as its input, optimizes and compiles the IR into CSA assembly, for example, adding appropriate buffering for performance on latency-insensitive channels. It then places and routes the CSA assembly onto the hardware architecture and configures the PEs and network for execution. In one embodiment, the toolchain supports CSA-specific compilation as just-in-time (JIT) compilation, thereby incorporating potential runtime feedback from actual execution. One of the key design features of the framework is compiling (LLVM) IR to obtain CSA, rather than using a higher-level language as input. While programs written in high-level programming languages specifically designed for CSA can achieve the highest performance and / or energy efficiency, adopting new high-level languages or programming frameworks can be slow and limited in practice due to the difficulty of converting existing code bases. Using (LLVM) IR as input enables a wide range of existing programs to potentially execute on the CSA, e.g. without the need to create a new language, nor to significantly modify the front end of a new language that one wants to run on the CSA.

[0278] 4.2 Prototype Compiler

[0279] Figure 33The figure shows a compiler 3300 for an accelerator according to an embodiment of the present disclosure. The compiler 3300 initially focuses on ahead-of-time compilation of C or C++ via a front-end (e.g., Clang). To compile (LLVM) IR, the compiler implements the CSA backend target within LLVM using three main stages. First, the CSA backend reduces the LLVM IR to target-specific machine instructions for a serialization unit, which implements most CSA operations as well as a traditional RISC-like control flow architecture (e.g., using branches and a program counter). The serialization unit in the toolchain can serve as a useful aid for both the compiler and application developers because it allows incremental transformation from control flow (CF) to data flow (DF), for example, converting a code segment at a certain moment from control flow to data flow and verifying program correctness. The serialization unit can also provide a model for handling code that does not fit in a spatial array. The compiler then converts these control flow instructions into data flow operators (e.g., code) for the CSA. This stage is described later in Section 4.3. The CSA backend can then run its own optimization rounds on the data flow instructions. Finally, the compiler can dump the instructions in CSA assembly format. This assembly format is taken as input to the subsequent tools, which place and route the data flow instructions on the actual CSA hardware.

[0280] 4.3 Control to Data Flow Conversion

[0281] The key part of the compiler can be implemented in the control-dataflow conversion pass (or simply the dataflow conversion pass). This pass takes a function expressed in control flow form, such as a control flow graph (CFG) with serialized machine instructions that operate on virtual registers, and converts it into a dataflow function, which is conceptually a graph of dataflow operations (instructions) connected by latency-insensitive channels (LICs). This section gives a high-level description of this pass, describing how it conceptually handles memory operations, branches, and loops in some embodiments.

[0282] Straight Line Code

[0283] Figure 34A Serialized assembly code 3402 is illustrated according to an embodiment of the present disclosure. Figure 34B FIG. 1 shows an embodiment of the present disclosure. Figure 34A The data flow assembly code 3404 of the serialization assembly code 3402. Figure 34C FIG. 1 illustrates a method for an accelerator according to an embodiment of the present disclosure. Figure 34B A data flow graph 3406 of the data flow assembly code 3404 is shown.

[0284] First, consider the simple case of converting a straight line serialization code into a data flow. The data flow conversion pass can convert the basic serialization code block (such as Figure 34A ) is converted to the CSA assembly code shown in FIG34B . Conceptually, Figure 34B The CSA assembly representation in Figure 34C . In this example, each serializing instruction is converted to matching CSA assembly. (For example, the .lic declaration for data declares a latency-insensitive lane corresponding to a virtual register (e.g., Rdata) in the serializing code. In practice, the input to the dataflow conversion pass can be in numbered virtual registers. However, for clarity, this section uses descriptive register names. Note that in this embodiment, load and store operations are supported in the CSA architecture, allowing for many more program runs than architectures that only support pure dataflow. Because the serializing code input to the compiler is in SSA (single static assignment) form, for simple basic blocks, the control-dataflow pass can convert each virtual register definition into the generation of a single value on the latency-insensitive lane. SSA form allows multiple uses of a single definition of a virtual register (such as in Rdata2). To support this model, the CSA assembly code supports multiple uses of the same LIC (e.g., data2), and the simulator implicitly creates the necessary copies of the LIC. A key difference between serialization code and dataflow code is the handling of memory operations. Figure 34A The code in is conceptually serial, meaning that in the case where the addresses of addr and addr3 overlap, the load32 (ld32) of addr3 should appear to occur after the st32 of addr.

[0285] branch

[0286] To convert a program with multiple basic blocks and conditional statements into dataflow, the compiler generates special dataflow operators to replace branches. More specifically, the compiler uses switch operators to direct outgoing data at the end of a basic block in the original CFG, and uses pick operators to select values from the appropriate incoming channels at the beginning of a basic block. As a concrete example, consider Figures 35A-35C The code and corresponding data flow graph in , which conditionally computes the value of y based on the following inputs: a, i, x, and n. After computing the branch conditional test, the data flow code uses the switch operator (see, for example, Figure 20B-Figure 20C ) to: if the test is 0, direct the value in channel x to channel xF, or if the test is 1, direct the value in channel x to channel xT. Similarly, the pick operator (see, for example, Figure 20B-Figure 20C) is used to send channel yF to y if the test is 0, or to send channel yT to y if the test is 1. In this example, it is demonstrated that even if the value of a is only used in the true branch of the conditional statement, the CSA will include a switch operator that directs the value of a to channel aT when the test is 1, and consumes (devours) the value when the test is 0. The latter case is expressed by setting the false output of the switch to %ign. Simply connecting the channel directly to the true path may not be correct because, in the event that execution actually takes the false path, the value of "a" will be left in the graph, resulting in an incorrect value of a for the next execution of the function. This example highlights the property of control equivalence, which is a key property in embodiments of correct dataflow transformations.

[0287] Control Equivalence : Consider a single-entry, single-exit control flow graph G with two basic blocks A and B. If all completion control flow paths through G visit A and B the same number of times, then A and B are control equivalent.

[0288] LIC replacement In a control flow graph G, suppose an operation in basic block A defines a virtual register x and an operation in basic block B uses x. Then a correct control-data flow transformation can replace x with a latency-insensitive channel only if A and B are control equivalent. The control equivalence relation partitions the basic blocks of the CFG into regions of strong control dependencies. Figure 35A Illustrated is C source code 3502 according to an embodiment of the present disclosure. Figure 35B FIG. 1 shows an embodiment of the present disclosure. Figure 35A The data flow assembly code 3504 of the C source code 3502. Figure 35C A data flow diagram 3506 is shown for the data flow assembly code 3504 of FIG. 35B according to an embodiment of the present disclosure. Figures 35A-35C In the example, the basic blocks before and after the conditional statement are control-equivalent to each other, but the basic blocks in the true and false paths are each located in their control dependency regions. A correct algorithm for converting a CFG to dataflow is for the compiler to: (1) insert switches to compensate for the mismatch in execution frequency for any values flowing between basic blocks that are not control-equivalent; and (2) insert pickers at the beginning of the basic block to correctly select from any incoming value to the basic block. Generating appropriate control signals for these pickers and switches can be a key part of the dataflow conversion.

[0289] cycle

[0290] Another important class of CFGs in dataflow transformations is the CFG for single-entry, single-exit loops, which is a common form of loop generated in (LLVM) IR. These loops can be almost acyclic except for a single backedge from the end of the loop back to the loop header block. Dataflow transformation passes can use the same high-level strategies to transform loops as for branches, for example, dataflow transformation passes insert switches at the end of the loop to direct values out of the loop (either out of the loop exit or around the backedge to the beginning of the loop), and insert pickers at the beginning of the loop to select between the initial value entering the loop and the value arriving via the backedge. Figure 36A Illustrated is C source code 3602 according to an embodiment of the present disclosure. Figure 36B FIG. 1 shows an embodiment of the present disclosure. Figure 36A C source code 3604 and data flow assembly code 3602. Figure 36C FIG. 1 shows an embodiment of the present disclosure. Figure 36B Data flow diagram 3600 of data flow assembly code 3604. Figure 36A - Figure 36C The C and CSA assembly code for an example do-while loop that adds up the values of the loop induction variable i, and the corresponding data flow diagram are shown. For each variable (i and sum) that conceptually loops around the loop, the diagram has a corresponding pick / switch pair that controls the flow of these values. Note that even though n is a loop invariant, this example uses a pick / switch pair to loop the value of n around the loop. This duplication of n enables the virtual register for n to be converted into the LIC because it matches the execution frequency between the conceptual definition of n outside the loop and the one or more uses of n inside the loop. Generally speaking, to achieve correct data flow conversion, when registers are converted into the LIC, registers that live in the loop will be repeated once for each iteration inside the loop body. Similarly, registers that are updated within the loop and live out of the loop will be consumed (e.g., with a single final value that is sent out of the loop). Looping introduces folds into the data flow conversion process, that is, the control for the pick at the top of the loop and the switch at the bottom of the loop are offset. For example, if Figure 35A Executes three iterations and exits, then the control for the picker should be 0, 1, 1, and the control for the switch should be 1, 1, 0. This control is achieved by starting the picker channel with an initial extra 0 when the function begins at loop 0 (which is specified in the assembly by the directives .value 0 and .avail 0), and then copying the output switch into the picker. Note that the last 0 in the switch restores the final 0 to the picker, ensuring that the final state of the dataflow graph matches its initial state.

[0291] Figure 37A 3700 is a flowchart according to an embodiment of the present disclosure. The flowchart 3700 includes: 3702: decoding an instruction into a decoded instruction using a decoder of a core of a processor; 3704: executing the decoded instruction using an execution unit of the core of the processor to perform a first operation; 3706: receiving an input of a dataflow graph including a plurality of nodes; 3708: overlaying the dataflow graph onto a plurality of processing elements of the processor and an interconnection network between the plurality of processing elements of the processor, with each node represented as a dataflow operator in the plurality of processing elements; and 3710: performing a second operation of the dataflow graph using the interconnection network and the plurality of processing elements when a corresponding incoming operand set arrives at each of the dataflow operators of the plurality of processing elements.

[0292] Figure 37B 3701 illustrates a flowchart according to an embodiment of the present disclosure. The depicted process includes: 3703: receiving an input of a dataflow graph comprising a plurality of nodes; 3705: overlaying the dataflow graph onto a plurality of processing elements of a processor, a data path network between the plurality of processing elements, and a flow control path network between the plurality of processing elements, with each node represented as a dataflow operator in the plurality of processing elements.

[0293] In one embodiment, the core writes commands to a memory queue, and the CSA (e.g., multiple processing elements) monitors the memory queue and begins execution when the command is read. In one embodiment, the core executes a first portion of a program, and the CSA (e.g., multiple processing elements) executes a second portion of the program. In one embodiment, while the CSA is executing operations, the core performs other work.

[0294] 5. CSA Advantages

[0295] In certain embodiments, CSA architectures and microarchitectures offer profound energy, performance, and usability advantages over roadmap processor architectures and FPGAs. In this section, these architectures are compared with embodiments of CSAs, highlighting the superiority of CSAs over each in accelerating parallel dataflow graphs.

[0296] 5.1 Processor

[0297] Figure 38 Graph 3800 illustrating throughput versus energy per operation according to an embodiment of the present disclosure. Figure 38As shown in [1], small cores are generally more energy efficient than large cores, and in some workloads, this advantage can be translated into absolute performance through higher core counts. The CSA microarchitecture follows these observations to their conclusion and removes (e.g., most) of the energy-hungry control structures associated with the von Neumann architecture (including most in the instruction-side microarchitecture). By removing this overhead and implementing simple single-operation PEs, embodiments of the CSA achieve dense, space-efficient arrays. Unlike small cores, which are typically very serial, a CSA can cluster its PEs together, for example, via a circuit-switched local network, to form an explicitly parallel aggregate dataflow graph. The result is performance not only in parallel applications, but also in serial applications. Unlike cores, which are expensive in terms of area and energy, CSAs are already parallel in their native execution model. In certain embodiments, CSAs require neither speculation to improve performance nor the need to repeatedly re-extract parallelism from a serialized program representation, thereby avoiding two of the main energy burdens of the von Neumann architecture. Most structures in CSA embodiments are distributed, small, and energy-efficient, in contrast to the centralized, bulky, and energy-hungry structures found in cores. Consider the case of registers in a CSA: Each PE may have a few (e.g., 10 or fewer) storage registers. Individually, these registers can be more efficient than a traditional register bank. When aggregated, these registers can provide the effect of a register bank in a larger architecture. As a result, CSA embodiments avoid most of the stack overflows and fill-ups caused by classic architectures, while using much less energy for each state access. Of course, applications can still access memory. In CSA embodiments, memory access requests and responses are architecturally decoupled, allowing workloads to maintain many more pending memory accesses per unit of area and energy. This property enables significantly higher performance for cache-bound workloads and reduces the area and energy required to saturate main memory in memory-bound workloads. CSA embodiments expose new forms of energy efficiency that are unique to non-von Neumann architectures. One consequence of executing a single operation (e.g., instruction) at (e.g., most) PEs is reduced operand entropy. In the case of incremental operation, each execution results in a small number of circuit-level switches and very little energy consumption, a situation examined in detail in Section 6.2. In contrast, von Neumann is multiplexed, resulting in a large number of bit transitions. The asynchronous style of CSA embodiments also enables microarchitectural optimizations, such as the floating-point optimizations described in Section 3.5, that are difficult to implement in a tightly scheduled core pipeline. Because PEs can be relatively simple and their behavior in a particular dataflow graph is statically known, clock gating and power gating techniques can be employed more efficiently than in coarser architectures.The graph execution style, small size, and scalability of embodiments of CSAs, PEs, and networks collectively enable the expression of many types of parallelism: instruction, data, pipeline, vector, memory, thread, and task parallelism can all be implemented. For example, in a CSA embodiment, one application can use the arithmetic units to provide high levels of address bandwidth, while another application can use those same units for computation. In many cases, multiple types of parallelism can be combined to achieve even higher performance. Many key HPC operations can be both replicated and pipelined, resulting in performance gains of multiple orders of magnitude. In contrast, von Neumann cores are typically optimized for a single parallelism style carefully chosen by the architect, resulting in an inability to capture all important application kernels. Precisely because CSA embodiments expose and facilitate many forms of parallelism, they do not mandate that specific forms of parallelism, or worse, specific subroutines, exist in an application to benefit from the CSA. For example, many applications (including single-stream applications) can gain both performance and energy benefits from CSA embodiments even when compiled without modification. This goes against the long-standing trend of requiring significant programmer effort to achieve significant performance gains in single-stream applications. In fact, in some applications, embodiments of the CSA achieve more performance from functionally equivalent, but less "modern," code than from its complex contemporary counterparts that have been hardened to target vector instructions.

[0298] 5.2 Comparison between CSA Implementation and FPGA

[0299] The choice of dataflow operators as the fundamental architecture of CSA embodiments distinguishes those CSAs from FPGAs, specifically as superior accelerators for HPC dataflow graphs generated from traditional programming languages. Dataflow operators are fundamentally asynchronous. This allows CSA embodiments not only to have implementation freedom in the microarchitecture, but also to adapt CSA embodiments to abstract architectural concepts simply and elegantly. For example, CSA embodiments naturally adapt to many memory microarchitectures, which are fundamentally asynchronous, using a simple load-store interface. One need only examine FPGA DRAM controllers to appreciate the difference in replication. CSA embodiments also exploit asynchrony to provide faster and more fully featured runtime services like configuration and extraction, believed to be 4-6 orders of magnitude faster than FPGA blocks. By narrowing the architectural interface, CSA embodiments provide control over most timing paths at the microarchitecture level. This allows CSA embodiments to operate at a much higher frequency than the more general control mechanisms provided in FPGAs. Similarly, clocks and resets, which may be architecturally fundamental to an FPGA, are microarchitectural in a CSA, eliminating, for example, the need to support clocks and resets as programmable entities. Dataflow operators can be coarse-grained for the most part. By performing processing only in coarse operators, embodiments of a CSA improve both the density of the structure and its energy consumption. The CSA performs operations directly rather than emulating them using lookup tables. A second consequence of coarseness is that the placement and routing problem is simplified. CSA dataflow graphs are many orders of magnitude smaller than FPGA netlists, and in embodiments of a CSA, placement and routing times are correspondingly reduced. The significant differences between embodiments of a CSA and an FPGA make the CSA superior as an accelerator, for example, for dataflows generated from traditional programming languages.

[0300] 6. Evaluation

[0301] CSAs are a novel computer architecture that offer significant performance and energy advantages over roadmap processors. Consider the case of computing a single-stride address for a walk across an array. This case can be important in HPC applications (e.g., where a large amount of integer work is spent computing address offsets). In address computations, especially strided address computations, one argument is constant for each computation, and the other argument changes only slightly. Therefore, in most cases, only a few bits switch per cycle. In fact, using a derivation similar to the constraint on the floating-point carry bit described in Section 3.5, it can be shown that for strided computations, on average, fewer than two input bits switch per computation, resulting in a 50% energy reduction for a random switching distribution. Many of these energy savings are lost if time multiplexing is used. In one embodiment, CSAs achieve approximately 3x (three times) the energy efficiency of a core while achieving an 8x (eight times) performance gain. The parallelism gains achieved by CSA embodiments result in reduced program runtimes, resulting in correspondingly significant leakage energy reductions. At the PE level, CSA embodiments are extremely energy efficient. The second important question for CSA is whether it uses a reasonable amount of energy at the chip level. Since CSA embodiments can exercise every floating-point PE in the fabric every cycle, it serves as a reasonable upper bound on energy and power consumption, for example, making most of the energy go into floating-point multiplication and addition.

[0302] 7. Future CSA Details

[0303] This section discusses further details of configuration and exception handling.

[0304] 7.1 Microarchitecture for Configuring CSA

[0305] This section discloses examples of how to configure a CSA (e.g., a structure), how to quickly implement that configuration, and how to minimize the resource overhead of the configuration. Quickly configuring structures is crucial for accelerating small parts of larger algorithms and, therefore, for broadening the applicability of CSAs. This section further discusses features that allow embodiments of a CSA to be programmed with configurations of varying lengths.

[0306] Embodiments of a CSA (e.g., a fabric) may differ from conventional cores in that they may utilize a configuration step in which (e.g., large) portions of the fabric are loaded in advance with program configuration, prior to program execution. An advantage of static configuration may be that very little energy is expended at runtime while configuring, in contrast, for example, to a serializing core that expends energy fetching configuration information (instructions) nearly every cycle. A previous disadvantage of configuration was that it was a coarse-grained step with potentially long latencies that set a lower bound on the size of programs that could be accelerated in the fabric due to the cost of context switches. The present disclosure describes a scalable microarchitecture for rapidly configuring spatial arrays in a distributed manner that, for example, avoids the previous disadvantages.

[0307] As discussed above, a CSA can include lightweight processing elements connected by an inter-PE network. By configuring configurable fabric elements (CFEs) (e.g., PEs and an interconnect (fabric) network), a program, viewed as a control-data flow graph, is then mapped onto the architecture. Generally speaking, a PE can be configured as a dataflow operator, and once all input operands arrive at a PE, some operation occurs, and the result is forwarded to another PE or PEs for consumption or output. PEs can communicate via dedicated virtual circuits, formed by statically configuring a circuit-switched communication network. These virtual circuits can be flow-controlled and fully backpressured, so that, for example, if the source has no data or the destination is full, the PE will stall. At runtime, data can flow through the PEs implementing the mapped algorithm. For example, data can flow from memory through the fabric and then outbound back to memory. This type of spatial architecture can achieve superior performance efficiency relative to traditional multi-core processors: in contrast to extended memory systems, computation can be simpler and more numerous in PE form than in larger cores, and communication can be direct.

[0308] Embodiments of a CSA may not utilize (e.g., software-controlled) packet switching (e.g., packet switching that requires significant software assistance to implement), which slows down configuration. Embodiments of a CSA include out-of-band signaling in the network (e.g., only 2-3 bits of out-of-band signaling, depending on the supported feature set) and a fixed configuration topology to avoid the need for extensive software support.

[0309] A key difference between CSA embodiments and the approach used in FPGAs is that the CSA approach can use wide data words, is distributed, and includes mechanisms for fetching program data directly from memory. CSA embodiments can avoid utilizing JTAG-style single-bit communication for area efficiency, for example because that can require several milliseconds to fully configure a large FPGA fabric.

[0310] Embodiments of the CSA include a distributed configuration protocol and a microarchitecture to support this protocol. Initially, the configuration state may reside in memory. Multiple (e.g., distributed) local configuration controllers (LCCs) can stream portions of the overall program to their local regions in the spatial fabric, for example, using a combination of a small set of control signals and a fabric-provided network. State elements can be used at each CFE to form a configuration chain, for example, allowing each CFE to self-program without global addressing.

[0311] Embodiments of the CSA include specific hardware support for forming configuration chains, e.g., rather than software that dynamically builds these chains at the expense of increased configuration time. Embodiments of the CSA are not purely packet-switched and do include additional out-of-band control lines (e.g., control is not sent over the data path, requiring additional cycles to gate and reserialize this information). Embodiments of the CSA reduce configuration latency (e.g., by at least half) by fixing configuration ordering and by providing explicit out-of-band control, while not significantly increasing network complexity.

[0312] Embodiments of the CSA do not use serial configuration for configurations where data is streamed into the fabric bit by bit using a JTAG-like protocol. Embodiments of the CSA utilize a coarse-grained fabric approach. In some embodiments, adding some control lines or status elements to a 64-bit or 32-bit oriented CSA fabric is less costly than adding those same control mechanisms to a 4-bit or 6-bit fabric.

[0313] Figure 39 An accelerator slice 3900 according to an embodiment of the present disclosure is illustrated, comprising an array of processing elements (PEs) and local configuration controllers 3902, 3906. Each PE, each network controller (e.g., a network data flow endpoint circuit), and each switch may be a configurable fabric element (CFE), for example, configured (e.g., programmed) by an embodiment of a CSA architecture.

[0314] An embodiment of a CSA includes hardware that provides efficient, distributed, low-latency configuration of heterogeneous spatial structures. This can be achieved based on four techniques. First, by utilizing a hardware entity (a local configuration controller (LCC)), such as Figures 39-41As shown in . LCCs can fetch a stream of configuration information from (e.g., virtual) memory. Second, a configuration data path can be included, for example, that is as wide as the native width of the PE structure and can be overlaid on top of the PE structure. Third, new control signals can be received into the PE structure that orchestrate the configuration process. Fourth, a state element can be located (e.g., in a register) at each configurable endpoint that tracks the state of adjacent CFEs, allowing each CFE to explicitly self-configure without the need for additional control signals. These four microarchitectural features allow a CSA to configure its chain of CFEs. To achieve low configuration latency, the configuration can be partitioned by establishing many LCCs and CFE chains. During configuration, these can operate independently to load structures in parallel, thereby dynamically reducing latency, for example. As a result of these combinations, structures configured using embodiments of the CSA architecture can be fully configured (e.g., in a few hundred nanoseconds). Detailed operation of various components of embodiments of the CSA configuration network is disclosed below.

[0315] Figures 40A-40C 4002 is shown configuring a datapath network according to an embodiment of the present disclosure. The depicted network includes a plurality of multiplexers (e.g., multiplexers 4006, 4008, 4010) that can be configured (e.g., via their respective control signals) to connect one or more datapaths (e.g., from PEs) together. Figure 40A Illustrated is a network 4000 (eg, structure) in some prior operational or procedural configuration (eg, setup). Figure 40B A local configuration controller 4002 (e.g., including network interface circuitry 4004 for sending and / or receiving signals) is shown gating configuration signals, and the local network is set to a default configuration (e.g., as depicted in the figure) that allows the LCC to send configuration data to all configurable fabric elements (CFEs) (e.g., muxes). Figure 40C The LCC is shown strobing configuration information across the network to configure the CFEs in a predetermined (e.g., silicon-defined) sequence. In one embodiment, when the CFEs are configured, they can begin operation immediately. In another embodiment, the CFEs wait to begin operation until the fabric has been fully configured (e.g., for each local configuration controller, by a configuration terminator (e.g., Figure 42 In one embodiment, the LCC gains control of the network fabric by sending a special message or driving a signal. It then gates the configuration data to the CFEs in the fabric (e.g., over a period of many cycles). In these figures, the multiplexer network is not identical to that in some figures (e.g., Figure 23 ) is similar to the “switching device” shown in ).

[0316] Local Configuration Controller

[0317] Figure 41 41. The diagram shows a (e.g., local) configuration controller 4102 according to an embodiment of the present disclosure. A local configuration controller (LCC) can be a hardware entity responsible for: loading local portions of a fabric program (e.g., in a subset of a slice or elsewhere); interpreting these program portions; and subsequently loading these program portions into the fabric by driving appropriate protocols over various configuration lines. In this capacity, an LCC can be a dedicated serialized microcontroller.

[0318] When an LCC operation receives a pointer to a code segment, it can begin. Depending on the LCB microarchitecture, this pointer (e.g., stored in pointer register 4106) arrives at the LCC either over the network (e.g., from within the CSA (structure) itself) or through a memory system access. When the LCC receives such a pointer, it can optionally drain the relevant state from its portion of the structure used for context storage and then proceed to immediately reconfigure the portion of the structure that the LCC is responsible for. The program loaded by the LCC can be a combination of configuration data for the structure and control commands for the LCC, for example, the configuration data and the control commands are lightly encoded. When the LCC has the program portion streamed in, it can interpret the program as a command stream and perform the appropriate encoded actions to configure (e.g., load) the structure.

[0319] exist Figure 39 , two different microarchitectures for LCCs are shown, one or both of which may be used in a CSA, for example. The first microarchitecture places LCC 3902 at the memory interface. In this case, the LCC can make direct requests to the memory system to load data. In the second case, LCC 3906 is placed on a memory network where it can only make indirect requests to memory. In both cases, the logical operation of the LCB remains unchanged. In one embodiment, the LCC is notified of the program to be loaded, for example, by a set of control status registers (e.g., visible to the OS), which are used to notify each LCC of a new program pointer, etc.

[0320] Additional out-of-band control channels (e.g., wires)

[0321] In some embodiments, configuration relies on 2-8 additional out-of-band control channels to improve configuration speed, as defined below. For example, the configuration controller 4102 may include the following control channels: e.g., CFG_START control channel 4108, CFG_START_PRIVILEDGE control channel 4109, CFG_VALID control channel 4110, and CFG_DONE control channel 4112, examples of each of which are discussed below in Table 2.

[0322] Table 2: Control channels

[0323]

[0324]

[0325] In general, the handling of the configuration information may be left to the implementer of the particular CFE. For example, a selectable-function CFE may have provisions to set registers using existing data paths, whereas a fixed-function CFE may simply set configuration registers.

[0326] Due to the long line delays when programming large sets of CFEs, the CFG_VALID signal can be considered a clock / latch enable for the CFE components. Because this signal is used as a clock, in one embodiment, the line's duty cycle is at most 50%. As a result, configuration throughput is approximately halved. Optionally, a second CFG_VALID signal can be added to allow continuous programming.

[0327] In one embodiment, only CFG_START is strictly passed on standalone coupling devices (eg, wires), for example, CFG_VALID and CFG_DONE may be overlaid on top of other network coupling devices.

[0328] Reuse of network resources

[0329] To reduce configuration overhead, certain embodiments of the CSA leverage existing network infrastructure to transfer configuration data. LCCs can leverage both the chip-level memory hierarchy and the fabric-level communication network to move data from storage to the fabric. As a result, in certain embodiments of the CSA, the configuration infrastructure adds no more than 2% to the total fabric area and power.

[0330] Reuse of network resources in certain embodiments of the CSA can enable networks with some hardware support for configuration mechanisms. Circuit-switched networks of embodiments of the CSA cause the LCC to set up their multiplexers in a specific manner for configuration when the 'CFG_START' signal is asserted. Packet-switched networks do not require extensions, but LCC endpoints (e.g., configuration terminators) use specific addresses in packet-switched networks. Network reuse is optional, and some embodiments may find a dedicated configuration bus more convenient.

[0331] Each CFE status

[0332] Each CFE may maintain a bit indicating whether it has been configured (see, for example, Figure 30 ). This bit may be de-asserted when the configuration start signal is driven, and subsequently asserted once a particular CFE has been configured. In one configuration protocol, the CFEs are arranged in a chain, and the CFE and configuration status bits determine the topology of the chain. A CFE may read the configuration status bits of an immediately adjacent CFE. If the adjacent CFE is configured and the current CFE is not configured, then the CFE and any current configuration data is determined to be for the current CFE. When the 'CFG_DONE' signal is asserted, the CFE may set its configuration bits to, for example, enable the upstream CFE to be configured. As a base case for the configuration process, the CFE asserts its configured configuration terminator (e.g., in Figure 39 A configuration terminator 3904 for LCC 3902 or a configuration terminator 3908 for LCC 3906) may be included at the end of the chain.

[0333] Within the CFE, this bit can be used to drive flow control ready signals. For example, when the configuration bit is deasserted, network control signals can be automatically clamped to values that prevent data flow, and within the PE, no operations or other actions will be scheduled.

[0334] Handling high-latency configuration paths

[0335] An embodiment of an LCC may, for example, drive signals over long distances through many multiplexers and utilize numerous loads. Therefore, it may be difficult for signals to reach the remote CFE within a short clock cycle. In some embodiments, configuration signals are at a certain frequency division (e.g., fractional) of the master (e.g., CSA) clock signal to ensure digital timing compliance during configuration. Clock division can be used in out-of-band signaling protocols and does not require any modifications to the master clock tree.

[0336] Ensure consistent structural behavior during configuration

[0337] Because some configuration schemes are distributed and because program and memory effects have non-deterministic timing, different parts of the fabric may be configured at different times. As a result, certain embodiments of the CSA provide mechanisms for preventing inconsistent operation between configured and unconfigured CFEs. In general, consistency is considered a property that is required and maintained by the CFE itself, for example using internal CFE state. For example, when a CFE is in an unconfigured state, it may declare its input buffers to be full and its outputs to be invalid. When configured, these values will be set to the true states of the buffers. As enough parts of the fabric come out of configuration, these techniques can allow the fabric to begin operation. This has the effect of further reducing context switch latency, for example, if long latency memory requests are issued early.

[0338] Variable width configuration

[0339] Different CFEs may have different configuration word widths. For smaller CFE configuration words, the implementer can balance delays by fairly assigning CFE configuration loading across network lines. To balance the loading on the network lines, one option is to assign configuration bits to different parts of the network lines to limit the net delay on any one line. Wide data words can be handled by using serialization / deserialization techniques. These decisions can be taken on a fabric-by-fabric basis to optimize the behavior of a particular CSA (e.g., a fabric). A network controller (e.g., one or more of network controller 3910 and network controller 3912) can communicate with each domain (e.g., a subset) of a CSA (e.g., a fabric) to, for example, send configuration information to one or more LCCs. The network controller can be part of a communications network (e.g., separate from a circuit-switched network). The network controller can include network data flow endpoint circuits.

[0340] 7.2 Microarchitecture for Low-Latency Configuration of CSAs and Timely Fetching of CSA Configuration Data

[0341] Embodiments of a CSA can be an energy-efficient and high-performance means of accelerating user applications. When considering whether a program (e.g., a data flow graph of a program) can be successfully accelerated by an accelerator, both the time used to configure the accelerator and the time used to run the program can be considered. If the run time is short, the configuration time will play a large role in determining successful acceleration. Therefore, in order to maximize the domain of accelerable programs, in some embodiments, the configuration time is made as short as possible. One or more configuration caches can be included in the CSA, for example to enable high-bandwidth, low-latency storage to achieve rapid reconfiguration. What follows is a description of several embodiments of the configuration cache.

[0342] In one embodiment, during configuration, the configuration hardware (e.g., LCC) may optionally access a configuration cache to obtain new configuration information. The configuration cache may operate as either a traditional address-based cache or in an OS-managed mode where the configuration is stored in a local address space and addressed by referencing that address space. If the configuration state is in the cache, then in some embodiments, no request to the backing store will be made. In some embodiments, the configuration cache is separate from any (e.g., lower-level) shared caches in the memory hierarchy.

[0343] Figure 42 An accelerator slice 4200 according to an embodiment of the present disclosure is illustrated, comprising an array of processing elements, a configuration cache (e.g., 4218 or 4220), and a local configuration controller (e.g., 4202 or 4206). In one embodiment, configuration cache 4214 is co-located with local configuration controller 4202. In one embodiment, configuration cache 4218 is located within a configuration domain of local configuration controller 4206, e.g., a first domain terminates at configuration terminator 4204 and a second domain terminates at configuration terminator 4208. The configuration cache may allow the local configuration controller to reference the configuration cache during configuration, e.g., to obtain configuration state with lower latency than referencing memory. The configuration cache (storage) may be either dedicated or accessible as a configuration mode within a storage element (e.g., local cache 4216) within the fabric.

[0344] Cache Mode

[0345] 1. Demand Caching - In this mode, the configuration cache operates as a true cache. The configuration controller issues address-based requests, which are checked against the tags in the cache. Misses can be loaded into the cache and can then be referenced during future reprogramming.

[0346] 2. In-Fabric Storage (Scratchpad) Cache - In this mode, the configuration cache receives references to configuration sequences in its own small address space rather than the host's larger address space. This can improve memory density because the portion of the cache used to store tags can instead be used to store configurations.

[0347] In some embodiments, the configuration cache may have configuration data preloaded therein (e.g., via external or internal instructions). This may allow for a reduction in latency for loading programs. Certain embodiments herein provide a structure for accessing the configuration cache that, for example, allows new configuration states to be loaded into the cache even while a configuration is already running within the structure. The initiation of this loading may occur from an internal or external source. Embodiments of the preloading mechanism further reduce latency by removing latency from cache loads of the configuration path.

[0348] Prefetch Mode

[0349] 1. Explicit Prefetch – The configuration path is augmented with a new command, ConfigurationCachePrefetch. Instead of programming the structure, this command simply causes the relevant program configuration to be loaded into the configuration cache. Because this mechanism piggybacks on the existing configuration infrastructure, it is exposed both within the structure and externally to cores and other entities accessing memory space, for example.

[0350] 2. Implicit prefetching - The global configuration controller may maintain a prefetch predictor and use it to initiate (eg, in an automated manner) explicit prefetches of the configuration cache.

[0351] 7.3 Hardware for Rapid Reconfiguration of CSA in Response to Exceptions

[0352] Certain embodiments of CSAs (e.g., space structures) include a large number of instructions and configuration states, for example, which are largely static during operation of the CSA. Consequently, the configuration states may be susceptible to soft errors. Rapid and error-free recovery from these soft errors may be critical to the long-term reliability and performance of space systems.

[0353] Certain embodiments herein provide a fast configuration recovery cycle, e.g., in which a configuration error is detected and portions of a structure are immediately reconfigured. Certain embodiments herein include, for example, a configuration controller with a reliability, availability, and durability (RAS) reprogramming feature. Certain embodiments of the CSA include circuitry for high-speed configuration, error reporting, and parity checking within a spatial structure. Using a combination of these three features and an optional configuration cache, the configuration / exception handling circuitry can recover from soft errors in the configuration. When detected, a soft error can be transferred to a configuration cache, which initiates an immediate reconfiguration of the structure (e.g., that portion of the structure). Certain embodiments provide dedicated reconfiguration circuitry, e.g., that is faster than any solution that would be implemented indirectly in the structure. In certain embodiments, the exception and configuration circuitry located together collaborate to reload the structure upon a configuration error detection.

[0354] Figure 43 The diagram illustrates an accelerator slice 4300 according to an embodiment of the present disclosure, comprising an array of processing elements and configuration and exception handling controllers 4302 and 4306 with reconfiguration circuitry 4318 and 4322. In one embodiment, when a PE detects a configuration error through its RAS feature, it sends a message (e.g., a configuration error or a reconfiguration error) to the configuration and exception handling controller (e.g., 4302 or 4306) via its exception generator. Upon receiving this message, the configuration and exception handling controller (e.g., 4302 or 4306) activates co-located reconfiguration circuitry (e.g., 4318 or 4322, respectively) to reload the configuration state. The microarchitecture continues to configure and (e.g., only) reloads the configuration state, and in some embodiments, only reloads the configuration state for the PE reporting the RAS error. After reconfiguration is complete, the architecture can resume normal operation. To reduce latency, the configuration state used by the configuration and exception handling controller (e.g., 4302 or 4306) can be sourced from a configuration cache. As a base case of the configuration or reconfiguration process, the configuration terminator (e.g., Figure 43 Configuration terminator 4304 for configuration and exception handling controller 4302 or configuration terminator 4308 for configuration and exception handling controller 4306) may be included at the end of the chain.

[0355] Figure 44 Reconfiguration circuitry 4418 is illustrated in accordance with an embodiment of the present disclosure. Reconfiguration circuitry 4418 includes a configuration state register 4420 for storing a configuration state (or a pointer to the configuration state).

[0356] 7.4 Hardware for fabric-initiated reconfiguration of CSA

[0357] Some parts of an application for a CSA (e.g., a spatial array) may be run infrequently or may be mutually exclusive with respect to other parts of the program. To save area, to improve performance and / or to reduce power, it may be useful to time-multiplex multiple parts of a spatial structure between several different parts of a program data flow graph. Certain embodiments herein include an interface through which a CSA (e.g., via a spatial program) may request that part of the structure be reprogrammed. This may enable the CSA to dynamically change itself based on dynamic control flow. Certain embodiments herein may allow structure-initiated reconfiguration (e.g., reprogramming). Certain embodiments herein provide a set of interfaces for triggering configuration from within a structure. In some embodiments, a PE issues a reconfiguration request based on a decision in the program data flow graph. The request may travel over the network to our new configuration interface, where it triggers the reconfiguration. Once the reconfiguration is complete, a message notifying the completion may optionally be returned. Certain embodiments of the CSA therefore provide program (e.g., data flow graph)-guided reconfiguration capabilities.

[0358] Figure 45 An accelerator slice 4500 is shown, according to an embodiment of the present disclosure, comprising an array of processing elements and a configuration and exception handling controller 4506 with reconfiguration circuitry 4518. Here, a portion of the fabric issues a request for (re)configuration to a configuration domain, such as the configuration and exception handling controller 4506 and / or the reconfiguration circuitry 4518. The domain (re)configures itself, and when the request is satisfied, the configuration and exception handling controller 4506 and / or the reconfiguration circuitry 4518 issues a response to the fabric, notifying the fabric of the completion of the (re)configuration. In one embodiment, the configuration and exception handling controller 4506 and / or the reconfiguration circuitry 4518 disables communication while the (re)configuration is in progress, so that during operation, there are no program consistency issues.

[0359] Configuration Mode

[0360] Configuration by Address - In this mode, the fabric makes a direct request to load configuration data from a specific address.

[0361] Configuration by reference - In this mode, the structure makes a request to load a new configuration, for example, by a predefined reference ID. This simplifies the determination of the code to load because the location of the code has been abstracted.

[0362] Configuring multiple domains

[0363] The CSA may include a higher-level configuration controller to support a multicast mechanism to broadcast configuration requests to multiple (e.g., distributed or local) configuration controllers (e.g., via a network indicated by a dashed box). This can enable a single configuration request to be replicated across multiple, larger portions of the fabric, for example, to trigger a wide reconfiguration.

[0364] 7.5 Exception Aggregator

[0365] Certain embodiments of the CSA may also experience exceptions (e.g., exceptional conditions), such as floating point underflow. When these conditions occur, special handling routines may be called to either correct the program or terminate it. Certain embodiments herein provide a system-level architecture for handling exceptions in a spatial structure. Because certain spatial structures emphasize area efficiency, embodiments herein minimize the total area while providing a general exception mechanism. Certain embodiments herein provide a low-area means for signaling exceptional conditions occurring in a CSA (e.g., a spatial array). Certain embodiments herein provide interfaces and signaling protocols for communicating such exceptions as well as PE-level exception semantics. Certain embodiments herein are dedicated exception handling capabilities and, for example, do not require explicit handling by the programmer.

[0366] One embodiment of the CSA exception architecture consists of four parts, such as Figure 46-47 These parts can be arranged in a hierarchy where exceptions flow from the generator and eventually flow up to a slice-level exception aggregator (e.g., a handler), which can meet with an exception maintainer, such as a core. The four parts can be:

[0367] 1.PE Exception Generator

[0368] 2. Local abnormal network

[0369] 3. Interlayer Abnormal Aggregator

[0370] 4. Slice-level exception aggregator

[0371] Figure 46 An accelerator slice 4600 is shown including an array of processing elements and a mezzanine anomaly aggregator 4602 coupled to a chip-level anomaly aggregator 4604 in accordance with an embodiment of the present disclosure. Figure 47 A processing element 4700 is shown with an exception generator 4744 in accordance with an embodiment of the present disclosure.

[0372] PE exception generator

[0373] Processing element 4700 may include Figure 26The processing element 2600, for example, similar numbers are similar components, such as local network 2602 and local network 4702. The additional network 4713 (e.g., channel) can be an abnormal network. The PE can be implemented to the abnormal network (e.g., Figure 47 An interface on an abnormal network 4713 (e.g., a channel). For example, Figure 47 The microarchitecture of such an interface is illustrated, wherein a PE has an exception generator 4744 (e.g., to initiate an exception finite state machine (FSM) 4740 to gate an exception packet (e.g., BOXID 4742) out onto an exception network). BOXID 4742 can be a unique identifier for an exception-generating entity (e.g., a PE or block) within a local exception network. When an exception is detected, exception generator 4744 senses the exception network and gates out the BOXID when the network is found to be idle. Exceptions can be caused by many conditions, such as, but not limited to, arithmetic errors, failed ECC checks on state, etc. However, it is also possible to introduce exceptional data flow operations using the idea of supporting constructs such as breakpoints.

[0374] Exceptions can be raised either explicitly through programmer-provided instructions or implicitly when a hardened error condition (e.g., floating-point underflow) is detected. When an exception occurs, PE 4700 may enter a wait state, in which it waits for service by, for example, a final exception handler external to PE 4700. The contents of the exception packet depend on the implementation of the particular PE, as described below.

[0375] Local abnormal network

[0376] The (e.g., local) anomaly network directs anomaly packets from PE 4700 to the mezzanine anomaly network. The anomaly network (e.g., 4713) can be a serial packet-switched network consisting of (e.g., a single control line) and one or more data lines, organized in a ring or tree topology, for example, for a subset of PEs. Each PE can have a (e.g., ring) station in the (e.g., local) anomaly network, at which, for example, the PE can arbitrate to inject messages into the anomaly network.

[0377] PE endpoints that need to inject anomaly packets can observe their local anomaly network exit point. If the control signal indicates busy, the PE will wait to start injecting packets for that PE. If the network is not busy, that is, the downstream station has no packets to forward, the PE will proceed with the injection.

[0378] Network packets can be of variable or fixed length. Each packet can begin with a fixed-length header field that identifies the source PE of the packet. This header field can be followed by a variable number of PE-specific fields containing information such as error codes, data values, or other useful status information.

[0379] Interlayer Abnormal Aggregator

[0380] Mezzanine anomaly aggregator 4604 is responsible for assembling local anomaly networks into larger packets and sending these larger packets to slice-level anomaly aggregator 4602. Mezzanine anomaly aggregator 4604 can prepend local anomaly packets with their own unique IDs, for example, to ensure that anomaly messages are unambiguous. Mezzanine anomaly aggregator 4604 can interface with a special virtual channel in the mezzanine network that is used only for anomalies, for example, to ensure that anomalies are deadlock-free.

[0381] The mezzanine anomaly aggregator 4604 may also be able to maintain certain categories of anomalies directly. For example, configuration requests from the fabric may be distributed out of the mezzanine network using a cache local to the mezzanine network station.

[0382] Slice-level exception aggregator

[0383] The final level of the exception system is the chip-level exception aggregator 4602. The chip-level exception aggregator 4602 is responsible for collecting exceptions from the various mezzanine-level exception aggregators (e.g., 4604) and forwarding them to the appropriate maintenance hardware (e.g., cores). Thus, the chip-level exception aggregator 4602 may include several internal tables and controllers for associating specific messages with handler routines. These tables can be indexed directly or with a small state machine to direct specific exceptions.

[0384] Like the mezzanine exception aggregator, the slice-level exception aggregator can service some exception requests. For example, it can initiate reprogramming of a large portion of the PE structure in response to a specific exception.

[0385] 7.6 Exception Controller

[0386] Certain embodiments of a CSA include extraction controller(s) for extracting data from a fabric. The following discusses embodiments of how to quickly implement this extraction and minimize the resource overhead of data extraction. Data extraction can be used for critical tasks such as exception handling and context switching. Certain embodiments herein extract data from a heterogeneous spatial fabric by introducing features that allow extractable fabric elements (EFEs) (e.g., PEs, network controllers, and / or switches) to have a variable and dynamically variable number of states to be extracted.

[0387] Embodiments of the CSA include a distributed data extraction protocol and a microarchitecture to support this protocol. Certain embodiments of the CSA include multiple local extraction controllers (LECs) that use a combination of a (e.g., small) set of control signals and a fabric-provided network to cause program data to flow from their local regions in the spatial fabric. State elements can be used at each extractable fabric element (EFE) to form an extraction chain, for example, allowing individual EFEs to extract themselves without requiring global addressing.

[0388] Embodiments of the CSA do not use the local network to extract program data. Embodiments of the CSA include, for example, specific hardware support (e.g., extraction controllers) for forming extraction chains and do not rely on software to dynamically establish these chains (e.g., at the expense of increased extraction time). Embodiments of the CSA are not purely packet-switched and do include additional out-of-band control lines (e.g., controls are not sent over the data path, requiring additional cycles to gate and reserialize this information). Embodiments of the CSA reduce extraction latency (e.g., by at least half) by fixing the extraction order and by providing explicit out-of-band control, while not significantly increasing network complexity.

[0389] Embodiments of the CSA do not use a serial mechanism for data extraction where data is streamed bit by bit from the fabric using a JTAG-like protocol. Embodiments of the CSA utilize a coarse-grained fabric approach. In some embodiments, adding some control lines or status elements to a 64-bit or 32-bit oriented CSA fabric is less costly than adding those same control mechanisms to a 4-bit or 6-bit fabric.

[0390] Figure 48 An accelerator slice 4800 according to an embodiment of the present disclosure is illustrated, comprising an array of processing elements and local extraction controllers 4802, 4806. Each PE, each network controller, and each switch may be an extractable fabric element (EFE), for example, configured (e.g., programmed) by an embodiment of the CSA architecture.

[0391] An embodiment of a CSA includes hardware that provides efficient, distributed, low-latency extraction of heterogeneous spatial structures. This can be achieved based on four techniques. First, by utilizing a hardware entity (a local extraction controller (LEC)), such as Figures 48-50As shown in [1], the LEC can accept commands from the host (e.g., a processor core), such as extracting a data stream from a spatial array and writing the data back to virtual memory for inspection by the host. Second, an extraction data path can be included, for example, one that is as wide as the native width of the PE structure and can be overlaid on top of the PE structure. Third, new control signals can be received into the PE structure to orchestrate the extraction process. Fourth, a state element can be located (e.g., in a register) at each configurable endpoint that tracks the state of adjacent EFEs, allowing each EFE to explicitly output its state without requiring additional control signals. These four microarchitectural features allow the CSA to extract data from a chain of EFEs. To achieve low data extraction latency, certain embodiments can partition the extraction problem by including multiple (e.g., many) LECs and EFE chains in the structure. During extraction, these chains can operate independently to extract data from the structure in parallel, thereby, for example, dramatically reducing latency. As a result of these features, the CSA can perform a complete state dump (e.g., in a few hundred nanoseconds).

[0392] Figures 49A-49C 49. A local extraction controller 4902 is illustrated configuring a datapath network according to an embodiment of the present disclosure. The depicted network includes a plurality of multiplexers (e.g., multiplexers 4906, 4908, 4910) that can be configured (e.g., via their respective control signals) to connect one or more datapaths (e.g., from PEs) together. Figure 49A Illustrated is a network 4900 (eg, structure) configured (eg, set up) for some prior operating procedures. Figure 49B The diagram shows a local extraction controller 4902 (e.g., including a network interface circuit 4904 for sending and / or receiving signals) strobing an extraction signal, and all PEs controlled by the LEC enter extraction mode. The last PE in the extraction chain (or extraction terminator) can master the extraction channel (e.g., bus) and send data based on (1) a signal from the LEC or (2) an internally generated signal (e.g., from the PE). Once completed, the PE can set its completion flag, for example, to enable the next PE to extract its data. Figure 49C The furthest PE is shown having completed the extraction process and, as a result, has set its one or more extraction status bits, e.g., which cause the mux to swing to the adjacent network to enable the next PE to begin the extraction process. The extracted PE may resume normal operation. In some embodiments, the PE may remain disabled until other action is taken. In these figures, the multiplexer network is not the same as in some figures (e.g., Figure 23 ) is similar to the “switching device” shown in ).

[0393] The next section describes the operation of the various components of an embodiment of the extraction network.

[0394] Local Extraction Controller

[0395] Figure 50 FIGURE 5002 illustrates an extraction controller according to an embodiment of the present disclosure. A local extraction controller (LEC) may be a hardware entity responsible for accepting extraction commands, coordinating the extraction process of the EFE, and / or storing the extracted data to, for example, virtual memory. In this capacity, the LEC may be a dedicated serialized microcontroller.

[0396] LEC operation can begin when the LEC receives a pointer to a buffer (e.g., in virtual memory) where the structure state is to be written and, optionally, a command controlling how much of the structure is to be extracted. Depending on the LEC microarchitecture, this pointer (e.g., stored in pointer register 5004) can come to the LEC either over the network or through a memory system access. When the LEC receives such a pointer (e.g., a command), it proceeds to extract state from the portion of the structure for which it is responsible. The LEC can stream this extracted data out of the structure into a buffer provided by the external caller.

[0397] exist Figure 48 Two different microarchitectures for LECs are shown in FIG. The first places LEC 4802 at the memory interface. In this case, the LEC can make direct requests to the memory system to write the data being fetched. In the second case, LEC 4806 is placed on a memory network, where LCC 4806 can only make requests to the memory indirectly. In both cases, the logical operation of the LEC may not change. In one embodiment, the LEC is informed of the desire to fetch data from the structure, for example, by a set of control status registers (e.g., visible to the OS), which will be used to notify each LEC of the new command.

[0398] Additional out-of-band control channels (e.g., wires)

[0399] In some embodiments, extraction relies on 2-8 additional out-of-band signals to improve configuration speed, as defined below. Signals driven by the LEC may be labeled LEC. Signals driven by the EFE (e.g., PE) may be labeled EFE. Configuration controller 5002 may include the following control channels, such as LEC_EXTRACT control channel 5006, LEC_START control channel 5008, LEC_STROBE control channel 5010, and EFE_COMPLETE control channel 5012, examples of each of which are discussed below in Table 3.

[0400] Table 3: Extraction channels

[0401]

[0402] In general, the handling of the extraction can be left to the implementer of the specific EFE. For example, a selectable function EFE may have provisions to use existing data paths to dump registers, while a fixed function EFE may simply have a multiplexer.

[0403] Due to the long line delays when programming a large set of EFEs, the LEC_STROBE signal can be considered a clock / latch enable for the EFE components. Because this signal is used as a clock, in one embodiment, the duty cycle of this line is at most 50%. As a result, the extraction throughput is approximately halved. Optionally, a second LEC_STROBE signal can be added to enable continuous extraction.

[0404] In one embodiment, only LEC_START is strictly communicated over a separate coupling device (eg, wire), for example, other control channels may be overlaid over an existing network (eg, wire).

[0405] Reuse of network resources

[0406] To reduce data fetch overhead, certain embodiments of the CSA leverage existing network infrastructure to deliver fetched data. LECs can leverage both the chip-level memory hierarchy and the fabric-level communication network to move data from the fabric to storage. As a result, in certain embodiments of the CSA, the fetch infrastructure contributes no more than 2% to the total fabric area and power.

[0407] Reuse of network resources in certain embodiments of the CSA may enable networks with some hardware support for the extraction protocol. Circuit-switched networks require certain embodiments of the CSA to have the LEC configure their multiplexers in a specific way when the "LEC_START" signal is asserted. Packet-switched networks do not require this extension, but LEC endpoints (e.g., extraction terminators) use specific addresses in packet-switched networks. Network reuse is optional, and some embodiments may find a dedicated configuration bus more convenient.

[0408] Each EFE status

[0409] Each EFE may maintain a bit that indicates whether it has output its status. This bit may be de-asserted when the extract start signal is driven, and then asserted once the particular EFE has completed the extract. In one extract protocol, the EFEs are arranged to form a chain, and the EFE extract status bits determine the topology of the chain. An EFE may read the extract status bits of the immediately adjacent EFEs. If the adjacent EFE has its extract bit set and the current EFE does not have its extract bit set, then the EFE may determine that it owns the extract bus. When an EFE dumps its last data value, it may drive the "EFE_DONE" signal and set its extract bit, thereby enabling, for example, an upstream EFE to be configured for extraction. The network adjacent to the EFE may observe this signal and also adjust its state to handle this transition. As a base case for the extract process, an extract terminator (e.g., in Figure 39 An extraction terminator 4804 for LEC 4802 or an extraction terminator 4808 for LEC 4806) may be included at the end of the chain.

[0410] Within the EFE, this bit can be used to drive flow control ready signals. For example, when the extract bit is deasserted, network control signals can be automatically clamped to values that prevent data flow, and within the PE, no operations or actions will be scheduled.

[0411] Handling high-latency paths

[0412] An embodiment of an LEC may, for example, drive signals over long distances through numerous multiplexers and utilize numerous loads. Consequently, it may be difficult for signals to reach the remote EFE within a short clock cycle. In some embodiments, the extracted signal is at a certain frequency division (e.g., fractional) of the master (e.g., CSA) clock signal to ensure digital timing compliance during extraction. Clock division can be used in out-of-band signaling protocols and does not require any modifications to the master clock tree.

[0413] Ensure consistent structural behavior during extraction

[0414] Because some extraction schemes are distributed and have non-deterministic timing due to program and memory effects, different members of the structure may be in the extraction state at different times. When LEC_EXTRACT is driven, all network flow control signals may be driven to logic low, for example, thereby freezing the operation of a particular segment of the structure.

[0415] The extraction process can be non-destructive. Thus, once extraction is complete, the set of PEs can be considered running. Extensions to the extraction protocol can allow PEs to be optionally disabled after extraction. Alternatively, in an embodiment, starting configuration during the extraction process will have a similar effect.

[0416] Single PE extraction

[0417] In some cases, extracting a single PE may be expedient. In this case, as part of the initialization of the extraction process, an optional address signal may be driven. This allows the extraction of that PE to be directly enabled. Once the PE has been extracted, the extraction process terminates with the LEC_EXTRACT signal falling. In this way, a single PE can be selectively extracted, for example, by a local extraction controller.

[0418] Disposal of extraction back pressure

[0419] In embodiments where the LEC writes the extracted data to memory (e.g., for post-processing, e.g., in software), it may be constrained by limited memory bandwidth. In the event that the LEC exhausts its buffer capacity or anticipates that it will exhaust its buffer capacity, the LEC may stop strobing LEC_STROBE until the buffering issue has been resolved.

[0420] Note that in some of the drawings (e.g. Figure 39 、 42 , 43, 45, 46 and 48), schematically illustrating communications. In some embodiments, those communications may occur via a (eg, interconnected) network.

[0421] 7.7 Flowchart

[0422] Figure 51 5100 is a flowchart according to an embodiment of the present disclosure. The flowchart 5100 includes: 5102: decoding an instruction into a decoded instruction using a decoder of a core of a processor; 5104: executing the decoded instruction to perform a first operation using an execution unit of the core of the processor; 5106: receiving an input of a dataflow graph comprising a plurality of nodes; 5108: overlaying the dataflow graph onto an array of processing elements of the processor, with each node represented as a dataflow operator in the array of processing elements; and 5110: executing a second operation of the dataflow graph using the array of processing elements when an incoming operand set arrives at the array of processing elements.

[0423] Figure 525200 according to an embodiment of the present disclosure. The depicted flow 5200 includes: 5202: decoding an instruction into a decoded instruction using a decoder of a core of a processor; 5204: executing the decoded instruction using an execution unit of the core of the processor to perform a first operation; 5206: receiving an input of a dataflow graph including a plurality of nodes; 5208: overlaying the dataflow graph onto a plurality of processing elements of the processor and an interconnection network between the plurality of processing elements of the processor, with each node represented as a dataflow operator in the plurality of processing elements; and 5210: executing a second operation of the dataflow graph using the interconnection network and the plurality of processing elements when an incoming operand set arrives at the plurality of processing elements.

[0424] 8. Summary

[0425] ExaFLOP-scale supercomputing can be a challenge in high-performance computing that may not be met by conventional von Neumann architectures. To achieve ExaFLOPs, embodiments of the CSA provide heterogeneous spatial arrays that are targeted for direct execution of dataflow graphs (e.g., generated by a compiler). In addition to laying out the architectural principles of embodiments of the CSA, embodiments of the CSA are described and evaluated above that demonstrate 10x (10 times) higher performance and energy than existing products. The code generated by the compiler can have significant performance and energy gains compared to roadmap architectures. As a heterogeneous parameterized architecture, embodiments of the CSA can be easily adapted to all computing use cases. For example, a mobile version of the CSA can be scaled to 32 bits, while an array focused on machine learning can feature a significant number of vectorized 8-bit multiplication units. The main advantages of embodiments of the CSA are high performance, extremely high energy efficiency, and features relevant to all forms of computing, from supercomputing and data centers to the Internet of Things.

[0426] In one embodiment, a processor includes: a plurality of processing elements; an interconnection network between the plurality of processing elements for receiving an input dataflow graph comprising a plurality of nodes, wherein the dataflow graph is configured to be overlaid onto the interconnection network and the plurality of processing elements, and each node is represented as a dataflow operator in the interconnection network and the plurality of processing elements, and the plurality of processing elements are configured to perform operations when an incoming operand set arrives at the plurality of processing elements; and a configuration controller coupled to the plurality of processing elements, the configuration controller configured to: configure the plurality of processing elements according to configuration information in the dataflow graph; and clock-gate at least one clocked component of the processing elements based on the configuration information. The at least one clocked component may be an input buffer of a plurality of parallel input buffers within the processing element. The at least one clocked component may be an input buffer of a plurality of parallel output buffers within the processing element. The at least one clocked component may be an operation configuration register within the processing element for storing an operation configuration of the configuration information. The configuration controller may clock-gate at least one clocked component of a second processing element based on the configuration information. The at least one clocked component may include: a plurality of parallel input buffers within the processing element; a plurality of parallel output buffers within the processing element; and an operational configuration register within the processing element for storing an operational configuration of configuration information, and the configuration controller is configured to independently clock gate each of those clocked components.

[0427] In another embodiment, a method includes configuring, using a configuration controller of a processor, a plurality of processing elements of the processor based on configuration information of a dataflow graph, wherein the processor includes the plurality of processing elements and an interconnection network between the plurality of processing elements and has the dataflow graph, the dataflow graph including a plurality of nodes, the plurality of nodes being overlaid into the plurality of processing elements of the processor and the interconnection network between the plurality of processing elements of the processor, and each node being represented as a dataflow operator in the interconnection network and the plurality of processing elements; clock-gating, using the configuration controller of the processor, at least one clocked component of the processing elements based on the configuration information of the dataflow graph; and executing operations of the dataflow graph using the interconnection network and the plurality of processing elements when an incoming operand set arrives at the plurality of processing elements. The clock gating may include clock-gating an input buffer of a plurality of parallel input buffers within the processing element. The clock gating may include clock-gating an output buffer of a plurality of parallel output buffers within the processing element. The clock gating may include clock-gating an operation configuration register within the processing element for storing an operation configuration of the configuration information. The clock gating may include clock-gating at least one clocked component of a second processing element based on the configuration information. The at least one clocked component may include: a plurality of parallel input buffers within the processing element; a plurality of parallel output buffers within the processing element; and an operational configuration register within the processing element for storing an operational configuration of configuration information, and the configuration controller independently performs clock gating for each of those clocked components.

[0428] In yet another embodiment, a processor includes: a plurality of processing elements; an interconnection network between the plurality of processing elements for receiving input of a dataflow graph comprising a plurality of nodes, wherein the dataflow graph is configured to be overlaid into the interconnection network and the plurality of processing elements, and each node is represented as a dataflow operator in the interconnection network and the plurality of processing elements, and the plurality of processing elements are configured to perform operations when an incoming operand set arrives at the plurality of processing elements; and means, coupled to the plurality of processing elements, for: configuring the plurality of processing elements according to configuration information of the dataflow graph; and clock-gating at least one clocked component of the processing elements based on the configuration information.

[0429] In another embodiment, a processor includes: a plurality of processing elements; an interconnection network between the plurality of processing elements for receiving an input of a dataflow graph comprising a plurality of nodes, wherein the dataflow graph is configured to be overlaid onto the interconnection network and the plurality of processing elements, and each node is represented as a dataflow operator in the interconnection network and the plurality of processing elements, and the plurality of processing elements are configured to perform operations when an incoming operand set arrives at the plurality of processing elements; and a configuration controller coupled to a first processing element and a second processing element of the plurality of processing elements, the first processing element having an output coupled to an input of the second processing element, the configuration controller configured to configure the second processing element to clock-gate at least one clocked component of the second processing element, and to configure the first processing element to send a re-enable signal on the interconnection network to the second processing element when data is to be sent from the first processing element to the second processing element to re-enable the at least one clocked component of the second processing element. The configuration controller may configure the first processing element to send the re-enable signal and the data from the first processing element to the second processing element during the same clock cycle. The at least one clocked component of the second processing element may be a plurality of parallel input buffers within the second processing element. The configuration controller may configure a first processing element to clock-gate a plurality of parallel output buffers within the first processing element and to re-enable the plurality of parallel output buffers when data is to be sent from the plurality of parallel output buffers within the first processing element to the plurality of parallel input buffers within the second processing element. The configuration controller is coupled to a third processing element of the plurality of processing elements, and the first processing element has an output coupled to an input of the third processing element. The configuration controller may configure the third processing element to not clock-gate any clocked component of the third processing element. The configuration controller may configure the third processing element to not clock-gate any clocked component of the third processing element when a distance between the first processing element and the third processing element on an interconnect network is greater than a threshold distance for communication between the first processing element and the third processing element within the same clock cycle.

[0430] In yet another embodiment, a method includes: configuring, with a configuration controller of a processor coupled to a first processing element and a second processing element of a plurality of processing elements, the second processing element to clock-gate at least one clocked component of the second processing element, the first processing element having an output coupled to an input of the second processing element, wherein the processor includes the plurality of processing elements and an interconnection network between the plurality of processing elements and has a dataflow graph including a plurality of nodes overlaid into the plurality of processing elements of the processor and the interconnection network between the plurality of processing elements of the processor, and each node being represented as a dataflow operator in the interconnection network and the plurality of processing elements; configuring, with the configuration controller, The first processing element is configured to: when data is to be sent from the first processing element to the second processing element, send a re-enable signal over an interconnect network to the second processing element to re-enable at least one clocked component of the second processing element; clock-gate the at least one clocked component of the second processing element using a configuration controller of a processor; send a re-enable signal over the interconnect network to the second processing element when data is to be sent from the first processing element to the second processing element to re-enable at least one clocked component of the second processing element; and execute the operations of the dataflow graph using the second processing element when an incoming operand set including the data arrives at the second processing element. The first processing element may be configured such that the first processing element sends the re-enable signal and the data from the first processing element to the second processing element during the same clock cycle. The clock gating may include clock-gating a plurality of parallel input buffers within the second processing element. The first processing element may be configured such that the first processing element clock-gates a plurality of parallel output buffers within the first processing element; and re-enables the plurality of parallel output buffers when data is sent from the plurality of parallel output buffers within the first processing element to the plurality of parallel input buffers within the second processing element. The method may include configuring, using a configuration controller coupled to a third processing element of the plurality of processing elements, a third processing element to not clock-gate any clocked component of the third processing element, the configuration controller being coupled to the third processing element of the plurality of processing elements, wherein the first processing element has an output coupled to an input of the third processing element. Configuring the third processing element to not clock-gate any clocked component of the third processing element may be based on a distance between the first processing element and the third processing element on an interconnect network being greater than a threshold distance for communication between the first processing element and the third processing element within the same clock cycle.

[0431] In another embodiment, a processor includes: a plurality of processing elements; an interconnection network between the plurality of processing elements for receiving an input of a dataflow graph including a plurality of nodes, wherein the dataflow graph is configured to be overlaid into the interconnection network and the plurality of processing elements, and each node is represented as a dataflow operator in the interconnection network and the plurality of processing elements, and the plurality of processing elements are configured to perform operations when an incoming operand set arrives at the plurality of processing elements; and means coupled to a first processing element and a second processing element of the plurality of processing elements, the first processing element having an output coupled to an input of the second processing element, the means for configuring the second processing element to clock-gate at least one clocked component of the second processing element, and configuring the first processing element to send a re-enable signal on the interconnection network to the second processing element to re-enable the at least one clocked component of the second processing element when data is to be sent from the first processing element to the second processing element.

[0432] In one embodiment, a processor includes: a plurality of processing elements; an interconnection network between the plurality of processing elements for receiving input of a dataflow graph including a plurality of nodes, the dataflow graph being configured to be overlaid into the interconnection network and the plurality of processing elements, with each node being represented as a dataflow operator in the interconnection network and the plurality of processing elements, the plurality of processing elements being configured to perform operations when an incoming operand set arrives at the plurality of processing elements; and a configuration controller coupled to a first subset and a second, different subset of the plurality of processing elements, the first subset having outputs coupled to inputs of the second, different subset, wherein the configuration controller is configured to: configure the first subset and the second, different subset of the plurality of processing elements according to configuration information for a first context of the dataflow graph; and after pending operations of the first context are completed in the first subset (e.g., until a backpressure signal is encountered and / or all input data is consumed), In one embodiment, a processor comprises a plurality of processing elements configured to execute a context switch (or pending operations for a first context are not completed, e.g., operations are stopped at a stopping point where a state can be extracted), and a first subset of the plurality of processing elements is configured according to configuration information for a second context of a dataflow graph, and the second context data flow is blocked from flowing from the outputs of the first subset to the inputs of a different second subset until pending operations for the first context are completed in the different second subset (e.g., until a backpressure signal is encountered and / or all input data is consumed). The processor may include a first local configuration controller for the first subset and a second local configuration controller for the different second subset, wherein the configuration controller is configured to send corresponding configuration information to each of the first and second local configuration controllers. The pending operations may be operations that are to be (e.g., required to be) completed in order to reach a (e.g., fully) savable state, e.g., as discussed above with respect to one or more of the figures. The configuration information may be stored and subsequently read in, while the extraction information may be written to a storage location, e.g., to virtual memory (e.g., via a RAF circuit). The configuration controller may include an extraction controller configured to cause state data from a first subset and a different second subset of the plurality of processing elements to be saved to a memory, and the extraction controller configured to extract the first state data from the first subset upon completion of pending operations of a first context in the first subset for a requested context switch. The plurality of processing elements may include a different third subset of the plurality of processing elements between outputs of the first subset and inputs of the different second subset, and the configuration controller configured to maintain the different third subset of the plurality of processing elements in an unconfigured state for the requested context switch to prevent flow of second context data from the outputs of the first subset to the inputs of the different second subset until pending operations of the first context in the different second subset are completed. The configuration controller may cause a backpressure signal for the different third subset to be output to the first subset of the plurality of processing elements for the unconfigured state.The configuration controller may allow operations on a first context in a different second subset to be concurrent with (eg, occur in) operations on a second context in the first subset.

[0433] In another embodiment, a method includes receiving an input of a dataflow graph comprising a plurality of nodes; overlaying the dataflow graph onto a plurality of processing elements of a processor and an interconnection network between the plurality of processing elements of the processor, with each node represented as a dataflow operator in the interconnection network and the plurality of processing elements; performing operations of the dataflow graph using the interconnection network and the plurality of processing elements when an incoming operand set arrives at the plurality of processing elements; configuring, using a configuration controller of the processor, a first subset of the plurality of processing elements and a different second subset according to configuration information for a first context of the dataflow graph; and, after pending operations of the first context are completed in the first subset, configuring, using the configuration controller of the processor, the first subset of the plurality of processing elements according to configuration information for a second context of the dataflow graph for a requested context switch, and preventing second context dataflow from an output of the first subset to an input of the different second subset until pending operations of the first context are completed in the different second subset. The method may include the configuration controller sending corresponding configuration information to each of a first local configuration controller of the first subset and a second local configuration controller of the different second subset. The method may include, after pending operations of the first context are completed in the first subset, extracting first state data from the first subset for a requested context switch. The method may include, for the requested context switch, maintaining, using a configuration controller, a different third subset of the plurality of processing elements between outputs of the first subset and inputs of a different second subset in an unconfigured state to prevent a flow of second context data from the outputs of the first subset to the inputs of the different second subset until pending operations of the first context are completed in the different second subset. The maintaining may include causing a backpressure signal of the different third subset to be output to the first subset of the plurality of processing elements for the unconfigured state. The method may include, using the configuration controller, allowing operations on the first context in the different second subset to be concurrent with (e.g., occur in) operation on the second context in the first subset.

[0434] In yet another embodiment, a processor includes: a plurality of processing elements; an interconnection device between the plurality of processing elements for receiving input of a dataflow graph including a plurality of nodes, the dataflow graph being adapted to be overlaid into the interconnection device and the plurality of processing elements, with each node being represented as a dataflow operator in the interconnection device and the plurality of processing elements, the plurality of processing elements being adapted to perform operations when an incoming set of operands arrives at the plurality of processing elements; and a device coupled to a first subset and a different second subset of the plurality of processing elements, the first subset having outputs coupled to inputs of the different second subset, wherein the device is adapted to: configure the first subset and the different second subset of the plurality of processing elements according to configuration information for a first context of the dataflow graph; and upon completion of pending operations of the first context in the first subset, for a requested context switch, configure the first subset of the plurality of processing elements according to configuration information for a second context of the dataflow graph, and prevent second context dataflow from the output of the first subset to the input of the different second subset until pending operations of the first context are completed in the different second subset.

[0435] In another embodiment, a processor includes: a plurality of processing elements; an interconnection network between the plurality of processing elements for receiving input of a dataflow graph including a plurality of nodes, the dataflow graph for being overlaid into the interconnection network and the plurality of processing elements, with each node represented as a dataflow operator in the interconnection network and the plurality of processing elements, the plurality of processing elements for performing operations when an incoming operand set arrives at the plurality of processing elements; a first configuration controller coupled to a first subset of the plurality of processing elements; and a second configuration controller coupled to a second, different subset of the plurality of processing elements, the first subset having outputs coupled to inputs of the second, different subset, wherein the first configuration controller and the second configuration controller are configured to: configure the first subset and the second, different subset of the plurality of processing elements according to configuration information for a first context of the dataflow graph; and upon completion of pending operations of the first context in the first subset, for a requested context switch, the first configuration controller is configured to configure the first subset of the plurality of processing elements according to configuration information for a second context of the dataflow graph and prevent second context dataflow from the output of the first subset to enter the input of the second, different subset until pending operations of the first context in the second subset are completed. The processor may include a higher-level configuration controller coupled to the first and second configuration controllers, wherein the higher-level configuration controller is configured to send corresponding configuration information to each of the first and second configuration controllers. The first configuration controller may include an extraction controller configured to cause state data from a first subset of the plurality of processing elements to be saved to a memory, and the extraction controller is configured to extract the first state data from the first subset for a requested context switch after pending operations of a first context in the first subset have completed. The plurality of processing elements may include a different third subset of the plurality of processing elements between outputs of the first subset and inputs of a different second subset, and a third configuration controller coupled to the different third subset is configured to maintain the different third subset of the plurality of processing elements in an unconfigured state for the requested context switch, thereby preventing the flow of second context data from the outputs of the first subset to the inputs of the different second subset until pending operations of the first context in the different second subset have completed. The third configuration controller may cause a backpressure signal for the different third subset to be output to the first subset of the plurality of processing elements for the unconfigured state. The first configuration controller and the second configuration controller may allow operation on the first context in the second, different subset to occur concurrently with operation on the second context in the first subset.

[0436] In yet another embodiment, a method includes receiving an input of a dataflow graph comprising a plurality of nodes; overlaying the dataflow graph onto a plurality of processing elements of a processor and an i...

Claims

1. A processor, comprising: multiple processing elements; an interconnect network between the plurality of processing elements, the interconnect network for receiving an input of a dataflow graph comprising a plurality of nodes, wherein the dataflow graph is adapted to be overlaid into the interconnect network and the plurality of processing elements, and each node is represented as a dataflow operator in the interconnect network and the plurality of processing elements, and the plurality of processing elements are adapted to perform an operation when an incoming operand set arrives at the plurality of processing elements; and A configuration controller is coupled to the plurality of processing elements, the configuration controller being configured to: configure the plurality of processing elements according to configuration information of the dataflow graph; and clock gate at least one clocked component of the processing elements based on the configuration information.

2. The processor of claim 1, wherein: The at least one clocked component is an input buffer of a plurality of parallel input buffers within the processing element.

3. The processor of claim 1, wherein: The at least one clocked component is an output buffer of a plurality of parallel output buffers within the processing element.

4. The processor of claim 1, wherein: The at least one clocked component is an operational configuration register within the processing element for storing an operational configuration of the configuration information.

5. The processor of claim 1, wherein: The configuration controller is configured to clock-gate at least one clocked component of the second processing element based on the configuration information.

6. The processor according to any one of claims 1 to 5, wherein: The at least one clocked component includes: a plurality of parallel input buffers within the processing element; a plurality of parallel output buffers within the processing element; and an operational configuration register within the processing element for storing an operational configuration of the configuration information, and the configuration controller is to independently clock gate each of those clocked components.

7. A method for clock gating, comprising: configuring, using a configuration controller of a processor, a plurality of processing elements of the processor according to configuration information of a dataflow graph, wherein the processor includes the plurality of processing elements and an interconnection network between the plurality of processing elements and has the dataflow graph, the dataflow graph including a plurality of nodes, the plurality of nodes being overlaid into the plurality of processing elements of the processor and the interconnection network between the plurality of processing elements of the processor, and each node being represented as a dataflow operator in the interconnection network and the plurality of processing elements; clock-gating, with the configuration controller of the processor, at least one clocked component of a processing element based on the configuration information of the dataflow graph; and When incoming operand sets arrive at the plurality of processing elements, operations of the dataflow graph are performed utilizing the interconnection network and the plurality of processing elements.

8. The method of claim 7, wherein: The clock gating includes clock gating an input buffer of a plurality of parallel input buffers within the processing element.

9. The method of claim 7, wherein: The clock gating includes clock gating an output buffer of a plurality of parallel output buffers within the processing element.

10. The method of claim 7, wherein: The clock gating includes: performing clock gating on an operation configuration register within the processing element for storing the operation configuration of the configuration information.

11. The method according to claim 7, wherein: The clock gating includes clock gating at least one clocked component of the second processing element based on the configuration information.

12. The method according to any one of claims 7 to 11, wherein: The at least one clocked component includes: a plurality of parallel input buffers within the processing element; a plurality of parallel output buffers within the processing element; and an operational configuration register within the processing element for storing an operational configuration of the configuration information, and the configuration controller is to independently clock gate each of those clocked components.

13. A processor comprising: multiple processing elements; an interconnect network between the plurality of processing elements, the interconnect network being operable to receive an input of a dataflow graph comprising a plurality of nodes, wherein the dataflow graph is operable to be overlaid into the interconnect network and the plurality of processing elements, and each node is represented as a dataflow operator in the interconnect network and the plurality of processing elements, and the plurality of processing elements are operable to perform an operation when an incoming operand set arrives at the plurality of processing elements; as well as a configuration controller coupled to a first processing element and a second processing element of the plurality of processing elements, the first processing element having an output coupled to an input of the second processing element, the configuration controller configured to configure the second processing element to clock-gate at least one clocked component of the second processing element, and to configure the first processing element to send a re-enable signal on the interconnect network to the second processing element to re-enable the at least one clocked component of the second processing element when data is to be sent from the first processing element to the second processing element.

14. The processor of claim 13, wherein: The configuration controller is configured to configure the first processing element to send the re-enable signal and the data from the first processing element to the second processing element within a same clock cycle.

15. The processor of claim 13, wherein: The at least one clocked component of the second processing element includes a plurality of parallel input buffers within the second processing element.

16. The processor of claim 15, wherein: The configuration controller is configured to configure the first processing element to: perform clock gating on a plurality of parallel output buffers within the first processing element; and re-enabling the plurality of parallel output buffers when the data is to be sent from the plurality of parallel output buffers within the first processing element to the plurality of parallel input buffers within the second processing element.

17. The processor of any one of claims 13 to 16, wherein: The configuration controller is coupled to a third processing element of the plurality of processing elements, and the first processing element has an output coupled to an input of the third processing element, the configuration controller being configured to configure the third processing element to not clock-gate any clocked component of the third processing element.

18. The processor of claim 17, wherein: The configuration controller is to configure the third processing element to not clock-gate any clocked component of the third processing element when a distance between the first processing element and the third processing element on the interconnect network is greater than a threshold distance for communication between the first processing element and the third processing element within a same clock cycle.

19. A method for clock gating, comprising: configuring, with a configuration controller of a processor coupled to a first processing element and a second processing element of a plurality of processing elements, the second processing element to clock-gate at least one clocked component of the second processing element, the first processing element having an output coupled to an input of the second processing element, wherein the processor includes the plurality of processing elements and an interconnection network between the plurality of processing elements and has a dataflow graph including a plurality of nodes overlaid into the plurality of processing elements of the processor and into the interconnection network between the plurality of processing elements of the processor, and each node being represented as a dataflow operator in the interconnection network and the plurality of processing elements; configuring, with the configuration controller, the first processing element to: send a re-enable signal to the second processing element over the interconnect network to re-enable the at least one clocked component of the second processing element when data is to be sent from the first processing element to the second processing element; clock-gating, with the configuration controller of the processor, the at least one clocked component of the second processing element; sending, by the first processing element, a re-enable signal to the second processing element over the interconnect network to re-enable the at least one clocked component of the second processing element when the data is sent from the first processing element to the second processing element; and When an incoming operand set including the data arrives at the second processing element, operations of the dataflow graph are performed using the second processing element.

20. The method of claim 19, wherein: The first processing element is configured to send the re-enable signal and the data from the first processing element to the second processing element in a same clock cycle.

21. The method of claim 19, wherein: The clock gating includes: performing clock gating on a plurality of parallel input buffers within the second processing element.

22. The method of claim 21, wherein: The configuration of the first processing element causes the first processing element to: clock gate a plurality of parallel output buffers within the first processing element; and re-enabling the plurality of parallel output buffers when data is sent from the plurality of parallel output buffers within the first processing element to the plurality of parallel input buffers within the second processing element.

23. The method of any one of claims 19 to 22, further comprising: A third processing element of the plurality of processing elements is configured, with the configuration controller, to not clock-gate any clocked component of the third processing element, the configuration controller being coupled to the third processing element, and the first processing element having an output coupled to an input of the third processing element.

24. The method of claim 23, wherein: Configuring the third processing element to not clock-gate any clocked component of the third processing element is based on a distance between the first processing element and the third processing element on the interconnect network being greater than a threshold distance for communication between the first processing element and the third processing element within the same clock cycle.

25. A processor comprising: multiple processing elements; an interconnect network between the plurality of processing elements, the interconnect network for receiving an input of a dataflow graph comprising a plurality of nodes, wherein the dataflow graph is adapted to be overlaid into the interconnect network and the plurality of processing elements, and each node is represented as a dataflow operator in the interconnect network and the plurality of processing elements, and the plurality of processing elements are adapted to perform an operation when an incoming operand set arrives at the plurality of processing elements; and Means, coupled to the plurality of processing elements, are configured to: configure the plurality of processing elements according to configuration information of the dataflow graph; and clock gate at least one clocked component of the processing elements based on the configuration information.

Images