Apparatus, methods, and systems for configurable space accelerator memory coherence

By using a Configurable Space Accelerator (CSA), the balance between high energy efficiency and high performance of processors is solved, achieving performance improvements for computationally intensive tasks with low complexity and high energy efficiency, and making it suitable for mainstream high-performance computing programs.

CN109992306BActive Publication Date: 2026-11-17INTEL CORP
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN201811497591.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2017-12-30
Filing Date
2018-12-07
Publication Date
2026-11-17
Estimated Expiration
2038-12-07

AI Technical Summary

Technical Problem

Existing processors or processor assemblies struggle to strike a balance between high energy efficiency and high performance when executing instruction sets, especially at exascale computing targets. The classic von Neumann architecture struggles to improve both program execution performance and energy efficiency simultaneously.

Method used

A Configurable Spatial Accelerator (CSA) is employed, which includes a spatial array of processing elements. It utilizes lightweight backpressure network connectivity to support direct execution of data flow graphs, reducing control overhead and enabling efficient data flow operations through a compilation toolchain.

Benefits of technology

It achieves significant improvements in performance and energy efficiency for computationally intensive tasks with low complexity and high energy efficiency, while maintaining a familiar high-performance computing programming model and being applicable to mainstream HPC programs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN109992306B_ABST
    Figure CN109992306B_ABST
Patent Text Reader

Abstract

This application discloses apparatus, methods, and systems for configurable space accelerator memory coherence. Methods and apparatus relating to coherence in an accelerator are described. In one embodiment, a first network couples Request Address File (RAF) circuitry to a space array, and a second network couples memory to the RAF circuitry. The RAF circuitry is configured to: not publish a request for memory marked with a program sequence dependency on a previous request to the second network until a first token generated by another RAF circuitry upon completion of the previous request for memory is received; and the second RAF circuitry is configured to: not publish a second request for memory marked with a program sequence dependency on the first request to the second network until a predetermined time period has elapsed since the first RAF circuitry published the first request to the second network until a second token sent by the first RAF circuitry is received.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Statement on federally funded research and development

[0002] This invention was carried out with government support under contract number H98230A-13-D-0124-0207 granted by the Ministry of Defense. The government holds certain rights in this invention. Technical Field

[0003] This disclosure is generally about electronics, and more specifically, embodiments of this disclosure relate to memory coherence in configurable space accelerators. Background Technology

[0004] A processor or set of processors executes instructions from an instruction set (e.g., an instruction set architecture (ISA)). An instruction set is the programming-related part of a computer architecture and generally includes primitive data types, instructions, register architecture, addressing modes, memory architecture, interrupt and exception handling, and external input and output (I / O). It should be noted that the term "instruction" in this document can refer to macro-instructions or micro-instructions, such as macro-instructions provided to the processor for execution, or micro-instructions generated by the processor's decoder from macro-instructions. Attached Figure Description

[0005] The present disclosure is illustrated by way of example and not by way of limitation in the accompanying drawings, in which similar reference numerals indicate similar elements, wherein:

[0006] Figure 1 The illustration shows an accelerator chip according to an embodiment of the present disclosure.

[0007] Figure 2 The illustration shows a hardware processor coupled to a memory according to an embodiment of the present disclosure.

[0008] Figure 3A The illustration shows a program source according to an embodiment of this disclosure.

[0009] Figure 3B The illustrations depict embodiments of the present disclosure. Figure 3A The data flow diagram of the program source.

[0010] Figure 3C The figure illustrates an accelerator according to an embodiment of the present disclosure, the accelerator having a configuration for performing Figure 3B The data flow graph consists of multiple processing elements.

[0011] Figure 4 The diagram illustrates an example of data flow diagram execution according to an embodiment of this disclosure.

[0012] Figure 5 The illustration shows a program source according to an embodiment of this disclosure.

[0013] Figure 6 The figure illustrates an accelerator chip according to an embodiment of the present disclosure, the accelerator chip including an array of processing elements.

[0014] Figure 7A The diagram illustrates a configurable data path network according to an embodiment of the present disclosure.

[0015] Figure 7B The diagram illustrates a configurable flow control path network according to an embodiment of the present disclosure.

[0016] Figure 8 The illustration shows a hardware processor chip including an accelerator according to an embodiment of the present disclosure.

[0017] Figure 9 The figure illustrates a processing element according to an embodiment of the present disclosure.

[0018] Figure 10 The diagram illustrates a Request Address File (RAF) circuit according to an embodiment of the present disclosure.

[0019] Figure 11 The figure illustrates multiple Request Address File (RAF) circuits coupled between multiple accelerator chips and multiple cache blocks according to an embodiment of the present disclosure.

[0020] Figure 12 The figure shows a data flow diagram of pseudocode function calls according to an embodiment of the present disclosure.

[0021] Figure 13 The figure illustrates a spatial array of processing elements having multiple network data stream endpoint circuits according to an embodiment of the present disclosure.

[0022] Figure 14 The diagram illustrates a network data stream endpoint circuit according to an embodiment of the present disclosure.

[0023] Figure 15 The diagram illustrates a data format for sending and receiving operations according to an embodiment of the present disclosure.

[0024] Figure 16 The illustration shows another data format for transmission operations according to an embodiment of the present disclosure.

[0025] Figure 17 The illustration shows the data format for configuring circuit elements (e.g., network data stream endpoint circuitry) for sending (e.g., switching) operations and receiving (e.g., picking) operations according to embodiments of the present disclosure.

[0026] Figure 18 The illustration shows a configuration data format for configuring circuit elements (e.g., network data stream endpoint circuitry) for a transmission operation using input, output, and control data of the transmission operation annotated on the circuit, according to an embodiment of the present disclosure.

[0027] Figure 19 The illustration shows a configuration data format for configuring circuit elements (e.g., network data stream endpoint circuitry) for a selected operation using input, output, and control data of the selected operation annotated on the circuit, according to an embodiment of the present disclosure.

[0028] Figure 20 The illustration shows a configuration data format for configuring circuit elements (e.g., network data stream endpoint circuitry) for a switch operation using input, output, and control data of a switch operation annotated on the circuit, according to an embodiment of the present disclosure.

[0029] Figure 21 The illustration shows a configuration data format for configuring circuit elements (e.g., network data stream endpoint circuitry) for a SwitchAny operation using input, output, and control data of the SwitchAny operation annotated on the circuit, according to an embodiment of the present disclosure.

[0030] Figure 22 The illustration shows a configuration data format for configuring circuit elements (e.g., network data stream endpoint circuitry) for a Pick operation using input, output, and control data of a Pick operation annotated on the circuit, according to an embodiment of the present disclosure.

[0031] Figure 23 The illustration shows a configuration data format for configuring circuit elements (e.g., network data stream endpoint circuitry) for a PickAny operation using input, output, and control data of the PickAny operation annotated on the circuit, according to an embodiment of the present disclosure.

[0032] Figure 24 The diagram illustrates the selection of operations for execution by a network data stream endpoint circuit according to an embodiment of the present disclosure.

[0033] Figure 25 The diagram illustrates a network data stream endpoint circuit according to an embodiment of the present disclosure.

[0034] Figure 26 The diagram illustrates a network data stream endpoint circuit that receives input zero (0) during a selection operation according to an embodiment of the present disclosure.

[0035] Figure 27 The diagram illustrates a network data stream endpoint circuit that receives input one (1) during a selection operation according to an embodiment of the present disclosure.

[0036] Figure 28 The diagram illustrates a network data stream endpoint circuit that outputs the selected input during a selection operation, according to an embodiment of the present disclosure.

[0037] Figure 29 The illustration shows a flowchart according to an embodiment of the present disclosure.

[0038] Figure 30 The diagram illustrates a Request Address File (RAF) circuit according to an embodiment of the present disclosure.

[0039] Figure 31 The diagram illustrates multiple Request Address File (RAF) circuits coupled between an accelerator and multiple cache blocks according to an embodiment of the present disclosure.

[0040] Figure 32 The diagram illustrates a Request Address File (RAF) circuit according to an embodiment of the present disclosure.

[0041] Figure 33 The illustration shows a flowchart according to an embodiment of the present disclosure.

[0042] Figure 34 The diagram illustrates multiple Request Address File (RAF) circuits coupled between an accelerator and multiple cache blocks according to an embodiment of the present disclosure.

[0043] Figure 35 The figure shows a first request address file (RAF) circuit coupled to a second request address file (RAF) circuit according to an embodiment of the present disclosure.

[0044] Figure 36 The illustration shows a flowchart according to an embodiment of the present disclosure.

[0045] Figure 37 The diagram illustrates multiple Request Address File (RAF) circuits coupled between an accelerator and multiple cache blocks according to an embodiment of the present disclosure.

[0046] Figure 38 The figure shows a first request address file (RAF) circuit coupled to a second request address file (RAF) circuit according to an embodiment of the present disclosure.

[0047] Figure 39 The illustration shows a flowchart according to an embodiment of the present disclosure.

[0048] Figure 40 The illustration shows C source code according to an embodiment of this disclosure.

[0049] Figure 41 The diagram illustrates a floating-point multiplier partitioned into three regions (a result region, three potential carry regions, and a gating region) according to an embodiment of the present disclosure.

[0050] Figure 42 The illustration shows an in-flight configuration of an accelerator with multiple processing elements according to an embodiment of the present disclosure.

[0051] Figure 43The illustration shows a snapshot extracted in a pipeline manner during flight according to an embodiment of the present disclosure.

[0052] Figure 44 The illustration shows a compiler toolchain for an accelerator according to an embodiment of the present disclosure.

[0053] Figure 45 The illustration shows a compiler for an accelerator according to an embodiment of the present disclosure.

[0054] Figure 46A The illustration shows serialized assembly code according to an embodiment of the present disclosure.

[0055] Figure 46B The illustrations depict embodiments of the present disclosure. Figure 46A The serialized assembly code is the data stream assembly code.

[0056] Figure 46C The illustration shows an accelerator according to an embodiment of the present disclosure. Figure 46B Data flow diagram of the data flow assembly code.

[0057] Figure 47A The illustration shows C source code according to an embodiment of this disclosure.

[0058] Figure 47B The illustrations depict embodiments of the present disclosure. Figure 47A The C source code's data flow assembly code.

[0059] Figure 47C The illustration shows an accelerator according to an embodiment of the present disclosure. Figure 47B Data flow diagram of the data flow assembly code.

[0060] Figure 48A The illustration shows C source code according to an embodiment of this disclosure.

[0061] Figure 48B The illustrations depict embodiments of the present disclosure. Figure 48A The C source code's data flow assembly code.

[0062] Figure 48C The illustration shows an accelerator according to an embodiment of the present disclosure. Figure 48B Data flow diagram of the data flow assembly code.

[0063] Figure 49A The illustration shows a flowchart according to an embodiment of the present disclosure.

[0064] Figure 49B The illustration shows a flowchart according to an embodiment of the present disclosure.

[0065] Figure 50 The figure illustrates a graph of throughput relative to the energy per operation according to an embodiment of the present disclosure.

[0066] Figure 51 The figure shows an accelerator chip according to an embodiment of the present disclosure, the accelerator chip including an array of processing elements and a local configuration controller.

[0067] Figures 52A-52C The figure illustrates a local configuration controller for a configuration data path network according to an embodiment of the present disclosure.

[0068] Figure 53 The diagram illustrates a configuration controller according to an embodiment of the present disclosure.

[0069] Figure 54 The figure shows an accelerator chip according to an embodiment of the present disclosure, the accelerator chip including an array of processing elements, a configuration cache and a local configuration controller.

[0070] Figure 55 The figure illustrates an accelerator chip according to an embodiment of the present disclosure, the accelerator chip including an array of processing elements and a configuration and anomaly handling controller having reconfiguration circuitry.

[0071] Figure 56 The diagram illustrates a reconfiguration circuit according to an embodiment of the present disclosure.

[0072] Figure 57 The figure illustrates an accelerator chip according to an embodiment of the present disclosure, the accelerator chip including an array of processing elements and a configuration and anomaly handling controller having reconfiguration circuitry.

[0073] Figure 58 The figure illustrates an accelerator chip according to an embodiment of the present disclosure, the accelerator chip including an array of processing elements and a mezzanine anomaly aggregator coupled to a chip-level anomaly aggregator.

[0074] Figure 59 The figure illustrates a processing element with an exception generator according to an embodiment of the present disclosure.

[0075] Figure 60 The figure shows an accelerator chip according to an embodiment of the present disclosure, the accelerator chip including an array of processing elements and a local extraction controller.

[0076] Figures 61A-61C The figure illustrates a local extraction controller for a configured data path network according to an embodiment of the present disclosure.

[0077] Figure 62 An extraction controller according to an embodiment of the present disclosure is shown.

[0078] Figure 63 A flowchart illustrating an embodiment according to this disclosure is shown.

[0079] Figure 64 A flowchart illustrating an embodiment according to this disclosure is shown.

[0080] Figure 65A This is a block diagram of a system according to an embodiment of the present disclosure, which employs a memory sorting circuit inserted between a memory subsystem and acceleration hardware.

[0081] Figure 65B According to embodiments of this disclosure Figure 65A The system block diagram is shown, but the system employs multiple memory sorting circuits.

[0082] Figure 66 This is a block diagram illustrating the general operation of memory operations for entering and leaving the acceleration hardware according to embodiments of the present disclosure.

[0083] Figure 67 This is a block diagram illustrating the space-dependent flow of storage operations according to an embodiment of the present disclosure.

[0084] Figure 68 This is a detailed block diagram of the memory sorting circuit in FIG65 according to an embodiment of the present disclosure.

[0085] Figure 69 This is a flowchart of the microarchitecture of the memory sorting circuit in FIG65 according to an embodiment of the present disclosure.

[0086] Figure 70 This is a block diagram of an executable determiner circuit according to an embodiment of the present disclosure.

[0087] Figure 71 This is a block diagram of a priority encoder according to an embodiment of the present disclosure.

[0088] Figure 72 This is a block diagram illustrating exemplary loading operations in both logical and binary forms according to embodiments of this disclosure.

[0089] Figure 73A This is a flowchart illustrating the logical execution of example code according to an embodiment of the present disclosure.

[0090] Figure 73B This illustration shows memory-level parallelism in an expanded version of example code according to embodiments of the present disclosure. Figure 73A The flowchart.

[0091] Figure 74A This is a block diagram of exemplary memory arguments for load and store operations according to embodiments of the present disclosure.

[0092] Figure 74B The illustration is based on an embodiment of the present disclosure, and is transmitted through... Figure 69 The microarchitecture of the memory sorting circuit in the memory performs load and store operations (such as, Figure 74AA flowchart of the flow of those operations.

[0093] Figure 75A , 75B 75C, 75D, 75E, 75F, 75G, and 75H are illustrated embodiments according to this disclosure. Figure 75B The diagram shows the functional flow of loading and storing operations performed by the queues in the microarchitecture of the exemplary program.

[0094] Figure 76 This is a flowchart of a method for ordering memory operations between acceleration hardware and an out-of-order memory subsystem according to embodiments of the present disclosure.

[0095] Figure 77A This is a block diagram illustrating a general vector-friendly instruction format and its Class A instruction template according to embodiments of the present disclosure.

[0096] Figure 77B This is a block diagram illustrating a general vector-friendly instruction format and its Class B instruction template according to embodiments of the present disclosure.

[0097] Figure 78A The illustration shows an embodiment of the present disclosure for use Figure 77A and Figure 77B A diagram of fields in the generic vector-friendly instruction format.

[0098] Figure 78B The diagram illustrates a complete opcode field according to an embodiment of this disclosure. Figure 78A A diagram of the fields in the dedicated vector-friendly instruction format.

[0099] Figure 78C The diagram illustrates the configuration of a register index field according to an embodiment of this disclosure. Figure 78A A diagram of the fields in the dedicated vector-friendly instruction format.

[0100] Figure 78D The diagram illustrates the configuration of an extended operation field 7750 according to an embodiment of this disclosure. Figure 78A A diagram of the fields in the dedicated vector-friendly instruction format.

[0101] Figure 79 This is a block diagram of a register architecture according to an embodiment of the present disclosure.

[0102] Figure 80A This is a block diagram illustrating both an exemplary ordered pipeline and an exemplary out-of-order release / execution pipeline for register renaming, according to embodiments of the present disclosure.

[0103] Figure 80BThis is a block diagram illustrating both an exemplary embodiment of an ordered architecture core to be included in a processor according to embodiments of the present disclosure and an exemplary out-of-order release / execution architecture core with register renaming.

[0104] Figure 81A This is a block diagram of a single processor core according to an embodiment of the present disclosure, its connection to the on-die interconnect network, and a local subset of its Level 2 (L2) cache.

[0105] Figure 81B According to embodiments of this disclosure Figure 81A An expanded diagram of a portion of the processor core.

[0106] Figure 82 This is a block diagram of a processor according to embodiments of the present disclosure, which may have more than one core, may have an integrated memory controller, and may have an integrated graphics device.

[0107] Figure 83 This is a block diagram of a system according to an embodiment of the present disclosure.

[0108] Figure 84 This is a block diagram of a more specific exemplary system according to embodiments of the present disclosure.

[0109] Figure 85 The diagram shown is a block diagram of a second, more specific, exemplary system according to an embodiment of the present disclosure.

[0110] Figure 86 The diagram shown is a block diagram of a system-on-a-chip (SoC) according to an embodiment of the present disclosure.

[0111] Figure 87 This is a block diagram illustrating the use of a software instruction converter to convert binary instructions in a source instruction set into binary instructions in a target instruction set, according to embodiments of the present disclosure. Detailed Implementation

[0112] In the following description, several specific details are set forth. However, it should be understood that embodiments of this disclosure can be practiced without these specific details. In other instances, well-known circuits, structures, and techniques have not been shown in detail so as not to obscure the understanding of this description.

[0113] References to "an embodiment," "embodiment," "example embodiment," etc., in the specification indicate that the described embodiment may include a specific feature, structure, or characteristic; however, each embodiment may not necessarily include that specific feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same embodiment. Additionally, when a specific feature, structure, or characteristic is described in connection with an embodiment, it is believed that the influence of such feature, structure, or characteristic on such feature, structure, or characteristic in conjunction with other embodiments, whether explicitly described or not, is within the knowledge of those skilled in the art.

[0114] A processor (e.g., having one or more cores) can execute instructions (e.g., instruction threads) to manipulate data, thereby performing, for example, arithmetic, logic, or other functions. For example, software can request an operation, and a hardware processor (e.g., one or more cores of the hardware processor) can perform the operation in response to the request. A non-limiting example of an operation is a mixing operation that takes multiple vector elements as input and outputs a vector having the mixed elements. In some embodiments, multiple operations are accomplished using the execution of a single instruction.

[0115] For example, the exascale performance defined by the U.S. Department of Energy requires system-level floating-point performance to exceed 10^10^12 volts per second within a given power budget (e.g., 20 MW). 18 Sub-float operations (exaFLOPs) or more. Some embodiments herein relate to spatial arrays (e.g., configurable spatial accelerators (CSAs)) of processing elements for high-performance computing (HPC) such as processors. Some embodiments herein of spatial arrays (e.g., CSAs) of processing elements are directed to the direct execution of dataflow graphs to achieve computationally intensive yet energy-efficient spatial microarchitectures far exceeding conventional roadmap architectures. Some embodiments herein provide one or more memory ordering implementations in the context of spatial architectures (e.g., CSAs) discussed herein. Some embodiments herein provide memory coherence in configurable spatial accelerators.

[0116] The following also includes a description of the architectural concept of embodiments of spatial arrays of processing elements (e.g., CSA) and certain features thereof. As with any revolutionary architecture, programmability can be a risk. To mitigate this issue, embodiments of the CSA architecture have been co-designed using compiler toolchains (also discussed below).

[0117] Introduction

[0118] Exascale computing targets can demand massive system-level floating-point performance (e.g., 1 ExaFLOP) within an aggressive power budget (e.g., 20MW). However, simultaneously improving program execution performance and energy efficiency using the classic von Neumann architecture has become difficult: out-of-order scheduling, simultaneous multithreading, complex register files, and other structures provide performance, but at a high energy cost. Some embodiments in this paper achieve both performance and energy requirements. Exascale computing power-performance targets can demand both high throughput and low energy consumption per operation. Some embodiments in this paper provide this by providing a large number of low-complexity, high-energy-efficiency processing (e.g., compute) elements that greatly eliminate the control overhead of previous processor designs. Guided by this observation, some embodiments in this paper include spatial arrays of processing elements, such as configurable spatial accelerators (CSAs), which, for example, comprise an array of processing elements (PEs) connected by a set of lightweight backpressure (e.g., communication) networks. An example of a CSA chip is shown in... Figure 1 Described in the text. Some embodiments of processing (e.g., computing) elements are data stream operators, such as multiple data stream operators that process only the input data when both (i) input data has arrived at the data stream operator and (ii) there is space available for storing the output data (e.g., otherwise no processing is taking place). Some embodiments (e.g., accelerators or CSAs) do not utilize triggered instructions.

[0119] Figure 1 The illustration shows an embodiment of an accelerator chip 100, a spatial array of processing elements according to an embodiment of the present disclosure. Accelerator chip 100 may be a portion of a larger chip. Accelerator chip 100 executes one or more data flow diagrams. Data flow Figure 1 Generally, this can refer to an explicit parallel program description that occurs during the compilation of serialized code. Certain embodiments herein (e.g., CSA) allow dataflow graphs to be directly configured onto a CSA array, for example, rather than being transformed into a serialized instruction stream. Certain embodiments herein allow a first (e.g., type-) dataflow operation to be executed by one or more processing elements (PEs) of the spatial array, and additionally or alternatively, allow a second (e.g., type-) dataflow operation to be executed by one or more network communication circuits (e.g., endpoints) of the spatial array.

[0120] The data flow graph, derived from the serialized compilation flow, allows CSA implementations to support familiar programming models and (e.g., without using worksheets) directly execute existing high-performance computing (HPC) code. CSA processing elements (PEs) can be energy efficient. Figure 1 In this context, the memory interface 102 can be coupled to a memory (e.g., Figure 2The accelerator chip 100 is configured to access (e.g., load and / or store) data from (e.g., off-die) memory (memory 202). The depicted accelerator chip 100 is a heterogeneous array of several types of PEs coupled together via an interconnect network 104. The accelerator chip 100 may store one or more of the following, for example, as part of a spatial array of processing elements 101: integer arithmetic PEs, floating-point arithmetic PEs, communication circuitry (e.g., network data stream endpoint circuitry), and structures. A data flow graph (e.g., a compiled data flow graph) may be overlaid on the accelerator chip 100 for execution. In one embodiment, for a given data flow graph, each PE handles only one or two (e.g., data stream) operations of the graph. The PE array may be heterogeneous, for example, such that no PE supports the full CSA data flow architecture and / or one or more PEs are programmed (e.g., customized) to perform only some but highly efficient operations. Some embodiments described herein thus enable processors or accelerators with arrays of compute-intensive processing elements compared to roadmap architectures, and achieve approximately orders of magnitude gains in energy efficiency and performance relative to existing HPC supplies.

[0121] Some embodiments herein provide performance improvements from parallel execution within a (e.g., dense) spatial array (e.g., CSA) of processing elements, where each utilized PE and / or network data stream endpoint circuit can perform its operation simultaneously, for example, if input data is available. The efficiency improvements may stem from the efficiency of each PE and / or network data stream endpoint circuit, where, for example, the operation (e.g., behavior) of each PE is fixed once each configuration (e.g., mapping) step and execution occurs at the local data arrival point at the PE, for example, without regard to other structural activities, and / or where the operation (e.g., behavior) of each network data stream endpoint circuit is variable (e.g., non-fixed) when configured (e.g., mapped). In some embodiments, the PE and / or network data stream endpoint circuits are data stream operators (e.g., each PE and / or network data stream endpoint circuit is a single data stream operator), for example, a data stream operator that operates on the input data only if both (i) the input data has arrived at the data stream operator and (ii) there is space available for storing the output data (e.g., otherwise no operation is taking place).

[0122] Some embodiments described herein include spatial arrays of processing elements as a highly energy-efficient and high-performance method for accelerating user applications. In one embodiment, the application is mapped in a highly parallel manner. For example, inner loops may be unrolled multiple times to improve parallelism. This method can provide high performance, for example, when the occupancy (e.g., usage) of the unrolled code is high. However, if there are less-used code paths (e.g., exception code paths like floating-point malformation) within the unrolled loop body, the spatial array of processing elements (e.g., its structural regions) may be wasted, and thus throughput may be lost.

[0123] One embodiment described herein for reducing stress on the spatial array (e.g., its structural regions) of a processing element (e.g., in the case of underutilized code segments) is time multiplexing. In this mode, a single instance of less frequently used (e.g., colder) code can be shared among several loop bodies, similar to function calls in a shared library. In one embodiment, the spatial array (e.g., of the processing element) supports direct implementations of the multiplexed code. However, for example, when multiplexing or demultiplexing in the spatial array involves selecting among multiple and geographically dispersed targets (e.g., sharers), a direct implementation using a dataflow manipulator (e.g., using the processing element) may be inefficient in terms of latency, throughput, implementation area, and / or energy. Some embodiments herein describe hardware mechanisms (e.g., network circuitry) that support (e.g., high-base) multiplexing or demultiplexing. Some embodiments herein (e.g., of network dataflow endpoint circuitry) allow multiple targets (e.g., sharers) to be aggregated with minimal hardware overhead or performance impact. Some embodiments herein allow (e.g., conventional) serialized code to be compiled into a parallel architecture within the spatial array.

[0124] In one embodiment, multiple network data stream endpoint circuits are combined as a single data stream operator, for example, as described below. Figure 13 The discussion focuses on certain (e.g., high (e.g., 4-6) radix) dataflow operators. As a non-limiting example, some (e.g., high-radix, 4-6) dataflow operators are listed below.

[0125] Embodiments of a “Pick” datastream operator are used to select data (e.g., tokens) from multiple input channels and provide that data as its (e.g., a single) output according to control data. The control data used for Pick may include input selector values. In one embodiment, the selected input channel is used to remove (e.g., discard) its data (e.g., tokens), for example, to complete the execution of that datastream operation (or a portion thereof). In another embodiment, those input channels that are not selected are also used to remove (e.g., discard) their data (e.g., tokens), for example, to complete the execution of that datastream operation (or a portion thereof).

[0126] Embodiments of the "PickSingleLeg" dataflow manipulator are used to select data (e.g., tokens) from multiple input channels and provide that data as its (e.g., single) output according to control data. However, in some embodiments, unselected input channels are ignored; for example, those unselected input channels are not used to remove (e.g., discard) their data (e.g., tokens), for example, to complete the execution of the dataflow operation (or a portion thereof). The control data used for PickSingleLeg may include input selector values. In one embodiment, the selected input channel is also used to remove (e.g., discard) its data (e.g., tokens), for example, to complete the execution of the dataflow operation (or a portion thereof).

[0127] Embodiments of the "PickAny" dataflow manipulator are used to select a first available (e.g., for the circuitry performing the operation) data (e.g., a token) from a plurality of input channels and provide that data as its (e.g., a single) output. In one embodiment, PickSingleLeg is also used to output an index (e.g., indicating which of the plurality of input channels) has selected its data. In one embodiment, the selected input channel is used to remove (e.g., discard) its data (e.g., the token), for example, to complete the execution of the dataflow operation (or a portion thereof). In some embodiments, unselected input channels (e.g., with or without input data) are ignored; for example, those unselected input channels are not used to remove (e.g., discard) their data (e.g., the token), for example, to complete the execution of the dataflow operation (or a portion thereof). Control data for PickAny may include values ​​corresponding to PickAny, for example, no input selector value.

[0128] Implementations of a “Switch” data stream operator are used to direct (e.g., a single) input data (e.g., a token) to one or more (e.g., fewer than all) outputs according to control data. The control data for the Switch may include one or more output selector values. In one embodiment, the input data (e.g., from an input channel) is used to remove (e.g., discard) its data (e.g., a token), for example, to complete the execution of that data stream operation (or a portion thereof).

[0129] Implementations of the "SwitchAny" data stream operator are used to guide (e.g., a single) input data (e.g., a token) to provide, for example, control data to one or more (e.g., fewer than all) outputs that can receive that data. In one embodiment, SwitchAny can provide input data to any coupled input buffer (e.g., its ingress buffer). Figure 14 The network ingress buffer (in the network) has output channels with availability (e.g., available storage space). Control data for SwitchAny may include values ​​corresponding to SwitchAny, for example, not having one or more output selector values. In one embodiment, input data (e.g., from an input channel) is used to remove (e.g., discard) its data (e.g., a token), for example, to complete the execution of an operation on that data stream (or a portion thereof). In one embodiment, SwitchAny is also used to output an index (e.g., indicating which of a plurality of output channels) to which input data was provided (e.g., sent). SwitchAny can be used to manage repeating subgraphs in a spatial array, for example, an expanded loop.

[0130] Therefore, some embodiments in this paper provide paradigm-shifting levels of performance and significant energy efficiency improvements across a broad range of existing single-stream and parallel programs (e.g., all programs) while maintaining a familiar HPC programming model. Some embodiments in this paper are targeted at HPC, making floating-point energy efficiency extremely important. Some embodiments in this paper not only achieve remarkable performance improvements and energy reductions but also pass these gains on to existing HPC programs written in mainstream HPC languages ​​and used in mainstream HPC frameworks. Some embodiments of the architectures in this paper (e.g., with contemplated compilation) provide several extensions in direct support for the internal representation of control dataflow generated by modern compilers. Some embodiments in this paper relate to CSA dataflow compilers (e.g., those accepting C, C++, and Fortran programming languages) for CSA architectures.

[0131] Figure 2The illustration depicts a hardware processor 200 coupled to (e.g., connected to) memory 202 according to an embodiment of the present disclosure. In one embodiment, the hardware processor 200 and memory 202 constitute a computing system 201. In some embodiments, one or more of the accelerators are CSAs according to the present disclosure. In some embodiments, one or more cores in the processor are those disclosed herein. The hardware processor 200 (e.g., each of its cores) may include a hardware decoder (e.g., a decoding unit) and a hardware execution unit. The hardware processor 200 may include registers. Note that the figures herein may not depict all data communication couplings (e.g., connections). Those skilled in the art will recognize that this is done to avoid obscuring certain details in the figures. Note that bidirectional arrows in the figures may not require bidirectional communication; for example, they may indicate unidirectional communication (e.g., to or from that component or device). Any path or all combinations of communication paths may be used in some embodiments herein. The depicted hardware processor 200 includes multiple cores (0 to N, where N may be 1 or greater) and hardware accelerators (0 to M, where M may be 1 or greater) according to embodiments of the present disclosure. Hardware processor 200 (e.g., its multiple accelerators and / or multiple cores) may be coupled to memory 202 (e.g., a data storage device). A hardware decoder (e.g., a core's) may receive (e.g., a single) instruction (e.g., a macro instruction) and decode it into, for example, microinstructions and / or microoperations. A hardware execution unit (e.g., a core's) may execute the decoded instructions (e.g., macro instructions) to perform one or more operations.

[0132] Section 1 below discloses embodiments of the CSA architecture. Specifically, a novel embodiment of integrating memory within a dataflow execution model is disclosed. Section 2 explores the microarchitectural details of the CSA embodiments. In one embodiment, the primary purpose of the CSA is to support compiler-generated programs. Section 3 below examines embodiments of the CSA compilation toolchain. In Section 4, the advantages of the CSA embodiments are compared with other architectures during the execution of compiled code. Finally, Section 5 discusses the performance of the CSA microarchitectural embodiments, Section 6 discusses further CSA details, and Section 7 provides a summary.

[0133] 1. CSA Architecture

[0134] Some embodiments of CSA aim to execute programs (e.g., programs generated by a compiler) rapidly and efficiently. Some embodiments of the CSA architecture provide programming abstractions to support compiler technologies and programming paradigms. Embodiments of CSA execute dataflow graphs, for example, program representations that are very similar to the compiler's own internal representation (IR) of a compiled program. In this model, a program is represented as a dataflow graph consisting of nodes (e.g., vertices) and edges, where nodes are drawn from a set of dataflow operators defined by the architecture (e.g., covering both computational and control operations), and edges represent the transfer of data between dataflow operators. Execution progresses by injecting dataflow tokens (e.g., as data values ​​or dataflow tokens representing data values) into the dataflow graph. Tokens can flow among them and can be transformed at each node (e.g., vertex), thereby forming, for example, a complete computation. Figures 3A-3C The diagram shows the sample data flow graph and its deviation from the high-level source code, and Figure 5 This shows an example of the execution of a data flow graph.

[0135] CSA implementations configure data flow graph execution by providing the exact data flow graph execution support required by the compiler. In one embodiment, CSA is an accelerator (e.g., Figure 2 (accelerators in the process), and it does not seek to provide performance in general-purpose processing cores (e.g., Figure 2 The CSA is a necessary but infrequently used mechanism available on the kernel (such as system calls). Therefore, in this embodiment, CSA can execute a lot of code, but not all of it. In exchange, CSA gains significant performance and energy advantages. To achieve the acceleration of code written in commonly used serialization languages, the embodiments described herein also introduce several novel architectural features to assist the compiler. One specific novelty is CSA's handling of memory, a subject previously neglected or poorly addressed. The CSA embodiment is also unique in using data stream manipulators (e.g., compared to using lookup tables (LUTs)) as its basic architectural interface.

[0136] Moving on to an implementation of CSA, the data stream manipulator is discussed below.

[0137] 1.1 Data Stream Manipulator

[0138] A key architectural interface of an accelerator (e.g., CSA) embodiment is a dataflow operator, for example, as a direct representation of a node in a dataflow graph. From an operational perspective, the dataflow operator is stream-based or data-driven. The dataflow operator is executable once its input operands are available. CSA dataflow execution can (e.g., only) depend on highly localized state, thus leading, for example, to a highly scalable architecture accompanying a distributed asynchronous execution model. Dataflow operators may include arithmetic dataflow operators, such as one or more of the following: floating-point addition and multiplication, integer addition, subtraction, and multiplication, various forms of comparison, logic operators, and shifting. However, CSA embodiments may also include a rich set of control operators for managing dataflow tokens in an auxiliary procedure graph. Examples of these control operators include "pick" operators (e.g., those that multiplex two or more logical input channels into a single output channel) and "switch" operators (e.g., those that operate as channel demultiplexers) (e.g., those that output a single channel from two or more logical input channels). These operators enable compilers to implement control paradigms (such as conditional expressions). Some embodiments of CSA may include (e.g., a finite set of dataflow operators relative to a small number of operations) to achieve a dense and energy-efficient PE microarchitecture. Some embodiments may include dataflow operators for complex operations common in HPC code. The CSA dataflow operator architecture is highly adaptable to deploying specific extensions. For example, more complex mathematical dataflow operators (e.g., trigonometric functions) may be included in some embodiments to accelerate certain mathematically intensive HPC workloads. Similarly, extensions adapted for neural networks may include dataflow operators for vectorized, low-precision arithmetic.

[0139] Figure 3A The illustration shows a program source according to an embodiment of this disclosure. The program source code includes a multiplication function (func). Figure 3B The illustrations depict embodiments of the present disclosure. Figure 3A The data flow graph 300 of the program source includes a pick node 304, a switching node 306, and a multiplication node 308. Buffers may optionally be included along one or more communication paths. The depicted data flow graph 300 can perform the following operations: select input X using the pick node 304, multiply X by Y (e.g., multiplication node 308), and then output the result from the left side of the switching node 306. Figure 3C The illustration shows an accelerator (e.g., CSA) according to an embodiment of the present disclosure, the accelerator having configurations for performing Figure 3BThe data flow graph 300 comprises multiple processing elements 301. More specifically, the data flow graph 300 is overlaid on an array of processing elements 301 (and, for example, multiple (e.g., interconnected) networks therebetween), such that each node in the data flow graph 300 is represented as a data flow operator in the array of processing elements 301. For example, some data flow operations may be implemented using processing elements, and / or some data flow operations may be implemented using a communication network (e.g., its network data flow endpoint circuitry). For example, Pick, PickSingleLeg, PickAny, Switch, and / or SwitchAny operations may be implemented using one or more components of the communication network (e.g., its network data flow endpoint circuitry), in contrast to the processing elements.

[0140] In one embodiment, one or more of the processing elements in the array of processing elements 301 are used to access memory via memory interface 302. In one embodiment, the pick node 304 of the data flow graph 300 thus corresponds to a pick operator 304A (e.g., represented by pick operator 304A), the switch node 306 of the data flow graph 300 thus corresponds to a switch operator 306A (e.g., represented by switch operator 306A), and the multiplier node 308 of the data flow graph 300 thus corresponds to a multiplier operator 308A (e.g., represented by multiplier operator 308A). Another processing element and / or flow control path network may provide control signals (e.g., control tokens) to the pick operator 304A and the switch operator 306A for execution. Figure 3A The operation in the process. In one embodiment, the array of processing elements 301 is configured to perform operations before execution begins. Figure 3B Data flow diagram 300. In one embodiment, the compiler executes from... Figures 3A to 3B The transformation. In one embodiment, the inputs to the data flow graph nodes of the array of processing elements logically embed the data flow graph into the array of processing elements (e.g., as discussed further below), such that the input / output paths are configured to produce the desired results.

[0141] 1.2 Channels that are not sensitive to waiting time

[0142] Communication arcs (arcs) are the second primary component of a data flow graph. Some embodiments of CSA describe these arcs as latency-insensitive channels, such as ordered, backpressure (e.g., output is generated or sent only when there is space available for storage), point-to-point communication channels. Like data flow manipulators, latency-insensitive channels are fundamentally asynchronous, thus providing freedom to combine many types of networks to implement channels for a particular graph. Latency-insensitive channels can have arbitrarily long latency times while still faithfully implementing the CSA architecture. However, in some embodiments, there is a strong incentive to minimize latency times in terms of performance and energy. Section 2.2 of this document discloses a network microarchitecture in which data flow graph channels are implemented in a pipelined manner with latency times of no more than one cycle. Embodiments of latency-insensitive channels provide a crucial abstraction layer that can be leveraged with the CSA architecture to provide numerous runtime services to application programmers. For example, CSA can utilize latency-insensitive channels when implementing CSA configuration (loading programs onto a CSA array).

[0143] Figure 4 The diagram illustrates an example of data flow diagram 400 according to an embodiment of this disclosure. In step 1, input values ​​(e.g., for...) are... Figure 3B X in 1 and for Figure 3B The 2 of Y in the data flow graph 400 can be loaded to perform a 1*2 multiplication operation. One or more of the data input values ​​can be static (e.g., constant) in the operation (e.g., refer to...). Figure 3B The data value 1 is output from select node 404 (where X is 1 and Y is 2) or updated during operation. In step 2, a processing element or other circuit (e.g., on a flow control path network) outputs 0 to the control input of select node 404 (e.g., a multiplexer control signal) (e.g., obtaining "1" from port as a source for its output) and outputs 0 to control the input of switching node 406 (e.g., a multiplexer control signal) (e.g., to make its input leave port "0" and be provided outward to the destination (e.g., a downstream processing element)). In step 3, the data value 1 is output from select node 404 (and, for example, consumes its control signal "0" at select node 404) to multiplier node 408 so as to be multiplied with the data value 2 at step 4. In step 4, the output of multiplier node 408 reaches switching node 406, for example, which causes switching node 406 to consume the control signal "0" to output the value 2 from port "0" of switching node 406 at step 5. The operation then completes. Therefore, the CSA can be programmed accordingly to enable the corresponding data stream operator of each node to execute. Figure 4 The operations in this example are serialized, but in principle all data stream operations can be executed in parallel. Figure 4The steps are used to distinguish data flow execution from any physical microarchitecture representation. In one embodiment, a downstream processing element is used (e.g., on a flow control path network) to send a signal (or not a ready signal) to switching device 406 to stop the output from switching device 406 until the downstream processing element is ready for output (e.g., with storage space).

[0144] 1.3 Memory

[0145] Dataflow architectures generally focus on communication and data manipulation, with less attention to state. However, enabling actual software, especially programs written in traditional serialization languages, requires a high degree of attention to memory interfacing. Some embodiments of CSA use architectural memory operations as their primary interface to (e.g., large) stateful memory. From a dataflow graph perspective, memory operations resemble other dataflow operations, except that they have the side effect of updating shared memory. Specifically, memory operations in some embodiments herein have the same semantics as every other dataflow operator; for example, these memory operations “execute” when their operands (e.g., addresses) are available and a response is generated after some latency. Some embodiments herein explicitly decouple operand inputs from result outputs, making memory operators inherently pipelined and capable of generating many concurrent pending requests, thus making memory operators highly adaptable to the latency and bandwidth characteristics of memory subsystems. Embodiments of CSA provide basic memory operations, such as load and store, which acquire an address channel and fill a response channel with a value corresponding to that address. Embodiments of CSA also provide more advanced operations, such as in-memory atomic and consistency operators. These operations can have semantics similar to their von Neumann equivalents. Implementations of CSA can accelerate existing programs described using serialization languages ​​such as C and Fortran. Supporting these language models results in addressing program memory order, for example, the serial ordering of memory operations typically defined by these languages.

[0146] Figure 5 The illustration shows a program source (e.g., C code) 500 according to an embodiment of this disclosure. According to the memory semantics of the C programming language, memory copying (memcpy) should be serialized. However, if it is known that array A and array B are disjoint, memcpy can be parallelized using embodiments of CSA. Figure 5The issue of program ordering is further illustrated. Generally, compilers cannot prove that array A is different from array B, for example, whether for the same index value or for different index values ​​across loop bodies. This is referred to as pointer or memory aliasing. Because compilers are used to generate statically correct code, they are often forced to serialize memory accesses. Typically, compilers for serialized von Neumann architectures use instruction ordering as a natural means of implementing program ordering. However, CSA implementations do not have the concept of instruction ordering or instruction-based program ordering as defined by the program counter. In some implementations, incoming dependency tokens (e.g., those that do not contain architecture-visible information) are like all other data flow tokens, and memory operations cannot execute until they receive a dependency token. In some implementations, memory operations generate outgoing dependency tokens once the operation of a memory operation becomes visible to logically subsequent dependent memory operations. In some implementations, dependency tokens are similar to other data flow tokens in the data flow graph. For example, because memory operations occur in a conditional context, dependency tokens can also be manipulated (e.g., like any other token) using the control operators described in Section 1.1. Dependency tokens can have the effect of serializing memory accesses, thereby providing the compiler, for example, with a means to define the order of memory accesses at the architectural level.

[0147] 1.4 Runtime Services

[0148] The main architectural aspects of CSA embodiments involve the actual execution of user-level programs, but providing several supporting mechanisms to solidify that execution is also desirable. The primary factors here are configuration (where a data flow graph is loaded into the CSA), fetching (where the state of the execution graph is moved to memory), and exception handling (where mathematical, soft, and other types of errors in the structure may be detected and handled by external entities). Section 2.8 below discusses the latency-insensitive data flow architecture properties of CSA embodiments used for efficient, highly pipelined implementations of these functions. Conceptually, configuration loads the state of the data flow graph (e.g., generally from memory) into interconnects (and / or communication networks (e.g., their network data flow endpoint circuits)) and processing elements (e.g., structures). During this step, all structures in the CSA can be loaded with a new data flow graph, and any data flow tokens survive in that graph, for example, as a result of a context switch. The latency-insensitive semantics of CSA allow for distributed asynchronous initialization of structures; for example, PEs can begin execution immediately upon configuration. Unconfigured PEs can backpush their channels until they are configured, thereby preventing, for example, communication between configured and unconfigured elements. CAS configuration can be partitioned into privileged and user-level states. Such a two-level partitioning allows the main configuration of the architecture to occur without invoking the operating system. In one embodiment of the extraction, a logical representation of the data flow graph is captured and committed to memory, including, for example, all realities in the graph, control and data flow tokens, and states.

[0149] Extraction can also play a role in providing reliability guarantees by creating structural checkpoints. Anomalies in CSA can generally be caused by the same events that lead to anomalies in the processor, such as illegal operator variables or reliability, availability, and durability (RAS) events. In some embodiments, anomalies are detected at the level of the dataflow operator (e.g., by verifying variable values) or through a modular arithmetic scheme. Upon detection of an anomaly, the dataflow operator (e.g., a circuit) may stop and emit an anomaly message, which may contain, for example, an operation identifier and some details about the nature of the problem that occurred. In some embodiments, the dataflow operator will remain stopped until it has been reconfigured. The anomaly message may then be passed to the associated processor (e.g., a core) for servicing (e.g., which may include an extraction graph for software analysis).

[0150] 1.5-chip architecture

[0151] Implementations of CSA computer architectures (e.g., for HPC and data center uses) are fragmented. Figure 6 and Figure 8 This shows the slice-level deployment of CSA. Figure 8This illustrates a full-chip implementation of CSA, which could be, for example, an accelerator for a processor with cores. A key advantage of this architecture is reduced design risk, such as allowing the CSA to be completely decoupled from the cores at manufacturing time. Besides allowing for better component reuse, this also allows components (like the CSA cache) to be designed solely for the CSA, rather than, for example, requiring more stringent latency requirements on the cores. Ultimately, the separate chips allow the CSA to be integrated with either small or large cores. One embodiment of CSA captures most vector-parallel workloads, allowing most vector-based workloads to run directly on the CSA; however, in some embodiments, vector-based instructions within the core can be included to, for example, support traditional binary instructions.

[0152] 2. Microarchitecture

[0153] In one embodiment, the purpose of the CSA microarchitecture is to provide a high-quality implementation of each dataflow operator specified by the CAS architecture. Implementations of the CSA microarchitecture provide that each processing element (and / or communication network (e.g., its network dataflow endpoint circuits)) of the microarchitecture corresponds to approximately one node (e.g., entity) in the architecture's dataflow graph. In one embodiment, the nodes in the dataflow graph are distributed across multiple network dataflow endpoint circuits. In some embodiments, this results in not only compact but also energy-efficient architectural elements; compactness leads to dense computational arrays, and energy efficiency, for example, processing elements (PEs) are both simple and highly unreused (e.g., performing a single dataflow operation for the configuration (e.g., programming) of the CSA). To further reduce energy consumption and implementation area, the CSA may include a configurable heterogeneous architecture style, where each PE implements only a subset of the dataflow operators (e.g., utilizing separate subsets of dataflow operators implemented as (multiple) network dataflow endpoint circuits). Peripheral and support subsystems (such as CSA caches) may be supplied to support the distributed parallelism present in the main CSA processing architecture itself. The CSA microarchitecture is implemented in a way that allows for data flow and latency-insensitive communication abstractions within the architecture. In some embodiments, there is a one-to-one correspondence (e.g., substantially) between nodes in the compiler-generated graph and data flow operators (e.g., data flow operator computational elements) in the CSA.

[0154] The following is a discussion of an example CSA, followed by a more detailed discussion of the microarchitecture. Some embodiments in this paper provide CSAs that allow for easy compilation, for example, in contrast to existing FPGA compilers that handle a small subset of programming languages ​​(e.g., C or C++) and require many hours to compile even small programs.

[0155] Some embodiments of the CSA architecture allow heterogeneous coarse-grained operations such as double-precision floating-point. Programs can be expressed with less coarse-grained operations, for example, making the disclosed compiler run faster than a traditional spatial compiler. Some embodiments include architectures with new processing elements to support serialization concepts such as program-ordered memory access. Some embodiments implement hardware for supporting coarse-grained dataflow communication channels. This communication model is abstract and very close to the control dataflow representation used by the compiler. Some embodiments in this document include network implementations that support single-cycle wait-time communication, for example, utilizing (e.g., small) PEs that support single control dataflow operations. In some embodiments, this not only improves energy efficiency and performance but also simplifies compilation because the compiler performs a one-to-one mapping between high-level dataflow constructs and structures. Some embodiments in this document thus simplify the task of compiling existing (e.g., C, C++, or Fortran) programs to CSA (e.g., structures).

[0156] Energy efficiency is often a primary consideration in modern computer systems. Certain embodiments described herein offer new paradigms for energy-efficient spatial architectures. In some embodiments, these architectures form a unique heterogeneous mix of small, energy-efficient, dataflow-oriented processing elements (PEs) (and / or packet-switched communication networks (e.g., their network dataflow endpoint circuits)) and lightweight circuit-switched communication networks (e.g., interconnects), for example, with enhanced support for flow control. Due to the energy advantages of each, the combination of these components can form a spatial accelerator (e.g., as part of a computer) suitable for executing compiler-generated parallel programs in an extremely energy-efficient manner. Because the architecture is heterogeneous, certain embodiments can be tailored for different application domains by introducing new domain-specific PEs. For example, an architecture for high-performance computing may include some customization for double-precision, fused multiply-add operations, while an architecture for deep neural networks may include low-precision floating-point operations.

[0157] Spatial architecture patterns (e.g., such as...) Figure 6 (As illustrated in the example) consists of lightweight PEs that are networked together between processing elements (PEs). Generally, a PE may include a dataflow operator, where, for example, once (e.g., all) input operands arrive at the dataflow operator, an operation (e.g., a microinstruction or set of microinstructions) is executed, and the result is forwarded to a downstream operator. Therefore, control, scheduling, and data storage can be distributed across multiple PEs, thereby eliminating, for example, the overhead of the centralized architecture that dominates classic processors.

[0158] A program can be converted into a data flow graph representing the program's control data flow graph by configuring PEs and networks, which is then mapped onto the architecture. Communication channels can be flow-controlled and fully backpressured, such that the PE will stop, for example, if the source communication channel has no data or the destination communication channel is full. In one embodiment, at runtime, data flows through PEs and channels that have been configured to implement operations (e.g., accelerated algorithms). For example, data may flow from memory through the structure and then flow back out to memory.

[0159] Implementations of this architecture achieve superior performance efficiency compared to conventional multi-core processors: computation (e.g., in the form of PE) can be simpler, more energy-efficient, and richer than in larger cores, and communication can be direct and primarily short-range, in contrast to, for example, occurring over a wide, full-chip network as in typical multi-core processors. Furthermore, because the implementations of this architecture are highly parallel, numerous robust circuit and device-level optimizations are possible without significantly impacting throughput, such as low-leakage devices and low operating voltages. These lower-level optimizations can achieve even greater performance advantages relative to conventional cores. The combination of efficiencies at the architecture, circuit, and device levels in these implementations is remarkable. As transistor density continues to increase, implementations of this architecture can achieve even larger effective areas.

[0160] The embodiments described herein offer a unique combination of data flow support and circuit switching to enable smaller, more energy-efficient architectures with higher converged performance compared to previous architectures. FPGAs are generally geared towards fine-grained bit manipulation, while the embodiments described herein are geared towards double-precision floating-point operations found in HPC applications. Some embodiments described herein may also include an FPGA in addition to the CSA according to this disclosure.

[0161] Some embodiments described herein combine lightweight networks with energy-efficient dataflow processing elements (and / or communication networks (e.g., their network dataflow endpoint circuitry)) to form a high-throughput, low-latency, energy-efficient HPC architecture. This low-latency network allows for the establishment of processing elements (and / or communication networks (e.g., their network dataflow endpoint circuitry)) with fewer functions (e.g., only one or two instructions, and perhaps only one architecture-visible register, since it is efficient to aggregate multiple PEs together to form a complete program).

[0162] Compared to processor cores, the CSA embodiments described herein offer greater computational density and energy efficiency. For example, when PE (e.g., compared to cores) is very small, a CSA can perform far more operations than a core and can have far more computational parallelism, perhaps up to 16 times the number of FMAs as vector processing units (VPUs). To utilize all these computational elements, in some embodiments, the energy per operation is very low.

[0163] The energy advantages of embodiments of the dataflow architecture of this application are numerous. Parallelism is explicit in the dataflow graph, and embodiments of the CSA architecture consume little or no energy to extract this parallelism, unlike out-of-order processors, which must rediscover parallelism every time an instruction is executed. In one embodiment, since each PE is responsible for a single operation, the register set and port count can be small, often only one, and therefore use less energy than their counterparts in the core. Some CSAs include many PEs, each of which holds the live program value, thus giving the aggregation effect of the massive register sets in conventional architectures, which significantly reduces memory accesses. In embodiments where memory is multi-ported and distributed, CSAs can handle far more pending memory requests and utilize more bandwidth than the core. These advantages can be combined to achieve energy levels that are only a small percentage of the cost per watt for bare arithmetic circuitry. For example, in the case of integer multiplication, a CSA can consume no more than 25% more energy than the underlying multiplication circuitry. In one embodiment relative to the core, integer operations in that CSA architecture consume less than 1 / 30th of the energy of each integer operation.

[0164] From a programming perspective, the application-specific compliant implementation of CSA architecture offers significant advantages over Vector Processing Units (VPUs). In traditional, inflexible architectures, the number of functional units, such as floating-point division or various transcendental math functions, must be selected at design time based on some desired use cases. In CSA architecture implementations, such functions can be configured into the architecture (e.g., by the user, not the manufacturer) based on the requirements of each application. Application throughput can thus be further increased. Simultaneously, by avoiding the fixation of such functions and instead provisioning more instances of primitive functions like floating-point multiplication, the computational density of CSA implementations is improved. These advantages can be significant in HPC workloads, some of which consume 75% of floating-point execution time in transcendental functions.

[0165] Some embodiments of CSA represent significant advancements in dataflow-oriented spatial architectures; for example, the PEs of this disclosure can be smaller yet more efficient. These improvements can stem directly from the combination of dataflow-oriented PEs with lightweight, circuit-switched interconnects, which, for example, have single-cycle latency, in contrast to packet-switched networks (e.g., with latency at least 300% higher). Some embodiments of the PE support 32-bit or 64-bit operation. Some embodiments herein permit the introduction of new application-specific PEs, for example, for machine learning or security, and not just homogeneous combinations. Some embodiments herein combine lightweight, dataflow-oriented processing elements with lightweight, low-latency networks to form energy-efficient computing architectures.

[0166] To enable certain spatial architectures to succeed, programmers will spend relatively little effort configuring them, for example, simultaneously achieving significant power and performance advantages relative to serialization kernels. Some embodiments described herein provide CSAs (e.g., spatial structures) that are easy to program (e.g., by a compiler), highly power-efficient, and highly parallel. Some embodiments described herein provide networks (e.g., interconnects) that achieve these three goals. From a programmability perspective, some network embodiments provide flow-controlled channels, which correspond, for example, to the control data flow graph (CDFG) model of execution used in a compiler. Some network embodiments utilize dedicated circuit-switched links, making program performance easier to deduce by both humans and compilers, as performance is predictable. Some network embodiments provide both high bandwidth and low latency. Some network embodiments (e.g., static, circuit-switched) provide latency of 0 to 1 cycles (e.g., depending on transmission distance). Some network embodiments provide high bandwidth by arranging several networks in parallel (and, for example, in low-level metal). Some network embodiments are in low-level metal and communicate over short distances, thus achieving very high power efficiency.

[0167] Some embodiments of the network include architectural support for flow control. For example, in a space accelerator composed of small processing elements (PEs), communication latency and bandwidth can be critical to overall program performance. Some embodiments herein provide lightweight, circuit-switched networks and the microarchitectural control features required to support such networks, which facilitate space processing arrays (such as…) Figure 6Communication between PEs (processors and devices) in the spatial array shown. Some embodiments of the network implement the construction of point-to-point, flow-controlled communication channels that support communication between data-stream-oriented processing elements (PEs). In addition to point-to-point communication, some networks described herein also support multicast communication. Communication channels can be formed by statically configuring the network to create virtual circuits between PEs. The circuit-switching techniques described herein can reduce communication latency and correspondingly minimize network buffering, thereby achieving, for example, both high performance and high energy efficiency. In some embodiments of the network, latency between PEs can be as low as zero cycles, meaning that downstream PEs can process the data within that cycle after it is generated. To obtain even higher bandwidth and to allow for more procedures, multiple networks can be arranged in parallel, for example, as shown in... Figure 6 As shown in the image.

[0168] Spatial architecture (such as, Figure 6 The spatial architecture shown can be composed of lightweight processing elements connected by a network (and / or communication network, e.g., its network data flow endpoint circuitry) between PEs. A program, viewed as a dataflow graph, can be mapped onto the architecture by configuring the PEs and the network. Generally, a PE can be configured as a dataflow operator, and once (e.g., all) input operands arrive at the PE, subsequent operations can occur, and the results are forwarded to the desired downstream PE. PEs can communicate via dedicated virtual circuits formed by statically configuring a circuit-switched communication network. These virtual circuits can be flow-controlled and fully back-pressured, such that the PE will stop, for example, if the source has no data or the destination is full. During runtime, data can flow through PEs implementing the mapped algorithm. For example, data can flow in from memory through the structure and then back out to memory. Embodiments of this architecture achieve superior performance efficiency compared to conventional multi-core processors: for example, computation in the form of PEs is simpler and more numerous than with larger cores, and communication is direct, unlike extended memory systems.

[0169] Figure 6The figure illustrates an accelerator chip 600 according to an embodiment of the present disclosure, which includes an array of processing elements (PEs). The interconnect network is depicted as a circuit-switched, statically configured communication channel. For example, a set of channels is coupled together by switching means (e.g., switching means 610 in a first network and switching means 611 in a second network). The first and second networks may be separate or may be coupled together. For example, switching means 610 may couple together one or more of four data paths 612, 614, 616, 618, for example, configured to perform operations according to a data flow graph. In one embodiment, the number of data paths is arbitrary. Processing elements (e.g., processing element 604) may be as disclosed herein, for example, as... Figure 9 As shown in the image. Accelerator chip 600 includes a memory / cache hierarchy interface 602 to interface, for example, accelerator chip 600 with memory and / or cache. Data paths (e.g., 618) may extend to another chip or may terminate at, for example, the edge of a chip. Processing elements may include input buffers (e.g., buffer 606) and output buffers (e.g., buffer 608).

[0170] Operations can be performed based on the availability of inputs for those operations and the state of the PE. The PE can obtain operands from input channels and write results to output channels, but can also use internal register states. Some embodiments described herein include configurable dataflow-friendly PEs. Figure 9 A detailed block diagram of such a PE is shown: an integer PE. This PE consists of several I / O buffers, an ALU, memory registers, some instruction registers, and a scheduler. In each cycle, the scheduler selects an instruction to execute based on the availability of the input and output buffers and the state of the PE. The result of the operation is then written to an output register, or to a register (e.g., local to the PE). Data written to the output buffer can be transferred to downstream PEs for further processing. This PE style can be extremely energy efficient; for example, instead of reading data from a complex multi-port register set, the PE reads data from a register. Similarly, instructions can be stored directly in registers instead of in a virtualized instruction cache.

[0171] The instruction register can be set during a specific configuration step. During this step, in addition to the inter-PE network, auxiliary control lines and states can also be used to allow configuration to flow across several PEs in the containing structure. As a result of parallelism, some embodiments of such networks can provide rapid reconfiguration; for example, a chip-sized structure can be configured in less than about 10 microseconds.

[0172] Figure 9This represents an example configuration of a processing element, where all architectural element sizes are set to a minimum. In other embodiments, each of the multiple components of the processing element is scaled independently to produce a new PE. For example, to handle more complex programs, a greater number of instructions executable by the PE can be introduced. A second dimension of configurability is the functionality of the PE's arithmetic logic unit (ALU). Figure 9 In this context, integer PEs are described as supporting addition, subtraction, and various logical operations. Other types of PEs can be created by replacing different types of functional units within the PE. For example, an integer multiplication PE may not have registers, have a single instruction, and a single output buffer. Some embodiments of PEs deconstruct the fused multiply-add (FMA) into separate but tightly coupled floating-point multiplication and addition units to improve support for multiply-add heavy workloads. PEs are discussed further below.

[0173] Figure 7A The illustrations are based on embodiments of this disclosure (e.g., see reference ). Figure 6 The configurable data path network 700 (in Network 1 or Network 2) discussed herein. Network 700 includes multiple multiplexers (e.g., multiplexers 702, 704, 706) that can be configured (e.g., via their respective control signals) to connect one or more data paths (e.g., from PE) together. Figure 7B The illustrations are based on embodiments of this disclosure (e.g., see reference ). Figure 6 The configurable flow control path network 701 (either Network 1 or Network 2 under discussion) can be a lightweight PE-to-PE network. Some embodiments of the network can be viewed as a collection of constituent primitives for constructing distributed peer-to-peer data channels. Figure 7A The diagram illustrates a network where two channels (bold black lines and dotted black lines) are enabled. The bold black line channel is multicast, meaning a single input is sent to two outputs. Note that even though a dedicated circuit-switched path is formed between the channel endpoints, the channels can cross at some points within a single network. Furthermore, this crossing does not introduce structural hazards between the two channels, allowing each to operate independently and at full bandwidth.

[0174] Implementing a distributed data channel may include Figures 7A-7B The two paths shown are forward or data paths that carry data from the producer to the consumer. Multiplexers can be configured to guide data and valid bits from the producer to the consumer, for example, as... Figure 7A As shown in the diagram. In the case of multicast, data will be directed to multiple consumer endpoints. The second part of this embodiment of the network is a flow control or backpressure path, which flows in the opposite direction to the forward data path, for example, as... Figure 7BAs shown in the diagram. The consuming endpoints can assert when they are ready to accept new data. Subsequently, configurable logical nodes (in...) can be used... Figure 7B The flow control function (labeled as, for example, backflow) directs these signals back to the production side. In one embodiment, each flow control function circuit can be multiple switching devices (e.g., multiple muxes), for example, with... Figure 7A Similarly, the flow control path can handle the return of control data from the consumer to the producer. The connection can enable multicast, for example, where each consumer is ready to receive data before the producer assumes it has been received. In one embodiment, the PE is a data flow operator having a data flow operator as its architectural interface. Additionally or alternatively, in one embodiment, the PE can be (e.g., in the architecture) any kind of PE, such as, but not limited to, a PE with an instruction pointer, triggered instruction, or state machine-based architectural interface.

[0175] In addition to the PE being statically configured, the network can also be statically configured. During this configuration step, configuration bits can be set at each network component. These bits control, for example, multiplexer selection and flow control functions. The network may include multiple networks, such as a data path network and a flow control path network. The network or multiple networks may utilize paths of different widths (e.g., a first width and narrower or wider widths). In one embodiment, the data path network has a wider width (e.g., bit transmission width) than the flow control path network. In one embodiment, each of the first and second networks includes its own data path network and flow control path network, for example, data path network A and flow control path network A, and a wider data path network B and flow control path network B.

[0176] Some embodiments of the network are unbuffered, and data is used to move between producers and consumers in a single cycle. Some embodiments of the network are also borderless, meaning the network spans the entire structure. In one embodiment, a PE is used to communicate with any other PE in a single cycle. In one embodiment, to improve routing bandwidth, several networks can be arranged in parallel across rows of PEs.

[0177] Compared to FPGAs, certain embodiments of the networks described herein offer three advantages: area, frequency, and programmability. Some embodiments of the networks operate with coarse-grained precision, reducing, for example, the number of configuration bits and thus the network area. Some embodiments of the networks also achieve area reduction by implementing flow control logic directly in the circuit (e.g., silicon). Certain embodiments of the enhanced network implementations also enjoy a frequency advantage relative to FPGAs. Due to the area and frequency advantages, a power advantage can exist when lower voltages are used at throughput parity. Finally, some embodiments of the networks provide better high-level semantics than FPGA lines, especially relative to variable timing aspects; therefore, those embodiments are easier for compilers to target. Some embodiments of the networks described herein can be viewed as a collection of constituent primitives for constructing distributed point-to-point data channels.

[0178] In some embodiments, a multicast source may not be able to assert the validity of its data unless the multicast source receives a ready signal from each sink. Therefore, in the case of multicast, additional junctions and control bits can be utilized.

[0179] Like some PEs, the network can be statically configured. During this step, configuration bits are set at each network component. These bits control, for example, multiplexer selection and flow control functions. The forward path of the network in this application requires some bits to allow the mux to swing along that forward path. Figure 7A In the example shown, four bits are required per hop: one bit for each of the east and west muxes, and two bits for the southbound multiplexer. In this embodiment, four bits are available for the data path, but seven bits can be used for flow control functions (e.g., in a flow control path network). Other embodiments may utilize more bits if, for example, the CSA further utilizes the north-south direction. Flow control functions can use control bits for each direction from which flow control can be implemented. This allows for statically setting the sensitivity of the flow control function. Table 1 below summarizes the requirements for... Figure 7B The network's flow control functionality is implemented using a Boolean algebra method, with configuration bits uppercase. In this example, seven bits are used.

[0180] Table 1: Stream Implementation Methods

[0181]

[0182] For from Figure 7B The third flow control box from the left in the diagram, EAST_WEST_SENSITIVE and NORTH_SOUTH_SENSITIVE, are described as being set to implement flow control for the bold line channel and the dotted line channel, respectively.

[0183] Figure 8The illustration shows a hardware processor chip 800 including an accelerator 802 according to an embodiment of the present disclosure. Accelerator 802 may be a CSA according to the present disclosure. Chip 800 includes multiple cache blocks (e.g., cache block 808). Request Address File (RAF) circuitry 810 may be included, for example, as discussed in Section 2.2 below. ODI may refer to on-die interconnect, e.g., an interconnect that extends across the entire die and connects all chips. OTI may refer to (e.g., an on-chip interconnect that extends across chips, e.g., connects on-chip cache blocks together).

[0184] 2.1 Processing Components

[0185] In some embodiments, the CSA includes an array of heterogeneous PEs, wherein the structure consists of several types of PEs, each of which implements only a subset of data stream operators. As an example, Figure 9 This illustrates a tentative implementation of a PE capable of implementing a broad set of integer and control operations. Other PEs (including those supporting floating-point addition, floating-point multiplication, buffering, and certain control operations) may also have similar implementation styles, for example, replacing the ALU with appropriate (dataflow manipulator) circuitry. Before execution begins, the CSA's PE (e.g., a dataflow manipulator) can be configured (e.g., programmed) to implement specific dataflow operations from the set supported by the PE. Configuration may include one or two control words specifying the opcodes controlling the ALU, directing various multiplexers within the PE, and driving data flow into and out of the PE channel. The dataflow manipulator can be implemented by micro-encoding these configuration bits. Figure 9 The integer PE 900 depicted is organized as a single-stage logic pipeline flowing from top to bottom. Data enters the PE 900 from one of the local networks in the local network set, where it is registered in an input buffer for subsequent operations. Each PE can support multiple wide data-oriented channels and narrow control-oriented channels. The number of channels supplied can vary depending on the PE's functionality, but one embodiment of an integer-oriented PE has two wide and one or two narrow input and output channels. While the integer PE is implemented as a single-cycle pipeline, other pipelined options can be utilized. For example, a multiplication PE can have multiple pipeline stages.

[0186] PE execution can continue in a data flow pattern. Based on the configuration microcode, the scheduler can check the status of the PE inlet and outlet buffers, and when all inputs for the configured operation have arrived and the outlet buffer for the operation is available, schedule the actual execution of the operation by a data manipulator (e.g., on the ALU). The resulting value can be placed in the configured outlet buffer. Transfers between the outlet buffer of one PE and the inlet buffer of another PE can occur asynchronously when the buffer becomes available. In some embodiments, providing a PE ensures that at least one data flow operation completes for each cycle. Section 2 discusses data flow manipulators covering primitive operations such as add, xor, or pick. Some embodiments offer advantages in terms of energy, area, performance, and latency. In one embodiment, more fusion combinations can be enabled by extending the PE control path. In one embodiment, the processing element is 64 bits wide, for example, for high utilization of double-precision floating-point calculations in HPC and for supporting 64-bit memory addressing.

[0187] 2.2 Communication Network

[0188] Implementations of the CSA microarchitecture provide a hierarchy of multiple networks that together provide an implementation of an architectural abstraction for latency-insensitive channels across multiple communication proportions. The lowest level of the CSA communication hierarchy can be a local network. Local networks can be statically circuit-switched, for example, using configuration registers to oscillate multiplexers(s) in the local network data path to form a fixed electrical path between communicating PEs. In one embodiment, the configuration of the local network is set once for each data flow graph (e.g., simultaneously with PE configuration). In one embodiment, static circuit-switching is optimized for energy efficiency, for example, where the vast majority (perhaps greater than 95%) of CSA communication traffic will traverse the local network. Procedures may include terms used in multiple expressions. To optimize for this situation, embodiments herein provide hardware support for multicast within the local network. Several local networks can be aggregated to form routing channels, which are spread (as a grid) between rows and columns of PEs, for example. As an optimization, several local networks can be included to carry control tokens. Compared to FPGA interconnects, CSA local networks can be routed at the granularity of data paths, and another difference may be the handling of control by CSA. One embodiment of CSA local networks is explicitly flow-controlled (e.g., backpressure). For example, for each forward data path and multiplexer set, CSA is used to provide a backward flow control path that is physically paired with the forward data path. The combination of two microarchitectural paths can provide a low latency, low energy, small area, point-to-point implementation of a latency-insensitive channel abstraction. In one embodiment, the flow control lines of CSA are not visible to the user program, but these flow control lines can be manipulated by the architecture maintaining the user program. For example, the exception handling mechanism described in Section 1.2 can be implemented by pulling the flow control lines to a "non-existent" state after an anomalous condition is detected. This action not only gently stops those parts of the pipeline involved in offensive computation, but also keeps the machine state prior to the exception for, for example, diagnostic analysis. The second network layer (e.g., a mezzanine network) can be a shared packet-switched network. A mezzanine network may include multiple distributed network controllers and network data flow endpoint circuits. A mezzanine network (e.g., composed of...) Figure 51The network (illustrated by the dashed box in the diagram) can provide more general long-distance communication at the cost of, for example, latency, bandwidth, and energy. In some applications, most communication can occur on the local network, thus significantly reducing the need for a mezzanine network. For example, each PE can connect to multiple local networks, but the CSA will only supply one mezzanine endpoint for each logical neighborhood of the PE. Since the mezzanine is essentially a shared network, each mezzanine network can carry multiple logically independent channels and be supplied, for example, as multiple virtual channels. In one embodiment, the primary function of the mezzanine network is to provide wide-range communication between PEs and between PEs and memory. In addition to this capability, the mezzanine may also include, for example, network data flow endpoint circuitry for certain data flow operations. Furthermore, the mezzanine can operate as a runtime support network, through which various services can access the full infrastructure in a user-program transparent manner. In this respect, mezzanine endpoints can act as controllers for their local neighborhoods, for example, during CSA configuration. To form a channel across a CSA slice, three sub-channels and two local network channels can be used (which carry traffic to and from a single channel in the mezzanine network). In one embodiment, one mezzanine channel is used, for example, one mezzanine and two local = a total of 3 network hops.

[0189] The combinability of channels across network layers is extended to higher-level network layers at the inter-chip, inter-die, and structural granularity levels.

[0190] Figure 9 The illustration shows a processing element 900 according to an embodiment of the present disclosure. In one embodiment, an operation configuration register 919 is loaded during configuration (e.g., mapping) and specifies a particular operation (or multiple operations) that the processing (e.g., computing element) will perform. The activity of register 920 may be controlled by that operation (the output of multiplexer 916, e.g., controlled by scheduler 914). For example, scheduler 914 may schedule one or more operations of processing element 900 when input data and control inputs arrive. A control input buffer 922 is connected to a local network 902 (e.g., and local network 902 may include, for example,...). Figure 7A Data path network and such Figure 7BThe control input buffer 922 is loaded with a value (e.g., the network has multiple data bits and multiple valid bits) when a value arrives (e.g., the network has multiple data bits and multiple valid bits). The control output buffer 932, data output buffer 934, and / or data output buffer 936 can receive the output of the processing element 900 (e.g., controlled by an operation (the output of multiplexer 916)). The status register 938 can be loaded whenever the ALU 918 executes (also controlled by the output of multiplexer 916). The data in the control input buffer 922 and control output buffer 932 can be a single bit. Multiplexers 921 (e.g., operand A) and 923 (e.g., operand B) can serve as sources of input.

[0191] For example, suppose the operation of the processing (e.g., calculation) element is (or includes) in Figure 3B This is referred to as the call-select operation. Processing element 900 is used to select data from data input buffer 924 or data input buffer 926, for example, to go to data output buffer 934 (e.g., by default) or data output buffer 936. Therefore, if selection is made from data input buffer 924, the control bit in 922 can indicate 0, or if selection is made from data input buffer 926, the control bit in 922 can indicate 1.

[0192] For example, suppose the operation of the processing (e.g., calculation) element is (or includes) in Figure 3B This is referred to as a call switching operation. Processing element 900 is used to output data from, for example, data input buffer 924 (e.g., by default) or data input buffer 926 to data output buffer 934 or data output buffer 936. Therefore, if the output is to data output buffer 934, the control bit in 922 can indicate 0, or if the output is to data output buffer 936, the control bit in 922 can indicate 1.

[0193] Multiple networks (e.g., interconnected) (e.g., input networks 902, 904, 906 and output networks 908, 910, 912) can be connected to the processing element. The connection can be, for example, referenced... Figure 7A and Figure 7B The switching device under discussion. In one embodiment, each network includes two subnetworks (or two channels on the network), for example, one for... Figure 7A The data path network in the middle, one used for Figure 7B The flow control (e.g., backpressure) path network in the network. As an example, a local network 902 (e.g., established as a control interconnect) is depicted as being switched (e.g., connected) to a control input buffer 922. In this embodiment, the data path (e.g., Figure 7AThe network in the process may carry control input values ​​(e.g., one or more bits) (e.g., control tokens), and the flow control path (e.g., the network) may carry backpressure signals (e.g., backpressure or no-backpressure tokens) from the control input buffer 922 to, for example, indicate to the upstream producer (e.g., PE) that a new control input value will not be loaded (e.g., sent to) the control input buffer 922 until the backpressure signal indicates that there is space in the control input buffer 922 for (e.g., from the upstream producer's control output buffer). In one embodiment, a new control input value may not enter the control input buffer 922 until both (i) the upstream producer receives a "space available" backpressure signal from the "control input" buffer 922 and (ii) a new control input value is sent from the upstream producer, for example, and this may cause the processing element 900 to stop until that happens (and space becomes available in the target, output buffers).

[0194] Data input buffers 924 and 926 can be implemented in a similar manner; for example, a local network 904 (e.g., established as a data (as opposed to control) interconnect) is depicted as being switched (e.g., connected) to data input buffer 924. In this embodiment, the data path (e.g., Figure 7A The network in the process may carry a data input value (e.g., one or more bits) (e.g., a data flow token), and the flow control path (e.g., the network) may carry a backpressure signal (e.g., a backpressure or no-backpressure token) from the data input buffer 924 to, for example, indicate to the upstream producer (e.g., PE) that a new data input value will not be loaded (e.g., sent to) the data input buffer 924 until the backpressure signal indicates that there is space in the data input buffer 924 for (e.g., from the upstream producer's data output buffer). In one embodiment, a new data input value may not enter the data input buffer 924 until both (i) the upstream producer receives a "space available" backpressure signal from the "data input" buffer 924 and (ii) a new data input value is sent from the upstream producer, for example, and this may cause the processing element 900 to stop until that happens (and space is available in the (multiple) target, output buffers). Control output values ​​and / or data outputs can be stopped in their respective output buffers (e.g., 932, 934, 936) until a back pressure signal indicates that there is available space in the input buffer for downstream processing elements(s).

[0195] Processing element 900 may cease execution until its operands (e.g., control input values ​​and one or more corresponding data input values ​​of the control input values) are received and / or until the processing element 900’s output buffer(s) have space for the data generated by performing operations on those operands.

[0196] 2.3 Memory Interface

[0197] Request Address File (RAF) circuit (in) Figure 10The simplified version shown in the diagram can be responsible for performing memory operations and acting as an intermediary between the CSA structure and the memory hierarchy. Thus, the primary microarchitectural task of the RAF can be to rationalize the out-of-order memory subsystem with the ordered semantics of the CSA structure. In this respect, the RAF circuitry can provide completion buffers (e.g., queue-like structures) that reorder memory responses and return these memory requests to the structure in the order they were requested. A second primary function of the RAF circuitry can be to provide support in the form of address translation and page walkers. Incoming virtual addresses can be translated into physical addresses using channel-associated translation lookaside buffers (TLBs). To provide sufficient memory bandwidth, each CSA chip can include multiple RAF circuits. Like the various PEs of the structure, the RAF circuitry can operate in a dataflow-style manner by checking the availability of input arguments and output buffers (if needed) before selecting the memory operation to be performed. However, unlike some PEs, the RAF circuitry is multiplexed among several co-located memory operations. Multiplexed RAF circuitry can be used to minimize the area overhead of its individual sub-components, such as shared accelerator cache interface (ACI) networks (described in more detail in Section 2.4), shared virtual memory (SVM) support hardware, mezzanine network interfaces, and other hardware management facilities. However, some procedural characteristics also facilitate this choice. In one embodiment, an efficient data flow graph is used to poll the memory in a shared virtual memory system. Memory latency-constrained procedures (like graph traversal) can utilize many separate memory operations to saturate memory bandwidth due to memory-dependent control flow. While each RAF can be multiplexed, CAS can include multiple (e.g., between 8 and 32) RAFs at a chip-granularity to ensure sufficient cache bandwidth. RAFs can communicate with the rest of the architecture via both the local network and the mezzanine network. In the case of RAF multiplexing, each RAF can be provisioned to the local network along with several ports. These ports can act as low-latency, highly deterministic paths to memory for use by latency-sensitive or high-bandwidth memory operations. In addition, RAF can be provided with mezzanine network endpoints, such as those that provide memory access to runtime services and remote user-level memory access providers.

[0198] Figure 10The diagram illustrates a Request Address File (RAF) circuit 1000 according to an embodiment of the present disclosure. In one embodiment, memory load and store operations already in the data flow graph are specified in register 1010 during configuration. Subsequently, arcs to those memory operations in the data flow graph can be connected to input queues 1022, 1024, and 1026. Arcs from those memory operations are thus used to exit completion buffers 1028, 1030, or 1032. Dependency tokens (which may each be a single bit) arrive at queues 1018 and 1020. Dependency tokens will exit from queue 1016. A dependency token counter 1014 may be a compact representation of the queues and can track the number of dependency tokens used for any given input queue. If the dependency token counter 1014 saturates, no additional dependency tokens can be generated for new memory operations. Accordingly, a memory sorting circuit (e.g., Figure 11 The RAF in the memory will stop scheduling new memory operations until the dependency token counter 1014 becomes unsaturated.

[0199] As an example of loading, an address arrives at queue 1022, and scheduler 1012 matches queue 1022 with the loads in 1010. The completion buffer slots for this load are assigned in the order the addresses arrived. Assuming that this particular load in the diagram has no specified dependencies, the address and completion buffer slot are dispatched to the memory system by the scheduler (e.g., via memory command 1042). When the result returns to multiplexer 1040 (illustratively shown), the result is stored in the completion buffer slot it was assigned (e.g., because the result carries the target slot throughout its journey through the memory system). The completion buffers send the results back to the local network (e.g., local networks 1002, 1004, 1006, or 1008) in the order the addresses arrived.

[0200] Storage can be simpler, except that both the address and the data must arrive before any operation is dispatched to the memory system.

[0201] 2.4 Cache

[0202] Data flow graphs can potentially generate a large number of (e.g., word-granular) requests in parallel. Therefore, some implementations of CSA dedicate sufficient bandwidth to the caching subsystem to maintain the CSA. Highly segmented caching microarchitectures (e.g., such as...) can be utilized. Figure 11 (as shown in the image). Figure 11The diagram illustrates a circuit 1100 according to an embodiment of the present disclosure, having multiple Request Address File (RAF) circuits (e.g., RAF circuit 1) coupled between multiple accelerator chips 1108, 1110, 1112, 1114 and multiple cache blocks (e.g., cache block 1102). In one embodiment, the number of RAFs and cache blocks may be in a 1:1 or 1:2 ratio. A cache block may include complete cache lines (e.g., in contrast to word-based partitioning), and each line has an exact home location within that cache. Cache lines may be mapped to cache blocks via a pseudo-random function. The CSA may employ a shared virtual memory (SVM) model for integration with other sharding architectures. Some embodiments include an Accelerator Cache Interface (ACI) network connecting the RAFs to cache blocks. This network can carry addresses and data between the RAFs and the cache. The ACI topology may be a cascaded crossbar switch, for example, as a trade-off between latency and implementation complexity.

[0203] 2.5 Network resources used to perform operations (e.g., data streams), such as circuits.

[0204] In some embodiments, the processing element (PE) communicates using dedicated virtual circuits formed by statically configuring (e.g., circuit-switched) communication networks. These virtual circuits can be flow-controlled and fully back-pressurized, such that the PE will stop, for example, if the source has no data or the PE's destination is full. During runtime, data can flow through the PE, which implements a mapped data flow graph (e.g., a mapped algorithm). For example, data can be fed from memory via a spatial array (e.g., a structural region) of the processing element, and then returned to memory.

[0205] Such architectures offer superior performance efficiency compared to traditional multi-core processors: for example, computation in PE form can be simpler and more frequent than in cores, and communication can be direct, for example, in contrast to the expansion of memory systems. However, the spatial array of processing elements (e.g., structural regions) can be tuned for the implementation of compiler-generated expression trees, which can be characterized by less reuse or de-reuse. Some embodiments herein extend the architecture (e.g., via network resources, such as, but not limited to, network data stream endpoint circuits) to support (e.g., high-radix) reuse and / or de-reuse, particularly in the context of function calls.

[0206] Spatial arrays, such as Figure 1 The spatial array 101 of processing elements can use (e.g., packet-switched) networks for communication. Some embodiments herein provide circuitry for overlaying high-radius data stream operations over these networks for communication. For example, some embodiments herein utilize existing networks for communication (e.g., reference...). Figure 1 The described interconnect network 104 provides data routing capabilities between processing elements and other components of the spatial array, but also enhances the network (e.g., network endpoints) to support the execution and / or control of some (e.g., less than all) of data stream operations (e.g., without requiring processing elements to perform those data stream operations). In one embodiment, special hardware structures within the spatial array (e.g., network data stream endpoint circuitry) are utilized to support (e.g., high-radix) data stream operations, e.g., without consuming processing resources or degrading (e.g., the performance of processing elements).

[0207] In one embodiment, a circuit-switched network between two points (e.g., between a data producer and a consumer) includes a dedicated communication line between those two points, for example, by setting up a (e.g., physical) switching device between the two points to create a (e.g., exclusive) physical circuit between them. In one embodiment, the circuit-switched network between the two points is established when the connection between them is first used and is maintained during the use of the connection. In another embodiment, a packet-switched network includes a shared communication line (e.g., a channel) between two (e.g., more) points, for example, where packets from different connections share the communication line (e.g., routed according to the data of each packet, the data of each packet being, for example, in the header of a packet including a header and a payload). Examples of packet-switched networks are discussed below, for example, referring to a mezzanine network.

[0208] Figure 12 The diagram illustrates a data flow diagram 1200 of pseudocode function call 1201 according to an embodiment of the present disclosure. Function call 1201 loads two input data operands (e.g., indicated by pointers *a and *b respectively), multiplies them together, and returns the result data. This function or other functions may be executed multiple times (e.g., in the data flow diagram). Figure 12The data flow diagram illustrates a PickAny data flow operator 1202 for performing operations to select control data (e.g., an index) (e.g., from call point 1202A) and copy the control data (e.g., the index) to each of the first Pick data flow operator 1206, the second Pick data flow operator 1208, and the Switch data flow operator 1216 using a copy data flow operator 1204. In one embodiment, the index (e.g., from PickAny) thus inputs and outputs data to the same index position, for example, [0, 1...M], where M is an integer. The first Pick data flow operator 1206 can then pull out one of a plurality of input data elements 1206A based on the control data, and use one input data element as (*a) to then load the input data value stored at *a using a load data flow operator 1210. The second Pick data stream operator 1208 can then pull out one of multiple input data elements 1208A based on control data, and use one input data element as (*b) to load the input data value stored at *b using the load data stream operator 1212. The two input data values ​​can then be multiplied by the multiplication data stream operator 1214 (e.g., as part of a processing element). The result of the multiplication can then be routed (e.g., to a downstream processing element or other component) by the Switch data stream operator 1216 to, for example, a call point 1216A, based on control data (e.g., an index) to the Switch data stream operator 1216.

[0209] Figure 12 This is an example of a function call where multiple dataflow operators for managing the routing of data (e.g., tokens) may be important for, for example, routing data to and / or from the call point. In one example, for instance, when multiple (e.g., many) call points exist, one or more of PickAny dataflow operator 1202, first Pick dataflow operator 1206, second Pick dataflow operator 1206, and Switch dataflow operator 1216 may be used to route (e.g., route) data. In embodiments where the (e.g., primary) goal of introducing multiplexing and / or demultiplexing of function calls is to reduce the implementation area of ​​a particular dataflow graph, certain embodiments herein (e.g., microarchitectures) reduce the area overhead of such multiplexing and / or demultiplexing (e.g., partial) of the dataflow graph.

[0210] Figure 13The illustration shows a spatial array 1301 of processing elements (PEs) having multiple network data flow endpoint circuits (1302, 1304, 1306) according to an embodiment of the present disclosure. The spatial array 1301 of processing elements may include a communication (e.g., interconnect) network between the components, such as those discussed herein. In one embodiment, the communication network is one or more packet-switched communication networks (e.g., their channels). In another embodiment, the communication network is one or more circuit-switched, statically configured communication channels. For example, the set of channels is coupled together by switching means (e.g., switching means 1310 in a first network and switching means 1311 in a second network). The first and second networks may be separate or may be coupled together. For example, switching means 1310 may couple together one or more of a plurality (e.g., four) of data paths, such as configured to perform operations according to a data flow graph. In one embodiment, the number of data paths is arbitrary. Processing elements (e.g., processing element 1308) may be as disclosed herein, such as those described herein. Figure 9 As shown in the image. Accelerator chip 1300 includes a memory / cache hierarchy interface 1312 to interface, for example, accelerator chip 1300 with memory and / or cache. Data paths may extend to another chip or terminate, for example, at the edge of a chip. Processing elements may include input buffers (e.g., buffer 1309) and output buffers.

[0211] Operations can be performed based on the availability of inputs for those operations and the state of the PE. The PE can obtain operands from input channels and write results to output channels, but can also use internal register states. Some embodiments described herein include configurable dataflow-friendly PEs. Figure 9 A detailed block diagram of such a PE is shown: an integer PE. This PE consists of several I / O buffers, an ALU, memory registers, some instruction registers, and a scheduler. In each cycle, the scheduler selects an instruction to execute based on the availability of the input and output buffers and the state of the PE. The result of the operation can then be written to an output buffer or to a register (e.g., locally on the PE). Data written to the output buffer can be transferred to downstream PEs for further processing. This PE style can be extremely energy efficient; for example, instead of reading data from a complex multi-port register file, the PE reads data from a register. Similarly, instructions can be stored directly in registers instead of in a virtualized instruction cache.

[0212] The instruction register can be set during a specific configuration step. During this step, in addition to the inter-PE network, auxiliary control lines and states can also be used to allow configuration to flow across several PEs in the containing structure. As a result of parallelism, some embodiments of such networks can provide rapid reconfiguration; for example, a chip-sized structure can be configured in less than about 10 microseconds.

[0213] Some embodiments herein overlay (e.g., high-base) dataflow operations onto a communication network, for example, in addition to the communication network routing data between processing elements, memory, etc., and / or the communication network performing other communication (e.g., non-data processing) operations. Some embodiments herein relate to a communication network (e.g., a packet-switched network) coupled to (e.g., a spatial array (e.g., CSA) of processing elements) to perform certain dataflow operations, for example, in addition to the communication network routing data between processing elements, memory, etc., or the communication network performing other communication operations. Some embodiments herein relate to (e.g., each) network dataflow endpoint circuitry that performs (e.g., a portion or all) one or more dataflow operations (e.g., a selection or switching dataflow operation of a dataflow graph). Some embodiments herein include enhanced network endpoints (e.g., network dataflow endpoint circuitry) to support control over (e.g., multiple or subsets) dataflow operations, for example, utilizing network endpoints to perform (e.g., dataflow) operations instead of processing elements (e.g., cores) or arithmetic logic units (e.g., for performing arithmetic and logic operations) performing the (e.g., dataflow) operation. In one embodiment, the network data stream endpoint circuitry is separate from the spatial array (e.g., its interconnection or structure) and / or processing elements.

[0214] Furthermore, the depicted accelerator chip 1300 includes a packet-switched communication network 1314, for example, as part of a mezzanine network as described below. Certain embodiments herein allow (e.g., distributed) data stream operations (e.g., operations that only route data) to be performed on the communication network (e.g., within) (e.g., and not in the processing element). As an example, in Figure 13The diagram depicts a distributed pick data flow operation. Specifically, distributed pick is implemented using three separate configurations on three separate network (e.g., global) endpoints (e.g., network data flow endpoint circuits (1302, 1304, 1306)). Data flow operations can be distributed, for example, using several endpoints configured in a cooperative manner. For example, the compiler tools can understand the need for cooperation. Endpoints (e.g., network data flow endpoint circuits) can be shared among several distributed operations; for example, a data flow operation (e.g., pick) endpoint can be compared with several transmissions involving the data flow operation (e.g., pick). Distributed data flow operations (e.g., pick) can produce the same results as non-distributed data flow operations (e.g., pick). In some embodiments, the distinction between distributed and non-distributed data flow operations is that distributed data flow operations cause their data (e.g., data to be routed, but which may not include control data) to run on, for example, a packet-switched communication network with associated flow control and distributed cooperation. Although processing elements (PEs) of different sizes are shown, in one embodiment, each processing element has the same size (e.g., silicon area). In one embodiment, a buffer element for buffering data may also be included, for example, separate from the processing element.

[0215] As an example, a pick-ahead data stream operation can have multiple inputs and guide (e.g., route) one of them as an output, such as... Figure 12 As described above. Instead of utilizing processing elements to perform the data stream selection operation, it can be implemented using one or more network communication resources (e.g., network data stream endpoint circuits). Additionally or alternatively, the network data stream endpoint circuits can route data between processing elements, for example, to allow processing elements to perform processing operations on the data. Therefore, the embodiments described herein can utilize a communication network to perform (e.g., guide) data stream operations. Additionally or alternatively, the network data stream endpoint circuits can be implemented as a mezzanine network discussed below.

[0216] In the depicted embodiments, the packet-switched communication network 1314 may handle certain (e.g., configuration) communications, such as programming processing elements and / or circuit-switched networks (e.g., network 1313, which may include switching devices). In one embodiment, the circuit-switched network is configured (e.g., programmed) to perform one or more operations (e.g., data flow operations of a data flow graph).

[0217] The packet-switched communication network 1314 includes multiple endpoints (e.g., network data stream endpoint circuits (1302, 1304, 1306)). In one embodiment, each endpoint includes an address or other indicator value to allow data to be routed to and / or from the endpoint, for example, based on data packets (e.g., headers).

[0218] For one or more additional or alternative locations performing the above operations, the packet-switched communication network 1314 can perform data flow operations. Network data flow endpoint circuits (1302, 1304, 1306) can be configured (e.g., programmed) to perform data flow graph (e.g., distributed picking) operations. Programming of components (e.g., circuits) is described herein. References Figure 14 Examples of configuring network data stream endpoint circuits (e.g., their operation configuration registers) are discussed.

[0219] As an example of distributed data stream selection operations Figure 13 The network data flow endpoint circuits (1302, 1304, 1306) can be configured (e.g., programmed) to perform distributed pick operations on the data flow graph. (See reference) Figure 14 Examples of configuring network data stream endpoint circuits (e.g., their operation configuration registers) are discussed. Local endpoint circuits can also be configured according to this disclosure, either additionally or alternatively, for configuring remote endpoint circuits.

[0220] Network data stream endpoint circuit 1302 can be configured to receive input data and output result data (e.g., such as network data stream endpoint circuit 1304 and network data stream endpoint circuit 1306) from multiple sources, for example, based on control data. Figure 12 (As in...). For example, upon receiving input data from processing element 1322, network data stream endpoint circuit 1304 can be configured to provide (e.g., send) the input data to network data stream endpoint circuit 1302. This is in... Figure 13 This can be referred to as input 0. In one embodiment, the circuit-switched network is configured (e.g., programmed) to provide a dedicated communication line along path 1324 between processing element 1322 and network data stream endpoint circuit 1304. For example, upon receiving input data from processing element 1320, network data stream endpoint circuit 1306 can be configured to provide (e.g., send) the input data to network data stream endpoint circuit 1302. This is in Figure 13 This can be referred to as input 1. In one embodiment, the circuit-switched network is configured (e.g., programmed) to provide a dedicated communication line along path 1316 between the processing element 1320 and the network data flow endpoint circuit 1306.

[0221] When network data stream endpoint circuit 1304 is used to transmit input data to network data stream endpoint circuit 1302 (e.g., when network data stream endpoint circuit 1302 has available storage space for data and / or network data stream endpoint circuit 1304 has its input data), network data stream endpoint circuit 1304 may generate packets (e.g., including input data and a header) to direct the data to network data stream endpoint circuit 1302 on packet-switched communication network 1314 (e.g., as a station on that (e.g., ring) network 1314). This in Figure 13 It is schematically shown in the middle with a dashed line 1326. Although Figure 13 The example shown uses two sources (e.g., two inputs), but it is possible to use one or any number of sources (e.g., more than two) (e.g., inputs).

[0222] When network data stream endpoint circuit 1306 is used to transmit input data to network data stream endpoint circuit 1302 (e.g., when network data stream endpoint circuit 1302 has available storage space for data and / or network data stream endpoint circuit 1306 has its input data), network data stream endpoint circuit 1306 may generate packets (e.g., including input data and a header) to direct the data to network data stream endpoint circuit 1302 on packet-switched communication network 1314 (e.g., as a station on that (e.g., ring) network 1314). This in Figure 13 The diagram is schematically shown in dashed line 1318. Although a mesh network is shown, other network topologies can be used.

[0223] (For example, upon receiving input 0 from network data stream endpoint circuit 1304, input 1 from network data stream endpoint circuit 1306, and / or control data) network data stream endpoint circuit 1302 can then execute a programmed data stream operation (e.g., a Pick operation in this example). Network data stream endpoint circuit 1302 can then output the corresponding result data from the operation to, for example... Figure 13 The processing element 1308 is located within the network. In one embodiment, the circuit-switched network is configured (e.g., programmed) to provide a dedicated communication line along path 1328 between the processing element 1308 (e.g., its buffer) and the network data stream endpoint circuitry 1302. Reference is made below. Figure 26-28 Further examples of distributed Pick operations are discussed.

[0224] In one embodiment, control data for performing an operation (e.g., a picking operation) comes from other components of the spatial array (e.g., processing elements) or enters via a network. See below for reference. Figure 14An example of this situation is discussed. Note that the Pick operator is schematically shown in terminal 1302 and may not be a multiplexer circuit, for example, see below for an example of... Figure 14 Discussion of the network data flow endpoint circuit 1400 in the middle.

[0225] In some embodiments, a data flow graph can allow certain operations to be performed by processing elements while allowing certain operations to be performed by a communication network (e.g., one or more network data flow endpoint circuits).

[0226] Figure 14 The figure illustrates a network data stream endpoint circuit 1400 according to an embodiment of the present disclosure. Although multiple components are shown in the network data stream endpoint circuit 1400, one or more instances of each component may be utilized in a single network data stream endpoint circuit. Embodiments of the network data stream endpoint circuit may include... Figure 14 Any one of the components (for example, not all of them).

[0227] Figure 14 The microarchitecture of a network interface (e.g., mezzanine) is depicted, illustrating an embodiment of the main data (solid lines) and control data (dashed lines) paths. This microarchitecture provides a configuration store and scheduler to enable (e.g., high-radix) dataflow manipulators. Some embodiments herein include data paths to the scheduler to enable foot selection and description. Figure 14 This illustrates a high-level microarchitecture of a network endpoint (e.g., a mezzanine network), which may be a component of a ring network for contextual purposes. To support (e.g., high-baseline) dataflow operations, the configuration of the endpoint (e.g., operation configuration storage 1426) includes checking the configuration of multiple network (e.g., virtual) channels (e.g., individual virtual channels relative to a baseline implementation). Some embodiments of the network dataflow endpoint circuitry 1400 include data paths from ingress to egress to control the selection of operations (e.g., pick-and-switch types) and / or describe the selection made by the scheduler in the case of a PickAny or SwitchAny dataflow operator. Flow control and backpressure behavior can be utilized in each communication channel, for example, in (e.g., packet-switched communication) networks and (e.g., circuit-switched) networks (e.g., structures of spatial arrays of processing elements).

[0228] As a description of an embodiment of the microarchitecture, a selector dataflow operator can function to select, for example, an output of resulting data from multiple inputs of input data based on control data. Network dataflow endpoint circuitry 1400 can be configured to treat one of the spatial array ingress buffers 1402 of circuitry 1400 (e.g., data from the architecture is control data) as a selection among multiple input data elements stored in network ingress buffer 1424 of circuitry 1400 to direct the resulting data to spatial array egress buffer 1408 of circuitry 1400. Therefore, network ingress buffer 1424 can be considered as an input to a virtual multiplexer, spatial array ingress buffer 1402 as a multiplexer selection, and spatial array egress buffer 1408 as a multiplexer output. In one embodiment, when a value (e.g., control data) is detected and / or arrives in spatial array ingress buffer 1402, scheduler 1428 (e.g., programmed by stored operation configuration 1426) is activated to check the corresponding network ingress channel. When data becomes available in the channel, it is removed from the network ingress buffer 1424 and moved to the space array egress buffer 1408. The ingress and egress control bits can then be updated to reflect the data transfer. This can cause control flow tokens or credits to propagate throughout the associated network. In some embodiments, all inputs (e.g., control or data) may occur locally or over the network.

[0229] Initially, using packet-switched networks to implement operators for multiplexing and / or demultiplexing code (e.g., high base segments) may appear to hinder performance. For example, in one embodiment, the packet-switched network is typically shared and the caller and callee dataflow graphs may be geographically distant. However, recalling that in some embodiments, the intention to support multiplexing and / or demultiplexing is to reduce the area consumed by infrequent code paths (e.g., spatial arrays) within the dataflow operator. Therefore, some embodiments herein reduce area and avoid the consumption of more expensive structural resources, such as PEs, without (materially) impacting the area and efficiency of individual PEs to support those (e.g., infrequent) operations.

[0230] Turn now Figure 14In further detail, the depicted network data flow endpoint circuit 1400 includes a spatial array (e.g., structural) ingress buffer 1402, for example, for inputting data (e.g., control data) from (e.g., circuit-switched) networks. As described above, although a single spatial array (e.g., structural) ingress buffer 1402 is depicted, multiple spatial array (e.g., structural) ingress buffers may be present in the network data flow endpoint circuit. In one embodiment, the spatial array (e.g., structural) ingress buffer 1402 is used to receive data (e.g., control data) from a communication network of spatial arrays (e.g., spatial arrays of processing elements), for example, from one or more of networks 1404 and 1406. In one embodiment, network 1404 is... Figure 13 It is part of network 1313.

[0231] The depicted network data flow endpoint circuit 1400 includes a spatial array (e.g., structural) exit buffer 1408, for example, for outputting data (e.g., control data) to (e.g., a circuit-switched) network. As described above, although a single spatial array (e.g., structural) exit buffer 1408 is depicted, multiple spatial array (e.g., structural) exit buffers may be present in the network data flow endpoint circuit. In one embodiment, the spatial array (e.g., structural) exit buffer 1408 is used to send (e.g., transmit) data (e.g., control data) to a communication network of a spatial array (e.g., a spatial array of processing elements), for example, to one or more of networks 1410 and 1412. In one embodiment, network 1410 is... Figure 13 It is part of network 1313.

[0232] Additionally or alternatively, the network data flow endpoint circuitry 1400 may be coupled to another network 1414, such as a packet-switched network. The other network 1414 (e.g., a packet-switched network) may be used to transmit (e.g., send or receive) data (e.g., input and / or result) to processing elements or other components of the space array and / or transmit one or more of the input or result data. In one embodiment, network 1414 is... Figure 13 It is part of a packet-switched communication network 1314 (e.g., a time-multiplexed network).

[0233] Network buffer 1418 (e.g., register) can be a station on network 1414 (e.g., ring) for receiving data from network 1414.

[0234] The depicted network data stream endpoint circuit 1400 includes a network egress buffer 1422, for example, for outputting data (e.g., result data) to (e.g., a packet-switched) network. As described above, although a single network egress buffer 1422 is depicted, multiple network egress buffers may be present in the network data stream endpoint circuit. In one embodiment, the network egress buffer 1422 is used to send (e.g., transmit) data (e.g., result data) to a communication network of a spatial array (e.g., a spatial array of processing elements), for example, to network 1414. In one embodiment, network 1414 is... Figure 13 It is part of the packet-switched network 1314. In some embodiments, the network egress buffer 1422 is used to output data (e.g., from the spatial array ingress buffer 1402) to (e.g., the packet-switched) network 1414, for example, to be routed (e.g., directed) to other components (e.g., other network data flow endpoint circuits).

[0235] The depicted network data stream endpoint circuit 1400 includes a network ingress buffer 1424, for example, for inputting data (e.g., input data) from (e.g., packet-switched) networks. As described above, although a single network ingress buffer 1424 is depicted, multiple network ingress buffers may be present in the network data stream endpoint circuit. In one embodiment, the network ingress buffer 1424 is used to receive (e.g., transmit) data (e.g., input data) from a communication network of a spatial array (e.g., a spatial array of processing elements), for example, from network 1414. In one embodiment, network 1414 is... Figure 13 It is part of the packet-switched network 1314. In some embodiments, the network ingress buffer 1424 is used to input data from (e.g., packet-switched) network 1414 (e.g., from spatial array ingress buffer 1402), for example, to be routed (e.g., directed) from other components (e.g., other network data flow endpoint circuits) to this location (e.g., to spatial array egress buffer 1408).

[0236] In one embodiment, the data format (e.g., data on network 1414) includes packets with data and a header (e.g., the destination of the data). In another embodiment, the data format (e.g., data on networks 1404 and / or 1406) includes only data (e.g., packets not with data and a header (e.g., the destination of the data)). Network data flow endpoint circuit 1400 may add headers (or other data) to packets (e.g., data output from circuit 1400) or remove headers (or other data) from packets (e.g., data input to circuit 1400). Coupling 1420 (e.g., a line) may send data received from network 1414 (e.g., from network buffer 1418) to network ingress buffer 1424 and / or multiplexer 1416. Multiplexer 1416 may (e.g., via control signals from scheduler 1428) output data from network buffer 1418 or from network egress buffer 1422. In one embodiment, one or more of the multiplexer 1416 or network buffer 1418 are components separate from the network data stream endpoint circuitry 1400. The buffer may include multiple (e.g., discrete) entries, such as multiple registers.

[0237] In one embodiment, an operation configuration store 1426 (e.g., one or more registers) is loaded during configuration (e.g., mapping) and specifies the particular operation (or operations) (e.g., data bootstrapping operations as opposed to logical and / or arithmetic operations) that the network data stream endpoint circuit 1400 (e.g., a processing element not part of a spatial array) will perform. Buffer (e.g., 1402, 1408, 1422, and / or 1424) activity can be controlled by this operation (e.g., by scheduler 1428). Scheduler 1428 can schedule one or more operations of the network data stream endpoint circuit 1400, for example, when (e.g., all) input (e.g., payload) data and / or control data arrive. Dashed lines indicating paths to and from scheduler 1428 can be used, for example, to and / or from control data. The scheduler can also control the multiplexer 1416, for example, to direct data to network data stream endpoint circuits 1400 and network 1414 and / or direct data from network data stream endpoint circuits 1400 and network 1414.

[0238] Refer to the above text Figure 13 In the distributed selection operation, the network data stream endpoint circuit 1302 can be configured (e.g., as...). Figure 14 (as in the operation configuration register 1426) to receive from each of network data stream endpoint circuits 1304 and 1306 (e.g., in such...) Figure 14The network ingress buffer 1424 (one of the two storage locations) receives the input data, and (for example, from such...) Figure 14 The output result data of its spatial array output buffer 1408 is, for example, based on (e.g., such as...) Figure 14 The network data stream endpoint circuit 1304 can be configured (e.g., as shown in its spatial array inlet buffer 1402) to control data. Figure 14 As in the operation of its configuration register 1426, input data is provided (e.g., via circuit 1304 as in the operation of the configuration register 1426). Figure 14 The network egress buffer 1422 sends data to the network data stream endpoint circuit 1302, for example, upon receiving data (e.g., in circuit 1304, such as...). Figure 14 When input data comes from processing element 1322 in the space array inlet buffer 1402. This is in Figure 13 This can be referred to as input 0. In one embodiment, the circuit-switched network is configured (e.g., programmed) to provide a dedicated communication line along path 1324 between processing element 1322 and network data stream endpoint circuitry 1304. Network data stream endpoint circuitry 1304 may include (e.g., add) header packets with the received data (e.g., in a format such as...). Figure 14 The network egress buffer 1422 is used to direct packets (e.g., input data) to the network data stream endpoint circuit 1302. The network data stream endpoint circuit 1306 can be configured (e.g., as...) Figure 14 As in the operation of its configuration register 1426, input data is provided (e.g., via circuit 1306 as in the operation of the configuration register 1426). Figure 14 The network egress buffer 1422 sends data to the network data stream endpoint circuit 1302, for example, upon receiving data (e.g., in circuit 1306, such as...). Figure 14 When input data comes from processing element 1320 (in the space array inlet buffer 1402). This is in Figure 13 This can be referred to as input 1. In one embodiment, the circuit-switched network is configured (e.g., programmed) to provide a dedicated communication line along path 1316 between processing element 1320 and network data stream endpoint circuitry 1306. Network data stream endpoint circuitry 1306 may include (e.g., add) header packets with the received data (e.g., in a format such as...). Figure 14 The network egress buffer 1422 is used to direct packets (e.g., input data) to the network data stream endpoint circuit 1302.

[0239] When network data stream endpoint circuit 1304 is used to transmit input data to network data stream endpoint circuit 1302 (e.g., when network data stream endpoint circuit 1302 has available storage space for data and / or network data stream endpoint circuit 1304 has its input data), network data stream endpoint circuit 1304 may generate packets (e.g., including input data and a header) to direct the data to network data stream endpoint circuit 1302 on packet-switched communication network 1314 (e.g., as a station on that (e.g., ring) network). This in Figure 13 The figure is schematically shown as 1326 (dashed line). Figure 13 Network 1314 is schematically shown in multiple dashed boxes. Network 1314 may include network controller 1314A, for example, for managing the inflow and / or outflow of data on network 1314A.

[0240] When network data stream endpoint circuit 1306 is used to transmit input data to network data stream endpoint circuit 1302 (e.g., when network data stream endpoint circuit 1302 has available storage space for data and / or network data stream endpoint circuit 1306 has its input data), network data stream endpoint circuit 1306 may generate packets (e.g., including input data and a header) to direct the data to network data stream endpoint circuit 1302 on packet-switched communication network 1314 (e.g., as a station on that (e.g., ring) network). This in Figure 13 The figure is schematically shown in the middle with a dashed line 1318.

[0241] Network data stream endpoint circuit 1302 (e.g., when it receives input 0 from network data stream endpoint circuit 1304 in the network inlet buffer of circuit 1302, input 1 from network data stream endpoint circuit 1306 in the network inlet buffer of circuit 1302, and / or control data from processing element 1308 in the space array inlet buffer of circuit 1302) can then execute a programmed data stream operation (e.g., a Pick operation in this example). Network data stream endpoint circuit 1302 can then output the corresponding result data from the operation to, for example... Figure 13 The processing element 1308 is located within the network. In one embodiment, the circuit-switched network is configured (e.g., programmed) to provide a dedicated communication line along path 1328 between the processing element 1308 (e.g., its buffer) and the network data stream endpoint circuitry 1302. Reference is made below. Figure 26-28 Further examples of distributed Pick operations are discussed. Figure 13 The buffer in the buffer can be a small, unlabeled box in each PE.

[0242] The following Figure 15-18This includes an example data format, but other data formats may be used. One or more fields may be included in the data format (e.g., in a group). The data format may be used by network data stream endpoint circuits, for example, to transmit (e.g., send and / or receive) data between first components (e.g., between a first network data stream endpoint circuit and a second network data stream endpoint circuit, components of a spatial array, etc.).

[0243] Figure 15 The illustration depicts data formats for transmit operation 1502 and receive operation 1504 according to embodiments of the present disclosure. In one embodiment, transmit operation 1502 and receive operation 1504 are data formats for data transmitted over a packet-switched communication network. The depicted transmit operation 1502 data format includes a destination field 1502A (e.g., indicating which component in the network the data will be sent to), a channel field 1502B (e.g., indicating which channel on the network the data will be sent on), and an input field 1502C (e.g., the payload or input data to be sent). The depicted receive operation 1504 includes an output field, and may also include a destination field (not depicted). These data formats can (e.g., for packets) be used to dispose of components that move data in and out. These configurations can occur separately and / or in parallel. These configurations can use separate resources. The term channel can generally refer to the communication resource associated with the request (e.g., in management hardware). The association of the configuration with queue management hardware can be explicit.

[0244] Figure 16 The illustration shows another data format for a transmission operation 1602 according to an embodiment of the present disclosure. In one embodiment, the transmission operation 1602 is a data format for data transmitted over a packet-switched communication network. The depicted transmission operation 1602 data format includes a type field (e.g., for annotating special control packets, such as, but not limited to, configuration, extraction, or exception packets), a destination field 1602B (e.g., indicating which component in the network the data will be sent to), a channel field 1602C (e.g., indicating on which channel the data will be transmitted on the network), and an input field 1602D (e.g., the payload or input data to be transmitted).

[0245] Figure 17The illustration depicts a configuration data format for configuring circuit elements (e.g., network data stream endpoint circuitry) for a transmit (e.g., switching) operation 1702 and a receive (e.g., picking) operation 1704 according to embodiments of the present disclosure. In one embodiment, transmit operation 1702 and receive operation 1704 are configuration data formats for data transmitted over a packet-switched communication network (e.g., between network data stream endpoint circuits). The depicted transmit operation configuration data format 1702 includes a destination field 1702A (e.g., indicating which component(s) in the network will send (input) data to), a channel field 1702B (e.g., indicating on which channel(s) on the network the (input) data will be sent), an input field 1702C (e.g., an identifier for the component sending the input data, e.g., a set of inputs in a buffer (e.g., a structure entry) to which the component is sensitive), and an operation field 1702D (e.g., indicating which of a plurality of operations will be performed). In one embodiment, (e.g., outbound) operation is one of the Switch or SwitchAny data flow operations, for example, a (e.g., the same) data flow operator corresponding to the data flow graph.

[0246] The described receive operation configuration data format 1704 includes an output field 1704A (e.g., indicating which component(s) in the network will send the (result) data to), an input field 1704B (e.g., an identifier for the component sending the input data), and an operation field 1704C (e.g., indicating which of a plurality of operations will be performed). In one embodiment, an operation (e.g., inbound) is one of a Pick, PickSingleLeg, PickAny, or Merge data stream operation, for example, a data stream operator corresponding to (e.g., the same) a data stream graph. In one embodiment, a merge data stream operation is a selection that requires all operands and dequeues all operands (e.g., using egress endpoint receive control).

[0247] The configuration data format used in this document may include one or more of the fields described herein, for example, in any order.

[0248] Figure 18The illustration depicts a configuration data format 1802 for configuring circuit elements (e.g., network data stream endpoint circuitry) for a transmission operation using input, output, and control data annotated on circuitry 1800, according to an embodiment of this disclosure. The depicted transmission operation configuration data format 1802 includes a destination field 1802A (e.g., indicating which component in the network to which data will be transmitted), a channel field 1802B (e.g., indicating which channel on a (packet-switched) network the data will be transmitted on), and an input field 1802C (e.g., an identifier for the component transmitting input data). In one embodiment, circuitry 1800 (e.g., network data stream endpoint circuitry) is configured to receive packets of data in the data format of transmission operation configuration data format 1802, for example, using the destination field to indicate which of a plurality of circuits the result will be transmitted to, the channel field to indicate which channel on a (packet-switched) network the data will be transmitted on, and the input field to indicate from which of a plurality of circuits the input data will be received. AND gate 1804 is used to allow an operation to be performed when input data is available and a credit status (e.g., a dependency token indication) indicates the existence of space for storing output data (e.g., in a buffer at the destination). In some embodiments, each operation annotation has its requirements (e.g., input, output, and control), and if all requirements are met, the configuration is 'executable' by circuitry (e.g., network data stream endpoint circuitry).

[0249] Figure 19 The illustration depicts a configuration data format 1902, according to an embodiment of the present disclosure, for configuring circuit elements (e.g., network data stream endpoint circuitry) for a selected operation (e.g., transmit) using input, output, and control data annotated on circuitry 1900. The depicted (e.g., transmit) operation configuration data format 1902 includes a destination field 1902A (e.g., indicating which component(s) in the network will send (input) data to), a channel field 1902B (e.g., indicating on which channel in the network the (input) data will be sent), an input field 1902C (e.g., an identifier for the component sending the input data), and an operation field 1902D (e.g., indicating which of a plurality of operations will be performed and / or the source of control data for that operation). In one embodiment, the (e.g., outward) operation is one of a transmit, switch, or switchAny data stream operation, e.g., a (e.g., the same) data stream operator corresponding to a data flow graph.

[0250] In one embodiment, circuitry 1900 (e.g., network data stream endpoint circuitry) is configured to receive packets of data in the data format of operation configuration data format 1902, for example, by utilizing the source of the input as a payload (e.g., input data) and an operation field indicating which operation will be performed (e.g., schematically shown as Switch or SwitchAny). The depicted multiplexer 1904 can select the operation to be performed from a plurality of available operations, for example, based on the value in the operation field 1902D. In one embodiment, circuitry 1900 is configured to perform the operation when input data is available and a credit status of yes (e.g., a dependency token indication) indicates the existence of space for storing output data (e.g., in a buffer at the destination).

[0251] In one embodiment, the send operation does not utilize control other than checking whether its input is available for sending. This allows the switch to perform the operation without credit on all feet. In one embodiment, the Switch and / or SwitchAny operation includes a multiplexer controlled by a value stored in the operation field 1902D to select the correct queue management circuitry.

[0252] The value stored in operation field 1902D can be selected in control options, for example, by utilizing different control (e.g., logic) circuits for each operation, such as in... Figure 20-23 In some embodiments, credit (e.g., credit on the internet) status is another input (e.g., as described herein). Figure 20-21 (As depicted in the text).

[0253] Figure 20The illustration illustrates a configuration data format for configuring circuit elements (e.g., network data stream endpoint circuitry) for a Switch operation configuration data format 2002, according to an embodiment of the present disclosure, using input, output, and control data annotated on circuitry 2000. In one embodiment, an operation value (e.g., outward) stored in operation field 1902D is used for a Switch operation, such as a Switch data stream operator corresponding to a data flow graph. In one embodiment, circuitry 2000 (e.g., network data stream endpoint circuitry) is used to receive packets of data in the data format of Switch operation 2002, for example, by using input in input field 2002A to indicate which component(s) will be used to send data and operation field 2002B to indicate which operation will be performed (e.g., schematically shown as Switch). The depicted circuitry 2000 can select the operation to be performed from a plurality of available operations based on operation field 2002B. In one embodiment, circuit 1900 performs the operation when input data (e.g., depending on the input state, e.g., space for data exists at the destination) is available and the credit state (e.g., selection operation (OP) state) is true (e.g., network credit indicates availability on the network to send the data to the destination). For example, multiplexers 2010, 2012, and 2014 can be used with corresponding input states and credit states for each input (e.g., where output data will be sent in a switching operation), for example, to prevent the input from appearing available until both the input state (e.g., space for data at the destination) and the credit state (e.g., space on the network to reach the destination) are true (e.g., yes). In one embodiment, the input state is an indication of the existence of space in a buffer at the destination for (output) data to be stored. In some embodiments, when input data is available (e.g., as output from multiplexer 2004) and the selection operation (e.g., control data) state is true (e.g., indicating a selection operation) (e.g., which of the multiple outputs the input will be sent to, see example...), circuit 1900 performs the operation. Figure 12 When an operation is performed, AND gate 2006 is used to allow the operation to be performed. In some embodiments, an operation (e.g., a selection operation) is performed using control data to cause input data from one of the inputs to be output to one or more (e.g., multiple) outputs (e.g., as indicated by the control data), for example, according to multiplexer selection bits from multiplexer 2008. In one embodiment, the selection operation selects which pin of the switching output will be used and / or selects the decoder to create the multiplexer selection bit.

[0254] Figure 21The illustration illustrates a configuration data format for configuring circuit elements (e.g., network data stream endpoint circuitry) for SwitchAny operation configuration data format 2102 using input, output, and control data annotated on circuitry 2100, according to an embodiment of this disclosure. In one embodiment, an operation value (e.g., outward) stored in operation field 1902D is used for a SwitchAny operation, such as a SwitchAny data stream operator corresponding to a data flow graph. In one embodiment, circuitry 2100 (e.g., network data stream endpoint circuitry) is used to receive packets of data in the data format of SwitchAny operation configuration data format 2102, for example, by using input in input field 2102A to indicate which component(s) will be used to send data and operation field 2102B to indicate which operation will be performed (e.g., schematically shown as SwitchAny) and / or the source of control data for that operation. In one embodiment, circuit 1900 performs the operation when any of the input data (e.g., depending on the input state, e.g., space for data exists at the destination) is available and the credit status is yes (e.g., network credit indicates availability on the network to send the data to the destination). For example, multiplexers 2110, 2112, 2114 can be used with a corresponding input state and credit status for each input (e.g., where the output data will be sent in a SwitchAny operation) to prevent the input from appearing available until both the input state (e.g., space for data at the destination) and the credit status (e.g., space on the network to reach the destination) are true (e.g., yes). In one embodiment, the input state is an indication of the existence of space in a buffer at the destination for (output) data to be stored. In some embodiments, OR gate 2104 allows the operation to be performed when any of the outputs is available. In some embodiments, the operation is performed to cause a first available input data from one of the inputs to be output on one or more (e.g., multiple) outputs, for example, according to the multiplexer selection bit from multiplexer 2106. In one embodiment, SwitchAny occurs (e.g., in contrast to a Switch utilizing the selection operation) whenever any output credit is available. The multiplexer selection bit can be used to direct input to an (e.g., network) egress buffer of network data stream endpoint circuitry.

[0255] Figure 22The illustration illustrates a configuration data format for configuring circuit elements (e.g., network data stream endpoint circuitry) for Pick operation configuration data format 2202 using input, output, and control data annotated on circuitry 2200, according to an embodiment of this disclosure. In one embodiment, an operation value (e.g., inward) stored in operation field 2202C is used for a Pick operation, e.g., a Pick data stream operator corresponding to a data flow graph. In one embodiment, circuitry 2200 (e.g., network data stream endpoint circuitry) is used to receive packets of data in the data format of Pick operation configuration data format 2202, e.g., using data in input field 2202B indicating which component(s) will be used to send input data, data in output field 2202A indicating which component(s) will be used to send input data, and operation field 2202C indicating which operation will be performed (e.g., schematically shown as Pick) and / or the source of control data for that operation. The depicted circuitry 2200 can select an operation to be performed from a plurality of available operations based on operation field 2202C. In one embodiment, circuit 2200 performs the operation when input data (e.g., based on the input (e.g., network ingress buffer) status, such as all input data has arrived) is available, a credit status (e.g., output status) is yes (e.g., spatial array exit buffer) indicating the existence of space for storing output data (e.g., in the destination buffer), and the selection operation (e.g., control data) status is yes. In some embodiments, circuit 2200 performs the operation when input data is available (e.g., as output from multiplexer 2204), output space is available, and the selection operation (e.g., control data) status is yes (e.g., indicating the selection operation) (e.g., which of the multiple outputs the input will be sent to, see example...). Figure 12 When the AND gate 2206 is used, it allows the operation to be performed. In some embodiments, an operation (e.g., a selection operation) is performed using control data to cause input data from one of a plurality of inputs (e.g., as indicated by the control data) to be output to one or more (e.g., multiple) outputs, for example, according to multiplexer selection bits from multiplexer 2208. In one embodiment, the selection operation selects which pin will be used and / or selects the decoder to create the multiplexer selection bits.

[0256] Figure 23The illustration illustrates a configuration data format for configuring circuit elements (e.g., network data stream endpoint circuitry) for PickAny operation 2302 using input, output, and control data annotated on circuitry 2300, according to an embodiment of this disclosure. In one embodiment, an operation value (e.g., inward) stored in operation field 2302C is used for the PickAny operation, e.g., a PickAny data stream operator corresponding to a data flow graph. In one embodiment, circuitry 2300 (e.g., network data stream endpoint circuitry) is used to receive packets of data in the data format of PickAny operation configuration data format 2302, e.g., using data in input field 2302B indicating which component(s) will be used to send input data, data in output field 2302A indicating which component(s) will be used to send input data, and operation field 2302C indicating which operation will be performed (e.g., schematically shown as PickAny). The depicted circuitry 2300 can select the operation to be performed from a plurality of available operations based on operation field 2302C. In one embodiment, circuit 2300 performs the operation when any of the input data (e.g., the first to arrive) is available (e.g., based on the input (e.g., network ingress buffer) state, such as any of the input data has arrived) and the credit state (e.g., output state) is yes (e.g., space array exit buffer indication) indicating the existence of space for storing output data (e.g., in the destination buffer). In some embodiments, AND gate 2306 allows the operation to be performed when any of the input data is available (e.g., as output from multiplexer 2304) and output space is available. In some embodiments, the operation is performed to cause input data from one of a plurality of inputs (e.g., the first to arrive) to be output on one or more (e.g., multiple) outputs, such as based on multiplexer selection bits from multiplexer 2308.

[0257] In one embodiment, PickAny performs and / or selects the decoder to create multiplexer selection bits for the presence of any data.

[0258] Figure 24 The diagram illustrates a network data stream endpoint circuit 2400 according to an embodiment of the present disclosure, selecting operations (2402, 2404, 2406) for execution. A pending operation store 2401 (e.g., in...) Figure 14The scheduler 1428 can store one or more data stream operations, for example, according to the format discussed herein. The scheduler (e.g., based on a fixed priority or, for example, the oldest operation with all its operands) can schedule operations for execution. For example, the scheduler can select operation 2402 and send corresponding control signals from multiplexer 2408 and / or multiplexer 2410 according to the values ​​stored in the operation field. As an example, several operations can be simultaneously executable in a single network data stream endpoint circuit. Assuming all data is here, the "executable" signal (e.g., such as...) Figure 18-23 The signal (shown) can be input as a signal to multiplexer 2412. Multiplexer 2412 can send a control signal as an output for a selected operation (e.g., one of operations 2402, 2404, and 2406), the control signal causing multiplexer 2408 to configure a connection in the network data stream endpoint circuit to perform the selected operation (e.g., to retrieve data from a buffer or send data to a buffer). Multiplexer 2412 can send a control signal as an output for a selected operation (e.g., one of operations 2402, 2404, and 2406), the control signal causing multiplexer 2410 to configure a connection in the network data stream endpoint circuit to remove data, such as consumed data, from the queue. See, as an example, the discussion herein regarding the removal of data (e.g., tokens). Figure 24 The “PE status” in this context can be control data from the PE, such as empty and full indicators for the queue (e.g., backpressure signals and / or network credits). In one embodiment, the PE status can include empty or full status for all buffers and / or data paths, as described herein. Figure 14 middle. Figure 24 The diagram illustrates a generalized scheduling for the embodiments described herein, for example, where a dedicated scheduling is used for reference. Figure 20-23 The embodiments discussed.

[0259] In one embodiment, (e.g., using scheduling) the selection of dequeueing is determined by the operation and its dynamic behavior, for example, to dequeue the operation after execution. In one embodiment, the circuitry is used to dequeue data (e.g., input, output, and / or control data) using operand selection bits.

[0260] Figure 25 The diagram illustrates a network data stream endpoint circuit 2500 according to an embodiment of the present disclosure. Figure 14 In contrast, the network data flow endpoint circuit 2500 has separated configuration and control into two separate schedulers. In one embodiment, the egress scheduler 2528A is used to control the data flow entering (e.g., from a circuit-switched communication network coupled to it) the data flow endpoint circuit 2500 (e.g., at the variable queue 2502, for example, as...). Figure 14 The spatial array ingress buffer 1402 and the output (e.g., from the packet-switched communication network coupled thereto) data stream endpoint circuit 2500 (e.g., at the network egress buffer 2522, for example, as shown in the image) are both located in the spatial array ingress buffer 1402 and the output (e.g., from the packet-switched communication network coupled thereto) data stream endpoint circuit 2500. Figure 14 The network egress buffer 1422 in the network is used for data scheduling operations. In one embodiment, the ingress scheduler 2528B is used to schedule the data flow into (e.g., from the packet-switched communication network coupled to it) the endpoint circuit 2500 (e.g., at the network ingress buffer 2524, for example, such as...) Figure 14 The network ingress buffer 2424 and the output (e.g., from the circuit-switched communication network coupled thereto) data flow endpoint circuit 2500 (e.g., at the output buffer 2508, for example, such as...) Figure 14 The data scheduling operation of the spatial array exit buffer 2408 in the circuit 2500. Scheduler 2528A and / or scheduler 2528B may include the (e.g., operational) state of circuit 2500 as input, such as the full level of inputs (e.g., buffers 2502A, 2502), the full level of outputs (e.g., buffer 2508), values ​​(e.g., values ​​in 2502A), etc. Scheduler 2528B may include credit return circuitry, for example, to indicate that credit has been returned to the sender, for example, after credit has been received in network entry buffer 2524 of circuit 2500.

[0261] Network 2514 may be a circuit-switched network, such as those discussed herein. Additionally or alternatively, packet-switched networks (such as those discussed herein) may also be utilized, for example, coupled to network egress buffer 2522, network ingress buffer 2524, or other components described herein. Variable queue 2502 may include control buffer 2502A, for example, to indicate when a corresponding input queue (e.g., a buffer) includes a (new) data item, for example, as a single bit. Now turning to... Figure 26-28 In one embodiment, these figures cumulatively illustrate the configuration used to create a distributed pick.

[0262] Figure 26 The diagram illustrates a network data stream endpoint circuit 2600 that receives input zero (0) during a selection operation according to an embodiment of the present disclosure, for example, as referenced above. Figure 13 As discussed. In one embodiment, the export configuration 2626A transmits data to different network data stream endpoint circuits (e.g., Figure 28A portion of the selection operation of circuit 2800 in the process is loaded (e.g., during the configuration step). In one embodiment, the exit scheduler 2628A is used to monitor the variable queue 2602 (e.g., the data queue) of input data (e.g., from the processing element). According to an embodiment of the depicted data format, "send" (e.g., its binary value) indicates the data to be sent according to fields X, Y, where X is a value indicating a specific target network data stream endpoint circuit (e.g., 0 is...). Figure 28 In the network data stream endpoint circuit 2800, Y is a value indicating the location of which network ingress buffer (e.g., buffer 2824) the value will be stored. In one embodiment, Y is a value indicating a specific channel in a multi-channel (e.g., packet-switched) network (e.g., 0 is...). Figure 28 The network data stream endpoint circuit 2800 includes channel 0 and / or buffer element 0. When input data arrives, it is then sent by the network data stream endpoint circuit 2600 (e.g., from the network egress buffer 2622) to different network data stream endpoint circuits (e.g., ...). Figure 28 (Network data stream endpoint circuit 2800 in the middle).

[0263] Figure 27 The diagram illustrates a network data stream endpoint circuit 2700 that receives input one (1) during a selection operation according to an embodiment of the present disclosure, for example, as referenced above. Figure 13 As discussed. In one embodiment, the export configuration 2726A transmits data to different network data stream endpoint circuits (e.g., Figure 28 A portion of the selection operation of circuit 2800 in the process is loaded (e.g., during the configuration step). In one embodiment, exit scheduler 2728A is used to monitor the variable queue 2720 (e.g., data queue 2702B) of input data (e.g., from processing elements). According to an embodiment of the depicted data format, "send" (e.g., its binary value) indicates the data to be sent according to fields X, Y, where X is a value indicating a specific target network data stream endpoint circuit (e.g., 0 is...). Figure 28 In the network data stream endpoint circuit 2800, Y is a value indicating the location of which network ingress buffer (e.g., buffer 2824) the value will be stored. In one embodiment, Y is a value indicating a specific channel in a multi-channel (e.g., packet-switched) network (e.g., 1 is...). Figure 28 The network data stream endpoint circuit 2800 includes channel 1 and / or buffer element 1. When input data arrives, it is then sent by the network data stream endpoint circuit 2700 (e.g., from the network egress buffer 2722) to different network data stream endpoint circuits (e.g., ...). Figure 28 (Network data stream endpoint circuit 2800 in the middle).

[0264] Figure 28 The diagram illustrates a network data stream endpoint circuit 2800 that outputs the selected input during a selection operation, according to an embodiment of the present disclosure, for example, as referenced above. Figure 13 As discussed. In one embodiment, other network data stream endpoint circuits (e.g., circuits 2600 and 2700) are used to send their input data to the network ingress buffer 2824 of circuit 2800. In one embodiment, ingress configuration 2826B is loaded as part of a selection operation for selecting data sent to network data stream endpoint circuit 2800 (e.g., according to a control value) (e.g., during a configuration step). In one embodiment, the control value will be received in ingress control 2832 (e.g., a buffer). In one embodiment, ingress scheduler 2828A is used to monitor the reception of control values ​​and input values ​​(e.g., in network ingress buffer 2824). For example, if the control value indicates selection from buffer element A (e.g., 0 or 1 in this example) from network ingress buffer 2824 (e.g., from channel A), the value stored in buffer element A is then output as a result of the operation of circuit 2800, e.g., to output buffer 2808, e.g., when the output buffer has storage space (e.g., as indicated by a backpressure signal). In one embodiment, the output data of circuit 2800 is sent out when the output buffer has a token (e.g., input data and control data) and the receiver asserts that it has a buffer (e.g., indicating that storage is available, but allocation of other resources is possible; this example is merely illustrative).

[0265] Figure 29 The diagram illustrates a flowchart 2900 according to an embodiment of the present disclosure. The depicted flowchart 2900 includes: providing a spatial array 2902 of processing elements; routing data within the spatial array between processing elements according to a data flow graph using a packet-switched communication network 2904; performing a first data flow operation of the data flow graph using the processing elements 2906; and performing a second data flow operation of the data flow graph using a plurality of network data flow endpoint circuits of the packet-switched communication network 2908.

[0266] RAF circuit with TLB

[0267] In some embodiments, the circuitry (e.g., a Request Address File (RAF) circuitry) (e.g., each of a plurality of RAF circuitry) includes a Translation Backing Buffer (TLB) (e.g., a TLB circuitry). The TLB can receive virtual address inputs and output physical addresses corresponding to the virtual-to-physical address mapping (e.g., an address mapping) (e.g., any mapping different from a dataflow graph to hardware). Virtual addresses can be addresses seen by a program running on the circuitry (e.g., on an accelerator and / or processor). Physical addresses can be addresses in memory hardware (e.g., different from virtual addresses). The TLB can include data structures (e.g., tables) to store (e.g., recently used) virtual-to-physical memory address translations, such that the translation does not need to be performed for every existing virtual address to obtain the physical memory address corresponding to that virtual address. If a virtual address entry is not in the TLB, the circuitry (e.g., a TLB manager circuitry) can perform a page traversal to determine the virtual-to-physical memory address translation. In one embodiment, the circuitry (e.g., the RAF circuitry) is used via a circuit-switched network (e.g., such as...) Figure 6-11 The circuit receives input of a virtual address from a requesting entity (e.g., PE or other hardware component) for translation in the TLB (e.g., the TLB in a RAF circuit). Additionally or alternatively, the circuit (e.g., the RAF circuit) may be transmitted via a packet-switched network (e.g., such as...). Figure 12-29 The system receives input of a virtual address from a requesting entity (e.g., a PE, network data stream endpoint circuitry, or other hardware component) for translation in a TLB (e.g., a TLB in a RAF circuit). In some embodiments, the data received for a memory (e.g., cache) access request is a memory command. The memory command may include the virtual address to be accessed, the operation to be performed (e.g., load or store), and / or payload data (e.g., for storage).

[0268] Request Address File (RAF) circuit (in) Figure 30The RAF (shown in its version) can be responsible for performing memory operations and acting as an intermediary between the CSA structure and the memory hierarchy. Thus, a primary microarchitectural task of the RAF can be to rationalize the out-of-order memory subsystem with the ordered semantics of the CSA structure. In this respect, the RAF circuitry can provide completion buffers (e.g., queue-like structures) that reorder memory responses and return them to the structure in the order requested. A second primary function of the RAF circuitry can be to provide support in the form of address translation and page walkers. Incoming virtual addresses can be translated into physical addresses using channel-associated translation lookaside buffers (TLBs). To provide sufficient memory bandwidth, each CSA chip or CSA accelerator can include multiple RAF circuits. Like the various PEs of the structure, the RAF circuitry can operate in a dataflow-style manner by checking the availability of input arguments and output buffers (if needed) before selecting the memory operation to be performed. However, unlike some PEs, the RAF circuitry can be multiplexed among several co-located memory operations. Multiplexed RAF circuitry can be used to minimize the area overhead of its individual sub-components, such as shared accelerator cache interface (ACI) ports (described in more detail in Section 2.4), shared virtual memory (SVM) support hardware, mezzanine network interfaces, and other hardware management facilities. However, some programmatic characteristics can also facilitate this choice. In one embodiment, an efficient data flow graph is used to poll the memory in the shared virtual memory system. Memory latency-constrained programs (e.g., graph traversal) may utilize numerous separate memory operations to saturate memory bandwidth due to memory-dependent control flow. While each RAF can be multiplexed, the CSA may include multiple (e.g., between 8 and 32) RAFs at a granular level to ensure sufficient cache bandwidth. RAFs can communicate with the rest of the architecture via both a local network and / or a mezzanine network. In the case of RAF multiplexing, each RAF may be provisioned to the local network along with several ports. These ports can act as low-latency, highly deterministic paths to memory for use by latency-sensitive or high-bandwidth memory operations. In addition, RAF can be provided with mezzanine network endpoints, such as those that provide memory access to runtime services and remote user-level memory access providers.

[0269] Figure 30The diagram illustrates a Request Address File (RAF) circuit according to an embodiment of the present disclosure. In one embodiment, memory load and store operations (e.g., in a data flow graph) are specified (e.g., stored) in register 3010 during configuration, for example, sent via one or more of networks 3002, 3004, 3006, or 3008. Arcs from those memory operations in the data flow graph can then be connected to input queues 3022, 3024, and 3026. Arcs from those memory operations are thus used to exit completion buffers 3028, 3030, or 3032. Memory dependency tokens (which may each be a single bit) arrive in queues 3018 and 3020. Memory dependency tokens will exit from queue 3016 into the network. In the depicted embodiment, memory dependency tokens can be generated and sent to queue 3218, for example, on path 3217. Dependency token counter 3014 can be a compact representation of the queues and can track the number of dependency tokens used for any given input queue. In some embodiments, if the dependency token counter 3014 saturates, no additional dependency tokens can be generated for new memory operations. Accordingly, the memory sorting circuit (e.g., Figure 30 The RAF in the memory will stop scheduling new memory operations until the dependency token counter 3014 becomes unsaturated.

[0270] As an example of loading, an address arrives in queue 3022, and scheduler 3012 matches that address with a load in 3010. The completion buffer slot for this load can be assigned in the order the addresses arrive. Assuming that this particular load in the diagram does not specify dependencies, the address and completion buffer slot can be dispatched to the memory system by the scheduler (e.g., via memory command 3042). When the result returns to mux 3040 (illustratively shown), the result can be stored in the completion buffer slot it was assigned (e.g., since the result carries the target slot throughout its journey through the memory system). The completion buffer can send the result back to the local network (e.g., local networks 3002, 3004, 3006, or 3008) in the order the addresses arrive. Storage can be similar, except that both the address and the data may have to arrive before any operation is dispatched to the memory system.

[0271] One or more of the networks can be communication networks (e.g., packet-switched and / or circuit-switched networks). Memory command 3042 can be a load or store command. Memory command 3042 can include the virtual address to be accessed. RAF circuit 3000 can include TLB 3001, for example, as discussed herein. In one embodiment, TLB 3001 includes one or more mapping entries 3003 (e.g., from 1 to X, where X is any integer) (e.g., storing virtual address (VA) to physical address (PA) mappings). Arrow 3007 from the TLB can indicate a connection to an accelerator cache network (e.g., to access cache blocks, CHA, etc.). Note that bidirectional arrows in the figures do not necessarily require bidirectional communication; for example, they can indicate unidirectional communication (e.g., to or from that component or device). Note that unidirectional arrows in the figures do not necessarily require unidirectional communication; for example, they can indicate bidirectional communication (e.g., to and from that component or device). In one embodiment, TLB 3001 sends a request with a virtual address for a miss (e.g., from arrow 3007), and padding with a physical address is returned and entered into TLB 3001 (e.g., in the corresponding mapping entry of entry 3003). In one embodiment, when a virtual address exists in TLB 3001 (e.g., ...), ... Figure 30 When a memory access request is missed (in ADDR), the request is sent back to the queue (e.g., one or more of the input queues 3022, 3024, and 3026).

[0272] Refer again Figure 8 Accelerator (e.g., CSA) 802 can perform (e.g., or request execution) access (e.g., load and / or store) data to one or more of multiple cache blocks (e.g., cache block 808). Memory interface circuitry (e.g., Request Address File (RAF) circuitry) may be included (e.g., as discussed herein) to provide access between memory (e.g., cache blocks) and accelerator 802. See again Figure 11 The request circuitry (e.g., a processing element) can perform (e.g., request execution) access (e.g., load and / or store) to data in one or more of a plurality of cache blocks (e.g., cache block 1102). Memory interface circuitry (e.g., request address file (RAF) circuitry) may be included (e.g., as discussed herein) to provide access between memory (e.g., one or more blocks of cache memory) and accelerators (e.g., one or more accelerator chips (1108, 1110, 1112, 1114)). See again Figure 13And / or 14, requesting circuitry (e.g., processing elements) can perform (e.g., request execution) access (e.g., load and / or store) to data in one or more of a plurality of cache blocks. Memory interface circuitry (e.g., Request Address File (RAF) circuitry, such as RAF / Cache Interface 1312) may be included (e.g., as discussed herein) to provide access between memory (e.g., one or more blocks of cache memory) and accelerators (e.g., processing elements and / or network data stream endpoint circuitry (e.g., circuits 1302, 1304, 1306)).

[0273] 2.6 Memory Coherence

[0274] In some embodiments, the accelerator (e.g., its PE) is coupled to the RAF circuit or multiple RAF circuits via networks such as: (i) circuit-switched networks (e.g., as discussed herein, such as references...). Figure 6-11 (i) via packet-switched networks (e.g., as discussed herein, for example, references) Figure 12-29In some embodiments, accelerators (e.g., non-x86 instruction set architecture (ISA) compatible accelerators) are free to adopt a relaxed memory model. Distributed accelerators (e.g., x86ISA compatible accelerators) may adopt a consistency model, where memory transactions originating from a single memory operation, or more specifically, memory transactions originating from a single memory input channel, are ordered relative to each other. This ordering can be from the perspective of the channel that generated the transaction and from the perspective of an external observer examining transactions on a single address. An external observer may see different orderings across addresses as well as accesses originating from other input channels, memory operations, and memory interface circuitry (e.g., RAF) may not be guaranteed to be ordered. One advantage of this model is that, in some embodiments, requests are not typically ordered relative to each other. This can simplify hardware, improve parallelism, and (e.g., largely) remove microarchitecture serial bottlenecks. For some data-parallel programs, this model ensures high performance at low energy costs. However, some largely data-parallel programs may (e.g., occasionally) require a certain degree of memory ordering. Some embodiments herein provide architectures and methods for allowing (e.g., programmer-imposed) control over memory ordering when needed. In one embodiment, unlike memory accesses that do not require global visibility and are significantly slowed down by waiting for global visibility (e.g., average case), special annotations are introduced to apply memory ordering (e.g., in distributed spatial memory systems). In cases requiring enhanced ordering, memory operations (e.g., optionally) are marked with these modifiers. Some embodiments herein provide one or more memory ordering implementations in the context of spatial architectures (e.g., CSAs) discussed herein. Memory ordering implementations can also be applied to field-programmable gate arrays (FPGAs). Some embodiments herein provide hardware for memory ordering implementations. Some embodiments herein provide multiple memory interface circuits (e.g., performing memory accesses in parallel), for example, ordering accesses occurring across a set of asymmetric memory access channels. Some embodiments herein allow for finer-grained ordering operations, for example, ordering requests within an accelerator (e.g., an accelerator die).

[0275] Some embodiments herein allow different memory ordering (e.g., coherence) modes, for example, within a space accelerator memory subsystem. Some embodiments herein allow individual memory accesses and access flows to be ordered within RAF circuitry, on-chip accelerator, and / or across (e.g., globally) coherent systems. Some embodiments herein provide memory coherence that allows for a mapping of sequential programs to a spatial architecture (e.g., CSA) (e.g., using out-of-order (e.g., capable) memory systems).

[0276] In some embodiments herein, memory operations are not implemented within the architecture (e.g., the architecture of a spatial array) to avoid making memory operations slow, area-starved, and feature-limited. Some embodiments herein do not allow the memory operation interface to be user-defined. Some embodiments herein provide a high-level consistent memory subsystem, for example, instead of merely providing a basic memory interface capable of achieving high performance but without the refined interface semantics necessary for (e.g., all) parallel programming models. Spatial architectures may utilize dependency flows, which can generally refer to memory dependency tokens used for ordering memory operations (e.g., as discussed herein). Some embodiments herein extend dependency flows to a more precise understanding of the behavior of the system memory.

[0277] Some embodiments herein provide memory consistency hardware with multiple granularity levels, such as (i) local-level memory consistency, (ii) chip-level memory consistency, and (iii) global-level memory consistency. For example, local-level memory consistency can be implemented by memory accesses (e.g., access requests) ordered at the granularity of a single RAF circuit (e.g., having one or more channels in a network to and / or coupled to that memory). Local-level memory consistency can be applied to individual operations at a fine-grained level, where such methods are valuable for fine-grained communication, as found in producer-consumer patterns. Chip-level memory consistency can be implemented by memory accesses (e.g., access requests) ordered at the granularity of a single accelerator chip. Global-level memory consistency can be implemented by memory accesses (e.g., access requests) ordered within a global memory consistency system. In some embodiments, the performance of code with (e.g., stronger) consistency requirements is improved by differentiating the levels of these memory consistency (e.g., visibility) levels within the memory system. Some embodiments herein provide memory coherence hardware that is switchable between multiple (e.g., three) granularity levels, such as switchable between (i) local-level memory coherence mode, (ii) slice-level memory coherence mode, and (iii) global-level memory coherence mode. Coherence requirements can be encoded into operational code for specific memory operations. For example, memory stored... Figure 30The operation code in register 3010. A memory dependency token can be a data value (e.g., a single bit) that stops outputting a memory access from the RAF circuit (e.g., a memory access request) until a memory dependency token value is received. For example, a first memory access may be ordered relative to a second memory access, and the second memory access may not access its memory location (e.g., be posted to memory) until a memory dependency token generated because the first memory access has reached or achieved a desired milestone (e.g., a memory access is initiated or completed, or in other ways discussed herein). Consistency can refer to the situation in which memory dependency tokens are generated (e.g., generated, transmitted, and / or stored) for the execution of a particular operation. Subsequent executions of the same memory operations (e.g., ordered in a pipelined manner to ensure local consistency) may not be completed simultaneously. In some embodiments, this enables pipelined requests even when dependencies exist. In some embodiments, local consistency involves the ordering of operations on a single RAF circuit (e.g., a single channel if multiple channels exist in the RAF circuit) and is assumed to apply to all operations. In one embodiment, the point of local ordering is the RAF circuit, and the RAF circuit may not reorder requests originating from the same memory operation or set of operation arguments. Although in some embodiments dependency tokens may not be generated for local ordering, e.g., when they are implicit in manipulation of the argument queue used for memory operations, memory dependency tokens can be logically generated within the RAF circuit (e.g., at the operation scheduler). In some embodiments, slice-level consistency allows distributed memory operations within a single accelerator (e.g., an accelerator slice) to see a consistent view of memory. One use of this pattern is for memory-dependent accesses, e.g., involving unresolved pointer ambiguity. In one embodiment, the point of slice ordering is a memory (e.g., cache) entry port or an exit from the RAF to a memory network (e.g., ACI). At this point, messages from various RAFs (e.g., within the slice) are fully ordered. In one embodiment, once a request (e.g., a message) has entered (e.g., is known to have entered) the memory (e.g., cache) entry queue, a memory dependency token associated with the operation is issued. In some embodiments, global horizontal consistency orders each operation against a full memory (e.g., cache) consistency protocol. In one embodiment, the point of global ordering is the memory (e.g., cache), specifically, the point where ownership of the data involved (e.g., cache lines) is acquired and access completion is guaranteed. At this point, a memory dependency token can be generated and sent to downstream operations. In one embodiment, this ensures that any external or internal observer will see the result of the access. Some embodiments herein allow individual memory accesses and access flows to be ordered within an accelerator (e.g., chip) and across consistency systems.

[0278] In some embodiments, this set of memory operations supported by RAF circuitry includes multiple sorting granularities. Some example operations (e.g., operation types) are listed in Table 2 below, but other operations may follow a similar pattern.

[0279] Table 2: Memory Operators

[0280]

[0281] Local horizontal memory coherence hardware

[0282] In some embodiments, memory operations are located at the RAF circuitry. These memory operations may be driven by the PE, which generates data and addresses. The RAF circuitry in the depicted embodiments controls memory accesses to memory (e.g., issuing memory access requests). In one embodiment, memory ordering is relaxed and the RAF circuitry (e.g., the RAF itself) is typically decoupled from each other unless necessary to maintain program correctness. Situations requiring additional RAF-to-RAF communication may include executing program ordering, where memory accesses derived from sequential programming languages ​​must be serialized, for example, due to the possible existence of aliased addresses, and where certain ordering is applied to the memory system to implement entity-to-entity communication protocols. Some embodiments herein provide hardware for ordering requests (e.g., memory access requests) in (e.g., a single) RAF circuitry (e.g., for each channel of a multi-channel RAF circuitry). In one embodiment, memory access requests originate from one or more PEs coupled to a single RAF circuitry. Cache blocks may include buffers, for example, for storing pending requests.

[0283] Figure 31 The illustration shows multiple Request Address File (RAF) circuits (1)-(8) coupled between an accelerator 3106 (formed by a plurality of accelerator chips 3108, 3109, 3110 and 3111) and a plurality of cache blocks (1)-(8) according to an embodiment of the present disclosure. The in-structure storage and communication circuitry depicted in the accelerator 3106 are each optional. The RAF circuitry can be as disclosed herein, for example, as... Figure 32 The RAF circuit depicted is coupled to the accelerator 3106 via a communication network 3102. The communication network 3102 can be a circuit-switched network, for example, such as... Figure 6-11 As shown in the diagram. Alternatively or additionally, the communication network 3102 may be a packet-switched network, for example, such as... Figure 12-29As shown in the diagram. The communication network 3102 can carry data between PEs and / or between PEs and RAF circuits. In one embodiment, accelerator 3106 (e.g., its PE) sends a memory access request to the RAF circuit. The memory access request may include an address (e.g., for a load operation or a store operation) and payload data (e.g., stored for a store operation). The requesting entity may be a PE, a network data stream endpoint circuit, or other hardware component.

[0284] The depicted RAF circuitry is coupled to a memory (e.g., multiple blocks depicted as a cache) via a second communication network 3104. The second communication network 3104 can couple any one of the RAF circuitry to any one of the cache blocks. The second communication network 3104 can be a fixed latency network. The second communication network 3104 can be a circuit-switched network, for example, as... Figure 6-11 As shown in the diagram. The second communication network 3104 can be a fixed latency crossover, for example, a crossover switch with multiple switching devices to couple any one of the RAF circuits to any one of the cache blocks. The number of RAF circuits and the number of cache blocks can be the same (e.g., as shown in the diagram). Figure 31 (Eight as shown) or different quantities.

[0285] Each RAF circuit may include a scheduler (e.g., an operation manager circuit) to control memory access to memory (e.g., issuing memory access requests). The RAF circuit may include one or more of the following components and / or features. The depicted RAF circuit (1) includes a scheduler 3112, a memory-dependent token queue 3118, and an input queue 3122. Each RAF circuit may include one or any number of memory-dependent token queues. Each RAF circuit may include one or any number of input queues and / or output queues. In some embodiments, the scheduler 3112 (e.g., its data storage) is used to monitor data sent to and / or sent from the queues, e.g., monitoring the input queue 3122. In one embodiment, the queue (e.g., the input queue 3122) is coupled (e.g., via network 3102) to a requesting entity (e.g., one or more PEs of accelerator 3106). Multiple memory access requests (e.g., from different PEs) may be sent (e.g., programmed to be sent, for example, via a circuit-switched network) to the RAF circuit (1), e.g., the input queue 3122. For example, two access requests, referred to as a first access request and a second access request, can be sent to the RAF circuit (1), such as input queue 3122 or a queue. It may be desirable to ensure (e.g., require) that the first access request occurs before the second access request, for example, when the first access request occurs in program order before the second access request. In one embodiment, request 1 is a load or store, and request 2 is a load or store, for example, to the same address. In some embodiments, scheduler 3112 is configured (e.g., programmed to) check whether a corresponding memory dependency token for the first access request has been received before issuing the second access request (e.g., issuing a request to memory (e.g., a cache block) or a network 3104 coupled (e.g., directly) to memory). In one embodiment, the second access request is stored in input queue 3122 until memory dependency token queue 3118 receives a memory dependency token indicating that the second access request can begin (e.g., be issued). In some embodiments, a memory dependency token for a second access request is generated (e.g., sent and / or stored) because the first memory access has reached or achieved a desired milestone (e.g., the memory access for the first access request is initiated or completed, or in other ways as discussed herein). The depicted RAF circuit (1) publishes access requests to memory (e.g., cache blocks) via network 3104. In one embodiment, the memory access request includes an address (e.g., physical) for each access request, and network 3104 directs the access requests (e.g., load requests or store requests with storage payloads) to the appropriate cache block. In one embodiment, multiple access requests are directed to the same cache block (e.g., the same address within the same cache block). Figure 31 The circled numbers may refer to sequential time steps, for example, but in some embodiments they may not be linear time steps.

[0286] In some embodiments, a first access request is received in a first RAF circuit, and a second access request is received in a second RAF circuit (e.g., statically configured to always be received in the same RAF circuits used for the program), and the first access request occurs before the second access request, for example, the first access request is in program order before the second access request. Therefore, the second access request can be flagged (e.g., in the scheduler) to request that a memory dependency token caused by the first access request be received before the second access request is issued. In one embodiment, before the first RAF circuit sends the memory dependency token to a second RAF circuit that will issue (e.g., cause issuance) a second access request that includes a dependency on the first access request for memory (e.g., program order), the first RAF circuit will wait until the first access request for memory reaches the cache and / or the memory access to the first access request is completed (e.g., and the second RAF circuit receives an acknowledgment).

[0287] In some embodiments, a first access request and a second access request are received (e.g., programmed to receive) in the same RAF circuit (1) (e.g., statically configured to always be received in the same RAF circuit (1) for a program), and the first access request occurs before the second access request, for example, the first access request is in program order before the second access request. Therefore, the second access request can be flagged (e.g., in the scheduler) to require a memory dependency token caused by the first access request to be received before the second access request is issued. When the first access request and the second access request (e.g., always for a program) which includes a dependency on the first access request (e.g., program order) are received in the same RAF circuit, a single RAF circuit can use local horizontal memory coherence.

[0288] exist Figure 31In the illustrated embodiment, the RAF circuit (1) receives a first memory access request (shown as MR1) and a second memory access request (shown as MR2). In one embodiment, the first memory access request may be received in a first slot 3122 (0) of the input queue 3122, and the second memory access request may be received in a second slot 3122 (1) of the input queue 3122, for example, at some point before time 1 (e.g., marked by circle 1). The second memory access request may include a corresponding slot in the memory dependency token queue 3118 (e.g., the second slot 3118 (1)). In this example, the issuance of the second memory access request is stopped until a memory dependency token is received in a slot in the memory dependency token queue 3118 (e.g., the second slot 3118 (1)). This memory dependency token is generated for the second memory access request because the first memory access has reached or achieved a desired milestone (e.g., the memory access is initiated or completed, or in other ways as discussed herein). Instead of waiting for multiple cycles to pass (e.g., after publication) for the first access request or waiting for the first access request to reach the cache and / or for the memory access used for the first access request to be completed (e.g., and the second RAF circuit receives an acknowledgment), in some embodiments, a single RAF circuit (1) will allow the second memory access request to be published immediately after the first memory access request is published to memory (e.g., into network 3104) (e.g., in the next cycle or two cycles later), for example, without the memory dependency token traveling on network 3102 and / or accelerator 3106. In one embodiment, the RAF circuit (1) (e.g., scheduler 3112) will provide the memory dependency token in a slot (e.g., second slot 3118(1)) in the memory dependency token queue 3118 for the second memory access request, for example, when scheduling or publishing the first access request to network 3104, for example, to access the address used for the access request. The second access request can then be scheduled or published to network 3104, for example, based on the presence of the memory dependency token in a slot (e.g., second slot 3118(1)) of the memory dependency token queue 3118 for the second memory access request (e.g., read by scheduler 3112). In one embodiment, the memory dependency token is provided (e.g., stored) to the memory dependency token queue 3118 without the memory dependency token traveling through network 3102, network 3104, and / or accelerator 3106. In one embodiment, the second memory access request is published to network 3104 in the next cycle (e.g., immediately after the first memory access request is published to network 3104).In some embodiments, network 3104 is a fixed wait time network, such that a first access request in time cannot pass through a second access request in time in memory, for example, to maintain program order in between.

[0289] A first memory access request (shown as MR1) and a second memory access request (shown as MR2) are received (e.g., programmed to receive) in the same RAF circuit (1) (e.g., statically configured to always be received in the same RAF circuit (1) used for a program), and the first access request will occur before the second access request, for example, the first access request is in program order before the second access request. Therefore, the second access request can be flagged (e.g., in the scheduler) to require the memory dependency token caused by the first access request to be received before the second access request is issued. When the first access request and the second access request (e.g., always used for a program) which includes a dependency on the first access request (e.g., program order) are received in the same RAF circuit, a single RAF circuit can use local horizontal memory consistency. The depicted RAF circuit (1) issues the first memory access request (shown as MR1) to network 3104 at time 1 (e.g., marked by circle 1). In a local horizontal memory coherence mode for a second memory access request (shown as MR2) that is marked (e.g., in the scheduler) to require receiving a memory dependency token caused by the first access request before issuing the second access request, the depicted RAF circuit (1) issues the second memory access request (shown as MR2) to network 3104 at time 2 (e.g., marked by circle 2). At time 2 (e.g., marked by circle 2), the first memory access request has traveled to the point in network 3104 marked by circle 2. At time 3 (e.g., marked by circle 3), the first memory access request has reached a cache block (e.g., cache block (3)), causing the memory access requested by the first memory access request to be performed. At time 3 (e.g., marked by circle 3), the second memory access request has traveled to the point in network 3104 marked by circle 3 but has not yet reached a cache block (e.g., cache block (6)). At time 4 (e.g., marked by circle 4), a second memory access request has arrived at a cache block (e.g., cache block (6)), causing the memory access requested by the second memory access request to be executed. In another embodiment, the first and second access requests will access the same address used for the access requests (e.g., within the same cache block), and the aforementioned timing prevents the second access request from being executed (e.g., started) before the first access request is executed (e.g., completed) to maintain memory consistency. In one embodiment, each cache block is accessed in the order in which the accesses are received (e.g., at the input of the cache block). In some embodiments, consistency is maintained in a single RAF circuit by sorting in network 3104 (e.g., ACI) and / or cache blocks. In some embodiments, local horizontal memory consistency eliminates or reduces pipeline latency for any RAF circuit or network (e.g., ACI).

[0290] In one embodiment, the first memory access request does not require (e.g., receives) a memory dependency token before being published to memory (e.g., a cache block). In another embodiment, the first memory access request requires (e.g., receives) a memory dependency token before being published to memory (e.g., a cache block), for example, in the first slot 3118(0) of the memory dependency token queue 3118. Figure 31 The RAF circuit in the document can be as discussed herein (for example, refer to the following). Figure 32 ).

[0291] In one embodiment, the address received by the RAF circuit (e.g., from other circuitry in the PE or CSA) is a virtual address, and a physical address is then determined and used to access memory at that physical address, for example, by using... Figure 30 The TLB in this document is added to any RAF circuit.

[0292] Figure 32 The figure illustrates a Request Address File (RAF) circuit 3200 according to an embodiment of the present disclosure. The RAF circuit 3200 may be coupled (e.g., to) a circuit-switched network (e.g., such as...). Figure 6-11 One or more of the networks (3202, 3204, 3206, 3208) shown in the diagram receive input (e.g., including address values) of access requests from one or more requesting entities (e.g., PEs or other hardware components). The RAF circuit 3200 can be coupled (e.g., to) a packet-switched network (e.g., such as...). Figure 12-29 One or more of the networks (3202, 3204, 3206, 3208) shown in the diagram receive input (e.g., including address values) of an access request from one or more requesting entities (e.g., PE, network data stream endpoint circuits, or other hardware components).

[0293] In some embodiments, the data received for a memory (e.g., cache) access request is a memory command. The memory command may include the address to be accessed (e.g., physical), the operation to be performed (e.g., load or store), and / or payload data (e.g., for storage).

[0294] In one embodiment, for example, during configuration, memory (e.g., load and / or store) operations (e.g., in the data flow graph) are specified in register 3210 (e.g., store), and sent, for example, via one or more of networks 3202, 3204, 3206, or 3208. Arcs from those memory operations in the data flow graph can then be connected to input queues 3222, 3224, and 3226. Arcs from those memory operations are thus used to exit completion buffers 3228, 3230, or 3232. Memory dependency tokens (which may each be a single bit) arrive in queues 3218 and 3220. Memory dependency tokens will exit from queue 3216 into the network, for example, in slice-level consistency mode or global-level consistency mode. In the depicted embodiments, memory dependency tokens can be generated and sent to memory dependency token queue 3218, for example, on path 3217, and / or memory dependency token queue 3220, for example, on path 3219, for example, on paths 3217 or 3219 of the RAF circuit 3200 without using a network (e.g., without using one or more of the networks (3202, 3204, 3206, 3208)). Dependency token counter 3214 can be a compact representation of the queues and can track the number of dependency tokens used for any given input queue. In some embodiments, if dependency token counter 3214 saturates, no additional dependency tokens can be generated for new memory operations. Accordingly, the memory sorting circuit (e.g., Figure 32 The RAF in the memory will stop scheduling new memory operations until the dependency token counter 3214 becomes unsaturated.

[0295] As an example of a load (e.g., a load request), an address arrives at queue 3222, and scheduler 3212 matches that address with a load operation in register 3210. Completion buffer slots for this load can be assigned in the order the addresses arrive. Assuming that a particular load specifies (e.g., in a “Load Memory Operation” stored in register 3210) dependencies, dispatching addresses (e.g., and completion buffer slots) to the memory system by the scheduler can be halted (e.g., as memory command 3242) until the corresponding memory dependency token arrives, for example, in memory dependency token queue 3218, e.g., in slot 3218(0). Once a memory dependency token is received in the memory dependency token queue for this load, the address (e.g., and completion buffer slot) 3241 can be dispatched by the scheduler to the memory system (e.g., as memory command 3242). In one embodiment, the address of the loaded instance is received in input queue 3222 (e.g., in slot 3222(0)), and a memory dependency token for allowing the issuance of memory dependency tokens for those loads is received in memory dependency token input queue 3218, e.g., from scheduler 3212 on path 3217. Assuming that a particular load does not specify dependencies, the address (e.g., and completion buffer slot) can be dispatched to the memory system by the scheduler (e.g., as memory command 3242). When the result returns to mux 3240 (illustratively shown), the result can be stored in the completion buffer slot it is designated to (e.g., since the result carries the target slot along the entire path through the memory system). The completion buffer can send the result back to the local network (e.g., local networks 3202, 3204, 3206, or 3208) in the order the addresses arrive. Storage can be similar, except that both the address and the data may have to arrive before any operation is dispatched to the memory system. As an example of storage (e.g., a storage request), an address arrives at queue 3224 (e.g., in slot 3224(0)), and payload data to be stored at that address arrives at queue 3226 (e.g., in slot 3226(0)) (e.g., at a different time than the address arrival at queue 3224). Scheduler 3212 matches this address with a storage operation in register 3210. When storage will generate a dependency token, the completion buffer slot for that storage can be allocated in the order the addresses arrive. Assuming that a particular storage specifies (e.g., in the "Store Memory Operation" stored in register 3210) a dependency, the dispatch of the address and payload data (e.g., and the completion buffer slot) to the memory system by the scheduler can be stopped (e.g., as memory command 3242) until the corresponding memory dependency token arrives, for example, in memory dependency token queue 3220.Once a memory dependency token is received in the memory dependency token queue of the storage, the scheduler can dispatch the address and payload data (e.g., and completion buffer slot) to the memory system (e.g., as memory command 3242). In one embodiment, the address of the storage instance is received in input queue 3224, the payload data of those storage instances is received in input queue 3226, and memory dependency tokens for allowing the issuance of those storage instances are received in memory dependency token input queue 3220, e.g., from scheduler 3212 on path 3219. Assuming that a particular storage does not specify a dependency, the scheduler can dispatch the address and payload data (e.g., and completion buffer slot) to the memory system (e.g., as memory command 3242). When the result returns to mux 3240 (illustratively shown), the result can be stored in the completion buffer slot it is designated for (e.g., since the result carries the target slot throughout its journey through the memory system). The completion buffer can send results back to the local network (e.g., local networks 3202, 3204, 3206, or 3208) in the order they arrive at the address.

[0296] In some embodiments, a first access request and a second access request are received (e.g., programmed to receive) in one or more input queues (e.g., input queues 3222, 3224, or 3226) of the same RAF circuit 3200 (e.g., statically configured to always be received in the same RAF circuit 3200 for a program), and the first access request will occur before the second access request, for example, the first access request is in program order before the second access request. Therefore, the second access request can be flagged (e.g., in register 3210 and / or scheduler 3212) to require the reception of a memory dependency token caused by the first access request (e.g., in queue 3218 or queue 3220) before the second access request is issued. When the first access request and the second access request (e.g., always for a program) that includes a dependency on the first access request (e.g., program order) are received in the same RAF circuit, a single RAF circuit 3200 can use local horizontal memory coherence. The first access request may be: (i) the address of a load request received in input queue 3222, or (ii) the address of a storage request received in input queue 3224 and payload data for the address of the storage request received in input queue 3226. In this example, a second memory access request will be (e.g., programmed) received in the same RAF circuit 3200, and the second memory access request is marked (e.g., in its memory operation field in register 3210) to request the receipt of a memory dependency token (e.g., in queue 3218 or queue 3220) caused by the arrival or achievement of a desired milestone by the first access request, for example, to release the stop of the issuance of the second access request. In one embodiment of local memory coherence, the desired milestone is that the first memory access is issued to memory, for example, as memory command 3242 enters memory 3241. After the expected milestone of the first request is reached or achieved (e.g., scheduler 3212 sends a memory command), the scheduler can then issue (e.g., a stopped) a second access request, assuming that the address and any payload data are ready for the second access request. In one embodiment, the second access request is stored in one or more of the input queues (3222, 3224, 3226), and one of the memory dependency token input queues (3218, 3220) will be populated with a memory dependency token for the second request indicating that the expected milestone of the first access request has been reached or achieved. In one embodiment, the scheduler will provide the memory dependency token to the memory dependency token input queue 3218 via path 3217, or to the memory dependency token input queue 3220 via path 3219.The scheduler can then read memory dependency tokens from the memory dependency token input queue and issue a second memory access request to the memory, for example, as memory command 3242 to memory 3241. In another embodiment, instead of actually filling the memory dependency tokens into the memory dependency token input queue 3218 via path 3217 or into the memory dependency token input queue 3220 via path 3219, the scheduler 3212 uses the arrival of the first access request or the achievement of a desired milestone as a trigger to issue the second memory access request to the memory (e.g., as memory command 3242 to memory 3241). In some embodiments, the first and second memory accesses are thus issued to the memory in procedural order and are guaranteed to be served in procedural order, for example, when the memory system maintains them in the order they are received in the memory system.

[0297] Therefore, the memory dependency token generated for the access request in the local horizontal memory coherence mode can be sent within the RAF circuit 3200 without sending the memory dependency token as a memory command 3242 or without sending the memory dependency token externally from the RAF circuit 3200, for example, without using any of the networks connected to the RAF circuit 3200 (e.g., without using any of the networks (3202, 3204, 3206, 3208)). In one embodiment, the memory dependency token is a single bit, and the address and payload data are multiple bits (e.g., 32 or 64 bits). In one embodiment, the memory dependency token input queue (3218, 3220) is backpush back such that path 3217 or path 3219 does not send the memory dependency token to one or more queues (3218, 3220), for example, when the queue is full. Although in Figure 32 Two memory-dependent token input queues (3218, 3220) are depicted, but a single memory-dependent token input queue or three or more memory-dependent token input queues may be used. In some embodiments, each queue will receive multiple instances of the same operation, for example, but not other operations. In some embodiments, local memory consistency is maintained even when the (e.g., physical) addresses accessed by the first and second access requests may be the same address.

[0298] In one embodiment, the RAF circuitry includes a TLB for providing an output of the physical address of the input for a memory access request in the order in which the inputs are received (e.g., program order). The RAF circuitry may include a TLB, such as a reference... Figure 30 The TLB under discussion.

[0299] Figure 33 The diagram illustrates a flowchart 3300 according to an embodiment of the present disclosure. Flow 3300 includes: coupling a plurality of request address file circuits to a spatial array of processing elements via a first communication network, and coupling memory to the plurality of request address file circuits via a second communication network (3302); receiving a first access request to memory and a second access request to memory marked with a program sequence dependency on the first access request from the spatial array of processing elements by a single request address file circuit among the plurality of request address file circuits (3304); stopping the publication of the second access request to the second communication network by the single request address file circuit (3306); providing a first memory dependency token for program sequence dependency by the single request address file circuit when publishing the first access request to the second communication network (3308); and publishing the second access request to the second communication network by the single request address file circuit based on reading the first memory dependency token (3310).

[0300] Slice-level memory coherence hardware

[0301] In some embodiments, memory operations are located at the RAF circuitry. These memory operations may be driven by PEs, which generate data and addresses. The RAF circuitry in the depicted embodiments controls memory accesses to memory (e.g., issuing memory access requests). In one embodiment, memory ordering is relaxed and the RAF circuitry (e.g., the RAF itself) is typically decoupled from each other unless necessary to maintain program correctness. Situations requiring RAF-to-RAF communication may include implementing program ordering, such as where memory accesses derived from sequential programming languages ​​must be serialized, for example, due to the possible existence of aliased addresses, and where certain ordering is applied to the memory system to implement entity-to-entity communication protocols. Some embodiments herein provide hardware for ordering requests (e.g., memory access requests) within RAF circuitry (e.g., RAF circuitry in a single accelerator or accelerator chip). In one embodiment, the memory access request originates from one or more PEs coupled to the RAF circuitry.

[0302] Figure 34 The illustration shows multiple Request Address File (RAF) circuits (1)-(8) coupled between an accelerator 3406 (formed by a plurality of accelerator chips 3408, 3409, 3410 and 3411) and a plurality of cache blocks (1)-(8) according to an embodiment of the present disclosure. The in-structure storage and communication circuitry depicted in the accelerator 3406 are each optional. The RAF circuitry can be as disclosed herein, for example, as... Figure 35 The RAF circuit depicted is coupled to the accelerator 3406 via a communication network 3402. The communication network 3402 can be a circuit-switched network, for example, such as... Figure 6-11 As shown in the diagram. Alternatively or additionally, the communication network 3402 may be a packet-switched network, for example, such as... Figure 12-29 As shown in the diagram. The communication network 3402 can carry data between PEs and / or between PEs and RAF circuits. In one embodiment, the accelerator 3406 (e.g., its PE) sends a memory access request to the RAF circuit. The memory access request may include an address (e.g., for a load operation or a store operation) and payload data (e.g., stored for a store operation). The requesting entity may be a PE, a network data stream endpoint circuit, or other hardware component.

[0303] The depicted RAF circuitry is coupled to a memory (e.g., multiple blocks depicted as a cache) via a second communication network 3404. The second communication network 3404 can couple any one of the RAF circuitry to any one of the cache blocks. The second communication network 3404 can be a fixed latency network. The second communication network 3404 can be a circuit-switched network, for example, as... Figure 6-11 As shown in the diagram. The second communication network 3404 can be a fixed latency crossover, for example, a crossover switch with multiple switching devices to couple any one of the RAF circuits to any one of the cache blocks. The number of RAF circuits and the number of cache blocks can be the same (e.g., as shown in the diagram). Figure 34 (Eight as shown) or different quantities.

[0304] Each RAF circuit may include a scheduler (e.g., an operation manager circuit) to control memory access to memory (e.g., issuing memory access requests). The RAF circuit may include one or more of the following components and / or features. The depicted RAF circuit (1) includes a scheduler 3412A, a memory-dependent token queue 3418A, and an input queue 3422A. The depicted RAF circuit (5) includes a scheduler 3412B, a memory-dependent token queue 3418B, and an input queue 3422B. Each RAF circuit may include one or more memory-dependent token queues. Each RAF circuit may include one or more input queues and / or output queues. In some embodiments, schedulers (e.g., schedulers 3412A and 3412B) monitor data sent to and / or sent out of their queues. For example, scheduler 3412A monitors input queue 3422A and memory-dependent token queue 3418A, and / or scheduler 3412B monitors input queue 3422B and memory-dependent token queue 3418B. In one embodiment, queues (e.g., input queues 3422A and 3422B) are coupled (e.g., via network 3402) to requesting entities (e.g., one or more PEs of accelerator 3406), for example, each RAF circuit is coupled to a separate PE. Multiple memory access requests (e.g., from different PEs) can be sent (e.g., programmed to be sent, for example, via a circuit-switched network) to RAF circuits, such as their input queues. Two access requests, referred to as a first access request and a second access request, may be sent. For example, the first access request may be sent to RAF circuit (1) (e.g., input queue 3422A), and the second access request may be sent to RAF circuit (5) (e.g., input queue 3422B). It may be desirable to ensure (e.g., require) that the first access request occurs before the second access request, for example, when the first access request occurs in program order before the second access request. In one embodiment, request 1 is a load or store, and request 2 is a load or store, for example, to the same address. In some embodiments, RAF circuit (5) (e.g., scheduler 3412B) is configured (e.g., programmed to) check whether the corresponding memory dependency token of the first access request (e.g., in its memory dependency token queue 3418B) has been received before the second access request is published (e.g., publishing the request to memory (e.g., cache block) or coupled (e.g., directly) to network 3404 of memory). In one embodiment, the second access request is stored in the input queue 3422B until the memory dependency token queue 3418B receives a memory dependency token indicating that the second access request can be initiated (e.g., issued).In some embodiments, a memory dependency token for a second access request is generated (e.g., sent and / or stored) because the first memory access has reached or achieved a desired milestone (e.g., the memory access for the first access request is initiated or completed, or in other ways as discussed herein). The depicted RAF circuitry publishes access requests to memory (e.g., cache blocks) via network 3404. In one embodiment, the memory access request includes an address (e.g., physical) for each access request, and network 3404 directs the access requests (e.g., load requests or store requests with storage payloads) to the appropriate cache block. In one embodiment, multiple access requests are directed to the same cache block (e.g., the same address within the same cache block). Figure 34 The circled numbers may refer to sequential time steps, for example, but in some embodiments they may not be linear time steps.

[0305] In some embodiments, a first access request is received in a first RAF circuit, and a second access request is received in a second RAF circuit (e.g., statically configured to always be received in the same RAF circuits used for the program), and the first access request occurs before the second access request, for example, the first access request is in program order before the second access request. Therefore, the second access request can be flagged (e.g., in the scheduler) to request that a memory dependency token caused by the first access request be received before the second access request is issued. In one embodiment, before the first RAF circuit sends the memory dependency token to a second RAF circuit that will issue (e.g., cause issuance) a second access request that includes a dependency on the first access request for memory (e.g., program order), the first RAF circuit will wait until the first access request for memory reaches the cache and / or the memory access to the first access request is completed (e.g., and the second RAF circuit receives an acknowledgment). In another embodiment, after the first access is published to the memory, the first RAF circuit will wait for a predetermined period of time (or, for example, multiple cycles) before sending a memory dependency token to a second RAF circuit that will publish (e.g., cause the publication) a second access request to the memory that includes a dependency on the first access request to the memory (e.g., program order).

[0306] In some embodiments, a first access request and a second access request are received (e.g., programmed to receive) in the RAF circuitry of a single accelerator (e.g., a single accelerator chip) (e.g., statically configured to always be received in the same RAF circuitry used for a program), and the first access request occurs before the second access request, for example, the first access request is in program order before the second access request. Therefore, the second access request can be flagged (e.g., in the scheduler) to require a memory dependency token caused by the first access request to be received before the second access request is issued. When the first access request and the second access request (e.g., always used for a program) that includes a dependency on the first access request (e.g., program order) are received in the RAF circuitry of the same accelerator (e.g., one or more accelerator chips sharing a single (e.g., ACI) network), the RAF circuitry can use chip-level memory coherence.

[0307] exist Figure 34In the illustrated embodiment, RAF circuit (1) receives a first memory access request (shown as MR1), and RAF circuit (5) receives a second memory access request (shown as MR2). In one embodiment, the first memory access request may be received in the first slot 3422A(0) of the input queue 3422A in RAF circuit (1), and the second memory access request may be received in the first slot 3422B(0) of the input queue 3422B in RAF circuit (5), for example, at some time before time 1 (e.g., marked by circle 1). The second memory access request may include a corresponding slot (e.g., the first slot 3418B(0)) in the memory dependency token queue 3418B. In this example, the issuance of the second memory access request is stopped until a memory dependency token is received in the slot (e.g., the first slot 3418B(0)) of the memory dependency token queue 3418B in RAF circuit (5). The memory dependency token is generated for the second memory access request because the first memory access has reached or achieved a desired milestone (e.g., a predetermined time period has elapsed or been achieved since the first memory access was initiated, or other ways as discussed herein). In some embodiments, instead of issuing the second memory access in a single cycle after the first access request is issued or after waiting for the first access request to reach the cache and / or after the memory access to the first access request is completed (e.g., and the second RAF circuitry receives an acknowledgment), the RAF circuitry in the same accelerator (e.g., the same chip) will allow the second memory access request to be issued over a predetermined time period (e.g., three or more cycles) after the first memory access request is issued to memory (e.g., into network 3404), for example, where the memory dependency token travels on its own communication path 3417 (illustrated) or on network 3402 and / or in accelerator 3406. In one embodiment, the RAF circuit (1) (e.g., scheduler 3412A) provides the memory dependency token in a slot (e.g., first slot 3418B(0)) of the memory dependency token queue 3418B in the RAF circuit (5) for a second memory access request, for example, after a predetermined time period following the scheduling or publication of the first access request to the network 3404 (e.g., to access the address used for the access request). The second access request can then be scheduled or published to the network 3404, for example, based on the presence (e.g., read by scheduler 3412B) of the memory dependency token in the slot (e.g., first slot 3418B(0)) of the memory dependency token queue 3418B in the RAF circuit (5) for the second memory access request. In one embodiment, the memory dependency token is provided (e.g., stored) in the memory dependency token queue 3418B by memory dependency tokens traveling on the network 3402.In some embodiments, network 3404 is a fixed wait time network, such that a first access request in time cannot pass through a second access request in time in memory, for example, to maintain program order in between.

[0308] A first memory access request (shown as MR1) is received (e.g., programmed to receive) in RAF circuit (1), and a second memory access request (shown as MR2) is received (e.g., programmed to receive) in RAF circuit (5) (e.g., statically configured to always be received in those same RAF circuits used for the program), and the first access request will occur before the second access request, for example, the first access request is in program order before the second access request. Therefore, the second access request can be flagged (e.g., in scheduler 3412B) to require a memory dependency token caused by the first access request to be received before the second access request is issued. When the first access request and the second access request (e.g., always used for the program) which includes a dependency on the first access request (e.g., program order) are received in the same accelerator circuit 3406 (e.g., to the RAF circuits coupled together via the same network 3404), RAF circuits (1) and (5) can use chip-level memory coherence. The depicted RAF circuit (1) publishes a first memory access request (shown as MR1) to network 3404 at time 1 (e.g., marked with circle 1). In a slice-level memory coherence mode for a second memory access request (shown as MR2) that is marked (e.g., in scheduler 3412B) to request the receipt of a memory dependency token caused by (e.g., thereafter) a predetermined time period elapsed since the first access request was published to memory before the second access request is published, the depicted RAF circuit (5) publishes the second memory access request (shown as MR2) to network 3404 at time 4 (e.g., marked with circle 4) (e.g., three cycles after time 1). At time 2 (e.g., marked with circle 2), the first memory access request has traveled to the point in network 3404 marked with circle 2. At time 3 (e.g., marked with circle 3), the first memory access request has reached a cache block (e.g., cache block (3)), causing the memory access requested by the first memory access request to be performed. At time 3 (e.g., marked with circle 3), a memory dependency token for the second memory access request is sent, and at time 4 (e.g., marked with circle 4), the process has progressed to the RAF circuit (5). In another embodiment, the first and second access requests will access the same address used for the access requests (e.g., in the same cache block), and the aforementioned timing prevents the second access request from being executed (e.g., started) before the first access request is executed (e.g., completed) to maintain memory consistency. In one embodiment, each cache block is accessed in the order in which the accesses are received (e.g., at the input of the cache block). In some embodiments, consistency is maintained in a single accelerator (e.g., accelerator chip) by sorting in the network 3404 (e.g., ACI) and / or cache blocks.In some embodiments, chip-level memory coherence reduces pipeline latency for any RAF circuitry or network (e.g., ACI).

[0309] In one embodiment, the first memory access request does not require (e.g., receives) a memory dependency token before being published to memory (e.g., a cache block). In another embodiment, the first memory access request requires (e.g., receives) a memory dependency token before being published to memory (e.g., a cache block), for example, in the first slot 3418A(0) of the memory dependency token queue 3418A. Figure 34 The RAF circuit in the document can be as discussed herein (for example, refer to the following). Figure 35 ).

[0310] In one embodiment, the address received by the RAF circuit (e.g., from other circuitry in the PE or CSA) is a virtual address, and a physical address is then determined and used to access memory at that physical address, for example, by using... Figure 30 The TLB in this document is added to any RAF circuit.

[0311] Figure 35 The illustration shows a first Request Address File (RAF) circuit 3500A coupled to a second Request Address File (RAF) circuit 3500B according to an embodiment of the present disclosure. In the depicted embodiment, a network (e.g., its channels) couples the first RAF circuit 3500A to the second RAF circuit 3500B. The network (e.g., one or more of networks (3502, 3504, 3506, 3508)) can be a circuit-switched network, such as... Figure 6-11 As shown, it could be a packet-switched network, for example, such as Figure 12-29 As shown in the diagram. In one embodiment, one of the networks (3502, 3504, 3506, 3508) couples one or more first request entities (e.g., PE, network data stream endpoint circuitry, or other hardware components) to the first RAF circuit 3500A, and another of the networks (3502, 3504, 3506, 3508) couples one or more second request entities (e.g., PE, network data stream endpoint circuitry, or other hardware components) to the first RAF circuit 3500A and to the second RAF circuit 3500B. In one embodiment, the request entity is... Figure 34 The components of the accelerator 3406, for example, wherein the RAF circuit (1) is a first RAF circuit 3500A and the RAF circuit (2) is a second RAF circuit 3500B. The first RAF circuit 3500A and / or the second RAF circuit 3500B may be coupled (e.g., to) a circuit-switched network (e.g., such as... Figure 6-11One or more of the networks (3502, 3504, 3506, 3508) shown receive input (e.g., including address values) of access requests from one or more requesting entities (e.g., PE or other hardware components). The first RAF circuit 3500A and / or the second RAF circuit 3500B can be coupled (e.g., to) a packet-switched network (e.g., such as...). Figure 12-29 One or more of the networks (3502, 3504, 3506, 3508) shown in the diagram receive input (e.g., including address values) of an access request from one or more requesting entities (e.g., PE, network data stream endpoint circuits, or other hardware components).

[0312] In some embodiments, the data received for a memory (e.g., cache) access request is a memory command. The memory command may include the address to be accessed (e.g., physical), the operation to be performed (e.g., load or store), and / or payload data (e.g., for storage).

[0313] In one embodiment, for example, during configuration, memory (e.g., load and / or store) operations (e.g., in the data flow graph) are specified (e.g., stored) in register 3510A for the first RAF circuit 3500A and in register 3510B for the second RAF circuit 3500B, for example, and are sent via one or more of networks 3502, 3504, 3506, or 3508. Subsequently, arcs to those memory operations in the data flow graph may be connected to input queues 3522A, 3524A, and 3526A for the first RAF circuit 3500A and input queues 3522B, 3524B, and 3526B for the second RAF circuit 3500B. Arcs from those memory operations are thus used to exit from completion buffers 3528A, 3530A, or 3532A for the first RAF circuit 3500A and completion buffers 3528B, 3530B, or 3532B for the second RAF circuit 3500B. Memory-dependent tokens (which may each be a single bit) arrive at queues 3518A and 3520A for the first RAF circuit 3500A and queues 3518B and 3520B for the second RAF circuit 3500B. Memory-dependent tokens will then exit from queue 3516A for the first RAF circuit 3500A and queue 3516B for the second RAF circuit 3500B into the network, for example, in on-chip horizontal consistency mode or global horizontal consistency mode. In the depicted embodiments, memory-dependent tokens may be generated by a first RAF circuit 3500A and sent from a memory-dependent token queue 3516A to a memory-dependent token queue (e.g., 3518B or 3520B) of a second RAF circuit 3500B on one or more of the networks (3502, 3504, 3506, 3508). In one embodiment, network 3504 couples (e.g., is programmed to couple) the memory-dependent token output queue 3516A of the first RAF circuit 3500A to the memory-dependent token input queue 3518B of the second RAF circuit 3500B, for example, by providing such a connection by providing switching devices 3505 and 3507 of a circuit-switched network.

[0314] Dependency token counters (3514A and 3514B) can be compact representations of queues in the corresponding RAF circuits and can track the number of dependency tokens used for any given input queue. In some embodiments, if the dependency token counter saturates, no additional dependency tokens can be generated for new memory operations. Accordingly, the memory sorting circuit (e.g., Figure 35 The RAF in the memory will stop scheduling new memory operations until the dependency token counter becomes unsaturated.

[0315] As an example of a load (e.g., a load request), an address arrives at queue 3522A, and scheduler 3512A matches that address with a load operation in register 3510A. The completion buffer slot in the RAF circuit 3500A for this load can be assigned in the order the addresses arrive. Assuming that a particular load specifies (e.g., in a “Load Memory Operation” stored in register 3510A) dependencies, dispatching addresses (e.g., and completion buffer slots) to the memory system (e.g., as memory commands 3542A) by the scheduler can be stopped until the corresponding memory dependency token arrives, for example, in memory dependency token queue 3518A. Once a memory dependency token is received in the memory dependency token queue for that load, the address (e.g., and completion buffer slot) can be dispatched to the memory system (e.g., as memory commands 3542A) by the scheduler. In one embodiment, the address of the instance of the load is received in input queue 3522A, and memory dependency tokens for allowing the issuance of those loads are received in memory dependency token input queue 3518A. Assuming that a particular load does not specify dependencies, the scheduler can dispatch the address (and completion buffer slot) to the memory system (e.g., as memory command 3542A). When the result returns to mux 3540A (illustratively shown), the result can be stored in the completion buffer slot it is designated for (e.g., since the result carries the target slot along its entire journey through the memory system). The completion buffer can send the results back to the local network (e.g., local networks 3502, 3504, 3506, or 3508) in the order the addresses arrived. Storage can be similar, except that both the address and the data may have to arrive before any operation is dispatched to the memory system. As an example of storage (e.g., a storage request), an address arrives at queue 3524A, and the payload data to be stored at that address arrives at queue 3526A (e.g., at a different time than the address arriving at queue 3524A), the scheduler 3512A matches that address with a storage operation in register 3510A. When a memory dependency token is generated, the completion buffer slot for that memory can be allocated in the order the addresses arrive. Assuming that a particular memory specifies (e.g., in the "Memory Operation" stored in register 3510A) a dependency, the dispatch of the address and payload data (e.g., and the completion buffer slot) to the memory system by the scheduler (e.g., as memory command 3542A) can be stopped until the corresponding memory dependency token arrives, for example, in memory dependency token queue 3520A. Once a memory dependency token is received in the memory dependency token queue for that memory, the scheduler can dispatch the address and payload data (e.g., and the completion buffer slot) to the memory system (e.g., as memory command 3542A).In one embodiment, the addresses of the stored instances are received in input queue 3524A, the payload data of those stored instances is received in input queue 3526A, and memory dependency tokens for allowing the publication of those stores are received in memory dependency token input queue 3520A. Assuming that a particular store does not specify a dependency, the scheduler can dispatch the address and payload data (e.g., and completion buffer slot) to the memory system (e.g., as memory command 3542A). When the result returns to mux 3540A (illustratively shown), the result can be stored in its designated completion buffer slot (e.g., since the result carries the target slot throughout its journey through the memory system). The completion buffer can send the results back to the local network (e.g., local networks 3502, 3504, 3506, or 3508) in the order the addresses arrived.

[0316] As another example of a load (e.g., a load request), an address arrives at queue 3522B, and scheduler 3512B matches that address with a load operation in register 3510B. The completion buffer slot in the RAF circuit 3500B for this load can be assigned in the order the addresses arrive. Assuming that a particular load specifies (e.g., in a “Load Memory Operation” stored in register 3510B) dependencies, dispatching addresses (e.g., and completion buffer slots) to the memory system by the scheduler can be stopped (e.g., as memory command 3542B) until the corresponding memory dependency token arrives, for example, in memory dependency token queue 3518B. Once a memory dependency token is received in the memory dependency token queue for that load, the address (e.g., and completion buffer slot) can be dispatched to the memory system by the scheduler (e.g., as memory command 3542B). In one embodiment, the address of the instance of the load is received in input queue 3522B, and memory dependency tokens for allowing the issuance of those loads are received in memory dependency token input queue 3518B. Assuming that a particular load does not specify dependencies, the scheduler can dispatch the address (and completion buffer slot) to the memory system (e.g., as memory command 3542B). When the result returns to mux 3540B (illustratively shown), the result can be stored in the completion buffer slot it is designated for (e.g., since the result carries the target slot throughout its journey through the memory system). The completion buffer can send the results back to the local network (e.g., local networks 3502, 3504, 3506, or 3508) in the order the addresses arrived. Storage can be similar, except that both the address and the data may have to arrive before any operation is dispatched to the memory system. As another example of storage (e.g., a storage request), an address arrives at queue 3524B, and the payload data to be stored at that address arrives at queue 3526B (e.g., at a different time than the address arriving at queue 3524B), the scheduler 3512B matches that address with a storage operation in register 3510B. When a memory dependency token is generated, the completion buffer slot for that memory can be allocated in the order the addresses arrive. Assuming that a particular memory specifies (e.g., in the "Memory Operation" stored in register 3510B) a dependency, the dispatch of the address and payload data (e.g., and the completion buffer slot) to the memory system by the scheduler (e.g., as memory command 3542B) can be stopped until the corresponding memory dependency token arrives, for example, in memory dependency token queue 3520B. Once a memory dependency token is received in the memory dependency token queue for that memory, the scheduler can dispatch the address and payload data (e.g., and the completion buffer slot) to the memory system (e.g., as memory command 3542B).In one embodiment, the addresses of the stored instances are received in input queue 3524A, the payload data of those stored instances is received in input queue 3526A, and memory dependency tokens for allowing the publication of those stores are received in memory dependency token input queue 3520B. Assuming that a particular store does not specify a dependency, the scheduler can dispatch the address and payload data (e.g., and completion buffer slot) to the memory system (e.g., as memory command 3542B). When the result returns to mux 3540B (illustratively shown), the result can be stored in the completion buffer slot it is designated to (e.g., since the result carries the target slot throughout its journey through the memory system). The completion buffer can send the results back to the local network (e.g., local networks 3502, 3504, 3506, or 3508) in the order the addresses arrived.

[0317] In some embodiments, a first access request is received (e.g., programmed to receive) in one or more input queues of a first RAF circuit 3500A (e.g., input queues 3522A, 3524A, or 3526A) (e.g., statically configured to always be received in the same RAF circuit used for the program), and a second access request is received (e.g., programmed to receive) in one or more input queues of a second RAF circuit 3500B (e.g., input queues 3522B, 3524B, or 3526B) (e.g., statically configured to always be received in the same RAF circuit used for the program), and the first access request will occur before the second access request, for example, the first access request is in the program sequence before the second access request. Therefore, the second access request can be flagged (e.g., in register 3510B and / or scheduler 3512B) to require the reception of a memory-dependent token caused by the first access request (e.g., in queue 3518B or queue 3520B) before the second access request is issued. When a first access request and a second access request (e.g., always for a program) that includes a dependency on the first access request (e.g., program order) are received in the RAF circuitry on the same accelerator (e.g., accelerator chip) (e.g., coupled to memory together by the same network), the RAF circuitry 3500B can utilize chip-level memory coherence. The first access request can be: (i) the address of a load request received in input queue 3522A (e.g., sent by the first PE) or (ii) the address of a storage request received in input queue 3524A (e.g., sent by the second PE) and payload data for the address of the storage request received in input queue 3526A (e.g., sent by the second or third PE). In this example, a second memory access request (e.g., sent by a third or fourth PE) will be received in the second RAF circuit 3500B (e.g., programmed to be received), and the second memory access request is marked (e.g., in its memory operation field in register 3510B) to request the receipt of a memory dependency token (e.g., in queue 3518B or queue 3520B) resulting from the arrival or fulfillment of a desired milestone by the first access request, for example, to release the stop on issuing the second access request, for example, assuming that the address and any payload data are ready for the second access request. In one embodiment of chip memory coherence, the desired milestone is a predetermined period of time that has elapsed or been fulfilled since the first memory access was initiated, or other such manner as discussed herein.In some embodiments, instead of issuing a second memory access in a single cycle after issuing a first access request or waiting for the first access request to the memory to reach the cache and / or the memory access to the first access request is completed (e.g., and the second RAF circuit receives an acknowledgment), the RAF circuit in the same accelerator (e.g., the same chip) will allow the first memory access request to be issued (e.g., as memory command 3542A) to the memory (e.g., as memory command 3542A enters). Figure 34 The first RAF circuit 3500A (e.g., scheduler 3512A) issues a second memory access request (e.g., by sending a memory dependency token from RAF circuit 3500A to RAF circuit 3500B) within a predetermined time period (e.g., three or more cycles) after the first memory access request is issued (e.g., by sending a memory dependency token from RAF circuit 3500A to RAF circuit 3500B). In one embodiment, the first RAF circuit 3500A (e.g., scheduler 3512A) is used to send the memory dependency token to a slot (e.g., first slot 3518B(0)) in the memory dependency token queue 3518B of the second RAF circuit 3500B for the second memory access request, for example, within a predetermined time period after the first access request is scheduled or issued to the network (e.g., as a memory command 3542A) to access the address for the access request. The scheduler 3512A includes a timer 3511 for tracking and counting down the time after the first memory access request is issued (e.g., as a memory command 3542A) to the memory (e.g., as a memory command 3542A enters the memory). Figure 34 The predetermined time period (e.g., three, four, five or more cycles, e.g., fewer cycles than acquiring ownership of the cache line to be accessed) after network 3404). In one embodiment, timer 3511 is set (e.g., before runtime) to have a maximum time, and memory command 3542A spends that maximum time traversing from RAF circuit 3500A to the memory to be accessed according to the memory command 3542A (e.g., network 3404). Figure 34 MR1 in the RAF circuit (1) traverses to the cache block (e.g., Figure 34The longest time spent in the cache block (3) in the memory (e.g., the travel time using a known latency architecture). Timer 3511 can be set to start counting time (e.g., controlling the periodic clock period of the RAF circuit) after memory command 3542A is issued to memory, and then issue a signal causing scheduler 3512A to send a memory dependency token to the second RAF circuit 3500B. At this time, the memory dependency token can be sent to the memory dependency token output queue 3516A or to a network that couples the output queue 3516A to the memory dependency token input queue (e.g., 3520A or 3520B) of the RAF circuit 3500B. Once the RAF circuit 3500B receives the memory dependency token for the second access request, the second access request (e.g., assuming address and payload data are ready for the second access request) can be scheduled or published to the network as a memory command 3542B, for example, based on the presence of the memory dependency token for the second memory access request in a slot (e.g., read by scheduler 3512B) in the memory dependency token queue 3518B of the second RAF circuit 3500B (e.g., the first slot 3518B(0)). In one embodiment, the memory dependency token is provided (e.g., stored) into the memory dependency token queue 3518B by the memory dependency token traveling on the network 3504. In some embodiments, the network that couples the RAF circuit to the memory is Figure 34 The network 3404 (e.g., a fixed latency network) ensures that a first time-based access request cannot pass through a second time-based access request in memory, for example, to maintain program order for the first memory command 3542A and the second memory command 3542B. In some embodiments, the first and second memory accesses are thus issued to memory in program order and guaranteed to be served in program order, for example, when the memory system maintains them in the order they are received in the memory system.

[0318] Therefore, in slice-level memory coherence mode, the memory dependency token generated for an access request can be sent from the first RAF circuit 3500A to the second RAF circuit 3500B without waiting for the memory access of the first access request to complete (e.g., or waiting for confirmation that the access request has been completed). In one embodiment, the network (e.g., interconnect) between the RAF circuit and the memory (e.g., cache block) is implemented as a fixed latency crossover, for example, a crossover switch with multiple switching devices to couple any one of the RAF circuits to any one of the cache blocks. As an example, initially, one of the multiple RAF circuits in a single accelerator (e.g., a single accelerator chip) issues an access to memory. When in slice-level memory coherence mode, a dependency token is issued once the access is accepted into the target cache block (e.g., instead of spending additional time waiting for memory confirmation of the access execution before the RAF circuit sends the dependency token). The fixed latency property of the interconnect can be utilized to allow the RAF circuit to directly generate dependency tokens for logically related operations.

[0319] In one embodiment, the memory-dependent token is a single bit, and the address and payload data are multiple bits (e.g., 32 or 64 bits). In one embodiment, the memory-dependent token input queues (3518A, 3518B, 3520A, 3520B) are backpush back, causing the network not to send memory-dependent tokens to one or more queues (3518A, 3518B, 3520A, 3520B), for example, when the queue is full. Although in Figure 35 Each RAF circuit depicts two memory-dependent token input queues, but a single memory-dependent token input queue or three or more memory-dependent token input queues can be utilized. In some embodiments, each queue will receive multiple instances of the same operation, e.g., but not other operations. In some embodiments, slice memory consistency is maintained even when the addresses accessed by the first and second access requests (e.g., physical addresses) may be the same address. In one embodiment, the RAF circuit includes a TLB for providing the output of the physical address of the input for the virtual address of the memory access request in the order in which the input is received (e.g., in program order). The RAF circuit may include a TLB, e.g., referenced to Figure 30 The TLB under discussion.

[0320] Figure 36The diagram illustrates a flowchart 3600 according to an embodiment of the present disclosure. Flow 3600 includes: coupling a plurality of request address file circuits to a spatial array of processing elements via a first communication network, and coupling memory to the plurality of request address file circuits via a second communication network (3602); receiving a second request address file circuit from the spatial array of the processing elements, which is marked with a program sequence dependency of a first access request to memory (3604); stopping the second request address file circuit from publishing the second access request to the second communication network (3606); publishing the first access request to memory to the second communication network by the first request address file circuit (3608); sending a first memory dependency token by the first request address file circuit (3610) when a predetermined time period has elapsed since the first request address file circuit published the first access request to memory to the second communication network; and publishing the second access request to the second communication network based on the first memory dependency token received by the second request address file circuit (3612).

[0321] In one embodiment, a dependency token is generated but not consumed for use in conditional execution, such as a dependency load or store operation where a dependency token generated by an unselected conditional path is ignored (e.g., deleted, for example, by a PE).

[0322] Global horizontal memory consistency hardware

[0323] In some embodiments, memory operations reside at the RAF circuitry. These memory operations may be driven by the PE, which generates data and addresses. The RAF circuitry in the depicted embodiments controls memory access to memory (e.g., issuing memory access requests). In one embodiment, memory ordering is relaxed and the RAF circuitry (e.g., the RAF) is typically decoupled from each other unless necessary to maintain program correctness. Situations requiring additional RAF-to-RAF communication may include executing program ordering, where memory accesses derived from sequential programming languages ​​must be serialized, for example, due to the possible existence of aliased addresses, and where certain ordering is applied to the memory system to implement entity-to-entity communication protocols. Some embodiments herein provide hardware for ordering requests (e.g., memory access requests) in a system, including RAF circuitry accessing memory (e.g., a cache) and other components (e.g., a processor or processor core) accessing that memory (e.g., a cache), such as a memory management unit (MMU). In one embodiment, the memory access request originates from one or more PEs coupled to the RAF circuitry.

[0324] Figure 37 The illustration shows multiple Request Address File (RAF) circuits (1)-(8) coupled between an accelerator 3706 (formed by multiple accelerator chips 3708, 3709, 3710, and 3711) and multiple cache blocks (1)-(8) according to an embodiment of the present disclosure. In this embodiment, other components (e.g., a processor or processor core 3703) also access the memory (e.g., cache), for example, via a memory management unit (MMU) 3701 or through a CHA (e.g., as in the embodiments herein). Core 3703 may be a processor core as discussed herein, for example, having a decoder for decoding instructions and an execution unit for executing the decoded instructions, for example, separate from the operation of accelerator 3706. Memory management unit (MMU) 3701 may maintain the memory (e.g., cache blocks (1)-(8)) according to a coherence protocol. For example, management unit (MMU) 3701 may ensure, according to a coherence protocol, that any changes to memory locations (e.g., physical addresses) are globally visible. In one embodiment, the memory management unit (MMU) 3701 is used to ensure ownership of data items (e.g., cache lines) for changes to cache blocks (1)-(8) and / or to make any changes globally visible (e.g., committed to a consensus protocol) before sending a signal indicating that the changes are globally visible. In one embodiment, the memory management unit (MMU) 3701 is used to send a signal indicating that the changes are globally visible to the RAF circuitry on path 3715 (e.g., network 3704 or its own (e.g., response) network). The in-structure storage and communication circuitry depicted in accelerator 3706 are each optional. The RAF circuitry may be as disclosed herein, e.g., as... Figure 38 The RAF circuit depicted is coupled to the accelerator 3706 via a communication network 3702. The communication network 3702 can be a circuit-switched network, for example, as disclosed in [the original text]. Figure 6-11 As shown in the diagram. Alternatively or additionally, the communication network 3702 may be a packet-switched network, for example, such as... Figure 12-29 As shown in the diagram. The communication network 3702 can carry data between PEs and / or between PEs and RAF circuits. In one embodiment, the accelerator 3706 (e.g., its PE) sends a memory access request to the RAF circuit. The memory access request may include an address (e.g., for a load operation or a store operation) and payload data (e.g., stored for a store operation). The requesting entity may be a PE, a network data stream endpoint circuit, or other hardware component.

[0325] The depicted RAF circuitry is coupled to a memory (e.g., multiple blocks depicted as a cache) via a second communication network 3704. The second communication network 3704 can couple any one of the RAF circuitry to any one of the cache blocks. The second communication network 3704 can be a fixed latency network. The second communication network 3704 can be a circuit-switched network, for example, as... Figure 6-11 As shown in the diagram. The second communication network 3704 can be a fixed latency crossover, for example, a crossover switch with multiple switching devices to couple any one of the RAF circuits to any one of the cache blocks. The number of RAF circuits and the number of cache blocks can be the same (e.g., as shown in the diagram). Figure 37 (Eight as shown) or different quantities.

[0326] Each RAF circuit may include a scheduler (e.g., an operation manager circuit) to control memory access to memory (e.g., issuing memory access requests). The RAF circuit may include one or more of the following components and / or features. The depicted RAF circuit (1) includes a scheduler 3712A, a memory-dependent token queue 3718A, and an input queue 3722A. The depicted RAF circuit (5) includes a scheduler 3712B, a memory-dependent token queue 3718B, and an input queue 3722B. Each RAF circuit may include one or any number of memory-dependent token queues. Each RAF circuit may include one or any number of input queues and / or output queues. In some embodiments, schedulers (e.g., schedulers 3712A and 3712B) monitor data sent to and / or sent out of their queues. For example, scheduler 3712A monitors input queue 3722A and memory-dependent token queue 3718A, and / or scheduler 3712B monitors input queue 3722B and memory-dependent token queue 3718B. In one embodiment, queues (e.g., input queues 3722A and 3722B) are coupled (e.g., via network 3702) to requesting entities (e.g., one or more PEs of accelerator 3706), for example, each RAF circuit is coupled to a separate PE. Multiple memory access requests (e.g., from different PEs) can be sent (e.g., programmed to be sent, for example, via a circuit-switched network) to RAF circuits, such as their input queues. Two access requests, referred to as a first access request and a second access request, may be sent. For example, the first access request may be sent to RAF circuit (1) (e.g., input queue 3722A), and the second access request may be sent to RAF circuit (5) (e.g., input queue 3722B). It may be desirable to ensure (e.g., require) that the first access request occurs before the second access request, for example, when the first access request occurs in program order before the second access request. In one embodiment, request 1 is a load or store, and request 2 is a load or store, for example, to the same address. In some embodiments, RAF circuit (5) (e.g., scheduler 3712B) is configured (e.g., programmed to) check whether the corresponding memory dependency token of the first access request (e.g., in its memory dependency token queue 3718B) has been received before the second access request is published (e.g., publishing the request to memory (e.g., cache block) or coupled (e.g., directly) to the network 3704 of memory). In one embodiment, the second access request is stored in the input queue 3722B until the memory dependency token queue 3718B receives a memory dependency token indicating that the second access request can be initiated (e.g., issued).In some embodiments, a memory dependency token for a second access request is generated (e.g., sent and / or stored) because the first memory access has reached or achieved a desired milestone (e.g., the memory access for the first access request is initiated or completed, or in other ways as discussed herein). The depicted RAF circuitry publishes access requests to memory (e.g., cache blocks) via network 3704. In one embodiment, the memory access request includes an address (e.g., physical) for each access request, and network 3704 directs the access requests (e.g., load requests or store requests with storage payloads) to the appropriate cache block. In one embodiment, multiple access requests are directed to the same cache block (e.g., the same address within the same cache block). Figure 37 The circled numbers may refer to sequential time steps, for example, but in some embodiments they may not be linear time steps.

[0327] In some embodiments, a first access request is received in a first RAF circuit, and a second access request is received in a second RAF circuit (e.g., statically configured to always be received in the same RAF circuits used for the program), and the first access request occurs before the second access request, for example, the first access request is in program order before the second access request. Therefore, the second access request can be flagged (e.g., in the scheduler) to request that a memory dependency token caused by the first access request be received before the second access request is issued. In one embodiment, the first RAF circuit will wait until the first access request reaches the cache and becomes globally visible (e.g., by MMU 3701) and / or the memory access of the first access request is completed and made globally visible (e.g., by MMU 3701) before the first RAF circuit sends the memory dependency token to the second RAF circuit that will issue (e.g., cause the issuance) a second access request that includes the (e.g., program order) dependency of the first access request on the memory. (e.g., and the first RAF circuit receives an acknowledgment of global visibility.)

[0328] In some embodiments, a first access request and a second access request are received (e.g., programmed to receive) in the system's RAF circuitry (e.g., statically configured to always be received in the same RAF circuitry used for programming), and the first access request occurs and / or becomes globally visible before the second access request (e.g., before the second access request is published to memory), for example, the first access request precedes the second access request in program order. Therefore, the first access request can be flagged (e.g., in the scheduler) to require receiving a signal indicating that the access is globally visible in memory (e.g., a cache block) before the memory dependency token causing the requested access (e.g., storage) is sent to other RAF circuitry. The second access request can be flagged (e.g., in the scheduler) to require receiving a memory dependency token caused by the first access request before the second access request is published. When managing (e.g., by MMU 3701) the first access request and the second access request, which includes a dependency on the first access request for memory (e.g., program order), for consistency purposes, the RAF circuitry can use global-level memory consistency.

[0329] exist Figure 37In the illustrated embodiment, RAF circuit (1) receives a first memory access request (shown as MR1), and RAF circuit (5) receives a second memory access request (shown as MR2). In one embodiment, the first memory access request may be received in a first slot 3722A(0) in input queue 3722A of RAF circuit (1), and the second memory access request may be received in a second slot 3722B(0) in input queue 3722B of RAF circuit (5), for example, at some time before time 1 (e.g., marked by circle 1). The second memory access request may include a corresponding slot in memory dependency token queue 3718B (e.g., first slot 3718B(0)). In this example, the issuance of the second memory access request is stopped until a memory dependency token is received in a slot (e.g., second slot 3718B(0)) in memory dependency token queue 3718B of RAF circuit (5). In one embodiment, the memory dependency token for the second memory access request is generated because the first memory access has reached the memory to be accessed (e.g., cache block) and the access is globally visible in the memory (e.g., cache block) before the memory dependency token causing the requested access (e.g., storage) is sent to other RAF circuits (e.g., and received as an acknowledgment by the first RAF circuit). In one embodiment, a signal indicating that the requested access (e.g., storage) is globally visible in the memory (e.g., cache block) is sent from cache block (3) and / or MMU 3701 to RAF circuit (1) on path 3715.

[0330] In one embodiment, the RAF circuit (1) (e.g., scheduler 3712A) provides the memory dependency token in a slot (e.g., first slot 3718B(0)) of the memory dependency token queue 3718B in the RAF circuit (5) for a second memory access request, for example, when the RAF circuit (1) receives a signal indicating that the requested access RF1 (e.g., storage) is globally visible in memory. The second access request can then be scheduled or published to the network 3704, for example, based on the presence (e.g., read by scheduler 3712B) of the memory dependency token in the slot (e.g., first slot 3718B(0)) of the memory dependency token queue 3718B in the RAF circuit (5) for the second memory access request. In one embodiment, the memory dependency token is provided (e.g., stored) in the memory dependency token queue 3718B by the memory dependency token traveling on its own communication path 3717 (illustrated) or on the network 3702. In some embodiments, network 3704 is a fixed wait time network, such that a first access request in time cannot pass through a second access request in time in memory, for example, to maintain program order in between.

[0331] A first memory access request (shown as MR1) is received (e.g., programmed to receive) in RAF circuit (1), and a second memory access request (shown as MR2) is received (e.g., programmed to receive) in RAF circuit (5) (e.g., statically configured to always be received in those same RAF circuits used for the program), and the first access request will occur before the second access request, for example, the first access request is in program order before the second access request. Therefore, the first access request can be flagged (e.g., in the first scheduler 3712A) to require receiving a signal indicating that the access is globally visible in memory (e.g., a cache block) before the memory dependency token causing the requested access (e.g., storage) is sent to RAF circuit (5). Therefore, the second access request can be flagged (e.g., in scheduler 3712B) to require receiving a memory dependency token caused by the first access request before the second access request is issued. When a first access request is made to memory managed for consistency (e.g., by MMU3701) and a second access request includes a dependency on the first access request (e.g., program order), the RAF circuit (1) (e.g., and RAF circuit (5)) can use global horizontal memory consistency. The depicted RAF circuit (1) publishes the first memory access request (shown as MR1) to network 3704 at time 1 (e.g., marked with circle 1). In global horizontal memory consistency mode, the first memory access request (shown as MR1) is marked (e.g., in scheduler 3712A) to request a signal indicating that the access is globally visible in memory (e.g., cache block) before the memory dependency token causing the requested access (e.g., storage) is sent to RAF circuit (5). At time 2 (e.g., marked with circle 2), the first memory access request has traveled to the point in network 3704 marked with circle 2. At time 3 (e.g., marked with circle 3), the first memory access request has arrived at the cache block (e.g., cache block (3)), such that the memory access requested by the first memory access request will be performed, for example, the memory access request will be accepted in the cache block (e.g., winning access) and / or it will be queued for service. At time 4 (e.g., marked with circle 4), the memory access of the first memory access request is performed and made globally visible (e.g., by MMU 3701). At time 5 (e.g., marked with circle 5), a signal indicating that the requested access (e.g., storage) is globally visible in memory (e.g., cache block (3)) is received at the RAF circuit (1). The RAF circuit (1) then sends a memory dependency token generated for the first access request that causes the access to be sent to the RAF circuit (5).At time 6, the depicted RAF circuit (5) has received a memory dependency token from the first memory access request (shown as MR1), and then at time 6 (e.g., marked by circle 6), publishes the second memory access request (shown as MR2) to network 3404. In another embodiment, the first and second access requests will access the same address used for the access requests (e.g., in the same cache block), and the aforementioned timing prevents the second access request from being executed (e.g., started) before the first access request is executed and globally visible (e.g., completed) to maintain memory consistency. In one embodiment, each cache block is accessed in the order in which the accesses are received (e.g., at the input of the cache block). In some embodiments, this is achieved by sorting in network 3704 (e.g., ACI) and / or cache blocks. Figure 37 Maintain consistency in the computational system of the circuit.

[0332] In one embodiment, the first memory access request does not require (e.g., receives) a memory dependency token before being published to memory (e.g., a cache block). In another embodiment, the first memory access request requires (e.g., receives) a memory dependency token before being published to memory (e.g., a cache block), for example, in the first slot 3718A(0) of the memory dependency token queue 3718A. Figure 37 The RAF circuit in the document can be as discussed herein (for example, refer to the following). Figure 38 ).

[0333] In one embodiment, the address received by the RAF circuit (e.g., from other circuitry in the PE or CSA) is a virtual address, and a physical address is then determined and used to access memory at that physical address, for example, by using... Figure 30 The TLB in this document is added to any RAF circuit.

[0334] Figure 38 The illustration shows a first Request Address File (RAF) circuit 3800A coupled to a second Request Address File (RAF) circuit 3800B according to an embodiment of the present disclosure. In the depicted embodiment, a network (e.g., its channels) couples the first RAF circuit 3800A to the second RAF circuit 3800B. The network (e.g., one or more of networks (3802, 3804, 3806, 3808)) can be a circuit-switched network, such as... Figure 6-11 As shown, it could be a packet-switched network, for example, such as Figure 12-29As shown in the diagram. In one embodiment, one of the networks (3802, 3804, 3806, 3808) couples one or more first request entities (e.g., PE, network data stream endpoint circuitry, or other hardware components) to the first RAF circuit 3800A, and another of the networks (3802, 3804, 3806, 3808) couples one or more second request entities (e.g., PE, network data stream endpoint circuitry, or other hardware components) to the first RAF circuit 3800A and to the second RAF circuit 3800B. In one embodiment, the request entity is... Figure 37 The components of the accelerator 3706, for example, wherein the RAF circuit (1) is a first RAF circuit 3800A and the RAF circuit (2) is a second RAF circuit 3800B. The first RAF circuit 3800A and / or the second RAF circuit 3800B may be coupled (e.g., to) a circuit-switched network (e.g., such as... Figure 6-11 One or more of the networks (3802, 3804, 3806, 3808) shown receive input (e.g., including address values) of an access request from one or more requesting entities (e.g., PE or other hardware components). The first RAF circuit 3800A and / or the second RAF circuit 3800B can be coupled (e.g., to) a packet-switched network (e.g., such as...). Figure 12-29 One or more of the networks (3802, 3804, 3806, 3808) shown in the diagram receive input (e.g., including address values) of an access request from one or more requesting entities (e.g., PE, network data stream endpoint circuits, or other hardware components).

[0335] In one embodiment, dependency tokens are generated but not consumed by conditional execution, where examples of dependency load or store operations have dependency tokens generated for unselected conditional paths that are ignored (e.g., deleted, for example, by PE).

[0336] Other status signals (e.g., in) Figure 10 , 30 (in 32, 35, 38) can refer to the state of the input queue, the completion queue, the dependency queue, or any combination thereof (e.g., empty or full).

[0337] In some embodiments, the data received for a memory (e.g., cache) access request is a memory command. The memory command may include the address to be accessed (e.g., physical), the operation to be performed (e.g., load or store), and / or payload data (e.g., for storage).

[0338] In one embodiment, for example, during configuration, memory (e.g., load and / or store) operations (e.g., in the data flow graph) are specified (e.g., stored) in register 3810A for the first RAF circuit 3800A and in register 3810B for the second RAF circuit 3800B, for example, and are sent via one or more of networks 3802, 3804, 3806, or 3808. Subsequently, arcs to those memory operations in the data flow graph may be connected to input queues 3822A, 3824A, and 3826A for the first RAF circuit 3800A and input queues 3822B, 3824B, and 3826B for the second RAF circuit 3800B. Arcs from those memory operations are thus used to exit from completion buffers 3828A, 3830A, or 3832A for the first RAF circuit 3800A and completion buffers 3828B, 3830B, or 3832B for the second RAF circuit 3800B. Memory-dependent tokens (which may each be a single bit) arrive at queues 3818A and 3820A for the first RAF circuit 3800A and queues 3818B and 3820B for the second RAF circuit 3800B. Memory-dependent tokens will then exit from queue 3816A for the first RAF circuit 3800A and queue 3816B for the second RAF circuit 3800B into the network, for example, in on-chip horizontal consistency mode or global horizontal consistency mode. In the depicted embodiments, memory-dependent tokens may be generated by a first RAF circuit 3800A and sent from a memory-dependent token queue 3816A to a memory-dependent token queue (e.g., 3818B or 3820B) of a second RAF circuit 3800B on one or more of the networks (3802, 3804, 3806, 3808). In one embodiment, network 3804 couples (e.g., is programmed to couple) the memory-dependent token output queue 3816A of the first RAF circuit 3800A to the memory-dependent token input queue 3818B of the second RAF circuit 3800B, for example, by providing such a connection by providing switching devices 3805 and 3807 of a circuit-switched network.

[0339] Dependency token counters (3814A and 3814B) can be compact representations of queues in the corresponding RAF circuits and can track the number of dependency tokens used for any given input queue. In some embodiments, if the dependency token counter saturates, no additional dependency tokens can be generated for new memory operations. Accordingly, the memory sorting circuit (e.g., Figure 38 The RAF in the memory will stop scheduling new memory operations until the dependency token counter becomes unsaturated.

[0340] As an example of a load (e.g., a load request), an address arrives at queue 3822A, and scheduler 3812A matches that address with a load operation in register 3810A. The completion buffer slot in the RAF circuit 3800A for this load can be assigned in the order the addresses arrive. Assuming that a particular load specifies (e.g., in a “Load Memory Operation” stored in register 3810A) dependencies, dispatching addresses (e.g., and completion buffer slots) to the memory system (e.g., as memory commands 3842A) by the scheduler can be stopped until the corresponding memory dependency token arrives, for example, in memory dependency token queue 3818A. Once a memory dependency token is received in the memory dependency token queue for that load, the address (e.g., and completion buffer slot) can be dispatched to the memory system (e.g., as memory commands 3842A) by the scheduler. In one embodiment, the address of the instance of the load is received in input queue 3822A, and memory dependency tokens for allowing the issuance of those loads are received in memory dependency token input queue 3818A. Assuming that a particular load does not specify dependencies, the scheduler can dispatch the address (and completion buffer slot) to the memory system (e.g., as memory command 3842A). When the result returns to mux 3840A (illustratively shown), the result can be stored in the completion buffer slot it is designated for (e.g., since the result carries the target slot throughout its journey through the memory system). The completion buffer can send the results back to the local network (e.g., local networks 3802, 3804, 3806, or 3808) in the order the addresses arrived. Storage can be similar, except that both the address and the data may have to arrive before any operation is dispatched to the memory system. As an example of storage (e.g., a storage request), an address arrives at queue 3824A, and the payload data to be stored at that address arrives at queue 3826A (e.g., at a different time than the address arriving at queue 3824A), the scheduler 3812A matches that address with a storage operation in register 3810A. When a memory dependency token is generated, the completion buffer slot for that memory can be allocated in the order the addresses arrive. Assuming that a particular memory specifies (e.g., in the "Memory Operation" stored in register 3810A) a dependency, the dispatch of the address and payload data (e.g., and the completion buffer slot) to the memory system by the scheduler (e.g., as memory command 3842A) can be stopped until the corresponding memory dependency token arrives, for example, in memory dependency token queue 3820A. Once a memory dependency token is received in the memory dependency token queue for that memory, the scheduler can dispatch the address and payload data (e.g., and the completion buffer slot) to the memory system (e.g., as memory command 3842A).In one embodiment, the addresses of the storage instances are received in input queue 3824A, the payload data of those storage instances is received in input queue 3826A, and memory dependency tokens for allowing the publication of those storage instances are received in memory dependency token input queue 3820A. Assuming that a particular storage instance does not have a specified dependency, the scheduler can dispatch the address and payload data (e.g., and completion buffer slot) to the memory system (e.g., as memory command 3842A). When the result returns to mux 3840A (illustratively shown), the result can be stored in its designated completion buffer slot (e.g., since the result carries the target slot throughout its journey through the memory system). The completion buffer can then send the results back to the local network (e.g., local networks 3802, 3804, 3806, or 3808) in the order the addresses arrived.

[0341] As another example of a load (e.g., a load request), an address arrives at queue 3822B, and scheduler 3812B matches that address with a load operation in register 3810B. The completion buffer slot in the RAF circuit 3800B for this load can be assigned in the order the addresses arrive. Assuming that a particular load specifies (e.g., in a “Load Memory Operation” stored in register 3810B) dependencies, dispatching addresses (e.g., and completion buffer slots) to the memory system by the scheduler can be stopped (e.g., as memory command 3842B) until the corresponding memory dependency token arrives, for example, in memory dependency token queue 3818B. Once a memory dependency token is received in the memory dependency token queue for that load, the address (e.g., and completion buffer slot) can be dispatched to the memory system by the scheduler (e.g., as memory command 3842B). In one embodiment, the address of the instance of the load is received in input queue 3822B, and memory dependency tokens for allowing the issuance of those loads are received in memory dependency token input queue 3818B. Assuming that a particular load does not specify dependencies, the scheduler can dispatch the address (and completion buffer slot) to the memory system (e.g., as memory command 3842B). When the result returns to mux 3840B (illustratively shown), the result can be stored in the completion buffer slot it is designated for (e.g., since the result carries the target slot throughout its journey through the memory system). The completion buffer can send the results back to the local network (e.g., local networks 3802, 3804, 3806, or 3808) in the order the addresses arrive. Storage can be similar, except that both the address and the data may have to arrive before any operation is dispatched to the memory system. As another example of storage (e.g., a storage request), an address arrives at queue 3824B, and the payload data to be stored at that address arrives at queue 3826B (e.g., at a different time than the address arriving at queue 3824B), the scheduler 3812B matches that address with a storage operation in register 3810B. When a memory dependency token is generated, the completion buffer slot for that memory can be allocated in the order the addresses arrive. Assuming that a particular memory specifies (e.g., in the "Memory Operation" stored in register 3810B) a dependency, the dispatch of the address and payload data (e.g., and the completion buffer slot) to the memory system by the scheduler (e.g., as memory command 3842B) can be stopped until the corresponding memory dependency token arrives, for example, in memory dependency token queue 3820B. Once a memory dependency token is received in the memory dependency token queue for that memory, the scheduler can dispatch the address and payload data (e.g., and the completion buffer slot) to the memory system (e.g., as memory command 3842B).In one embodiment, the address of the storage instance is received in input queue 3824B, the payload data of those storage instances is received in input queue 3826B, and a memory dependency token for allowing the publication of those storage instances is received in memory dependency token input queue 3820B. Assuming that a particular storage instance does not have a specified dependency, the scheduler can dispatch the address and payload data (e.g., and completion buffer slot) to the memory system (e.g., as memory command 3842B). When the result returns to mux 3840B (illustratively shown), the result can be stored in its designated completion buffer slot (e.g., since the result carries the target slot throughout its journey through the memory system). The completion buffer can then send the result back to the local network (e.g., local networks 3802, 3804, 3806, or 3808) in the order the addresses arrived.

[0342] In some embodiments, a first access request is received (e.g., programmed to receive) in one or more input queues of a first RAF circuit 3800A (e.g., input queues 3822A, 3824A, or 3826A) (e.g., statically configured to always be received in the same RAF circuit used for the program), and a second access request is received (e.g., programmed to receive) in one or more input queues of a second RAF circuit 3800B (e.g., input queues 3822B, 3824B, or 3826B) (e.g., statically configured to always be received in the same RAF circuit used for the program), and the first access request occurs and / or is globally visible before the second access request (e.g., before the second access request is published to memory), for example, the first access request precedes the second access request in the program sequence. Therefore, a first access request can be flagged (e.g., in scheduler 3812A) to require a signal (e.g., from MMU 3801) indicating that the access is globally visible in memory (e.g., a cache block) before the memory dependency token causing the requested access (e.g., storage) is sent to the second RAF circuit 3800B. The signal indicating global visibility (e.g., an MMU signal) can be coupled through input port 3841A (e.g., to...). Figure 37The path 3715 in the MUX 3840A is sent directly to the scheduler 3812A, or it can be traversed to the scheduler 3812A via the MUX 3840A. Therefore, a second access request can be flagged (e.g., in register 3810B and / or scheduler 3812B) to request that a memory dependency token caused by the first access request be received (e.g., in queue 3818B or queue 3820B) before the second access request is issued. When the first access request is directed to memory managed for consistency (e.g., by MMU 3701) and the second access request includes a dependency on the first access request (e.g., program order), the first RAF circuit 3800A (e.g., and the second RAF circuit 3800B) can use global horizontal memory consistency.

[0343] The first access request may be: (i) the address of a load request received in input queue 3822A (e.g., sent by a first PE) or (ii) the address of a storage request received in input queue 3824A (e.g., sent by a second PE) and payload data for the address of the storage request received in input queue 3826A (e.g., sent by a second or third PE). In this example, a second memory access request (e.g., sent by a third or fourth PE) (e.g., programmed to) be received in the second RAF circuit 3800B, and the second memory access request is marked (e.g., in its memory operation field in register 3810B) to request the reception of a memory-dependent token (e.g., in queue 3818B or queue 3820B) caused by receiving a signal indicating that the requested access (e.g., storage) in memory is globally visible. In one embodiment, a first RAF circuit 3800A (e.g., scheduler 3812A) sends the memory dependency token to a slot (e.g., first slot 3818B(0)) in a memory dependency token queue 3818B of a second RAF circuit 3800B for a second memory access request, for example, after a first access request, issued to the network as a memory command 3842A, has accessed the address for the access request and the requested access (e.g., store or load) is globally visible in memory. In one embodiment, scheduler 3812A includes storage coupled to input port 3841A, including one or more bits set when a signal is detected indicating that the requested access of the access request sent by RAF circuit 3800A is globally visible in memory (e.g., after the first memory access request is issued (e.g., as a memory command 3842A) to memory (e.g., as a memory command 3842A enters memory). Figure 37(After network 3704). In some embodiments, one or more bits are set to cause the scheduler to issue a memory dependency token (e.g., from memory dependency token output queue 3816A). In one embodiment, when a signal indicating that the requested access is globally visible in memory is detected, the memory dependency token can be sent to the memory dependency token output queue 3816A or to the network of the memory dependency token input queue (e.g., 3820A or 3820B) that couples the output queue 3816A to the RAF circuit 3800B. Once the RAF circuit 3800B receives the memory dependency token for the second access request, the second access request (e.g., assuming the address and payload data are ready for the second access request) can be scheduled or issued to the network as a memory command 3842B, for example, based on the presence of the memory dependency token for the second memory access request in a slot (e.g., first slot 3818B(0)) of the memory dependency token queue 3818B of the second RAF circuit 3800B (e.g., read by the scheduler 3812B). In one embodiment, memory dependency tokens are provided (e.g., generated and / or stored) to memory dependency token queue 3818B by memory dependency tokens traveling on network 3804. In some embodiments, the network that couples the RAF circuitry to the memory is Figure 37 The network 3704 (e.g., a fixed latency network) ensures that a first time-based access request cannot pass through a second time-based access request in memory, for example, to maintain program order between the first memory command 3842A and the second memory command 3842B. In some embodiments, the first and second memory accesses are thus issued to memory in program order and guaranteed to be served in program order, for example, when the memory system maintains them in the order they are received in the memory system. Optionally, the scheduler 3812B includes an input port 3841B (e.g., coupled to MMU 3801) and / or storage (e.g., coupled to the input port) including one or more bits set when a signal (e.g., from MMU 3801) is detected, indicating that the requested access of the access request sent by the RAF circuit 3800B is globally visible in memory (e.g., when the second memory access request is issued (e.g., as memory command 3842B) to memory (e.g., as memory command 3842B enters memory). Figure 37 (Network 3704) after.

[0344] Therefore, after the memory access awaiting the first access request is globally visible, the memory dependency token generated for the access request in the global level memory consistency mode can be sent from the first RAF circuit 3800A to the second RAF circuit 3800B. In one embodiment, the network (e.g., interconnect) between the RAF circuit and the memory (e.g., cache block) is implemented with fixed latency crossover, for example, a crossover switch with multiple switching devices to couple any one of the RAF circuits to any one of the cache blocks. As an example, initially, one of the multiple RAF circuits in the system issues an access to memory. When in the global level memory consistency mode, once the access is accepted into the target cache block and the access is made visible at the desired level (e.g., globally), a dependency token is issued.

[0345] In one embodiment, the memory-dependent token is a single bit, and the address and payload data are multiple bits (e.g., 32 or 64 bits). In one embodiment, the memory-dependent token input queues (3818A, 3818B, 3820A, 3820B) are backpush back, causing the network not to send memory-dependent tokens to one or more queues (3818A, 3818B, 3820A, 3820B), for example, when the queue is full. Although in Figure 38 Each RAF circuit depicts two memory-dependent token input queues, but a single memory-dependent token input queue or three or more memory-dependent token input queues may be utilized. In some embodiments, each queue will receive multiple instances of the same operation (e.g., it will receive dependency tokens associated with the same operation), but for example, it will not receive other operations. In some embodiments, global memory consistency is maintained even when the addresses accessed by the first and second access requests may be the same address (e.g., physical addresses). In one embodiment, the RAF circuit includes a TLB for providing the output of the physical address of the input for the virtual address of the memory access request in the order in which the input is received (e.g., in program order). The RAF circuit may include a TLB, for example, referencing Figure 30 The TLB under discussion.

[0346] Figure 39The diagram illustrates a flowchart 3900 according to an embodiment of the present disclosure. Flow 3900 includes: coupling a plurality of request address file circuits to a spatial array of processing elements via a first communication network, and coupling memory to the plurality of request address file circuits via a second communication network (3902); receiving a second request address file circuit from the spatial array of the processing elements, the second request address file circuit marking a program sequence dependency of a first access request to memory (3904); stopping the second request address file circuit from publishing the second access request to the second communication network (3906); and the first request address file circuit receiving the first access request to memory from the spatial array of the processing elements. Publish to the second communication network (3908); couple the memory management unit to the memory and maintain the memory according to the consensus protocol (3910); when the access to the memory of the first access request from the first request address file circuit of the plurality of request address file circuits is completed and the memory management unit makes the access of the first access request globally visible according to the consensus protocol, the first request address file circuit of the plurality of request address file circuits sends a first memory dependency token (3912); and based on the first memory dependency token received by the second request address file circuit, publish the second access request to the second communication network (3914).

[0347] Figure 40 The illustration shows C source code 4000 according to an embodiment of this disclosure. C source code 4000 is an example of fine-grained communication. Here, execution code segment 4002 is used to enable an accelerator (e.g., Figure 37 The accelerator 3706 in the middle) is written to the first memory location, and then (in program order) to the cooperating core (e.g., executed by code segment 4004) Figure 37 The markers observed in nucleus 3703 (e.g., to Figure 37 In the cache block (3), for example, to enable the kernel to read the flag (e.g., in Figure 37 Within the cache block (3) in memory). In some embodiments, execution code 4000 may fail to operate correctly if writes to flag locations in memory and writes to payload locations in memory are not correctly ordered. For reference Figure 37 and Figure 40 One example, Figure 37 The accelerator 3706 in the middle will write pointer(*) payload (e.g., store payload data) to Figure 37 In the cache block (3), it is used as the first access request MR1, and then the pointer (*) flag (e.g., to set the flag for the core) is written to. Figure 37In the cache block. To maintain the program order before the first write and the second write, an instr...

Claims

1. An apparatus comprising: A spatial array of processing elements is coupled by a first communication network; Multiple request address file circuits are coupled to a spatial array of the processing element via the first communication network; as well as The memory, coupled to the plurality of request address file circuits via a second communication network, wherein: The request address file circuit in the plurality of request address file circuits is configured to: not publish access requests to the memory that are marked with program sequence dependencies on previous access requests in the program sequence to the second communication network until a first memory dependency token generated by another request address file circuit in the plurality of request address file circuits in the program sequence is received upon completion of the previous access request to the memory. A single request address file circuit in the plurality of request address file circuits is configured to: provide a second memory dependency token for the program order dependency when publishing the first access request to the second communication network, for a first access request to the memory received by the single request address file circuit and a second access request to the memory received by the single request address file circuit that is marked with a program order dependency on the first access request, and then publish the second access request to the second communication network based on reading the second memory dependency token.

2. The apparatus as claimed in claim 1, characterized in that, The single request address file circuit is used to: generate and store the second memory dependency token in the same cycle of the single request address file circuit, and publish the second access request to the second communication network in the next cycle.

3. The apparatus as described in claim 1, characterized in that, The first access request and the second access request are directed to the same location in the memory.

4. The apparatus as claimed in claim 1, characterized in that, The second request address file circuit in the plurality of request address file circuits is configured to: not issue a fourth access request to the memory that marks a program sequence dependency on the third access request, until a third memory dependency token generated by the first request address file circuit in the plurality of request address file circuits upon completion of the access to the memory of the third access request is received from the second communication network.

5. The apparatus as claimed in claim 1, characterized in that, The second request address file circuit in the plurality of request address file circuits is configured to: not publish a second access request to the memory marked with a program sequence dependency on the first access request to the second communication network until a predetermined time period has elapsed since the first request address file circuit in the plurality of request address file circuits published the first access request to the memory to the second communication network, and receive a third memory dependency token sent by the first request address file circuit.

6. The apparatus as claimed in claim 5, characterized in that, The system further includes a memory management unit coupled to the memory for maintaining the memory according to a consistency protocol, wherein a fourth request address file circuit in the plurality of request address file circuits is configured to: not issue a fourth access request to the memory that marks a program order dependency on the third access request until a fourth memory dependency token generated from the second communication network under the following conditions is received: The completion of the access to the third access request to the memory from the third request address file circuit among the plurality of request address file circuits, and The memory management unit makes the access of the third access request globally visible according to the consistency protocol.

7. A non-transitory machine-readable medium storing code, said code causing the machine to execute a method when executed by the machine, said method comprising: A first communication network couples multiple request address file circuits to a spatial array of processing elements, and a second communication network couples a memory to the multiple request address file circuits. A single request address file circuit in the plurality of request address file circuits receives a first access request to the memory and a second access request to the memory marked with a program sequence dependency on the first access request from the spatial array of the processing elements. The single request address file circuit stops publishing the second access request to the second communication network; When the first access request is published to the second communication network, the single request address file circuit provides a first memory dependency token for the program order dependency; as well as Based on reading the first memory dependency token, the second access request is published to the second communication network by the single request address file circuit.

8. The non-transitory machine-readable medium as described in claim 7, characterized in that, The provisioning and publishing steps include: the single request address file circuit generating and storing the first memory dependency token in the same cycle of the single request address file circuit, and publishing the second access request to the second communication network in the next cycle.

9. The non-transitory machine-readable medium as described in claim 7, characterized in that, The first access request and the second access request are directed to the same location in the memory.

10. The non-transitory machine-readable medium as claimed in claim 7, characterized in that, The method further includes: the second request address file circuit in the plurality of request address file circuits does not issue a fourth access request to the memory that is marked with a program order dependency on the third access request, until it receives a second memory dependency token generated by the first request address file circuit in the plurality of request address file circuits upon completion of the access to the memory of the third access request from the second communication network.

11. The non-transitory machine-readable medium as claimed in claim 7, characterized in that, The method further includes: a second request address file circuit in the plurality of request address file circuits does not publish a second access request to the memory that is marked with a program sequence dependency on the first access request to the second communication network until a predetermined time period has elapsed since the first request address file circuit in the plurality of request address file circuits published the first access request to the memory to the second communication network, and receives a second memory dependency token sent by the first request address file circuit.

12. The non-transitory machine-readable medium as claimed in claim 11, characterized in that, The method further includes: coupling a memory management unit to the memory to maintain the memory according to a consistency protocol, wherein a fourth request address file circuit in the plurality of request address file circuits does not issue a fourth access request to the memory that marks a program order dependency on a third access request until a third memory dependency token generated from the second communication network under the following conditions is received: The completion of the access to the third access request to the memory from the third request address file circuit among the plurality of request address file circuits, and The memory management unit makes the access of the third access request globally visible according to the consistency protocol.

13. An apparatus comprising: A spatial array of processing elements is coupled by a first communication network; Multiple request address file circuits are coupled to a spatial array of the processing element via the first communication network; as well as The memory, coupled to the plurality of request address file circuits via a second communication network, wherein: The request address file circuit in the plurality of request address file circuits is configured to: not publish access requests to the memory that are marked with program sequence dependencies on previous access requests in the program sequence to the second communication network until a first memory dependency token generated by another request address file circuit in the plurality of request address file circuits in the program sequence is received upon completion of the previous access request to the memory. The second request address file circuit in the plurality of request address file circuits is configured to: not publish a second access request to the memory that is marked as program sequence dependency on the first access request to the second communication network until a predetermined time period has elapsed since the first request address file circuit in the plurality of request address file circuits published the first access request to the memory to the second communication network and receives a second memory dependency token sent by the first request address file circuit.

14. The apparatus as claimed in claim 13, characterized in that, The predetermined time period is the maximum time period during which the first access request traverses the first request address file circuit, passes through the second communication network, and is enqueued in the memory.

15. The apparatus as claimed in claim 13, characterized in that, The system further includes a memory management unit coupled to the memory for maintaining the memory according to a consistency protocol, wherein a fourth request address file circuit in the plurality of request address file circuits is configured to: not issue a fourth access request to the memory that marks a program order dependency on the third access request until a third memory dependency token generated from the second communication network under the following conditions is received: The completion of the access to the third access request of the memory by the third request address file circuit in the plurality of request address file circuits, and The memory management unit makes the access of the third access request globally visible according to the consistency protocol.

16. The apparatus as claimed in claim 15, characterized in that, The sixth request address file circuit in the plurality of request address file circuits is configured to: not issue a sixth access request to the memory that is marked with a program order dependency on the fifth access request, until a fourth memory dependency token generated by the fifth request address file circuit in the plurality of request address file circuits upon completion of the access to the memory of the fifth access request is received from the second communication network.

17. The apparatus as claimed in claim 13, characterized in that, The fourth request address file circuit in the plurality of request address file circuits is configured to: not issue a fourth access request to the memory that marks a program order dependency on the third access request, until a third memory dependency token generated by the third request address file circuit in the plurality of request address file circuits upon completion of the access to the memory of the third access request is received from the second communication network.

18. The apparatus as claimed in claim 13, characterized in that, A single request address file circuit in the plurality of request address file circuits is configured to: provide a third memory dependency token for the program order dependency when publishing the third access request to the second communication network, for a third access request to the memory received by the single request address file circuit and a fourth access request to the memory received by the single request address file circuit that is marked with a program order dependency on the third access request, and then publish the fourth access request to the second communication network based on reading the third memory dependency token.

19. A non-transitory machine-readable medium storing code, said code causing the machine to perform a method when executed by a machine, said method comprising: A first communication network couples multiple request address file circuits to a spatial array of processing elements, and a second communication network couples a memory to the multiple request address file circuits. The second request address file circuit in the plurality of request address file circuits receives from the spatial array of the processing element a second access request to the memory that is program-sequence dependent on the first access request to the memory; The second request address file circuit stops publishing the second access request to the second communication network; The first request address file circuit in the plurality of request address file circuits publishes the first access request for the memory to the second communication network; When a predetermined time period has elapsed since the first request address file circuit published the first access request for the memory to the second communication network, the first request address file circuit among the plurality of request address file circuits sends a first memory dependency token. as well as Based on the first memory dependency token received by the second request address file circuit, the second access request is published to the second communication network.

20. The non-transitory machine-readable medium as claimed in claim 19, characterized in that, The predetermined time period is the maximum time period during which the first access request traverses the first request address file circuit, passes through the second communication network, and is enqueued in the memory.

21. The non-transitory machine-readable medium as claimed in claim 19, characterized in that, The method further includes: coupling a memory management unit to the memory to maintain the memory according to a consistency protocol, wherein a fourth request address file circuit in the plurality of request address file circuits does not issue a fourth access request to the memory that marks a program order dependency on the third access request until a second memory dependency token generated from the second communication network under the following conditions is received: The completion of the access to the third access request of the memory by the third request address file circuit in the plurality of request address file circuits, and The memory management unit makes the access of the third access request globally visible according to the consistency protocol.

22. The non-transitory machine-readable medium as claimed in claim 21, characterized in that, The method further includes: the sixth request address file circuit in the plurality of request address file circuits does not issue a sixth access request to the memory that is marked with a program order dependency on the fifth access request, until it receives a third memory dependency token generated by the fifth request address file circuit in the plurality of request address file circuits upon completion of the access to the memory of the fifth access request from the second communication network.

23. The non-transitory machine-readable medium as claimed in claim 19, characterized in that, The method further includes: the fourth request address file circuit in the plurality of request address file circuits does not issue a fourth access request to the memory that is marked with a program order dependency on the third access request, until it receives a second memory dependency token generated by the third request address file circuit in the plurality of request address file circuits upon completion of the access to the memory of the third access request from the second communication network.

24. The non-transitory machine-readable medium as claimed in claim 19, characterized in that, The method further includes: for a third access request to the memory received by the single request address file circuit and a fourth access request to the memory received by the single request address file circuit that is marked with a program order dependency on the third access request, when publishing the third access request to the second communication network, providing a second memory dependency token for the program order dependency, and then publishing the fourth access request to the second communication network based on reading the second memory dependency token.

25. An apparatus comprising: A spatial array of processing elements is coupled by a first communication network; Multiple devices are coupled to a spatial array of the processing elements via the first communication network; as well as A memory, coupled to the plurality of devices via a second communication network, wherein: a device among the plurality of devices is configured to: not publish an access request to the memory to the second communication network that indicates a program order dependency on a previous access request in the program order, until a first memory dependency token generated by another device in the plurality of devices upon completion of access to the memory in the program order; and a single device among the plurality of devices is configured to: for a first access request to the memory received by the single device and a second access request to the memory received by the single device that indicates a program order dependency on the first access request, provide a second memory dependency token for the program order dependency when publishing the first access request to the second communication network, and then publish the second access request to the second communication network based on reading the second memory dependency token.

Citation Information

Patent Citations

  • Instruction set for eliminating misaligned memory accesses during processing of an array having misaligned data rows

    CN105278921A

  • Processor reduction unit for accumulation of multiple operands with or without saturation

    CN1820246A