Indexing external memory in a reconfigurable computing fabric

Abstract

The present disclosure relates to indexing external memory in a reconfigurable computing fabric. Various examples relate to systems and methods in which a stream controller of a first synchronous stream can receive an instruction to execute a first loop using the first synchronous stream. The stream controller can determine a first iteration index for a first iteration of the first loop. The stream controller can send a first synchronization message to a first compute element of the first synchronous stream to initiate a first synchronization stream thread for executing the first iteration of the first loop. The first synchronization message can include the iteration index. The first compute element can be capable of performing an input / output operation at a first location of a first compute element memory indicated by the first iteration index.

Description

Technical Field

[0001] This disclosure relates to indexing external memory in a reconfigurable computing architecture. Background Technology

[0002] Various computer architectures, such as the von Neumann architecture, typically use shared memory for data, buses for accessing the shared memory, arithmetic units, and program control units. However, moving data between the processor and memory can be time-consuming and energy-intensive, which can constrain the performance and capacity of a computer system. Given these limitations, new computing architectures and devices are needed to drive computing performance beyond the transistor scale (i.e., Moore's Law). Summary of the Invention

[0003] In one aspect, this disclosure provides an apparatus comprising: a reconfigurable computing structure including a plurality of computing elements and at least one flow controller, the reconfigurable computing structure being arranged to include a first synchronization stream, the first synchronization stream including one of the at least one flow controller and a plurality of the plurality of computing elements, the first synchronization stream being programmed using stored instructions to perform operations including: executing a first synchronization stream thread for performing at least a portion of a loop iteration for generating at least one value of a multidimensional array; sending a write request from a computing element of the first synchronization stream to a memory interface, the write request including an iteration index indicating the loop iteration and payload data including at least one value of the multidimensional array; and a memory interface being programmed using stored instructions to perform operations including: determining an external memory offset using the indication of the iteration index; and writing the payload data to an external memory location indicated by the external memory offset in external memory.

[0004] In another aspect, this disclosure further provides a method comprising: executing a first synchronous stream thread by a first synchronous stream of a reconfigurable computational structure, the first synchronous stream thread being configured to perform at least a portion of a loop iteration to generate at least one value of a multidimensional array, the reconfigurable computational structure including a plurality of computational elements and at least a flow controller, the reconfigurable computational structure being arranged to include the first synchronous stream, the first synchronous stream including the flow controller of the reconfigurable computational structure and a plurality of the plurality of computational elements; sending a write request from a computational element of the first synchronous stream to a memory interface, the write request including an iteration index indicating the loop iteration and payload data including at least one value of the multidimensional array; using the iteration index to determine an external memory offset by the memory interface; and writing the payload data to an external memory location indicated by the external memory offset by the memory interface.

[0005] In another aspect, this disclosure further provides a machine-readable medium including instructions that, when executed by a processor, cause the processor to perform operations including: executing a first synchronization stream thread by a first synchronization stream, the first synchronization stream thread being configured to perform at least a portion of a loop iteration to generate at least one value of a multidimensional array; sending a write request to a memory interface by a computational element of the first synchronization stream, the write request including an iteration index indicating the loop iteration and payload data including at least one value of the multidimensional array; using the iteration index by the memory interface to determine an external memory offset; and writing the payload data to an external memory location indicated by the external memory offset by the memory interface. Attached Figure Description

[0006] To facilitate identification of any particular element or action, one or more of the most significant digits in the reference numerals refer to the figure number in which the element is first introduced.

[0007] Figure 1 This generally illustrates a first instance of a first memory computing device in the context of a memory computing system according to an embodiment.

[0008] Figure 2 An example of a memory subsystem of a memory computing device according to an embodiment is generally shown.

[0009] Figure 3 An example of a programmable atomic unit for a memory controller according to an embodiment is generally shown.

[0010] Figure 4An example of a hybrid thread processor (HTP) accelerator for a memory computing device according to an embodiment is shown.

[0011] Figure 5 An example of a representation of a hybrid threaded structure (HTF) of a storage computing device according to an embodiment is shown.

[0012] Figure 6A An example of a chiplet system according to an embodiment is shown in general.

[0013] Figure 6B Generally showing the source Figure 6A A block diagram of the various components in an example chiplet system.

[0014] Figure 7 This section generally illustrates an example of a chiplet-based implementation for a memory computing device according to an embodiment.

[0015] Figure 8 This illustrates an exemplary chipletization of a memory computing device according to an embodiment.

[0016] Figure 9 This is a diagram illustrating an example of a workflow used to perform operations at a reconfigurable computational structure using more than one synchronous stream.

[0017] Figure 10 This is a diagram illustrating an example of a workflow for performing operations at a reconfigurable computing structure, where a synchronization flow interacts with a memory interface.

[0018] Figure 11 This is a diagram illustrating an example of a workflow for performing loops using a reconfigurable computational structure.

[0019] Figure 12 This is a diagram illustrating an example of a workflow that uses a reconfigurable computational structure to perform a triple nested loop.

[0020] Figure 13 This is a diagram illustrating an instance of a synchronous flow arranged with a flow controller configured to generate iterative indices.

[0021] Figure 14 This is a flowchart illustrating an example of a processing flow that can be executed by a flow controller to perform a loop at a synchronous flow using iterative indexes.

[0022] Figure 15 This is a diagram illustrating an example of a flow controller and computational element that utilizes a packed iteration index for a synchronous flow.

[0023] Figure 16This is a flowchart illustrating an example of a process flow that can be performed using synchronous streams and memory interfaces to write the values ​​of a multidimensional array to external memory.

[0024] Figure 17 This is a diagram illustrating an instance of a multidimensional array that maps external memory offsets to external memory locations.

[0025] Figure 18 This is a flowchart illustrating an example of a processing flow that can be executed by the memory interface to generate an external memory offset.

[0026] Figure 19 A block diagram of an exemplary machine is shown, in which any one or more of the techniques (e.g., methods) discussed herein can be implemented. Detailed Implementation

[0027] Recent advances in materials, devices, and integration technologies can be leveraged to provide memory-centric computing topologies. These topologies enable advancements in computational efficiency and workload handling for applications constrained by size, weight, or power requirements. Topologies can facilitate low-latency computing in or near memory or other data storage elements. The approach is particularly well-suited for various computationally intensive operations utilizing sparse lookups, such as in transform computations (e.g., Fast Fourier Transform (FFT) calculations), or in applications such as neural networks or artificial intelligence (AI), financial analysis, or simulation or modeling, such as for computational fluid dynamics (CFD), engineer's enhanced acoustic simulators (EASE), integrated circuit-centric simulation programs (SPICE), etc.

[0028] The systems, apparatuses, and methods discussed herein may include or utilize memory computing systems with processors or processing capabilities provided in, near, or integrated with memory or data storage components. Such systems are generally referred to herein as compute-near-memory (CNM) systems. CNM systems may be node-based systems, where individual nodes in the system are coupled using a system-scale architecture. Particularly in environments where high cache miss rates are expected, each node may include or utilize dedicated or general-purpose processors and user-accessible accelerators, and have custom computational architectures to facilitate intensive operations.

[0029] In this example, each node in a CNM system can have one or more host processors. Within each node, a dedicated hybrid-threaded processor can occupy discrete endpoints of the on-chip network. The hybrid-threaded processor can access some or all of the memory in a specific node of the system, or it can access the memory of a network spanning multiple nodes via a system-scale architecture. Custom computational architectures or hybrid-threaded architectures at each node can have their own processors or accelerators and can operate at higher bandwidths than the hybrid-threaded processors. Different nodes in a near-memory computing system can be configured differently, such as having different computing capabilities, different types of memory, different interfaces, or other differences. However, nodes can be coupled together to share data and computational resources within a defined address space.

[0030] In this example, near-memory computing systems (CNMs) or nodes within such systems can be configured by the user for custom operations. Users can provide instructions using high-level programming languages ​​(such as C / C++), which can be compiled and directly mapped into the system's dataflow architecture, or into the dataflow architecture of one or more nodes within the CNM system. That is, nodes in the system can contain hardware blocks (e.g., memory controllers, atomic cells, other client accelerators, etc.) that can be configured to directly implement or support user instructions, thereby enhancing system performance and reducing latency.

[0031] In practice, near-memory computing systems are particularly well-suited for implementing hierarchical structures of instructions and nested loops (e.g., two, three, or more loop depths, or multidimensional loops). Standard compilers can be used to accept high-level language instructions and compile them directly into one or more dataflow architectures within a node. For example, nodes in the system can contain hybrid threading accelerators. These hybrid threading accelerators can execute in the user space of the CNM system and can initiate their own threads or sub-threads that can operate in parallel. Each thread can be mapped to a different loop iteration, thereby supporting multidimensional loops. Leveraging the ability to initiate such nested loops, along with other capabilities, CNM systems can achieve significant time savings and latency improvements for computationally intensive operations.

[0032] In some instances, near-memory computing systems are programmed to arrange components of a reconfigurable computing architecture (such as the various HTFs described herein) into one or more synchronous streams. The reconfigurable computing architecture includes one or more hardware flow controllers and one or more hardware computing elements, which can be arranged to form one or more synchronous streams, as described herein.

[0033] Computing elements include computing element memory and processors or other suitable logic circuitry that form a computing pipeline for processing received data. In some instances, computing elements include multiple parallel processing lanes, such as Single Instruction Multiple Data (SIMD) processing lanes. Computing elements also include circuitry for sending and receiving synchronous and asynchronous messages to and from flow controllers, other computing elements, and other system components, as described herein. This document is about Figure 5 Pieces 504, 510, and 512 describe exemplary computing elements.

[0034] A flow controller includes a processor or other logic circuitry for managing synchronous flows, as described herein. A flow controller also includes circuitry for sending synchronous and asynchronous messages to computing elements, other flow controllers, and other system components, as described herein. In some instances, a flow controller is implemented using the base of one or more of the tiles 504, 510, and 512 described herein.

[0035] Synchronous flow is a hardware arrangement in a reconfigurable computing architecture that includes a hardware flow controller and an ordered synchronous data path between one or more hardware computing elements.

[0036] A synchronous stream can execute one or more worker threads. To execute threads, the hardware components of the synchronous stream pass synchronization messages and execute a predetermined set of operations in the order of the synchronous stream. The stream controller of the synchronous stream initiates a thread at the synchronous stream by providing a first synchronization message to a first computing element of the synchronous stream. The first synchronization message contains data (e.g., data for processing by the computing element) and may also contain control information providing various flags and other configuration and / or instruction data to the computing element. The first computing element is programmed to perform one or more operations, for example, based on the data provided by the first synchronization message. The first computing element generates a second synchronization message, which may also contain data and control information. For example, the second synchronization message may describe the result of one or more operations performed by the first computing element.

[0037] The first computing element provides a second synchronization message to the next computing element according to the sequence of the synchronization flow. The next computing element in the synchronization flow can be another computing element with a reconfigurable computing structure, although in some instances, a single computing element can perform consecutive operations of the synchronization flow, meaning that in some arrangements, the computing element of the synchronization flow can direct synchronization messages to itself. The next computing element is programmed to perform one or more operations, which may include preparing a third synchronization message and sending the third synchronization message to subsequent computing elements.

[0038] A thread completes when all computational elements of the synchronous stream have completed their programming operations in the predetermined order of the synchronous stream. Once the thread is complete, the pipeline of synchronous messages propagates between the computational elements in the predetermined order of the synchronous stream, starting from the stream controller. Because this arrangement is synchronous, thread completion can occur within a fixed amount of time (e.g., a predictable number of clock cycles from when the synchronous stream is started by the stream controller).

[0039] Arranging an HTF (Hybrid Flow Controller) to include synchronous streams can facilitate parallel processing. For example, a flow controller used for a synchronous stream does not need to wait for a thread to complete before launching additional threads. Consider an exemplary synchronous stream comprising a flow controller and multiple compute elements. The flow controller launches a first thread by providing a synchronization message to a first compute element in the synchronous stream. The first compute element performs its processing and directs a second synchronization message to the next compute element, and so on. After the first compute element completes its processing and directs a synchronization message to the next compute element, the flow controller can launch additional threads at the synchronous stream, for example, by providing additional synchronization messages to the first compute element.

[0040] Additional parallelization of the synchronization stream at a reconfigurable computational structure can be achieved by utilizing computational elements (such as the various pieces described herein) that operate at a predefined rhythm or spoke counting. For example, a computational element can perform various operations, such as receiving synchronization messages, performing processing operations, sending synchronization messages, etc., using a predetermined number of clock cycles. A computational element can be configured to receive a new synchronization message and begin operations on a thread, while operations from previous threads still propagate through the computational element. The new thread can be a different thread from the same synchronization stream as the previous thread, or it can be a thread from a different synchronization stream.

[0041] Synchronous flows can use asynchronous structures within reconfigurable compute structures to communicate with other synchronous flows and / or other components of the reconfigurable compute structure using asynchronous messages. For example, a flow controller can receive asynchronous messages from a dispatch interface and / or from another flow controller, instructing the flow controller to start a thread at a synchronous flow. The dispatch interface can interface between the reconfigurable compute structure and other system components. Furthermore, in some instances, synchronous flows can send asynchronous messages to the dispatch interface to indicate thread completion.

[0042] Asynchronous messages can also be used by various synchronous streams to access external memory. For example, a reconfigurable computing architecture can include one or more memory interfaces. A memory interface is a hardware component used by a synchronous stream or its components to access external memory that is not part of the synchronous stream. Threads executing in a synchronous stream can include sending read and / or write requests to the memory interface. Because reads and writes are asynchronous, the thread that initiates a read or write request to the memory interface may not receive the result of the request. Instead, the result of the read or write request can be provided to different threads executing at different synchronous streams.

[0043] Consider an exemplary reconfigurable computing structure configured with a first synchronous stream for initiating read requests and a second synchronous stream for receiving the results of the read requests. A first thread in the first synchronous stream sends an asynchronous read request message to a memory interface. The first thread may also send an asynchronous continue message of type A to the flow controller of the second synchronous stream, where the continue message indicates a read request. The memory interface retrieves the requested data from memory and directs the read data to the appropriate computing element in the second synchronous stream. The computing element then directs an asynchronous message to the second flow controller, indicating that the data has been received. (In some instances, the memory interface provides the read data directly to the second flow controller). Upon receiving the indication that the read data has been received, the second flow controller initiates a thread in the second synchronous stream to further process the results of the read request.

[0044] In some instances, reconfigurable computational structures of HFTs, as described herein, are used to execute one or more loops, such as a set of nested loops as described herein. To execute the loops, the reconfigurable computational structures utilize flow controllers and computational elements arranged as one or more synchronous flows, as described herein. For example, the flow controller of the synchronous flows can start threads at the synchronous flow for each iteration of the loop. Consider a simple exemplary loop given by the following code snippet [1]:

[0045]

[0046] The flow controller can begin an exemplary loop by starting a first thread at the synchronous flow where i=1 is iterated. In this example, the flow controller passes the initial value of x[i-1] along with the payload data of the initial synchronization message to the first computational element. One or more computational elements of the synchronous flow determine the value of x[1] and return the value of x[1] as a synchronous or asynchronous message to the flow controller. Then, the flow controller starts a second thread at the synchronous flow where i=2 is iterated, passing the return value of x[1] as x[i-1] to the synchronization message. This process continues until all iterations of the loop are completed and the value of x

[10] is returned.

[0047] The exemplary loop above uses a single synchronization stream for each iteration of the loop. However, in some instances, multiple synchronization streams can be used for each loop iteration. Consider the exemplary loop given by the following code snippet [2]:

[0048]

[0049] In this example, each loop iteration involves multiplying i by the value y[i] read from memory and then writing the result back to memory. Therefore, each loop iteration includes asynchronous memory reads and asynchronous memory writes. As described herein, a memory read involves sending an asynchronous message to the memory interface and then waiting for the memory interface to respond with another asynchronous message containing the requested data. Because the memory read is asynchronous, each loop iteration can utilize synchronous stream threads executing at two different synchronous streams. For the i=1 loop iteration, the thread at the first synchronous stream sends an asynchronous message to the memory interface containing a request to read the value of y[1]. The thread at the first synchronous stream can also send an asynchronous message to the second stream controller of the second synchronous stream, indicating that the second stream controller expects the result of the read request (either directly from the memory interface or from a computational element of the second synchronous stream that has already received the read data). The memory interface initiates a read of the value of y[1] and provides the value of y[1] to the second synchronous stream via an asynchronous message. Upon receiving an asynchronous message indicating that read data has been received, the second stream controller starts a thread at the second synchronous stream. (The return value of y[1] can be provided to the computing element, for example, through thread synchronization communication and / or directly from the memory interface before the thread starts). The second thread determines the value of x[1] and sends a synchronization message to the memory interface containing a write request for x[1].

[0050] In some instances, the number of threads a synchronous stream controller can launch at a synchronous stream is limited by the resources of the synchronous stream's components. For example, synchronous stream threads may write data to the local compute element memory at the synchronous stream compute element. If too many synchronous stream threads are launched simultaneously, some threads may lack sufficient local memory or other resources in a hybrid threading architecture. This could prevent a synchronous stream thread from writing its data and / or cause it to overwrite data in the local memory of other synchronous stream threads.

[0051] To prevent this, a reconfigurable compute structure can limit the number of synchronous streaming threads that can be started at a given time. For example, a reconfigurable compute structure can implement a pool of thread identifiers (IDs). The stream controller can determine the availability of thread IDs before implementing synchronous streaming threads. In some instances, the synchronization message of a synchronous streaming thread can include an indication of the thread ID of a given thread.

[0052] When a synchronous stream thread completes, it can, for example, send an asynchronous idle message to the stream controller that started the synchronous stream thread. This indicates to the stream controller that the thread ID (and associated resources) of the completed synchronous stream thread is now available for new synchronous stream threads.

[0053] In some instances, a single thread ID can be used for more than one synchronization stream simultaneously. For example, when a single operation string is executed on more than one synchronization stream, the corresponding threads at multiple synchronization streams can utilize the same thread ID. Consider again the example of code snippet [2]. Recall that for the i=1 loop iteration, the thread at the first synchronization stream sends an asynchronous message to the memory interface with a request to read the value of y [1], and sends an asynchronous continuation type message to the second stream controller of the second synchronization stream. When the value of y [1] is received at the second synchronization stream, the second stream controller starts a thread at the second synchronization stream. In this example, the i=1 thread at the first synchronization stream and the i=1 thread at the second synchronization stream can use a common thread ID (and associated resources).

[0054] Furthermore, in some instances, thread IDs can be described by different levels, where different numbers of thread IDs (and corresponding resources) are available at different levels. Consider nested loops. The synchronized stream thread executing the first level of the nested loop can be assigned a thread ID (and corresponding resources) from the first level thread pool. The synchronized stream thread from the second level loop can be assigned a thread ID (and corresponding resources) from the second level thread pool, and so on.

[0055] In some instances, thread IDs can be tracked by the flow controller based on asynchronous messages received by the flow controller to indicate that the flow controller is initiating a synchronous flow thread. Consider an instance where a dispatch interface sends asynchronous loop messages to the flow controller of a synchronous flow. The loop message may indicate the number of loop iterations to be executed, the number of available thread IDs, and, in some instances, one or more conditions under which a previously used thread ID becomes available again. For example, the thread ID used by the flow controller to initiate the synchronous flow thread may become available again when the synchronous flow thread completes (if a single thread ID is used for only one synchronous flow), or when subsequent synchronous flow threads at different synchronous flows complete (if a single thread ID is used for more than one synchronous flow, as described herein). Near-memory computing systems or nodes or components of near-memory computing systems may include or use various memory devices, controllers, and interconnects, etc. In instances, the system may include various interconnect nodes, and nodes or groups of nodes may be implemented using chiplets. Chiplets are an emerging technology for integrating various processing functionalities. Typically, chiplet systems consist of discrete chips (e.g., integrated circuits (ICs) on different substrates or dies) integrated on an interposer and packaged together. This arrangement differs from a single chip (e.g., an IC) containing different blocks of devices (e.g., intellectual property (IP) blocks) on a substrate (e.g., a single die), such as a system-on-a-chip (SoC), or discrete packaged devices integrated on a board. Generally, chiplets offer manufacturing benefits compared to single-die chips, including higher yields or reduced development costs. The following discussion... Figure 6A and Figure 6B Examples of chiplet systems are generally shown, such as near-memory computing systems.

[0056] When a synchronous stream is used to execute a loop, the synchronous stream threads executing different iterations of the loop may need to read data from and / or write data to external memory (e.g., memory other than local compute element memory or tile memory). For example, during the execution of a synchronous stream thread that implements all or part of the loop iterations, the compute element may perform loop-iteration-specific reads or writes to external memory via a memory interface.

[0057] When computational elements of a synchronous stream read from and / or write to external memory during loop iterations, it is sometimes desirable for the computational elements to read from and / or write to memory locations specific to the loop iteration being executed. For example, if a synchronous stream thread executing a first loop iteration reads operand data from a first location in the computational element's memory, a synchronous stream thread executing a second iteration may read its operand data from a different location. A synchronous stream thread executing a third iteration may read its operand data from yet another different location, and so on. Similarly, synchronous streams can be arranged such that threads write result data to loop iteration-specific memory locations.

[0058] One way to enable a synchronous streaming computing element to read from and write to a loop-specific memory location is to use an iteration index. The iteration index is derived from the number of iterations of the loop and points to a specific memory location or set of memory locations that can be read from and / or written to during iteration execution. When a synchronous streaming computing element reads from or writes to memory, it uses the iteration index to refer to the iteration-specific memory location.

[0059] Consider the exemplary code segment [2] above. The computational elements at the first and second synchronization streams can derive the iteration index using the iteration number i. In the example of code segment [2], each iteration of the i loop uses two memory locations. Operand data x[i] is read from the memory location, and output data y[i] is written to the memory location. Therefore, a simple iteration index can be determined by simply multiplying the value of i for the iteration by 2.

[0060] Consider another instance given by the following code snippet [3]:

[0061]

[0062] This example contains nested loops with a first-level x loop and a second-level y loop. In this example, the first-level x loop will execute 2048 times for the value of x from 1 to 2048. The second-level y loop will execute 2048 times for each iteration of the outer loop, for a total of 4,235,364 iterations. It may be desirable to determine the iteration index of a code segment that has a unique value for each of the 4,235,364 iterations. In some instances, this can be achieved by generating the iteration index from the number of iterations of the first-level loop and the iterations of the second-level loop. Using the code segment [3], a unique iteration index can be derived by performing operations on the number of iterations of the first-level x loop and the number of iterations of the second-level y loop. For example, the iteration index of the loop iteration where x = 588 and y = 2043 can be derived by performing operations on 588 and 2043.

[0063] Iteration indices can be exported during loop execution. However, exporting iteration indices in a synchronous stream may involve using additional computational element resources, or even constructing the synchronous stream with additional computational elements. For example, referring to the code snippet above [3], a synchronous stream for computing the value s[x,y] could contain computational elements that fetch the value r[x,y] from computational element memory, perform the indicated multiplication, and write the result s[x,y] back to computational element memory. However, a computational element could first export the iteration index, then use that iteration index to read r[x,y] from the appropriate computational element memory location and write s[x,y] to the attribute computational element memory location. Iteration indices can be exported at the computational element, or in some instances, additional computational elements located before the first computational element can be used. This can lead to inefficient use of computational elements.

[0064] Furthermore, in some reconfigurable computational structures, computing iterative indices may not be the optimal use of computational elements. Consider again the exemplary code snippet above [3]. In this instance, generating an iterative index involves processing two 12-bit values ​​(representing 1-2048). However, in various instances, computational elements in reconfigurable computational structures are arranged to operate on wider or even much wider data than 12 bits. In some instances described herein, computational elements may be arranged to operate on a 512-bit data path. Using one or more cycles of a 512-bit data path to perform operations on 12-bit values ​​may not be efficient. Therefore, in some instances, the flow controller is configured to generate iterative indices. The generated iterative index is then passed to the computational elements of the synchronized flow via synchronization messages, as control information is passed between computational elements during the execution of the synchronized flow thread.

[0065] In some instances, the computational structure can be reconfigured to execute two or more nested loops to generate values ​​for a multidimensional array. Each iteration of the nested loop can generate one (or more) values ​​for the multidimensional array, where the inner loop iteration corresponds to one dimension of the multidimensional array and the outer loop iteration corresponds to another dimension. Consider again the exemplary code snippet above [3]. In this instance, each iteration of the nested loop generates the value of the multidimensional array s[x,y]. The coordinates of the value generated by each iteration are given by the values ​​of the outer loop iteration and the inner loop iteration. For example, in the exemplary code snippet [3], the iteration result where x = 321 and y = 832 is the multidimensional array value s[321,832].

[0066] In some instances, it is desirable for reconfigurable computational structures to store multidimensional arrays in external memory. For example, arrays stored in the memory of one or more computational elements, while easily accessible by a synchronous stream, may not be accessible by other components of the near-memory computational system. Furthermore, in some instances, more than one synchronous stream may be used to generate multidimensional arrays. Referring again to the exemplary code snippet [3], iterations of x and / or y over a certain range may be performed by one synchronous stream, while iterations of x and y over another range may be performed by another synchronous stream. For example, one synchronous stream may perform iterations where x is between 1 and 64. Another synchronous stream may perform iterations where x is between 65 and 128, and so on.

[0067] When a multidimensional array is stored in external memory, it may be necessary to store the values ​​of the multidimensional array according to their positions in the array. Again, using the exemplary code snippet [3], it might be desirable to store the values ​​of array s[x,y] in external memory at the locations that identify the corresponding x and y positions of these values. This could make it easier for other components to access the array later. However, writing array values ​​to external memory locations that identify the spatial locations of the array values ​​presents at least two obstacles. For example, the external memory location address may be linear, or if it is not linear, it may not match the dimensions of the multidimensional array. This problem can be solved by simply writing the array values ​​to memory locations allocated in the order in which the values ​​are written. However, this approach may present challenges in reconfigurable computational structures or alternative arrangements where the values ​​of the array may not be determined in order. For example, if one or more synchronous stream threads determining the value of s[1,3] complete before one or more synchronous stream threads determining the value of s[1,1], the values ​​may be written out of order.

[0068] Various instances address these and other challenges by configuring the memory interface to translate iteration indices or other iteration indicators into external memory offsets. Consider again the exemplary code snippet [3]. A synchronous stream thread executing iterations of an inner “y” loop can receive or compute iteration indices indicating iterations of both the outer “x” loop and the inner “y” loop. To save the value s[x,y] to external memory, the thread can send an asynchronous write request containing payload data (s[x,y] in this instance) and the iteration index or other indicators of the corresponding iterations of the x and y loops. The memory interface can translate the iteration index or other indicators of the loop iterations into corresponding external memory offsets and write the payload data to one or more external memory locations indicated by the external memory offsets relative to the base address.

[0069] Figure 1A first instance of a near-memory computing system, or CNM system 102, is generally illustrated. The instance of CNM system 102 comprises multiple different memory computing nodes, each of which may contain various near-memory computing devices. Each node in the system can operate within its own operating system (OS) domain (e.g., Linux and others). In this instance, the nodes may coexist in a common OS domain of CNM system 102.

[0070] Figure 1 An instance of the CNM system 102 includes an instance of the first memory compute node 104. The CNM system 102 may have multiple nodes, such as different instances of the first memory compute node 104, coupled using a scaling structure 106. In an instance, the architecture of the CNM system 102 can support scaling with up to n different memory compute nodes (e.g., n = 2048) using the scaling structure 106. As discussed further below, each node in the CNM system 102 may be a collection of multiple devices.

[0071] CNM system 102 may include a global controller for each node in the system, or a specific memory compute node in the system may optionally act as a host or controller for one or more other memory compute nodes in the same system. The nodes in CNM system 102 may therefore be configured similarly or differently.

[0072] In this example, each node in CNM system 102 may include a host system using a specified operating system. The operating systems can be common or different across the nodes in CNM system 102. Figure 1In this example, the first memory computing node 104 includes a host system 108, a first switch 110, and a first memory computing device 112. The host system 108 may include a processor, such as an x86, ARM, RISC-V, or other type of processor. The first switch 110 may be configured to facilitate communication between or between devices of the first memory computing node 104 and devices of the CNM system 102, such as using a dedicated or other communication protocol (generally referred to herein as the Chip-to-Chip Protocol Interface (CTCPI)). That is, CTCPI may include a dedicated interface unique to the CNM system 102, or may include or use other interfaces, such as a Compute High-Speed ​​Link (CXL) interface, a Peripheral Component Interconnect High-Speed ​​(PCIe) interface, or a Chiplet Protocol Interface (CPI) interface, and others. The first switch 110 may include switches configured to use CTCPI. For example, the first switch 110 may include a CXL switch, a PCIe switch, a CPI switch, or other types of switches. In this example, the first switch 110 may be configured to couple disparately configured endpoints. For example, the first switch 110 can be configured to convert packet formats between formats such as PCIe and CPI.

[0073] The CNM system 102 is described herein with various exemplary configurations (such as systems including nodes), and each node may include various chips (e.g., processors, switches, memory devices, etc.). In an example, the first memory computing node 104 in the CNM system 102 may include various chips implemented using chiplets. In the chiplet-based configuration of the CNM system 102 discussed below, inter-chiplet communication and additional intra-system communication may use a CPI network. The CPI network described herein is an example of CTCPI, that is, a chiplet-specific implementation of CTCPI. Therefore, the structure, operation, and functionality of CPI described below are equally applicable to structures, operations, and functions that may be implemented using non-chiplet-based CTCPI implementations. Unless otherwise expressly indicated, any discussion of CPI herein applies equally to CTCPI.

[0074] The CPI interface comprises a packet-based network that supports virtual channels to enable flexible and high-speed interaction between chiplets, such as portions of the first memory compute node 104 or the CNM system 102. CPI can bridge intra-chiplet networks to a wider range of chiplet networks. For example, the Advanced Scalable Interface (AXI) is a specification for intra-chip communication. However, the AXI specification covers a large number of physical design options, such as the number of physical channels, signal timing, power, etc. Within a single chip, these options are typically selected to meet design goals such as power consumption, speed, etc. However, to achieve the flexibility of chiplet-based memory compute systems, adapters using CPI can intersect between various AXI design options that can be implemented in various chiplets. By utilizing packetization protocols to enable physical-to-virtual channel mapping and encapsulating time-based signaling, CPI can be used to bridge intra-chiplet networks (such as those within a specific memory compute node) across a wider chiplet network (such as across the first memory compute node 104 or across the CNM system 102).

[0075] CNM system 102 is scalable to accommodate multi-node configurations. That is, multiple different instances of the first memory compute node 104 or other differently configured memory compute nodes can be coupled using scaling structure 106 to provide a scalable system. Each of the memory compute nodes can run its own operating system and can be configured to jointly coordinate the use of wide system resources.

[0076] exist Figure 1 In this example, the first switch 110 of the first memory compute node 104 is coupled to the scaling structure 106. The scaling structure 106 can provide switches (e.g., CTCPI switches, PCIe switches, CPI switches, or other switches) that can facilitate communication between different memory compute nodes. In this example, the scaling structure 106 can facilitate communication between nodes within a partitioned global address space (PGAS).

[0077] In this example, the first switch 110 of the first memory computing node 104 is coupled to one or more different memory computing devices, such as a first memory computing device 112. The first memory computing device 112 may include a chiplet-based architecture, referred to herein as a near-memory computing (CNM) chiplet. A packaged version of the first memory computing device 112 may include, for example, one or more CNM chiplets. The chiplet may be communicatively coupled using CTCPI for high bandwidth and low latency.

[0078] exist Figure 1 In an example, the first memory computing device 112 may include a network on-chip (NOC) or a first NOC 118. Typically, an NOC is an interconnect network within a device that connects a specific set of endpoints. Figure 1 In this context, the first NOC 118 can provide communication and connectivity between various memories, computing resources and ports of the first memory computing device 112.

[0079] In an example, the first NOC 118 may include a folded Clos topology, such as within each instance of a memory computing device, or as a grid coupling multiple memory computing devices in a node. Clos topologies offer various benefits, such as the ability to use multiple lower-radix cross switches to provide the functionality associated with higher-radix cross switch topologies. For example, Clos topologies can present consistent latency and bi-directional bandwidth across the NOC.

[0080] The first NOC 118 can include various types of switches, including hub switches, edge switches, and endpoint switches. Each switch can be configured as a crossbar switch to provide substantially uniform latency and bandwidth between input and output nodes. In an example, endpoint switches and edge switches may contain two separate crossbar switches, one for traffic destined for the hub switch and the other for traffic destined for the hub switch. Hub switches can be configured as a single crossbar switch to switch all inputs to all outputs.

[0081] In this example, hub switches may each have multiple ports (e.g., four or six ports each) depending on whether a particular hub switch participates in inter-chip communication. The number of hub switches participating in inter-chip communication can be set by inter-chip bandwidth requirements.

[0082] The first NOC 118 can support various payloads between computing elements and memory (e.g., 8 to 64 bytes of payload; other payload sizes can be used similarly). In an instance, the first NOC 118 can be optimized for relatively small payloads (e.g., 8 to 16 bytes) to efficiently handle access to sparse data structures.

[0083] In this example, the first NOC 118 may be coupled to an external host via a first physical layer interface 114, a PCIe slave module 116 or endpoint, and a PCIe master module 126 or root port. That is, the first physical layer interface 114 may include an interface that allows the external host processor to couple to the first memory computing device 112. The external host processor may optionally be coupled to one or more different memory computing devices, such as using a PCIe switch or other local protocol switches. Communication with the external host processor via a PCIe-based switch limits device-to-device communication to device-to-device communication supported by the switch. In contrast, communication via a memory computing device local protocol switch, such as using CTCPI, allows for more complete communication between or within different memory computing devices, including support for partitioned global address spaces, such as for generating worker threads and sending events.

[0084] In this example, the CTCPI protocol may be used by a first NOC 118 in a first memory computing device 112, and a first switch 110 may include a CTCPI switch. The CTCPI switch may allow CTCPI packets to be transmitted from a source memory computing device, such as the first memory computing device 112, to different destination memory computing devices (e.g., on the same or another node) without being converted to another packet format.

[0085] In one example, the first memory computing device 112 may include an internal host processor 122. The internal host processor 122 may be configured to communicate with the first NOC 118 or other components or modules of the first memory computing device 112, for example, using an internal PCIe master module 126, which can help eliminate the time- and energy-consuming physical layer. In one example, the internal host processor 122 may be based on a RISC-VISA processor and may use a first physical layer interface 114 to communicate externally to the first memory computing device 112, such as with other storage devices, networks, or other peripherals of the first memory computing device 112. The internal host processor 122 may control the first memory computing device 112 and may act as a proxy for operating system-related functionalities. The internal host processor 122 may include a relatively small number of processing cores (e.g., 2 to 4 cores) and a host memory device 124 (e.g., including DRAM modules).

[0086] In one example, the internal host processor 122 may include a PCI root port. When the internal host processor 122 is in use, one of its root ports may be connected to a PCIe slave module 116. Another root port of the internal host processor 122 may be connected to a first physical layer interface 114, such as to provide communication with external PCI peripherals. When the internal host processor 122 is disabled, the PCIe slave module 116 may be coupled to the first physical layer interface 114 to allow an external host processor to communicate with a first NOC 118. In an example of a system with multiple memory computing devices, the first memory computing device 112 may be configured to act as a system host or controller. In this example, the internal host processor 122 may be in use, and other instances of internal host processors in corresponding other memory computing devices may be deactivated.

[0087] The internal host processor 122 can be configured at power-up of the first memory computing device 112 to allow host initialization. In an example, the internal host processor 122 and its associated data paths (e.g., including the first physical layer interface 114, PCIe slave module 116, etc.) can be configured to the first memory computing device 112 from input pins. One or more pins can be used to enable or disable the internal host processor 122 and configure the PCI (or other) data paths accordingly.

[0088] In this example, the first NOC 118 can be coupled to scale structure 106 via scale structure interface module 136 and the second physical layer interface 138. Scale structure interface module 136, or SIF, can facilitate communication between the first memory computing device 112 and device space (such as Partitioned Global Address Space (PGAS)). The PGA can be configured to allow a specific memory computing device, such as the first memory computing device 112, to access memory or other resources on different memory computing devices (e.g., on the same or different nodes) using a load / store paradigm. Various scalable architecture technologies can be used, including CTCPI, CPI, Gen-Z, PCI, or Ethernet bridged via CXL. Scale structure 106 can be configured to support various packet formats. In this example, scale structure 106 supports out-of-order packet communication or ordered packets, such as using path identifiers to extend bandwidth across multiple equivalent paths. Scale structure 106 typically supports remote operations such as remote memory reads, writes, and other built-in atoms, remote memory atoms, remote memory computing device send events, and remote memory computing device call and return operations.

[0089] In this example, the first NOC 118 can be coupled to one or more different memory modules, such as a first memory device 128. The first memory device 128 can include various types of memory devices, such as LPDDR5 or GDDR6, and others. Figure 1 In one example, the first NOC 118 can coordinate communication with the first memory device 128 via a memory controller 130 dedicated to a specific memory module. In this example, the memory controller 130 may include a memory module cache and an atomic operation module. The atomic operation module may be configured to provide relatively high-volume atomic operators, such as those including integer and floating-point operators. The atomic operation module may be configured to apply its operators to data within a memory module cache (e.g., including an SRAM memory-side cache), thereby allowing back-to-back atomic operations using the same memory location with minimal volume degradation.

[0090] The memory module cache can provide storage for frequently accessed memory locations, such as those that do not require re-accessing the first memory device 128. In one example, the memory module cache can be configured to cache only data for a specific instance of the memory controller 130. In another example, the memory controller 130 includes a DRAM controller configured to interface with the first memory device 128, such as a DRAM device. The memory controller 130 can provide access scheduling and bit error management, as well as other functions.

[0091] In this example, the first NOC 118 may be coupled to the hybrid thread processor (HTP 140), the hybrid thread architecture (HTF142), and the host interface and dispatch module (HIF 120). HIF 120 may be configured to facilitate access to host-based command request and response queues. In this example, HIF 120 may dispatch new execution threads on the processors or compute elements of the HTP 140 or HTF 142. In this example, HIF 120 may be configured to maintain workload balancing between the HTP 140 and HTF 142 modules.

[0092] The hybrid-threaded processor, or HTP 140, may include accelerators, such as those based on the RISC-V instruction set. The HTP 140 may include a highly threaded event-driven processor, where threads can execute in a single instruction round to maintain high instruction throughput. The HTP 140 includes relatively few custom instructions to support low-overhead threading capabilities, event send / receive, and shared-memory atomic operators.

[0093] The hybrid thread architecture or HTF 142 may include accelerators, such as non-von Neumann coarse-grained reconfigurable processors. The HTF 142 may be optimized for high-level language operations and data types (e.g., integers or floating-point). In some instances, the HTF 142 may support dataflow computation. The HTF 142 may be configured to utilize substantially all memory bandwidth available on the first memory computing device 112, such as when executing a memory-bound computational core.

[0094] The HTP and HTF accelerators of the CNM system 102 can be programmed using various high-level structured programming languages. For example, the HTP and HTF accelerators can be programmed using C / C++, or using the LLVM compiler framework. The HTP accelerator can leverage an open-source compiler environment through various additional custom instruction sets configured to improve memory access efficiency, provide message passing mechanisms, and manage events, etc. In one example, the HTF accelerator can be designed to be programmed using a high-level programming language, and the compiler can generate simulator configuration files or binaries that run on the HTF 142 hardware. The HTF 142 provides a mid-level language for precise and concise expression of algorithms while hiding the configuration details of the HTF accelerator itself. In another example, the HTF accelerator toolchain can use an LLVM front-end compiler and an LLVM intermediate representation (IR) to interface with the HTF accelerator back-end.

[0095] Figure 2 An example of a memory subsystem 200 of a memory computing device according to an embodiment is generally illustrated. The example of memory subsystem 200 includes a controller 202, a programmable atom unit 208, and a second NOC 206. The controller 202 may include or utilize the programmable atom unit 208 to perform operations using information in the memory device 204. In this example, the memory subsystem 200 includes components from… Figure 1 A portion of the first memory computing device 112, such as a portion including the first NOC 118 or memory controller 130.

[0096] exist Figure 2 In this example, the second NOC 206 is coupled to the controller 202, and the controller 202 may include a memory control module 210, a local cache module 212, and a built-in atom module 214. In this example, the built-in atom module 214 may be configured to handle relatively simple single-cycle integer atoms. The built-in atom module 214 can perform atoms with the same processing power as, for example, a normal memory read or write operation. In this example, an atomic memory operation may include a combination of: storing data into memory, performing an atomic memory operation, and then responding by loading data from memory.

[0097] A local cache module 212, which may include an SRAM cache, can be provided to help reduce latency for frequently accessed memory locations. In an example, the local cache module 212 may provide a read buffer for sub-memory line access. The local cache module 212 may be particularly advantageous for computing elements with relatively small or no data cache.

[0098] A memory control module 210, which may include a DRAM controller, can provide low-level request buffering and scheduling to provide efficient access to the memory device 204 (which may include a DRAM device). In an example, the memory device 204 may include or use a GDDR6 DRAM device, such as one with a density of 16 Gb and a peak bandwidth of 64 Gb / s. Other devices may be used similarly.

[0099] In this example, programmable atomic unit 208 may include single-loop or multi-loop operators, such as being configured to perform integer addition or more complex multi-instruction operations, such as Bloom filter insertion. In this example, programmable atomic unit 208 may be configured to perform load and store-to-memory operations. Programmable atomic unit 208 may be configured to utilize RISC-V ISA via a dedicated instruction set to facilitate interaction with controller 202 to atomically execute user-defined operations.

[0100] Programmable atomic requests (such as those received from on-node or off-node hosts) can be routed to programmable atomic units 208 via the second NOC 206 and controller 202. In this example, custom atomic operations (e.g., implemented by programmable atomic unit 208) can be identical to built-in atomic operations (e.g., implemented by built-in atomic module 214), except that the programmable atomic operations can be defined or programmed by a user rather than a system architect. In this example, programmable atomic request packets can be sent to controller 202 via the second NOC 206, and controller 202 can recognize the requests as custom atoms. Controller 202 can then forward the recognized requests to programmable atomic units 208.

[0101] Figure 3 An example of a programmable atom unit 302 for use with a memory controller, according to an embodiment, is generally illustrated. In this example, the programmable atom unit 302 may include or correspond to data from... Figure 2 The instance is a programmable atomic unit 208. That is to say, Figure 3 Components in an example of programmable atom unit 302 (PAU) are shown, as referenced above. Figure 2 (For example, in programmable atom unit 208) or reference Figure 1 (For example, those components described in the atomic operation module of memory controller 130.) Figure 3 As shown, the programmable atom unit 302 includes a PAU processor or PAU core 306, a PAU thread control 304, an instruction SRAM 308, a data cache 310, and a memory interface 312 that interfaces with a memory controller 314. In this example, the memory controller 314 includes components from... Figure 2 An instance of controller 202.

[0102] In this example, the PAU core 306 is a pipelined processor, allowing multiple stages of different instructions to execute together per clock cycle. The PAU core 306 may contain a barrel-shaped multithreaded processor, where the thread control 304 circuitry switches between different register files (e.g., a set of registers containing the current processing state) after each clock cycle. This enables efficient context switching between currently executing threads. In this example, the PAU core 306 supports eight threads, resulting in eight register files. In this example, some or all of the register files are not integrated into the PAU core 306, but reside in the local data cache 310 or instruction SRAM 308. This reduces the circuit complexity in the PAU core 306 by eliminating the traditional flip-flops used for registers in such memories.

[0103] The local PAU memory may contain instruction SRAM 308, such as instructions for various atoms. The instructions include a set of instructions to support atomic operators loaded by various applications. When an atomic operator is requested, such as via an application chiplet, a set of instructions corresponding to the atomic operator is executed via the PAU core 306. In this example, the instruction SRAM 308 may be partitioned to establish the instruction set. In this example, a specific programmable atomic operator requested by the request process can be identified by a partition number. A partition number can be established when a programmable atomic operator is registered to programmable atomic unit 302 (e.g., loaded into programmable atomic unit 302). Other metadata of the programmable instructions may be stored in the local memory of programmable atomic unit 302 (e.g., in a partition table).

[0104] In this example, atomic operators manipulate data cache 310, which is typically synchronized (e.g., flushed) when the thread used for the atomic operator completes. Therefore, latency can be reduced for most memory operations during the execution of the programmable atomic operator thread, except for initial loads from external memory such as memory controller 314.

[0105] If a potentially dangerous condition would prevent a memory request, the pipelined processor (such as PAU core 306) may experience problems when a thread attempts to make such a request. In this paper, the memory request is for retrieving data from memory controller 314, whether it comes from cache on memory controller 314 or off-die memory. To address this issue, PAU core 306 is configured to deny memory requests from threads. Typically, PAU core 306 or thread control 304 may contain circuitry for enabling one or more thread rescheduling points in the pipeline. In this paper, the denial occurs at points in the pipeline outside (e.g., after) these thread rescheduling points. In this instance, the danger occurs outside the rescheduling point. In this paper, the danger is created by a preceding instruction in the thread after the memory request instruction has passed through the last thread rescheduling point before the pipeline stage where the memory request can be made.

[0106] In this example, to reject a memory request, PAU core 306 is configured to determine (e.g., detect) the presence of a danger on the memory indicated in the memory request. In this document, a danger represents any condition that would cause an inconsistent state of the thread if the memory request were allowed (e.g., executed). In this example, the danger is an ongoing memory request. In this document, the existence of an ongoing memory request makes it uncertain what the data at that address in the data cache 310 should be, regardless of whether the data cache 310 contains data at the requested memory address. Therefore, the thread must wait for the ongoing memory request to complete before operating on the current data. The danger is cleared when the memory request completes.

[0107] In this example, the danger lies in a dirty cache line in data cache 310 used for the requested memory address. While a dirty cache line typically indicates that the data in the cache is current and the memory controller version of such data is not current, problems can arise with thread instructions that do not operate from the cache. Examples of such instructions use the built-in atomic operators of memory controller 314 or other separate hardware blocks. In the case of the memory controller, the built-in atomic operators may be separate from programmable atomic units 302 and cannot access the cache within the PAU or data cache 310. If the cache line is dirty, the built-in atomic operators will not operate on the latest data until the cache is flushed to synchronize the cache and other or off-die memory. The same situation can occur using other hardware blocks of the memory controller, such as encryption blocks, encoders, etc.

[0108] Figure 4An example of a hybrid thread processor (HTP) accelerator or HTP accelerator 400 is shown. According to an embodiment, the HTP accelerator 400 may include part of a memory computing device. In the example, the HTP accelerator 400 may include or include components from… Figure 1 The HTP 140 is an example. The HTP accelerator 400 includes, for example, an HTP core 402, an instruction cache 404, a data cache 406, a translation block 408, a memory interface 410, and a thread controller 412. The HTP accelerator 400 may further include a dispatch interface 414 and a NOC interface 416, such as for communication with NOCs (e.g., from...). Figure 1 The first NOC 118 instance, from Figure 2 The second NOC 206 or other NOCs of the instance are introduced.

[0109] In this example, the HTP accelerator 400 includes modules based on the RISC-V instruction set and may include a relatively small number of other or additional custom instructions to support low-overhead, thread-enabled hybrid threading (HT) languages. The HTP accelerator 400 may include a highly threaded processor core, HTP core 402, in which threads can execute in a single instruction round-robin to maintain high instruction throughput. In this example, threads can pause while waiting for other pending events to complete. This allows computational resources to be used efficiently for related work, rather than polling. In this example, multithreaded barrier synchronization can utilize efficient HTP-to-HTP and HTP-to / from host message passing, allowing thousands of threads to initialize or wake up in, for example, tens of clock cycles.

[0110] In an example, dispatch interface 414 may include function blocks for handling hardware-based thread management of HTP accelerator 400. That is, dispatch interface 414 can manage the dispatch of work to HTP core 402 or other accelerators. However, non-HTP accelerators typically cannot dispatch work. In an example, work dispatched from the host may use dispatch queues residing in, for example, host main memory (e.g., DRAM-based memory). On the other hand, work dispatched from HTP accelerator 400 may use dispatch queues residing in SRAM, such as within a dispatch for a target HTP accelerator 400 within a specific node.

[0111] In an example, HTP core 402 may include one or more cores that execute instructions on behalf of threads. That is, HTP core 402 may contain instruction processing blocks. HTP core 402 may further include or be coupled to thread controller 412. Thread controller 412 may provide thread control and state for each active thread within HTP core 402. Data cache 406 may include a cache for the host processor (e.g., for local and remote memory computing devices, including a cache for HTP core 402), and instruction cache 404 may include a cache for use by HTP core 402. In an example, data cache 406 may be configured for read and write operations, and instruction cache 404 may be configured for read-only operations.

[0112] In this example, data cache 406 provides a small cache per hardware thread. Data cache 406 can temporarily store data for use by the owning thread. Data cache 406 can be managed by hardware or software within the HTP accelerator 400. For example, when load and store operations are performed by the HTP core 402, the hardware can be configured to automatically allocate or evict lines as needed. Software, such as using RISC-V instructions, can determine which memory accesses should be cached and when lines should be invalidated or written back to other memory locations.

[0113] Cached data on the HTP accelerator 400 offers various benefits, including making memory controller access more efficient, thus allowing execution threads to avoid stalling. However, there are situations that can lead to inefficiencies when using caches. Examples include accesses where data is accessed only once, resulting in cache line thrashing. To help address this issue, the HTP accelerator 400 can use a custom set of load instructions to force load instructions to check for cache hits and, if a cache miss occurs, to issue a memory request for the requested operand without placing the retrieved data into the data cache 406. Therefore, the HTP accelerator 400 includes various types of load instructions, including uncached and cache line loads. If dirty data exists in the cache, the uncached load instructions use the cached data. Uncached load instructions ignore clean data in the cache and do not write the accessed data to the data cache. For cache line load instructions, a complete data cache line (e.g., including 64 bytes) can be loaded from memory into the data cache 406, and the addressed memory can be loaded into a specified register. If clean or dirty data is in data cache 406, these loads can use the cached data. If the referenced memory location is not in data cache 406, the entire cache line can be accessed from memory. The use of cache line load instructions when referencing sequential memory locations (such as in memory copy operations) can reduce cache misses, but it may also waste memory and bandwidth at NOC interface 416 if the referenced memory data is not used.

[0114] In this example, the HTP accelerator 400 includes non-cached custom store instructions. These non-cached store instructions help avoid thrashing the data cache 406 by utilizing write data that is not sequentially written to memory.

[0115] In this example, the HTP accelerator 400 further includes a translation block 408. Translation block 408 may contain a virtual-to-physical translation block for the local memory of the memory computing device. For example, a host processor, such as in the HTP core 402, may execute load or store instructions, and these instructions may generate virtual addresses. These virtual addresses can be translated into physical addresses for the host processor, such as using a translation table from translation block 408. For example, memory interface 410 may contain an interface between the HTP core 402 and NOC interface 416.

[0116] Figure 5 An example of a hybrid threading architecture (HTF) or HTF 500 of a memory computing device according to an embodiment is shown. In the example, HTF 500 may include or include components from... Figure 1The HTF 142 is an example of this. The HTF 500 is a coarse-grained, reconfigurable computational architecture that can be optimized for high-level language operand types and operators (e.g., using C / C++ or other high-level languages). In an instance, the HTF 500 may contain a configurable n-bit-wide (e.g., 512-bit-wide) data path for interconnect-hardened SIMD arithmetic units.

[0117] In this example, HTF 500 includes an HTF cluster 502 containing multiple HTF tiles, including an exemplary tile 504 or tile N. Each HTF tile may implement one or more computing elements with local tile or computing element memory and arithmetic capabilities. For example, each tile may contain a computing pipeline with support for integer and floating-point operations. In this example, data paths, computing elements, and other infrastructure may be implemented as hardened IP to provide maximum performance while minimizing power consumption and reconfiguration time.

[0118] exist Figure 5 In this example, the tiles of HTF cluster 502 are linearly arranged, and each tile in the cluster can be coupled to one or more other tiles in HTF cluster 502. Figure 5 In this example, exemplary tile 504 or tile N is coupled to four other tiles, including being coupled to the tile base of tile 510 (e.g., tile N-2) via a port labeled SF IN N-2, coupled to an adjacent tile 512 (e.g., tile N-1) via a port labeled SF IN N-1, and coupled to tile N+1 via a port labeled SF IN N+1 and to tile N+2 via a port labeled SF IN N+2. The tile base is the hardware portion of a tile (such as tiles 504, 510, 512) configured to initiate threads and / or otherwise act as a flow controller. Exemplary tile 504 may be coupled to the same or other tiles via corresponding output ports, such as output ports labeled SF OUT N-1, SF OUT N-2, SF OUT N+1, and SF OUT N+2. In this example, the ordered list of tile names is a conceptual indication of the tile's location. In other instances, tiles, including HTF cluster 502, can be arranged in a grid or other configuration, where each tile is similarly coupled to one or more of its nearest neighbors in the grid. Tiles positioned at the edge of the cluster may optionally have fewer connections to adjacent tiles. For example, Figure 5 In the example, the base of the tile at tile N-2 or tile 510 can be coupled only to the adjacent tile 512 (tile N-1) and the exemplary tile 504 (tile N). Similarly, fewer or additional inter-tile connections can be used.

[0119] HTF cluster 502 may further include a memory interface module, including a first memory interface module 506. The memory interface module can couple HTF cluster 502 to a NOC, such as a first NOC 118. In an example, the memory interface module can allow tiles within the cluster to make requests to other locations in the memory-computing system (e.g., on the same or different nodes in the system). That is, the representation of HTF 500 may include part of a larger structure that can be distributed across multiple nodes, such as having one or more HTF tiles or HTF clusters at each of the nodes. Requests can be made between tiles or nodes within the context of the larger structure.

[0120] exist Figure 5 In this example, the tiles in HTF cluster 502 are coupled using a synchronous structure (SF). The synchronous structure provides communication between a specific tile in HTF cluster 502 and its neighboring tiles, as described above. Each HTF cluster 502 may further include an asynchronous structure (AF), which provides, for example, communication between tiles in the cluster, memory interfaces in the cluster, and dispatch interfaces 508 in the cluster.

[0121] In this example, the synchronization structure can exchange messages containing data and control information. Control information may include, among other things, instruction RAM address information or thread identifiers. Control information can be used to set data paths, and data message fields can be selected as the source of the path. Typically, control fields can be provided or received earlier, making them available for configuring the data path. For example, to help minimize any delays in the synchronous pipeline through the tile, control information can arrive at the tile several clock cycles before the data fields. Various registers can be provided to help coordinate the timing of data flow in the pipeline.

[0122] In this example, each tile in the HTF cluster 502 may contain one or more tile memories. Each tile memory may have the same width as the data path (e.g., 512 bits) and may have a specified depth, such as in the range of 512 to 1024 elements. The tile memory can be used to store data that supports data path operations. For example, the stored data may contain constants loaded as part of the kernel's cluster configuration, or it may contain variables computed as part of a data stream. In this example, the tile memory may be written from an asynchronous structure as a data transfer from another synchronous stream, or it may contain the result of a load operation initiated by another synchronous stream. The tile memory can be read via synchronous data path instructions executed within the synchronous stream.

[0123] In the example, each tile in the HTF cluster 502 can have a dedicated instruction RAM (INST RAM). In an instance of an HTF cluster 502 with 16 tiles and a corresponding instruction RAM instance with 64 entries, the cluster can allow algorithms to be mapped with up to 1024 multiplication-shift and / or ALU operations. Various tiles can optionally be pipelined together using a synchronous architecture to allow computation with minimal memory access to the data stream, thus minimizing latency and reducing power consumption. In the example, an asynchronous architecture allows memory referencing and computation to continue in parallel, thereby providing a more efficient streaming kernel. In the example, various tiles can include built-in support for loop-based constructions and can support nested loop kernels.

[0124] Synchronization architectures can allow multiple pieces (e.g., their multiple computing elements) to be pipelined, such as without data queuing. For example, the computing elements participating in a synchronized flow can act as a single pipelined data path. The flow controller used for a synchronized flow can be or includes pieces (e.g., in...) Figure 5 In the example, this includes the piece N-2, the computing elements on the piece, and / or the piece base or controller on the piece. A synchronous flow controller can initiate threads of work through pipelined pieces. The flow controller can be responsible for initiating threads at a predefined rhythm (referred to herein as spoke counting). For example, if the spoke count is 3, then the piece base can initiate a thread every three clock cycles.

[0125] In this example, a synchronization flow comprises a set of connected compute elements within an HTF cluster 502. Execution of a thread can begin at the flow controller and can proceed from the flow controller to other compute elements (e.g., to other pieces within the same synchronization flow) via the synchronization structure. The flow controller can provide instructions to be executed for the first compute element. By default, the first compute element can provide the same instructions to execute for other connected compute elements. However, in some instances, the flow controller or subsequent compute elements can implement conditional operations that conditionally specify or use alternative instructions. Alternative instructions can be selected by causing the compute element's data path to produce a Boolean conditional value, which can then be used to select between the current compute element's instruction set and the alternative instructions.

[0126] Asynchronous structures can be used to perform operations that occur asynchronously relative to a synchronous stream. Each piece in an HTF cluster 502 can contain an interface to an asynchronous structure. The in interface can contain, for example, a FIFO buffer or a queue (e.g., an asynchronous structure enqueue) to provide storage for messages that cannot be processed immediately. Similarly, the out interface of an asynchronous structure can contain a FIFO buffer or a queue (e.g., an asynchronous structure dequeue) to provide storage for messages that cannot be sent immediately.

[0127] In this example, messages in an asynchronous architecture can be categorized as data messages or control messages. Data messages may contain SIMD-width data values ​​written to tile memory 0 (MEM_0) or memory 1 (MEM_1). Control messages can be configured to control threads to create or release resources or issue external memory references.

[0128] In an HTF cluster 502, tiles can perform various computational operations for the HTF. These operations can be performed by configuring the data paths within the tile and / or its computational elements. In this example, a tile contains two function blocks that perform computational operations for the tile: a multiplication and shift operation block (MS OP) and an arithmetic, logic, and bit operation block (ALB OP). Both blocks can be configured to perform pipelined operations such as multiplication and addition, or shift and addition, etc.

[0129] In one instance, each instance of the memory computing device in the system may have a full set of supported instructions for its operator blocks (e.g., MSOP and ALB OP). In this case, binary compatibility can be achieved across all devices in the system. However, in some instances, it may be helpful to maintain a basic set of functionalities and optional instruction set categories to accommodate various design trade-offs, such as die size. The approach can be similar to how the RISC-V instruction set has a basic set and multiple optional instruction subsets.

[0130] In this example, exemplary tile 504 may include spoke RAM. The spoke RAM can be used to specify which input (e.g., from four SF tile inputs and a base tile input) is the master input for each clock cycle. The spoke RAM read address input may originate from a counter counting from zero to the spoke count minus one. In this example, different spoke counts can be used for different tiles within the same HTF cluster 502 to allow the performance of a particular application or instruction set to be determined by several tiles used in an inner loop or a unique tile instance. In this example, the spoke RAM may specify when a synchronization input will be written to the tile memory, such as when multiple inputs are used for a particular tile instruction and when one of the inputs arrives before the others. The earlier arriving input can be written to the tile memory and can be read later when all inputs are available. In this example, the tile memory can be accessed as a FIFO memory, and the FIFO read and write pointers can be stored in register-based memory areas or structures within the tile memory.

[0131] Figure 6A and Figure 6BAn example of a chiplet system that can be used to implement one or more aspects of CNM system 102 is generally illustrated. As similarly mentioned above, nodes in CNM system 102, or devices within nodes in CNM system 102, can contain chiplet-based architectures or near-memory computing (CNM) chiplets. Packaged memory computing devices can contain, for example, one, two, or four CNM chiplets. Chipslets can be interconnected using high-bandwidth, low-latency interconnects (such as using CPI interfaces). Typically, a chiplet system consists of discrete modules (each “chiplet”) integrated on an interposer layer, and in many instances, interconnected as needed via one or more established networks to provide a system with desired functionality. The interposer layer and the contained chiplets can be packaged together to facilitate interconnection with other components of a larger system. Each chiplet can contain one or more individual integrated circuits (ICs) or “chips,” which may be combined with discrete circuit components and can be coupled to a corresponding substrate for attachment to the interposer layer. Most or all of the chiplets in the system can be individually configured for communication via established networks.

[0132] Chiplets, configured as individual modules within a system, differ from such systems implemented on a single chip containing different device blocks (e.g., intellectual property (IP) blocks) on a substrate (e.g., a single die), such as a system-on-a-chip (SoC) or multiple discrete packaged devices integrated on a printed circuit board (PCB). Generally, chiplets offer better performance (e.g., lower power consumption, reduced latency, etc.) than discrete packaged devices, and provide greater manufacturing benefits than a single die. These manufacturing benefits may include higher yields or reduced development costs and time.

[0133] Chiplet systems may comprise, for example, one or more application (or processor) chiplets and one or more support chiplets. In this document, the distinction between application chiplets and support chiplets is merely a reference to possible design scenarios for chiplet systems. Thus, for example, a synthetic vision chiplet system may include (by way of example only) application chiplets for generating synthetic vision output, and support chiplets such as memory controller chiplets, sensor interface chiplets, or communication chiplets. In typical use cases, synthetic vision designers may design the application chiplets and obtain the support chiplets from other sources. Therefore, design expenditures (e.g., in terms of time or complexity) are reduced by avoiding the design and manufacture of the functionality embodied in the support chiplets.

[0134] Chiplets also support the tight integration of IP blocks that might otherwise be difficult, such as those manufactured using different processing technologies or with different feature sizes (or utilizing different contact technologies or spacing). Therefore, multiple ICs or IC components with different physical, electrical, or communication characteristics can be assembled in a modular manner to provide components with a variety of required functionalities. Chiplet systems also facilitate adaptation to the needs of different larger systems that will be incorporated into chiplet systems. In examples, ICs or other components can be optimized for power, speed, or heat generation for specific functions—as might happen with sensors—making integration with other devices much easier than attempting to integrate them on a single die. Furthermore, by reducing the overall die size, chiplet yields are often higher than those of more complex single-die devices.

[0135] Figure 6A and Figure 6B An example of a chiplet system according to an embodiment is generally shown. Figure 6A This is an illustration of a chiplet system 602 mounted on a peripheral board 604. The chiplet system 602 can be connected to a wider range of computer systems via, for example, Peripheral Component Interconnect (PCIe). The chiplet system 602 includes a package substrate 606, an interposer 608, and four chips: an application chiplet 610, a host interface chiplet 612, a memory controller chiplet 614, and a memory device chiplet 616. Other systems may include numerous additional chipsets to provide additional functionality, as will become apparent from the following discussion. The package of the chiplet system 602 is illustrated with a cap or cover plate 618, but other packaging techniques and structures used for chiplet systems may be used. Figure 6B This is a block diagram for the purpose of clearly labeling the components in a chiplet system.

[0136] Application chip 610 is shown to include chiplet system NOC 620 to support chiplet network 622 for inter-chiplet communication. In an exemplary embodiment, chiplet system NOC 620 may be included on application chip 610. In an example, the communication from the selected supporting chiplet (e.g., host interface chiplet 612, memory controller chiplet 614, and memory device chiplet 616) can be defined. Figure 1 The first NOC 118 in this example allows designers to select an appropriate number of chiplet network connections or switches for the chiplet system NOC 620. In this example, the chiplet system NOC 620 can reside on a single chiplet or within the interposer layer 608. In the example discussed herein, the chiplet system NOC 620 implements a chiplet protocol interface (CPI) network.

[0137] In this example, the chiplet system 602 may include or comprise a portion of the first memory computing node 104 or the first memory computing device 112. That is, various blocks or components of the first memory computing device 112 may include chiplets that can be mounted on the peripheral board 604, the package substrate 606, and the interposer layer 608. The interface components of the first memory computing device 112 may typically include a host interface chiplet 612; the memory and memory control-related components of the first memory computing device 112 may typically include a memory controller chiplet 614; the various accelerator and processor components of the first memory computing device 112 may typically include an application chiplet 610 or an instance thereof, and so on.

[0138] For example, the CPI interface, which can be used for communication between or within chiplets in a system, is a packet-based network that supports virtual channels to enable flexible and high-speed interaction between chiplets. CPI can bridge the chiplet network 622. For instance, the Advanced Extensible Interface (AXI) is a widely used specification for designing intra-chip communication. However, the AXI specification covers a large number of physical design options, such as the number of physical channels, signal timing, and power. Within a single chip, these options are typically selected to meet design goals such as power consumption and speed. However, to achieve flexibility in chiplet systems, adapters such as CPI are used to intersect between various AXI design options that can be implemented in various chiplets. By implementing a mapping from physical channels to virtual channels and encapsulating time-based signaling using packetization protocols, CPI bridges the intra-chiplet network 622.

[0139] CPI can utilize various physical layers to transmit packets. A physical layer may contain simple conductive connections, or it may contain drivers to increase voltage, or otherwise facilitate signal transmission over longer distances. One instance of such a physical layer may contain an Advanced Interface Bus (AIB), which in various instances can be implemented in the intermediate layer 608. The AIB uses source-synchronous data transmission with a forwarding clock to send and receive data. Packets are transmitted across the AIB at Single Data Rate (SDR) or Double Data Rate (DDR) relative to the transmitted clock. The AIB supports various channel widths. Channels can be configured with a symmetrical number of transmit (TX) and receive (RX) inputs / outputs (I / O), or with an asymmetrical number of transmitters and receivers (e.g., all transmitters or all receivers). A channel can act as either the AIB body or slave depending on which chip provides the master clock. The AIB I / O unit supports three clock modes: asynchronous (i.e., non-timing), SDR, and DDR. In various instances, the non-timing mode is used for clocking and some control signals. SDR mode can use a dedicated SDR-only I / O unit or a dual-purpose SDR / DDR I / O unit.

[0140] In this example, the CPI packet protocol (e.g., point-to-point or routable) can use symmetrical receive and transmit I / O units within an AIB channel. The CPI streaming protocol allows for more flexible utilization of AIB I / O units. In this example, the streaming mode AIB channel can configure I / O units as all TX, all RX, or half TX and half RX. The CPI packet protocol can use the AIB channel in SDR or DDR operating modes. In this example, for SDR mode, the AIB channel is configured in increments of 80 I / O units (i.e., 40 TX and 40 RX), and for DDR mode, the AIB channel is configured in increments of 40 I / O units. The CPI streaming protocol can use the AIB channel in SDR or DDR operating modes. In this document, in this example, the AIB channel is configured in increments of 40 I / O units for both SDR and DDR modes. In this example, a unique interface identifier is assigned to each AIB channel. This identifier is used during CPI reset and initialization to determine paired AIB channels across neighboring chiplets. In this example, the interface identifier is a 20-bit value comprising a seven-bit chiplet identifier, a seven-bit column identifier, and a six-bit link identifier. The AIB physical layer uses an AIB out-of-band shift register to transmit the interface identifier. Bits 32 to 51 of the shift register are used to transmit the 20-bit interface identifier in both directions across the AIB interface.

[0141] AIB defines a stacked group of AIB channels as an AIB channel column. An AIB channel column has a certain number of AIB channels, plus auxiliary channels. The auxiliary channels contain signals used for AIB initialization. All AIB channels within a column (except for the auxiliary channels) have the same configuration (e.g., all TX, all RX, or half TX and half RX, and have the same number of data I / O signals). In this example, AIB channels are numbered sequentially in ascending order, starting with the AIB channel adjacent to the AUX channel. The AIB channel adjacent to the AUX channel is defined as AIB channel zero.

[0142] Typically, the CPI interface on a single chiplet may include serialization-deserialization (SERDES) hardware. SERDES interconnects are well-suited for scenarios requiring high-speed signaling and low signal counts. However, SERDES can introduce additional power consumption and longer latency for multiplexing and demultiplexing, error detection or correction (e.g., using block-level cyclic redundancy check (CRC)), link-level retries, or forward error correction. However, when low latency or power consumption is a primary concern for ultra-short-distance chiplet-to-chiplet interconnects, parallel interfaces that allow data transfer with minimal latency can be utilized. CPIs contain elements designed to minimize both latency and power consumption in these ultra-short-distance chiplet interconnects.

[0143] For flow control, CPI employs a credit-based technique. The receiving side of application chip 610, and the sender side of memory controller chip 614, provide credits representing available buffers. In this example, the CPI receiver contains buffers for each virtual channel for a given transmit time unit. Therefore, if the CPI receiver supports five messages and a single virtual channel in time, the receiver has five buffers arranged in five rows (e.g., one row per unit time). If four virtual channels are supported, then the receiver has twenty buffers arranged in five rows. Each buffer holds the payload of one CPI packet.

[0144] When a sender transmits to a receiver, the sender decrements its available credits based on the transmission. Once the receiver has exhausted all its credits, the sender stops sending packets to the receiver. This ensures that the receiver always has an available buffer to store transmissions.

[0145] When the receiver processes the received packet and releases the buffer, it sends the available buffer space back to the sender. The sender can then use this credit to allow the transmission of additional information.

[0146] Figure 6A Examples include chiplet mesh network 624, which uses direct chiplet-to-chiplet technology without requiring a chiplet system NOC 620. Chiplet mesh network 624 can be implemented in CPI or another chiplet-to-chiplet protocol. Chiplet mesh network 624 typically implements a chiplet pipeline, where one chiplet acts as an interface to the pipeline, while other chipslets in the pipeline interface only interface with themselves.

[0147] Alternatively, a dedicated device interface can be used to connect devices to the chiplet, such as one or more industry-standard memory interfaces (e.g., synchronous memory interfaces, such as DDR5, DDR6). Connections between the chiplet system or a single chiplet and external devices (such as larger systems) can be made via a desired interface (e.g., a PCIe interface). In this example, such an external interface can be implemented via a host interface chiplet 612, which, in the depicted example, provides a PCIe interface external to the chiplet system. This type of dedicated chiplet interface 626 is typically used when industry practice or standards have already focused on such interfaces. The illustrated example of connecting a memory controller chiplet 614 to a dual data rate (DDR) interface of a dynamic random access memory (DRAM) memory device chiplet 616 is such an industry practice.

[0148] Among the various possible supporting chiplets, the memory controller chiplet 614 is likely to be present in chiplet systems due to the near-ubiquitous use of storage for computer processing and mature, advanced technologies for memory devices. Therefore, using the memory device chiplet 616 and the memory controller chiplet 614, produced through other technologies, enables chiplet system designers to obtain robust products manufactured by established companies. Typically, the memory controller chiplet 614 provides a memory device-specific interface for reading, writing, or erasing data. Typically, the memory controller chiplet 614 can provide additional features such as error detection, error correction, maintenance operations, or atomic operator execution. For some types of memory, maintenance operations are often specific to the memory device chiplet 616, such as garbage collection in NAND flash memory or storage-class memory, or temperature regulation (e.g., cross-temperature management) in NAND flash memory. In instances, maintenance operations may involve logic-to-physical (L2P) mapping or management to provide an indirection hierarchy between the physical and logical representations of data. In other types of memory, such as DRAM, some memory operations, such as refresh, may be controlled at certain times by the host processor or memory controller, and at other times by the DRAM memory device or logic associated with one or more DRAM devices, such as an interface chip (in this example, a buffer).

[0149] An atomic operator is a data operation that can be performed, for example, by the memory controller chip 614. In other chip systems, atomic operators can be performed by other chips. For example, an application chip 610 can specify an "increment" atomic operator in a command that includes a memory address and possibly an increment value. Upon receiving the command, the memory controller chip 614 retrieves a number from the specified memory address, increments the number by the amount specified in the command, and stores the result. Upon successful completion, the memory controller chip 614 provides the application chip 610 with an indication that the command was successful. Atomic operators avoid data transfer across the chip mesh network 624, resulting in lower latency execution for such commands.

[0150] Atomic operators can be classified as built-in atoms or programmable (e.g., custom) atoms. Built-in atoms are a limited set of operations that are implemented immutably in the hardware. Programmable atoms are small programs that can be executed on programmable atom units (PAUs) (e.g., custom atom units (CAUs)) of the memory controller chip 614.

[0151] The memory device chip 616 may be or contain any combination of volatile memory devices or non-volatile memory. Examples of volatile memory devices include, but are not limited to, random access memory (RAM)—such as DRAM, synchronous DRAM (SDRAM), graphics double data rate type 6 SDRAM (GDDR6 SDRAM), and others. Examples of non-volatile memory devices include (but are not limited to) NAND flash memory, memory-class memory (e.g., phase-change memory or memristor-based technology), ferroelectric RAM (FeRAM), and others. The illustrated example includes memory device chip 616 as a chip; however, the device may be located elsewhere, such as in different packages on peripheral board 604. For many applications, multiple memory device chips may be provided. In examples, these memory device chips may each implement one or more memory technologies and may contain an integrated computing host. In examples, the memory chip may contain multiple stacked memory dies of different technologies, such as one or more static random access memory (SRAM) devices stacked with or otherwise communicating with one or more dynamic random access memory (DRAM) devices. In this example, the memory controller chiplet 614 can be used to coordinate the operation between multiple memory chipsets in the chiplet system 602, for example, using one or more memory chipsets in one or more cache tiers, and using one or more additional memory chipsets as main memory. The chiplet system 602 can contain multiple instances of the memory controller chiplet 614, which can be used to provide memory control functionality for individual hosts, processors, sensors, networks, etc. The chiplet architecture in the illustrated system offers advantages in allowing adaptation to different memory storage technologies and different memory interfaces through updated chiplet configurations, such as without requiring redesign of the rest of the system architecture.

[0152] Figure 7 This section generally illustrates an example of a chiplet-based implementation of a memory computing device according to an embodiment. The example includes an implementation having four near-memory computing (CNM) chiplets, and each of the CNM chiplets may include or include components from… Figure 1 This refers to a portion of the first memory computing device 112 or the first memory computing node 104. Each portion may itself contain or include a corresponding chiplet. Chiplet-based implementations may include or utilize CPI-based intra-system communication, as described above in [the context of...]. Figure 6A and Figure 6B Similar discussions are made in the exemplary chiplet system 602.

[0153] Figure 7An example includes a first CNM package 700 comprising multiple chiplets. The first CNM package 700 includes a first chiplet 702, a second chiplet 704, a third chiplet 706, and a fourth chiplet 708 coupled to a CNM NOC hub 710. Each of the first through fourth chiplets may include instances of the same or substantially the same components or modules. For example, each chiplet may include a corresponding instance of an HTP accelerator, an HTF accelerator, and a memory controller for accessing internal or external memory.

[0154] exist Figure 7 In one example, the first chiplet 702 includes a first NOC hub edge 714 coupled to the CNM NOC hub 710. Other chipsets in the first CNM package 700 similarly include NOC hub edges or endpoints. Switches at the NOC hub edges facilitate chiplet-level or chiplet-system communication via the CNM NOC hub 710.

[0155] The first chiplet 702 may further include one or more memory controllers 716. The memory controllers 716 may correspond to different NOC endpoint switches that interface with the first NOC hub edge 714. In examples, the memory controller 716 may include a memory controller chiplet 614, or a memory controller 130, or a memory subsystem 200, or other memory computing implementations. The memory controller 716 may be coupled to different memory devices, such as a first external memory module 712a or a second external memory module 712b. The external memory modules may include, for example, GDDR6 memory that is selectively accessible by different chipsets in the system.

[0156] The first chiplet 702 may further include a first HTP chiplet 718 and a second HTP chiplet 720, which are coupled to the edge 714 of the first NOC hub via corresponding different NOC endpoint switches. The HTP chiplets may correspond to HTP accelerators, such as those from… Figure 1 The instance of HTP 140, or from Figure 4 An example of an HTP accelerator 400. The HTP chiplet can communicate with the HTF chiplet 722. The HTF chiplet 722 can correspond to an HTF accelerator, such as those from... Figure 1 The instance of HTF142, or from Figure 5 The HTF 500 instance.

[0157] The CNM NOC hub 710 can be coupled to other chiplets or other CNM packaged NOC hub instances via various interfaces and switches. For example, the CNM NOC hub 710 can be coupled to a CPI interface via multiple different NOC endpoints on the first CNM package 700. Each of the multiple different NOC endpoints can be coupled to a different node, for example, outside the first CNM package 700. In an example, the CNM NOC hub 710 can be coupled to other peripheral devices, nodes, or apparatuses using CTCPI or other non-CPI protocols. For example, the first CNM package 700 may include a PCIe scale interface (PCIe / SFI) or a CXL interface (CXL) configured to interface the first CNM package 700 with other apparatuses. In an example, apparatuses coupled to the first CNM package 700 using various CPI, PCIe, CXL, or other architectures can form a common global address space.

[0158] exist Figure 7 In this example, the first CNM package 700 includes a host interface 724 (HIF) and a host processor (R5). The host interface 724 may correspond to, for example, a source... Figure 1 The instance is HIF 120. The host processor or R5 can correspond to the one from... Figure 1 The instance includes an internal host processor 122. The host interface 724 may include a PCI interface for coupling the first CNM package 700 to other external devices or systems. In this instance, operation can be initiated on the first CNM package 700 or a chip cluster within the first CNM package 700 via the host interface 724. For example, the host interface 724 can be configured to command individual HTF chip clusters between the individual chips in the first CNM package 700 to enter and exit power / clock gate modes.

[0159] Figure 8 An exemplary tiled configuration of a memory computing device according to an embodiment is illustrated. Figure 8 In this context, the chiplet instance 800 comprises four instances of different near-memory computing clusters of chiplets, where the clusters are coupled together. Each instance of a near-memory computing chiplet may itself contain one or more constituent chiplets (e.g., host processor chiplets, memory device chiplets, interface chiplets, etc.).

[0160] The 800 small chip instances in the mosaic contain data from... Figure 7An instance of the first CNM package 700 serves as one or more of its near-memory computing (CNM) clusters. For example, a chiplet instance 800 may include a first CNM cluster 802, which includes a first chiplet 810 (e.g., corresponding to the first chiplet 702), a second chiplet 812 (e.g., corresponding to the second chiplet 704), a third chiplet 814 (e.g., corresponding to the third chiplet 706), and a fourth chiplet 816 (e.g., corresponding to the fourth chiplet 708). The chipsets in the first CNM cluster 802 may be coupled to a common NOC hub, which in turn may be coupled to NOC hubs in one or more adjacent clusters (e.g., in the second CNM cluster 804 or the fourth CNM cluster 808).

[0161] exist Figure 8 In this example, the chiplet instance 800 of the lattice includes a first CNM cluster 802, a second CNM cluster 804, a third CNM cluster 806, and a fourth CNM cluster 808. The various CNM chipsets can be configured in a common address space, allowing chipsets to allocate and share resources across different lattices. In this example, chipsets within a cluster can communicate with each other. For instance, the first CNM cluster 802 can be communicatively coupled to the second CNM cluster 804 via an inter-chiplet CPI interface 818, and the first CNM cluster 802 can be communicatively coupled to the fourth CNM cluster 808 via another or the same CPI interface. The second CNM cluster 804 can be communicatively coupled to the third CNM cluster 806 via the same or another CPI interface, and so on.

[0162] In the example, one of the near-memory computing chiplets in chiplet instance 800 may include a host interface (e.g., corresponding to a host interface from...). Figure 7 The host interface 724 of the instance is responsible for workload balancing across the chiplet instance 800 of the chiplet. The host interface facilitates access to host-based command request and response queues, as from outside the chiplet instance 800. The host interface can use one or more hybrid thread processors and hybrid thread architectures in the near-memory computing chiplets of the chiplet instance 800 to dispatch new execution threads.

[0163] Figure 9 This is a diagram illustrating an example of a workflow 900 that performs operations on a reconfigurable computing structure (such as HTF 142 or HTF 500) using more than one synchronous stream. Figure 9 Workflow 900 utilizes dispatch interface 902 and two synchronization streams 901 and 903. Dispatch interface 902 is configured to receive instructions and / or data from other parts of the near-memory computing system, as described herein. In some instances, dispatch interface 902 is arranged in a manner similar to dispatch interface 508 described herein.

[0164] Synchronization flows 901, 903 include corresponding flow controllers 904, 906 and computing elements arranged in ordered synchronization data paths 908, 910, as described herein. Synchronization data paths 908, 910 may, for example, be located between the computing elements of the corresponding synchronization flows 901, 903, as described herein. Flow controllers 904, 906 and synchronization data paths 908, 910 are arranged by reconfigurable computing structures, such as the HTF 142 or HTF 500 described herein.

[0165] The reconfigurable computing architecture can be arranged to implement synchronous flows 901, 903, which include various computing elements such as flow controllers 904, 906 and synchronous data paths 908, 910. For example, flow controllers 904, 906 can be implemented using corresponding tile bases of tiles 510, 512, etc. Synchronous data paths 908, 910 can be implemented among tiles 510, 512, etc., which implement the computing elements described herein. As described herein and Figure 5 The tiles, tile bases, etc. shown may be components of a hybrid threaded structure (HTF) (such as HTF 142), which is a memory computing device (such as... Figure 1 It is part of the memory computing device 112. Other synchronization streams described herein can be arranged similarly using the tiles and tile bases of the HTF described herein.

[0166] exist Figure 9 In workflow 900, domain interface 902 initiates a processing task by providing an asynchronous message 912 to flow controller 904 via an asynchronous structure of the reconfigurable computing architecture. The asynchronous message 912 may be a continuation-type message. Domain interface 902 may initiate the processing task based on its own programming and / or in response to instructions from another component of the near-memory computing system's reconfigurable computing architecture. In some instances, domain interface 902 also configures the reconfigurable computing architecture for the correct kernel configuration used to perform the processing task, which includes, for example, programming the various flow controllers 904, 906 and computing elements to implement synchronous flows 901, 903.

[0167] Asynchronous message 912 can instruct flow controller 904 to initiate a synchronous flow thread. For example, when the spoke RAM selects flow controller 904 for a synchronous data path 908 between components of synchronous flow 901, the flow controller can initiate the thread by providing a first synchronization message to a first computing element of synchronous data path 908. As described herein, the synchronization message can contain data and control information. The first computing element of synchronous data path 908 begins thread execution by performing a processing task and generating a next synchronization message to be sent to a second computing element of synchronous data path 908. The second computing element performs a processing task and generates a third synchronization message, which is sent to a third computing element of synchronous data path 908, and so on, until the thread has been executed at all computing elements of synchronous data path 908.

[0168] The computing elements of synchronous data path 908 are programmed to perform various operations during thread execution, including, for example, operations for configuring synchronous stream 903. For instance, during thread execution at synchronous stream 901, one or more computing elements of synchronous data path 908 send an asynchronous message 914 to the flow controller 906 of synchronous stream 903. Asynchronous message 914 instructs flow controller 906 to start a thread at synchronous stream 903 conditionally upon the occurrence of at least one condition. In this example, one or more conditions include receiving another asynchronous message from synchronous stream 901 indicating that thread completion or starting a thread at synchronous stream 903 is otherwise acceptable. Later, while the thread is executing at synchronous stream 901, the computing elements of synchronous data path 908 send an asynchronous message 916 to the flow controller 906 of synchronous stream 903. Asynchronous message 916 instructs that the thread at synchronous stream 901 has completed and / or reached a point where the corresponding thread at synchronous stream 903 can begin. In some instances, asynchronous message 916 indicates the result of processing performed during thread execution at synchronous stream 903.

[0169] Upon receiving asynchronous message 916 (and if any other conditions of asynchronous message 914 are met), flow controller 906 initiates a thread at synchronous flow 903, for example, by sending a first synchronization message to the first computing element of synchronous data path 910. In some instances, for example, when the thread at synchronous flow 903 completes, the computing element of synchronous data path 910 will send asynchronous message 918 to dispatch interface 902 to indicate that the requested processing task has been completed.

[0170] Figure 10This diagram illustrates an example of a workflow 1000 for performing operations on a reconfigurable computing architecture (such as HTF 142 or HTF 500), where synchronous flows 1001, 1003 interact with a memory interface 1016. In workflow 1000, domain interface 1002 initiates a processing task by providing an asynchronous message 1010 to flow controller 1004 via an asynchronous structure of the reconfigurable computing architecture. Asynchronous message 1010 instructs flow controller 1004 to start a thread at synchronous flow 1001 when one or more conditions occur. (In some instances, message 1010 may instruct flow controller 1004 to begin executing a thread immediately without waiting for any conditions to occur.)

[0171] Flow controller 1004 initiates a thread by providing a synchronization message to a first computing element of synchronous data path 1022, as described herein. During thread execution at synchronous stream 1001, the computing element of synchronous data path 1022 provides an asynchronous message 1012 to flow controller 1006 of synchronous stream 1003. Asynchronous message 1012 may instruct flow controller 1006 to initiate a thread at synchronous stream 1003 upon receiving an indication that data has been received from memory interface 1016.

[0172] As part of thread execution at synchronous stream 1001, another computing element on synchronous data path 1022 provides asynchronous message 1014 to memory interface 1016. Asynchronous message 1014 includes a read request to memory of a near-memory computing system, such as a memory element of a memory computing device containing a reconfigurable computing architecture. In some instances, asynchronous message 1014 may also identify the computing element of synchronous stream 1003 that will receive the result of the read request.

[0173] Memory interface 1016 executes a read request on appropriate external memory to retrieve loaded data. Upon receiving the loaded data, memory interface 1016 directs an asynchronous message 1018 containing the loaded data to the compute element of synchronous stream 1003 (e.g., one of the compute elements in synchronous data path 1024) indicated by asynchronous message 1014. Upon receiving the loaded data, the compute element writes the loaded data to its memory and sends an asynchronous message 1017 to the flow controller 1006 of synchronous stream 1003. Message 1017 indicates that the loaded data has been received. In some instances, memory interface 1016 provides asynchronous messages containing loaded data, similar to message 1018, to multiple compute elements of synchronous stream 1003. Each compute element that has received the loaded data can provide a corresponding asynchronous message, similar to message 1017, to flow controller 1006.

[0174] Upon receiving asynchronous message 1017 (and after any other condition of continuation type message 1012 is met), flow controller 1006 initiates a thread at synchronous flow 1003, for example, by providing a synchronous message to the first compute element of synchronous data path 1024. If more than one compute element of synchronous flow 1003 is to receive loaded data, flow controller 1006 may initiate a thread when it has received all expected asynchronous messages from compute elements that have already received loaded data. In some instances, upon completion of the thread at synchronous flow 1003, one or more compute elements of synchronous data path 1024 (e.g., its last compute element) send an asynchronous message 1020 indicating completion to dispatch interface 1002.

[0175] Figure 11 This diagram illustrates an example of a workflow 1100 that executes loop 1122 using a reconfigurable computational structure such as HTF 142 or HTF 500. Workflow 1100 is illustrated using three synchronization flows 1101, 1103, and 1105. In this example, each iteration of loop 1122 is executed using a single thread at a single synchronization flow 1103. In this example, threads in synchronization flows 1101 and 1105 can execute threads with a common thread ID. Threads at synchronization flow 1103 can have different thread IDs, which can be lower-level thread IDs. For example, if threads in synchronization flows 1101 and 1105 are at level T1, then threads in synchronization flow 1103 can be at level T0.

[0176] Flow controller 1102 initiates a first thread at synchronous stream 1101, for example, by providing a synchronization message to a first computing element of synchronous data path 1108. Executing the thread at synchronous stream 1101 includes sending an asynchronous message 1114 to flow controller 1106 at synchronous stream 1105. Asynchronous message 1114 may be a continuation-type message instructing flow controller 1106 to execute the thread at synchronous stream 1105 after receiving an indication that loop 1122 has completed. In some instances, asynchronous message 1114 also indicates the thread ID of the thread executing at synchronous stream 1101, allowing the corresponding thread at synchronous stream 1105 to use the same thread ID (and associated resources). Executing the thread at synchronous stream 1101 also includes sending an asynchronous message 1116 to flow controller 1104 at synchronous stream 1103. Asynchronous message 1116 may be a loop-type message instructing flow controller 1104 to execute a loop. For example, asynchronous message 1116 may contain an iteration count indicating the number of iterations in the loop.

[0177] Flow controller 1104 initiates a thread for the first iteration of the loop. In some instances, flow controller 1104 generates an iteration index, as described herein. The iteration index can be included in control information propagated, for example, via a synchronization message through a synchronization stream. The computational element of the synchronization data path 1110 of synchronization domain 1103 performs the first iteration of the loop. The computational element of synchronization data path 1110 can use the iteration index generated by flow controller 1104. When the thread completes, one or more computational elements send an asynchronous message 1120 to flow controller 1104 indicating that the first iteration of the loop has been completed. The asynchronous message generated by synchronization stream 1103 (such as message 1118) can carry the iteration index of loop 1122.

[0178] In some instances, after starting the first thread for the first iteration of a loop, the flow controller 1104 continues to start additional threads for executing additional iterations of the loop. For example, if the flow controller 1104 has a three-spoke count, it can start a new thread at the synchronization flow 1103 for a new iteration of the loop every three clock cycles. This can happen without regard to whether the threads of previous loop iterations have already completed propagating through the computational elements of the synchronization data path 1110. In some instances, individual threads implementing different loop iterations can execute simultaneously at the computational elements of the synchronization data path 1110 of the synchronization flow 1103 (e.g., offset by the spoke count).

[0179] When all threads implementing iterations of loop 1122 have completed, flow controller 1104 will receive asynchronous messages from all threads implementing the iterations. Once this has occurred, flow controller 1104 sends asynchronous message 1118 to flow controller 1106, indicating that loop 1122 has completed. Flow controller 1106 can respond by starting a thread at computational element 1112 to implement the next part of programming the reconfigurable computational structure.

[0180] Figure 11 Various exemplary elements are shown that can be combined to implement loops in different contexts. First, although Figure 11 The example shows a loop 1122 initiated by a thread at synchronization stream 1101, but in some instances, the loop can be initiated by another component, such as a dispatch interface. Furthermore, although in Figure 11 In some instances, the completion of loop 1122 triggers another thread at another synchronization stream 1105; however, in others, the completion of the loop can indicate the completion of a processing task. Therefore, in addition to providing... Figure 11 In addition to the other flow controller 1106 shown, or instead provided to, for example Figure 11 Another flow controller 1106 shown can provide a synchronization message 1118 indicating the completion of a loop to the dispatch interface.

[0181] also, Figure 11 Elements that can be used to implement nested loops are shown. For example, a thread at synchronization stream 1101 that initiates loop 1122 may be executing an iteration of an outer loop that calls loop 1122 as an inner loop. Furthermore, in some instances, for example, when a thread at synchronization stream 1101 causes synchronization stream 1103 to execute loop 1122, each thread at synchronization stream 1103 that implements iterations of loop 1122 may cause another synchronization stream (not shown) to implement another loop.

[0182] exist Figure 11 In one instance, a single synchronous stream 1103 is used to execute each iteration of loop 1122. However, in some instances, the execution of the loop iteration can utilize multiple synchronous streams. For example, the thread executing the iteration for loop 1122 may include sending asynchronous messages to a second stream controller and / or memory interface. The second stream controller starts a thread at another synchronous stream (not shown) to execute another portion of the loop iteration. In this instance, when the loop iteration completes, the synchronous stream that has completed the loop iteration sends an asynchronous message to the stream controller that started the loop (e.g., stream controller 1104).

[0183] In some instances, modifications can be made. Figure 11 The arrangement ensures that asynchronous messages of type cyclic 1116 and asynchronous messages of type continuation 1114 are sent by the flow controller 1102, rather than by the computing elements of the synchronous flow 1101, as follows: Figure 11 As shown. This can reduce the number of computational elements used at synchronization data path 1108. In some instances, it can allow the computational elements of synchronization stream 1101 to be omitted.

[0184] Figure 12 This is a diagram illustrating an example of a workflow 1200 that performs a triple nested loop using a reconfigurable computational structure such as HTF 142 or HTF 500. Workflow 1200 is illustrated using a dispatch interface 1202 and four synchronization streams 1203, 1205, 1207, and 1209. Figure 12 Workflow 1200 can be used to execute the following code snippet [4]:

[0185]

[0186]

[0187] exist Figure 12 In this instance, the reconfigurable computational architecture is configured to contain three thread levels, T2, T1, and T0. However, in Figure 12In this instance, dispatch interface 1202 is configured to send a loop-type asynchronous message 1212 directly to flow controller 1204. This allows dispatch interface 1202 to operate effectively as an additional thread level with an available thread ID, referred to herein as thread level T3. Asynchronous message 1212 may indicate the number of iterations of the first-level x loop (in this instance, L iterations).

[0188] Flow controller 1204 responds to a loop-type asynchronous message 1212 by initiating a thread at synchronous stream 1203 for executing iterations of a first-level x-loop. For example, as described herein, flow controller 1204 can generate an iteration index value for each thread initiated at synchronous stream 1203 and provide the iteration index to the computational elements of synchronous stream 1203 via a synchronous message. Threads initiated at synchronous stream 1203 can be assigned a T2-level thread ID. Each thread at synchronous stream 1203 can send an asynchronous message 1214 to flow controller 1206. Asynchronous message 1214 can be a loop-type message instructing flow controller 1206 to initiate a thread at synchronous stream 1205 for executing iterations of a second-level y-loop. Asynchronous message 1214 can indicate the number of iterations of the second-level y-loop (M iterations in this example).

[0189] Flow controller 1206 can determine the corresponding iteration index for each initiated thread and initiate the thread at synchronous stream 1205, for example, by sending one or more synchronous messages, as described herein. Threads initiated at synchronous stream 1205 can be assigned a T1-level thread ID. Each thread at synchronous stream 1205 can send asynchronous message 1216 to flow controller 1208. Asynchronous message 1216 can be a loop-type message instructing flow controller 1208 to initiate a thread at synchronous stream 1207 for execution of iterations of a third-level z-loop. Asynchronous message 1216 can indicate the number of iterations of the third-level z-loop (in this example, N iterations).

[0190] Flow controller 1208 can determine the corresponding iteration index for each initiated thread and initiate the thread at synchronization stream 1207, for example, by sending one or more synchronization messages, as described herein. Threads initiated at synchronization stream 1207 can be assigned a T0-level thread ID. Threads at synchronization stream 1207 can update the accumulated value s, for example, by writing an iteration-specific value of s to the computational element memory location indicated by the iteration index. Threads at synchronization stream 1207 can also send an asynchronous message 1218 to flow controller 1208 upon completion. Asynchronous message 1218 can be a complete and idle message indicating that the thread at synchronization stream 1207 has completed and that the T0 thread ID (and associated resources) used by the completed thread is now available for reassignment to another synchronization stream thread.

[0191] When flow controller 1208 starts N threads and receives N instances of asynchronous message 1218, it can be determined that the instance of the third-level z-loop has completed (e.g., N iterations of the third-level z-loop have completed). Flow controller 1208 sends asynchronous message 1220 to flow controller 1206, indicating that the instance of the third-level z-loop requested by the specific loop asynchronous message 1216 has completed, and that the T1-level thread ID of the thread at the synchronization stream 1205 that sent the loop asynchronous message 1216 is now idle. Because asynchronous message 1220 is sent by flow controller 1208, it may not be necessary to include an additional synchronization stream to end the z-loop, e.g., as... Figure 12 As in the example.

[0192] Flow controller 1206 can start threads for each of the M iterations of the second-level y-loop, and each of these threads can send an asynchronous message 1216 to start an instance of the third-level z-loop. Therefore, flow controller 1206 can eventually receive N instances of asynchronous message 1220. Upon receiving M instances of asynchronous message 1220, flow controller 1206 can determine that all iterations of an instance of the second-level y-loop have been executed. In response, flow controller 1206 sends an asynchronous message 1222 to flow controller 1204. Asynchronous message 1222 can be a complete and idle message indicating that the instance of the second-level y-loop has completed and requesting that the T2-level thread ID of the thread at its synchronization stream 1203 is now idle.

[0193] Flow controller 1208 can start threads for each of the L iterations of the first-level x-loop, and each of these threads can send an asynchronous message 1214 to start an instance of the second-level y-loop. Therefore, flow controller 1206 can eventually receive L instances of asynchronous message 1222. Upon receiving L instances of asynchronous message 1222, flow controller 1208 can determine that all iterations of the first-level x-loop have been executed. In response, flow controller 1208 sends asynchronous message 1224 to flow controller 1204. Asynchronous message 1222 can be a full-type message indicating the completion of a triple-nested loop. In response to asynchronous message 1224 from dispatch interface 1202 and asynchronous message 1213 of the previously continued type, flow controller 1210 starts a thread at synchronous stream 1209 to perform the saving of the accumulated value s. Upon completion, the thread at synchronous stream 1209 sends asynchronous message 1226 to dispatch interface 1202. Asynchronous message 1226 can be a complete and idle type message indicating that the code segment [5] is complete and (single) T3 level thread ID is available.

[0194] Figure 13The diagram illustrates an example of a synchronization stream 1300, which is arranged with a stream controller 1302 configured to generate iterative indices. The synchronization stream 1300 can be arranged by a reconfigurable computational structure, such as the HTF142 or HTF 500 described herein. Figure 13 An exemplary synchronous stream 1300 includes a stream controller 1302 and exemplary computing elements 1304, 1306, 1308, and 1310. One or more tiles, tile bases, or other components of a reconfigurable computing structure can be programmed to implement the stream controller 1302, and each tile (or a portion thereof) can be configured to implement the corresponding computing element 1304, 1306, 1308, and 1310.

[0195] Figure 13 The synchronization process 1300 illustrates an exemplary synchronization data path for transmitting synchronization messages 1328, 1330, 1332, and 1334 between components from process controller 1302 to computing element 1304, between components from computing element 1304 to computing element 1308, between components from computing element 1308 to computing element 1310, and between components from computing element 1310 to computing element 1306.

[0196] Flow controller 1302 may receive asynchronous message 1326, which instructs flow controller 1302 to execute a loop using synchronous flow 1300. Asynchronous message 1326 may contain, for example, an iteration count of the loop to be executed using synchronous flow 1300. If the loop to be executed by synchronous flow 1300 is a second-level or lower-level nested loop, asynchronous message 1326 may also contain information about one or more higher-level loops, including, for example, an iteration count of the higher-level loops, the number of iterations of the higher-level loop that sent asynchronous message 1326, etc. The iteration count is the total number of iterations of the loop to be executed. For example, a loop executed for values ​​of x between 1 and 2048 has a loop count of 2048. The number of iterations is the number of specific iterations performed.

[0197] Referring again to the exemplary code segment above [4], if asynchronous message 1326 instructs flow controller 1302 to implement a third-level z-loop, it may include the number of iterations of the first-level x-loop and the second-level y-loop associated with the execution of the third-level z-loop call. In some instances, as described herein, asynchronous message 1326 may also include iteration counts of the first-level x-loop and / or the second-level y-loop.

[0198] Flow controller 1302 can implement a loop indicated by asynchronous message 1326 by initiating a synchronous flow thread at synchronous flow 1300. The synchronous flow thread corresponds to a different iteration of the requested loop. Flow controller 1302 initiates the synchronous flow thread by sending a synchronization message 1328 to the first computing element 1304 of synchronous flow 1300. Flow controller 1302 may include iteration indexing circuitry 1324, which generates an iteration index for each iteration of the loop. The iteration index generated by iteration indexing circuitry 1324 is provided to the first computing element 1304 of synchronous flow 1300 as part of the synchronization message 1328. The iteration index may be included, for example, in control information.

[0199] The iteration indexing circuit 1324 can determine the iteration index in any suitable manner. In some instances, the iteration indexing circuit 1324 generates the iteration index using the number of iterations and / or iteration count of the loop executed at synchronization flow 1300 and / or the number of iterations and / or iteration count of one or more higher-level loops. Consider an instance where there exists a higher-level loop, referred to in this instance as the outer loop, and the loop incremented by synchronization flow 1300 is referred to as the inner loop. The iteration indexing circuit 1324 can perform a first operation on the number of iterations of the higher-level loop and combine the result with the number of iterations of the inner loop.

[0200] In some instances, operations on the number of outer loop iterations involve applying an outer loop binary mask to the number of outer loop iterations. This may return the masked number of outer loop iterations. In other instances, operations on the number of outer loop iterations involve multiplying the number of outer loop iterations by a constant to produce a product of the number of outer loop iterations. The result of an operation on the number of outer loop iterations can be combined with the masked number of outer loop iterations, for example, through addition, a Boolean OR, or any other suitable function. If there is more than one outer loop (e.g., if the loop executed at synchronization flow 1300 is a third-level or lower loop), the iteration index may consider the iteration counts and / or iteration counts of more than one or all outer loops.

[0201] In some instances, applying the operation to the number of outer loop iterations limits the number of outer loop iterations to an indication of the outer loop iterations associated with synchronization flow 1300. Consider the example of the code snippet above [3], where N equals 2048. In this text, the first-level outer loop of x executes 2048 times, and the second-level inner loop of y also executes 2048 times, meaning that the inner loop executes a total of 16,777,216 iterations. The computational element of synchronization flow 1300 may not have enough computational element memory to store data for all 16,777,216 iterations. Therefore, multiple synchronization flows may be used to execute the inner loops and / or the computational element memory may be written to external memory and cleared after a predetermined number of iterations. Thus, masking and / or multiplying by the number of outer loop iterations can limit the resulting masked number of outer loop iterations to an indication of the iterations of the outer loop to be executed at synchronization flow 1300.

[0202] In some instances, the iteration indexing circuit 1324 generates a unique iteration index for each iteration of the loop to be executed by the synchronization stream 1300, for example, from the number of iterations of the outer loop and the unique number of iterations of the inner loop. In other instances, the iteration circuit 1324 generates an initial iteration index for the first iteration of the loop to be executed by the synchronization stream 1300. The iteration indexing circuit 1324 adds a margin to the initial iteration index for subsequent iterations of the loop. The margin may reflect the amount of space available in the memory of the corresponding computing element that can be used by the computing element in each iteration. In some instances, the margin is or is based on the data width of computing elements 1304, 1306, 1308, 1310. For example, each computing element 1304, 1306, 1308, 1310 may process a predetermined amount of data in each operation. The iteration indexing circuit 1324 finds the iteration index of a loop iteration by adding the data width (or a multiple of the data width) to the iteration index of the previous iteration.

[0203] As described above, the iterative index generated by the iterative indexing circuit 1324 for a given synchronous stream thread corresponds to the loop iteration to be performed by that thread. The flow controller 1302 includes the iterative index in a synchronization message 1328 sent to the computing element 1304 to initiate the synchronous stream 1300. The computing element 1304 uses the iterative index to perform I / O operations. Figure 13In some instances, computing element 1304 uses iteration indexes to perform one or more I / O operations at computing element memory 1314. For example, computing element 1304 may read operand data from computing element memory and / or write result data to computing element memory 1314 at a memory location indicated by the iteration index. For example, computing element 1304 may read data from and / or write data to a memory location whose offset from a base memory location is equal to the iteration index. In some instances, iteration indexes are used for different operations relative to different base memory locations. For example, computing element 1304 may read operand data from a memory location offset by an iteration index amount from a first base location and write result data to another memory location offset by an iteration index amount from a second base location.

[0204] Computing element 1304 can pass the iterative index generated by iterative indexing circuit 1324 to the next computing element 1308 via synchronization message 1330. In this way, the iterative index can continue to be passed between computing elements 1306, 1308, and 1310, which are part of the same synchronization stream thread. Computing elements 1308, 1310, and 1306 can use the iterative index, for example, for I / O operations to the corresponding computing element memories 1318, 1320, and 1316.

[0205] In some instances, one or more computing elements 1304, 1306, 1308, and 1310 can use iterative indexes to reference external memory locations. Figure 13 In one example, computing element 1306 sends asynchronous message 1336 to memory interface 1322. Asynchronous message 1336 may contain an iteration index, which is used to refer to one or more memory locations in external memory corresponding to an iteration of the execution thread. For example, asynchronous message 1336 may contain a write message containing data to be written, and use an iteration index to refer to the memory location used to store the data. In other instances, in addition to a write request or instead of a write request, asynchronous message 1336 may contain a read request. A read request may request data to be stored in external memory at the memory location indicated by the iteration index.

[0206] Figure 14 This is a flowchart illustrating an example of a processing flow 1400, which can be executed by a flow controller to perform a loop at a synchronous flow using an iteration index. The flow controller executing processing flow 1400 is associated with the synchronous flow and starts a thread at the synchronous flow, as described herein. At operation 1402, the flow controller receives an asynchronous message instructing it to perform a loop at the synchronous flow. The asynchronous message contains an iteration count for the loop.

[0207] At operation 1404, the flow controller determines the iteration index of the synchronization flow thread to execute the first iteration of the loop. This may involve, for example, performing an operation on the iteration count of one or more associated outer loops and combining the result with the requested iteration count of the loop, as described herein. In operation 1406, the flow controller initiates the thread at the synchronization flow, for example, by directing a synchronization message to the first computational element of the synchronization flow.

[0208] At operation 1408, the flow controller determines whether there are more pending iterations of the requested loop. If so, at operation 1404, the flow controller generates an iteration index for the next iteration of the loop. This may involve performing an operation on the outer loop iteration count and combining the result with the inner loop iteration count, as described above. In other instances, determining the iteration index for the next iteration at operation 1404 involves adding a margin value to the iteration index of the previous iteration. When the thread ID of the new synchronization flow thread is available, and for example, at the appropriate spoke count, at operation 1406, the flow controller starts the next thread at the synchronization flow to execute the next iteration. If at operation 1408, there are no more pending loop iterations, then at operation 1410, the flow controller waits for its next message.

[0209] Figure 15 This is a diagram illustrating an example of a flow controller and computing elements utilizing a synchronous stream with packaged iterative indexing. In some instances, the tiles or other computing elements of the reconfigurable computing architecture each include multiple parallel processing channels, such as SIMD processing channels. Figure 15 An example including a flow controller 1502 is shown, which has an iterative indexing circuit 1528 to generate an iterative index, as described herein. The flow controller 1502 generates a packaged iterative index and provides the packaged iterative index to an exemplary computing element 1504 via a synchronization message 1508.

[0210] Computing element 1504 includes parallel processing channels 1510, 1512, 1514, and 1516. N parallel processing channels are shown, where N can be any suitable number, including, for example, 8. Furthermore, the width of the parallel processing channels can be any suitable value. In some instances, there are eight 64-bit processing channels, with a total data path width of 512 bits. Other numbers of processing channels and processing channel widths can also be used. In this example, each parallel processing channel uses a different iteration index IIL0, IIL1, IIL2, and IILN. Each channel-specific iteration index can be offset by multiple channel data widths. For example, if the parallel processing channels have an 8-bit data width, then each channel-specific iteration index can be offset by 8 bits from the iteration index of the previous channel.

[0211] The iterative indexing circuit 1528 of the flow controller 1502 can be configured to generate packaged iterative indices. A packaged iterative index contains channel-specific iterative indices for more than one parallel processing channel, packaged into a single value. For example, the iterative indexing circuit 1528 can generate a first iterative index for the first channel 1510 as described herein. The iterative indexing circuit 1528 can generate a corresponding iterative index for the next channel 1512 by adding the channel width to the first channel iterative index. An iterative index for channel 1514 can be generated by adding the channel width to the iterative index for channel 1512, and so on. Index interfaces for all channels 1510, 1512, 1514, and 1516 can be generated and packaged into a single packaged iterative index. In some instances, the width of the packaged iterative index is equal to the combined data width of the parallel processing channels 1510, 1512, and 1514. For example, as shown, packaged iterative interfaces can be provided in parallel to the respective channels 1510, 1512, 1514, and 1516. Each channel 1510, 1512, 1514, 1516 can use its channel-specific iterative index to perform I / O operations to computing element memory 1506 and / or to external memory, as described herein.

[0212] Figure 16 This is a flowchart illustrating an example of a processing flow 1600 that can be executed using a synchronous stream and a memory interface to write the values ​​of a multidimensional array to external memory. Processing flow 1600 comprises three columns 1601, 1603, and 1605. Column 1601 contains operations performed by the flow controller of the synchronous stream. Column 1603 contains operations performed by the computational elements of the synchronous stream. Column 1605 contains operations performed by the memory interface. Figure 16 In an example, a synchronous stream executes an inner loop consisting of two or more nested loops. For instance, referring to the exemplary code snippet [3], a synchronous stream can execute all or part of the inner "y" loop. Figure 12 Examples, for example, similar to Figure 12 Synchronous stream 1207, Figure 16 The synchronous stream can execute all or part of the internal "z" loop.

[0213] At operation 1602, the flow controller receives an instruction to begin executing a loop. This instruction may be, for example, an asynchronous loop message received from a dispatch interface, another flow controller, or other suitable component. The loop message may contain an indication of the iteration count of the inner loop and an indication of the corresponding iteration count of one or more outer loops. For example, if process flow 1600 is used to execute the inner “x” loop of the exemplary code segment [3], the loop message may indicate the iteration count 2048 of the inner “x” loop and the corresponding iteration count of the outer “y” loop that initiated the loop message. If process flow 1600 is used to execute Figure 12 If the instance has an inner "z" loop, then the loop message can indicate the iteration count N of the inner "z" loop, the corresponding iteration count of the outer "y" loop that initiated the loop message, and the corresponding iteration count of another "x" loop that initiated the iteration of the "y" loop.

[0214] At operation 1604, the flow controller initiates a first synchronous flow thread 1610A to execute iterations of an inner loop. To initiate synchronous flow thread 1610A, the flow controller sends a synchronization message 1609A to a first computational element in the synchronous flow computational element. Synchronization message 1609A may contain an indication of the inner loop iterations to be executed by the synchronous flow thread and an indication of the number of iterations of one or more corresponding outer loops. In some instances, the indication of the number of iterations included in synchronization message 1609A is or includes an iteration index, as described herein.

[0215] At operation 1606, the flow controller determines whether the iteration initiated at operation 1604 is the last iteration of the inner loop. If so, at operation 1608, the flow controller sends an asynchronous completion message. The asynchronous completion message can be sent, for example, to the dispatch interface of another synchronous flow and / or the flow controller, as described herein.

[0216] If, at operation 1606, the flow controller determines that there are still iterations to be started, it returns to operation 1604 and starts another thread 1610B to execute the next iteration of the inner loop by sending an additional synchronization message 1609B to the first computational element of the synchronized flow computational element. Synchronization message 1609B may be sent, for example, with an appropriate spoke count to avoid interfering with the previous synchronized flow thread 1610A. In some instances, the flow controller starts a new synchronized flow thread at operation 1604 only if an additional thread ID is available.

[0217] Referring now to synchronous stream threads 1610A, 1610B, and 1610N, the synchronous stream computation element receives messages 1609A, 1609B, and 1609N. At operations 1608A, 1608B, and 1608N, the synchronous stream computation element determines one or more array values ​​for a multidimensional array, for example, as described herein. At operations 1614A, 1614B, and 1614N, the synchronous stream computation element sends corresponding asynchronous write requests 1607A, 1607B, and 1607N to the memory interface. Each corresponding write request 1607A, 1607B, and 1607N contains payload data indicating one or more array values ​​generated by the sending synchronous stream thread, as well as indications of the number of iterations of the inner loop and one or more outer loops. In some instances, write requests 1607A, 1607B, and 1607N contain iteration indices indicating the number of iterations for all relevant loops, for example, similar to those regarding... Figure 13 Write request 1336 as described.

[0218] The memory interface can perform operations 1616, 1618, and 1620 upon receiving each write request 1607A, 1607B, 1607N. At operation 1616, the memory interface receives the write request. At operation 1618, the memory interface determines an external memory offset associated with the number of iterations indicated by the write requests 1607A, 1607B, 1607N. The external memory offset indicates the external memory location where one or more values ​​of an array indicated by the payload data will be written. For example, the memory interface can add the external memory offset to a base external memory location to determine the location used to write the payload data. As described in more detail herein, the offset can be mapped between a multidimensional array and a set of external memory locations.

[0219] At operation 1618, the memory array writes payload data to external memory at one or more external memory locations indicated by the external memory offset determined at operation 1616. For example, the memory interface can add the external memory offset to the base external memory location and write the payload data to the corresponding address.

[0220] Figure 17 This is a diagram illustrating an example of a multidimensional array 1702 that uses external memory offsets to map to external memory location 1704. Figure 17 The arrangement illustrates one way in which the memory interface can determine the external memory offset by the number of iterations containing write requests (such as write requests 1607A, 1607B, 1607N). Figure 17In the example, multidimensional array 1702 is a two-dimensional array with dimensions of 2048 × 2048 or 800H × 800H. For example, a multidimensional array can be generated using a set of two nested loops (an outer loop with an iteration count of 2048 and an inner loop with an iteration count of 2048). For each iteration of the outer loop, the inner loop generates 2048 array values, resulting in a total of 2048 × 2048 or 4,194,304 array values.

[0221] The memory interface determines the external memory offset to convert array 1702 into a set of linear external memory locations relative to the underlying external memory location. Figure 17 In the example, external memory location 1704 begins at the base external memory address 0x10000000 and increases from top to bottom. Figure 17 In this example, the external memory address is represented in hexadecimal.

[0222] As described herein, each value of array 1702 is stored in external memory location 1704, which is offset from the base external memory address corresponding to the value of the cyclic iteration that generated these values. Figure 17 An example of this being done line by line is shown. That is, the bottom row of array 1702 is written starting from the base external memory address 0x10000000, then the next row after the bottom row of array 1702 is written, and so on. However, the technique described herein can also be applied to writing array 1702 column by column.

[0223] The memory interface uses the inner loop number and the outer loop number to generate the external memory offset for a given array value. The row and column values ​​of each value in array 1702 can be characterized by the number of iterations of the outer and inner loops that generated the value. For example, the array value [0,0] can be generated by the first iteration of the outer loop and the corresponding first iteration of the inner loop. The array value [137,820] can be generated by the 138th iteration of the outer loop and the corresponding 820th iteration of the inner loop.

[0224] The memory interface can use a combination of outer and inner loop offsets to generate an external memory offset for array values. In some instances, the outer loop offset can be summarized by the following equation [5]:

[0225] Outer loop_offset = (outer loop_iteration * (data_width * outer loop_dimension))[5]

[0226] In [5], outer loop_offset is the outer loop offset. Outer loop_iteration is the number of iterations of the outer loop. Data_width is the number of external memory locations used to store each value of array 1702. For example, if the external memory has one byte of memory locations, then data_width is the number of bytes in the array values. And, outer loop_dimension is the dimension of the multidimensional array corresponding to the outer loop. Figure 17 In the example, the outer loop dimension of the multidimensional array is 2048, and the data width is eight bytes. Therefore, the outer loop offset of the array values ​​in the first row is zero, and the outer loop offset of the array values ​​in the second row is 16,384 (4000H). In some instances, the inner loop offset can be summarized by equation [6]:

[0227] Inner loop offset = (inner loop iteration * data width)[6]

[0228] In equation [6], inner loop_offset is the inner loop offset, and inner loop_iteration is the number of iterations of the inner loop. In some instances, the total external memory offset is the sum of the inner loop offset and the outer loop offset, given by the following equation [7]:

[0229] External memory offset = outer loop offset + inner loop offset[7]

[0230] Now consider Figure 17 In this example, the underlying external memory location address is 0x10000000, and the data width is eight (e.g., each array value occupies eight one-byte external memory locations). For the array value [0,0], the outer loop offset and the inner loop offset will be zero, and the total loop offset will be zero. Therefore, the memory interface can write the array value [0,0] to the underlying external memory address 0x10000000.

[0231] For the value [0,1] in array 1702, the outer loop offset is still zero, while the inner loop offset is eight, therefore the external memory offset is eight. Thus, array value [0,1] can be stored starting at external memory location 0x10000008, which is eight bytes offset from the base external memory address 0x10000000. For the value [0,2] in array 1702, the outer loop offset is still zero, while the inner loop offset is 16. Therefore, array value [0,2] is written to external memory location 0x10000010, which is sixteen bytes offset from the base external memory address 0x10000000 (remember that external memory address representation is in hexadecimal).

[0232] This pattern runs through rows [0, i] of the array, where the outer loop offset is zero and the inner loop offset is based on the number of inner loop iterations. The last iteration of the inner loop, corresponding to the first iteration 0 of the outer loop, generates the array value [0, 2047], which is the last value of the first row. For the array value [0, 2047], the outer loop offset remains zero, while the inner loop offset is 2047 times 8, or 16,376 (3FF8H). Therefore, the memory interface can store the array value [0, 2047] starting from external memory address 0x10003FF8, which is offset by 16,376 or 3FF8H from the base external memory address.

[0233] Also consider the value in the first column of the second row of array 1702 (value [1,0]). For this array value, the outer loop offset is now 16,384 (4000H). The inner loop offset is zero, so the total offset is 16,384 (4000H). Therefore, the memory interface writes the value [1,0] of array 1702 starting from external memory address 0x1004000, as... Figure 17 As shown. The memory interface can similarly locate the external memory offset for the other values ​​in array 1702, as shown in the figure.

[0234] In some instances, the array values ​​of a multidimensional array can be determined by a synchronous stream having computational elements with multiple parallel processing channels. In these arrangements, each synchronous stream thread can execute multiple iterations of its inner loop in parallel. Therefore, the payload data generated by each synchronous stream contains multiple values ​​for array 1702.

[0235] Consider an instance where the computational elements of a synchronous stream comprise eight parallel processing channels, each generating an eight-byte array value. Thus, each synchronous stream thread can simultaneously evaluate eight iterations of its inner loop. The payload data generated by each thread can contain eight array values ​​generated for the eight iterations, each array value comprising eight bytes. Therefore, the payload data from the first synchronous stream thread can contain the values ​​[0,0], [0,1], [0,2], [0,3], [0,4], [0,5], [0,6], and [0,7] of array 1702. The payload data generated by the next synchronous stream thread can contain the values ​​[0,8], [0,9], [0,10], [0,11], [0,12], [0,13], [0,14], and [0,15] of array 1702, and so on. Therefore, executing 2048 iterations of the inner loop to generate a single row of values ​​for array 1702 can involve executing only 256 synchronous stream threads.

[0236] In instances where the synchronous stream computing element includes multiple parallel processing channels, the memory interface can take this into account by considering the number of threads and the number of parallel processing channels when searching for the inner loop offset. An example is given by the following equation [7]:

[0237] Outer loop offset = (inner loop thread count * data width * number of parallel processing channels) [7]

[0238] In Equation [7], the inner loop thread count is the number of threads in the synchronous stream. For example, the thread that returns array values ​​[i,0], [i,1], [i,2], [i,3], [i,4], [i,5], [i,6], and [i,7] will be thread 0, the thread that returns array values ​​[i,8], [i,9], [i,10], [i,11], [i,12], [i,13], [i,14], and [i,15] will be thread 1, and so on. The number of parallel processing channels is the number of parallel processing channels in the computing element. When using parallel processing channels, the memory interface can use Equation [7] to generate the inner loop offset, and Equation [5] to generate one or more outer loop offsets.

[0239] In some instances, the external memory offsets described herein can be used in arrangements where different synchronization streams are used to generate different values ​​for a multidimensional array. For example, different synchronization streams can generate external memory offsets, as described herein, and apply these offsets to different base locations. See again Figure 17 Consider an example of array 1702 with 64 columns indicated by 1706. Column 1706 corresponds to column numbers 129-191. Using the inner and outer loop offsets described herein, and using 0x10000400 as the base external memory location, a single synchronization stream can perform sixty-four iterations of the inner loop corresponding to columns 129-191. Other synchronization streams can similarly find the array values ​​of different columns of array 1702 using other suitable base external memory locations.

[0240] It should be understood that, despite Figure 17The example shows a two-dimensional array, but in some instances, the technique can be used for arrays with additional dimensions. For instance, consider a set of three nested loops used to generate an a×b×c array. The first-level loop will execute a times. The second-level loop will execute b times, and the third-level or inner loop will execute c times. In this example, the memory interface will find the outer memory offset for each iteration of the inner loop by summing the outer loop offset of the first-level loop, the outer loop offset of the second-level loop, and the inner loop offset of the third-level or inner loop. In this example, the outer loop dimension of the first-level loop will be a, and the outer loop dimension of the second-level loop will be b. Additional dimensions can be added to the output array, for example, by adding additional outer loop offsets as shown.

[0241] In some instances, write requests generated by a synchronous stream thread can use iteration indices to indicate the number of inner and outer loop iterations, as described herein. In these instances, the memory interface can be configured to translate the iteration index into an inner loop portion indicating the number of inner loop iterations and an outer loop portion indicating the number of outer loop iterations. This can be achieved, for example, by masking and (if necessary) division (or bit shifting). Consider an exemplary eight-bit iteration index where the outer loop iteration count is represented by the four most significant bits and the inner loop iteration count by the four least significant bits. To find the inner loop offset, the memory interface can apply an inner loop mask that masks the four most significant bits. To find the outer loop offset, the memory interface can apply an outer loop mask that masks the four least significant bits and right-shifts the remaining bits by four bits to produce the outer loop iteration count.

[0242] Consider another example implemented by a synchronous stream with computational elements having multiple parallel processing channels, as described herein. Recall that in such an arrangement, the number of synchronous stream threads performing inner loop iterations is less than the number of inner loop iterations. Therefore, the number of bits in the iteration index used to indicate the number of threads can be less than the number of bits used to indicate the total number of inner loop iterations.

[0243] Figure 18 This is a flowchart illustrating an example of a processing flow 1800 that can be executed by a memory interface to generate an external memory offset. For example, processing flow 1800 illustrates an exemplary manner in which the memory interface can perform operation 1618 of processing flow 1600. At operation 1802, the memory interface receives a write request including payload data and other indications such as an iteration index or the number of outer loop iterations and the number of inner loop iterations.

[0244] At operation 1804, the memory interface determines the outer loop offset. In an instance where an iteration index is used, the memory interface may apply appropriate masking and shifting of the iteration index to generate the number of outer loop iterations. As described herein, the number of outer loop iterations can then be used to generate the outer loop offset. If there is more than one outer loop, the memory interface may generate more than one outer loop offset, as described herein. At operation 1806, the memory interface determines the inner loop offset. In an instance where an iteration index is used, the memory interface may apply appropriate masking and shifting of the iteration index to generate the number of inner loop iterations and / or the number of inner loop threads. As described herein, the number of inner loop iterations and / or the number of inner loop threads can be used to generate the inner loop offset.

[0245] At operation 1808, the memory interface uses the inner loop offset determined at operation 1806 and one or more outer loop offsets determined at operation 1804 to determine the external memory offset for the write request. For example, the memory interface may sum the inner offset and one or more outer offsets. At operation 1810, the memory interface begins writing payload data at the external memory location indicated by the offset. For example, the memory interface may determine the address of the external memory location indicated by the offset by adding the offset to the underlying external memory location, as described herein.

[0246] Figure 19A block diagram of an exemplary machine 1900 is shown, in which any one or more techniques (e.g., methods) discussed herein may be implemented, or in which any one or more techniques (e.g., methods) discussed herein may be implemented. As described herein, an instance may contain, or may be operated by, logic or components or mechanisms in machine 1900. A circuit system (e.g., a processing circuit system) is a collection of circuits implemented in a tangible entity of machine 1900 containing hardware (e.g., simple circuits, gates, logic, etc.). Circuit system membership may be flexible over time. A circuit system contains components that can perform a particular operation individually or in combination during operation. In an instance, the hardware of the circuit system may be designed in an immutable manner to perform a particular operation (e.g., hardwired). In an instance, the hardware of the circuit system may contain variably connected physical components (e.g., execution units, transistors, simple circuits, etc.) containing machine-readable media that are physically modified (e.g., magnetic ground, electrical ground, movable placement of invariant aggregate particles, etc.) to encode instructions for a particular operation. When connecting physical components, the fundamental electrical properties of the hardware components are altered, for example, from an insulator to a conductor, or vice versa. Instructions enable embedded hardware (e.g., an execution unit or loading mechanism) to hardware-create components of a circuit system via variable connections to perform specific operations when in operation. Thus, in an example, a machine-readable media element is part of the circuit system or communicatively coupled to other components of the circuit system during device operation. In an example, any one of the physical components can be used in more than one component of more than one circuit system. For example, in operation, an execution unit may be used at one point in time in a first circuit of a first circuit system and reused by a second circuit of the first circuit system, or reused at a different time by a third circuit of a second circuit system. Further examples of these components of machine 1900.

[0247] In alternative embodiments, machine 1900 may operate as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, machine 1900 may operate as a server machine, a client machine, or both in a server-client network environment. In an example, machine 1900 may act as a peer-to-peer (P2P) (or other distributed) network environment. Machine 1900 may be a personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), mobile phone, network appliance, network router, switch, or bridge, or any machine capable of executing (in sequence or otherwise) instructions specifying actions to be taken by said machine. Furthermore, although only a single machine is shown, the term "machine" should also be considered as encompassing any collection of machines that individually or jointly execute one or more sets of instructions to perform any one or more of the methodologies discussed herein, such as cloud computing, software as a service (SaaS), and other computer cluster configurations.

[0248] Machine 1900 (e.g., a computer system) may include a hardware processor 1902 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a hardware processor core, or any combination thereof), main memory 1904, static memory 1906 (e.g., memory or storage device for firmware, microcode, basic input / output (BIOS), unified extensible firmware interface (UEFI), etc.), and mass storage device 1908 (e.g., a hard disk drive, tape drive, flash memory device, or other block device), some or all of which may communicate with each other via interconnect 1930 (e.g., a bus). Machine 1900 may further include a display device 1910, an alphanumeric input device 1912 (e.g., a keyboard), and a user interface (UI) navigation device 1914 (e.g., a mouse). In an example, the display device 1910, the input device 1912, and the UI navigation device 1914 may be a touchscreen display. Machine 1900 may additionally include a mass storage device 1908 (e.g., a drive unit), a signal generation device 1918 (e.g., a speaker), a network interface device 1920, and one or more sensors 1916, such as a Global Positioning System (GPS) sensor, a compass, an accelerometer, or other sensors. Machine 1900 may include an output controller 1928, which is serially (e.g., Universal Serial Bus (USB)), parallelly, or otherwise wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connected to communicate with or control one or more peripheral devices (e.g., a printer, a card reader, etc.).

[0249] The registers of the hardware processor 1902, main memory 1904, static memory 1906, or mass storage device 1908 may be or contain a machine-readable medium 1922, on which one or more sets of data structures or instructions 1924 (e.g., software) are stored, embodying or used by any one or more of the techniques or functions described herein. During execution via machine 1900, the instructions 1924 may also reside wholly or at least partially in any register of the hardware processor 1902, main memory 1904, static memory 1906, or mass storage device 1908. In this example, one or any combination of the hardware processor 1902, main memory 1904, static memory 1906, or mass storage device 1908 may constitute the machine-readable medium 1922. Although machine-readable media 1922 is shown as a single medium, the term "machine-readable media" can include a single medium or multiple media configured to store one or more instructions 1924 (e.g., a centralized or distributed database, or associated caches and servers).

[0250] The term "machine-readable media" can include any media capable of storing, encoding, or carrying any one or more of the technologies disclosed herein that are executed by machine 1900 and cause machine 1900 to perform, or any media capable of storing, encoding, or carrying data structures used by or associated with such instructions. Examples of non-limiting machine-readable media can include solid-state memory, optical media, magnetic media, and signals (e.g., radio frequency signals, other photon-based signals, sound signals, etc.). In examples, non-transitory machine-readable media includes machine-readable media having a plurality of particles having invariant (e.g., rest) mass and therefore being composed of matter. Thus, non-transitory machine-readable media is machine-readable media that does not contain transiently propagating signals. Specific examples of non-transitory machine-readable media can include: non-volatile memory, such as semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)) and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0251] In this example, information stored or otherwise provided on machine-readable medium 1922 may represent instruction 1924, such as instruction 1924 itself or a format from which instruction 1924 can be derived. This format from which instruction 1924 can be derived may include source code, encoded instructions (e.g., in compressed or encrypted form), encapsulated instructions (e.g., split into multiple encapsulations), etc. The information representing instruction 1924 in machine-readable medium 1922 may be processed by a processing circuitry system into instructions to perform any of the operations discussed herein. For example, deriving instruction 1924 from information (e.g., by a processing circuitry system) may include: compiling (e.g., from source code, object code, etc.), interpreting, loading, organizing (e.g., dynamically or statically linking), encoding, decoding, encrypting, decrypting, encapsulating, decapsulating, or otherwise manipulating information into instruction 1924.

[0252] In an example, the derivation of instruction 1924 may involve assembling, compiling, or interpreting information (e.g., by processing a circuit system) to create instruction 1924 from some intermediate or preprocessed format provided by machine-readable media 1922. Information, when provided in multiple parts, can be combined, decapsulated, and modified to create instruction 1924. For example, information may be contained in multiple compressed source code packages (or object code, or binary executable code, etc.) on one or more remote servers. The source code packages may be encrypted when transmitted over a network and may be decrypted, decompressed, assembled (e.g., linked), and compiled or interpreted at the local machine (e.g., compiled or interpreted into a library, a standalone executable, etc.), and executed by the local machine.

[0253] Instruction 1924 can be further transmitted or received via network interface device 1920 using a communication network 1926 through a transmission medium, utilizing any of a variety of transport protocols (e.g., frame relay, Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), etc.). Exemplary communication networks may include local area networks (LANs), wide area networks (WANs), packet data networks (e.g., the Internet), mobile phone networks (e.g., cellular networks), conventional telephone (POTS) networks, and wireless data networks (e.g., referred to as…). The Institute of Electrical and Electronics Engineers (IEEE) 802.11 family of standards, known as The network interface device 1920 may include one or more physical jacks (e.g., Ethernet, coaxial, or telephone jacks) or one or more antennas for connection to the network 1926. In an example, the network interface device 1920 may include multiple antennas for wireless communication using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) technologies. The term "transmission medium" should be considered as any intangible medium capable of storing, encoding, or carrying instructions for execution by the machine 1900, and containing digital or analog communication signals or other intangible media used to facilitate communication of such software. The transmission medium is a machine-readable medium.

[0254] To better illustrate the methods and apparatus described herein, a set of non-limiting exemplary embodiments are set forth below as numbered examples.

[0255] The above detailed description includes reference to the accompanying drawings, which form a part of the detailed description. The drawings illustrate, by means of illustration, specific embodiments in which the invention may be practiced. These embodiments are also referred to herein as “examples.” Such examples may include elements other than those shown or described. However, the inventors also contemplate that only examples of those elements shown or described are provided herein. Furthermore, the inventors also contemplate examples (or examples of those elements) using any combination or arrangement of those elements shown or described relative to a particular example (or one or more aspects thereof) or relative to other examples (or one or more aspects thereof) shown or described herein.

[0256] In this document, as is common in patent documents, the terms “a” or “an” are used to include one or more, regardless of any other instances or uses of “at least one” or “one or more”. In this document, unless otherwise indicated, the term “or” is used to refer to a non-exclusive “or”, such that “A or B” can include “A but not B,” “B but not A,” and “A and B.” In the appended claims, the terms “including” and “in which” are used as concise English equivalents to the corresponding terms “comprising” and “wherein.” Furthermore, in the appended claims, the terms “comprising” and “including” are open-ended, meaning that a system, apparatus, article of manufacture, or process that includes elements other than those listed after such terms in the claims is still considered to fall within the scope of the claims. Additionally, in the following claims, the terms “first,” “second,” and “third,” etc., are used merely as labels and are not intended to impose numerical requirements on their objects.

[0257] The above description is intended to be illustrative and not restrictive. For example, the examples described above (or one or more aspects thereof) can be used in combination with each other. Other embodiments may be used by those skilled in the art upon review of the above description. This document is submitted under the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Similarly, in the above detailed description, various features may be grouped together to simplify this disclosure. This should not be construed as expecting that any unclaimed disclosed features are necessary for any claim. In fact, the subject matter of the invention may lie in fewer features than all of the particular disclosed embodiments. Therefore, the appended claims are hereby incorporated into the detailed description, wherein each claim exists independently as a separate embodiment, and such embodiments are contemplated to be combined or arranged in various ways. The scope of the invention should be determined by reference to the appended claims and the full scope of the equivalents granted by the claims.

Claims

1. An apparatus comprising: A reconfigurable computing architecture includes multiple computing elements and at least one flow controller. The reconfigurable computing architecture is arranged to include a first synchronous flow, which includes one of the at least one flow controller and multiple of the multiple computing elements. The first synchronous flow is programmed using stored instructions to perform operations including: Execute a first synchronous stream thread, the first synchronous stream thread being used to perform at least a portion of a loop iteration for generating at least one value of a multidimensional array; A write request is sent from the computing element of the first synchronous stream to the memory interface. The write request includes an iteration index indicating the loop iteration and payload data including at least one value of the multidimensional array. and A memory interface, which is programmed using stored instructions to perform operations including: The external memory offset is determined using the iterative index; and The payload data is written to the external memory location indicated by the external memory offset.

2. The device of claim 1, wherein the loop iteration is an inner loop iteration associated with an outer loop iteration, the iteration index indicates the inner loop iteration and the outer loop iteration, and the memory interface is further configured to perform operations including: Determine the outer loop offset; and The inner loop offset is determined, and the external memory offset is based at least in part on the outer loop offset and the inner loop offset.

3. The device of claim 2, further comprising using a first dimension of the multidimensional array and the iterative index to determine the outer loop offset.

4. The device of claim 2, wherein the memory interface is further configured to perform operations including: Apply an outer loop mask to the iterative index to generate the outer loop portion of the iterative index; and Determine the product of the outer loop portion of the iterative index and the first dimension of the multidimensional array.

5. The device of claim 4, wherein the memory interface is further configured to perform an operation including determining the product of the outer loop portion of the iterative index, the data width of the at least one value of the multidimensional array, and the first dimension of the multidimensional array, the outer loop offset being based on the product of the outer loop portion of the iterative index, the data width of the at least one value of the multidimensional array, and the first dimension of the multidimensional array.

6. The device of claim 2, wherein the memory interface is further configured to perform an operation including determining the inner loop offset using the data width of the at least one value of the multidimensional array.

7. The device of claim 2, wherein the memory interface is further configured to perform an operation including applying an inner loop mask to the iterative index to generate an inner loop portion of the iterative index, the inner loop offset being at least partially based on the inner loop portion of the iterative index.

8. The device of claim 7, wherein the computing element comprises a plurality of parallel processing channels, and the payload data comprises at least one value of the multidimensional array, and the memory interface is further configured to perform operations including: Determine the product of the inner loop portion of the iterative index, the number of parallel processing channels, and the data width of at least one value of the multidimensional array.

9. A method comprising: A first synchronous stream thread is executed by a first synchronous stream of a reconfigurable computational structure. The first synchronous stream thread is used to perform at least a portion of a loop iteration to generate at least one value of a multidimensional array. The reconfigurable computational structure includes a plurality of computational elements and at least one stream controller. The reconfigurable computational structure is arranged to include the first synchronous stream, which includes the at least one stream controller of the reconfigurable computational structure and a plurality of the plurality of computational elements. A write request is sent from the computing element of the first synchronous stream to the memory interface. The write request includes an iteration index indicating the loop iteration and payload data including at least one value of the multidimensional array. The external memory offset is determined by the iterative index used by the memory interface; and The payload data is written to the external memory location indicated by the external memory offset via the memory interface.

10. The method of claim 9, wherein the loop iteration is an inner loop iteration associated with an outer loop iteration, the iteration index indicating the inner loop iteration and the outer loop iteration, the method further comprising: Determine the outer loop offset; and The inner loop offset is determined, and the external memory offset is based at least in part on the outer loop offset and the inner loop offset.

11. The method of claim 10, further comprising using a first dimension of the multidimensional array and the iteration index to determine the outer loop offset.

12. The method of claim 10, further comprising: Apply an outer loop mask to the iterative index to generate the outer loop portion of the iterative index; and Determine the product of the outer loop portion of the iterative index and the first dimension of the multidimensional array.

13. The method of claim 12, the method further comprising determining the product of the outer loop portion of the iterative index, the data width of the at least one value of the multidimensional array, and the first dimension of the multidimensional array, the outer loop offset being based on the product of the outer loop portion of the iterative index, the data width of the at least one value of the multidimensional array, and the first dimension of the multidimensional array.

14. The method of claim 10, further comprising using the data width of the at least one value of the multidimensional array to determine the inner loop offset.

15. The method of claim 10, further comprising applying an inner loop mask to the iterative index to generate an inner loop portion of the iterative index, the inner loop offset being at least partially based on the inner loop portion of the iterative index.

16. The method of claim 15, wherein the computing element comprises a plurality of parallel processing channels, and the payload data comprises at least one value of the multidimensional array, the method further comprising: Determine the product of the inner loop portion of the iterative index, the number of parallel processing channels, and the data width of at least one value of the multidimensional array.

17. A machine-readable medium including instructions that, when executed by a processor, cause the processor to perform operations including: A first synchronous stream thread is executed by a first synchronous stream, the first synchronous stream thread being used to perform at least a portion of a loop iteration to generate at least one value of a multidimensional array; A write request is sent from the computing element of the first synchronous stream to the memory interface. The write request includes an iteration index indicating the loop iteration and payload data including at least one value of the multidimensional array. The external memory offset is determined by the iterative index used by the memory interface; and The payload data is written to the external memory location indicated by the external memory offset via the memory interface.

18. The machine-readable medium of claim 17, wherein the cyclic iteration is an inner cyclic iteration associated with an outer cyclic iteration, the iteration index indicating the inner cyclic iteration and the outer cyclic iteration, further comprising: Determine the outer loop offset; and The inner loop offset is determined, and the external memory offset is based at least in part on the outer loop offset and the inner loop offset.

19. The machine-readable medium of claim 18, further comprising using a first dimension of the multidimensional array and the iterative index to determine the outer loop offset.

20. The machine-readable medium of claim 18, further comprising: Apply an outer loop mask to the iterative index to generate the outer loop portion of the iterative index; and Determine the product of the outer loop portion of the iterative index and the first dimension of the multidimensional array.

Images