Memory computing systems and related methods

Abstract

Various examples relate to memory computing systems and related methods. The first flow controller may launch a first thread at a first synchronization flow to execute a first portion of a first iteration of the loop. A second flow controller may receive a first asynchronous message instructing the second flow controller to initiate a first thread at a second synchronization flow to perform a second portion of the first iteration. The second flow controller may determine that the first iteration of the loop is a last iteration of the loop to be executed, and launch the first thread at the second synchronization flow if a last iteration flag is set.

Description

[0001] Divisional application

[0002] This application is a divisional application of the invention patent application with the application date of December 29, 2021, application number 202111637668.3, and invention name “Memory Computing System and Related Methods”.

[0003] Priority application

[0004] This application claims priority to U.S. Provisional Application Serial No. 63 / 132,779, filed on December 31, 2020, which is incorporated herein by reference in its entirety, and U.S. Application Serial No. 17 / 405,371, filed on August 18, 2021, which is incorporated herein by reference in its entirety. Technical Field

[0005] The present application relates generally to computing architectures, and more particularly to loop execution in reconfigurable computing structures. Background Art

[0006] Various computer architectures (e.g., von Neumann architecture) typically use shared memory for data and a bus for accessing the shared memory, arithmetic units, and program control units. However, moving data between the processor and memory can require significant time and energy, which in turn can limit the performance and capacity of computer systems. Given these limitations, new computing architectures and devices are needed to improve computing performance beyond the practice of transistor scaling (i.e., Moore's Law). Summary of the Invention

[0007] According to one aspect of the present application, a method is provided. The method includes: launching, by a first flow controller of a first synchronous flow, a first thread at the first synchronous flow, the first thread at the first synchronous flow executing a first portion of a first iteration of a loop, the loop having multiple iterations, the first synchronous flow being configured by a reconfigurable computing structure comprising a plurality of flow controllers and a plurality of computing elements; receiving, by a second flow controller of a second synchronous flow, a first asynchronous message instructing the second flow controller to launch the first thread at the second synchronous flow to execute a second portion of the first iteration, the second portion of the first iteration executing after the first portion of the first iteration; determining, by the second flow controller, that the first iteration of the loop is a last iteration of the loop to be executed; and in response to determining that the first iteration of the loop is the last iteration of the loop to be executed, launching, by the second flow controller, the first thread at the second synchronous flow, the launching comprising providing a first synchronization message to a first computing element of the second synchronous flow, the first synchronization message comprising a last iteration flag indicating that the first iteration is the last iteration of the loop to be executed.

[0008] According to another aspect of the present application, a reconfigurable computing structure is provided. The reconfigurable computing structure includes: a first synchronous flow, the first synchronous flow including a first flow controller and at least one computing element, the first flow controller being programmed to start a first thread at the first synchronous flow to execute a first portion of a first iteration of a loop, the loop having multiple iterations; a second synchronous flow, the second synchronous flow including a second flow controller and at least one computing element, the second flow controller being programmed to perform operations including: receiving a first asynchronous message, the first asynchronous message instructing the second flow controller to start a first thread at the second synchronous flow to execute a second portion of the first iteration, the second portion of the first iteration being executed after the first portion of the first iteration; and determining that the first iteration of the loop is the last iteration of the loop to be executed; and in response to determining that the first iteration of the loop is the last iteration of the loop to be executed, starting the first thread at the second synchronous flow, the starting including providing a first synchronization message to a first computing element of the second synchronous flow, the first synchronization message including a last iteration flag indicating that the first iteration is the last iteration of the loop to be executed.

[0009] According to yet another aspect of the present application, a machine-readable medium is provided. The machine-readable medium includes instructions stored thereon that, when executed at a reconfigurable computing fabric, cause the reconfigurable computing fabric to perform operations comprising: launching, by a first flow controller of a first synchronous flow, a first thread at the first synchronous flow, the first thread at the first synchronous flow executing a first portion of a first iteration of a loop, the loop having a plurality of iterations, the first synchronous flow being configured by the reconfigurable computing fabric comprising a plurality of flow controllers and a plurality of computing elements; receiving, by a second flow controller of a second synchronous flow, a first asynchronous message instructing the second flow controller to launch the first thread at the second synchronous flow to execute a second portion of the first iteration, the second portion of the first iteration executing after the first portion of the first iteration; determining, by the second flow controller, that the first iteration of the loop is a last iteration of the loop to be executed; and in response to determining that the first iteration of the loop is the last iteration of the loop to be executed, launching, by the second flow controller, the first thread at the second synchronous flow, the launching comprising providing a first synchronization message to a first computing element of the second synchronous flow, the first synchronization message comprising a last iteration flag indicating that the first iteration is the last iteration of the loop to be executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] To easily identify the discussion of any particular element or act, the most significant digit(s) in a reference number refers to the figure number in which the element is first introduced.

[0011] Figure 1 A first example of a first memory computing device in the context of a memory computing system according to an embodiment is generally shown.

[0012] Figure 2 An example of a memory subsystem of a memory computing device according to an embodiment is generally shown.

[0013] Figure 3 An example of a programmable atomic cell for a memory controller according to an embodiment is generally shown.

[0014] Figure 4 An example of a hybrid thread processor (HTP) accelerator of a memory computing device according to an embodiment is shown.

[0015] Figure 5 An example of a representation of a hybrid thread structure (HTF) of a memory computing device according to an embodiment is shown.

[0016] Figure 6A An example of a chiplet system according to an embodiment is generally shown.

[0017] Figure 6B Overall, the diagram shows the Figure 6A A block diagram of the various components of an example chiplet system.

[0018] Figure 7 An example of a chiplet-based implementation for a memory computing device according to an embodiment is generally shown.

[0019] Figure 8 An exemplary tiling of chiplets of a memory computing device according to an embodiment is shown.

[0020] Figure 9 is a diagram illustrating an example of a workflow for performing operations on a reconfigurable computing fabric using more than one synchronization stream.

[0021] Figure 10 is a diagram illustrating one example of a workflow for performing operations at a reconfigurable computing fabric, where synchronization flows interact with a memory interface.

[0022] Figure 11 is a diagram illustrating an example of a workflow for executing a loop using a reconfigurable computing fabric.

[0023] Figure 12 is a flow chart illustrating one example of a process flow that may be implemented using a reconfigurable computing structure to execute a loop using a last iteration flag.

[0024] Figure 13 is a diagram illustrating an example of a workflow for executing a loop with a reconfigurable computing structure using a last iteration flag.

[0025] Figure 14 A block diagram is shown of an exemplary machine with which, in which, or by which any one or more of the techniques (eg, methodologies) discussed herein may be implemented. DETAILED DESCRIPTION

[0026] The latest advances in materials, devices, and integration technologies can be used to provide memory-centric computing topologies. Such topologies can achieve improvements in computing efficiency and workload throughput, for example, for applications limited by size, weight, or power requirements. The topology can be used to promote low-latency computing near or inside memory or other data storage elements. These methods can be particularly suitable for various computationally intensive operations with sparse lookups, such as in transform calculations (e.g., fast Fourier transform calculations (FFT)), or in applications such as neural networks or artificial intelligence (AI), financial analysis, or simulation or modeling for computational fluid dynamics (CFD), Engineer Enhanced Acoustic Simulator (EASE), Simulation Program with Integrated Circuit Emphasis (SPICE), etc.

[0027] The systems, devices and methods discussed herein may include or use a memory computing system with a processor or processing power that is provided in, near or integrated with a memory or data storage component. Such systems are generally referred to herein as neighboring memory computing (CNM) systems. A CNM system may be a node-based system in which individual nodes in the system are coupled using a system-level architecture. Each node may include or use a dedicated or general-purpose processor and a user-accessible accelerator, with a customized computing architecture for intensive operations, particularly in environments where a higher cache miss rate is expected.

[0028] In an example, each node in a CNM system can have one or more host processors. In each node, a dedicated hybrid thread processor can occupy a discrete endpoint of the network on chip. The hybrid thread processor can access some or all of the memory in a specific node of the system, or the hybrid thread processor can access the memory across a network of multiple nodes via a system-level structure. The custom computing structure or hybrid thread structure on each node can have its own processor or accelerator and can operate at a higher bandwidth than the hybrid thread processor. Different nodes in a neighboring memory computing system can be configured differently, for example, with different computing capabilities, different types of memory, different interfaces or other differences. However, these nodes can be coupled together to share data and computing resources within a defined address space.

[0029] In an example, a near-memory computing system or a node in the system can be configured by a user for custom operations. The user can provide instructions using a high-level programming language (such as C / C++), which can be compiled and mapped directly to the data flow architecture of the system, or the data flow architecture of one or more nodes in the CNM system. In other words, the nodes in the system can include hardware blocks (e.g., memory controllers, atomic units, other customer accelerators, etc.) that can be configured to directly implement or support user instructions, thereby enhancing system performance and reducing latency.

[0030] In an example, the adjacent memory computing system may be particularly suitable for implementing a hierarchy of instructions and nested loops (e.g., two, three or more loops, deep loops, or multi-dimensional loops). A standard compiler may be used to accept high-level language instructions and then directly compile into the data flow architecture of one or more nodes. For example, a node in the system may include a hybrid thread structure accelerator. The hybrid thread structure accelerator may be executed in the user space of the CNM system and may start its own threads or sub-threads that may operate in parallel. Each thread may be mapped to a different loop iteration, thereby supporting multi-dimensional loops. In addition to other capabilities, by enabling the ability to start such nested loops, the CNM system may achieve significant time savings and latency improvements for computationally intensive operations.

[0031] In some examples, the near-memory computing system is programmed to arrange components of a reconfigurable computing structure (such as the various HTFs described herein) into one or more synchronized flows. The reconfigurable computing structure includes one or more hardware flow controllers and one or more hardware computing elements, which can be arranged to form one or more synchronized flows, as described herein.

[0032] The computing element includes a computing element memory and a processor or other suitable logic circuitry that forms a computing pipeline for processing received data. In some examples, the computing element includes multiple parallel processing channels, such as single instruction multiple data (SIMD) processing channels. As described herein, the computing element also includes circuitry for sending and receiving synchronous and asynchronous messages to the flow controller, other computing elements, and other system components. Figure 5 Blocks 504 , 510 , 512 of EMBODIMENT 1 depict exemplary computing elements.

[0033] As described herein, a flow controller includes a processor or other logic circuitry for managing synchronous flows. As described herein, a flow controller includes circuitry for sending synchronous and asynchronous messages to computing elements, other flow controllers, and other system components. In some examples, a flow controller is implemented using a block basis of one or more of blocks 504, 510, and 512 described herein.

[0034] A synchronization flow is a hardware arrangement within a reconfigurable computing fabric that includes a hardware flow controller and an ordered, synchronized data path between one or more hardware compute elements. A synchronization flow can execute one or more worker threads. To execute a thread, the hardware components of the synchronization flow pass synchronization messages and execute a predetermined set of operations in the order of the synchronization flow.

[0035] The flow controller of the synchronization flow starts a thread at the synchronization flow by providing a first synchronization message to the first computing element of the synchronization flow. The first synchronization message includes data (e.g., data processed by the computing element) and may also include control information that provides various flags and other configuration and / or instruction data to the computing element. The first computing element is programmed to perform one or more operations based on the data provided by the first synchronization message, for example. The first computing element generates a second synchronization message, which may also include data and control information. For example, the second synchronization message may describe the results of the one or more operations performed by the first computing element.

[0036] The first computing element provides the second synchronization message to the next computing element according to the order of the synchronization flow. The next computing element in the synchronization flow can be another computing element of the reconfigurable computing structure, although in some instances, a single computing element can perform consecutive operations of the synchronization flow, meaning that in some arrangements, the computing elements of the synchronization flow can direct the synchronization message to themselves. The next computing element is programmed to perform one or more operations, which can include preparing a third synchronization message and sending the third synchronization message to a subsequent computing element.

[0037] A thread completes when all computational elements of the synchronization flow have completed their programmed operations in the predetermined order of the synchronization flow. When a thread has completed, a pipeline of synchronization messages will propagate from the flow controller to the various computational elements in the predetermined order of the synchronization flow. Because the arrangement is synchronous, the completion of a thread can occur within a fixed amount of time (e.g., a predictable number of clock cycles from when the flow controller initiates the synchronization flow).

[0038] Arranging the HTF to include synchronization flows may facilitate parallel processing. For example, a flow controller for a synchronization flow does not need to wait for one thread to complete before launching additional threads. Consider an exemplary synchronization flow that includes a flow controller and multiple computing elements. The flow controller starts a first thread by providing a synchronization message to a first computing element in the synchronization flow. The first computing element performs its processing and directs a second synchronization message to the next computing element, and so on. After the first computing element completes its processing and directs the synchronization message to the next computing element, the flow controller can launch additional threads at the synchronization flow, for example, by providing additional synchronization messages to the first computing element.

[0039] Additional parallelization of synchronization flows at a reconfigurable compute structure can be achieved by utilizing compute elements that operate with a predetermined cadence or branch count (such as the various tiles described herein). For example, a compute element can use a predetermined number of clock cycles to perform various operations, such as receiving a synchronization message, performing a processing operation, sending a synchronization message, etc. A compute element can be configured to receive a new synchronization message and begin operations on a thread while operations from the previous thread are still propagating through the compute element. The new thread can be a different thread of the same synchronization flow as the previous thread, or it can be a thread of a different synchronization flow.

[0040] Synchronous flows can use the asynchronous structure of the reconfigurable computing structure to communicate with other synchronous flows and / or other components of the reconfigurable computing structure using asynchronous messages. For example, a flow controller can receive an asynchronous message from a scheduling interface and / or from another flow controller, the asynchronous message instructing the flow controller to start a thread at the synchronous flow. The scheduling interface can interface between the reconfigurable computing structure and other system components. In addition, in some instances, a synchronous flow can send an asynchronous message to the scheduling interface to indicate the completion of a thread.

[0041] Asynchronous messages can also be used by various synchronous flows to access memory. For example, a reconfigurable computing structure may include one or more memory interfaces. A memory interface is a hardware component or component thereof used by a synchronous flow to access external memory that does not belong to the synchronous flow. Threads executed at a synchronous flow may include sending read and / or write requests to the memory interface. Because reading and writing are asynchronous, a thread that initiates a read or write request to a memory interface may not receive the result of the request. Instead, the result of the read or write request may be provided to different threads executing at different synchronous flows.

[0042] Consider an exemplary reconfigurable computing structure that is arranged with a first synchronous flow for initiating a read request and a second synchronous flow for receiving the results of the read request. A first thread at the first synchronous flow sends an asynchronous read request message to a memory interface. The first thread may also send an asynchronous continue type message to a flow controller of a second synchronous flow, where the continue message indicates a read request. The memory interface obtains the requested data from the memory and directs the read data to the appropriate computing element of the second synchronous flow. The computing element then directs an asynchronous message to the second flow controller, indicating that the data has been received. (In some instances, the memory interface provides the read data directly to the second flow controller). After receiving an indication that the read data has been received, the second flow controller starts a thread at the second synchronous flow to further process the results of the read request.

[0043] An adjacent memory computing system or a node or component of an adjacent memory computing system may include or use various memory devices, controllers, and interconnects, among others. In an example, the system may include various interconnected nodes, and the nodes or groups of nodes may be implemented using chiplets. Chiplets are an emerging technology for integrating various processing functions. Typically, a chiplet system consists of discrete chips (e.g., integrated circuits (ICs) on different substrates or dies) integrated on an interposer and packaged together. This arrangement is different from a single chip (e.g., an IC) containing different device blocks (e.g., intellectual property (IP) blocks) on one substrate (e.g., a single die), such as a system on a chip (SoC), or discrete packaged devices integrated on a board. In general, chiplets offer more production benefits than single die chips, including higher yields or reduced development costs. As discussed below Figure 6A and Figure 6B An example of a chiplet system as may include an adjacent memory computing system is generally shown.

[0044] In some examples, a reconfigurable computation structure of an HFT as described herein is used to execute one or more loops, such as a set of nested loops. As described herein, to execute the loops, the reconfigurable computation structure utilizes a flow controller and computation elements arranged into one or more synchronized flows. For example, a flow controller of a synchronized flow may start a thread at the synchronized flow for each iteration of the loop. Consider the simple exemplary loop given by the following code segment [1]: [1]

[0046] For i=1,10{

[0047] x[i]=x[i-1]*2;

[0048] }

[0049] The flow controller may begin an exemplary loop by starting the first thread at the synchronization stream for loop iteration i=1. In this example, the initial value of x[i-1] is passed to the first computational element by the flow controller along with the payload data of the initial synchronization message. One or more computational elements of the synchronization stream determine the value of x[1] and return the value of x[1] to the flow controller as a synchronization message or an asynchronous message. The flow controller then starts the second thread at the synchronization stream for loop iteration i=2 and passes the return value of x[1] into the synchronization message as x[i-1]. This process continues until all iterations of the loop are completed and the value of x

[10] is returned.

[0050] The example loop above uses a single synchronization stream for each iteration of the loop. However, in some instances, multiple synchronization streams may be used for each loop iteration. Consider the example loop given by the following code segment [2]: [2]

[0052]

[0053] In this example, each loop iteration involves multiplying i by a value y[i] read from memory and then writing the result to memory. Thus, each loop iteration includes an asynchronous memory read and an asynchronous memory write. As described herein, a memory read involves sending an asynchronous message to a memory interface and then waiting for the memory interface to reply with another asynchronous message including the requested data. Because memory reads are asynchronous, each loop iteration may use threads executing at two different synchronous streams. For the i=1 loop iteration, the thread at the first synchronous stream sends an asynchronous message to the memory interface that includes a read request for the value of y[1]. The thread at the first synchronous stream may also send an asynchronous message to the second flow controller of the second synchronous stream that indicates that the second flow controller expects the result of the read request (directly from the memory interface or from a computational element of the second synchronous stream that has received the read data). The memory interface initiates the read of the value of y[1] and provides the value of y[1] to the second synchronous stream via an asynchronous message. Upon receiving the asynchronous message indicating that the read data has been received, the second flow controller starts the thread at the second synchronous stream. (The return value of y[1] may be provided to the computation unit, for example, via synchronous communication of the threads and / or directly from the memory interface before the threads are started.) The second thread determines the value of x[1] and sends a synchronization message to the memory interface that includes a write request for x[1].

[0054] In loops that use more than one synchronization stream per loop iteration, it can be difficult, and sometimes impossible, to predict the order in which the loop iterations will complete. Consider again the exemplary loop above. A first flow controller starts a first thread at a first synchronization stream for the i=1 iteration of the loop. After waiting a predetermined number of clock cycles (e.g., 4 cycles), the first flow controller starts a second thread at the first synchronization stream for the i=2 iteration of the loop. After again waiting a predetermined number of clock cycles, the first flow controller starts a third thread at the first synchronization stream for the i=3 iteration of the loop, and so on.

[0055] However, as explained above, each iteration of the loop includes an asynchronous operation (in this instance, a read request). Even if the read requests are provided to the memory interface by the various threads in the order of the loop iterations (e.g., first i=1, then i=2, and so on), the system may not return the requested data to the second synchronous stream in the same order. For example, the second flow controller may receive an asynchronous message indicating that the i=10 iteration value (y

[10] ) has been received before receiving an asynchronous message indicating that the i=9 iteration value (y[9]) has been received. If this occurs, the second flow controller may start a thread at the second synchronous stream using the i=10 iteration value y

[10] . After the second synchronous stream subsequently receives the i=9 iteration value y[9], the second flow controller starts another thread at the second synchronous stream using y[9].

[0056] Sometimes, asynchronous or out-of-order completion of loop iterations is not a problem. However, in some instances, one or more operations in one loop iteration depend on operations performed in one or more previous loop iterations. Therefore, if the loop iterations are executed out of order, incorrect results may be returned. Consider the example loop given by the following code segment [3]: [3]

[0058]

[0059] In this example, each loop iteration loads a data value (data[i]) from memory and adds the data value to a local accumulation variable (localAccum). In the last loop iteration (i=10), the local accumulation variable is loaded into memory and then cleared.

[0060] When implemented using a reconfigurable computing structure (such as the HTF described herein), this example can use two synchronization streams for each loop iteration. For each loop iteration, the first thread at the first synchronization stream sends an asynchronous read request to the memory interface to load data[i], and also sends an asynchronous message to the second flow controller of the second domain, the asynchronous message indicating that the second synchronization stream expects the data[i] value from the memory interface. After the second flow controller receives the asynchronous message indicating that the data[i] value is received, it starts the thread at the second synchronization stream. The thread at the second synchronization stream updates the local accumulation variable, and if i=10, sends an asynchronous message to the memory interface to save and clear the local accumulation variable.

[0061] However, as can be seen from this example, if the second synchronous flow thread of the i=10 loop iteration is not the last thread to start at the second synchronous flow, the loop will not execute correctly. For example, if the second synchronous flow thread of the i=10 loop iteration is started at the second synchronous flow before the thread of the i=9 loop iteration, the value of the local accumulation variable loaded into the memory will be incorrect because it will not take into account the data [9] value considered in the i=9 loop iteration that has not yet completed.

[0062] There are various approaches to this problem, but many introduce their own additional problems. Loops (such as the example above) can be implemented in a serialized manner. In a serialized loop execution, the second loop iteration does not begin until the first loop iteration is complete. The third loop iteration does not begin until the second loop iteration is complete, and so on. However, this approach can significantly slow down the execution of the loop and may reduce or eliminate the benefits of an in-memory computing system.

[0063] Another exemplary approach uses additional synchronization streams to perform looping. Consider the arrangement given by the following code segment [4]: [4]

[0065]

[0066] In this example, the saving and clearing of local accumulated variables occurs outside the loop. For example, each thread at the second synchronous flow can send an asynchronous message to the third flow controller of the third synchronous flow. The asynchronous message indicates to the third flow controller that the corresponding loop iteration has been completed. When the third flow controller receives ten asynchronous messages indicating that all ten iterations of the loop have been completed, it starts the thread at the third synchronous flow to save and clear the local accumulated variables. This arrangement allows the loop to execute correctly, however, it also requires additional resources of the reconfigurable computing structure. For example, the resources of the flow controller and computing elements used to implement the third synchronous flow may not be available for other processing.

[0067] Various embodiments address these and other issues by utilizing a last iteration flag managed by the flow controller of the last synchronized flow of a loop iteration. For example, various flow controllers of a reconfigurable computing structure can be configured with a programmable field that controls whether the last iteration flag of a thread is set locally in the flow controller or provided by a previously executed thread. The flow controller of the synchronized flow that executes the last portion of a loop iteration is configured to set the last iteration flag locally. The flow controller is also provided with an iteration count that indicates the total number of loop iterations it should execute.

[0068] The flow controller counts the number of loop iterations it processes, for example, by counting the number of threads launched. When launching the final loop iteration, the flow controller sets the last iteration flag in a synchronization message sent to the first compute element in the synchronization flow to launch the final thread. This thread executes using operations conditional on the state of the last iteration flag. In this way, conditional operations are executed in the thread of the last loop iteration to be executed, even if that iteration is not the last to be launched.

[0069] To illustrate further, consider the following example code segment [5], which provides a variation of the loop given by example code segments [3] and [4] above: [5]

[0071]

[0072] In this example, as with the arrangement of the exemplary code segment [3] above, two synchronization streams are used for each loop iteration. The thread at the first synchronization stream sends an asynchronous read request to the memory interface to load data [i], and also sends an asynchronous message to the second flow controller of the second synchronization stream. After the second flow controller receives an indication that the value of data [i] has been received, it starts the thread at the second synchronization stream to update the local accumulation variable. However, in this example, the second flow controller is configured to track the total number of threads it starts at the second synchronization stream. When the number of threads started at the second synchronization stream meets the iteration count, the second flow controller sets the last iteration flag (lastlter) in the synchronization message that starts the last thread. The computing element of the last synchronization stream is configured to send one or more asynchronous messages to the memory interface to save and clear the local accumulation variable conditioned on the setting of the last iteration flag.

[0073] In this arrangement, execution of the loop can be robust even if one or more asynchronous operations of the loop iteration cause the iteration to complete out of order. Consider the exemplary execution of the code segment [5] above, in which the second synchronous stream receives the values of data [i] in order, except that data

[10] for the i=10 loop iteration is received before data [9] for the i=9 loop iteration. In this example, the second flow controller will start the thread at the second synchronous stream for the i=10 loop when data

[10] is received. Because, in this example, the i=10 iteration is not the last iteration to be completed, the second flow controller will not set the last iteration flag when starting the second synchronous stream thread for the i=10 iteration. Therefore, the second synchronous stream thread for the i=10 iteration will not save and clear the local accumulated variables. Thereafter, when the flow controller receives data [9] for the i=9 iteration, it starts the thread at the second synchronous stream for the i=9 iteration and sets the last iteration flag. Therefore, the thread for the i=9 iteration (which is the last to be executed in this example) saves and clears the local accumulated variables. In this manner, as described herein, using a last iteration flag at the last flow controller of a loop iteration may allow for robust and correct loop execution without slowing down processing by serializing loop iterations and without the additional resource requirements of using additional synchronization flows.

[0074] Figure 1 A first example of a neighboring memory computing system or CNM system 102 is generally shown. The example of the CNM system 102 includes a plurality of different memory computing nodes, each of which may include various neighboring memory computing devices. Each node in the system can operate in its own operating system (OS) domain (e.g., Linux, etc.). In an example, the nodes can coexist in a common OS domain of the CNM system 102.

[0075] Figure 1 104 includes an instance of a first memory compute node 104 of the CNM system 102. The CNM system 102 can have multiple nodes, such as different instances of the first memory compute node 104, coupled using an expansion fabric 106. In an example, the architecture of the CNM system 102 can support expansion using up to n different memory compute nodes (e.g., n=4096) using the expansion fabric 106. As discussed further below, each node in the CNM system 102 can be an assembly of multiple devices.

[0076] The CNM system 102 may include a global controller for various nodes in the system, or a particular memory computing node in the system may optionally serve as a host or controller for one or more other memory computing nodes in the same system. Thus, the various nodes in the CNM system 102 may be configured similarly or differently.

[0077] In an example, each node in the CNM system 102 may include a host system that uses a specific operating system. The operating system may be common or different between the various nodes in the CNM system 102. Figure 1 In an example, the first memory computing node 104 includes a host system 108, a first switch 110, and a first memory computing device 112. The host system 108 may include a processor, such as an X86, ARM, RISC-V, or other type of processor. The first switch 110 may be configured to facilitate communication between the first memory computing node 104 or the devices of the CNM system 102, such as using a dedicated or other communication protocol, generally referred to herein as a chip-to-chip protocol interface (CT). That is, CTCPI may include a dedicated interface unique to the CNM system 102, or may include or use other interfaces, such as a Compute Express Link (CXL) interface, a Peripheral Component Interconnect Express (PCIe) interface, or a Chiplet Protocol Interface (CPI). The first switch 110 may include a switch configured to use CTCPI. For example, the first switch 110 may include a CXL switch, a PCIe switch, a CPI switch, or other types of switches. In an example, the first switch 110 may be configured to couple endpoints of different configurations. For example, the first switch 110 may be configured to convert packet formats, such as converting between PCIe and CPI formats.

[0078] The CNM system 102 is described herein in various exemplary configurations, such as a system comprising nodes, and each node may comprise various chips (e.g., processors, switches, memory devices, etc.). In an example, the first memory compute node 104 in the CNM system 102 may comprise various chips implemented using chiplets. In the chiplet-based configuration of the CNM system 102 discussed below, inter-chiplet communications as well as additional communications within the system may use a CPI network. The CPI network described herein is an example of CTCPI, that is, a chiplet-specific implementation of CTCPI. Therefore, the structure, operation, and functionality of the CPI described below may be equally applicable to structures, operations, and functionality that may be implemented in other ways using a non-chiplet-based CTCPI. Unless otherwise expressly stated, any discussion herein about CPI is equally applicable to CTCPI.

[0079] The CPI interface includes a packet-based network that supports virtual channels to enable flexible and high-speed interaction between chiplets, such as may comprise the first memory compute node 104 or part of the CNM system 102. CPI enables bridging from a network within a chiplet to a broader network of chiplets. For example, the Advanced Extensible Interface (AXI) is a specification for intra-chip communication. However, the AXI specification covers a large number of physical design options, such as the number of physical channels, signal timing, power, etc. Within a single chip, these options are typically selected to meet design goals, such as power consumption, speed, etc. However, to enable flexibility in chiplet-based memory compute systems, adapters (such as using CPI) can interface between various AXI design options that may be implemented in various chiplets. By implementing a mapping of physical channels to virtual channels and encapsulating time-based signaling with a packetized protocol, CPI can be used to bridge intra-chiplet networks, such as within a specific memory compute node, across a wider network of chiplets, such as across the first memory compute node 104 or across the CNM system 102.

[0080] The CNM system 102 is scalable to include multi-node configurations. That is, multiple different instances of the first memory compute node 104 or other differently configured memory compute nodes can be coupled using the expansion fabric 106 to provide an extended system. Each memory compute node can run its own operating system and can be configured to jointly coordinate system-wide resource usage.

[0081] exist Figure 1 In the example of FIG, the first switch 110 of the first memory compute node 104 is coupled to the expansion fabric 106. The expansion fabric 106 can provide a switch (e.g., a CTCPI switch, a PCIe switch, a CPI switch, or other switches) that can facilitate communication between different memory compute nodes. In an example, the expansion fabric 106 can help various nodes communicate in a partitioned global address space (PGAS).

[0082] In an example, a first switch 110 from a first memory computing node 104 is coupled to one or more different memory computing devices, such as a first memory computing device 112. The first memory computing device 112 can include a chiplet-based architecture, which is referred to herein as a compute-near-memory (CNM) chiplet. A packaged version of the first memory computing device 112 can include, for example, one or more CNM chiplets. These chiplets can be communicatively coupled using CTCPI for high bandwidth and low latency.

[0083] exist Figure 1In an example of , the first memory computing device 112 may include a network on chip (NOC) or a first NOC 118. Typically, a NOC is an interconnect network within a device that connects a specific set of endpoints. Figure 1 In the embodiment, the first NOC 118 can provide communication and connection between various memories, computing resources and ports of the first memory computing device 112.

[0084] In an example, the first NOC 118 can include a folded Clos topology, such as within each instance of a memory computing device, or as a mesh coupling multiple memory computing devices in a node. The Clos topology (e.g., multiple smaller radix crossbars can be used to provide functionality associated with a higher radix crossbar topology) provides various benefits. For example, the Clos topology can exhibit consistent latency and equally divided bandwidth across the NOC.

[0085] The first NOC 118 can include a variety of switch types, including hub switches, edge switches, and endpoint switches. Each switch can be configured as a crossbar that provides substantially uniform latency and bandwidth between input nodes and output nodes. In an example, the endpoint switches and edge switches can include two separate crossbars, one for traffic heading to the hub switch and the other for traffic heading away from the hub switch. The hub switch can be configured as a single crossbar that switches all inputs to all outputs.

[0086] In an example, the hub switches can each have multiple ports (eg, four or six ports each), such as depending on whether a particular hub switch participates in inter-chip communication. The number of hub switches participating in inter-chip communication can be set by the inter-chip bandwidth requirements.

[0087] The first NOC 118 can support various payloads between the computing elements and the memory (e.g., payloads from 8 to 64 bytes; similarly, other payload sizes can be used). In an example, the first NOC 118 can be optimized for relatively small payloads (e.g., 8-16 bytes) to efficiently handle accesses to sparse data structures.

[0088] In an example, the first NOC 118 can be coupled to an external host via a first physical layer interface 114, a PCIe slave module 116 or endpoint, and a PCIe master module 126 or root port. That is, the first physical layer interface 114 can include an interface that allows an external host processor to be coupled to the first memory computing device 112. The external host processor can optionally be coupled to one or more different memory computing devices, such as using a PCIe switch or other local protocol switch. Communication with the external host processor through a PCIe-based switch can limit device-to-device communication to the range supported by the switch. In contrast, communication through a memory computing device local protocol switch (such as using CTCPI) can allow more complete communication between different memory computing devices, including support for partitioned global address spaces, such as creating threads of work and sending events.

[0089] In an example, the CTCPI protocol can be used by the first NOC 118 in the first memory computing device 112, and the first switch 110 can include a CTCPI switch. The CTCPI switch can allow CTCPI packets to be transmitted from a source memory computing device (such as the first memory computing device 112) to a different destination memory computing device (e.g., on the same or another node) without being converted to another packet format.

[0090] In an example, the first memory computing device 112 may include an internal host processor 122. The internal host processor 122 may be configured to communicate with the first NOC 118 or other components or modules of the first memory computing device 112 using, for example, an internal PCIe master module 126, which may help eliminate physical layers that consume time and energy. In an example, the internal host processor 122 may be based on a RISC-V ISA processor and may use a first physical layer interface 114 to communicate externally to the first memory computing device 112, such as with other storage, network, or other peripheral devices of the first memory computing device 112. The internal host processor 122 may control the first memory computing device 112 and may act as a proxy for operating system-related functions. The internal host processor 122 may include a relatively small number of processing cores (e.g., 2-4 cores) and a host memory device 124 (e.g., including DRAM modules).

[0091] In an example, the internal host processor 122 may include a PCI root port. When the internal host processor 122 is in use, one of its root ports may be connected to the PCIe slave module 116. Another of the root ports of the internal host processor 122 may be connected to the first physical layer interface 114, such as to provide communication with external PCI peripheral devices. When the internal host processor 122 is disabled, the PCIe slave module 116 may be coupled to the first physical layer interface 114 to allow the external host processor to communicate with the first NOC 118. In an example of a system having multiple memory computing devices, the first memory computing device 112 may be configured to act as a system host or controller. In this example, the internal host processor 122 may be used, and the other instances of the internal host processor in the corresponding other memory computing devices may be disabled.

[0092] The internal host processor 122 can be configured upon power-up of the first memory computing device 112, such as to allow host initialization. In an example, the internal host processor 122 and its associated data path (e.g., including the first physical layer interface 114, the PCIe slave module 116, etc.) can be configured from input pins to the first memory computing device 112. One or more pins can be used to enable or disable the internal host processor 122 and configure the PCI (or other) data path accordingly.

[0093] In an example, the first NOC 118 can be coupled to the extended fabric 106 via an extended fabric interface module 136 and a second physical layer interface 138. The extended fabric interface module 136 or SIF can facilitate communication between the first memory computing device 112 and a device space such as a partitioned global address space (PGAS). The PGAS can be configured to enable a particular memory computing device, such as the first memory computing device 112, to access memory or other resources on a different memory computing device (e.g., on the same or different nodes), such as using a load / store paradigm. Various scalable fabric technologies can be used, including CTCPI, CPI, Gen-Z, PCI, or Ethernet bridged over CXL. The extended fabric 106 can be configured to support various packet formats. In an example, the extended fabric 106 supports out-of-order packet communication, or supports sequential packets, such as using path identifiers to extend bandwidth across multiple equivalent paths. The extended fabric 106 can generally support remote operations, such as remote memory reads, writes, and other built-in atoms, remote memory atoms, remote memory computing device send events, and remote memory computing device call and return operations.

[0094] In an example, the first NOC 118 may be coupled to one or more different memory modules, such as a first memory device 128. The first memory device 128 may include various types of memory devices, such as LPDDR5 or GDDR6. Figure 1 In an example, the first NOC 118 can coordinate communications with the first memory device 128 via a memory controller 130 that can be dedicated to a particular memory module. In an example, the memory controller 130 can include a memory module cache and an atomic operation module. The atomic operation module can be configured to provide relatively high throughput atomic operators, such as integer and floating point operators. The atomic operation module can be configured to apply its operators to data within a memory module cache (e.g., including an SRAM memory-side cache), thereby allowing back-to-back atomic operations using the same memory location with minimal throughput degradation.

[0095] The memory module cache can provide storage for frequently accessed memory locations, such as without having to re-access the first memory device 128. In an example, the memory module cache can be configured to cache data only for a specific instance of the memory controller 130. In an example, the memory controller 130 includes a DRAM controller configured to interface with the first memory device 128, such as a DRAM device. The memory controller 130 can provide access scheduling and bit error management, among other functions.

[0096] In an example, the first NOC 118 can be coupled to a hybrid thread processor (HTP 140), a hybrid thread fabric (HTF 142), and a host interface and scheduling module (HIF 120). The HIF 120 can be configured to facilitate access to host-based command request queues and response queues. In an example, the HIF 120 can schedule new execution threads on processors or computing elements of the HTP 140 or HTF 142. In an example, the HIF 120 can be configured to maintain workload balance across the HTP 140 and HTF 142 modules.

[0097] A hybrid thread processor or HTP 140 may include an accelerator, such as one based on the RISC-V instruction set. The HTP 140 may include a highly threaded, event-driven processor in which threads may execute in a single instruction cycle, such as to maintain high instruction throughput. The HTP 140 includes relatively few custom instructions to support low-overhead threading capabilities, event send / receive, and shared memory atomic operators.

[0098] The hybrid threading structure or HTF 142 may include an accelerator, such as a non-von Neumann, coarse-grained, reconfigurable processor. The HTF 142 may be optimized for high-level language operations and data types (e.g., integer or floating point). In an example, the HTF 142 may support dataflow computations. The HTF 142 may be configured to utilize substantially all of the memory bandwidth available on the first memory computing device 112, such as when executing a memory-bound computation kernel.

[0099] The HTP and HTF accelerators of the CNM system 102 can be programmed using various high-level structured programming languages. For example, the HTP and HTF accelerators can be programmed using C / C++, such as using the LLVM compiler framework. The HTP accelerator can utilize an open source compiler environment, such as with the addition of various custom instruction sets that are configured to improve memory access efficiency, provide message passing mechanisms, and manage events. In an example, the HTF accelerator can be designed to be able to program the HTF 142 using a high-level programming language, and the compiler can generate a simulator configuration file or binary file that runs on the HTF 142 hardware. The HTF 142 can provide a mid-level language for accurately and concisely expressing algorithms while hiding the configuration details of the HTF accelerator itself. In an example, the HTF accelerator tool chain can use the LLVM front-end compiler and the LLVM intermediate representation (IR) to interface with the HTF accelerator back-end.

[0100] Figure 2 An example of a memory subsystem 200 of a memory computing device according to an embodiment is generally shown. The example of the memory subsystem 200 includes a controller 202, a programmable atomic unit 208, and a second NOC 206. The controller 202 may include or use the programmable atomic unit 208 to perform operations using information in the memory device 204. In the example, the memory subsystem 200 includes Figure 1 An instance of a portion of the first memory computing device 112 , such as including a portion of the first NOC 118 or the memory controller 130 .

[0101] exist Figure 2 In an example, the second NOC 206 is coupled to the controller 202, and the controller 202 may include a memory control module 210, a local cache module 212, and a built-in atomic module 214. In an example, the built-in atomic module 214 may be configured to handle relatively simple single-cycle integer atomics. The built-in atomic module 214 may execute atomics with the same throughput as, for example, normal memory read or write operations. In an example, an atomic memory operation may include a combination of storing data to memory, performing an atomic memory operation, and then responding with the load data from memory.

[0102] A local cache module 212 (e.g., which may include an SRAM cache) may be provided to help reduce latency for repeatedly accessed memory locations. In an example, the local cache module 212 may provide a read buffer for sub-memory line accesses. The local cache module 212 may be particularly beneficial for computing elements that have relatively small or no data caches.

[0103] The memory control module 210 (e.g., which may include a DRAM controller) may provide low-level request buffering and scheduling, e.g., to provide efficient access to the memory device 204 (e.g., which may include a DRAM device). In an example, the memory device 204 may include or utilize a GDDR6 DRAM device, e.g., having a density of 16 Gb and a peak bandwidth of 64 Gb / s. Similarly, other devices may be used.

[0104] In an example, the programmable atomic unit 208 can include single-cycle or multi-cycle operators, such as being configured to perform integer addition or more complex multi-instruction operations, such as Bloom filter insertion. In an example, the programmable atomic unit 208 can be configured to perform load and store operations to memory. The programmable atomic unit 208 can be configured to utilize the RISC-V ISA with a set of dedicated instructions to facilitate interaction with the controller 202 to atomically perform user-defined operations.

[0105] Programmable atomic requests, such as those received from a host on or off a node, can be routed to the programmable atomic unit 208 via the second NOC 206 and the controller 202. In an example, custom atomic operations (e.g., performed by the programmable atomic unit 208) can be the same as built-in atomic operations (e.g., performed by the built-in atomic module 214), except that programmable atomic operations can be defined or programmed by a user rather than a system architect. In an example, a programmable atomic request packet can be sent to the controller 202 via the second NOC 206, and the controller 202 can identify the request as a custom atomic. The controller 202 can then forward the identified request to the programmable atomic unit 208.

[0106] Figure 3 An example of a programmable atomic unit 302 for use with a memory controller according to an embodiment is generally shown. In an example, the programmable atomic unit 302 may include or may correspond to a programmable atomic unit 302 from Figure 2 Instance of the programmable atomic unit 208. That is, Figure 3 Components of an example of a programmable atomic unit 302 (PAU) are shown, as described above with respect to Figure 2 (e.g., in programmable atomic unit 208) or with respect to Figure 1(eg, in the atomic operation module of the memory controller 130). Figure 3 As shown, the programmable atomic unit 302 includes a PAU processor or PAU core 306, a PAU thread controller 304, an instruction SRAM 308, a data cache 310, and a memory interface 312 that interfaces with a memory controller 314. In this example, the memory controller 314 includes Figure 2 An instance of the controller 202 of an instance of .

[0107] In an example, the PAU core 306 is a pipelined processor such that multiple stages of different instructions are executed together per clock cycle. The PAU core 306 may include a barrel-shaped multithreaded processor with a thread control 304 circuit system to switch between different register files (e.g., a set of registers containing the current processing state) at each clock cycle. This enables efficient context switching between currently executing threads. In an example, the PAU core 306 supports eight threads, resulting in eight register files. In an example, some or all of the register files are not integrated into the PAU core 306, but rather reside in the local data cache 310 or instruction SRAM 308. This reduces circuit complexity in the PAU core 306 by eliminating traditional flip-flops for registers in such memories.

[0108] The local PAU memory may include an instruction SRAM 308, which may include instructions for various atomics. These instructions include an instruction set that supports atomic operators loaded by various applications. When an atomic operator is requested, such as by an application chiplet, a set of instructions corresponding to the atomic operator is executed by the PAU core 306. In an example, the instruction SRAM 308 may be partitioned to establish an instruction set. In this example, a specific programmable atomic operator requested by a requesting process may identify the programmable atomic operator by a partition number. The partition number may be established when the programmable atomic operator is registered to (e.g., loaded onto) the programmable atomic unit 302. Other metadata for the programmable instructions may be stored in a memory in a memory local to the programmable atomic unit 302 (e.g., in a partition table).

[0109] In an example, the atomic operator manipulates the data cache 310, which is typically synchronized (e.g., flushed) when the thread of the atomic operator completes. Thus, latency can be reduced for most memory operations during execution of a programmable atomic operator thread, except for the initial load from external memory (e.g., from the memory controller 314).

[0110] When an executing thread attempts to make a memory request, a pipeline processor such as the PAU core 306 may experience problems if a potential hazard condition would prevent such a request. Here, the memory request is to retrieve data from the memory controller 314, whether it is from a cache on the memory controller 314 or from off-die memory. To address this issue, the PAU core 306 is configured to deny memory requests to the thread. Generally speaking, the PAU core 306 or thread control 304 may include circuitry to enable one or more thread rescheduling points in the pipeline. Here, the denial occurs at a point in the pipeline that is outside (e.g., after) these thread rescheduling points. In an example, the hazard occurs outside of a rescheduling point. Here, the previous instruction in the thread created the hazard after the memory request instruction passed through the last thread rescheduling point before the pipeline stage where the memory request could be made.

[0111] In an example, to deny a memory request, the PAU core 306 is configured to determine (e.g., detect) the presence of a hazard on the memory indicated in the memory request. Here, a hazard represents any condition where allowing (e.g., executing) the memory request would cause an inconsistent state for the thread. In this example, the hazard is an ongoing memory request. Here, regardless of whether the data cache 310 includes data for the requested memory address, the presence of the ongoing memory request makes it uncertain what data should be in the data cache 310 at that address. Therefore, the thread must wait for the ongoing memory request to complete before operating on the current data. When the memory request completes, the hazard is cleared.

[0112] In the example, the hazard is a dirty cache line in the data cache 310 for the requested memory address. While a dirty cache line generally indicates that the data in the cache is current and the memory controller's version of the data is not current, problems may arise on thread instructions that do not operate from the cache. Examples of such instructions use the built-in atomic operators of the memory controller 314 or other separate hardware blocks. In the context of the memory controller, the built-in atomic operators may be separate from the programmable atomic unit 302 and do not have access to the cache within the PAU or the data cache 310. If the cache line is dirty, the built-in atomic operators will not operate on the latest data until the cache is flushed to synchronize the cache with other or off-die memory. The same situation may also occur with other hardware blocks of the memory controller (such as the encryption block, encoder, etc.).

[0113] Figure 4An example of a hybrid thread processor (HTP) accelerator or HTP accelerator 400 is shown. According to an embodiment, the HTP accelerator 400 may include a portion of a memory computing device. In an example, the HTP accelerator 400 may include or include a processor from Figure 1 The HTP accelerator 400 includes, for example, an HTP core 402, an instruction cache 404, a data cache 406, a conversion block 408, a memory interface 410, and a thread controller 412. The HTP accelerator 400 may further include a scheduling interface 414 and a NOC interface 416, such as for communicating with a NOC (e.g., from Figure 1 In the example of the first NOC 118, from Figure 2 The second NOC 206 or other NOC) interface in the example.

[0114] In an example, the HTP accelerator 400 includes a module based on the RISC-V instruction set, and may include a relatively small amount of other or additional custom instructions to support low overhead, threadable hybrid thread (HT) language. The HTP accelerator 400 may include a highly threaded processor core, i.e., an HTP core 402, in which or using the processor core, a thread may be executed in a single instruction cycle, such as to maintain high instruction throughput. In an example, a thread may be suspended while waiting for other pending events to complete. This may allow computing resources to be effectively used for related work, rather than polling. In an example, multi-threaded barrier synchronization may use efficient HTP to HTP and HTP to host / from host message passing, such as allowing thousands of threads to be initialized or woken up in, for example, tens of clock cycles.

[0115] In an example, the scheduling interface 414 may include a functional block of the HTP accelerator 400 for handling hardware-based thread management. In other words, the scheduling interface 414 may manage the scheduling of the work of the HTP core 402 or other accelerators. However, non-HTP accelerators are generally unable to schedule work. In an example, work scheduled from the host may use a scheduling queue residing in, for example, the host main memory (e.g., a DRAM-based memory). On the other hand, work scheduled from the HTP accelerator 400 may use a scheduling queue residing in SRAM, such as within the scheduling of the target HTP accelerator 400 within a particular node.

[0116] In an example, the HTP core 402 may include one or more cores that execute instructions on behalf of threads. That is, the HTP core 402 may include an instruction processing block. The HTP core 402 may further include a thread controller 412 or may be coupled to the thread controller. The thread controller 412 may provide thread control and status for each active thread within the HTP core 402. The data cache 406 may include a cache for the host processor (e.g., for local and remote memory computing devices, including for the HTP core 402), and the instruction cache 404 may include a cache for use by the HTP core 402. In an example, the data cache 406 may be configured for read and write operations, and the instruction cache 404 may be configured for read-only operations.

[0117] In an example, data cache 406 is a small cache provided for each hardware thread. Data cache 406 can temporarily store data for use by the owning thread. Data cache 406 can be managed by hardware or software in HTP accelerator 400. For example, the hardware can be configured to automatically allocate or evict lines as needed as load and store operations are performed by HTP core 402. Software (e.g., using RISC-V instructions) can determine which memory accesses should be cached and when lines should be invalidated or written back to other memory locations.

[0118] The data cache on the HTP accelerator 400 has various benefits, including making larger access more effective for the memory controller, thereby allowing the execution thread to avoid stagnation. However, there is a situation where the efficiency is low when using the cache. Examples include accesses where data is only accessed once, and the impact of the cache line is caused. To help solve this problem, the HTP accelerator 400 can use a set of customized load instructions to force the load instruction to check for cache hits, and when the cache misses, a memory request for the requested operand is issued, and the obtained data is not placed in the data cache 406. Therefore, the HTP accelerator 400 includes various different types of load instructions, including non-cached and cached line loads. If there is dirty data in the cache, the non-cached load instruction uses the cached data. The non-cached load instruction can ignore the clean data in the cache and will not write the accessed data to the data cache. For cache line load instructions, a complete data cache line (e.g., comprising 64 bytes) can be loaded from the memory into the data cache 406, and the addressed memory can be loaded into a specified register. These loads can use cached data if the clean or dirty data is in the data cache 406. If the referenced memory location is not in the data cache 406, the entire cache line can be accessed from memory. The use of cache line load instructions can reduce cache misses when sequential memory locations are referenced (such as in a memory copy operation), but can also waste memory and bandwidth at the NOC interface 416 if the referenced memory data is not used.

[0119] In an example, the HTP accelerator 400 includes custom non-cached store instructions. The non-cached store instructions can help avoid flooding the data cache 406 with write data that is not sequentially written to memory.

[0120] In an example, the HTP accelerator 400 further includes a translation block 408. The translation block 408 may include a virtual-to-physical translation block for the local memory of the memory computing device. For example, a host processor, such as in the HTP core 402, may execute a load or store instruction, and the instruction may generate a virtual address. The virtual address may be translated into a physical address for the host processor, such as using a translation table from the translation block 408. The memory interface 410 may include, for example, an interface between the HTP core 402 and the NOC interface 416.

[0121] Figure 5 An example of a representation of a hybrid thread structure (HTF) or HTF 500 of a memory computing device according to an embodiment is shown. In the example, the HTF 500 may include or contain Figure 15. The HTF 500 is a coarse-grained, reconfigurable computational structure that can be optimized for high-level language operand types and operators (e.g., using C / C++ or other high-level languages). In an example, the HTF 500 can include a configurable n-bit wide (e.g., 512-bit wide) data path that interconnects the hardened SIMD arithmetic units.

[0122] In the example, HTF 500 includes an HTF cluster 502, which includes multiple HTF tiles (including exemplary tile 504 or tile N). Each HTF tile can implement one or more compute elements using local tile or compute element memory and arithmetic functions. For example, each tile can include a compute pipeline that supports integer and floating-point operations. In the example, the data path, compute elements, and other infrastructure can be implemented as hardened IP to provide maximum performance while minimizing power consumption and reconfiguration time.

[0123] exist Figure 5 In the example of , the tiles comprising the HTF cluster 502 are arranged linearly, and each tile in the cluster can be coupled to one or more other tiles in the HTF cluster 502. Figure 5 In the example of FIG, exemplary tile 504 or tile N is coupled to four other tiles, including a tile base (e.g., tile N-2) coupled to tile 510 via a port labeled SF Input N-2, coupled to an adjacent tile 512 (e.g., tile N-1) via a port labeled SF Input N-1, and coupled to tile N+1 via a port labeled SF Input N+1 and to tile N+2 via a port labeled SF Input N+2. A tile base is a hardware portion of a tile (e.g., tiles 504, 510, 512) that is configured to launch threads and / or otherwise act as a flow controller. Exemplary tile 504 can be coupled to the same or other tiles via corresponding output ports (e.g., output ports labeled SF Output N-1, SF Output N-2, SF Output N+1, and SF Output N+2). In this example, the ordered list of the names of the various tiles is a conceptual indication of the locations of the tiles. In other examples, the tiles comprising the HTF cluster 502 may be arranged in a grid or other configuration, where each tile is similarly coupled to one or more of its nearest neighbors in the grid. Tiles located at the edges of a cluster may optionally have fewer connections to adjacent tiles. For example, tile N-2 or Figure 5 The tile basis of the example tile 510 may be coupled only to the adjacent tile 512 (Tile N-1) and the example tile 504 (Tile N). Similarly, fewer or additional inter-tile connections may be used.

[0124] The HTF cluster 502 may further include memory interface modules, including a first memory interface module 506. The memory interface module may couple the HTF cluster 502 to a NOC (e.g., the first NOC 118). In an example, the memory interface module may allow tiles within the cluster to issue requests to memory elsewhere in the computing system, such as within the same or different nodes in the system. In other words, the representation of the HTF 500 may comprise a portion of a larger structure that may be distributed across multiple nodes, such as having one or more HTF tiles or HTF clusters on each node. Requests may be made between tiles or nodes within the context of the larger structure.

[0125] exist Figure 5 In the example of FIG. 5 , a synchronization fabric (SF) is used to couple tiles in an HTF cluster 502. As described above, the synchronization fabric can provide communication between a particular tile and its neighboring tiles in the HTF cluster 502. Each HTF cluster 502 can further include an asynchronous fabric (AF) that can provide communication between, for example, tiles in the cluster, memory interfaces in the cluster, and a scheduling interface 508 in the cluster.

[0126] In an example, the synchronization structure can exchange messages that include data and control information. The control information can include instruction RAM address information or thread identifiers, among other things. The control information can be used to establish a data path, and the data message field can be selected as the source of the path. Typically, the control fields can be provided or received earlier so that they can be used to configure the data path. For example, to help minimize any delays in the tile through the synchronization flow pipeline, the control information can arrive at the tile several clock cycles before the data field. Various registers can be provided to help coordinate the timing of data flow in the pipeline.

[0127] In an example, each tile in the HTF cluster 502 may include one or more tile memories. Each tile memory may have the same width as the datapath (e.g., 512 bits) and may have a specified depth, such as in the range of 512 to 1024 elements. Tile memories may be used to store data that supports datapath operations. For example, the stored data may include constants loaded as part of the cluster configuration of the kernel, or may include variables calculated as part of the dataflow. In an example, tile memories may be written from an asynchronous structure as a data transfer from another synchronous flow, or may include the results of a load operation, such as initiated by another synchronous flow. Tile memories may be read via synchronous datapath instruction execution in a synchronous flow.

[0128] In an example, each tile in the HTF cluster 502 can have a dedicated instruction RAM (INST RAM). In an example of an HTF cluster 502 with sixteen tiles and a corresponding instruction RAM instance with sixty-four entries, the cluster can allow algorithms with up to 1024 multiply-shift and / or ALU operations to be mapped. The various tiles can optionally be pipelined together, such as using synchronous structures, to allow data flow computations to be performed with minimal memory accesses, thereby minimizing latency and reducing power consumption. In an example, asynchronous structures can allow memory references to be performed in parallel with computations, thereby providing a more efficient streaming kernel. In an example, the various tiles can include built-in support for loop-based constructs and can support nested loop kernels.

[0129] The synchronization structure can allow multiple tiles (e.g., multiple computational elements thereof) to be pipelined, e.g., without the need for data queuing. For example, the computational elements participating in a synchronized flow can act as a single pipeline data path. The flow controller for the synchronized flow can be or include a tile (e.g., in Figure 5 In the example of tile N-2, a computational element on the tile, and / or a tile base or controller on the tile. The flow controller of the synchronized flow can launch threads of work through the pipelined tiles. The flow controller can be responsible for launching threads at a predefined cadence (referred to herein as a branch count). For example, if the branch count is 3, the tile base can launch a thread every three clock cycles.

[0130] In an example, a synchronization flow includes a set of connected computing elements in the HTF cluster 502. The execution of a thread can be started at a flow controller and can be advanced from the flow controller to other computing elements (e.g., other tiles in the same synchronization flow) via a synchronization structure. The flow controller can provide instructions to be executed for a first computing element. By default, the first computing element can provide the same instructions for execution for other connected computing elements. However, in some examples, the flow controller or subsequent computing elements can implement conditional operations that conditionally specify or use alternative instructions. The alternative instruction can be selected by causing the data path of the computing element to generate a Boolean conditional value, which can then be used to select between the instruction set of the current computing element and the alternative instruction.

[0131] The asynchronous structure can be used to perform operations that occur asynchronously with respect to the synchronous flow. Each tile in the HTF cluster 502 can include an interface to the asynchronous structure. The inbound interface can include, for example, a FIFO buffer or queue (e.g., an AF entry queue) to provide storage for messages that cannot be processed immediately. Similarly, the outbound interface of the asynchronous structure can include a FIFO buffer or queue (e.g., an AF departure queue) to provide storage for messages that cannot be sent out immediately.

[0132] In an example, messages in an asynchronous structure can be classified as data messages or control messages. Data messages can include SIMD-width data values written to tile memory 0 (MEM_0) or memory 1 (MEM_1). Control messages can be configured to control thread creation, release resources, or issue external memory references.

[0133] The tiles in HTF cluster 502 can perform various computational operations for the HTF. Computational operations can be performed by configuring the data paths within the tile and / or its computational elements. In one example, the tile includes two functional blocks that perform the tile's computational operations: a multiplication and shift operation block (MS OP) and an arithmetic, logic, and bitwise operation block (ALB OP). These two blocks can be configured to perform pipelined operations, such as multiplication and addition, or shift and addition.

[0134] In an example, each instance of a memory computing device in a system may have a complete set of supported instructions for its operator blocks (e.g., MS OP and ALB OP). In this case, binary compatibility may be achieved across all devices in the system. However, in some examples, it may be helpful to maintain a base set of functions and optional instruction set classes (e.g., to meet various design tradeoffs, such as die size). This approach may be similar to how the RISC-V instruction set has a base set and multiple optional instruction subsets.

[0135] In an example, the exemplary tile 504 may include a branch RAM. The branch RAM may be used to specify which input (e.g., from the four SF tile inputs and the tile base input) is the primary input for each clock cycle. The branch RAM read address input may be derived from a counter that counts from zero to a branch count minus one. In an example, different branch counts may be used on different tiles, such as within the same HTF cluster 502, to allow the use of multiple slices or unique tile instances by an internal loop to determine the performance of a specific application or instruction set. In an example, the branch RAM may specify when synchronous inputs are written to the tile memory, such as when multiple inputs to a specific tile instruction are used and one input arrives before the others. The earliest arriving input may be written to the tile memory and may be read later when all inputs are available. In this example, the tile memory may be accessed as a FIFO memory, and the FIFO read and write pointers may be stored in a register-based memory area or structure in the tile memory.

[0136] Figure 6A and Figure 6BGenerally, an example of a chiplet system that can be used to implement one or more aspects of the CNM system 102 is shown. As similarly mentioned above, a node in the CNM system 102 or a device within a node in the CNM system 102 may include a chiplet-based architecture or a neighboring memory computing (CNM) chiplet. The packaged memory computing device may include, for example, one, two, or four CNM chiplets. The chiplets may be interconnected using a high-bandwidth, low-latency interconnect (such as using a CPI interface). Typically, a chiplet system consists of discrete modules (each a "chiplet") integrated on an interposer and, in many instances, interconnected as needed through one or more established networks to provide a system with the desired functionality. The interposer and the included chiplets may be packaged together to facilitate interconnection with other components of the larger system. Each chiplet may include one or more separate integrated circuits (ICs) or "chips," potentially in combination with discrete circuit components, and may be coupled to a corresponding substrate for attachment to the interposer. Most or all of the chiplets in the system may be individually configured for communication over an established network.

[0137] The configuration of chiplets as separate modules of a system differs from such systems implemented on a single chip containing different device blocks (e.g., intellectual property (IP) blocks) on one substrate (e.g., a single die), such as a system on a chip (SoC), or multiple discrete packaged devices integrated on a printed circuit board (PCB). In general, chiplets provide better performance (e.g., lower power consumption, reduced latency, etc.) than discrete packaged devices, and chiplets provide greater production benefits than a single bare die chip. These production benefits can include higher yields or reduced development costs and time.

[0138] A chiplet system may include, for example, one or more application (or processor) chiplets and one or more support chiplets. The distinction between application chiplets and support chiplets here is merely a reference to possible design scenarios for a chiplet system. Thus, for example, a synthetic vision chiplet system may include (by way of example only) an application chiplet to generate a synthetic vision output, and support chiplets such as a memory controller chiplet, a sensor interface chiplet, or a communication chiplet. In a typical use case, a synthetic vision designer may design the application chiplet and obtain support chiplets from other parties. Thus, design expenditure (e.g., in terms of time or complexity) is reduced by avoiding the design and production of functionality embodied in the support chiplets.

[0139] Chiplets also support tight integration of IP blocks that might otherwise be difficult, such as IP blocks manufactured using different process technologies or with different feature sizes (or with different contact technologies or pitches). Thus, multiple ICs or IC assemblies with different physical, electrical, or communication characteristics can be assembled in a modular fashion to provide assemblies with various desired functions. Chiplet systems can also facilitate adaptation to the needs of different larger systems into which the chiplet system will be incorporated. In an example, an IC or other component can be optimized for power, speed, or heat generation for a specific function—as might happen with a sensor—and can be more easily integrated with other devices than attempting to integrate them on a single die. Additionally, by reducing the overall size of the die, the yield of chiplets tends to be higher than that of more complex single-die devices.

[0140] Figure 6A and Figure 6B An example of a chiplet system according to an embodiment is generally shown. Figure 6A is a representation of a chiplet system 602 mounted on a peripheral board 604, which can be connected to a wider computer system via, for example, Peripheral Component Interconnect Express (PCIe). The chiplet system 602 includes a package substrate 606, an interposer 608, and four chiplets, an application chiplet 610, a host interface chiplet 612, a memory controller chiplet 614, and a memory device chiplet 616. Other systems may include many additional chiplets to provide additional functionality, as will be apparent from the discussion below. The package for the chiplet system 602 is shown as having a lid or cover 618, although other packaging techniques and structures for chiplet systems may be used. Figure 6B is a block diagram that labels components in a chiplet system for clarity.

[0141] The application chiplet 610 is shown to include a chiplet system NOC 620 to support a chiplet network 622 for inter-chiplet communication. In an exemplary embodiment, the chiplet system NOC 620 can be included on the application chiplet 610. In an example, the chiplet system NOC 620 from Figure 1 The first NOC 118 of an example can be defined in response to the selected supporting chiplets (e.g., host interface chiplet 612, memory controller chiplet 614, and memory device chiplet 616), thereby enabling the designer to select the appropriate number or chiplet network connections or switches for the chiplet system NOC 620. In examples, the chiplet system NOC 620 can be located on a separate chiplet or within the interposer 608. In examples as discussed herein, the chiplet system NOC 620 implements a chiplet protocol interface (CPI) network.

[0142] In an example, the chiplet system 602 may include or comprise a portion of the first memory compute node 104 or the first memory compute device 112. That is, the various blocks or components of the first memory compute device 112 may include chiplets that may be mounted on the peripheral board 604, the package substrate 606, and the interposer 608. The interface components of the first memory compute device 112 may generally comprise a host interface chiplet 612, the memory and memory control-related components of the first memory compute device 112 may generally comprise a memory controller chiplet 614, the various accelerator and processor components of the first memory compute device 112 may generally comprise an application chiplet 610 or instances thereof, and so on.

[0143] The CPI interface (such as can be used for communication between chiplets in a system) is a packet-based network that supports virtual channels to enable flexible and high-speed interaction between chiplets. CPI implements a bridge from the intra-chiplet network to the chiplet network 622. For example, the Advanced Extensible Interface (AXI) is a widely used specification for designing intra-chip communications. However, the AXI specification covers a very large number of physical design options, such as the number of physical channels, signal timing, power, etc. Within a single chip, these options are typically selected to meet design goals, such as power consumption, speed, etc. However, in order to achieve flexibility in chiplet systems, adapters such as CPI are used to interface between various AXI design options that can be implemented in various chiplets. CPI bridges the intra-chiplet network across the chiplet network 622 by implementing a mapping of physical channels to virtual channels and encapsulating time-based signaling with a packetized protocol.

[0144] The CPI can use a variety of different physical layers to transmit packets. The physical layer can include simple conductive connections or can include drivers to increase voltage or otherwise facilitate signal transmission over longer distances. One example of such a physical layer may include the Advanced Interface Bus (AIB), which, in various embodiments, may be implemented in interposer 608. The AIB uses source-synchronous data transmission with a forwarded clock to transmit and receive data. Packets are transferred across the AIB at either single data rate (SDR) or double data rate (DDR) relative to the transmitted clock. Various channel widths are supported by the AIB. Channels can be configured to have a symmetrical number of transmit (TX) and receive (RX) inputs / outputs (I / Os), or an asymmetrical number of transmitters and receivers (e.g., all transmitters or all receivers). A channel can act as an AIB master or slave depending on which chiplet provides the master clock. The AIB I / O unit supports three clock modes: asynchronous (i.e., unclocked), SDR, and DDR. In various embodiments, the unclocked mode is used for clocks and some control signals. SDR mode can use a dedicated SDR I / O-only unit, or a dual purpose SDR / DDR I / O unit.

[0145] In an example, a CPI packet protocol (e.g., point-to-point or routable) can use symmetrical receive and transmit I / O cells within an AIB channel. The CPI streaming protocol allows for more flexible use of AIB I / O cells. In an example, an AIB channel in streaming mode can configure I / O cells as all TX, all RX, or half TX and half RX. The CPI packet protocol can use AIB channels in SDR or DDR operating modes. In an example, AIB channels are configured in increments of 80 I / O cells (i.e., 40 TX and 40 RX) for SDR mode and in increments of 40 I / O cells for DDR mode. The CPI streaming protocol can use AIB channels in SDR or DDR operating modes. Here, in an example, AIB channels are in increments of 40 I / O cells for both SDR and DDR modes. In an example, each AIB channel is assigned a unique interface identifier. The identifier is used during CPI reset and initialization to determine paired AIB channels across adjacent chiplets. In this example, the interface identifier is a 20-bit value consisting of a seven-bit chiplet identifier, a seven-bit column identifier, and a six-bit link identifier. The AIB physical layer uses the AIB out-of-band shift register to transmit the interface identifier. Bits 32 to 51 of the shift register are used to transmit the 20-bit interface identifier in both directions across the AIB interface.

[0146] AIB defines a stacked group of AIB channels as an AIB channel column. An AIB channel column has a certain number of AIB channels, plus auxiliary channels. The auxiliary channels contain signals used for AIB initialization. All AIB channels in a column (except the auxiliary channels) have the same configuration (for example, all TX, all RX, or half TX and half RX, and the same number of data I / O signals). In this example, the AIB channels are numbered in consecutive increasing order, starting with the AIB channel adjacent to the AUX channel. The AIB channel adjacent to the AUX is defined as AIB channel zero.

[0147] Typically, the CPI interface on an individual chiplet may include serialization-deserialization (SERDES) hardware. SERDES interconnects are well suited for scenarios requiring high-speed signaling and low signal counts. However, SERDES may result in additional power consumption and longer latency for multiplexing and demultiplexing, error detection or correction (e.g., using block-level cyclic redundancy checks (CRCs)), link-level retries, or forward error correction. However, when low latency or energy consumption is the primary concern for ultra-short distance chiplet-to-chiplet interconnects, a parallel interface may be utilized whose clock rate allows data transfer with minimal latency. CPI includes elements for minimizing both latency and energy consumption for these ultra-short distance chiplet interconnects.

[0148] For flow control, CPI employs a credit-based technique. A receiver, such as application chiplet 610, provides credits representing available buffers to a transmitter, such as memory controller chiplet 614. In this example, a CPI receiver includes a buffer for each virtual channel for a given transmit time unit. Thus, if a CPI receiver supports five messages and a single virtual channel at a time, the receiver has five buffers arranged in five rows (e.g., one row per unit of time). If four virtual channels are supported, the receiver has twenty buffers arranged in five rows. Each buffer holds the payload of a CPI packet.

[0149] As the sender transmits to the receiver, the sender decrements the available credits based on the transmission. Once all credits of the receiver have been consumed, the sender stops sending packets to the receiver. This ensures that the receiver always has an available buffer to store the transmission.

[0150] When the receiver processes the received packet and frees up the buffer, the receiver transfers the available buffer space back to the sender. The sender can then use this credit back to allow the transmission of additional information.

[0151] Figure 6A Examples include a chiplet mesh network 624 that uses direct chiplet-to-chiplet technology without the need for a chiplet system NOC 620. The chiplet mesh network 624 can be implemented using CPI or another chiplet-to-chiplet protocol. The chiplet mesh network 624 typically implements a pipeline of chiplets, where one chiplet acts as an interface to the pipeline, while the other chiplets in the pipeline only interface with themselves.

[0152] In addition, dedicated device interfaces, such as one or more industry standard memory interfaces (such as, for example, synchronous memory interfaces, such as DDR5, DDR6), can be used to connect the device to the chiplet. The connection of the chiplet system or an individual chiplet to an external device (such as a larger system) can be through a desired interface (e.g., a PCIe interface). In an example, such an external interface can be implemented through a host interface chiplet 612, which in the depicted example provides a PCIe interface external to the chiplet system. Such dedicated chiplet interfaces 626 are typically used when industry conventions or standards have converged on such interfaces. The illustrated example of a double data rate (DDR) interface connecting a memory controller chiplet 614 to a dynamic random access memory (DRAM) memory device chiplet 616 is such an industry convention.

[0153] Among the various possible supporting chiplets, the memory controller chiplet 614 is likely to be present in a chiplet system due to the almost ubiquitous use of memory for computer processing and the advanced technology of memory devices. Therefore, using a memory device chiplet 616 and a memory controller chiplet 614 produced by others enables chiplet system designers to have access to robust products produced by experienced manufacturers. Typically, the memory controller chiplet 614 provides a memory device-specific interface to read, write, or erase data. Typically, the memory controller chiplet 614 can provide additional features such as error detection, error correction, maintenance operations, or atomic operator execution. For some types of memory, maintenance operations are often specific to the memory device chiplet 616, such as garbage collection in NAND flash memory or storage class memory, temperature adjustment in NAND flash memory (e.g., cross temperature management). In an example, maintenance operations can include logical to physical (L2P) mapping or management to provide a level of indirection between the physical and logical representations of data. In other types of memory, such as DRAM, some memory operations, such as refresh, may be controlled at certain times by a host processor or memory controller and at other times by the DRAM memory device or logic associated with one or more DRAM devices, such as an interface chip (in an example, a buffer).

[0154] Atomic operators are data manipulations that can be performed, for example, by the memory controller chiplet 614. In other chiplet systems, atomic operators can be performed by other chiplets. For example, an "increment" atomic operator can be specified by the application chiplet 610 in a command that includes a memory address and possibly an increment value. Upon receiving the command, the memory controller chiplet 614 retrieves a number from the specified memory address, increments the number by the amount specified in the command, and stores the result. Upon successful completion, the memory controller chiplet 614 provides an indication of command success to the application chiplet 610. Atomic operators avoid transmitting data across the chiplet mesh network 624, resulting in lower latency execution of such commands.

[0155] Atomic operators can be categorized as built-in atomics or programmable (e.g., custom) atomics. Built-in atomics are a limited set of operations that are immutably implemented in hardware. Programmable atomics are small programs that can be executed on the programmable atomic unit (PAU) (e.g., custom atomic unit (CAU)) of the memory controller chiplet 614.

[0156] The memory device chiplet 616 can be or include any combination of volatile memory devices or non-volatile memory. Examples of volatile memory devices include, but are not limited to, random access memory (RAM) such as DRAM, synchronous DRAM (SDRAM), Graphics Double Data Rate Type 6 SDRAM (GDDR6 SDRAM), and the like. Examples of non-volatile memory devices include, but are not limited to, NAND flash memory, storage class memory (e.g., phase change memory or memristor-based technology), ferroelectric RAM (FeRAM), and the like. The illustrated example includes the memory device chiplet 616 as a chiplet; however, the device can reside elsewhere, such as in a different package on the peripheral board 604. For many applications, multiple memory device chiplets can be provided. In an example, these memory device chiplets can each implement one or more memory technologies and can include an integrated computing host. In an example, a memory chiplet can include multiple stacked memory dies of different technologies, such as one or more static random access memory (SRAM) devices stacked or otherwise communicating with one or more dynamic random access memory (DRAM) devices. In an example, a memory controller chiplet 614 can be used to coordinate operations between multiple memory chiplets in a chiplet system 602, for example, using one or more memory chiplets in one or more levels of cache storage and using one or more additional memory chiplets as main memory. The chiplet system 602 may include multiple memory controller chiplet 614 instances, such as can be used to provide memory control functionality for separate hosts, processors, sensors, networks, etc. The chiplet architecture in the system shown provides advantages in allowing different memory storage technologies and different memory interfaces to be accommodated through updated chiplet configurations, such as without requiring a redesign of the rest of the system architecture.

[0157] Figure 7 An example of a chiplet-based implementation for a compute-in-memory device according to an embodiment is generally shown. The example includes an implementation with four compute-in-memory or CNM chiplets, and each CNM chiplet may include or contain a Figure 1 112 or a portion of a first memory computing device 112 or a first memory computing node 104 of an instance of the present invention. Each portion may itself include or comprise a respective chiplet. A chiplet-based implementation may include or use CPI-based intra-system communication, as described above in the context of Figure 6A and Figure 6B The example chiplet system 602 is similarly discussed.

[0158] Figure 77. The example of a first CNM package 700 includes a plurality of chiplets. The first CNM package 700 includes a first chiplet 702, a second chiplet 704, a third chiplet 706, and a fourth chiplet 708 coupled to a CNM NOC hub 710. Each of the first to fourth chiplets can include instances of the same or substantially the same components or modules. For example, the chiplets can each include an HTP accelerator, an HTF accelerator, and an instance of a memory controller for accessing internal or external memory.

[0159] exist Figure 7 In the example of FIG. 7 , the first chiplet 702 includes a first NOC hub edge 714 coupled to a CNM NOC hub 710. Other chiplets in the first CNM package 700 similarly include NOC hub edges or endpoints. Switches in the NOC hub edge facilitate communication within a chiplet or within a system of chiplets via the CNM NOC hub 710.

[0160] The first chiplet 702 can further include one or more memory controllers 716. The memory controllers 716 can correspond to respective different NOC endpoint switches that interface with the first NOC hub edge 714. In an example, the memory controller 716 includes the memory controller chiplet 614, or includes the memory controller 130, or includes the memory subsystem 200, or other memory computing implementations. The memory controller 716 can be coupled to respective different memory devices, such as the first external memory module 712a or the second external memory module 712b. The external memory modules can include, for example, GDDR6 memory, which can be selectively accessed by respective different chiplets in the system.

[0161] The first chiplet 702 may further include a first HTP chiplet 718 and a second HTP chiplet 720, which are coupled to the first NOC hub edge 714, such as via respective NOC endpoint switches. The HTP chiplet may correspond to an HTP accelerator, such as from Figure 1 HTP 140 for example, or from Figure 4 The HTP chiplet can communicate with the HTF chiplet 722. The HTF chiplet 722 can correspond to the HTF accelerator (e.g., from Figure 1 HTF 142 of the example, or from Figure 5 HTF 500 of the example).

[0162] The CNM NOC hub 710 can be coupled to other chiplets or NOC hub instances in other CNM packages through various interfaces and switches. For example, the CNM NOC hub 710 can be coupled to a CPI interface through multiple different NOC endpoints on the first CNM package 700. Each of the multiple different NOC endpoints can be coupled to, for example, a different node external to the first CNM package 700. In an example, the CNM NOC hub 710 can be coupled to other peripheral devices, nodes, or devices using CTCPI or other non-CPI protocols. For example, the first CNM package 700 can include a PCIe Extended Fabric Interface (PCIE / SFI) or a CXL interface (CXL) configured to interface the first CNM package 700 with other devices. In an example, the devices to which the first CNM package 700 is coupled using various CPI, PCIe, CXL, or other fabrics can form a common global address space.

[0163] exist Figure 7 In the example of FIG, the first CNM package 700 includes a host interface 724 (HIF) and a host processor (R5). The host interface 724 may correspond to, for example, a host processor from Figure 1 The host processor or R5 may correspond to the HIF 120 from Figure 1 The host interface 724 may include a PCI interface for coupling the first CNM package 700 to other external devices or systems. In an example, operations may be initiated on the first CNM package 700 or a cluster of tiles within the first CNM package 700 via the host interface 724. For example, the host interface 724 may be configured to command individual HTF tile clusters, such as in various chiplets within the first CNM package 700, to enter and exit power / clock gating modes.

[0164] Figure 8 An exemplary diagram of a memory computing device according to an embodiment is shown. Figure 8 In FIG, tiled chiplet instance 800 includes four instances of different adjacent memory compute clusters of a chiplet, where the clusters are coupled together. Each instance of the adjacent memory compute chiplet may itself include one or more constituent chiplets (e.g., a host processor chiplet, a memory device chiplet, an interface chiplet, and so on).

[0165] The tiled chiplet example 800 includes Figure 7700 as one or more of its neighboring compute-in-memory (CNM) clusters. For example, tiled chiplet instance 800 may include a first CNM cluster 802, which includes a first chiplet 810 (e.g., corresponding to first chiplet 702), a second chiplet 812 (e.g., corresponding to second chiplet 704), a third chiplet 814 (e.g., corresponding to third chiplet 706), and a fourth chiplet 816 (e.g., corresponding to fourth chiplet 708). The chiplets in first CNM cluster 802 may be coupled to a common NOC hub, which in turn may be coupled to a NOC hub in one or more neighboring clusters (e.g., in second CNM cluster 804 or fourth CNM cluster 808).

[0166] exist Figure 8 In the example of FIG. 8 , a tiled chiplet instance 800 includes a first CNM cluster 802, a second CNM cluster 804, a third CNM cluster 806, and a fourth CNM cluster 808. The various CNM chiplets can be configured in a common address space so that the chiplets can allocate and share resources across different tiles. In the example, the chiplets in the clusters can communicate with each other. For example, the first CNM cluster 802 can be communicatively coupled to the second CNM cluster 804 via an inter-chiplet CPI interface 818, and the first CNM cluster 802 can be communicatively coupled to the fourth CNM cluster 808 via another or the same CPI interface. The second CNM cluster 804 can be communicatively coupled to the third CNM cluster 806 via the same or another CPI interface, and so on.

[0167] In an example, one of the neighboring memory compute chiplets in the tiled chiplet instance 800 may include a host interface (e.g., corresponding to a host interface from the tiled chiplet instance 800) responsible for workload balancing across the tiled chiplet instance 800. Figure 7 The host interface can facilitate access to host-based command request queues and response queues, such as from outside the tiled chiplet instance 800. In the tiled chiplet instance 800, the host interface can schedule new execution threads using a hybrid threaded processor and a hybrid threaded fabric in one or more adjacent memory compute chiplets.

[0168] Figure 9 is a diagram illustrating one example of a workflow 900 for performing operations on a reconfigurable computing structure such as HTF 142 or HTF 500 using more than one simultaneous stream. Figure 9The workflow 900 utilizes a scheduling interface 902 and two synchronization streams 901, 903. As described herein, the scheduling interface 902 is configured to receive instructions and / or data from other parts of the adjacent memory computing system. In some examples, the scheduling interface 902 is arranged in a manner similar to the scheduling interface 508 described herein.

[0169] The synchronization flows 901, 903 include respective flow controllers 904, 906 and ordered synchronization data paths 908, 910, as described herein. The synchronization data paths 908, 910 may, for example, be between the computing elements of the respective synchronization flows 901, 903, as described herein. The flow controllers 904, 906 and the synchronization data paths 908, 910 are arranged from a reconfigurable computing structure (such as the HTF 142 or HTF 500 described herein).

[0170] The reconfigurable computing structure can be arranged to implement various computing elements of the synchronization flow 901, 903, including flow controllers 904, 906 and synchronization data paths 908, 910. For example, the flow controllers 904, 906 can be implemented using the corresponding tile base of the tiles 510, 512, etc. The synchronization data paths 908, 910 can be implemented between the tiles 510, 512, etc., where the tiles 510, 512, etc. implement the computing elements described herein. As described herein and Figure 5 The tiles, tile bases, etc. shown may be components of a hybrid threading structure (HTF) such as HTF 142, which is a memory computing device such as Figure 1 The other synchronization flows described herein may be similarly arranged using the tiles and tile basis of the HTF as described herein.

[0171] exist Figure 9 In the workflow 900 of FIG. 1 , the domain interface 902 initiates a processing task by providing an asynchronous message 912 to the flow controller 904 via the asynchronous structure of the reconfigurable computing structure. The asynchronous message 912 can be a continuous type message. The domain interface 902 can initiate the processing task based on its own programming and / or in response to instructions from another component of the reconfigurable computing structure of the adjacent memory computing system. In some examples, the domain interface 902 also configures the reconfigurable computing structure to the correct kernel configuration for executing the processing task, which includes, for example, programming the various flow controllers 904, 906 and computing elements to implement the synchronous flows 901, 903.

[0172] Asynchronous message 912 may instruct flow controller 904 to start a synchronous flow thread. For example, when a branch RAM selects flow controller 904 for a synchronous data path 908 between components of synchronous flow 901, the flow controller may initiate the thread by providing a first synchronization message to a first computing element of synchronous data path 908. As described herein, a synchronization message may include data and control information. The first computing element of synchronous data path 908 begins execution of the thread by performing a processing task and generating a next synchronization message that is sent to a second computing element of synchronous data path 908. The second computing element performs a processing task and generates a third synchronization message that is sent to a third computing element of synchronous data path 908, and so on, until the thread is executed on all computing elements of synchronous data path 908.

[0173] The computing elements of synchronization data path 908 are programmed to perform various operations when executing threads, including, for example, operations for configuring synchronization flow 903. For example, during execution of a thread at synchronization flow 901, one or more computing elements of synchronization data path 908 send an asynchronous message 914 to flow controller 906 of synchronization flow 903. Asynchronous message 914 instructs flow controller 906 to launch a thread at synchronization flow 903 conditional on the occurrence of at least one condition. In this example, the one or more conditions include receiving another asynchronous message from synchronization flow 901 indicating that the thread has completed or that it is otherwise acceptable to launch a thread at synchronization flow 903. Later in the execution of the thread at synchronization flow 901, the computing elements of synchronization data path 908 send an asynchronous message 916 to flow controller 906 of synchronization flow 903. Asynchronous message 916 indicates that a thread at synchronization flow 901 has completed and / or has reached a point at which a corresponding thread at synchronization flow 903 can begin. In some examples, asynchronous message 916 indicates the results of processing during execution of a thread at synchronous flow 903 .

[0174] Upon receiving asynchronous message 916 (and when other conditions (if any) of asynchronous message 914 are met), flow controller 906 starts the thread at synchronous flow 903, for example, by sending a first synchronous message to a first computing element of synchronous data path 910. In some examples, for example, when the thread at synchronous flow 903 completes, the computing element of synchronous data path 910 will send an asynchronous message 918 to scheduling interface 902 to indicate that the requested processing task has been completed.

[0175] Figure 101 is a diagram illustrating an example of a workflow 1000 for executing operations on a reconfigurable computing fabric, such as HTF 142 or HTF 500, wherein synchronization flows 1001 and 1003 interact with a memory interface 1016. In workflow 1000, domain interface 1002 initiates a processing task by providing an asynchronous message 1010 to flow controller 1004 via the asynchronous fabric of the reconfigurable computing fabric. Asynchronous message 1010 instructs flow controller 1004 to initiate a thread at synchronization flow 1001 when one or more conditions occur. (In some examples, message 1010 may instruct flow controller 1004 to begin execution of the thread immediately, without waiting for any conditions to occur.)

[0176] As described herein, flow controller 1004 initiates a thread by providing a synchronization message to a first computing element of synchronization data path 1022. During execution of the thread at synchronization flow 1001, the computing element of synchronization data path 1022 provides an asynchronous message 1012 to flow controller 1006 of synchronization flow 1003. Asynchronous message 1012 may instruct flow controller 1006 to initiate the thread at synchronization flow 1003 upon receiving an indication that data from memory interface 1016 has been received.

[0177] As part of the execution of the thread at synchronization flow 1001, another computing element of synchronization data path 1022 provides an asynchronous message 1014 to memory interface 1016. Asynchronous message 1014 contains a read request to be directed to the memory of a memory-adjacent computing system (such as, for example, a memory computing device including a reconfigurable computing fabric). In some instances, asynchronous message 1014 may also identify the computing element of synchronization flow 1003 that will receive the result of the read request.

[0178] Memory interface 1016 executes a read request on the appropriate system memory to obtain the load data. When the load data is received, memory interface 1016 directs an asynchronous message 1018 including the load data to the computing element of the synchronous flow 1003 indicated by asynchronous message 1014 (e.g., one of the computing elements of synchronous data path 1024). Upon receiving the load data, the computing element writes the load data to the computing element memory and sends an asynchronous message 1017 to the flow controller 1006 of the synchronous flow 1003. Message 1017 indicates that the load data has been received. In some instances, similar to message 1018, memory interface 1016 provides asynchronous messages including the load data to multiple computing elements of the synchronous flow 1003. Similar to message 1017, each computing element that has received the load data can provide a corresponding asynchronous message to the flow controller 1006.

[0179] Upon receiving the asynchronous message 1017 (and after any other conditions of the continue-type message 1012 are met), the flow controller 1006 starts the thread at the synchronous flow 1003, for example, by providing a synchronization message to the first computing element of the synchronous data path 1024. If more than one computing element of the synchronous flow 1003 is to receive load data, the flow controller 1006 can start the thread when it has received all expected asynchronous messages from the computing elements that have received the load data. In some examples, when the thread at the synchronous flow 1003 is completed, one or more computing elements (e.g., the last computing element thereof) of the synchronous data path 1024 sends an asynchronous message 1020 indicating completion to the scheduling interface 1002.

[0180] Figure 11 1 is a diagram illustrating one example of a workflow 1100 for executing a loop 1122 using a reconfigurable computing structure such as HTF 142 or HTF 500. The workflow 1100 is illustrated using three synchronization streams 1101, 1103, 1105. In this example, each iteration of the loop 1122 is executed at a single synchronization stream 1103 using a single thread.

[0181] Flow controller 1102 initiates the first thread at synchronous flow 1101, for example, by providing a synchronization message to the first computational element of synchronous data path 1108. Execution of the thread at synchronous flow 1101 includes sending an asynchronous message 1114 to flow controller 1106 of synchronous flow 1105. Asynchronous message 1114 may indicate that flow controller 1106 will execute the thread at synchronous flow 1105 after receiving an indication that loop 1122 has completed. Execution of the thread at synchronous flow 1101 also includes sending an asynchronous message 1116 to flow controller 1104 of synchronous flow 1103. Asynchronous message 1116 instructs flow controller 1104 to execute the loop. For example, asynchronous message 1116 may include an iteration count indicating the number of iterations in the loop.

[0182] Flow controller 1104 starts a thread for the first iteration of the loop. The computational elements of the synchronization data path 1110 of synchronization domain 1103 execute the first iteration of the loop. When the thread completes, one or more computational elements send an asynchronous message 1120 to flow controller 1104 indicating that the first iteration of the loop has completed.

[0183] In some examples, after launching the first thread for the first iteration of the loop, the flow controller 1104 continues to launch additional threads for executing additional iterations of the loop. For example, if the flow controller 1104 has a branch count of three, it can launch a new thread at the synchronization flow 1103 for a new iteration of the loop every three clock cycles. This can occur regardless of whether the thread of the previous loop iteration has completed propagation through the computational elements of the synchronization data path 1110. In some examples, various threads implementing different loop iterations can execute simultaneously at the computational elements of the synchronization data path 1110 of the synchronization flow 1103 (e.g., offset branch counts).

[0184] When the threads implementing all iterations of loop 1122 have completed, flow controller 1104 will have received asynchronous messages from all threads implementing the iterations. When this has occurred, flow controller 1104 sends an asynchronous message 1118 to flow controller 1106 indicating that loop 1122 has completed. Flow controller 1106 can respond by starting threads at compute elements 1112 to implement the next portion of the programming of the reconfigurable compute fabric.

[0185] Figure 11 Various exemplary elements are shown that can be combined to implement loops in different contexts. First, although Figure 11 The example of shows a loop 1122 started by a thread at the synchronization flow 1101, but in some examples, the loop can be started by another component such as a scheduling interface. Figure 11 In the example of , the completion of loop 1122 triggers another thread at another synchronization flow 1105, but in some examples, the completion of the loop can indicate the completion of the processing task. Therefore, the synchronization message 1118 indicating the completion of the loop can be provided to the scheduling interface in addition to or instead of being provided to the scheduling interface such as Figure 11 Another flow controller 1106 is shown.

[0186] also, Figure 11 Elements that can be used to implement nested loops are shown. For example, a thread at synchronization flow 1101 that initiates loop 1122 may be executing an iteration of an outer loop that calls loop 1122 as an inner loop. Furthermore, in some instances, for example, when a thread at synchronization flow 1101 causes synchronization flow 1103 to execute loop 1122, each thread at synchronization flow 1103 that implements an iteration of loop 1122 may cause another synchronization flow (not shown) to implement another loop.

[0187] exist Figure 11In the example of , each iteration of loop 1122 is executed using a single synchronization stream 1103. However, in some examples, the execution of the loop iteration can utilize multiple synchronization streams. For example, executing a thread for an iteration of loop 1122 can include sending an asynchronous message to a second flow controller and / or a memory interface. The second flow controller starts a thread at another synchronization stream (not shown) to execute another part of the loop iteration. In this example, when the loop iteration is completed, the synchronization stream that completed the loop iteration sends an asynchronous message to the flow controller that started the loop (e.g., flow controller 1104).

[0188] Figure 12 142 or HTF 500 to execute a loop using a last iteration flag. Column 1200 illustrates an example of a process flow 1200 that can be implemented using a reconfigurable computing structure, such as HTF 142 or HTF 500, to execute a loop using a last iteration flag. Process flow 1200 includes four columns 1201, 1203, 1205, and 1207. Each column 1201, 1203, 1205, and 1207 illustrates actions performed by different components of the near-memory computing system (including, for example, other components of the reconfigurable computing structure). Column 1201 includes actions performed by the loop start component. Column 1203 includes actions performed by the first loop flow controller (e.g., the flow controller that executes the synchronization flow for the thread that starts the loop iteration execution). Column 1205 indicates one or more asynchronous operation components that perform asynchronous operations that may occur during the execution of the loop iteration. For example, column 1205 may indicate operations performed by a memory interface or any other asynchronous operations. Column 1207 indicates operations performed by the last loop flow controller. The last loop flow controller is the flow controller at the synchronization flow that executes the last thread of the loop iteration.

[0189] At operation 1202, a loop initiation component sends an asynchronous message 1209 to a first loop flow controller via an asynchronous structure of the reconfigurable computing fabric. Asynchronous message 1209 instructs the first loop flow controller to begin execution of a loop. Asynchronous message 1209 may include an iteration count indicating the number of iterations of the loop to be completed. The initiation component that sends asynchronous message 1209 may be, for example, a scheduling interface, a memory interface, or a component of another synchronization flow.

[0190] At operation 1204, the first loop flow controller receives an asynchronous message 1209. At operation 1206, the first loop flow controller starts a thread at the first synchronous flow to execute the first iteration of the loop. At operation 1208, the first loop flow controller determines whether there are any remaining or remaining iterations for which threads have not yet been started. If so, the first loop flow controller returns to operation 1206 and starts the next thread at the first synchronous flow for the next iteration of the loop. For example, the next thread may be started after the branch count number of clock cycles has passed since the thread of the previous iteration of the loop was started.

[0191] If the first loop flow controller determines at operation 1208 that threads at the first synchronization flow have been started for all iterations of the loop, in some instances, it determines at operation 1210 whether it has received an indication that all iterations of the loop have completed. For example, the component that completed the loop iteration may send an asynchronous message to the first loop flow controller indicating that the particular loop iteration has completed. If the first loop flow controller has not yet received an indication that all iterations of the loop have completed, it may continue to wait and recheck at operation 1210. When the first loop flow controller determines that all iterations of the loop have completed, it may send an asynchronous message 1211 to the loop starting component at operation 1212 indicating that the loop has completed. At operation 1214, the loop starting component may receive message 1211. After receiving the load complete message, the loop starting component may continue its processing.

[0192] At operation 1206, various threads launched by the first loop flow controller for different loop iterations may prompt at least one asynchronous operation to be performed by one or more asynchronous operation components (e.g., column 1205). For example, a synchronization thread may initiate a memory load and / or write performed by a memory interface, or any other operation that is not completed synchronously. Based on the initial thread launched at operation 1206, any suitable number of asynchronous operations and / or synchronization threads may be executed.

[0193] Completion of the asynchronous operations performed by one or more asynchronous operation components at column 1205 can be indicated by an asynchronous message 1215 provided to the last loop flow controller. Asynchronous message 1215 can be sent to the last loop flow controller, for example, via a memory interface, a computing element of the last loop synchronization flow, or the like. Asynchronous message 1215 can instruct the last loop flow controller to launch a thread at the last loop synchronization flow to complete the corresponding loop iteration. In some instances, asynchronous message 1215 includes data, such as load data, to be used by the last loop synchronization flow to execute the last loop thread, or otherwise indicates that load data has been received by the last loop synchronization flow. The last loop flow controller may receive an instance of asynchronous message 1215 for each iteration of the loop. As explained herein, messages 1215 may not be received in the order in which the loop iterations were launched. Because the threads launched by the last loop flow controller are synchronous, they may complete in the order in which they were launched. For example, the loop iteration corresponding to the last thread launched by the last loop flow controller will also be the last loop iteration to be completed.

[0194] The last loop flow controller may also be provided with an iteration count 1213 indicating the number of iterations in the loop. The iteration count 1213 may be provided to the last loop flow controller in any suitable manner. In some instances, the iteration count 1213 is included in a synchronization message provided by the first loop flow controller to start threads for various iterations. The iteration count may be passed between asynchronous operating components, for example, via asynchronous messages, and ultimately provided to the last loop flow controller via asynchronous messages 1215. In another instance, the iteration count 1213 is provided to the last loop flow controller when the configuration computing structure is programmed. In another instance, the iteration count 1213 is provided to the last loop flow controller by the scheduling interface, for example, when the scheduling interface instructs the first flow controller to begin executing the first thread of the processing task.

[0195] At operation 1216, the last loop flow controller receives the asynchronous message 1215. At operation 1218, the last loop flow controller updates the iteration counter. The iteration counter is used to track the total number of loop iterations that have been processed by the last loop flow controller. For example, the last loop flow controller can start the counter at zero and increment the counter each time an asynchronous message 1215 for a loop iteration is received. When the counter is equal to the iteration count 1213, the current asynchronous message 1215 corresponds to the last loop iteration to be completed. In another example, the last loop flow controller can start the counter at the iteration count 1213 and decrement the counter each time an asynchronous message 1215 for a loop iteration is received. When the counter is equal to zero, the current asynchronous message 1215 corresponds to the last loop iteration to be completed.

[0196] At operation 1220, the last loop flow controller determines whether the current iteration (e.g., the iteration corresponding to asynchronous message 1215) is the last iteration to be completed. For reasons described herein, the last loop iteration to be completed may not be the last loop iteration to be started. If the asynchronous message 1215 does not correspond to the last loop iteration to be completed, then at operation 1222, the last loop flow controller starts a thread at the last loop synchronization flow to continue execution of the loop iteration without setting the last iteration flag and waits for the next asynchronous message 1215. If the asynchronous message 1215 does correspond to the last loop iteration to be completed, then at operation 1224, the last loop flow controller starts a thread to continue execution of the loop iteration with the last iteration flag set.

[0197] The computational elements of the last loop synchronization flow are programmed to execute a thread including at least one conditional operation, the conditional operation being conditional on the value of a last iteration flag. If the last iteration flag is set, such an operation will be executed in the thread, and if the last iteration flag is not set, such an operation will not be executed. Therefore, if the last iteration flag is not set, the thread launched at operation 1222 may not execute the conditional operation. If the last iteration flag is set, the thread launched at operation 1224 may execute the conditional operation.

[0198] Figure 13 is a diagram illustrating an example of a workflow 1300 for executing loop 1340 using a reconfigurable computing structure such as HTF 142 or HTF 500 using a last iteration flag. For example, workflow 1300 provides an example of how loop 1340 may be executed. Figure 12 1300 shows four exemplary synchronization flows 1301, 1303, 1305, 1307, including respective flow controllers 1302, 1304, 1306, 1308 and computational elements comprising respective synchronization data paths 1310, 1312, 1314, 1316. Workflow 1300 also shows a scheduling interface 1318 and memory interfaces 1320A, 1320B.

[0199] exist Figure 13 In the example of , at least two synchronization streams 1303 and 1305 are used to perform loop 1340. Synchronization stream 1303 corresponds to Figure 12 The first loop synchronization flow of , and the execution thread to start the execution of the loop iteration. Synchronous flow 1305 execution thread to end the execution of the loop iteration, and corresponds to Figure 12 The final loop synchronizes the stream.

[0200] Loop 1340 can be started in any suitable manner. In some instances, loop 1340 is started by a thread at synchronous stream 1301. The thread at synchronous stream 1301 can implement iterations of the loop, which is an external loop relative to loop 1340. In other instances, the thread at synchronous stream 1301 implements other non-loop work. The thread at synchronous stream 1301 starts loop 1340 by providing an asynchronous message 1322A to the flow controller 1304, which instructs the flow controller 1304 to start threads for various iterations of loop 1340. The asynchronous message 1322A may include an iteration count indicating the number of loop iterations to be performed. In some instances, the thread at synchronous stream 1301 also provides another asynchronous message indicating the iteration count to the flow controller 1306, as described herein.

[0201] In another example, loop 1340 is initiated by scheduling interface 1318. For example, scheduling interface 1318 can provide an asynchronous message 1322B to flow controller 1304, instructing flow controller 1304 to start threads for iterations of loop 1340. Asynchronous message 1322B can include an iteration count indicating the number of loop iterations to be executed. In some examples, scheduling interface 1318 also provides another asynchronous message indicating the iteration count to flow controller 1306, as described herein.

[0202] Iterations of loop 1340 can switch between synchronization stream 1303 (the first loop synchronization stream) and synchronization stream 1305 (the last loop synchronization stream) in any suitable manner. In some instances, a thread at synchronization stream 1303 directly prompts flow controller 1306 to start a corresponding thread to end the loop iteration. For example, a thread at synchronization stream 1303 can provide an asynchronous message 1324 to flow controller 1306. Asynchronous message 1324 can instruct flow controller 1306 to start a thread when a condition occurs, where the condition can be receiving another asynchronous message 1330A from the thread at synchronization stream 1303, receiving a synchronization message 1330B from memory interface 1320B, and / or receiving an asynchronous message 1330C from an execution element of synchronization stream 1305 indicating that load data from a read request has been received (e.g., in asynchronous message 1331 from memory interface 1320A). In some examples, a thread at synchronous stream 1303 may send an asynchronous message 1333 to memory interface 1320A, causing memory interface 1320A to perform the memory operation reported to synchronous stream 1305 via asynchronous message 1331 or 1330B.

[0203] For example, as described herein, flow controller 1306 receives an iteration count indicating the total number of iterations of loop 1340. When flow controller 1306 launches threads for various loop iterations, it tests whether the thread to be launched corresponds to the last iteration of loop 1340 to be completed. If the thread corresponds to the last iteration to be completed, flow controller 1306 launches the thread with the last iteration flag set. The threads of synchronization flow 1305 include at least one conditional operation that is executed (or not executed) based on the value of the last iteration flag.

[0204] Figure 13 Also shown are examples of one or more conditional operations performed by the synchronous flow 1305 in the thread with the last iteration flag set. For example, the conditional operation can be or include sending an asynchronous message 1332A to the flow controller 1308 of another synchronous flow 1307 to execute one or more additional threads to perform additional operations. In other examples, the conditional operation can be or include sending an asynchronous message 1332B to a memory interface (e.g., memory interface 1320B) to, for example, write to a memory location, clear a memory location, read from a memory location, or perform any other suitable operation.

[0205] Figure 14A block diagram of an exemplary machine 1400 is shown, in which any one or more of the techniques (e.g., methods) discussed herein may be implemented, implemented, or performed by logic or multiple components or mechanisms within the machine 1400. A circuit system (e.g., processing circuit system) is a collection of circuits implemented within the tangible physical structure of the machine 1400, including hardware (e.g., simple circuits, gates, logic, etc.). Circuit system membership can be flexible over time. A circuit system includes components that, when operated, can perform specific operations individually or in combination. In an example, the hardware of the circuit system can be designed in an immutable manner to perform specific operations (e.g., hard-wired). In an example, the hardware of the circuit system can include variably connected physical components (e.g., execution units, transistors, simple circuits, etc.), including physically modified machine-readable media (e.g., magnetic, electrical, removable placement of fixed aggregated particles, etc.) to encode instructions for specific operations. When connecting physical components, the fundamental electrical properties of the hardware components are changed, such as from an insulator to a conductor, or vice versa. Instructions enable embedded hardware (e.g., an execution unit or loading mechanism) to create a component of a circuit system in hardware via variable connections to perform a portion of a specific operation when in operation. Thus, in an example, the machine-readable media element is part of the circuit system or is communicatively coupled to other components of the circuit system when the device is in operation. In an example, any of the physical components can be used in more than one component in more than one circuit system. For example, in operation, an execution unit can be used in a first circuit in a first circuit system at one point in time and reused by a second circuit in the first circuit system, or reused by a third circuit in the second circuit system at a different time. Additional examples of these components are described herein with respect to machine 1400.

[0206] In alternative embodiments, machine 1400 can operate as a standalone device or can be connected (e.g., networked) to other machines. In a networked deployment, machine 1400 can operate in a server-client network environment as a server machine, a client machine, or both. In an example, machine 1400 can act as a peer machine in a peer-to-peer (P2P) (or other distributed) network environment. Machine 1400 can be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile phone, a network appliance, a network router, a switch or a bridge, or any machine capable of executing (sequentially or otherwise) instructions specifying actions to be taken by the machine. In addition, although only a single machine is shown, the term "machine" should also be construed to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methods discussed herein, such as cloud computing, software as a service (SaaS), and other computer cluster configurations.

[0207] The machine 1400 (e.g., a computer system) may include a hardware processor 1402 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a hardware processor core, or any combination thereof), a main memory 1404, a static memory 1406 (e.g., memory or storage for firmware, microcode, basic input and output (BIOS), a unified extensible firmware interface (UEFI), etc.), and a mass storage device 1408 (e.g., a hard drive, a tape drive, a flash memory device, or other block device), some or all of which may communicate with each other via an interconnect 1430 (e.g., a bus). The machine 1400 may further include a display device 1410, an alphanumeric input device 1412 (e.g., a keyboard), and a user interface (UI) navigation device 1414 (e.g., a mouse). In an example, the display device 1410, the input device 1412, and the UI navigation device 1414 may be a touch screen display. The machine 1400 may further include a mass storage device 1408 (e.g., a drive unit), a signal generating device 1418 (e.g., a speaker), a network interface device 1420, and one or more sensors 1416, such as a global positioning system (GPS) sensor, a compass, an accelerometer, or other sensors. The machine 1400 may include an output controller 1428, such as a serial (e.g., universal serial bus (USB), parallel or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connection to communicate with or control one or more peripheral devices (e.g., a printer, a card reader, etc.).

[0208] The registers of the hardware processor 1402, main memory 1404, static memory 1406, or mass storage 1408 may be or include machine-readable media 1422 on which one or more sets of data structures or instructions 1424 (e.g., software) are stored that embody or are used by any one or more of the techniques or functionality described herein. During execution of the instructions 1424 by the machine 1400, the instructions may also reside, completely or at least partially, in any register of the hardware processor 1402, main memory 1404, static memory 1406, or mass storage 1408. In an example, one or any combination of the hardware processor 1402, main memory 1404, static memory 1406, or mass storage 1408 may constitute the machine-readable media 1422. Although the machine-readable medium 1422 is shown as a single medium, the term “machine-readable medium” may include a single medium or multiple media (eg, a centralized or distributed database, or associated caches and servers) configured to store one or more instructions 1424 .

[0209] The term "machine-readable medium" may include any medium capable of storing, encoding, or carrying instructions for execution by machine 1400 that causes machine 1400 to perform any one or more of the techniques of this disclosure, or any medium capable of storing, encoding, or carrying data structures used by or associated with such instructions. Non-limiting examples of machine-readable media may include solid-state memory, optical media, magnetic media, and signals (e.g., radio frequency signals, other photon-based signals, acoustic signals, etc.). In an example, a non-transitory machine-readable medium comprises a machine-readable medium having a set of particles having a constant (e.g., stationary) mass and, therefore, being a constituent of matter. Thus, a non-transitory machine-readable medium is a machine-readable medium that does not include a transitory propagating signal. Specific examples of non-transitory machine-readable media may include: non-volatile memory, such as semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)) and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0210] In an example, information stored or otherwise provided on machine-readable medium 1422 may represent instructions 1424, such as the instructions 1424 themselves or a format from which instructions 1424 may be derived. Such formats from which instructions 1424 may be derived may include source code, encoded instructions (e.g., in compressed or encrypted form), packaged instructions (e.g., split into multiple packages), etc. The information representing instructions 1424 in machine-readable medium 1422 may be processed by processing circuitry into instructions to implement any of the operations discussed herein. For example, deriving instructions 1424 from the information (e.g., processing by processing circuitry) may include compiling (e.g., from source code, object code, etc.), interpreting, loading, organizing (e.g., dynamically or statically linking), encoding, decoding, encrypting, decrypting, packing, unpacking, or otherwise manipulating the information into instructions 1424.

[0211] In an example, the derivation of instructions 1424 may include the assembly, compilation, or interpretation of information (e.g., by processing circuitry) to create instructions 1424 from some intermediate or pre-processed format provided by machine-readable media 1422. When provided in multiple parts, the information may be combined, unpacked, and modified to create instructions 1424. For example, the information may be in multiple compressed source code packages (or object code, or binary executable code, etc.) on one or more remote servers. The source code packages may be encrypted when transmitted over the network and may be decrypted, decompressed, assembled (e.g., linked) as necessary, and compiled or interpreted at the local machine (e.g., into a library, a stand-alone executable, etc.) and executed by the local machine.

[0212] The instructions 1424 may further be sent or received over a communication network 1426 using a transmission medium via the network interface device 1420 using any of a variety of transmission protocols (e.g., frame relay, Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), etc.). Exemplary communication networks may include a local area network (LAN), a wide area network (WAN), a packet data network (e.g., the Internet), a mobile telephone network (e.g., a cellular network), a plain old telephone (POTS) network, and a wireless data network (e.g., the Institute of Electrical and Electronics Engineers (IEEE) 802.11 family of standards (known as ), IEEE 802.16 family of standards (known as )), IEEE 802.15.4 family of standards, peer-to-peer (P2P) networks, and the like. In an example, network interface device 1420 may include one or more physical jacks (e.g., Ethernet, coaxial, or telephone jacks) or one or more antennas to connect to network 1426. In an example, network interface device 1420 may include a set of multiple antennas to enable wireless communication using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) technology. The term "transmission media" shall be deemed to include any intangible medium capable of storing, encoding, or carrying instructions for execution by machine 1400, and includes digital or analog communication signals or other intangible media used to facilitate communication of such software. Transmission media is machine-readable media.

[0213] To better illustrate the methods and apparatus described herein, a set of non-limiting exemplary embodiments are set forth below as numbered examples.

[0214] The above detailed description includes reference to the accompanying drawings, which form a part of the detailed description. The accompanying drawings show, by way of illustration, specific embodiments of the present invention that can be put into practice. These embodiments are also referred to as "examples" in this article. Such examples may include elements other than those shown or described. However, the present inventors also contemplate examples in which only those elements shown or described are provided. In addition, the present inventors also contemplate examples (or one or more aspects thereof) of any combination or arrangement of those elements shown or described with respect to a specific example (or one or more aspects thereof) or with respect to other examples (or one or more aspects thereof) shown or described herein.

[0215] In this document, as is common in patent documents, the terms "a" and "an" are used to include one or more than one, regardless of any other examples or uses of "at least one" or "one or more." In this document, unless otherwise indicated, the term "or" is used to refer to a non-exclusive or, such that "A or B" may include "A but not B," "B but not A," and "A and B." In the appended claims, the terms "including" and "in which" are used as the plain English equivalents of the respective terms "comprising" and "wherein." Furthermore, in the appended claims, the terms "including" and "comprising" are open-ended, that is, systems, devices, articles, or processes that include these elements in addition to the elements listed after such terms in the claim are still considered to fall within the scope of the claim. Furthermore, in the appended claims, the terms "first," "second," and "third," etc., are used merely as labels and are not intended to impose numerical requirements on their objects.

[0216] The above description is intended to be illustrative, not restrictive. For example, the examples described above (or one or more aspects thereof) can be used in combination with each other. Other embodiments can be used by those skilled in the art after consulting the above description. The abstract is submitted with the following understanding: it will not be used to interpret or limit the scope or meaning of the claims. Similarly, in the above specific embodiments, various features can be grouped together to simplify the present disclosure. This situation should not be interpreted as expecting that the disclosed features that are not claimed for protection are necessary for any claim. In fact, the subject matter of the present invention may be less than all the features of the specific disclosed embodiments. Therefore, the attached claims are incorporated into the specific embodiments accordingly, wherein each claim exists independently as a separate embodiment, and it is expected that such embodiments can be combined with each other in various combinations or arrangements. The scope of the present invention should be determined by reference to the complete scope of the equivalents granted by the attached claims and such claims.

Claims

1. A method for using a reconfigurable computing fabric, the method comprising: launching a thread at a first synchronous flow controlled by a first flow controller to perform a first portion of an iteration of a loop, and subsequently launching at least one additional thread within the first synchronous flow for subsequent iterations of the loop; receiving, at a second flow controller of a second synchronous flow, an asynchronous message to start a thread at the second synchronous flow for executing a second portion of the iteration; determining, by the second flow controller, that the iteration is a last iteration of the loop to be executed; starting the thread at the second synchronization flow with a synchronization message including a final iteration flag; as well as Executing the thread within the second synchronization flow includes checking the last iteration flag in the synchronization message and performing a conditional operation based on the presence of the last iteration flag, the conditional operation based at least in part on output produced by the iteration of the loop and at least in part on output produced by a second iteration of the loop. 2 . The method of claim 1 , launching the thread at the second synchronization flow in response to determining that the iteration of the loop is the last iteration of the loop to be executed.

3. The method according to claim 1, further comprising starting a second thread at the second synchronization flow, executing the second thread at the second synchronization flow comprising: providing a second synchronization message to the computing elements of the second synchronization flow; Determining that the second synchronization message does not include the last iteration flag; as well as The conditional operation is omitted in response to determining that the second synchronization message does not include the last iteration flag.

4. The method according to claim 1, further comprising: receiving, by the second flow controller, an iteration count indicating a number of iterations of the loop; as well as The number of threads previously launched by the second flow controller is compared to the iteration count. The method of claim 4 , wherein the asynchronous message includes the iteration count.

6. The method according to claim 4, further comprising: Sending, by a scheduling interface of the reconfigurable computing fabric, a second asynchronous message to the first flow controller, the second asynchronous message instructing the first flow controller to start execution of the loop; as well as A third asynchronous message is sent by the scheduling interface to the second flow controller, where the third asynchronous message includes the iteration count.

7. The method of claim 4, further comprising programming the reconfigurable computing structure to include the first synchronization flow and the second synchronization flow, the programming comprising providing the iteration count to the second flow controller.

8. The method of claim 4, further comprising sending the asynchronous message to the second flow controller by at least a portion of the first synchronous flow.

9. The method according to claim 1, further comprising: After starting the thread at the first synchronization flow, starting a second thread at the first synchronization flow by the first flow controller, the second thread at the first synchronization flow executing the first part of the second iteration of the loop; before receiving the asynchronous message, receiving a second asynchronous message to start a second thread at the second synchronous flow to execute a second portion of a second iteration of the loop; determining, by the second flow controller, that the second iteration of the loop is not the last iteration of the loop to be executed; as well as The second thread at the second synchronization flow is started by the second flow controller, the starting comprising providing a second synchronization message to the computing elements of the second synchronization flow, the second synchronization message not including the last iteration flag.

10. A reconfigurable computing structure comprising: A first synchronization flow comprising a first flow controller and at least one computational element, the first flow controller being programmed to perform operations comprising: Starting a thread within the first synchronous flow to execute a first portion of an iteration of the loop; as well as subsequently launching at least one additional thread within the first synchronous stream for subsequent iterations of the loop; as well as A second synchronization flow comprising a second flow controller and at least one computational element, the second flow controller being programmed to perform operations comprising: receiving an asynchronous message to start a thread at the second synchronous flow for executing a second part of the iteration; determining that the iteration is a last iteration of the loop to be executed; starting the thread at the second synchronization flow with a synchronization message including a final iteration flag; as well as Executing the thread within the second synchronization flow includes checking the last iteration flag in the synchronization message and performing a conditional operation based on the presence of the last iteration flag, the conditional operation based at least in part on output produced by the iteration of the loop and at least in part on output produced by a second iteration of the loop.

11. The reconfigurable computing structure of claim 10, wherein the thread at the second synchronization flow is started in response to determining that the iteration of the loop is the last iteration of the loop to be executed.

12. The reconfigurable computing structure of claim 10 , wherein the second flow controller is programmed to perform operations further comprising starting a second thread at the second synchronization flow, wherein executing the second thread at the second synchronization flow comprises: providing a second synchronization message to the computing elements of the second synchronization flow; Determining that the second synchronization message does not include the last iteration flag; as well as The conditional operation is omitted in response to determining that the second synchronization message does not include the last iteration flag.

13. The reconfigurable computing structure of claim 10 , wherein the second flow controller is programmed to perform operations further comprising: receiving an iteration count indicating a number of iterations of the loop; and The number of threads previously launched by the second flow controller is compared to the iteration count.

14. The reconfigurable computing structure of claim 13, wherein the asynchronous message includes the iteration count.

15. The reconfigurable computing structure of claim 13, comprising a dispatch interface programmed to perform operations comprising: sending a second asynchronous message to the first flow controller, the second asynchronous message instructing the first flow controller to start execution of the loop; and A third asynchronous message is sent to the second flow controller, the third asynchronous message including the iteration count.

16. The reconfigurable computing structure of claim 13, the second flow controller being programmed to perform operations further comprising sending the asynchronous message to the second flow controller.

17. The reconfigurable computing structure of claim 10 , wherein the first flow controller is programmed to perform operations further comprising: after starting the thread at the first synchronization flow, starting, by the first flow controller, a second thread at the first synchronization flow, the second thread at the first synchronization flow executing a first portion of a second iteration of the loop; before receiving the asynchronous message, receiving a second asynchronous message to start a second thread at the second synchronous flow to execute a second portion of a second iteration of the loop; determining, by the second flow controller, that the second iteration of the loop is not the last iteration of the loop to be executed; as well as The second thread at the second synchronization flow is started by the second flow controller, the starting comprising providing a second synchronization message to the computing elements of the second synchronization flow, the second synchronization message not including the last iteration flag.

18. A non-transitory machine-readable medium comprising instructions stored thereon that, when executed at a reconfigurable computing structure, cause the reconfigurable computing structure to perform operations comprising: launching a thread at a first synchronous flow controlled by a first flow controller to perform a first portion of an iteration of a loop, and subsequently launching at least one additional thread within the first synchronous flow for subsequent iterations of the loop; receiving, at a second flow controller of a second synchronous flow, an asynchronous message to start a thread at the second synchronous flow for executing a second portion of the iteration; determining, by the second flow controller, that the iteration is a last iteration of the loop to be executed; starting the thread at the second synchronization flow with a synchronization message including a final iteration flag; as well as Executing the thread within the second synchronization flow includes checking the last iteration flag in the synchronization message and performing a conditional operation based on the presence of the last iteration flag, the conditional operation based at least in part on output produced by the iteration of the loop and at least in part on output produced by a second iteration of the loop.

19. The non-transitory machine-readable medium of claim 18, launching the thread at the second synchronization flow in response to determining that the iteration of the loop is the last iteration of the loop to be executed.

20. The non-transitory machine-readable medium of claim 18, the operations further comprising launching a second thread at the second synchronization flow, executing the second thread at the second synchronization flow comprising: providing a second synchronization message to the computing elements of the second synchronization flow; Determining that the second synchronization message does not include the last iteration flag; as well as The conditional operation is omitted in response to determining that the second synchronization message does not include the last iteration flag.

Images