System and apparatus for preventing thread completion including non-blocking external device calls by non-return indication

Abstract

Devices and techniques for unblocked external device calls are described herein. Specifically, when a processor receives an instruction with a no return indication from a thread for a device, the processor can increment a counter corresponding to the thread based on the no return indication. The processor can then continue execution of the thread without waiting for a return value from the device. When a return value for the instruction is received, the processor can decrement the counter. While the counter is not zero, the processor prevents the thread from completing (exiting).

Description

Technical Field

[0001] This application relates to unobstructed access to external devices. Background Technology

[0002] For example, various computer architectures based on the von Neumann architecture conventionally use shared memory for data, buses for accessing shared memory, arithmetic units, and program control units. However, moving data between the processor and memory can be time-consuming and energy-intensive, which can in turn constrain the performance and capacity of computer systems. Given these limitations, new computing architectures and devices are needed to drive the practice of computing performance beyond the transistor scale (i.e., Moore's Law). Summary of the Invention

[0003] In one aspect, this application provides a system comprising: a first memory computing node including: means configured to accept remote calls; and a second memory computing node including: a hybrid thread processor (HTP) including processing circuitry configured to: receive instructions from a thread for the means, the instructions corresponding to a no-return indication; increment a counter corresponding to the thread based on the no-return indication; execute the thread regardless of a return value from the means; decrement the counter based on receiving the return value; and prevent the thread from completing until the counter is zero.

[0004] In another aspect, this application provides an apparatus comprising: an interface to the device; and a processing circuitry configured to: receive an instruction from a thread for the device, the instruction corresponding to a no-return indication; in response to receiving the instruction, make a call to the device across the interface; increment a counter corresponding to the thread based on the no-return indication; continue execution of the thread without waiting for a return value from the device; decrement the counter based on receiving the return value; and prevent the thread from completing until the counter reaches zero. Attached Figure Description

[0005] To facilitate identification of any particular element or action, one or more of the most significant digits in the reference numerals refer to the drawing number in which the element is first introduced.

[0006] Figure 1 This section provides a general description of a first instance of a first memory computing device in the context of a memory computing system according to an embodiment.

[0007] Figure 2 Examples of memory subsystems of memory computing devices according to embodiments are described in general.

[0008] Figure 3 This section provides a general description of examples of programmable atomic units for a memory controller according to embodiments.

[0009] Figure 4 An example of a hybrid thread processor (HTP) accelerator for a memory computing device according to an embodiment is described.

[0010] Figure 5 This describes an example of thread execution in a processor under unobstructed calls to external devices, according to an embodiment.

[0011] Figure 6 This section describes an example comparing the operation sequence for blocking calls to external devices with the operation sequence for unblocking calls to external devices, according to an embodiment.

[0012] Figure 7 An example illustrating a representation of a hybrid thread mesh (HTF) architecture of a memory computing device according to an embodiment is provided.

[0013] Figure 8A This section provides a general description of an example of a chiplet system based on an embodiment.

[0014] Figure 8B General description shows from Figure 8A A block diagram of the components in an example chiplet system.

[0015] Figure 9 This section provides a general description of examples of chiplet-based implementation schemes for memory computing devices according to embodiments.

[0016] Figure 10 This describes an example of tiling a memory computing device chip according to an embodiment.

[0017] Figure 11 This is a flowchart of an example of a method for unobstructed external device invocation according to an embodiment.

[0018] Figure 12 This is a block diagram of an example machine, in which one or more of the techniques (e.g., methods) discussed herein may be implemented using, in, or by the machine. Detailed Implementation

[0019] Recent advances in materials, devices, and integration technologies can be leveraged to provide memory-centric computing topologies. Such topologies enable advancements in computational efficiency and workload handling for applications constrained by size, weight, or power requirements. Topologies can facilitate low-latency computing in or near memory or other data storage elements. The approach is particularly well-suited for various computationally intensive operations utilizing sparse lookups, such as in transform computations (e.g., Fast Fourier Transform (FFT) computations), or in applications such as neural networks or artificial intelligence (AI), financial analysis, or simulation or modeling, for example, computational fluid dynamics (CFD), engineer-as-a-system (EASE) augmented acoustic simulators, integrated circuit-centric simulation programs (SPICE), etc.

[0020] The systems, apparatuses, and methods discussed herein may include or utilize memory computing systems with processors or processing capabilities provided in, near, or integrated with memory or data storage components. Such systems are generally referred to herein as compute-near-memory (CNM) systems. CNM systems may be node-based systems in which individual nodes are coupled using a system-scale mesh architecture. Especially in environments where high cache miss rates are anticipated, each node may include or utilize a dedicated or general-purpose processor and a user-accessible accelerator (with a custom compute mesh architecture to facilitate intensive operations).

[0021] In this example, each node in a CNM system can have one or more host processors. Within each node, a dedicated hybrid-threaded processor can occupy discrete endpoints of the on-chip network. The hybrid-threaded processor can access some or all of the memory in a specific node of the system, or it can access the memory of a network spanning multiple nodes via a system-scaled mesh architecture. The custom compute mesh or hybrid-threaded mesh architecture at each node can have its own processor or accelerator and can operate at higher bandwidths than the hybrid-threaded processor. Different nodes in a near-memory compute system can be configured differently, for example, with different compute capabilities, different types of memory, different interfaces, or other differences. However, nodes can be coupled together to share data and compute resources within a defined address space.

[0022] In this example, near-memory computing systems or nodes within a system can be user-configurable for custom operations. Users can provide instructions using a high-level programming language (e.g., C / C++), which can be compiled and directly mapped to the dataflow architecture of one or more nodes in the system or CNM system. That is, nodes in the system can contain hardware blocks (e.g., memory controllers, atomic cells, other client accelerators, etc.) that can be configured to directly implement or support user instructions to thereby enhance system performance and reduce latency.

[0023] In practice, near-memory computing systems are particularly well-suited for implementing hierarchical structures of instructions and nested loops (e.g., two, three, or more loop depths, or multidimensional loops). Standard compilers can be used to accept high-level language instructions and compile them directly into one or more dataflow architectures within a node. For example, nodes in the system can contain hybrid threaded mesh accelerators. These accelerators can execute in the user space of the CNM system and can initiate their own threads or sub-threads that can operate in parallel. Each thread can be mapped to a different loop iteration, thereby supporting multidimensional loops. Leveraging the ability to initiate such nested loops, along with other capabilities, CNM systems can achieve significant time savings and latency improvements for computationally intensive operations.

[0024] Near-memory computing systems, or nodes or components of near-memory computing systems, may include or utilize various memory devices, controllers, and interconnects, etc. In examples, the system may include various interconnect nodes, and nodes or groups of nodes may be implemented using chiplets. Chiplets are an emerging technology for integrating various processing functionalities. Typically, chiplet systems consist of discrete chips (e.g., integrated circuits (ICs) on different substrates or dies) integrated on an interposer and packaged together. This arrangement differs from a single chip (e.g., an IC) containing different blocks of devices (e.g., blocks of intellectual property (IP)) on a substrate (e.g., a single die), such as a system-on-a-chip (SoC), or discrete packaged devices integrated on a board. Generally, chiplets offer manufacturing benefits compared to single die chips, including higher yields or reduced development costs. The following discussion... Figure 8A and Figure 8B Examples typically described include chiplet systems, which may include near-memory computing systems.

[0025] Various components that can constitute the near-memory computing system described herein are processors configured to execute threads that interact with other components of the near-memory computing system. For example, thread execution on an HTP can perform atomic operations on a memory controller (on the same node or a different node) to modify values ​​in memory managed by the memory controller. Typically, this type of interaction involves a thread making a call to a device (e.g., the memory controller) and blocking that call while waiting for a response from the device. Typically, once the response is received, it is written to the processor's target register and the thread resumes execution. Therefore, the thread does not perform any useful work while the device is processing the call.

[0026] The traditional practice of pausing thread activity during an external device call is often necessary, for example, when a thread's subsequent instructions depend on the data returned by the call. Therefore, if a value needs to be retrieved from memory for subsequent computations by the thread, the thread must wait for the value to be returned before executing subsequent instructions to operate correctly. However, there are situations where subsequent thread operations do not depend on the result of a call to a device. For example, if maintaining a flowing sum of the number of sensor samples being processed, each thread processing the sensor samples can call an atomic operation at the memory controller to increment the flowing sum by one. Here, once the call is sent, the thread's subsequent operations are independent of the actual value of the flowing count. In situations like these, blocking the thread immediately after the call would only slow down the thread's execution without any benefit. To address the problems in these situations, the thread can indicate whether to use the return value from the external device call. If the thread indicates that it will use the return value, then the traditional practice of the processor aborting, pausing, or blocking the thread can be used while waiting for the return. However, if a thread indicates that it will not use a return value (e.g., a request, call, or instruction with no return, no blocking, etc.), then the processor can enable the thread to continue execution after a remote device call (e.g., a request) (e.g., immediately rescheduling the thread) without waiting for a return value. In other words, the thread can execute without considering the return value. In systems with many running threads, such as the memory-based computing systems described herein, enabling threads to continue execution after an unblocking call can significantly reduce thread idle time and increase processing throughput on the hardware.

[0027] Even when the calling thread doesn't need a return value, ensuring that calls to external devices between threads complete their operations can be important. That is, allowing a thread to exit while an unblocked call is still running, thus making it seem as if it has done all its work, can lead to incorrect software behavior.

[0028] To address this issue, the processor can maintain a pending no-return counter, indicating how many no-return memory accesses are awaiting a response. Thus, a thread can execute its next instruction immediately after making any no-return call, but is prevented from exiting the thread before the processor receives a response to a no-return access (e.g., by executing a thread return instruction (ETR)). This avoids the traditional dependency on instructions responding to external devices (e.g., memory accesses), which can cause significant performance degradation, and instead maintains data consistency with actors outside the thread.

[0029] In the additional details and examples described below, HTP (e.g., Figure 4 The HTP accelerator 400 described herein is an instance processor implementing a pending no-return counter. However, any processing circuitry that affects thread execution and thread calls to external devices (e.g., regarding...) Figure 1-3 The processing circuit systems described in 7-8B or 12 can all implement the apparatus and techniques for the non-return call or pending non-return counter described herein.

[0030] Figure 1 This typically describes a first instance of a near-memory computing system or CNM system 102. An instance of CNM system 102 comprises multiple different memory computing nodes, each of which may contain various near-memory computing devices. Each node in the system can operate within its own operating system (OS) domain (e.g., in particular Linux). In the instance, the nodes may coexist in a common OS domain of CNM system 102.

[0031] Figure 1 Examples include instances of the first memory compute node 104 of the CNM system 102. The CNM system 102 may have multiple nodes, for example, including different examples of the first memory compute node 104 coupled using a scaling mesh architecture 106. In the examples, the architecture of the CNM system 102 may support a scale with up to n different memory compute nodes (e.g., n = 4096) using the scaling mesh architecture 106. As discussed further below, each node in the CNM system 102 may be a combination of multiple devices.

[0032] CNM system 102 may include a global controller for various nodes in the system, or a specific memory compute node in the system may optionally act as a host or controller for one or more other memory compute nodes in the same system. The various nodes in CNM system 102 may therefore be configured similarly or differently.

[0033] In this example, each node in CNM system 102 may include a host system using a specified operating system. The operating system may be common or different across the various nodes in CNM system 102. Figure 1 In this example, the first memory computing node 104 includes a host system 108, a first switch 110, and a first memory computing device 112. The host system 108 may include a processor, such as an x86, ARM, RISC-V, or other type of processor. The first switch 110 may be configured to facilitate communication between or between devices of the first memory computing node 104 or devices of the CNM system 102, for example, using a dedicated or other communication protocol (generally referred to herein as the Chip-to-Chip Protocol Interface (CTCPI)). That is, CTCPI may include a dedicated interface unique to the CNM system 102, or may include or use other interfaces, such as a High-Speed ​​Compute Link (CXL) interface, a High-Speed ​​Peripheral Component Interconnect (PCIe) interface, or a Chiplet Protocol Interface (CPI), etc. The first switch 110 may include a switch configured to use CTCPI. For example, the first switch 110 may include a CXL switch, a PCIe switch, a CPI switch, or other types of switches. In this example, the first switch 110 may be configured to couple dissimilarly configured endpoints. For example, the first switch 110 can be configured to convert packet formats between, for example, PCIe and CPI formats.

[0034] The CNM system 102 is described herein with various example configurations (e.g., systems including nodes), and each node may include various chips (e.g., processors, switches, memory devices, etc.). In an example, the first memory computing node 104 in the CNM system 102 may include various chips implemented using chiplets. In the chiplet-based configuration of the CNM system 102 discussed below, inter-chiplet communication and additional intra-system communication may use a CPI network. The CPI network described herein is an example of CTCPI, that is, a chiplet-specific implementation of CTCPI. Therefore, the structure, operation, and functionality described below for CPI are equally applicable to structures, operations, and functions that may be implemented using non-chiplet-based CTCPI implementations. Unless otherwise expressly indicated, any discussion of CPI herein applies equally to CTCPI.

[0035] The CPI interface includes a packet network that supports virtual channels to enable flexible and high-speed interaction between chiplets, such as portions of the first memory compute node 104 or the CNM system 102. CPI can bridge intra-chiplet networks to wider chiplet networks. For example, the Advanced Extensible Interface (AXI) is a specification for intra-chip communication. However, the AXI specification covers a large number of physical design options, such as the number of physical channels, signal timing, power, etc. Within a single chip, these options are typically selected to meet design goals, such as power consumption, speed, etc. However, to achieve the flexibility of chiplet-based memory compute systems, adapters using CPI can intersect between various AXI design options that can be implemented in various chiplets. By enabling physical-to-virtual channel mapping using packetization protocols and encapsulating time-based signaling, CPI can be used to bridge intra-chiplet networks (e.g., within a specific memory compute node) across wider chiplet networks (e.g., across the first memory compute node 104 or across the CNM system 102).

[0036] The CNM system 102 is scalable to include multi-node configurations. That is, multiple different examples of the first memory compute node 104 or other differently configured memory compute nodes can be coupled using a scalable mesh architecture 106 to provide a scalable system. Each of the memory compute nodes can run its own operating system and can be configured to jointly coordinate the use of wide system resources.

[0037] exist Figure 1 In one example, a first switch 110 of the first memory compute node 104 is coupled to a scaling mesh architecture 106. The scaling mesh architecture 106 provides switches (e.g., CTCPI switches, PCIe switches, CPI switches, or other switches) that facilitate communication among and between different memory compute nodes. In this example, the scaling mesh architecture 106 can facilitate communication between various nodes in a partitioned global address space (PGAS).

[0038] In this example, a first switch 110 from a first memory computing node 104 is coupled to one or more different memory computing devices, such as a first memory computing device 112. The first memory computing device 112 may include a chiplet-based architecture, referred to herein as a near-memory computing (CNM) chiplet. A packaged version of the first memory computing device 112 may include, for example, one or more CNM chiplets. The chiplets may be communicatively coupled using CTCPI for high bandwidth and low latency.

[0039] exist Figure 1 In an example, the first memory computing device 112 may include a network on-chip (NOC) or a first NOC 118. Typically, an NOC is an interconnect network within a device that connects a specific set of endpoints. Figure 1 In this context, the first NOC 118 can provide communication and connectivity between various memories, computing resources and ports of the first memory computing device 112.

[0040] In an example, the first NOC 118 may include a folded Clos topology, such as within each instance of a memory computing device, or as a grid coupling multiple memory computing devices in a node. The Clos topology offers various benefits, such as the ability to use multiple lower-radix cross switches to provide functionality associated with higher-radix cross switch topologies. For example, the Clos topology can present consistent latency and bi-directional bandwidth across the NOC.

[0041] The first NOC 118 can include various types of switches, including hub switches, edge switches, and endpoint switches. Each switch can be configured as a crossbar switch that provides substantially uniform latency and bandwidth between input and output nodes. In an example, endpoint switches and edge switches can contain two separate crossbar switches, one for traffic destined for the hub switch and the other for traffic destined for the hub switch. A hub switch can be configured as a single crossbar switch that switches all inputs to all outputs.

[0042] In this example, hub switches may each have multiple ports (e.g., four or six ports each) depending on whether a particular hub switch participates in inter-chip communication. The number of hub switches participating in inter-chip communication can be set by inter-chip bandwidth requirements.

[0043] The first NOC 118 can support a variety of payloads between computing elements and memory (e.g., 8 to 64-byte payloads; other payload sizes can be used similarly). In an example, the first NOC 118 can be optimized for relatively small payloads (e.g., 8 to 16 bytes) to efficiently handle access to sparse data structures.

[0044] In this example, the first NOC 118 may be coupled to an external host via a first physical layer interface 114, a PCIe slave module 116 or endpoint, and a PCIe master module 126 or root port. That is, the first physical layer interface 114 may include an interface to allow an external host processor to couple to the first memory computing device 112. The external host processor may optionally couple to one or more different memory computing devices, for example, using a PCIe switch or other native protocol switch. Communication with the external host processor via a PCIe-based switch limits device-to-device communication to device-to-device communication supported by the switch. In contrast, communication via a memory computing device native protocol switch, such as using CTCPI, allows for more complete communication between or among different memory computing devices, including support for partitioned global address spaces, such as for generating worker threads and sending events.

[0045] In this example, the CTCPI protocol may be used by a first NOC 118 in a first memory computing device 112, and a first switch 110 may include a CTCPI switch. The CTCPI switch may allow CTCPI packets to be transmitted, for example, from a source memory computing device of the first memory computing device 112 to (e.g., on the same or another node) different destination memory computing devices without being converted to another packet format.

[0046] In one example, the first memory computing device 112 may include an internal host processor 122. The internal host processor 122 may be configured to communicate with the first NOC 118 or other components or modules of the first memory computing device 112, for example, using an internal PCIe master module 126, which can help eliminate the time- and energy-consuming physical layer. In one example, the internal host processor 122 may be based on a RISC-VISA processor and may use a first physical layer interface 114 to communicate externally to the first memory computing device 112, for communication with other storage, networking, or other peripheral devices of the first memory computing device 112. The internal host processor 122 may control the first memory computing device 112 and may act as a proxy for operating system-related functionalities. The internal host processor 122 may include a relatively small number of purpose processing cores (e.g., 2 to 4 cores) and a host memory device 124 (e.g., including DRAM modules).

[0047] In one example, the internal host processor 122 may include a PCI root port. When the internal host processor 122 is in use, one of its root ports may be connected to a PCIe slave module 116. Another root port of the internal host processor 122 may be connected to a first physical layer interface 114 to provide communication with external PCI peripherals. When the internal host processor 122 is deactivated, the PCIe slave module 116 may be coupled to the first physical layer interface 114 to allow an external host processor to communicate with a first NOC 118. In an example of a system with multiple memory computing devices, the first memory computing device 112 may be configured to act as a system host or controller. In this example, the internal host processor 122 may be in use, and other examples of internal host processors in corresponding other memory computing devices may be deactivated.

[0048] The internal host processor 122 can be configured at power-up of the first memory computing device 112 to allow host initialization. In an example, the internal host processor 122 and its associated data paths (e.g., including the first physical layer interface 114, PCIe slave module 116, etc.) can be configured to the first memory computing device 112 from input pins. One or more of these pins can be used to enable or disable the internal host processor 122 and configure the PCI (or other) data paths accordingly.

[0049] In this example, the first NOC 118 may be coupled to the scaling mesh architecture 106 via a scaling mesh interface module 136 and a second physical layer interface 138. The scaling mesh interface module 136, or SIF, facilitates communication between the first memory computing device 112 and a device space such as a Partitioned Global Address Space (PGAS). The PGA may be configured such that a particular memory computing device, such as the first memory computing device 112, can access memory or other resources on different memory computing devices (e.g., on the same or different nodes) using a load / store paradigm. Various scalable mesh technologies may be used, including CTCPI, CPI, Gen-Z, PCI, or Ethernet bridged via CXL. The scaling mesh architecture 106 may be configured to support various packet formats. In this example, the scaling mesh architecture 106 supports out-of-order packet communication or supports ordered packets, for example, by using path identifiers to extend bandwidth across multiple equivalent paths. The scaled mesh architecture 106 typically supports remote operations, such as remote memory reads, writes and other built-in atoms, remote memory atoms, remote memory computing device send events, and remote memory computing device call and return operations.

[0050] In this example, the first NOC 118 may be coupled to one or more different memory modules, such as including a first memory device 128. The first memory device 128 may include various types of memory devices, such as LPDDR5 or GDDR6, etc. Figure 1 In one example, the first NOC 118 may coordinate communication with the first memory device 128 via a memory controller 130, which may be dedicated to a specific memory module. In this example, the memory controller 130 may include a memory module cache and an atomic operation module. The atomic operation module may be configured to provide relatively high throughput atomic operators, such as integer and floating-point operators. The atomic operation module may be configured to apply its operators to data within a memory module cache (e.g., including an SRAM memory-side cache), thereby allowing back-to-back atomic operations using the same memory location with minimal throughput degradation.

[0051] The memory module cache can provide storage for frequently accessed memory locations, such as eliminating the need to re-access the first memory device 128. In one example, the memory module cache can be configured to cache data only for a specific instance of the memory controller 130. In another example, the memory controller 130 includes a DRAM controller configured to interface with, for example, the first memory device 128 containing a DRAM device. The memory controller 130 can provide access scheduling and bit error management, among other functions.

[0052] In this example, the first NOC 118 may be coupled to a hybrid thread processor (HTP 140), a hybrid thread mesh architecture (HTF 142), and a host interface and dispatch module (HIF 120). HIF 120 may be configured to facilitate access to host-based command request and response queues. In this example, HIF 120 may dispatch new execution threads on the processors or compute elements of the HTP 140 or HTF 142. In this example, HIF 120 may be configured to maintain workload balancing across the HTP 140 and HTF 142 modules.

[0053] The hybrid-threaded processor, or HTP 140, may include accelerators, such as those based on the RISC-V instruction set. The HTP 140 may contain a highly threaded event-driven processor, where threads can execute in a single instruction round to maintain high instruction throughput. The HTP 140 includes relatively few custom instructions to support low-overhead threading capabilities, event send / receive, and shared-memory atomic operators.

[0054] The hybrid threaded mesh architecture, or HTF 142, may include accelerators, such as non-von Neumann coarse-grained configurable processors. The HTF 142 may be optimized for high-level language operations and data types (e.g., integers or floating-point). In some instances, the HTF 142 may support dataflow computation. The HTF 142 may be configured to utilize substantially all memory bandwidth available on the first memory computing device 112, for example, when executing a memory-bound computational core.

[0055] The HTP and HTF accelerators of the CNM system 102 can be programmed using various high-level structured programming languages. For example, the HTP and HTF accelerators can be programmed using C / C++ (e.g., using the LLVM compiler framework). The HTP accelerator can, for example, utilize the open-source compiler environment through various additional custom instruction sets configured to improve memory access efficiency, provide message passing mechanisms, and manage events, etc. In an example, the HTF accelerator can be designed to be programmed using a high-level programming language, and the compiler can generate simulator configuration files or binaries that run on the HTF 142 hardware. The HTF 142 provides a mid-level language for precise and concise expression of algorithms while hiding the configuration details of the HTF accelerator itself. In an example, the HTF accelerator toolchain can use an LLVM front-end compiler and an LLVM intermediate representation (IR) to interface with the HTF accelerator back-end.

[0056] Figure 2 An example of a memory subsystem 200 of a memory computing device according to an embodiment is generally described. The example of memory subsystem 200 includes a controller 202, a programmable atom unit 208, and a second NOC 206. The controller 202 may include or use the programmable atom unit 208 to perform operations using information in the memory device 204. In the example, memory subsystem 200 includes components from… Figure 1 A portion of the first memory computing device 112, such as a portion of the first NOC 118 or memory controller 130.

[0057] exist Figure 2 In this example, the second NOC 206 is coupled to the controller 202, and the controller 202 may include a memory control module 210, a local cache module 212, and a built-in atom module 214. In this example, the built-in atom module 214 may be configured to handle relatively simple single-loop integer atoms. The built-in atom module 214 can perform atoms with the same processing power as, for example, a normal memory read or write operation. In this example, an atomic memory operation may include a combination of: storing data into memory, performing an atomic memory operation, and then responding by loading data from memory.

[0058] A local cache module 212, which may include, for example, an SRAM cache, can be provided to help reduce latency for repeatedly accessed memory locations. In an example, the local cache module 212 may provide a read buffer for sub-memory line access. The local cache module 212 may be particularly advantageous for computing elements with relatively small or no data cache.

[0059] For example, a memory control module 210, which may include a DRAM controller, can provide low-level request buffering and scheduling to provide efficient access to memory device 204 (e.g., which may include a DRAM device). In an example, memory device 204 may include or use a GDDR6 DRAM device, for example, with a density of 16 Gb and a peak bandwidth of 64 Gb / s. Other devices may be used similarly.

[0060] In this example, programmable atomic unit 208 may include single-loop or multi-loop operators, such as being configured to perform integer addition or more complex multi-instruction operations, such as Bloom filter insertion. In this example, programmable atomic unit 208 may be configured to perform load and store-to-memory operations. Programmable atomic unit 208 may be configured to utilize RISC-VISA via a dedicated instruction set to facilitate interaction with controller 202 to atomically execute user-defined operations.

[0061] Programmable atomic requests received, for example, from a host on or outside the node, can be routed to programmable atomic units 208 via a second NOC 206 and controller 202. In an example, custom atomic operations (e.g., performed by programmable atomic units 208) can be identical to built-in atomic operations (e.g., performed by built-in atomic modules 214), except that the programmable atomic operations can be defined or programmed by a user rather than a system architect. In an example, programmable atomic request packets can be sent to controller 202 via the second NOC 206, and controller 202 can recognize the requests as custom atoms. Controller 202 can then forward the recognized requests to programmable atomic units 208.

[0062] Figure 3 This section typically describes an example of a programmable atom unit 302 for use with a memory controller according to an embodiment. In this example, the programmable atom unit 302 may include or correspond to data from... Figure 2 The instance is a programmable atomic unit 208. That is to say, Figure 3 This describes the components in an example of the programmable atom unit (PAU) 302, such as those described above relative to... Figure 2 (e.g., in programmable atom unit 208) or relative to Figure 1 (For example, in the atomic operation module of memory controller 130) the mentioned components. Figure 3 As described, the programmable atomic unit 302 includes a PAU processor or PAU core 306, a PAU thread control 304, an instruction SRAM 308, a data cache 310, and a memory interface 312 for interfacing with a memory controller 314. In an example, the memory controller 314 includes components from... Figure 2 An instance of controller 202.

[0063] In this example, the PAU core 306 is a pipelined processor, allowing multiple stages of different instructions to execute together per clock cycle. The PAU core 306 may contain a barrel-shaped multithreaded processor, where the thread control 304 circuitry switches between different register files (e.g., a set of registers containing the current processing state) after each clock cycle. This enables efficient context switching between currently executing threads. In this example, the PAU core 306 supports eight threads, resulting in eight register files. In this example, some or all of the register files are not integrated into the PAU core 306, but instead actually reside in the local data cache 310 or instruction SRAM 308. This reduces the circuit complexity of the PAU core 306 by eliminating conventional flip-flops used for registers in such memories.

[0064] The local PAU memory may contain instruction SRAM 308, such as instructions for various atoms. The instructions include instruction sets to support atomic operators loaded by various applications. When an atomic operator is requested, for example, by an application chiplet, the instruction set corresponding to the atomic operator is executed by the PAU core 306. In this example, the instruction SRAM 308 may be partitioned to establish the instruction set. In this example, a specific programmable atomic operator requested by the request process can be identified by a partition number. The partition number can be established when a programmable atomic operator is registered (e.g., loaded onto) using programmable atomic unit 302. Other metadata for the programmable instructions may be stored in memory (e.g., in a partition table) within the local memory of programmable atomic unit 302.

[0065] In this example, atomic operators manipulate data cache 310, which is typically synchronized (e.g., flushed) when the thread used for the atomic operator completes. Therefore, latency can be reduced for most memory operations during the execution of the programmable atomic operator thread, except for initial loads from external memory such as memory controller 314.

[0066] If a potentially dangerous condition would prevent a memory request, the pipelined processor (e.g., PAU core 306) may experience problems when an execution thread attempts to make such a request. Here, the memory request is for retrieving data from memory controller 314, whether it comes from cache on memory controller 314 or off-die memory. To address this, PAU core 306 is configured to deny memory requests from threads. Typically, PAU core 306 or thread control 304 may contain circuitry for enabling one or more thread rescheduling points in the pipeline. Here, the denial occurs at points in the pipeline outside (e.g., after) these thread rescheduling points. In this instance, the danger occurs outside the rescheduling point. Here, the danger is created by a previous instruction in the thread after the memory request instruction has passed through the last thread rescheduling point before the pipeline stage where the memory request can be made.

[0067] In this example, to reject a memory request, the PAU core 306 is configured to determine (e.g., detect) the presence of a danger on the memory indicated in the memory request. Here, a danger represents any condition that would cause an inconsistent state of the thread if the memory request were allowed (e.g., executed). In this example, the danger is a memory request in progress. Here, the existence of a memory request in progress makes it uncertain what the data at that address in the data cache 310 should be, regardless of whether the data cache 310 contains data at the requested memory address. Therefore, the thread must wait for the memory request to complete before operating on the current data. The danger is cleared when the memory request completes.

[0068] In this instance, the danger lies in a dirty cache line in data cache 310 used for the requested memory address. While a dirty cache line typically indicates that the data in the cache is current and the memory controller version of such data is not current, problems can arise with thread instructions that do not operate from the cache. Examples of such instructions utilize the built-in atomic operators of memory controller 314 or other separate hardware blocks. In the context of the memory controller, the built-in atomic operators may be separate from programmable atomic units 302 and do not access the data cache 310 or instruction SRAM 308 within the PAU. If the cache line is dirty, the built-in atomic operators will not operate on the latest data until data cache 310 is flushed to synchronize the cache with another or off-chip memory. This same situation can occur using other hardware blocks of the memory controller, such as encryption blocks, encoders, etc.

[0069] Figure 4This describes an example of a hybrid-threaded processor (HTP) accelerator or HTP accelerator 400. HTP accelerator 400 may include a portion of a memory computing device according to an embodiment. In this example, HTP accelerator 400 may include or include components from... Figure 1 The HTP 140 is an example of this. The HTP accelerator 400 includes, for example, an HTP core 402, an instruction cache 404, a data cache 406, a translation block 408, a memory interface 410, and a thread controller 412. The HTP accelerator 400 may additionally include, for example, a dispatch interface 414 and an NOC interface 416 for interfacing with a NOC, the NOC being, for example, from... Figure 1 The first NOC 118 instance, from Figure 2 The second NOC 206 or other NOCs for the example.

[0070] In this example, the HTP accelerator 400 includes modules based on the RISC-V instruction set and may include a relatively small number of other or additional custom instructions for supporting low-overhead, thread-enabled hybrid threading (HT) languages. The HTP accelerator 400 may include a highly threaded processor core, HTP core 402, in which threads can execute in a single instruction round-robin to maintain high instruction throughput. In this example, threads can pause while waiting for other pending events to complete. This allows computational resources to be used efficiently for related work, rather than polling. In this example, multithreaded barrier synchronization can utilize efficient HTP-to-HTP and HTP-to / from host message passing, allowing thousands of threads to initialize or wake up in, for example, tens of clock cycles.

[0071] In an example, dispatch interface 414 may include function blocks for handling hardware-based thread management of HTP accelerator 400. That is, dispatch interface 414 can manage the dispatch of work to HTP core 402 or other accelerators. However, non-HTP accelerators are typically unable to dispatch work. In an example, work dispatched from the host may use dispatch queues residing in, for example, host main memory (e.g., DRAM-based memory). On the other hand, work dispatched from HTP accelerator 400 may use dispatch queues residing in SRAM, for example, within a dispatch for a target HTP accelerator 400 within a specific node.

[0072] In an example, HTP core 402 may include one or more cores that execute instructions on behalf of threads. That is, HTP core 402 may contain instruction processing blocks. HTP core 402 may additionally include or be coupled to thread controller 412. Thread controller 412 may provide thread control and state for each active thread within HTP core 402. Data cache 406 may include a cache for the host processor (e.g., for local and remote memory computing devices, including caches for HTP core 402), and instruction cache 404 may include a cache for use by HTP core 402. In an example, data cache 406 may be configured for both read and write operations, and instruction cache 404 may be configured for read-only operations.

[0073] In this example, data cache 406 provides a small cache for each hardware thread. Data cache 406 can temporarily store data for use by the owning thread. Data cache 406 can be managed by hardware or software within the HTP accelerator 400. For example, the hardware can be configured to automatically allocate or evict lines as needed when load and store operations are performed by the HTP core 402. Software, for example, using RISC-V instructions, can determine which memory access should be cached and when a line should be invalidated or written back to another memory location.

[0074] Cached data on the HTP accelerator 400 offers various benefits, including making memory controller access more efficient, thus allowing execution threads to avoid stalling. However, there are situations that can lead to inefficiency when using cache. Examples include accesses where data is accessed only once, resulting in cache line thrashing. To help address this issue, the HTP accelerator 400 can use a custom load instruction set to cause load instructions to check for cache hits and, if a cache miss occurs, issue a memory request for the requested operand without placing the retrieved data in the data cache 406. Therefore, the HTP accelerator 400 includes various types of load instructions, including non-cached and cached line loads. If dirty data exists in the cache, non-cached load instructions use the cached data. Non-cached load instructions ignore clean data in the cache and do not write the accessed data to the data cache. For cache line load instructions, a complete data cache line (e.g., including 64 bytes) can be loaded from memory into data cache 406, and the addressed memory can be loaded into a specified register. If clean or dirty data is in data cache 406, then these loads can use the cached data. If the referenced memory location is not in data cache 406, then the entire cache line can be accessed from memory. The use of cache line load instructions can reduce cache misses when referencing sequential memory locations (e.g., memory copy operations), but it can also waste memory and bandwidth at NOC interface 416 if the referenced memory data is not used.

[0075] In this example, the HTP accelerator 400 includes non-cached custom store instructions. These non-cached store instructions help avoid thrashing the data cache 406 by utilizing write data that is not sequentially written to memory.

[0076] In this example, the HTP accelerator 400 further includes a translation block 408. Translation block 408 may contain a virtual-to-physical translation block for the local memory of the memory computing device. For example, a host processor in HTP core 402 may execute load or store instructions, and these instructions may generate virtual addresses. The virtual addresses may be translated into physical addresses of the host processor, for example, using a translation table from translation block 408. For example, memory interface 410 may contain an interface between HTP core 402 and NOC interface 416.

[0077] The HTP accelerator 400 can handle multiple threads to complete multiple computational tasks within the memory computing device on the CNM node. Therefore, implementing unblocked external device calls on the HTP accelerator 400 can provide the increased thread processing throughput benefits described above. That is, when a thread indicates that a particular call is a no-return request, and the hardware supports such requests, the same hardware resources can execute more thread instructions in a given time unit. In the context of several memory computing devices that may be distributed across several nodes, this efficiency can provide a significant positive impact on overall system efficiency.

[0078] HTP accelerator 400 can be configured to implement unblocked external device calls. Unblocked external device calls enable thread execution on HTP accelerator 400 to specify memory calls as, for example, unblocked (e.g., no waiting). Therefore, HTP core 402 or thread controller 412 can be configured to reschedule threads after the request is made without waiting for a response to the request. Typically, an external device call is a processing request at an external device. Therefore, when a memory call is made, it is typically an atomic operation compared to a read (e.g., where a return is required) or a write (e.g., where data is typically not returned). External device calls may also include calls to another HTPO accelerator to start a thread, remote program requests to another node, etc.

[0079] Specifically, HTP core 402 is configured to receive instructions from a thread on a specified device. This device has a latency that would require the HTP accelerator to pause the thread to wait for the instruction to complete under normal circumstances. Typically, such requests are external to HTP accelerator 400, for example, to memory, another HTP accelerator, HTF, or other devices accessed via NOC interface 416. However, it is possible that some undescribed components of HTP accelerator 400 external to HTP core 402 may invoke such latency scenarios. In any case, the instruction or call to the thread typically involves putting the thread to sleep (e.g., not rescheduling another instruction to the thread) until a response to the instruction is received. However, if the instruction has a no-return indicator, then once the instruction is executed (e.g., once HTP core 402 stops sending requests to memory interface 410, NOC interface 416, etc.), HTP core 402 or thread controller 412 enables the thread to continue execution. In this example, the indicator is a state variable or other latch or trigger external to the instruction. In this example, the no-return indicator is included in the instruction. This example involves a specific instruction (e.g., EXTERN_REQ_NO_RETURN vs. EXTERN_REQ), field, flag, or other bits signaled as indicating that this is a no-return instruction. In this example, the no-return indicator is contained within a field of the instruction. In this example, the no-return indicator is a single bit. In this example, a single bit is one of several bits in the opcode field of the instruction.

[0080] As described above, the device to which the instruction points can be a memory controller, such as memory controller 200. In this instance, the instruction is an atomic operation. In this instance, the atomic operation is a programmable atomic operation. This type of instruction is the only instance involving latency exceeding that of common thread instructions and traditionally involves pausing the calling thread.

[0081] When the HTP accelerator 400, such as the HTP core 402, detects a no-return instruction, it increments a counter corresponding to the thread based on the no-return instruction. The counter can be part of the HTP core 402, the thread controller 412, the memory interface 410, or another component of the HTP accelerator. In the following example, the counter is part of the thread controller 412. As described above, the counter is thread-specific and increments by one to maintain a count of uncompleted no-return requests. While the following example starts the counter at zero and increments by one for each no-return instruction executed and decrements by one for each return received on the instruction, other numbering systems can be used. However, the counter is adjusted to account for unprocessed instructions that will operate in the same manner (e.g., those requests that have been generated but not yet completed).

[0082] Due to the presence of a no-return indication in the instruction, thread controller 412 (or in some cases, HTP core 402) is configured to continue thread execution without waiting for a return from the device. Typically, enabling a thread to continue execution involves placing the thread back into the scheduler of HTP core 402. Therefore, the thread does not block (e.g., pause, go to sleep, etc.) waiting for a requested result. In an example, continuing thread execution involves placing the thread's identifier into the ready-to-run queue of HTP core 402 or other thread instruction scheduling device. Regardless of the implementation, the thread can continue execution as if the instruction had been sent to the device and to the extractable logic unit (ALU) of HTP core 402.

[0083] HTP core 402 is configured to decrement (e.g., decrease) a counter based on a received return value for an instruction. The return value may be an indication of whether the instruction was successfully executed. The return value may also contain data. Thus, for example, the return value may be the result of an atomic operation, such as a read data atomic operation from memory and incremented by one. In an example, the return value may indicate whether an atomic operation was successfully completed without containing the result of the atomic operation. In either case, the return value signals to HTP accelerator 400 that the instruction has completed, and the counter is decremented.

[0084] In this example, a return value is received (e.g., by processing the circuitry) and then discarded. Discarding the return value reduces unnecessary operations on the HTP accelerator 400. For instance, in a typical (e.g., where there is no no-return indication) instruction sequence, the result is placed in a register for use by the thread. However, storing the return value involves unsuccessful processing because the thread will not use the value, since the thread has indicated that the return is not useful. Generally, the earlier a return value can be discarded in the receive pipeline, the better. Therefore, discarding the return value frees up communication resources (e.g., bandwidth on the bus or network) when undesirable data is not returned. In this case, only the data of the return value is omitted, not the result of the request (e.g., the exit code). This allows a counter to be decremented when the request is completed.

[0085] In this example, the counter is a unique counter for each no-return instruction issued by a thread to a device outside the processing circuitry system. Therefore, the HTP accelerator 400 maintains only one counter for each (current) thread. Thus, if a thread makes a memory request against a first memory controller and performs a programmable atomic operation against a second memory controller, the single counter increments twice. In this example, a separate counter can be maintained for each device. Therefore, in the above example, both counters will each increment once. Typically, the number of counters per thread is fixed. Therefore, in a per-device counter arrangement, if three devices are supported, each thread will have three counters. However, in large CNM systems, it may be impossible to know in advance how many devices a thread can request, or maintaining a counter for each of these devices may be impractical. To address this, a separate counter can also be used per device type (and thread). Typically, a single counter is sufficient to confirm whether a thread is in a state consistent with exit. Additional counters increase complexity, but this can provide greater flexibility in some cases, such as allowing a thread to exit under certain circumstances even if there are unprocessed (e.g., incomplete) requests. For example, during a shutdown procedure, a thread can be prevented from exiting if there are unprocessed requests to memory or storage, but it can exit if there are unprocessed requests to the sensor system, because the sensor system does not maintain state and will lose state as part of the shutdown process. Device types can be defined through various classifications. For example, classifications such as storage, actuators, and sensors can be used. Device types can also be more specific, such as volatile memory, non-volatile memory, etc.

[0086] The HTP accelerator 400 (e.g., HTP core 402 or thread controller 412) prevents the thread from completing until the counter returns to its initial (e.g., empty) state (e.g., zero). For example, in some architectures, the thread may execute a last instruction (e.g., ETR) to indicate thread completion. In this instance, the last instruction will not be scheduled until the counter returns to zero. Other techniques may also be used, such as pausing the thread after its final instruction until the counter returns to its initial state. In any case, thread completion is prevented while there are still unprocessed return values ​​for instructions issued to the thread. Once the counter reaches its initial state, the consistency of the thread's data or actions is equivalent to the thread never issuing an unblocking request, and the thread can exit without any other entity having to consider the fact that the thread used an unblocking request.

[0087] The following is about Figure 1-4The described architecture context includes instances of instructions with no return indication. For example, floating-point memory atomic instructions. Floating-point atomic memory operations can be sent to a memory controller with built-in atoms. The floating-point operations performed can be MIN, MAX, and ADD for both 32-bit and 64-bit data types.

[0088] Here is an example of the command format:

[0089]

[0090]

[0091] HTP floating-point atomic instructions can specify several parameters for the operation. The following table lists examples of these parameters:

[0092]

[0093] Assuming the nr instruction bit is not set, the result value is written to the F register rd.

[0094] The following are example assembly instructions for threads, including both return indicators and no-return indicators: In this example, atomic instructions have the following potential pitfalls:

[0095]

[0096]

[0097] Another type of instruction is the Programmable Atomic Memory (PMA) instruction. The HTP accelerator 400 can send programmable atomic memory operations to the PAU located within the memory controller. Typically, as described above, when an application points to a PMA routine, the PMA routine is loaded by the operating system. The PMA routine is loaded into the PAU instruction RAM. The PAU instruction RAM is divided into fixed-size segments. The PMA routine is loaded into one or more partitions determined by the size of the routine. The operating system determines the partition where the routine is loaded and initializes a mapping table within each HTP accelerator. When the HTP accelerator 400 executes a PMA instruction, it references the mapping table to determine the partition on the PAU and then begins execution of the programmable atomic routine. Execution of an HTP PMA instruction that references an uninitialized programmable atomic mapping table entry creates a trap. The described mechanism virtualizes PAU system-wide resources.

[0098] The following diagram illustrates an example of the PMA command format:

[0099]

[0100] In practice, PMA instructions can include several parameters for programmable atomic operations. The following table lists some of these parameters:

[0101]

[0102]

[0103] In this example, the number of result values ​​returned from the PAM routine can be in the range of 0-2. The number of return values ​​can be specified by the PAM routine's initialization instruction. In this example, when the nr instruction bit is not set, the result value is written to registers a0-a1.

[0104] An example assembly of executing PAM instructions:

[0105] EAC.C0 rs1# Zero call variable parameters, waiting for result, relaxed consistency

[0106] EAC.C4.NR.AQRL rs1# Four call variable parameters, no waiting, strong consistency.

[0107] The `cc` field is used to specify the number of variable parameters (0, 1, 2, or 4). The following shows the encoding.

[0108] Programmable atomic instruction call variable parameter count suffix:

[0109] cc field encoding suffix Call variable parameter count 0 .C0 0 1 .C1 1 2 .C2 2 3 .C4 4

[0110] Programmable atomic instructions have no return suffix:

[0111] NR field encoding suffix describe 0 No suffix As a result, the HTP thread was paused. 1 .NR No result, do not pause the HTP thread

[0112] Programmable atomic instructions can have the following pitfalls:

[0113] trap describe Invalid address Perform programmable atoms on invalid virtual addresses

[0114] In these instances, a decoded flag indicating that the response value of a memory access instruction is not needed is maintained, and this decoded flag is used to prevent the response value from being written to a general-purpose register when the HTP accelerator 400 receives a response. Furthermore, a count of the number of unprocessed, non-returning memory responses is maintained and used to prevent thread completion to ensure that the parent thread will properly see the correct result in memory.

[0115] These unobstructed external device technologies provide an optimal mechanism for reducing unnecessary thread pauses across various architectures while maintaining data consistency between threads. Figure 5 and 6 This section provides some additional instance details regarding unobstructed external device calls.

[0116] Figure 5 This describes an example of thread execution in a processor under unobstructed calls to external devices, according to an embodiment. Figure 5 The components described above for HTP accelerator 400 are reduced to illustrate that unblocked call technology can be applied to a variety of processor architectures. Here, processor 502 includes execution devices 508 (e.g., one or more ALUs, execution units (EUs), registers, pointers, counters, accumulators, processing pipeline components, etc.) and external interfaces 510 for one or more devices (e.g., memory, memory controllers, sensors, etc.). The processor is described as having a ready-to-run queue 518 with scheduling threads, although other thread scheduling devices may be used.

[0117] When a non-timed call to device 516 is made, thread 504 is executed on processor 502. Thread 504 follows a conceptual path 506 of unobstructed call to device 516 via execution device 508 (e.g., execution, request, etc.). The call (e.g., instruction) enters external interface 510, where, for example, the instruction is translated in a transmission (e.g., CPI packet, bus transfer, etc.) arriving at device 516. In response to the transmission, device 516 provides a return value (e.g., whether the call was successful, data, etc.) to form exchange 514. Time period 518 represents the time required to complete exchange 514.

[0118] When the call is an unblocking (e.g., no-return) call, processor 502 immediately places thread 504 into the ready-to-run queue 518 upon call execution. In contrast, in a conventional scenario, or if thread 504 does not indicate an unblocking call, processor 502 will suspend thread 504 for a period of time 518 to complete the exchange 514 of device 516.

[0119] To maintain data consistency between thread 504 and other actors of processor 502, processor 502 maintains a counter 512 to track how many swaps 514 have been started but not yet completed for thread 504. Therefore, when thread 504 makes an unblocking call, processor 502 increments counter 512. When a return value indicating completion of a swap 514 is received, processor 502 decrements counter 512. If counter 512 is initialized to zero, incremented by one for each call from thread 504 traversing external interface 510, and decremented by one for each return value received at external interface 510, then when the counter is above zero, processor 502 prevents the thread from completing (e.g., exits).

[0120] While unblocking calls are not always appropriate, enabling unblocking calls can reduce the thread execution latency of each of these calls by up to time period 518, for example, when thread 504 applies future instructions to data retrieved from device 516. In environments with many threads, such savings can be significant.

[0121] Figure 6 This section describes an example comparing the operation sequence of a blocked call to an external device with the operation sequence of an unblocked call to an external device according to an embodiment. The process begins by making a standard (e.g., blocked or returned) request for memory at the thread (1.) and then pausing (2.). The processor executes the memory request (3.). The memory receives the request (4.) and returns a result from the request completion (5.). As described above, this result generally includes the request status (e.g., successful completion, failure, error code, etc.) and may also include data (e.g., bits at the memory address specified in the request). Once the processor receives the result (6.), the thread restarts (7.) and continues with additional instructions.

[0122] As explained, the thread's next instruction is a no-return memory request (8.), such as performing an atomic operation at memory. Like a standard request, the no-return request is executed by the processor (10.). However, the thread immediately proceeds to the next instruction (9.) and the processor increments the unprocessed no-return request counter (11.).

[0123] Next, the memory receives the request (12.) and returns a result (14.). In this example, the request indicates that it is a no-return request to the memory. In this example, the memory may ignore data that might otherwise be included in the result. This eliminates unnecessary transfers of data not used by the thread. As explained, the thread attempts to terminate (13.) before the memory returns the result (14.). The processor prevents successful termination because the count indicates that the request has not been processed. However, once the processor receives the result, it decrements the count (15.) to zero, allowing the thread to terminate successfully (16.).

[0124] Figure 7 Examples of a hybrid threaded mesh (HTF) architecture or HTF 700 representing a memory computing device according to embodiments are described. In these examples, HTF 700 may include or include components from... Figure 1 The HTF 142 is an example of this. The HTF 700 is a coarse-grained configurable computation mesh architecture that can be optimized for high-level language operand types and operators (e.g., using C / C++ or other high-level languages). In an instance, the HTF 700 may contain configurable n-bit-wide (e.g., 512-bit-wide) data paths for interconnecting hardened SIMD arithmetic units.

[0125] In this instance, HTF 700 includes an HTF cluster 702, which contains multiple HTF tiles, including instance tiles 704 or tiles N. Each HTF tile may contain one or more compute elements with local memory and arithmetic capabilities. For example, each tile may contain a compute pipeline with support for integer and floating-point operations. In this instance, data paths, compute elements, and other infrastructure may be implemented as hardened IP to provide maximum performance while minimizing power consumption and reconfiguration time.

[0126] exist Figure 7 In this example, the tiles of HTF cluster 702 are arranged linearly, and each tile in the cluster can be coupled to one or more other tiles in HTF cluster 702. Figure 7 In this example, instance tile 704 or tile N is coupled to four other tiles, including coupling to base tile 710 (e.g., tile N-2) via a port labeled SF IN N-2, coupling to adjacent tile 712 (e.g., tile N-1) via a port labeled SF IN N-1, coupling to tile N+1 via a port labeled SF IN N+1, and coupling to tile N+2 via a port labeled SF IN N+2. Instance tile 704 may be coupled to the same or other tiles via corresponding output ports (e.g., ports labeled SF OUT N-1, SF OUT N-2, SF OUT N+1, and SF OUT N+2). In this example, the ordered list of names of the various tiles is a conceptual indication of the tile's location. In other examples, tiles, including HTF cluster 702, may be arranged in a grid or other configuration, where each tile is similarly coupled to one or more of its nearest neighbors in the grid. Tiles positioned at the edge of a cluster may optionally have fewer connections to adjacent tiles. For example, tiles N-2 or... Figure 7 In an instance, the base tile 710 may be coupled only to the adjacent tile 712 (tile N-1) and to the instance tile 704 (tile N). Similarly, fewer or additional inter-tile connections may be used.

[0127] HTF cluster 702 may additionally include a memory interface module, which includes a first memory interface module 706. The memory interface module couples HTF cluster 702 to a NOC, such as a first NOC 118. In an example, the memory interface module allows tiles within the cluster to request other locations in the memory computing system (e.g., within the same or different nodes in the system). That is, the representation of HTF 700 may include part of a larger mesh architecture that can be distributed across multiple nodes, such as having one or more HTF tiles or HTF clusters at each of the nodes. Requests can be made between tiles or nodes within the context of the larger mesh architecture.

[0128] exist Figure 7 In this example, a synchronous mesh (SF) architecture is used to couple tiles within the HTF cluster 702. The synchronous mesh architecture provides communication between a specific tile in the HTF cluster 702 and its neighboring tiles, as described above. Each HTF cluster 702 may additionally include an asynchronous mesh (AF) architecture, which provides communication, for example, between tiles within the cluster, between memory interfaces within the cluster, and between dispatch interfaces 708 within the cluster.

[0129] In this example, a synchronous mesh architecture can exchange messages containing data and control information. Control information may include, among other things, instruction RAM address information or thread identifiers. Control information can be used to set data paths, and data message fields can be selected as the source of the path. Typically, control fields can be provided or received earlier, making them available for configuring the data path. For example, to help minimize any delays in the synchronous domain pipeline within a tile, control information may arrive at the tile several clock cycles before the data fields. Various registers can be provided to help coordinate the timing of data flow in the pipeline.

[0130] In this example, each tile in the HTF cluster 702 may contain multiple memories. Each memory may have the same width as the data path (e.g., 512 bits) and a specified depth, for example, ranging from 512 to 1024 features. The tile memory can be used to store data that supports data path operations. For example, the stored data may contain constants that are part of the cluster configuration of the kernel, or variables that are computed as part of a data stream. In this example, the tile memory may be written from an asynchronous mesh architecture as a data transfer from another synchronization domain, or may contain, for example, the result of a load operation initiated by another synchronization domain. Readings from the tile memory can be performed via synchronous data path instructions in the synchronization domain.

[0131] In the example, each tile in the HTF cluster 702 may have a dedicated instruction RAM (INST RAM). In an example of an HTF cluster 702 with sixteen tiles and an instruction RAM with sixty-four entries, the cluster may allow mapping algorithms with up to 1024 multiplication-shift and / or ALU operations. Various tiles may be optionally pipelined together, for example, using a synchronous mesh architecture to allow computation with minimal memory access to the data stream, thus minimizing latency and reducing power consumption. In the example, an asynchronous mesh architecture may allow memory referencing and computation to continue in parallel, thereby providing a more efficient streaming kernel. In the example, various tiles may include built-in support for loop-based constructions and may support nested loop kernels.

[0132] Synchronous mesh architecture allows for the pipelined execution of multiple tiles, eliminating the need for data queues. Tiles participating in a synchronization domain can, for example, act as a single pipelined data path. The first or base tile of the synchronization domain (e.g., tile N-2, in...) Figure 7 In this example, worker threads can be initiated via pipelined tiles. The base tile is responsible for starting work on a predefined cadence, referred to in this document as the Spoke Count. For instance, if the Spoke Count is 3, the base tile can initiate work every two clock cycles.

[0133] In this example, the synchronization domain comprises a set of connected tiles in the HTF cluster 702. Thread execution can begin at the base tile of the domain and can proceed from the base tile to other tiles in the same domain via the synchronization mesh architecture. The base tile provides instructions to be executed against the first tile. The first tile may by default provide the same instructions for another connected tile to be executed. However, in some instances, the base tile or subsequent tiles may conditionally specify or use alternative instructions. Alternative instructions are selected by causing the tile's data path to produce a Boolean condition value, and then the Boolean value can be used to select between the instruction set of the current tile and the alternative instructions.

[0134] An asynchronous mesh architecture can be used to perform operations that occur asynchronously relative to a synchronous domain. Each piece in an HTF cluster 702 may contain an interface to the asynchronous mesh architecture. The ingress interface may contain, for example, a FIFO buffer or a queue (e.g., AFINQUEUE) to provide storage for messages that cannot be processed immediately. Similarly, the egress interface of the asynchronous mesh architecture may contain a FIFO buffer or a queue (e.g., AFOUTQUEUE) to provide storage for messages that cannot be sent immediately.

[0135] In this example, messages in an asynchronous mesh architecture can be categorized as data messages or control messages. Data messages may contain SIMD-width data values ​​written to tile memory 0 (MEM_0) or memory 1 (MEM_1). Control messages can be configured to control threads to create and release resources or issue external memory references.

[0136] In an HTF cluster 702, tildes can perform various computational operations for HTF. These computational operations can be performed by configuring the data paths within the tilde. In this instance, a tilde contains two function blocks that perform computational operations for the tilde: a multiplication and shift operation block (MS OP) and an arithmetic, logical, and bit operation block (ALB OP). Both blocks can be configured to perform pipelined operations, such as multiplication and addition, or shift and addition, etc.

[0137] In one instance, each example of a memory computing device in the system may have a full set of supported instructions for its operator blocks (e.g., MSOP and ALB OP). In this case, binary compatibility can be achieved across all devices in the system. However, in some instances, it may be helpful to maintain a basic set of functionalities and optional instruction set categories to accommodate various design trade-offs, such as die size. The approach can be similar to how the RISC-V instruction set has a basic set and multiple optional instruction subsets.

[0138] In this example, the instance tile 704 may include spoke RAM. The spoke RAM can be used to specify which input (e.g., from four SF tile inputs and a base tile input) is the master input for each clock cycle. The spoke RAM read address input may originate from a counter counting from zero to the spoke count minus one. In this example, different spoke counts can be used for different tiles, for example, within the same HTF cluster 702, to allow the performance of a particular application or instruction set to be determined by several tiles or unique tile examples used in the inner loop. In this example, the spoke RAM may specify when synchronization inputs will be written to the tile memory, for example, using multiple inputs for a particular tile instruction and the time when one of the inputs arrives before the others. The earlier arriving input can be written to the tile memory and can be read later when all inputs are available. In this example, the tile memory may be accessed as a FIFO memory, and the FIFO read and write pointers may be stored in register-based memory areas or structures within the tile memory.

[0139] Figure 8A and Figure 8B This section provides a general description of examples of chiplet systems that can be used to implement one or more aspects of CNM system 102. As similarly mentioned above, nodes in CNM system 102 or devices within nodes in CNM system 102 may contain chiplet-based architectures or near-memory computing (CNM) chiplets. Packaged memory computing devices may contain, for example, one, two, or four CNM chiplets. Chipslets may be interconnected using high-bandwidth, low-latency interconnects (e.g., using CPI interfaces). Typically, a chiplet system consists of discrete modules (each referred to as a "chiplet") integrated on an interposer layer and, in many instances, interconnected as needed via one or more established networks to provide the desired functionality to the system. The interposer layer and the contained chiplets may be packaged together to facilitate interconnection with other components of a larger system. Each chiplet may contain one or more individual integrated circuits (ICs) or "chips," which may be combined with discrete circuit components and may be coupled to a corresponding substrate for attachment to the interposer layer. Most or all of the chiplets in the system may be individually configured for communication via established networks.

[0140] Chipsets, configured as individual modules within a system, differ from a system implemented on a single chip containing different device blocks (e.g., intellectual property (IP) blocks) on a substrate (e.g., a single die), such as a system-on-a-chip (SoC) or multiple discrete packaged devices integrated on a printed circuit board (PCB). Generally, chiplets offer better performance (e.g., lower power consumption, reduced latency, etc.) than discrete packaged devices, and offer greater manufacturing benefits than a single die chip. These manufacturing benefits may include higher yields or reduced development costs and time.

[0141] A chiplet system may comprise, for example, one or more application (or processor) chiplets and one or more support chiplets. Here, the distinction between application chiplets and support chiplets is merely a reference to possible design scenarios for chiplet systems. Thus, for example, a synthetic vision chiplet system may comprise (by way of example only) application chiplets for generating synthetic vision output, and support chiplets such as memory controller chiplets, sensor interface chiplets, or communication chiplets. In typical use cases, synthetic vision designers may design the application chiplets and obtain the support chiplets from other sources. Therefore, design costs (e.g., in terms of time or complexity) are reduced by avoiding the design and manufacture of the functionality embodied in the support chiplets.

[0142] Chiplets also support the tight integration of IP blocks that might otherwise be difficult, such as IP blocks manufactured using different processing technologies or with different feature sizes (or utilizing different contact technologies or spacing). Therefore, multiple ICs or IC assemblies with different physical, electrical, or communication characteristics can be assembled in a modular manner to provide assemblies with a variety of desired functions. Chiplet systems also facilitate adaptation to the needs of different larger systems that will be incorporated into chiplet systems. In examples, ICs or other assemblies can be optimized for power, speed, or heat generation for specific functions, as might be the case with sensors, and can be integrated with other devices more easily than attempting integration on a single die. Furthermore, by reducing the overall die size, chiplet yields are often higher than those of more complex single-die devices.

[0143] Figure 8A and Figure 8B This section provides a general description of an example of a chiplet system based on an embodiment. Figure 8AThis is an illustration of a chiplet system 802 mounted on a peripheral board 804, which can be connected to a wider range of computer systems, for example, via a high-speed peripheral component interconnect (PCIe). The chiplet system 802 includes a package substrate 806, an interposer 808, and four chiplets: an application chiplet 810, a host interface chiplet 812, a memory controller chiplet 814, and a memory device chiplet 816. Other systems may include numerous additional chiplets to provide additional functionality, as will be apparent from the following discussion. The package of the chiplet system 802 is illustrated with a cap or cover plate 818, but other packaging techniques and structures used for chiplet systems may be used. Figure 8B This is a block diagram illustrating the components in a chiplet system for clarity.

[0144] Application chip 810 is described as a chip system NOC 820 that includes a chip network 822 for supporting inter-chip communication. In an example embodiment, the chip system NOC 820 may be included on application chip 810. In an example, the chip network 822 may be defined in response to selected supporting chips (e.g., host interface chip 812, memory controller chip 814, and memory device chip 816). Figure 1 The first NOC 118 in this example allows designers to choose the appropriate number of chiplet network connections or switches for the chiplet system NOC 820. In this example, the chiplet system NOC 820 may reside on a single chiplet or within the interposer layer 808. In the example discussed herein, the chiplet system NOC 820 implements a chiplet protocol interface (CPI) network.

[0145] In this example, the chiplet system 802 may include or comprise a portion of the first memory computing node 104 or the first memory computing device 112. That is, various blocks or components of the first memory computing device 112 may include chiplets that can be mounted on the peripheral board 804, the package substrate 806, and the interposer 808. The interface components of the first memory computing device 112 may typically include a host interface chiplet 812, the memory and memory control-related components of the first memory computing device 112 may typically include a memory controller chiplet 814, and the various accelerator and processor components of the first memory computing device 112 may typically include an application chiplet 810 or examples thereof.

[0146] For example, the CPI interface, which can be used for communication between or within chiplets in a system, is a packet network that supports virtual channels to enable flexible and high-speed interaction between chiplets. CPI bridges the chiplet network to the chiplet network 822. For instance, the Advanced Extensible Interface (AXI) is a widely used specification for designing intra-chip communication. However, the AXI specification covers a large number of physical design options, such as the number of physical channels, signal timing, and power. Within a single chip, these options are typically selected to meet design goals, such as power consumption and speed. However, to achieve flexibility in chiplet systems, adapters such as CPI are used to intersect between various AXI design options that can be implemented in various chiplets. By implementing a mapping from physical channels to virtual channels and encapsulating time-based signaling using packetization protocols, CPI bridges the intra-chiplet network across the chiplet network 822.

[0147] CPI can utilize various physical layers to transmit packets. A physical layer may contain simple conductive connections, or it may contain drivers to increase voltage, or otherwise facilitate signal transmission over longer distances. Examples of such physical layers may include an Advanced Interface Bus (AIB), which in various instances can be implemented within the intermediate layer 808. The AIB uses source-synchronous data transfer with a forwarding clock to transmit and receive data. Packets are transmitted across the AIB at Single Data Rate (SDR) or Double Data Rate (DDR) relative to the transmitted clock. The AIB supports various channel widths. Channels can be configured with a symmetrical number of transmit (TX) and receive (RX) inputs / outputs (I / O), or an asymmetrical number of transmitters and receivers (e.g., all transmitters or all receivers). A channel may act as either the AIB master or slave depending on which chip provides the master clock. The AIB I / O unit supports three timing modes: asynchronous (i.e., non-timing), SDR, and DDR. In various instances, the non-timing mode is used for clock and some control signals. SDR mode can use a dedicated SDR / DDR I / O unit or a dual-purpose SDR / DDR I / O unit.

[0148] In this example, the CPI packet protocol (e.g., point-to-point or routable) can use symmetrical receive and transmit I / O units within an AIB channel. The CPI streaming protocol allows for more flexible utilization of AIB I / O units. In this example, a streaming-mode AIB channel can configure I / O units as all TX, all RX, or half TX and half RX. The CPI packet protocol can use AIB channels in SDR or DDR operating modes. In this example, AIB channels are configured in increments of 80 I / O units (i.e., 40 TX and 40 RX) for SDR mode and in increments of 40 I / O units for DDR mode. The CPI streaming protocol can use AIB channels in either SDR or DDR operating modes. Here, in this example, the AIB channels are configured in increments of 40 I / O units for both SDR and DDR modes. In this example, a unique interface identifier is assigned to each AIB channel. This identifier is used during CPI reset and initialization to determine paired AIB channels across neighboring chiplets. In this example, the interface identifier is a 20-bit value comprising a seven-bit chiplet identifier, a seven-bit column identifier, and a six-bit link identifier. The AIB physical layer uses an AIB out-of-band shift register to transmit the interface identifier. Bits 32 to 51 of the shift register are used to transmit the 20-bit interface identifier in both directions across the AIB interface.

[0149] AIB defines a stacked group of AIB channels as an AIB channel column. An AIB channel column has a certain number of AIB channels, plus auxiliary channels. The auxiliary channels contain signals used for AIB initialization. All AIB channels within a column (except for the auxiliary channels) have the same configuration (e.g., all TX, all RX, or half TX and half RX, and the same number of data I / O signals). In this example, AIB channels are numbered sequentially in ascending order, starting with the AIB channel adjacent to the AUX channel. The AIB channel adjacent to the AUX channel is defined as AIB channel zero.

[0150] Typically, the CPI interface on an individual chiplet may include serialization-deserialization (SERDES) hardware. SERDES interconnects are well-suited for scenarios requiring high-speed signaling and low signal counts. However, SERDES can introduce additional power consumption and longer latency for multiplexing and demultiplexing, error detection or correction (e.g., using block-level cyclic redundancy check (CRC)), link-level retries, or forward error correction. However, when low latency or power consumption is a primary concern for ultra-short-distance chiplet-to-chiplet interconnects, parallel interfaces that allow data transfer with minimal latency can be utilized. CPIs contain elements designed to minimize both latency and power consumption in these ultra-short-distance chiplet interconnects.

[0151] For flow control, CPI employs a credit-based technique. For example, the receiving side of application chip 810, or the sender side of memory controller chip 814, provides credits indicating available buffers. In this example, the CPI receiver contains buffers for each virtual channel for a given transmit time unit. Therefore, if the CPI receiver supports five messages and a single virtual channel in time, the receiver has five buffers arranged in five rows (e.g., one row per unit time). If four virtual channels are supported, then the receiver has twenty buffers arranged in five rows. Each buffer holds the payload of one CPI packet.

[0152] When a sender transmits to a receiver, the sender decrements its available credits based on the transmission. Once the receiver has exhausted all its credits, the sender stops sending packets to the receiver. This ensures that the receiver always has an available buffer to store transmissions.

[0153] When the receiver processes the received packet and releases the buffer, it sends the available buffer space back to the sender. The sender can then use this credit return to allow the transmission of additional information.

[0154] Figure 8A Examples include a chiplet mesh network 824 that uses direct chiplet-to-chiplet technology without requiring a chiplet system NOC 820. The chiplet mesh network 824 can be implemented in CPI or another chiplet-to-chiplet protocol. The chiplet mesh network 824 typically implements a chiplet pipeline, where one chiplet acts as an interface to the pipeline, while other chipslets in the pipeline interface only interface with themselves.

[0155] Alternatively, a dedicated device interface can be used to connect devices to the chiplet, such as one or more industry-standard memory interfaces (e.g., synchronous memory interfaces, such as DDR5, DDR6). This allows for connections from the chiplet system or individual chiplets to external devices (e.g., larger systems can connect via a desired interface, such as a PCIe interface). In an example, such an external interface can be implemented via a host interface chiplet 812, which, in the depicted example, provides a PCIe interface external to the chiplet system. This type of interface is typically used when industry practice or standards have converged on such a dedicated chiplet interface 826. The illustrated example of connecting a memory controller chiplet 814 to the Double Data Rate (DDR) interface of a dynamic random access memory (DRAM) memory device chiplet 816 exemplifies this industry practice.

[0156] Among the various possible supporting chiplets, the memory controller chiplet 814 is likely to be present in chiplet systems due to the ubiquitous use of storage devices for computer processing and the current level of technological sophistication of memory devices. Therefore, using the memory device chiplet 816 and the memory controller chiplet 814, both designed by other designers, allows chiplet system designers to obtain robust products from established manufacturers. Typically, the memory controller chiplet 814 provides a memory device-specific interface for reading, writing, or erasing data. Often, the memory controller chiplet 814 can provide additional functionality such as error detection, error correction, maintenance operations, or atomic operator execution. For some types of memory, maintenance operations are often specific to the memory device chiplet 816, such as garbage collection in NAND flash or storage-class memory, or temperature regulation (e.g., cross-temperature management) in NAND flash memory. In instances, maintenance operations may involve logic-to-physical (L2P) mapping or management to provide an indirection hierarchy between the physical and logical representations of data. In other types of memory, such as DRAM, some memory operations, such as refresh, may be controlled at some times by the host processor or memory controller, and at other times by the DRAM memory device or logic associated with one or more DRAM devices, such as interface chips (in this example, buffers).

[0157] Atomic operators are data manipulations that can be performed, for example, by the memory controller chip 814. In other chiplet systems, atomic operators can be performed by other chipsets. For example, an application chip 810 can specify an "increment" atomic operator in a command, which includes a memory address and may contain an increment value. Upon receiving the command, the memory controller chip 814 retrieves a number from the specified memory address, increments the number by the amount specified in the command, and stores the result. Upon successful completion, the memory controller chip 814 provides the application chip 810 with an indication that the command was successful. Atomic operators avoid transferring data across the chiplet mesh network 824, thereby reducing latency in executing such commands.

[0158] Atomic operators can be classified as built-in atoms or programmable (e.g., custom) atoms. Built-in atoms are a limited set of operations that are implemented immutably in the hardware. Programmable atoms are small programs that can be executed on programmable atom units (PAUs) (e.g., custom atom units (CAUs)) of the memory controller chip 814.

[0159] The memory device chiplet 816 may be or contain any combination of volatile memory devices or non-volatile memory. Examples of volatile memory devices include, but are not limited to, random access memory (RAM), such as DRAM, synchronous DRAM (SDRAM), graphics double data rate type 6 SDRAM (GDDR6 SDRAM), and so on. Examples of non-volatile memory devices include, but are not limited to, NAND flash memory, memory-class memory (e.g., phase-change memory or memristor-based technology), ferroelectric RAM (FeRAM), and so on. The illustrated example includes the memory device chiplet 816 as a chiplet; however, the device may reside elsewhere, such as in different packages on peripheral board 804. For many applications, multiple memory device chipsets may be provided. In examples, these memory device chipsets may each implement one or more memory technologies and may contain an integrated computing host. In examples, the memory chiplet may contain multiple stacked memory dies of different technologies, such as one or more static random access memory (SRAM) devices stacked with or otherwise communicating with one or more dynamic random access memory (DRAM) devices. In this example, the memory controller chiplet 814 can be used to coordinate the operation of multiple memory chipsets in the chiplet system 802, for example, using one or more memory chipsets in one or more tiers of cache memory, and using one or more additional memory chipsets as main memory. The chiplet system 802 may also include multiple memory controllers 814, which can be used to provide memory control functionality for individual hosts, processors, sensors, networks, etc. For example, the chiplet architecture in the illustrated system offers the advantage of allowing adaptation to different memory storage technologies and different memory interfaces through updated chiplet configurations, without requiring redesign of the rest of the system architecture.

[0160] Figure 9 This section provides examples of chiplet-based implementations for a memory computing device according to embodiments. Examples include implementations having four near-memory computing (CNM) chiplets, each of which may include or incorporate components from... Figure 1 This refers to a portion of the first memory computing device 112 or the first memory computing node 104. Various portions may contain or include corresponding chiplets. Chiplet-based implementations may include or utilize CPI-based intra-system communication, as described above in [the document from...]. Figure 8A and Figure 8B The example of System-on-Chips 802 is discussed similarly.

[0161] Figure 9An example includes a first CNM package 900, which comprises a plurality of chiplets. The first CNM package 900 includes a first chiplet 902, a second chiplet 904, a third chiplet 906, and a fourth chiplet 908 coupled to a CNM NOC hub 910. Each of the first to fourth chiplets may include examples of the same or substantially the same components or modules. For example, each chiplet may each contain a corresponding example of an HTP accelerator, an HTF accelerator, and a memory controller for accessing internal or external memory.

[0162] exist Figure 9 In one example, the first chiplet 902 includes a first NOC hub edge 914 coupled to the CNM NOC hub 910. Other chipslets in the first CNM package 900 similarly include NOC hub edges or endpoints. Switches in the NOC hub edges facilitate communication within the chiplet or within the chiplet system via the CNM NOC hub 910.

[0163] The first chiplet 902 may additionally include one or more memory controllers 916. The memory controller 916 may correspond to a different NOC endpoint switch that interfaces with the first NOC hub edge 914. In examples, the memory controller 916 may include a memory controller chiplet 814 or a memory controller 130, or a memory subsystem 200 or other memory computing implementation. The memory controller 916 may be coupled to a different memory device, such as a first external memory module 912a or a second external memory module 912b. The external memory module may include, for example, GDDR6 memory that is selectively accessible by a different chiplet in the system.

[0164] The first chip 902 may further include, for example, a first HTP chip 918 and a second HTP chip 920 coupled to the edge 914 of the first NOC hub via a corresponding different NOC endpoint switch. The HTP chips may correspond to an HTP accelerator, for example, from... Figure 1 The instance of HTP 140 or from Figure 4 An example of an HTP accelerator 400. The HTP chiplet can communicate with the HTF chiplet 922. The HTF chiplet 922 can correspond to an HTF accelerator, for example, from... Figure 1 The instance of HTF 142 or from Figure 7 The example of HTF 700.

[0165] The CNM NOC hub 910 can be coupled to other chiplets or other CNM packages via various interfaces and switches. For example, the CNM NOC hub 910 can be coupled to a CPI interface via multiple different NOC endpoints on a first CNM package 900. Each of the multiple different NOC endpoints can be coupled to a different node, for example, outside the first CNM package 900. In an example, the CNM NOC hub 910 can be coupled to other peripheral devices, nodes, or devices using CTCPI or other non-CPI protocols. For example, the first CNM package 900 may include a PCIe Scaled Mesh Interface (PCIe / SFI) or a CXL interface (CXL) configured to interface the first CNM package 900 with other devices. In an example, devices coupled to the first CNM package 900 using various CPI, PCIe, CXL, or other mesh architectures can form a common global address space.

[0166] exist Figure 9 In this example, the first CNM package 900 includes a host interface 924 (HIF) and a host processor (R5). The host interface 924 may correspond to, for example, a processor from... Figure 1 The instance is HIF 120. The host processor or R5 can correspond to the one from... Figure 1 The instance includes an internal host processor 122. The host interface 924 may include a PCI interface for coupling the first CNM package 900 to other external devices or systems. In this instance, work can be initiated via the host interface 924 on the first CNM package 900 or on a chip cluster within the first CNM package 900. For example, the host interface 924 may be configured to command individual HTF chip clusters (e.g., within various chiplets in the first CNM package 900) to enter and exit power / clock gate modes.

[0167] Figure 10 This describes an example tile of a memory computing device according to an embodiment. Figure 10 In the example, the tiled chiplet instance 1000 contains examples of four different near-memory computing clusters of chips, where the clusters are coupled together. Each example of a near-memory computing chiplet may itself contain one or more constituent chips (e.g., host processor chips, memory device chips, interface chips, etc.).

[0168] The 1000 tiled chip instances contain data from... Figure 9An example of the first CNM package 900 is one or more of its near-memory computing (CNM) clusters. For example, the tiled chiplet instance 1000 may include a first CNM cluster 1002, which includes a first chiplet 1010 (e.g., corresponding to the first chiplet 902), a second chiplet 1012 (e.g., corresponding to the second chiplet 904), a third chiplet 1014 (e.g., corresponding to the third chiplet 906), and a fourth chiplet 1016 (e.g., corresponding to the fourth chiplet 908). The chipsets in the first CNM cluster 1002 may be coupled to a common NOC hub, which may in turn be coupled to NOC hubs in one or more adjacent clusters (e.g., in the second CNM cluster 1004 or the fourth CNM cluster 1008).

[0169] exist Figure 10 In this example, the tiled chiplet instance 1000 includes a first CNM cluster 1002, a second CNM cluster 1004, a third CNM cluster 1006, and a fourth CNM cluster 1008. These different CNM chipsets can be configured in a common address space, allowing chipsets to be allocated and share resources across different tiles. In this example, chipsets within a cluster can communicate with each other. For instance, the first CNM cluster 1002 can be communicatively coupled to the second CNM cluster 1004 via the inter-chiplet CPI interface 1018, and the first CNM cluster 1002 can be communicatively coupled to the fourth CNM cluster 1008 via another or the same CPI interface. The second CNM cluster 1004 can be communicatively coupled to the third CNM cluster 1006 via the same or another CPI interface, and so on.

[0170] In the example, one of the near-memory computing chiplets in the tiled chiplet instance 1000 may contain a host interface (e.g., corresponding to a source from...). Figure 9 The host interface (924) of the instance is responsible for workload balancing across the tiled chiplet instances 1000. The host interface can facilitate, for example, access to host-based command request queues and response queues from outside the tiled chiplet instances 1000. The host interface can dispatch new execution threads using hybrid-threaded processors and hybrid-threaded mesh architectures in one or more near-memory computing chips within the tiled chiplet instances 1000.

[0171] Figure 11 This is a flowchart illustrating an example of a method 1100 for unobstructed external device invocation according to an embodiment. The operation of method 1100 is, for example, related to... Figure 1-7The computer hardware described in 8A-10 and 12 executes, for example, a memory computing device 112, a memory controller 200, a PAU 208 or PAU 302, an HTP 400 or an HTF 700, or various combinations thereof. The computer hardware performing the operation of method 1100 includes a processing circuitry configured to operate (e.g., hardwired, via software containing firmware, or a combination of both). In an example, the processing circuitry is a portion of the processor (e.g., an HTP 400 or PAU 302) that includes external interfaces (e.g., access to memory), execution pipelines, and thread management circuitry (e.g., a thread ready-to-run queue).

[0172] At operation 1102, an instruction is received from the thread via the processing circuitry; the instruction points to a device, such as external memory or other hardware accessed via an external interface. Here, the instruction corresponds to a no-return indicator. In an example, the no-return indicator is contained within the instruction. In an example, the no-return indicator is contained within a field of the instruction. In an example, the no-return indicator is a single bit. In an example, the single bit is one of several bits in the opcode field of the instruction.

[0173] In this example, the device is a memory controller. In this example, the instruction is an atomic operation. In this example, the atomic operation is a programmable atomic operation.

[0174] At operation 1104, a counter corresponding to the thread is incremented based on the no-return indication. For example, when processing a circuit system disposal instruction, the counter used for the thread is incremented by one to maintain the count of unprocessed no-return requests.

[0175] At operation 1106, thread execution continues without waiting for a return value from the device. In this instance, the return value is the result of an atomic operation. Therefore, the thread does not block (e.g., pause, sleep, etc.) waiting for a requested result. In this instance, continuing thread execution involves placing the thread's identifier onto the execution stack of the processing circuitry system (e.g., a thread-ready run queue or other thread instruction scheduling device).

[0176] At operation 1108, the counter is decremented (e.g., reduced) based on the received return value for the request. In this instance, the counter is a unique counter used by the thread for each non-returning instruction sent to a device outside the processing circuitry. In this instance, the return value is received (e.g., via the processing circuitry) and discarded. In this instance, the return value indicates whether the atomic operation was successfully completed without containing the result of the atomic operation.

[0177] At operation 1110, thread completion is prevented until the counter reaches zero. In this example, preventing thread completion includes removing the thread return instruction from execution when the counter is greater than zero.

[0178] Figure 12 A block diagram of an example machine 1200 is shown, which may be used, in, or through which any one or more techniques (e.g., methods) discussed herein may be implemented. As described herein, the example may contain logic or several components or mechanisms in, or operable by, machine 1200. A circuit system (e.g., a processing circuit system) is a collection of circuits implemented in the tangible entity of machine 1200, which includes hardware (e.g., simple circuits, gates, logic, etc.). The membership of the circuit system may be flexible over time. The circuit system includes members that can perform specific tasks individually or in combination during operation. In the example, the hardware of the circuit system may be designed in an immutable manner to perform a specific operation (e.g., hardwired). In the example, the hardware of the circuit system may include variably connected physical components (e.g., execution units, transistors, simple circuits, etc.) that include machine-readable media that are physically modified (e.g., invariant magnetic properties of concentrated particles, electrically movable placement, etc.) to encode instructions for a specific operation. When connecting physical components, the fundamental electrical properties of the hardware components are altered, for example, from an insulator to a conductor, or vice versa. Instructions enable embedded hardware (e.g., an execution unit or loading mechanism) to hardware-create components of a circuit system via variable connections to perform specific operations when in operation. Thus, in an example, a machine-readable media element is part of the circuit system or communicatively coupled to other components of the circuit system during device operation. In an example, any one of the physical components can be used in more than one component of more than one circuit system. For example, in operation, an execution unit may be used at one point in time in a first circuit of a first circuit system and reused by a second circuit of the first circuit system, or reused at a different time by a third circuit of a second circuit system. Further examples of these components of machine 1200 are given below.

[0179] In alternative embodiments, machine 1200 may operate as a standalone device or be connected (e.g., networked) to other machines. In a networked deployment, machine 1200 may operate as a server machine, a client machine, or both in a server-client network environment. In an example, machine 1200 may act as a peer-to-peer (P2P) (or other distributed) network environment. Machine 1200 may be a personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), mobile phone, network appliance, network router, switch, or bridge, or any machine capable of executing (sequentially or otherwise) instructions specifying actions to be taken by said machine. Furthermore, although only a single machine is shown, the term "machine" should also be considered to include any collection of machines, such as cloud computing, software as a service (SaaS), and other computer cluster configurations, that individually or jointly execute one or more sets of instructions to perform any one or more of the methodologies discussed herein.

[0180] Machine 1200 (e.g., a computer system) may include a hardware processor 1202 (e.g., a central processing unit (CPU), graphics processing unit (GPU), hardware processor core, or any combination thereof), main memory 1204, static memory 1206 (e.g., memory or storage device for firmware, microcode, basic input / output (BIOS), unified extensible firmware interface (UEFI), etc.), and mass storage device 1208 (e.g., hard disk drive, tape drive, flash storage device, or other block device), some or all of which may communicate with each other via interconnect 1230 (e.g., a bus). Machine 1200 may additionally include a display device 1210, an alphanumeric input device 1212 (e.g., a keyboard), and a user interface (UI) navigation device 1214 (e.g., a mouse). In an example, the display device 1210, the input device 1212, and the UI navigation device 1214 may be a touchscreen display. Machine 1200 may additionally include a mass storage device 1208 (e.g., a drive unit), a signal generation device 1218 (e.g., a speaker), a network interface device 1220, and one or more sensors 1216, such as a Global Positioning System (GPS) sensor, a compass, an accelerometer, or other sensors. Machine 1200 may include an output controller 1228, for example, serial (e.g., Universal Serial Bus (USB), parallel, or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connection, to communicate with or control one or more peripheral devices (e.g., a printer, a card reader, etc.).

[0181] The registers of the hardware processor 1202, main memory 1204, static memory 1206, or mass storage device 1208 may be or contain machine-readable medium 1222, on which one or more sets of data structures or instructions 1224 (e.g., software) embodying or used by any one or more of the techniques or functions described herein are stored. Instructions 1224 may also remain wholly or at least partially within the registers of the hardware processor 1202, main memory 1204, static memory 1206, or mass storage device 1208 during execution by the machine 1200. In this example, one or any combination of the hardware processor 1202, main memory 1204, static memory 1206, or mass storage device 1208 may constitute the machine-readable medium 1222. Although machine-readable media 1222 is described as a single medium, the term "machine-readable media" may include a single medium or multiple media (e.g., a centralized or distributed database, or associated cache and server) configured to store one or more instructions 1224.

[0182] The term "machine-readable media" can include any medium capable of storing, encoding, or carrying instructions for execution by machine 1200 and causing machine 1200 to perform any or more of the technologies disclosed herein, or any medium capable of storing, encoding, or carrying data structures used by or associated with such instructions. Examples of non-limiting machine-readable media can include solid-state memory, optical media, magnetic media, and signals (e.g., radio frequency signals, other photon-based signals, sound signals, etc.). In examples, non-transitory machine-readable media includes machine-readable media having a plurality of particles having invariant (e.g., rest) mass and therefore being composed of matter. Therefore, non-transitory machine-readable media is machine-readable media that does not contain transiently propagating signals. Specific examples of non-transitory machine-readable media can include: non-volatile memory, such as semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)) and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0183] In this example, information stored or otherwise provided on machine-readable medium 1222 may represent instructions 1224, such as instructions 1224 themselves or a format from which instructions 1224 can be derived. This format from which instructions 1224 can be derived may include source code, encoded instructions (e.g., in compressed or encrypted form), encapsulated instructions (e.g., split into multiple encapsulations), etc. Information representing instructions 1224 on machine-readable medium 1222 may be processed by a processing circuitry system into instructions to perform any of the operations discussed herein. For example, deriving instructions 1224 from information (e.g., processed by a processing circuitry system) may include: compiling (e.g., from source code, object code, etc.), interpreting, loading, organizing (e.g., dynamically or statically linking), encoding, decoding, encrypting, decrypting, encapsulating, decapsulating, or otherwise manipulating information into instructions 1224.

[0184] In an example, the derivation of instruction 1224 may involve assembling, compiling, or decompiling information (e.g., by processing a circuit system) to produce instruction 1224 from some intermediate or preprocessed format provided by machine-readable media 1222. When information is provided in multiple parts, the information may be combined, decapsulated, and modified to create instruction 1224. For example, the information may be in multiple compressed source code packages (or object code, or binary executable code, etc.) on one or more remote servers. The source code packages may be encrypted when transmitted over a network and, if necessary, decrypted, decompressed, assembled (e.g., linked), and compiled or decompiled at the local machine (e.g., into a stand-alone executable library, etc.) and executed by the local machine.

[0185] Instruction 1224 can further utilize any of a plurality of transport protocols (e.g., Frame Relay, Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), etc.) to transmit or receive on the communication network 1226 via the network interface device 1220 using the transport medium. Example communication networks may include local area networks (LANs), wide area networks (WANs), packet data networks (e.g., the Internet), mobile phone networks (e.g., cellular networks), conventional telephone (POTS) networks, and wireless data networks (e.g., referred to as…). The Institute of Electrical and Electronics Engineers (IEEE) 802.11 standard series, known as The IEEE 802.16 series of standards, the IEEE 802.15.4 series of standards, peer-to-peer (P2P) networks, etc., are examples of such standards. In an example, network interface device 1220 may include one or more physical jacks (e.g., Ethernet, coaxial, or telephone jacks) or one or more antennas for connection to communication network 1226. In an example, network interface device 1220 may include multiple antennas for wireless communication using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) technologies. The term "transmission medium" should be considered as including any intangible medium capable of storing, encoding, or carrying instructions for execution by machine 1200, and including digital or analog communication signals or other intangible media used to facilitate communication of this software. The transmission medium is a machine-readable medium.

[0186] To better illustrate the methods and apparatus described herein, a set of non-limiting exemplary embodiments are set forth below as numbered examples.

[0187] Example 1 is a system comprising: a first memory computing node including: means configured to accept remote calls; and a second memory computing node including: a hybrid thread processor (HTP) including processing circuitry configured to: receive instructions from a thread for the means, the instructions corresponding to a no-return indication; increment a counter corresponding to the thread based on the no-return indication; continue execution of the thread without waiting for a return value from the means; decrement the counter based on receiving the return value; and prevent the thread from completing until the counter is zero.

[0188] In Example 2, the object according to Example 1 includes, in order to continue the execution of the thread, the processing circuitry is configured to place the thread identifier into the processor's ready-to-run queue.

[0189] In Example 3, the subject matter according to Examples 1 and 2 includes, in order to prevent the thread from completing, the processing circuitry is configured to remove the thread return instruction from execution when the counter is greater than zero.

[0190] In Example 4, the object described according to Examples 1 to 3 includes wherein the processing circuitry is further configured to: receive the return value; and discard the return value.

[0191] In Example 5, the object described according to Examples 1 to 4 includes, wherein the device is a memory controller.

[0192] In Example 6, the object according to Example 5 includes, wherein the instruction is an atomic operation.

[0193] In Example 7, the object described in Example 6 includes the return value being the result of the atomic operation.

[0194] In Example 8, the object described in Examples 6 and 7 includes a return value that indicates whether the atomic operation was successfully completed without including the result of the atomic operation.

[0195] In Example 9, the object described according to Examples 6 to 8 includes wherein the atomic operation is a programmable atomic operation.

[0196] In Example 10, the subject matter according to Examples 1 to 9 includes, wherein the first memory computing node is the second memory computing node.

[0197] In Example 11, the subject matter according to Examples 1 to 10 includes, wherein the counter is a unique counter for each no-return instruction sent by the thread to a device outside the HTP.

[0198] In Example 12, the object described in Examples 1 to 11 includes, wherein the no-return indication is included in the instruction.

[0199] In Example 13, the object described in Example 12 includes, wherein the no-return indication is a field of the instruction.

[0200] In Example 14, the object described according to Examples 12 to 13 includes, wherein the no-return indication is a single bit.

[0201] In Example 15, the object according to Example 14 includes, wherein the single bit is one of several bits in the operation code field of the instruction.

[0202] Example 16 is a device comprising: an interface to the device; and a processing circuitry configured to: receive an instruction from a thread for the device, the instruction corresponding to a no-return indication; in response to receiving the instruction, make a call to the device across the interface; increment a counter corresponding to the thread based on the no-return indication; continue execution of the thread without waiting for a return value from the device; decrement the counter based on receiving the return value; and prevent the thread from completing until the counter reaches zero.

[0203] In Example 17, the object according to Example 16 includes, in order to continue the execution of the thread, the processing circuitry is configured to place the thread identifier into the processor's ready-to-run queue.

[0204] In Example 18, the subject matter according to Examples 16 and 17 includes, in order to prevent the thread from completing, the processing circuitry is configured to remove the thread return instruction from execution when the counter is greater than zero.

[0205] In Example 19, the subject matter according to Examples 16 to 18 includes wherein the processing circuitry is further configured to: receive the return value; and discard the return value.

[0206] In Example 20, the subject matter according to Examples 16 to 19 includes, wherein the means is a memory controller.

[0207] In Example 21, the object according to Example 20 includes, wherein the instructions are atomic operations.

[0208] In Example 22, the object described in Example 21 includes the return value being the result of the atomic operation.

[0209] In Example 23, the object described in Examples 21 to 22 includes a return value that indicates whether the atomic operation was successfully completed without including the result of the atomic operation.

[0210] In Example 24, the object described according to Examples 21 to 23 includes wherein the atomic operation is a programmable atomic operation.

[0211] In Example 25, the subject matter described in Examples 16 to 24 includes, wherein the device is a hybrid thread processor.

[0212] In Example 26, the subject matter described in Examples 16 to 25 includes, wherein the counter is a unique counter for each no-return instruction sent by the thread to a device outside the device.

[0213] In Example 27, the object described in Examples 16 to 26 includes, wherein the no-return indication is included in the instruction.

[0214] In Example 28, the object described in Example 27 includes, wherein the no-return indication is a field of the instruction.

[0215] In Example 29, the object described according to Examples 27 and 28 includes, wherein the no-return indication is a single bit.

[0216] In Example 30, the object according to Example 29 includes, wherein the single bit is one of several bits in the operation code field of the instruction.

[0217] Example 31 is a method comprising: receiving, via a processor, an instruction from a thread for a device, the instruction corresponding to a no-return indication; incrementing a counter corresponding to the thread via the processor based on the no-return indication; continuing execution of the thread via the processor without waiting for a return value from the device; decrementing the counter via the processor based on the received return value; and preventing the thread from completing until the counter reaches zero via the processor.

[0218] In Example 32, the object according to Example 31 includes, wherein continuing the execution of the thread includes placing the thread identifier into the processor's ready-to-run queue.

[0219] In Example 33, the object inclusion according to Examples 31 to 32 is provided, wherein a thread return instruction is removed from execution to prevent thread completion inclusion when a counter is greater than zero.

[0220] In Example 34, the object described in Examples 31 to 33 includes receiving the return value and discarding the return value.

[0221] In Example 35, the subject matter according to Examples 31 to 34 includes, wherein the means is a memory controller.

[0222] In Example 36, the object according to Example 35 includes, wherein the instruction is an atomic operation.

[0223] In Example 37, the object described in Example 36 includes the return value being the result of the atomic operation.

[0224] In Example 38, the object described in Examples 36 and 37 includes a return value that indicates whether the atomic operation was successfully completed without including the result of the atomic operation.

[0225] In Example 39, the object described according to Examples 36 to 38 includes wherein the atomic operation is a programmable atomic operation.

[0226] In Example 40, the subject matter described according to Examples 31 to 39 includes, wherein the processor is a hybrid-threaded processor.

[0227] In Example 41, the subject matter described in Examples 31 to 40 includes a counter that is a unique counter for each no-return instruction sent by a thread to a device outside the processor.

[0228] In Example 42, the object described in Examples 31 to 41 includes, wherein the no-return indication is included in the instruction.

[0229] In Example 43, the object described in Example 42 includes, wherein the no-return indication is a field of the instruction.

[0230] In Example 44, the object described according to Examples 42 to 43 includes, wherein the no-return indication is a single bit.

[0231] In Example 45, the object according to Example 44 includes, wherein the single bit is one of several bits in the operation code field of the instruction.

[0232] Example 46 is a machine-readable medium containing instructions that, when executed by a processing circuitry system, cause the processing circuitry system to perform operations including: receiving an instruction from a thread for a device, the instruction corresponding to a no-return indication; incrementing a counter corresponding to the thread based on the no-return indication; continuing the execution of the thread without waiting for a return value from the device; decrementing the counter based on receiving a return value; and preventing the thread from completing until the counter is zero.

[0233] In Example 47, the object described in Example 46 includes, wherein continuing the execution of the thread includes placing the thread identifier into the processor's ready-to-run queue.

[0234] In Example 48, the object is included according to Examples 46 and 47, wherein thread completion includes the removal of the thread return instruction from execution when the counter is greater than zero.

[0235] In Example 49, the object described according to Examples 46 to 48 includes, wherein the operation further includes: receiving the return value; and discarding the return value.

[0236] In Example 50, the subject matter according to Examples 46 to 49 includes, wherein the means is a memory controller.

[0237] In Example 51, the object according to Example 50 includes, wherein the instructions are atomic operations.

[0238] In Example 52, the object described in Example 51 includes the return value being the result of the atomic operation.

[0239] In Example 53, the object described in Examples 51 to 52 includes a return value that indicates whether the atomic operation was successfully completed without including the result of the atomic operation.

[0240] In Example 54, the object described according to Examples 51 to 53 includes wherein the atomic operation is a programmable atomic operation.

[0241] In Example 55, the subject matter described according to Examples 46 to 54 includes, wherein the processor is a hybrid-threaded processor.

[0242] In Example 56, the subject matter described in Examples 46 to 55 includes a counter that is a unique counter for each no-return instruction sent by a thread to a device outside the processor.

[0243] In Example 57, the object described in Examples 46 to 56 includes, wherein the no-return indication is included in the instruction.

[0244] In Example 58, the object described in Example 57 includes, wherein the no-return indication is a field of the instruction.

[0245] In Example 59, the object described according to Examples 57 to 58 includes, wherein the no-return indication is a single bit.

[0246] In Example 60, the object according to Example 59 includes, wherein the single bit is one of several bits in the operation code field of the instruction.

[0247] Example 61 is a system comprising: means for receiving, via a processor, an instruction from a thread for a device, the instruction corresponding to a no-return indication; means for incrementing a counter corresponding to the thread via the processor based on the no-return indication; means for continuing execution of the thread via the processor without waiting for a return value from the device; means for decrementing the counter via the processor based on the received return value; and means for preventing the thread from completing until the counter reaches zero via the processor.

[0248] In Example 62, the object according to Example 61 includes, wherein the means for continuing the execution of the thread includes means for placing the thread identifier into the processor's ready-to-run queue.

[0249] In Example 63, the object according to Examples 61 to 62 includes, wherein the means for preventing thread completion includes means for removing a thread return instruction from execution when a counter is greater than zero.

[0250] In Example 64, the object described according to Examples 61 to 63 includes means for receiving the return value; and means for discarding the return value.

[0251] In Example 65, the subject matter according to Examples 61 to 64 includes, wherein the means is a memory controller.

[0252] In Example 66, the object described according to Example 65 includes, wherein the instructions are atomic operations.

[0253] In Example 67, the object described in Example 66 includes the return value being the result of the atomic operation.

[0254] In Example 68, the object described in Examples 66 and 67 includes a return value that indicates whether the atomic operation was successfully completed without including the result of the atomic operation.

[0255] In Example 69, the object described according to Examples 66 to 68 includes wherein the atomic operation is a programmable atomic operation.

[0256] In Example 70, the subject matter according to Examples 61 to 69 includes, wherein the processor is a hybrid-threaded processor.

[0257] In Example 71, the subject matter described in Examples 61 to 70 includes a counter that is a unique counter for each no-return instruction issued by a device outside the processor for a thread.

[0258] In Example 72, the object described in Examples 61 to 71 includes, wherein the no-return indication is included in the instruction.

[0259] In Example 73, the object described in Example 72 includes, wherein the no-return indication is a field of the instruction.

[0260] In Example 74, the object described according to Examples 72 to 73 includes, wherein the no-return indication is a single bit.

[0261] In Example 75, the object according to Example 74 includes, wherein the single bit is one of several bits in the operation code field of the instruction.

[0262] Example 76 is at least one machine-readable medium containing instructions that, when executed by a processing circuitry system, cause the processing circuitry system to perform an operation to implement any one of Examples 1-75.

[0263] Example 77 is an apparatus that includes means for implementing any of Examples 1-75.

[0264] Example 78 is a system that implements any of the examples 1-75.

[0265] Example 79 is a method for implementing any of the examples in Examples 1-75.

[0266] The above detailed description includes reference to the accompanying drawings, which form a part of the detailed description. The drawings illustrate, by means of illustration, specific embodiments in which the invention may be practiced. These embodiments are also referred to herein as “examples.” Such examples may include elements other than those shown or described. However, the inventors also contemplate that only examples of those elements shown or described are provided herein. Furthermore, the inventors also contemplate examples (or one or more aspects thereof) of any combination or arrangement of those elements shown or described with respect to a particular example (or one or more aspects thereof) or with respect to other examples (or one or more aspects thereof) shown or described herein.

[0267] In this document, as is common in patent documents, the term “a” is used to include one or more, regardless of any other examples or uses of “at least one” or “one or more.” In this document, unless otherwise indicated, the term “or” is used to refer to a non-exclusive “or,” such that “A or B” can include “A but not B,” “B but not A,” and “A and B.” In the appended claims, the terms “comprising” and “in which” are used as common equivalents to the corresponding terms “including” and “wherein.” Furthermore, in the appended claims, the terms “comprising” and “including” are open-ended, meaning that a system, apparatus, article, or process that includes elements other than those listed after this term in the claims is still considered to be within the scope of the claims. Additionally, in the following claims, the terms “first,” “second,” and “third,” etc., are used merely as labels and are not intended to impose numerical requirements on their objects.

[0268] The above description is intended to be illustrative and not restrictive. For example, the examples described above (or one or more aspects thereof) can be used in combination with each other. Other embodiments may be used by those skilled in the art upon review of the above description. The abstract is submitted under the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Similarly, in the above detailed description, various features may be grouped together to simplify this disclosure. This should not be construed as expecting that any unclaimed disclosed features are necessary for any claim. In fact, the subject matter of the invention may lie in fewer features than all of the particular disclosed embodiments. Therefore, the appended claims are hereby incorporated into the detailed description, wherein each claim exists independently as a separate embodiment, and these embodiments are considered to be combined or arranged in various ways with each other. The scope of the invention should be determined by reference to the appended claims and the full scope of the equivalents granted by the claims.

Claims

1. A system comprising: The first memory computing node includes: A device configured to accept remote calls; and The second memory computing node includes: A hybrid thread processor (HTP) comprising a processing circuitry configured to perform the following operations: Receive instructions from a thread for the device, the instructions corresponding to a no-return indication; The counter corresponding to the thread is incremented based on the no-return indication; The thread continues execution regardless of the return value from the device; The counter is decremented based on the received return value; and When the counter is greater than zero, the thread is prevented from completing until the counter is zero by removing the thread return instruction from the execution.

2. The system of claim 1, wherein, in order to continue the execution of the thread, the processing circuitry is configured to place the identifier of the thread into the ready-to-run queue of the hybrid thread processor.

3. The system of claim 1, wherein the processing circuitry is further configured to: Receive the return value; and Discard the returned value.

4. The system of claim 1, wherein the device is a memory controller.

5. The system of claim 4, wherein the instruction is an atomic operation.

6. The system of claim 5, wherein the return value is the result of the atomic operation.

7. The system of claim 5, wherein the return value indicates whether the atomic operation was successfully completed without including the result of the atomic operation.

8. The system of claim 5, wherein the atomic operation is a programmable atomic operation.

9. The system according to claim 1, wherein the first memory computing node is the second memory computing node.

10. The system of claim 1, wherein the counter is a unique counter for each no-return instruction sent by the thread to a device outside the HTP.

11. The system of claim 1, wherein the no-return indication is included in the instruction.

12. The system of claim 11, wherein the no-return indication is a field of the instruction.

13. The system of claim 11, wherein the no-return indication is a single bit.

14. The system of claim 13, wherein the single bit is one of several bits in the operation code field of the instruction.

15. An apparatus comprising: Interface leading to the device; and The processing circuit system is configured as follows: Receive instructions from a thread for the device, the instructions corresponding to a no-return indication; In response to receiving the instruction, a call to the device is made across the interface; The counter corresponding to the thread is incremented based on the no-return indication; The execution of the thread continues without waiting for a return value from the device; The counter is decremented based on the received return value; and When the counter is greater than zero, the thread is prevented from completing until the counter is zero by removing the thread return instruction from the execution.

16. The device of claim 15, wherein, in order to continue the execution of the thread, the processing circuitry is configured to place an identifier of the thread into the ready-to-run queue of the device.

17. The device of claim 15, wherein the means is a memory controller.

18. The device of claim 17, wherein the instruction is an atomic operation.

19. The apparatus of claim 18, wherein the atomic operation is a programmable atomic operation.

20. The device of claim 15, wherein the device is a hybrid thread processor.

21. The device of claim 15, wherein the counter is a unique counter for each no-return instruction sent by the thread to a device outside the device.

22. The device of claim 15, wherein the no-return indication is included in the instruction.

Images