Saturated local cache in memory computing systems

Abstract

Embodiments of this application relate to saturated local caches in memory computing systems. Latency can be problematic in node-based near-memory computing systems. Solutions to this problem may include or utilize a dedicated software-based cache at each node. The cache may be configured to store information received from each of the other nodes in the system. In an example, the cache may be populated during a breadth-first search algorithm to store boundary information from each of the other nodes.

Description

[0001] Statement regarding federally funded research or development

[0002] This invention was made with government support under Contract No. HR0011-19-3-0002 granted by DARPA. The government enjoys certain rights in this invention.

[0003] Cross-reference to related applications

[0004] This application claims priority to U.S. Patent Application No. 63 / 132,799, filed December 31, 2020, which is incorporated herein by reference in its entirety. Technical Field

[0005] Embodiments of this application relate to memory computing systems, and more specifically, to saturated local caches within memory computing systems. Background Technology

[0006] For example, various computer architectures based on the von Neumann architecture conventionally use shared memory for data, buses for accessing shared memory, arithmetic units, and program control units. However, moving data between the processor and memory can be time-consuming and energy-intensive, which can constrain the performance and capacity of computer systems. Given these limitations, new computing architectures and devices are needed to drive computing performance beyond the transistor scale (i.e., Moore's Law). Summary of the Invention

[0007] According to embodiments of this disclosure, a method is provided, comprising: establishing N cache blocks at a first memory computing node in a near-memory computing system comprising N different hardware nodes, wherein the first memory computing node is one of the N different hardware nodes, and wherein the N different hardware nodes are coupled by a computing structure; and for each initial request from the first memory computing node to read information from a particular one of the N different nodes, filling a corresponding one of the N cache blocks with boundary information received from the particular one of the N different nodes, wherein the boundary information indicates whether a parent object of the particular one of the N different nodes was found during some or all of the previous searches in the nodes coupled by the computing structure.

[0008] According to embodiments of the present disclosure, an apparatus is provided, and the apparatus includes a first memory computing node among a plurality of memory computing nodes in a near-memory computing system, wherein the first memory computing node includes a processor configured to perform operations including: establishing N blocks of software cache, each of the N blocks corresponding to a corresponding memory computing node in the near-memory computing system; and filling the corresponding block in the cache with boundary information received from the corresponding memory computing node in the near-memory computing system.

[0009] According to embodiments of this disclosure, a memory computing system is provided, comprising N distinct memory computing nodes coupled by a scale structure to provide a computing architecture, wherein each of the N nodes includes a hybrid thread processor and a hybrid thread structure, wherein a first memory computing node among the N distinct memory computing nodes includes a processor configured to perform operations including: establishing N blocks of software cache, wherein each of the N blocks corresponds to a corresponding one of the memory computing nodes coupled by the scale structure; and filling the corresponding block in the cache with boundary information received from the corresponding node, wherein the boundary information indicates whether a parent object of the corresponding one of the N nodes was found during some or all of the previous searches in the nodes coupled by the hybrid thread structure. Attached Figure Description

[0010] To facilitate identification of any particular element or action, one or more of the most significant digits in the reference numerals refer to the figure number in which the element is first introduced.

[0011] Figure 1 This typically describes a first instance of a first memory computing device in the context of a memory computing system according to an embodiment.

[0012] Figure 2 Examples of memory subsystems of memory computing devices according to embodiments are typically described.

[0013] Figure 3 Examples of programmable atomic units for a memory controller, according to embodiments, are typically described.

[0014] Figure 4 An example of a hybrid thread processor (HTP) accelerator for a memory computing device according to an embodiment is described.

[0015] Figure 5 An example of a representation of a hybrid thread structure (HTF) of a memory computing device according to an embodiment is described.

[0016] Figure 6A Examples of chiplet systems according to embodiments are typically described.

[0017] Figure 6B The description usually indicates that the information is from Figure 6A A block diagram of the various components in an example chiplet system.

[0018] Figure 7 Examples of chiplet-based implementation schemes for memory computing devices according to embodiments are typically described.

[0019] Figure 8 This describes an example of tiling a memory computing device chip according to an embodiment.

[0020] Figure 9 This typically illustrates an example of a data structure distributed across different memory computing devices.

[0021] Figure 10 This typically describes an example of a data structure distributed across different memory computing devices, each of which contains a saturated cache.

[0022] Figure 11 This typically describes instances where boundary information from different memory computing devices is mapped to corresponding different saturated caches.

[0023] Figure 12 Typically, the description may include instances of creating and populating a node-based saturation cache.

[0024] Figure 13 Typically, the description may include instances that use information from the local cache to satisfy read requests.

[0025] Figure 14 A block diagram illustrating an example machine may be used, in, or through which any one or more techniques (e.g., methods) discussed herein may be implemented. Detailed Implementation

[0026] Recent advances in materials, devices, and integration technologies can be leveraged to provide memory-centric computing topologies. Such topologies enable advancements in computational efficiency and workload handling for applications constrained by size, weight, or power requirements. Topologies can facilitate low-latency computing in or near memory or other data storage elements. The approach is particularly well-suited for various computationally intensive operations utilizing sparse lookups, such as in transform computations (e.g., Fast Fourier Transform (FFT) computations), or in applications such as neural networks or artificial intelligence (AI), financial analysis, or simulation or modeling, for example, computational fluid dynamics (CFD), engineer-as-a-system (EASE) augmented acoustic simulators, integrated circuit-centric simulation programs (SPICE), etc.

[0027] The systems, apparatuses, and methods discussed herein may include or utilize memory computing systems with processors or processing capabilities provided in, near, or integrated with memory or data storage components. Such systems are generally referred to herein as compute-near-memory (CNM) systems. CNM systems may be node-based systems, where individual nodes in the system are coupled using a system-scale architecture. Especially in environments where high cache miss rates are expected, each node may include or utilize a dedicated or general-purpose processor and a user-accessible accelerator (with a custom computational architecture to facilitate intensive operations).

[0028] This will be discussed in more detail below, especially in relation to Figures 9 to 13 Latency can be problematic in node-based CNM systems. The problem may involve efficiently using a breadth-first search (BFS) algorithm across a large number of nodes in the system, where each node may contain a local memory storage device accessible by the same node or by another node in the system. The inventors have recognized that a solution to the problem can include or utilize a dedicated, saturated, software-based cache at each node. For example, in a BFS algorithm, the frontier of a vertex with a relatively high degree or the most "counts" is more likely to be accessed during the search. To improve performance, the bottom-up (BU) phase of the BFS algorithm can use a local cache (e.g., a software-implemented cache of, for example, 1024 × 64-bit words per memory computing device in the system) to store the results of remote accesses to the frontier information corresponding to the highest-degree vertex. For example, at a first node, a first access to one of the high-degree vertices may read an element from a remote location (i.e., a location far from the first node) and then store that element in the local cache at the first node. In this way, subsequent attempts to access the same remote location from the first node can be satisfied by the local software cache at the first node, which helps to avoid remote access activities that cause latency.

[0029] In this example, each node in a CNM system can have one or more host processors. Within each node, a dedicated hybrid-threaded processor can occupy discrete endpoints of the on-chip network. The hybrid-threaded processor can access some or all of the memory in a specific node of the system, or it can access the memory of a network spanning multiple nodes via a system-scale architecture. Custom computational architectures or hybrid-threaded architectures at each node can have their own processors or accelerators and can operate at higher bandwidths than the hybrid-threaded processors. Different nodes in a near-memory computing system can be configured differently, for example, with different computational capabilities, different types of memory, different interfaces, or other differences. However, nodes can be coupled together to share data and computational resources within a defined address space.

[0030] In this example, near-memory computing systems or nodes within a system can be user-configurable for custom operations. Users can provide instructions using a high-level programming language (e.g., C / C++), which can be compiled and directly mapped to the dataflow architecture of one or more nodes in the system or CNM system. That is, nodes in the system can contain hardware blocks (e.g., memory controllers, atomic cells, other client accelerators, etc.) that can be configured to directly implement or support user instructions to thereby enhance system performance and reduce latency.

[0031] In practice, near-memory computing systems are particularly well-suited for implementing hierarchical structures of instructions and nested loops (e.g., two, three, or more loop depths, or multidimensional loops). Standard compilers can be used to accept high-level language instructions and compile them directly into one or more dataflow architectures within a node. For example, nodes in the system can contain hybrid threading accelerators. These hybrid threading accelerators can execute in the user space of the CNM system and can initiate their own threads or sub-threads that can operate in parallel. Each thread can be mapped to a different loop iteration, thereby supporting multidimensional loops. Leveraging the ability to initiate such nested loops, along with other capabilities, CNM systems can achieve significant time savings and latency improvements for computationally intensive operations.

[0032] Near-memory computing systems, or nodes or components of near-memory computing systems, may include or utilize various memory devices, controllers, and interconnects, etc. In examples, the system may include various interconnect nodes, and nodes or groups of nodes may be implemented using chiplets. Chiplets are an emerging technology for integrating various processing functionalities. Typically, chiplet systems consist of discrete chips (e.g., integrated circuits (ICs) on different substrates or dies) integrated on an interposer and packaged together. This arrangement differs from a single chip (e.g., an IC) containing different blocks of devices (e.g., blocks of intellectual property (IP)) on a substrate (e.g., a single die), such as a system-on-a-chip (SoC), or discrete packaged devices integrated on a board. Generally, chiplets offer manufacturing benefits compared to single die chips, including higher yields or reduced development costs. The following discussion... Figure 6A and Figure 6B Examples typically described include chiplet systems, which may include near-memory computing systems.

[0033] Figure 1 This typically describes a first instance of a near-memory computing system or CNM system 102. An instance of CNM system 102 comprises multiple different memory computing nodes, each of which may contain various near-memory computing devices. Each node in the system can operate within its own operating system (OS) domain (e.g., in particular Linux). In the instance, the nodes may coexist in a common OS domain of CNM system 102.

[0034] Figure 1 Examples include instances of the first memory compute node 104 of the CNM system 102. The CNM system 102 may have multiple nodes, for example, including different instances of the first memory compute node 104 coupled using a scaling structure 106. In the examples, the architecture of the CNM system 102 may support a scale with up to n different memory compute nodes (e.g., n = 4096) using the scaling structure 106. As discussed further below, each node in the CNM system 102 may be a component of multiple devices.

[0035] CNM system 102 may include a global controller for various nodes in the system, or a specific memory compute node in the system may optionally act as a host or controller for one or more other memory compute nodes in the same system. The various nodes in CNM system 102 may therefore be configured similarly or differently.

[0036] In this example, each node in CNM system 102 may include a host system using a specified operating system. The operating system may be common or different across the various nodes in CNM system 102. Figure 1In this example, the first memory computing node 104 includes a host system 108, a first switch 110, and a first memory computing device 112. The host system 108 may include a processor, such as an x86, ARM, RISC-V, or other type of processor. The first switch 110 may be configured to facilitate communication between or between devices of the first memory computing node 104 and devices of the CNM system 102, for example, using a dedicated or other communication protocol (generally referred to herein as the Chip-to-Chip Protocol Interface (CTCPI)). That is, CTCPI may include a dedicated interface unique to the CNM system 102, or may include or use other interfaces, such as a Compute High-Speed ​​Link (CXL) interface, a Peripheral Component Interconnect High-Speed ​​(PCIe) interface, or a Chiplet Protocol Interface (CPI), etc. The first switch 110 may include a switch configured to use CTCPI. For example, the first switch 110 may include a CXL switch, a PCIe switch, a CPI switch, or other types of switches. In this example, the first switch 110 may be configured to couple dissimilarly configured endpoints. For example, the first switch 110 can be configured to convert packet formats between, for example, PCIe and CPI formats.

[0037] The CNM system 102 is described herein with various example configurations (e.g., systems including nodes), and each node may include various chips (e.g., processors, switches, memory devices, etc.). In an example, the first memory computing node 104 in the CNM system 102 may include various chips implemented using chiplets. In the chiplet-based configuration of the CNM system 102 discussed below, inter-chiplet communication and additional intra-system communication may use a CPI network. The CPI network described herein is an example of CTCPI, that is, a chiplet-specific implementation of CTCPI. Therefore, the structure, operation, and functionality described below for CPI are equally applicable to structures, operations, and functions that may be implemented using non-chiplet-based CTCPI implementations. Unless otherwise expressly indicated, any discussion of CPI herein applies equally to CTCPI.

[0038] The CPI interface includes a packet network that supports virtual channels to enable flexible and high-speed interaction between chiplets, such as portions of the first memory compute node 104 or the CNM system 102. CPI enables bridging from intra-chiplet networks to wider chiplet networks. For example, the Advanced Scalable Interface (AXI) is a specification for intra-chip communication. However, the AXI specification covers various physical design options, such as the number of physical channels, signal timing, power, etc. Within a single chip, these options are typically selected to meet design goals, such as power consumption, speed, etc. However, to achieve the flexibility of chiplet-based memory compute systems, adapters using CPI can intersect between various AXI design options that can be implemented in various chiplets. By enabling physical-to-virtual channel mapping using packetization protocols and encapsulating time-based signaling, CPI can be used to bridge intra-chiplet networks (e.g., within a specific memory compute node) across wider chiplet networks (e.g., across the first memory compute node 104 or across the CNM system 102).

[0039] CNM system 102 is scalable to accommodate multi-node configurations. That is, multiple different instances of the first memory compute node 104 or other differently configured memory compute nodes can be coupled using scaling structure 106 to provide a scalable system. Each of the memory compute nodes can run its own operating system and can be configured to jointly coordinate the use of wide system resources.

[0040] exist Figure 1 In one example, a first switch 110 of the first memory compute node 104 is coupled to a scaling structure 106. The scaling structure 106 can provide switches (e.g., CTCPI switches, PCIe switches, CPI switches, or other switches) that facilitate communication among and between different memory compute nodes. In this example, the scaling structure 106 can facilitate communication between various nodes in a partitioned global address space (PGAS).

[0041] In this example, a first switch 110 from a first memory computing node 104 is coupled to one or more different memory computing devices, such as a first memory computing device 112. The first memory computing device 112 may include a chiplet-based architecture, referred to herein as a near-memory computing (CNM) chiplet. A packaged version of the first memory computing device 112 may include, for example, one or more CNM chiplets. The chiplets may be communicatively coupled using CTCPI for high bandwidth and low latency.

[0042] exist Figure 1 In an example, the first memory computing device 112 may include a network on-chip (NOC) or a first NOC 118. Typically, an NOC is an interconnect network within a device that connects a specific set of endpoints. Figure 1 In this context, the first NOC 118 can provide communication and connectivity between various memories, computing resources and ports of the first memory computing device 112.

[0043] In an example, the first NOC 118 may include a folded Clos topology, such as within each instance of a memory computing device, or as a grid coupling multiple memory computing devices in a node. Clos topologies offer various benefits, such as the ability to use multiple lower-radix cross switches to provide functionality associated with higher-radix cross switch topologies. For instance, a Clos topology can present consistent latency and bi-directional bandwidth across the NOC.

[0044] The first NOC 118 can include various types of switches, including hub switches, edge switches, and endpoint switches. Each switch can be configured as a crossbar switch that provides substantially uniform latency and bandwidth between input and output nodes. In an example, endpoint switches and edge switches can contain two separate crossbar switches, one for traffic destined for the hub switch and the other for traffic destined for the hub switch. A hub switch can be configured as a single crossbar switch that switches all inputs to all outputs.

[0045] In this example, hub switches may each have multiple ports (e.g., four or six ports each) depending on whether a particular hub switch participates in inter-chip communication. The number of hub switches participating in inter-chip communication can be set by inter-chip bandwidth requirements.

[0046] The first NOC 118 can support a variety of payloads between computing elements and memory (e.g., 8 to 64-byte payloads; other payload sizes can be used similarly). In an example, the first NOC 118 can be optimized for relatively small payloads (e.g., 8 to 16 bytes) to efficiently handle access to sparse data structures.

[0047] In this example, the first NOC 118 may be coupled to an external host via a first physical layer interface 114, a PCIe slave module 116 or endpoint, and a PCIe master module 126 or root port. That is, the first physical layer interface 114 may include an interface to allow an external host processor to couple to the first memory computing device 112. The external host processor may optionally couple to one or more different memory computing devices, for example, using a PCIe switch or other native protocol switch. Communication with the external host processor via a PCIe-based switch limits device-to-device communication to device-to-device communication supported by the switch. In contrast, communication via a memory computing device native protocol switch, such as using CTCPI, allows for more complete communication between or among different memory computing devices, including support for partitioned global address spaces, such as for generating worker threads and sending events.

[0048] In this example, the CTCPI protocol may be used by a first NOC 118 in a first memory computing device 112, and a first switch 110 may include a CTCPI switch. The CTCPI switch may allow CTCPI packets to be transmitted, for example, from a source memory computing device of the first memory computing device 112 to (e.g., on the same or another node) different destination memory computing devices without being converted to another packet format.

[0049] In one example, the first memory computing device 112 may include an internal host processor 122. The internal host processor 122 may be configured to communicate with the first NOC 118 or other components or modules of the first memory computing device 112, for example, using an internal PCIe master module 126, which can help eliminate the time- and energy-consuming physical layer. In one example, the internal host processor 122 may be based on a RISC-VISA processor and may use a first physical layer interface 114 to communicate externally to the first memory computing device 112, for communication with other storage, networking, or other peripheral devices of the first memory computing device 112. The internal host processor 122 may control the first memory computing device 112 and may act as a proxy for operating system-related functionalities. The internal host processor 122 may include a relatively small number of purpose processing cores (e.g., 2 to 4 cores) and a host memory device 124 (e.g., including DRAM modules).

[0050] In one example, the internal host processor 122 may include a PCI root port. When the internal host processor 122 is in use, one of its root ports may be connected to a PCIe slave module 116. Another root port of the internal host processor 122 may be connected to a first physical layer interface 114 to provide communication with external PCI peripherals. When the internal host processor 122 is deactivated, the PCIe slave module 116 may be coupled to the first physical layer interface 114 to allow an external host processor to communicate with a first NOC 118. In an example of a system with multiple memory computing devices, the first memory computing device 112 may be configured to act as a system host or controller. In this example, the internal host processor 122 may be in use, and other instances of internal host processors in corresponding other memory computing devices may be deactivated.

[0051] The internal host processor 122 can be configured at power-up of the first memory computing device 112 to allow host initialization. In an example, the internal host processor 122 and its associated data paths (e.g., including the first physical layer interface 114, PCIe slave module 116, etc.) can be configured to the first memory computing device 112 from input pins. One or more of these pins can be used to enable or disable the internal host processor 122 and configure the PCI (or other) data paths accordingly.

[0052] In this example, the first NOC 118 may be coupled to scale structure 106 via scale structure interface module 136 and second physical layer interface 138. Scale structure interface module 136 or SIF facilitates communication between the first memory computing device 112 and a device space such as a Partitioned Global Address Space (PGAS). The PGA may be configured such that a particular memory computing device, such as the first memory computing device 112, may access memory or other resources on different memory computing devices (e.g., on the same or different nodes) using a load / store paradigm. Various scalable architecture technologies may be used, including CTCPI, CPI, Gen-Z, PCI, or Ethernet bridged via CXL. Scale structure 106 may be configured to support various packet formats. In this example, scale structure 106 supports out-of-order packet communication or ordered packets, for example, by using path identifiers to extend bandwidth across multiple equivalent paths. Scale structure 106 typically supports remote operations such as remote memory reads, writes, and other built-in atoms, remote memory atoms, remote memory computing device send events, and remote memory computing device call and return operations.

[0053] In this example, the first NOC 118 may be coupled to one or more different memory modules, such as including a first memory device 128. The first memory device 128 may include various types of memory devices, such as LPDDR5 or GDDR6, etc. Figure 1 In one example, the first NOC 118 may coordinate communication with the first memory device 128 via a memory controller 130, which may be dedicated to a specific memory module. In this example, the memory controller 130 may include a memory module cache and an atomic operation module. The atomic operation module may be configured to provide relatively high-volume atomic operators, such as integer and floating-point operators. The atomic operation module may be configured to apply its operators to data within a memory module cache (e.g., including an SRAM memory-side cache), thereby allowing back-to-back atomic operations using the same memory location with minimal volume degradation.

[0054] The memory module cache can provide storage for frequently accessed memory locations, such as eliminating the need to re-access the first memory device 128. In one example, the memory module cache can be configured to cache data only for specific entries of the memory controller 130. In another example, the memory controller 130 includes a DRAM controller configured to interface with, for example, the first memory device 128 containing a DRAM device. The memory controller 130 can provide access scheduling and bit error management, among other functions.

[0055] In this example, the first NOC 118 may be coupled to the hybrid thread processor (HTP 140), the hybrid thread architecture (HTF 142), and the host interface and dispatch module (HIF 120). HIF 120 may be configured to facilitate access to host-based command request and response queues. In this example, HIF 120 may dispatch new execution threads on the processors or compute elements of the HTP 140 or HTF 142. In this example, HIF 120 may be configured to maintain workload balancing across the HTP 140 and HTF 142 modules.

[0056] The hybrid-threaded processor, or HTP 140, may include accelerators, such as those based on the RISC-V instruction set. The HTP 140 may contain a highly threaded event-driven processor, where threads can execute in a single instruction round to maintain high instruction throughput. The HTP 140 includes relatively few custom instructions to support low-overhead threading capabilities, event send / receive, and shared-memory atomic operators.

[0057] The hybrid thread architecture or HTF 142 may include accelerators, such as non-von Neumann coarse-grained configurable processors. The HTF 142 may be optimized for high-level language operations and data types (e.g., integers or floating-point). In an example, the HTF 142 may support dataflow computation. The HTF 142 may be configured to utilize substantially all memory bandwidth available on the first memory computing device 112, for example, when executing a memory-bound computational core.

[0058] The HTP and HTF accelerators of the CNM system 102 can be programmed using various high-level structured programming languages. For example, the HTP and HTF accelerators can be programmed using C / C++ (e.g., using the LLVM compiler framework). The HTP accelerator can, for example, utilize the open-source compiler environment through various additional custom instruction sets configured to improve memory access efficiency, provide message passing mechanisms, and manage events, etc. In an example, the HTF accelerator can be designed to be programmed using a high-level programming language, and the compiler can generate simulator configuration files or binaries that run on the HTF 142 hardware. The HTF 142 provides a mid-level language for precise and concise expression of algorithms while hiding the configuration details of the HTF accelerator itself. In an example, the HTF accelerator toolchain can use an LLVM front-end compiler and an LLVM intermediate representation (IR) to interface with the HTF accelerator back-end.

[0059] Figure 2 An example of a memory subsystem 200 of a memory computing device according to an embodiment is generally described. The example of memory subsystem 200 includes a controller 202, a programmable atom unit 208, and a second NOC 206. The controller 202 may include or use the programmable atom unit 208 to perform operations using information in the memory device 204. In the example, memory subsystem 200 includes components from… Figure 1 A portion of the first memory computing device 112, such as a portion of the first NOC 118 or memory controller 130.

[0060] exist Figure 2 In this example, the second NOC 206 is coupled to the controller 202, and the controller 202 may include a memory control module 210, a local cache module 212, and a built-in atom module 214. In this example, the built-in atom module 214 may be configured to handle relatively simple single-loop integer atoms. The built-in atom module 214 can perform atoms with the same processing power as, for example, a normal memory read or write operation. In this example, an atomic memory operation may include a combination of: storing data into memory, performing an atomic memory operation, and then responding by loading data from memory.

[0061] A local cache module 212, which may include, for example, an SRAM cache, can be provided to help reduce latency for repeatedly accessed memory locations. In an example, the local cache module 212 may provide a read buffer for sub-memory line access. The local cache module 212 may be particularly advantageous for computing elements with relatively small or no data cache.

[0062] For example, a memory control module 210, which may include a DRAM controller, can provide low-level request buffering and scheduling to provide efficient access to memory device 204 (e.g., which may include a DRAM device). In an example, memory device 204 may include or use a GDDR6 DRAM device, for example, with a density of 16 Gb and a peak bandwidth of 64 Gb / s. Other devices may be used similarly.

[0063] In this example, programmable atomic unit 208 may include single-loop or multi-loop operators, such as being configurable to perform integer addition or more complex multi-instruction operations, such as Bloom filter insertion. In this example, programmable atomic unit 208 may be configured to perform load and store-to-memory operations. Programmable atomic unit 208 may be configured to utilize a dedicated instruction set, such as RISC-V ISA, to facilitate interaction with controller 202 to atomically execute user-defined operations.

[0064] Programmable atomic requests received, for example, from a host on or outside the node, can be routed to programmable atomic units 208 via a second NOC 206 and controller 202. In an example, custom atomic operations (e.g., performed by programmable atomic units 208) can be identical to built-in atomic operations (e.g., performed by built-in atomic modules 214), except that the programmable atomic operations can be defined or programmed by a user rather than a system architect. In an example, programmable atomic request packets can be sent to controller 202 via the second NOC 206, and controller 202 can recognize the requests as custom atoms. Controller 202 can then forward the recognized requests to programmable atomic units 208.

[0065] Figure 3 This section typically describes an example of a programmable atom unit 302 for use with a memory controller according to an embodiment. In this example, the programmable atom unit 302 may include or correspond to data from... Figure 2 The instance is a programmable atomic unit 208. That is to say, Figure 3 This describes the components in an example of the programmable atom unit (PAU) 302, such as those described above relative to... Figure 2 (e.g., in programmable atom unit 208) or relative to Figure 1 (For example, in the atomic operation module of memory controller 130) the mentioned components. Figure 3 As described, the programmable atomic unit 302 includes a PAU processor or PAU core 306, a PAU thread control 304, an instruction SRAM 308, a data cache 310, and a memory interface 312 for interfacing with a memory controller 314. In an example, the memory controller 314 includes components from... Figure 2 An instance of controller 202.

[0066] In this example, the PAU core 306 is a pipelined processor, allowing multiple stages of different instructions to execute together per clock cycle. The PAU core 306 may contain a barrel-shaped multithreaded processor, where the thread control 304 circuitry switches between different register files (e.g., a set of registers containing the current processing state) after each clock cycle. This enables efficient context switching between currently executing threads. In this example, the PAU core 306 supports eight threads, resulting in eight register files. In this example, some or all of the register files are not integrated into the PAU core 306, but instead actually reside in the local data cache 310 or instruction SRAM 308. This reduces the circuit complexity of the PAU core 306 by eliminating the traditional flip-flops used for registers in such memories.

[0067] The local PAU memory may contain instruction SRAM 308, such as instructions for various atoms. These instructions include instruction sets to support atomic operators loaded by various applications. When an atomic operator is requested, for example, by an application chiplet, the instruction set corresponding to the atomic operator is executed by the PAU core 306. In one example, the instruction SRAM 308 can be partitioned to establish the instruction set. In this example, a specific programmable atomic operator requested by the request process can be identified by a partition number. The partition number can be established when a programmable atomic operator is registered (e.g., loaded onto) using programmable atomic unit 302. Other metadata for the programmable instructions may be stored in memory (e.g., in a partition table) within the local memory of programmable atomic unit 302.

[0068] In this example, atomic operators manipulate data cache 310, which is typically synchronized (e.g., flushed) when the thread used for the atomic operator completes. Therefore, latency can be reduced for most memory operations during the execution of the programmable atomic operator thread, except for initial loads from external memory such as memory controller 314.

[0069] If a potentially dangerous condition would prevent a memory request, the pipelined processor (e.g., PAU core 306) may experience problems when an execution thread attempts to make such a request. Here, the memory request is for retrieving data from memory controller 314, whether it comes from cache on memory controller 314 or off-die memory. To address this, PAU core 306 is configured to deny memory requests from threads. Typically, PAU core 306 or thread control 304 may contain circuitry for enabling one or more thread rescheduling points in the pipeline. Here, the denial occurs at points in the pipeline outside (e.g., after) these thread rescheduling points. In this instance, the danger occurs outside the rescheduling point. Here, the danger is created by a previous instruction in the thread after the memory request instruction has passed through the last thread rescheduling point before the pipeline stage where the memory request can be made.

[0070] In this example, to reject a memory request, the PAU core 306 is configured to determine (e.g., detect) the presence of a danger on the memory indicated in the memory request. Here, a danger represents any condition that would cause an inconsistent state of the thread if the memory request were allowed (e.g., executed). In this example, the danger is an ongoing memory request. Here, the existence of an ongoing memory request makes it uncertain what the data at that address in the data cache 310 should be, regardless of whether the data cache 310 contains data at the requested memory address. Therefore, the thread must wait for the ongoing memory request to complete before operating on the current data. The danger is cleared when the memory request completes.

[0071] In this example, the danger lies in a dirty cache line in data cache 310 used for the requested memory address. While a dirty cache line typically indicates that the data in the cache is current and the memory controller version of such data is not current, problems can arise with thread instructions that do not operate from the cache. Examples of such instructions use the built-in atomic operators of memory controller 314 or other separate hardware blocks. In the context of the memory controller, the built-in atomic operators may be separate from programmable atomic units 302 and do not access the data cache 310 or instruction SRAM 308 within the PAU. If a cache line is dirty, the built-in atomic operators will not operate on the latest data until data cache 310 is flushed to synchronize the cache with another or off-chip memory. The same situation can occur using other hardware blocks of the memory controller, such as encryption blocks, encoders, etc.

[0072] Figure 4This describes an example of a hybrid-threaded processor (HTP) accelerator or HTP accelerator 400. HTP accelerator 400 may include a portion of a memory computing device according to an embodiment. In this example, HTP accelerator 400 may include or include components from... Figure 1 The HTP 140 is an example of this. The HTP accelerator 400 includes, for example, an HTP core 402, an instruction cache 404, a data cache 406, a translation block 408, a memory interface 410, and a thread controller 412. The HTP accelerator 400 may further include, for example, a dispatch interface 414 and an NOC interface 416 for interfacing with a NOC, the NOC being, for example, from... Figure 1 The first NOC 118 instance, from Figure 2 The second NOC 206 or other NOCs for the example.

[0073] In this example, the HTP accelerator 400 includes modules based on the RISC-V instruction set and may include a relatively small number of other or additional custom instructions for supporting low-overhead, thread-enabled hybrid threading (HT) languages. The HTP accelerator 400 may include a highly threaded processor core, HTP core 402, in which threads can execute in a single instruction round-robin to maintain high instruction throughput. In this example, threads can pause while waiting for other pending events to complete. This allows computational resources to be used efficiently for related work, rather than polling. In this example, multithreaded barrier synchronization can utilize efficient HTP-to-HTP and HTP-to / from host message passing, allowing thousands of threads to initialize or wake up in, for example, tens of clock cycles.

[0074] In an example, dispatch interface 414 may include function blocks for handling hardware-based thread management of HTP accelerator 400. That is, dispatch interface 414 can manage the dispatch of work to HTP core 402 or other accelerators. However, non-HTP accelerators are typically unable to dispatch work. In an example, work dispatched from the host may use dispatch queues residing in, for example, host main memory (e.g., DRAM-based memory). On the other hand, work dispatched from HTP accelerator 400 may use dispatch queues residing in SRAM, for example, within a dispatch for a target HTP accelerator 400 within a specific node.

[0075] In an example, HTP core 402 may include one or more cores that execute instructions on behalf of threads. That is, HTP core 402 may contain instruction processing blocks. HTP core 402 may further include or be coupled to thread controller 412. Thread controller 412 may provide thread control and state for each active thread within HTP core 402. Data cache 406 may include a cache for the host processor (e.g., for local and remote memory computing devices, including caches for HTP core 402), and instruction cache 404 may include a cache for use by HTP core 402. In an example, data cache 406 may be configured for both read and write operations, and instruction cache 404 may be configured for read-only operations.

[0076] In this example, data cache 406 provides a small cache for each hardware thread. Data cache 406 can temporarily store data for use by the owning thread. Data cache 406 can be managed by hardware or software within the HTP accelerator 400. For example, the hardware can be configured to automatically allocate or evict lines as needed when load and store operations are performed by the HTP core 402. Software, for example, using RISC-V instructions, can determine which memory access should be cached and when a line should be invalidated or written back to another memory location.

[0077] Data caching on the HTP accelerator 400 offers various benefits, including making memory controller access more efficient, thus allowing execution threads to avoid stalling. However, there are situations that can lead to inefficiency when using caching. Examples include accesses where data is accessed only once, resulting in cache line thrashing. To help address this issue, the HTP accelerator 400 can use a custom load instruction set to prompt load instructions to check for cache hits and, if a cache miss occurs, issue a memory request for the requested operand without placing the retrieved data in the data cache 406. Therefore, the HTP accelerator 400 includes various types of load instructions, including non-cached and cache line loads. If dirty data exists in the cache, non-cached load instructions use the cached data. Non-cached load instructions ignore clean data in the cache and do not write the accessed data to the data cache. The cache line load instruction can load an entire data cache line (e.g., 64 bytes) from memory into data cache 406, and can also load the addressed memory into a specified register. If clean or dirty data is in data cache 406, these loads can use the cached data. If the referenced memory location is not in data cache 406, the entire cache line can be accessed from memory. The use of cache line load instructions reduces cache misses when referencing sequential memory locations (e.g., in memory copy operations), but it can also waste memory and bandwidth at NOC interface 416 if the referenced memory data is not used.

[0078] In this example, the HTP accelerator 400 includes non-cached custom store instructions. These non-cached store instructions help avoid thrashing the data cache 406 by utilizing write data that is not sequentially written to memory.

[0079] In this example, the HTP accelerator 400 further includes a translation block 408. Translation block 408 may include a virtual-to-physical translation block for the local memory of the memory computing device. For example, a host processor in HTP core 402 may execute load or store instructions, and these instructions may generate virtual addresses. A translation table from translation block 408 may be used, for example, to translate the virtual addresses into physical addresses for the host processor. For example, memory interface 410 may include an interface between HTP core 402 and NOC interface 416.

[0080] Figure 5 Examples of a hybrid threading architecture (HTF) or HTF 500 of a memory computing device according to embodiments are described. In these examples, HTF 500 may include or include components from... Figure 1The HTF 142 is an example of this. The HTF 500 is a coarse-grained configurable computational architecture that can be optimized for high-level language operand types and operators (e.g., using C / C++ or other high-level languages). In an instance, the HTF 500 may contain a configurable n-bit-wide (e.g., 512-bit-wide) data path for interconnect-hardened SIMD arithmetic units.

[0081] In this instance, HTF 500 includes HTF cluster 502, which contains multiple HTF tiles, including instance tile 504 or tile N. Each HTF tile may contain one or more compute elements with local memory and arithmetic capabilities. For example, each tile may contain a compute pipeline with support for integer and floating-point operations. In this instance, data paths, compute elements, and other infrastructure may be implemented as hardened IP to provide maximum performance while minimizing power consumption and reconfiguration time.

[0082] exist Figure 5 In this example, the tiles of HTF cluster 502 are arranged linearly, and each tile in the cluster can be coupled to one or more other tiles in HTF cluster 502. Figure 5 In this example, instance tile 504 or tile N is coupled to four other tiles, including coupling to base tile 510 (e.g., tile N-2) via a port labeled SF IN N-2, coupling to adjacent tile 512 (e.g., tile N-1) via a port labeled SF IN N-1, and coupling to tile N+1 via a port labeled SF IN N+1 and to tile N+2 via a port labeled SF IN N+2. Instance tile 504 may be coupled to the same or other tiles via corresponding output ports (e.g., ports labeled SF OUT N-1, SF OUT N-2, SF OUT N+1, and SF OUT N+2). In this example, the ordered list of names for the various tiles is a conceptual indication of the tile's location. In other examples, tiles, including HTF cluster 502, may be arranged in a grid or other configuration, where each tile is similarly coupled to one or more of its nearest neighbors in the grid. Tiles positioned at the edge of a cluster may optionally have fewer connections to adjacent tiles. For example, tiles N-2 or... Figure 5 In an instance, the base tile 510 may be coupled only to the adjacent tile 512 (tile N-1) and to the instance tile 504 (tile N). Similarly, fewer or additional inter-tile connections may be used.

[0083] HTF cluster 502 may further include a memory interface module, which includes a first memory interface module 506. The memory interface module couples HTF cluster 502 to a NOC, such as a first NOC 118. In an example, the memory interface module may allow tiles within the cluster to request other locations in the memory computing system (e.g., within the same or different nodes in the system). That is, the representation of HTF 500 may include part of a larger structure that can be distributed across multiple nodes, such as having one or more HTF tiles or HTF clusters at each of the nodes. Requests can be made between tiles or nodes within the context of the larger structure.

[0084] exist Figure 5 In this example, a synchronous structure (SF) is used to couple tiles within HTF cluster 502. The synchronous structure provides communication between a specific tile in HTF cluster 502 and its neighboring tiles, as described above. Each HTF cluster 502 may further include an asynchronous structure (AF) that provides communication, for example, between tiles within the cluster, between memory interfaces within the cluster, and between dispatch interfaces 508 within the cluster.

[0085] In this example, the synchronization structure can exchange messages containing data and control information. Control information may include, among other things, instruction RAM address information or thread identifiers. Control information can be used to set the data path, and data message fields can be selected as the source of the path. Typically, control fields can be provided or received earlier, making them available for configuring the data path. For example, to help minimize any delays in the pipeline through the synchronization domain in the tile, control information may arrive at the tile several clock cycles before the data fields. Various registers can be provided to help coordinate the timing of data flow in the pipeline.

[0086] In this example, each tile in HTF cluster 502 may contain multiple memories. Each memory may have the same width as the data path (e.g., 512 bits) and a specified depth, for example, ranging from 512 to 1024 features. The tile memory can be used to store data that supports data path operations. For example, the stored data may contain constants that are part of the cluster configuration of the kernel, or variables that are computed as part of a data stream. In this example, the tile memory may be written from an asynchronous structure as a data transfer from another synchronization domain, or may contain, for example, the result of a load operation initiated by another synchronization domain. Reading from the tile memory can be performed via synchronous data path instructions in the synchronization domain.

[0087] In the example, each tile in the HTF cluster 502 may have a dedicated instruction RAM (INST RAM). In an example of an HTF cluster 502 with sixteen tiles and an instruction RAM entry of sixty-four entries, the cluster may allow the use of up to 1024 multiply-shift and / or ALU operation mapping algorithms. Various tiles may be optionally pipelined together, for example, using a synchronous structure to allow computation with minimal memory access to the data stream, thus minimizing latency and reducing power consumption. In the example, an asynchronous structure may allow memory referencing and computation to continue in parallel, thereby providing a more efficient streaming kernel. In the example, various tiles may include built-in support for loop-based constructions and may support nested loop kernels.

[0088] Synchronization structures allow for the pipelined execution of multiple tiles, eliminating the need for data queues. Tiles participating in a synchronization domain can, for example, act as a single pipelined data path. The first or base tile of the synchronization domain (e.g., tile N-2, in...) Figure 5 In this example, worker threads can be initiated via pipelined tiles. The base tile is responsible for starting work on a predefined cadence, referred to in this document as the SpokeCount. For instance, if the SpokeCount is 3, the base tile can initiate work every two clock cycles.

[0089] In this example, the synchronization domain comprises a set of connected tiles in HTF cluster 502. Thread execution can begin at the base tile of the domain and can proceed from the base tile to other tiles in the same domain via the synchronization structure. The base tile provides instructions to be executed against the first tile. The first tile may by default provide the same instructions for another connected tile to be executed. However, in some instances, the base tile or subsequent tiles may conditionally specify or use alternative instructions. Alternative instructions are selected by causing the tile's data path to produce a Boolean condition value, and then the Boolean value can be used to select between the instruction set of the current tile and the alternative instructions.

[0090] Asynchronous structures can be used to perform operations that occur asynchronously relative to synchronous domains. Each piece in the HTF cluster 502 may contain an interface to an asynchronous structure. The in interface may contain, for example, a FIFO buffer or a queue (e.g., AF IN QUEUE) to provide storage for messages that cannot be processed immediately. Similarly, the out interface of the asynchronous structure may contain a FIFO buffer or a queue (e.g., AF OUT QUEUE) to provide storage for messages that cannot be sent immediately.

[0091] In this example, messages in an asynchronous architecture can be categorized as data messages or control messages. Data messages may contain SIMD-width data values ​​written to tile memory 0 (MEM_0) or memory 1 (MEM_1). Control messages can be configured to control threads to create or release resources or issue external memory references.

[0092] In an HTF cluster 502, tildes can perform various computational operations for HTF. These operations can be executed by configuring the data paths within the tilde. In this example, a tilde contains two function blocks that perform computational operations for the tilde: a multiplication and shift operation block (MS OP) and an arithmetic, logical, and bitwise operation block (ALB OP). Both blocks can be configured to perform pipelined operations, such as multiplication and addition, or shift and addition, etc.

[0093] In one instance, each instance of a memory computing device in the system can have a full set of supported instructions for its operator blocks (e.g., MSOP and ALB OP). In this case, binary compatibility can be achieved across all devices in the system. However, in some instances, it may be helpful to maintain a basic set of functionalities and optional instruction set categories to accommodate various design trade-offs, such as die size. The approach can be similar to how the RISC-V instruction set has a basic set and multiple optional instruction subsets.

[0094] In this example, instance tile 504 may include spoke RAM. The spoke RAM can be used to specify which input (e.g., from four SF tile inputs and a base tile input) is the master input for each clock cycle. The spoke RAM read address input may originate from a counter counting from zero to the spoke count minus one. In this example, different spoke counts can be used for different tiles, for example, within the same HTF cluster 502, to allow the performance of a particular application or instruction set to be determined by several tiles or unique tile instances used in the inner loop. In this example, the spoke RAM may specify when synchronization inputs will be written to the tile memory, for example, using multiple inputs for a particular tile instruction and the time when one of the inputs arrives before the others. The earlier arriving input can be written to the tile memory and can be read later when all inputs are available. In this example, the tile memory may be accessed as a FIFO memory, and the FIFO read and write pointers may be stored in register-based memory areas or structures within the tile memory.

[0095] Figure 6A and Figure 6BThis section typically describes examples of chiplet systems that can be used to implement one or more aspects of CNM system 102. As similarly mentioned above, nodes in CNM system 102 or devices within nodes in CNM system 102 may contain chiplet-based architectures or near-memory computing (CNM) chiplets. Packaged memory computing devices may contain, for example, one, two, or four CNM chiplets. Chipslets may be interconnected using high-bandwidth, low-latency interconnects (e.g., using CPI interfaces). Typically, a chiplet system consists of discrete modules (each referred to as a "chiplet") integrated on an interposer layer and, in many instances, interconnected as needed via one or more established networks to provide the desired functionality to the system. The interposer layer and the contained chiplets may be packaged together to facilitate interconnection with other components of a larger system. Each chiplet may contain one or more individual integrated circuits (ICs) or "chips," which may be combined with discrete circuit components and may be coupled to a corresponding substrate for attachment to the interposer layer. Most or all of the chiplets in the system may be individually configured for communication via established networks.

[0096] Chipsets, configured as individual modules within a system, differ from systems implemented on a single chip containing different blocks of devices (e.g., intellectual property (IP) blocks) on a substrate (e.g., a single die), such as a system-on-a-chip (SoC) or multiple discrete packaged devices integrated on a printed circuit board (PCB). Generally, chiplets offer better performance (e.g., lower power consumption, reduced latency, etc.) than discrete packaged devices, and offer greater manufacturing benefits than a single die chip. These manufacturing benefits may include higher yields or reduced development costs and time.

[0097] A chiplet system may comprise, for example, one or more application (or processor) chiplets and one or more support chiplets. Here, the distinction between application chiplets and support chiplets is merely a reference to possible design scenarios for chiplet systems. Thus, for example, a synthetic vision chiplet system may comprise (by way of example only) application chiplets for generating synthetic vision output, and support chiplets such as memory controller chiplets, sensor interface chiplets, or communication chiplets. In typical use cases, a synthetic vision designer may design the application chiplets and obtain the support chiplets from other sources. Therefore, design costs (e.g., in terms of time or complexity) are reduced by avoiding the design and manufacture of the functionality embodied in the support chiplets.

[0098] Chiplets also support the tight integration of IP blocks that might otherwise be difficult, such as IP blocks manufactured using different processing technologies or with different feature sizes (or utilizing different contact technologies or spacing). Therefore, multiple ICs or IC components with different physical, electrical, or communication characteristics can be assembled in a modular manner to provide a component with the desired functionality. Chiplet systems also facilitate adaptation to the needs of different larger systems that will be incorporated into chiplet systems. In examples, ICs or other components can be optimized for power, speed, or heat generation for specific functions, as might be the case with sensors, and can be integrated with other devices more easily than attempting integration on a single die. Furthermore, by reducing the overall die size, chiplet yields are often higher than those of more complex single-die devices.

[0099] Figure 6A and Figure 6B Examples of chiplet systems according to embodiments are typically illustrated. Figure 6A This is an illustration of a chiplet system 602 mounted on a peripheral board 604, which can be connected to a wider range of computer systems, for example, via peripheral component interconnect (PCIe). The chiplet system 602 includes a package substrate 606, an interposer 608, and four chips: an application chiplet 610, a host interface chiplet 612, a memory controller chiplet 614, and a memory device chiplet 616. Other systems may include numerous additional chipsets to provide additional functionality, as will be apparent from the following discussion. The package of the chiplet system 602 is illustrated with a cap or cover plate 618, but other packaging techniques and structures used for chiplet systems may be used. Figure 6B This is a block diagram illustrating the components in a chiplet system for clarity.

[0100] Application chip 610 is described as a chip system NOC 620 that includes a chip network 622 for supporting inter-chip communication. In an example embodiment, the chip system NOC 620 may be included on application chip 610. In an example, the chip network 620 may be defined in response to selected supporting chips (e.g., host interface chip 612, memory controller chip 614, and memory device chip 616). Figure 1 The first NOC 118 in this example allows designers to choose the appropriate number of chiplet network connections or switches for the chiplet system NOC 620. In this example, the chiplet system NOC 620 may reside on a single chiplet or within the interposer 608. In the example discussed herein, the chiplet system NOC 620 implements a chiplet protocol interface (CPI) network.

[0101] In this example, the chiplet system 602 may include or comprise a portion of the first memory computing node 104 or the first memory computing device 112. That is, various blocks or components of the first memory computing device 112 may include chiplets that can be mounted on the peripheral board 604, the package substrate 606, and the interposer 608. The interface components of the first memory computing device 112 may typically include a host interface chiplet 612, the memory and memory control-related components of the first memory computing device 112 may typically include a memory controller chiplet 614, and the various accelerator and processor components of the first memory computing device 112 may typically include an application chiplet 610 or examples thereof.

[0102] For example, the CPI interface, which can be used for communication between or within chiplets in a system, is a packet network that supports virtual channels to enable flexible and high-speed interaction between chiplets. CPI bridges the chiplet-internal network to chiplet network 622. For instance, the Advanced Extensible Interface (AXI) is a widely used specification for designing intra-chip communication. However, the AXI specification covers a large number of physical design options, such as the number of physical channels, signal timing, and power. Within a single chip, these options are typically selected to meet design goals, such as power consumption and speed. However, to achieve flexibility in chiplet systems, adapters such as CPI are used to intersect between various AXI design options that can be implemented in various chiplets. By implementing a mapping from physical channels to virtual channels and encapsulating time-based signaling using packetization protocols, CPI bridges the chiplet-internal network across chiplet network 622.

[0103] CPI can utilize various physical layers to transmit packets. A physical layer may contain simple conductive connections, or it may contain drivers to increase voltage, or otherwise facilitate signal transmission over longer distances. Examples of such physical layers may include an Advanced Interface Bus (AIB), which in various instances can be implemented within the intermediate layer 608. The AIB uses source-synchronous data transfer with a forwarding clock to transmit and receive data. Packets are transmitted across the AIB at Single Data Rate (SDR) or Double Data Rate (DDR) relative to the transmitted clock. The AIB supports various channel widths. Channels can be configured with a symmetrical number of transmit (TX) and receive (RX) inputs / outputs (I / O), or an asymmetrical number of transmitters and receivers (e.g., all transmitters or all receivers). A channel may act as the AIB body or slave depending on which chip provides the master clock. The AIB I / O unit supports three clock modes: asynchronous (i.e., non-timing), SDR, and DDR. In various instances, the non-timing mode is used for clocking and some control signals. SDR mode can use a dedicated SDR-only I / O unit or a dual-purpose SDR / DDR I / O unit.

[0104] In this example, CPI packet protocols (e.g., point-to-point or routable) can use symmetrical receive and transmit I / O units within an AIB channel. CPI streaming protocols allow for more flexible utilization of AIB I / O units. In this example, a streaming AIB channel can be configured with I / O units as all TX, all RX, or half TX and half RX. CPI packet protocols can use AIB channels in SDR or DDR operating modes. In this example, AIB channels are configured in increments of 80 I / O units (i.e., 40 TX and 40 RX) for SDR mode and in increments of 40 I / O units for DDR mode. CPI streaming protocols can use AIB channels in SDR or DDR operating modes. Here, in this example, AIB channels are configured in increments of 40 I / O units for both SDR and DDR modes. In this example, a unique interface identifier is assigned to each AIB channel. This identifier is used during CPI reset and initialization to determine paired AIB channels across neighboring chiplets. In this example, the interface identifier is a 20-bit value comprising a seven-bit chiplet identifier, a seven-bit column identifier, and a six-bit link identifier. The AIB physical layer uses an AIB out-of-band shift register to transmit the interface identifier. Bits 32 to 51 of the shift register are used to transmit the 20-bit interface identifier in both directions across the AIB interface.

[0105] AIB defines a stacked group of AIB channels as an AIB channel column. An AIB channel column has a certain number of AIB channels, plus auxiliary channels. The auxiliary channels contain signals used for AIB initialization. All AIB channels within a column (except for the auxiliary channels) have the same configuration (e.g., all TX, all RX, or half TX and half RX, and the same number of data I / O signals). In this example, AIB channels are numbered sequentially in ascending order, starting with the AIB channel adjacent to the AUX channel. The AIB channel adjacent to the AUX channel is defined as AIB channel zero.

[0106] Typically, the CPI interface on an individual chiplet may include serialization-deserialization (SERDES) hardware. SERDES interconnects are well-suited for scenarios requiring high-speed signaling and low signal counts. However, SERDES can introduce additional power consumption and longer latency for multiplexing and demultiplexing, error detection or correction (e.g., using block-level cyclic redundancy check (CRC)), link-level retries, or forward error correction. However, when low latency or power consumption is a primary concern for ultra-short-distance chiplet-to-chiplet interconnects, parallel interfaces that allow data transmission with minimal latency can be utilized. CPIs contain elements designed to minimize both latency and power consumption in these ultra-short-distance chiplet interconnects.

[0107] For flow control, CPI employs a credit-based technique. For example, the receiver side of chip 610, or the transmitter side of memory controller chip 614, provides credits indicating available buffers. In this example, the CPI receiver contains buffers for each virtual channel for a given transmit time unit. Therefore, if the CPI receiver supports five messages and a single virtual channel in time, the receiver has five buffers arranged in five rows (e.g., one row per unit time). If four virtual channels are supported, the receiver has twenty buffers arranged in five rows. Each buffer holds the payload of one CPI packet.

[0108] When a sender transmits to a receiver, the sender decrements its available credits based on the transmission. Once the receiver has exhausted all its credits, the sender stops sending packets to the receiver. This ensures that the receiver always has an available buffer to store transmissions.

[0109] When the receiver processes the received packet and releases the buffer, it sends the available buffer space back to the sender. The sender can then use this credit return to allow the transmission of additional information.

[0110] Figure 6A An example is a chiplet mesh network 624 that uses direct chiplet-to-chiplet technology without requiring a chiplet system NOC 620. The chiplet mesh network 624 can be implemented in a CPI or another chiplet-to-chiplet protocol. The chiplet mesh network 624 typically implements a chiplet pipeline, where one chiplet acts as an interface to the pipeline, while other chipslets in the pipeline interface only interface with themselves.

[0111] Alternatively, a dedicated device interface can be used to connect devices to the chiplet, such as one or more industry-standard memory interfaces (e.g., synchronous memory interfaces, such as DDR5, DDR6). Connections from the chiplet system or individual chiplets to external devices (e.g., larger systems) can be made via a desired interface (e.g., a PCIe interface). In an example, such an external interface can be implemented via a host interface chiplet 612, which, in the depicted example, provides a PCIe interface external to the chiplet system. This type of interface is typically used when industry practice or standards have converged on such a dedicated chiplet interface 626. The illustrated example of connecting a memory controller chiplet 614 to the dual data rate (DDR) interface of a dynamic random access memory (DRAM) memory device chiplet 616 is such an industry practice.

[0112] Among the various possible supporting chiplets, the memory controller chiplet 614 is likely to be present in chiplet systems due to the ubiquitous use of storage devices for computer processing and the current level of technological sophistication of memory devices. Therefore, using the memory device chiplet 616 and the memory controller chiplet 614, produced through other technologies, allows chiplet system designers to obtain robust products manufactured by established companies. Typically, the memory controller chiplet 614 provides a memory device-specific interface for reading, writing, or erasing data. Typically, the memory controller chiplet 614 can provide additional functionality such as error detection, error correction, maintenance operations, or atomic operator execution. For some types of memory, maintenance operations are often specific to the memory device chiplet 616, such as garbage collection in NAND flash or storage-class memory, or temperature regulation (e.g., cross-temperature management) in NAND flash memory. In instances, maintenance operations may involve logic-to-physical (L2P) mapping or management to provide an indirection hierarchy between the physical and logical representations of data. In other types of memory, such as DRAM, some memory operations, such as refresh, may be controlled at some times by the host processor or memory controller, and at other times by the DRAM memory device or logic associated with one or more DRAM devices, such as interface chips (in this example, buffers).

[0113] Atomic operators are data manipulations that can be performed, for example, by the memory controller chiplet 614. In other chiplet systems, atomic operators can be performed by other chipsets. For instance, the application chiplet 610 can specify an "increment" atomic operator in a command that includes a memory address and may include an increment value. Upon receiving the command, the memory controller chiplet 614 retrieves a number from the specified memory address, increments the number by the amount specified in the command, and stores the result. Upon successful completion, the memory controller chiplet 614 provides the application chiplet 610 with an indication that the command was successful. Atomic operators avoid transferring data across the chiplet mesh network 624, thereby reducing latency in executing such commands.

[0114] Atomic operators can be categorized into built-in atoms or programmable (e.g., custom) atoms. Built-in atoms are a limited set of operations that are implemented immutably in the hardware. Programmable atoms are small programs that can be executed on programmable atom units (PAUs) (e.g., custom atom units (CAUs)) of the memory controller chip 614.

[0115] The memory device chip 616 may be or contain any combination of volatile memory devices or non-volatile memory. Examples of volatile memory devices include, but are not limited to, random access memory (RAM), such as DRAM, synchronous DRAM (SDRAM), graphics dual data rate type 6 SDRAM (GDDR6 SDRAM), and so on. Examples of non-volatile memory devices include, but are not limited to, NAND flash memory, memory-class memory (e.g., phase-change memory or memristor-based technology), ferroelectric RAM (FeRAM), and so on. The illustrated example includes the memory device chip 616 as a chip; however, the device may reside elsewhere, such as in different packages on peripheral board 604. For many applications, multiple memory device chips may be provided. In examples, these memory device chips may each implement one or more memory technologies and may contain an integrated computing host. In examples, the memory chip may contain multiple stacked memory dies of different technologies, such as one or more static random access memory (SRAM) devices stacked with or otherwise communicating with one or more dynamic random access memory (DRAM) devices. In this example, the memory controller chip 614 can be used to coordinate the operation between multiple memory chips in the chiplet system 602, for example, using one or more memory chips in one or more tiers of cache memory, and using one or more additional memory chips as main memory. The chiplet system 602 may include multiple examples of the memory controller chip 614, which can be used to provide memory control functions for individual hosts, processors, sensors, networks, etc. For example, the chiplet architecture in the illustrated system offers advantages in allowing adaptation to different memory storage technologies; and provides different memory interfaces through updated chiplet configurations, for example, without requiring redesign of the rest of the system architecture.

[0116] Figure 7 Examples of chiplet-based implementations for a memory computing device according to embodiments are typically described. Examples include implementations having four near-memory computing (CNM) chiplets, each of which may include or include components from… Figure 1 This refers to a portion of the first memory computing device 112 or the first memory computing node 104. Various portions may contain or include corresponding chiplets. Chiplet-based implementations may include or utilize CPI-based intra-system communication, as described above in [the context of...]. Figure 6A and Figure 6B The example chip system 602 is discussed similarly.

[0117] Figure 7An example includes a first CNM package 700, which comprises a plurality of chiplets. The first CNM package 700 includes a first chiplet 702, a second chiplet 704, a third chiplet 706, and a fourth chiplet 708 coupled to a CNM NOC hub 710. Each of the first to fourth chiplets may include instances of the same or substantially the same components or modules. For example, each chiplet may each include instances of an HTP accelerator, an HTF accelerator, and a memory controller for accessing internal or external memory.

[0118] exist Figure 7 In one example, the first chiplet 702 includes a first NOC hub edge 714 coupled to the CNM NOC hub 710. Other chipslets in the first CNM package 700 similarly include NOC hub edges or endpoints. Switches in the NOC hub edges facilitate communication within the chiplet or within the chiplet system via the CNM NOC hub 710.

[0119] The first chiplet 702 may further include one or more memory controllers 716. The memory controllers 716 may correspond to different NOC endpoint switches that interface with the first NOC hub edge 714. In examples, the memory controllers 716 may include a memory controller chiplet 614 or a memory controller 130, or a memory subsystem 200 or other memory computing implementations. The memory controllers 716 may be coupled to different memory devices, such as a first external memory module 712a or a second external memory module 712b. The external memory modules may include, for example, GDDR6 memory that is selectively accessible by the corresponding different chiplets in the system.

[0120] The first chiplet 702 may further include, for example, a first HTP chiplet 718 and a second HTP chiplet 720 coupled to the edge 714 of the first NOC hub via a corresponding different NOC endpoint switch. The HTP chipsets may correspond to an HTP accelerator, for example, from... Figure 1 The instance of HTP 140 or from Figure 4 An example of an HTP accelerator 400. The HTP chiplet can communicate with the HTF chiplet 722. The HTF chiplet 722 can correspond to an HTF accelerator, for example, from... Figure 1 The instance of HTF 142 or from Figure 5 The HTF 500 instance.

[0121] The CNM NOC hub 710 can be coupled to other chiplets or other CNM packages via various interfaces and switches. For example, the CNM NOC hub 710 can be coupled to a CPI interface via multiple different NOC endpoints on the first CNM package 700. Each of the multiple different NOC endpoints can be coupled to a different node, for example, outside the first CNM package 700. In an example, the CNM NOC hub 710 can be coupled to other peripheral devices, nodes, or devices using CTCPI or other non-CPI protocols. For example, the first CNM package 700 may include a PCIe scale interface (PCIe / SFI) or a CXL interface (CXL) configured to interface the first CNM package 700 with other devices. In an example, devices coupled to the first CNM package 700 using various CPI, PCIe, CXL, or other structures can form a common global address space.

[0122] exist Figure 7 In this example, the first CNM package 700 includes a host interface 724 (HIF) and a host processor (R5). The host interface 724 may correspond to, for example, a source... Figure 1 The instance is HIF 120. The host processor or R5 can correspond to the one from... Figure 1 The instance includes an internal host processor 122. The host interface 724 may include a PCI interface for coupling the first CNM package 700 to other external devices or systems. In this instance, work can be initiated via the host interface 724 on the first CNM package 700 or on a chip cluster within the first CNM package 700. For example, the host interface 724 may be configured to command individual HTF chip clusters (e.g., within various chiplets in the first CNM package 700) to enter and exit power / clock gate modes.

[0123] Figure 8 This describes an example tile of a memory computing device according to an embodiment. Figure 8 In the example, the tiled chiplet instance 800 contains instances of four different near-memory computing clusters of chiplets, where the clusters are coupled together. Each instance of a near-memory computing chiplet may itself contain one or more constituent chips (e.g., host processor chiplets, memory device chiplets, interface chiplets, etc.).

[0124] The 800 tiled chip instances contain data from... Figure 7An example of the first CNM package 700 is one or more of its near-memory computing (CNM) clusters. For example, a tiled chiplet instance 800 may include a first CNM cluster 802, which includes a first chiplet 810 (e.g., corresponding to the first chiplet 702), a second chiplet 812 (e.g., corresponding to the second chiplet 704), a third chiplet 814 (e.g., corresponding to the third chiplet 706), and a fourth chiplet 816 (e.g., corresponding to the fourth chiplet 708). The chipsets in the first CNM cluster 802 may be coupled to a common NOC hub, which may in turn be coupled to NOC hubs in one or more adjacent clusters (e.g., in the second CNM cluster 804 or the fourth CNM cluster 808).

[0125] exist Figure 8 In this example, the tiled chiplet instance 800 includes a first CNM cluster 802, a second CNM cluster 804, a third CNM cluster 806, and a fourth CNM cluster 808. Various different CNM chipsets can be configured in a common address space, allowing chipsets to be allocated and share resources across different tiles. In this example, chipsets within a cluster can communicate with each other. For instance, the first CNM cluster 802 can be communicatively coupled to the second CNM cluster 804 via a chiplet-to-chiplet CPI interface 818, and the first CNM cluster 802 can be communicatively coupled to the fourth CNM cluster 808 via another or the same CPI interface. The second CNM cluster 804 can be communicatively coupled to the third CNM cluster 806 via the same or another CPI interface, and so on.

[0126] In the example, one of the near-memory computing chiplets in the tiled chiplet instance 800 may contain a host interface (e.g., corresponding to a source from...). Figure 7 The host interface 724 of the instance is responsible for workload balancing across the tiled chiplet instances 800. The host interface can facilitate, for example, access to host-based command request queues and response queues from outside the tiled chiplet instances 800. The host interface can dispatch new execution threads using hybrid thread processors and hybrid thread architectures in one or more near-memory computing chips within the tiled chiplet instances 800.

[0127] The inventors have particularly recognized that the problem to be solved includes, for example, reducing latency in a computing system when performing memory accesses. The problem may further include efficiently using a breadth-first search (BFS) algorithm across a large number of nodes, where each node may contain a local memory storage device accessible by the same node in the system or by one or more other nodes.

[0128] The inventors have recognized that solutions to latency problems can include or utilize a cache for each node. For example, in the BFS algorithm, the boundary of a vertex with a relatively high number of times or the most "counts" is more likely to be accessed during the search. To improve performance, the bottom-up (BU) phase of the algorithm can use a local cache to store the results of remote accesses to the boundary information corresponding to the highest-count vertex. In an example, a software-implemented cache, such as 1024 × 64-bit words per memory computing device in the system, can exist. The cache may be empty at the start of the BU iteration. At the first node, a first access to one of the high-count features during execution can read the feature from a remote location (i.e., a location far from the first node), and then store the feature in the local cache at the first node. In this way, subsequent accesses to the same remote location from the first node or the first memory computing device can be satisfied alternatively by the local cache of the first node, and remote access activities that cause latency can be reduced or eliminated.

[0129] The inventors have recognized that latency in systems using the BFS algorithm on many nodes can be reduced by organizing requests around node-local requests. Significant differences in bandwidth utilization can be achieved by utilizing the data layout of the BFS algorithm's data structure and establishing a relatively small saturation cache for each node (e.g., for the first 1k requests). A relatively small saturation cache can help, for example, convert many remote requests into local requests without requiring a complete reload on each iteration.

[0130] Various benefits can be associated with the use of a saturated cache. By caching the boundary bits or boundary information of remote nodes frequently, future requests become local and have a relatively high hit rate. A saturated design can be relatively simple, as, for example, only the first few words (e.g., 1024 words) received from adjacent memory computing devices are typed into the cache. In some cases, the cache can be static, i.e., without an eviction policy, thus avoiding the expenditure of execution time on cache balancing. Furthermore, given a given cache structure, there is no need to reload all boundary information at the start of execution. This can help save computational resources and time by triggering the BU termination condition earlier (e.g., before fetching all boundary information).

[0131] Breadth-first search, or BFS, can represent a data-intensive workload that traverses a computational graph to identify nodes and connectivity levels from the starting source node. In the context of CNM system 102, the graph may include various first-memory computational nodes 104 as vertices and a scale structure 106 between nodes as edges in the graph.

[0132] The challenges presented by BFS include a particularly increasing ratio of remote to local memory accesses that grows proportionally with the number of nodes used, leading to a corresponding increase in average latency and network processing requirements. Consequently, traditional clusters exhibit poor scalability with increasing node counts. Furthermore, using numerous parallel threads to mask latency requires significant system processing power. Additionally, many memory references may exist for the same location (hotspot patterns). Memory access patterns can be data-dependent, and data references can be random, rendering data prefetching relatively useless. Moreover, the balanced distribution of work can be challenging due to large variations in work units coupled with inaccurate estimates of the work to be performed.

[0133] Solutions to these and other challenges may include using a hybrid BFS implementation. Various data structures can be used in a hybrid BFS implementation that employs both bottom-up and top-down approaches. In an example, a proximity list can be used to represent the search graph. For each vertex in the graph (e.g., corresponding to a specific node in CNM system 102), the corresponding neighbor index list may contain the number of vertices to which it is connected. During setup, the number of neighbors for each vertex can be calculated and stored in a count list. The count list can thus represent the connectivity of nodes in the graph. A parent list may contain the results of the search. For example, the parent of a source vertex is itself, and it is the only vertex at level 0 of the search. For subsequent levels, and for each vertex where the parent value has not been previously updated, if there are one or more neighbors that have been updated in previous levels, then the parent value is updated to contain the number of vertices with one of those neighbors. In an example, a boundary bitmap may contain information about whether a parent of a given vertex was found during the search of previous levels. In the example, the boundary bitmap can be read only for the duration of the bottom-up iteration of the search.

[0134] Figure 9 This typically illustrates examples of data structures distributed across different memory computing devices. For example, Figure 9 The demonstration may include multiple different nodes as part of an N-node near-memory computing system (e.g., CNM system 102). Figure 9 It includes a first node 902 (Node_0), a second node 904 (Node_1), and an Nth node 906 (Node_N-1). Fewer or additional nodes can be used similarly. Nodes can be configured to communicate using, for example, a scale structure 106, to satisfy read and write requests originating from one of the nodes in the system.

[0135] In this example, due to the distributed nature of computation on a node cluster, for example, using the CNM system 102, the data can be determined according to... Figure 9 The elements are placed or distributed according to their representation in the system. The allocation of the adjacent index list, count list, parent object list, and boundaries can be uniformly distributed across all nodes in the system as piecewise vectors. A piecewise vector can be a single vector and can be accessed as a normal vector for convenience. However, a piecewise vector can be divided into equal-sized segments based on the number of nodes in the system, and each segment can be physically allocated across different nodes. A series of segment pointers can be used to access each allocated segment.

[0136] Access to elements of a segmented vector from a processing core that also locates segments is referred to as local access in this paper. Otherwise, access to elements from another remote node is referred to as remote access in this paper. Since the latency and requirements of placing elements on the memory system by local access are less than those of placing elements on the memory system by remote access, minimizing the number of remote accesses can be a primary consideration when designing efficient search algorithms.

[0137] Figure 9 Examples illustrate various local and remote access methods. For instance, a first pointer on first node 902 may point to a portion of the parent segment on first node 902, and a second pointer on first node 902 may point to a portion of the parent segment on second node 904 or Nth node 906. Similarly, a pointer on second node 904 may point to a portion of the node's local or remote parent segment (e.g., on first node 902, Nth node 906, or elsewhere in CNM system 102).

[0138] The number of vertices or nodes, or the number of edges connected to a vertex, can be used to estimate the amount of work associated with a particular vertex. To achieve a balanced workload across all computing elements in the system, vertices can be assigned to specific nodes such that the sum of the vertex counts assigned to each node is approximately equal.

[0139] Compared to the top-down algorithm, which attempts to add the new vertex to the boundary bitmap by processing each element in a list of neighbors associated with elements found at previous levels, the bottom-up algorithm considers each vertex based on whether a parent object was found in a previous level. The bottom-up algorithm processes its list of neighbors, searching for vertices whose parent objects were found at previous levels.

[0140] The boundary bitmap indicates vertices where a parent object was found during retrieval at all previous levels. Vertices where no parent object was found have a corresponding bit in the boundary bitmap set to 1. For each of these vertices, each of its neighbors is considered until a neighbor whose corresponding boundary bit is 0 or whose corresponding value in the boundary bitmap is 0 is found. When such a neighbor is found, the parent segment can be updated by setting the current vertex's parent object to the neighbor's ID. The next boundary or one or more neighboring nodes can be updated by setting the current vertex ID bit to 0. A post-zero count instruction can be used to process the list to skip consecutive unmarked nodes.

[0141] During bottom-up processing, for example, each node can handle parent, access counts, and neighbor list accesses for vertices that are local to the same node, and can perform remote accesses for vertices that have not previously found a parent or for the boundaries of each of the node's neighbors. Therefore, the number of remote accesses can be minimized to improve search performance. Figure 10 This typically describes an example of a data structure distributed across different memory computing devices, each of which contains a saturated cache. Figure 10 It may include multiple different nodes that may be part of an N-node near-memory computing system (e.g., CNM system 102). Figure 10 It includes a first node 1002 (Node_0), a second node 1004 (Node_1), and an Nth node 1006 (Node_N-1). Fewer or additional nodes can be used similarly. Nodes can be configured to communicate using, for example, a scale structure 106, to satisfy read and write requests originating from various nodes in the system.

[0142] Figure 10 The corresponding nodes in the instance can be similarly configured as Figure 9 Nodes within an instance. That is, each node can be configured to communicate with other nodes in the system, and each node can have or use vector-based memory. Unlike from... Figure 9 Instance nodes, Figure 10 Each of the instance nodes contains a corresponding local software cache. For example, the first node 1002 contains a first node local cache 1008, the second node 1004 contains a second node local cache 1010, and the Nth node 1006 contains an Nth node local cache 1012. The different node local caches can be software-implemented saturation caches. In this example, the caches can be filled during the bottom-up phase of the BFS algorithm. For instance, each of the different node local caches can be filled with different information depending on the specific read request issued by each of the respective different nodes.

[0143] In this example, each of the first node local cache 1008, the second node local cache 1010, and the Nth node local cache 1012 is the same size. Each cache can be configured to have N different blocks corresponding to N different nodes in a system, such as CNM system 102. In this example, each block can be configured to store 1024 × 64-bit words, and each block can be filled with information received from the respective different nodes.

[0144] Various cache blocks can be populated during the initial remote access. For example, information received from one or more segments on second node 1004 (e.g., including 1024×64 bits) can be used to populate a first portion of the first node's local cache 1008 when it is first accessed by first node 1002. Subsequent requests for the same information from first node 1002 can be retrieved from the first portion of the first node's local cache 1008 (e.g., not from the location at second node 1004). Similarly, other information received from one or more segments on Nth node 1006 can be used to populate a second portion of the first node's local cache 1008 when it is first accessed by first node 1002. Subsequent requests for second information from first node 1002 can be retrieved from the second portion of the first node's local cache 1008, and so on.

[0145] Figure 11 This typically describes instances that map boundary information from different memory computing devices to corresponding saturated caches. Instances include, for example, those corresponding to... Figure 10 This example represents the first node local cache 1008 (cache_0), the second node local cache 1010 (cache_1), and the Nth node local cache 1012 (cache_N-1). Each of the node local caches may include N different cache blocks, where N corresponds to the number of nodes in the system containing the first node 1002, the second node 1004, and the Nth node 1006. In this example, each cache block may be the same size. In other examples, the caches may be of different sizes.

[0146] Typically, boundary information corresponding to the vertex with the highest count is most likely to be accessed during the search. To improve performance, bottom-up algorithms can use node-local caches to store the results of remote boundary accesses corresponding to the highest-count vertex. In an example, the node-local cache may include N blocks corresponding to N different nodes in the system, and each block can receive up to 1024 × 64 bits.

[0147] In an example, the cache of a specific node, such as the first node 1002, may be empty at the initialization of a bottom-up search initiated by computing resources on the first node 1002. A first access to one of the high-order features may read the target information from its remote location (i.e., at a node outside the first node 1002, such as the second node 1004, the Nth node 1006, or another node in the CNM system 102), and then store the target information in the first node's local cache 1008. Subsequent requests for the same target information from the first node 1002 may be satisfied by the first node's local cache 1008, thus excluding other remote accesses.

[0148] In this example, the cache can be a saturated cache because not every single feature has the opportunity to be cached. Instead, only specific features (e.g., the first 1024 features or words at each remote boundary) can be cached on reads. Features further or deeper within the boundary (e.g., those with indices greater than 1024) can bypass the cache. This solution helps ensure that larger features are eventually cached without incurring computation cycles while maintaining the caching protocol.

[0149] Figure 11 Examples are illustrated graphically to show how elements from a particular node can be stored or cached across the same or other nodes in the system. For instance, a first node 1002 may include a first node memory device 1110, a second node 1004 may include a second node memory device 1112, and an Nth node 1006 may include an Nth node memory device 1114. Each of the different memory devices may correspond to a different example of, for example, a first memory device 128, which comes from... Figure 1 An example of a first memory computing device 112. Different memory devices may store correspondingly different elements or bits, which may be used by computing resources on the same or different nodes. For example, a first node memory device 1110 may include a first element 1116, which may be used by a first node 1002, a second node 1004, or other nodes in the CNM system 102. The first node memory device 1110 may include information other than the first element 1116. A second node memory device 1112 may include a second element 1118, which may be used by a first node 1002, a second node 1004, or other nodes in the CNM system 102, and so on. Figure 11 In one example, the Nth node memory device 1114 includes a third element 1120 stored in memory at the Nth node 1006.

[0150] exist Figure 11In an example, different node-local caches may include corresponding different blocks, such as N different blocks for storing information from N different nodes. For instance, the first node-local cache 1008 or cache_0 may contain a first block 1102a, a second block 1104a, a third block 1106a, and an Nth block 1108a, which, for example, correspond to each of the N different nodes in a memory computing system containing N nodes. Figure 11 In one example, the first block 1102a may be configured to store a first element 1116 from the first node memory device 1110 (e.g., from the first node 1002). The first element 1116 may represent first information (e.g., including boundary bits) accessed from memory by the first node 1002 and stored on the first node 1002. In one example, the size of the first block 1102a (or one or more other blocks) may be the same as the size of the element to be cached or stored in the first block 1102a. For example, the first element 1116 may include 1024 elements, and the first block 1102a may be correspondingly sized to store 1024 elements.

[0151] In this example, during BFS processing, such as during the bottom-up phase of BFS processing, the first element 1116 can be used to populate the first block 1102a. The second block 1104a can be configured to store the second element 1118 from the second node's memory device 1112 (e.g., received from the second node 1004). The second element 1118 may represent first information (e.g., boundary bits) accessed from memory by the first node 1002 stored on the second node 1004, and the second element 1118 may represent less than all the information in the second node's memory device 1112. Subsequent requests for the second element 1118 from the first node 1002 can be satisfied using the first node's local cache 1008 or cache_0 (e.g., using information at the second block 1104a).

[0152] exist Figure 11 In this example, the same information (e.g., boundary bits) from the first node memory device 1110 can be used to fill the first block 1102a on the first node local cache 1008, and the first block 1102b on the second node local cache 1010, and the first block 1102c on the Nth node local cache 1012. That is, the same information, for example, from the first element 1116 can be used to satisfy the initial requests from each of the different nodes.

[0153] In other instances, different information can be used to populate corresponding cache blocks on different nodes, such as boundary information corresponding to these specific nodes. For example, the Nth node memory device 1114 may include a third element 1120. The boundary information relative to the first node 1002 may include the third element 1120, and therefore, the third element 1120 may be used to populate the Nth block 1108a of the first node local cache 1008 when such an element is first accessed by the first node 1002. The boundary information relative to the second node 1004 may include the third element 1120, and therefore, the third element 1120 may be used to populate the Nth block 1108b of the second node local cache 1010. In one instance, the Nth block 1108c of the third Nth node local cache 1012 may be empty or unused, for example, unless or until the third Nth node 1006 accesses information from the Nth node memory device 1114, such as information corresponding to the third element 1120.

[0154] A specific node's local cache can be populated with less than all of the feature or boundary information from a particular memory device. In a representative example, the first node memory device 1110 may include a first feature 1116, for example, a 1024×64-bit word. During a particular iteration of the search, and depending on the connectivity details used for that particular iteration, some, but not all, of the 1024×64-bit word from the first feature 1116 may be cached at the node responsible for the search. For example, a second node 1004 may perform a search that returns less than all of the first feature 1116, and the returned features may be stored in the first block 1102b of the second node's local cache 1010 (cache_1). Since the first block 1102b can be sized to receive up to 1024×64-bit words, a portion of the first block 1102b may be unused. Similarly, the third node 1006 can perform a search that returns fewer than all of the first elements 1116, and the returned elements can be stored in the first block 1102c of the local cache 1012 (cache_N-1) of the Nth node. The returned elements to each of the corresponding local caches of different nodes may or may not be the same elements.

[0155] Figure 12 The description typically includes a first method 1200 for building and populating a node-based saturation cache. The first method 1200 can be executed using multiple nodes in a near-memory computing system, for example, using multiple different instances of a first memory computing device 112 in a CNM system 102.

[0156] At step 1202, the first method 1200 may include establishing a first N-block cache at a first memory computing node in a memory computing system, and the system may include N nodes. For example, step 1202 may include establishing a first node-local cache 1008 at a first node 1002, and the first node 1002 may include one of N different nodes in the CNM system 102. Step 1202 may include establishing N blocks in the cache, and the size of the blocks may be equal or unequal. In an example, each of the N blocks may be configured to hold approximately 1024 different elements or words, which may, for example, correspond to information received from N (or N-1) corresponding different nodes in the system. In an example, step 1202 may include establishing a first block (e.g., first block 1102a) in the N-block cache to store boundary information from the first node 1002, establishing a second block (e.g., second block 1104a) in the N-block cache to store boundary information from the second node 1004, and so on.

[0157] At step 1204, the first method 1200 may include providing a first read request for first data from a first memory computing node. The first read request may be provided to a specific one of N different nodes in the system. For example, the first data may be stored in a second memory computing node in the system, and therefore the first read request may include sending a request for the first data to the second memory computing node. At step 1206, the first method 1200 may include receiving the first data at the first memory computing node in response to the request provided at step 1204. In an example, steps 1204 and / or 1206 may include or use a computing structure, such as scale structure 106, to exchange information between different nodes.

[0158] At step 1208, the first method 1200 may include filling at least one of the N blocks in a cache at the first memory computing node using the first data received at step 1206. For example, if the first data corresponds to a second node in the system, then step 1208 may include filling a specific block in the N caches established at the first node at step 1202 that corresponds to the second node.

[0159] At step 1210, the first method 1200 may include using information from N cache blocks. For example, step 1210 may include using the first memory compute node to request the same first data requested at step 1204. However, since the first data is stored in the node-local cache of the first memory compute node, information from the node-local cache can be used to satisfy the read request for the first data. For example, if the first data is stored in a specific block of the N cache blocks that are local to the first memory compute node, then information retrieved locally from that specific block (rather than remotely from a different node) can be used to satisfy the request at step 1210.

[0160] Figure 13 The description typically includes a second method 1300 that conditionally uses information from the local cache to satisfy a read request. The second method 1300 can be executed using one or more nodes in a near-memory computing system, for example, using multiple different instances of the first memory computing device 112 in the CNM system 102.

[0161] At step 1302, the second method 1300 may include establishing N cache blocks at the first memory computing node of the near-memory computing system. For example, step 1302 may include or correspond to... Figure 12 Examples include steps 1202 through 1208. After step 1302, the node-local cache at the N-block cache or the first memory compute node may include information received from one or more of the same nodes or other nodes in the system, wherein such information is segmented or partitioned in the cache according to its source location (see example...). Figure 11 Examples include how information from different nodes can be distributed or stored in portions of the local caches of different nodes within the system.

[0162] At step 1304, the second method 1300 may include issuing a read request for specific data at the first memory computing node. In an example, step 1304 may include using computing resources at the first node 1002 to prepare or issue a request for the specific data. At decision step 1306, the second method 1300 may include using computing resources at the first node 1002 to determine whether the requested specific data exists in a node-local cache, such as a cache established at step 1302. If the data exists in the node-local cache, then at step 1310, the second method 1300 may include reading the specific data from the node-local cache. If the data does not exist in the node-local cache, then the second method 1300 may proceed to step 1308. At step 1308, the second method 1300 may include issuing a read request to a remote resource, such as a remote node in a system that includes the first memory computing node. For example, step 1308 may include step 1204 from the first method 1200, which includes providing a read request to a different node in the memory computing system. After step 1308, the second method 1300 may include using the received information to populate a portion of N cache blocks at the first memory compute node, so that the cache can be used to satisfy subsequent requests for the same data.

[0163] Figure 14This illustrates a block diagram of example machine 1400, which may utilize, be in, or be operated by any one or more of the techniques (e.g., methods) discussed herein. As described herein, an example may contain logic or several components or mechanisms in, or be operable by, machine 1400. A circuit system (e.g., a processing circuit system) is a collection of circuits implemented in the tangible entity of machine 1400, which includes hardware (e.g., simple circuits, gates, logic, etc.). The membership of a circuit system may be flexible over time. A circuit system includes components that can perform specific operations individually or in combination during operation. In an example, the hardware of a circuit system may be designed in an immutable manner to perform specific operations (e.g., hardwiring). In an example, the hardware of a circuit system may include variably connected physical components (e.g., execution units, transistors, simple circuits, etc.) that include machine-readable media that are physically modified (e.g., invariant magnetic properties of concentrated particles, electrically movable placement, etc.) to encode instructions for specific operations. When connecting physical components, the fundamental electrical properties of the hardware components are altered, for example, from an insulator to a conductor, or vice versa. Instructions enable embedded hardware (e.g., an execution unit or loading mechanism) to hardware-create parts of a circuit system via variable connections to perform specific operations when in operation. Thus, in an example, a machine-readable media element is part of the circuit system or communicatively coupled to other components of the circuit system during device operation. In an example, any one of the physical components can be used in more than one part of more than one circuit system. For example, in operation, an execution unit may be used at one point in time in a first circuit of a first circuit system and reused at different times by a second circuit of the first circuit system or by a third circuit of the second circuit system. Additional examples of these components of machine 1400.

[0164] In alternative embodiments, machine 1400 may operate as a standalone device or be connectable (e.g., network-connected) to other machines. In a networked deployment, machine 1400 may operate as a server machine, a client machine, or both in a server-client network environment. In an example, machine 1400 may act as a peer-to-peer (P2P) (or other distributed) network environment. Machine 1400 may be a personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), mobile phone, network appliance, network router, switch, or bridge, or any machine capable of executing (sequentially or otherwise) instructions specifying actions to be taken by said machine. Furthermore, while only a single machine is described, the term "machine" should also be considered as encompassing any collection of machines that individually or jointly execute one or more sets of instructions to perform any one or more of the methodologies discussed herein, such as cloud computing, Software as a Service (SaaS), and other computer cluster configurations.

[0165] Machine 1400 (e.g., a computer system) may include a hardware processor 1402 (e.g., a central processing unit (CPU), graphics processing unit (GPU), hardware processor core, or any combination thereof), main memory 1404, static memory 1406 (e.g., memory or storage device for firmware, microcode, basic input / output (BIOS), unified extensible firmware interface (UEFI), etc.), and mass storage device 1408 (e.g., hard disk drive, tape drive, flash storage device, or other block device), some or all of which may communicate with each other via interconnect 1430 (e.g., a bus). Machine 1400 may further include a display device 1410, an alphanumeric input device 1412 (e.g., a keyboard), and a user interface (UI) navigation device 1414 (e.g., a mouse). In an example, the display device 1410, the input device 1412, and the UI navigation device 1414 may be a touchscreen display. Machine 1400 may additionally include a mass storage device 1408 (e.g., a drive unit), a signal generation device 1418 (e.g., a speaker), a network interface device 1420, and one or more sensors 1416, such as a Global Positioning System (GPS) sensor, a compass, an accelerometer, or other sensors. Machine 1400 may include an output controller 1428, for example, serial (e.g., Universal Serial Bus (USB), parallel, or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connection, to communicate with or control one or more peripheral devices (e.g., a printer, a card reader, etc.).

[0166] The registers of the hardware processor 1402, main memory 1404, static memory 1406, or mass storage device 1408 may be or contain machine-readable medium 1422, on which one or more sets of data structures or instructions 1424 (e.g., software) embodying or used by any one or more of the techniques or functions described herein are stored. Instructions 1424 may also reside wholly or at least partially in any of the registers of the hardware processor 1402, main memory 1404, static memory 1406, or mass storage device 1408 during execution by the machine 1400. In this example, one or any combination of the hardware processor 1402, main memory 1404, static memory 1406, or mass storage device 1408 may constitute machine-readable medium 1422. Although machine-readable media 1422 is described as a single medium, the term "machine-readable media" may include a single medium or multiple media (e.g., a centralized or distributed database, or associated caches and servers) configured to store one or more instructions 1424.

[0167] The term "machine-readable media" can include any medium capable of storing, encoding, or carrying instructions for execution by machine 1400 and causing machine 1400 to perform any one or more of the technologies disclosed herein, or any medium capable of storing, encoding, or carrying data structures used by or associated with such instructions. Examples of non-limiting machine-readable media can include solid-state memory, optical media, magnetic media, and signals (e.g., radio frequency signals, other photon-based signals, sound signals, etc.). In examples, non-transitory machine-readable media includes machine-readable media having a plurality of particles having invariant (e.g., rest) mass and therefore being composed of matter. Thus, non-transitory machine-readable media is machine-readable media that does not contain transiently propagating signals. Specific examples of non-transitory machine-readable media can include: non-volatile memory, such as semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)) and flash memory devices; magnetic disks, such as internal hard disks and removable hard disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0168] In an example, information stored or otherwise provided on machine-readable medium 1422 may represent instructions 1424, such as instructions 1424 itself or a format from which instructions 1424 can be derived. This format from which instructions 1424 can be derived may include source code, encoded instructions (e.g., in compressed or encrypted form), encapsulated instructions (e.g., split into multiple encapsulations), etc. Information representing instructions 1424 on machine-readable medium 1422 may be processed by a processing circuitry system into instructions to perform any of the operations discussed herein. For example, deriving instructions 1424 from information (e.g., processed by a processing circuitry system) may include: compiling (e.g., from source code, object code, etc.), interpreting, loading, organizing (e.g., dynamically or statically linking), encoding, decoding, encrypting, decrypting, encapsulating, decapsulating, or otherwise manipulating information into instructions 1424.

[0169] In an example, the derivation of instruction 1424 may involve assembling, compiling, or decompiling information (e.g., by processing a circuit system) to produce instruction 1424 from some intermediate or preprocessed format provided by machine-readable media 1422. When information is provided in multiple parts, the information may be combined, decapsulated, and modified to create instruction 1424. For example, the information may be in multiple compressed source code packages (or object code, or binary executable code, etc.) on one or more remote servers. The source code packages may be encrypted when transmitted over a network and, if necessary, decrypted, decompressed, assembled (e.g., linked), and compiled or decompiled at the local machine (e.g., into a stand-alone executable library, etc.) and executed by the local machine.

[0170] The system can further utilize any of several transport protocols (e.g., Frame Relay, Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), etc.) to transmit or receive instructions 1424 on the communication network 1426 via the network interface device 1420 using the transport medium. Example communication networks may include local area networks (LANs), wide area networks (WANs), packet data networks (e.g., the Internet), mobile phone networks (e.g., cellular networks), conventional telephone (POTS) networks, and wireless data networks (e.g., the Institute of Electrical and Electronics Engineers (IEEE) 802.11 series standards, referred to as… The IEEE 802.16 series of standards are called This includes standards such as IEEE 802.15.4 series and peer-to-peer (P2P) networks. In an example, network interface device 1420 may include one or more physical jacks (e.g., Ethernet, coaxial, or telephone jacks) or one or more antennas to connect to communication network 1426. In an example, network interface device 1420 may include multiple antennas to wirelessly communicate using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) technologies. The term "transmission medium" should be considered as any intangible medium capable of storing, encoding, or carrying instructions for execution by machine 1400, and containing digital or analog communication signals or other intangible media used to facilitate communication of such software. The transmission medium is a machine-readable medium.

[0171] To better illustrate the methods and apparatus described herein, a set of non-limiting exemplary embodiments are set forth below as numbered examples.

[0172] Example 1 may include or use objects (e.g., devices, systems, apparatus, methods, components for performing actions, or apparatus-readable media or articles containing instructions that, when performed by an apparatus, cause the apparatus to perform actions), for example, it may include or use a method comprising: establishing N cache blocks at a first memory computing node in a near-memory computing system comprising N different hardware nodes, wherein the first memory computing node is one of the N different hardware nodes, and wherein the N different hardware nodes are coupled by a computing architecture; and for each initial request from the first memory computing node to read information from a particular of the N different nodes, filling the corresponding one of the N blocks in the cache with boundary information received from the particular of the N different nodes. The boundary information may indicate whether the parent object of the particular of the N different nodes was found during some or all of the previous searches in the nodes coupled by the computing architecture. In this example, the boundary information includes a bitmap.

[0173] Example 2 may include or use the object of Example 1 or may optionally combine with the object of Example 1 to optionally include: for an initial request from a first memory computing node to read target information from a first node among N different nodes, retrieving the target information from the first node via a computing structure and filling the first of N blocks in the cache with the target information retrieved from the first node; and for subsequent requests from a first memory computing node to read target information from the first node, reading the target information from the first of N blocks in the cache.

[0174] Example 3 may include or use the object of any of Examples 1 or 2, or may optionally combine with the object of any of Examples 1 or 2 to optionally include: for a subsequent request from the first memory computing node to read information from a specific one of the N different nodes, read information from a corresponding cache block corresponding to the specific one of the N different nodes.

[0175] Example 4 may include or use the object of any of Examples 1 to 3 or may optionally combine with the object of any of Examples 1 to 3 to optionally include: at the first memory computing node, determining whether information from the cache can be used to satisfy a subsequent read request before a request for information is made from another node in the near-memory computing system.

[0176] Example 5 may include or use objects from any of Examples 1 to 4, or may optionally combine objects from any of Examples 1 to 4 to optionally include: establishing a cache as a first software object instantiated at the first memory compute node.

[0177] Instance 6 may contain or use objects from any of Instances 1 through 5, or may optionally combine objects from any of Instances 1 through 5 to optionally contain caches that have N blocks of equal size.

[0178] Instance 7 may contain or use objects from any of Instances 1 through 6, or may optionally combine objects from any of Instances 1 through 6 to optionally contain objects that saturate N blocks of cache.

[0179] Example 8 may include or use the object of any of Examples 1 to 7 or may optionally combine with the object of any of Examples 1 to 7 to optionally include: establishing a cache block for each of N different nodes in a near-memory computing system, and each cache block may include a 1024×64-bit word or element, or may include words or elements of other sizes.

[0180] Example 9 may include or use the object of any of Examples 1 to 8, or may optionally combine with the object of any of Examples 1 to 8 to optionally include: filling the corresponding block in the cache with the result from the bottom-up search algorithm, and the result may include boundary information received from the corresponding different nodes.

[0181] Instance 10 may contain or use the object of any of the instances 1 to 9, or may optionally combine with the object of any of the instances 1 to 9 to optionally contain: filling the corresponding block in the cache with the result from the breadth-first search algorithm, and the result may contain boundary information received from the corresponding different nodes.

[0182] Example 11 may include or use the object of Example 10 or may optionally combine with the object of Example 10 to optionally include: filling the corresponding block in the cache by prioritizing the search algorithm based on the connectivity of the node to be searched, and nodes with more neighbors or connections to other nodes may be prioritized over nodes with fewer neighbors or connections to other nodes.

[0183] Instance 12 may contain or use objects from any of Instances 1 to 11, or may optionally combine objects from any of Instances 1 to 11 to optionally include N blocks of cache as static cache without an eviction policy.

[0184] Example 13 may include or use the object of any of the preceding examples, or may optionally combine with the object of any of the preceding examples to optionally include a device comprising a first memory computing node among a plurality of memory computing nodes in a near-memory computing system, wherein the first memory computing node includes a processor configured to perform operations including: establishing N blocks of software cache, each of the N blocks corresponding to a corresponding memory computing node in the near-memory computing system; and filling the corresponding block in the cache with boundary information received from the corresponding memory computing node in the near-memory computing system. The boundary information may indicate whether a parent object of a particular node was found during some or all of the searches in the nodes coupled by the computational structure. In this example, the boundary information includes a bitmap.

[0185] Instance 14 may include or use the object of Instance 13 or may optionally combine with the object of Instance 13 to optionally include or use the processor to further perform operations including using a breadth-first search algorithm to populate blocks in the cache.

[0186] Instance 15 may contain or use the objects of Instance 14 or may optionally combine them with the objects of Instance 14 to optionally contain blocks that populate the cache using a bottom-up search method.

[0187] Instance 16 may contain or use objects from any of Instances 13 to 15, or may optionally combine objects from any of Instances 13 to 15 to optionally contain blocks populated in the cache using a search algorithm that prioritizes based on the connectivity of nodes in the system.

[0188] Example 17 may include or use the object of any of Examples 13 to 16, or may optionally combine with the object of any of Examples 13 to 16 to optionally include a second memory computing node among a plurality of memory computing nodes in a near-memory computing system, and the second memory computing node includes a second processor configured to perform operations including: establishing N blocks of software cache for use by the second memory computing node, wherein each of the N blocks corresponds to a corresponding one of the memory computing nodes in the near-memory computing system; filling one of the N blocks of software cache with boundary information received from a first memory computing node in the near-memory computing system; and filling the other N-1 blocks of software cache with boundary information received from other corresponding memory computing nodes in the near-memory computing system.

[0189] Example 18 may include or use objects from any of the preceding examples, or may optionally combine objects from any of the preceding examples to optionally include a memory computing system comprising N distinct memory computing nodes coupled by a scale structure to provide a computing architecture. In Example 18, each of the N nodes includes a hybrid thread processor and a hybrid thread structure, and a first memory computing node among the N distinct memory computing nodes includes a processor configured to perform operations including: building N blocks of software cache, each of the N blocks corresponding to a corresponding one of the memory computing nodes coupled by the scale structure; and filling the corresponding block in the cache with boundary information received from the corresponding node. The boundary information may indicate whether a parent object of the corresponding one of the N nodes was found during some or all of the previous searches in the nodes coupled by the hybrid thread structure.

[0190] Instance 19 may include or use the object of Instance 18 or may optionally combine with the object of Instance 18 to optionally include a processor of the first memory computing node, which is configured to fill blocks in the cache during the breadth-first search algorithm.

[0191] Example 20 may include or use the objects of Example 18 or 19, or may optionally combine them with the objects of Example 18 or 19 to optionally include, at each of the N different memory computing nodes, a corresponding processor and memory circuit configured to implement a corresponding N blocks of software-based caches, and each software-based cache on a particular node may be configured to store information received from each of the other nodes.

[0192] Each of these non-restrictive instances may exist independently or may be combined with one or more of the other instances in various permutations or combinations.

[0193] The above detailed description includes reference to the accompanying drawings, which form a part of the detailed description. The drawings illustrate, by means of illustration, specific embodiments in which the invention may be practiced. These embodiments are also referred to herein as “examples.” Such examples may include elements other than those shown or described. However, the inventors also contemplate that only examples of those elements shown or described are provided herein. Furthermore, the inventors also contemplate examples (or aspects thereof) of any combination or arrangement of those elements shown or described relative to a particular example (or one or more aspects thereof) or relative to other examples (or one or more aspects thereof) shown or described herein.

[0194] In this document, as is common in patent documents, the term “a (or an)” is used to include one or more, regardless of any other instances or uses of “at least one” or “one or more”. In this document, unless otherwise indicated, the term “or” is used to refer to a non-exclusive “or”, such that “A or B” may include “A but not B”, “B but not A”, and “A and B”. In the appended claims, the terms “comprising” and “in which” are used as common equivalents to the corresponding terms “including” and “wherein”. Furthermore, in the following claims, the terms “comprising” and “including” are open-ended, meaning that a system, apparatus, article, or process that includes elements other than those listed in the claims following such terms is still considered to be within the scope of the claims. Additionally, in the following claims, the terms “first,” “second,” and “third,” etc., are used merely as designations and are not intended to impose numerical requirements on their objects.

[0195] The above description is intended to be illustrative and not restrictive. For example, the examples described above (or one or more aspects thereof) can be used in combination with each other. Other embodiments may be used by those skilled in the art upon review of the above description. It is submitted under the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Similarly, in the above detailed description, various features may be grouped together to simplify this disclosure. This should not be construed as expecting that any unclaimed disclosed features are necessary for any claim. In fact, the subject matter of the invention may consist of fewer features than all of the particular disclosed embodiments. Therefore, the appended claims are hereby incorporated into the detailed description, wherein each claim exists independently as a separate embodiment, and these embodiments are contemplated to be combined or arranged in various ways. The scope of the invention should be determined by reference to the appended claims and the full scope of the equivalents granted by the claims.

Claims

1. A method comprising: N caches are established at the first memory computing node in a near-memory computing system containing N different hardware nodes, wherein the first memory computing node is one of the N different hardware nodes, and wherein the N different hardware nodes are coupled by a computing structure. For each initial request from the first memory compute node to read information from a specific one of the N different hardware nodes, the corresponding one of the N blocks in the cache is filled with boundary information received from the specific one of the N different hardware nodes, wherein the boundary information indicates whether the parent object of the specific one of the N different hardware nodes was found during some or all of the previous searches in the nodes coupled by the compute structure. as well as For an initial request from the first memory computing node to read target information from the first node among the N different hardware nodes, the target information from the first node is retrieved via the computing structure and the target information retrieved from the first node is used to fill the first of the N blocks in the cache; as well as For subsequent requests from the first memory computing node to read the target information from the first node, the target information is read from the first of the N blocks in the cache.

2. The method of claim 1, further comprising, at the first memory computing node, determining whether information from the cache can be used to satisfy further subsequent requests before requesting information from another node in the near-memory computing system.

3. The method of claim 1, wherein establishing the N cache blocks comprises establishing the cache as a first software object instantiated at the first memory compute node.

4. The method of claim 1, wherein establishing the N-block cache comprises establishing the cache having N blocks of equal size.

5. The method according to claim 1, wherein establishing the N cache blocks comprises establishing a saturated N cache blocks.

6. The method of claim 1, wherein establishing the N cache blocks comprises establishing cache blocks for each of the N different hardware nodes in the near-memory computing system, and wherein each cache block comprises a 1024×64-bit word.

7. The method of claim 1, further comprising populating the corresponding blocks in the cache with results from a bottom-up search algorithm, wherein the results include boundary information received from the respective different nodes.

8. The method of claim 1, further comprising populating the corresponding blocks in the cache with results from a breadth-first search algorithm, wherein the results contain boundary information received from the respective different nodes.

9. The method of claim 8, wherein filling the corresponding block in the cache comprises prioritizing the search algorithm based on the connectivity of the node to be searched, wherein a node with more neighbors or connections to other nodes is preferred over a node with fewer neighbors or connections to other nodes.

10. The method of claim 1, wherein establishing the N cache blocks comprises establishing a static cache without an eviction policy.

11. An apparatus comprising: A first memory compute node among multiple memory compute nodes in a near-memory computing system, wherein the first memory compute node includes a processor configured to perform operations including the following: Establish N software cache blocks, wherein each of the N blocks corresponds to a corresponding memory computing node in the near-memory computing system; as well as The corresponding blocks in the cache are populated with boundary information received from the corresponding memory computing nodes in the near-memory computing system, wherein populating the corresponding blocks includes prioritizing the search algorithm according to the connectivity of the nodes in the system.

12. The device of claim 11, wherein the processor is configured to further perform operations including using a breadth-first search algorithm to populate the block in the cache.

13. The device of claim 12, wherein using the breadth-first search algorithm includes using a bottom-up approach to populate the blocks in the cache.

14. The apparatus of claim 11, further comprising a second memory computing node among the plurality of memory computing nodes in the near-memory computing system, wherein the second memory computing node includes a second processor configured to perform operations including: N software cache blocks are established for use by the second memory computing node, wherein each of the N blocks corresponds to a corresponding memory computing node in the near-memory computing system; One of the N blocks in the software cache is filled with boundary information received from the first memory computing node in the near-memory computing system; as well as The other N-1 blocks in the software cache are filled with boundary information received from other corresponding memory computing nodes in the near-memory computing system.

15. A memory computing system, comprising: N distinct memory compute nodes are coupled by a scale structure to provide compute architecture, each of the N nodes including a hybrid-threaded processor and a hybrid-threaded architecture, wherein a first memory compute node of the N distinct memory compute nodes includes a processor configured to perform operations including: Establish N software cache blocks, wherein each of the N blocks corresponds to a corresponding one of the memory compute nodes coupled by the scale structure; The corresponding block in the cache is filled with boundary information received from the corresponding node, wherein the boundary information indicates whether the parent object of the corresponding one of the N nodes was found during some or all of the previous searches in the nodes coupled by the hybrid thread structure. as well as For an initial request from the first memory computing node to read target information from a second node among the N different memory computing nodes, the target information from the second node is retrieved via the computing structure and the target information retrieved from the second node is used to fill the first of the N blocks in the cache; and for a subsequent request from the first memory computing node to read the target information from the second node, the target information is read from the first of the N blocks in the cache.

16. The memory computing system of claim 15, wherein the processor of the first memory computing node is configured to fill the block in the cache during the execution of a breadth-first search algorithm.

17. The memory computing system of claim 15, wherein each of the N different memory computing nodes includes a corresponding processor and memory circuitry configured to implement corresponding N software-based caches, wherein each software-based cache on a particular node is configured to store information received from each of the other nodes.

18. The memory computing system of claim 15, wherein establishing the N-block cache comprises establishing the cache having N blocks of equal size.

19. The memory computing system of claim 15, wherein the corresponding block filling the cache comprises results using a breadth-first search algorithm, wherein the results comprise boundary information received from the respective different nodes.

20. The memory computing system of claim 19, wherein filling the corresponding block in the cache comprises prioritizing the search algorithm based on the connectivity of the nodes, wherein nodes with more neighbors or connections to other nodes are preferred over nodes with fewer neighbors or connections to other nodes.

Images