Direct connections between logic boards and disaggregated memory
The system addresses memory access challenges in disaggregated computing by using hash functions and NOC routing to connect compute devices directly to memory devices on separate boards, improving efficiency and reducing latency in memory access.
Patent Information
- Application Number
- PCT/US2026/026380
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-07
- Filing Date
- 2026-05-01
- Publication Date
- 2026-11-12
AI Technical Summary
Conventional systems face limitations in memory access and sharing across separate devices due to architectural, electrical, and protocol-related constraints, leading to signal integrity challenges and limited memory capacity in disaggregated computing architectures, especially when using PCIe or CXL communication.
A system that enables memory access without PCIe or CXL by applying hash functions to memory requests, routing them through a Network on a Chip (NOC) to connect compute devices directly to memory devices on separate boards, using optical and copper interconnects to facilitate efficient data transfer.
This approach improves load balancing, reduces latency and complexity, and enhances memory access efficiency in disaggregated systems by distributing memory requests across multiple memory devices without the limitations of traditional high-speed interconnects.
Smart Images

Figure US2026026380_12112026_PF_FP_ABST
Abstract
Description
000472-0005-W01DIRECT CONNECTIONS BETWEEN LOGIC BOARDS AND DISAGGREGATED MEMORYCross Reference to Related Application
[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 801,539, filed May 7, 2025, which is hereby incorporated by reference herein in its entirety.Background
[0002] The present disclosure relates to interconnections among microelectronic devices, and in particular, to improved connections between compute boards and memory appliances.Summary
[0003] Recent advancements in natural language processing (e.g., the development and deployment of large language models (LLMs)) has resulted in increased computational and memory demands for processing systems (e.g., central processing unit (CPU), graphical processing unit (GPU), etc.). LLMs typically rely heavily on extensive neural networks that require substantial computing power and large memory capacities for optimal performance. In some conventional systems, compute devices and memory are tightly collocated, such that the processing device and associated memory modules are located on the same motherboard or device package. For example, certain processing devices may utilize memory modules / devices (e.g., high bandwidth memory (HBM) chips) that are physically positioned along the periphery (or “shoreline”) of a processor die. The available shoreline area is limited, and therefore the number and capacity of memory devices that can be placed along the edges of the logic die are constrained by the finite physical space available.
[0004] In addition to constrictions related to the size and number of memory devices of existing server architectures, there are also limitations on sharing memory between devices. Accessing memory across devices (e.g., on separate printed circuit boards (PCBs)) is generally more complex than accessing local memory due to architectural, electrical, and protocol-related constraints. For example, processors are typically designed to interface with memory through well-defined, highspeed memory buses (e.g., double data rate (DDR) interfaces) that assume tight physical proximity, controlled impedance, and precise timing relationships. When memory is located on a different device, these assumptions are no longer valid. The increased physical separation introduces signal integrity challenges such as attenuation, reflections, crosstalk, and timing skew, all of which may degrade reliable data transfer.000472-0005-W01
[0005] In some approaches, disaggregated memory architectures are used in which compute resources and memory resources are physically separated into different hardware modules (e.g., compute boards or sleds and memory boards or sleds). In such systems, high-speed interconnect technologies may be used to enable communication between the compute resources and the remote memory resources. For example, a compute device may communicate with memory resources over an interconnect such as PCI Express (PCIe), which provides a high-speed interface for connecting devices within a server, and / or Compute Express Link (CXL), which is built on the PCIe physical layer and allows memory-oriented communication protocols (e.g., memory expansion and memory sharing across devices). However, PCIe comprises a fixed number of connection lanes such that increasing the amount of memory accessible to one compute device is limited by the PCIe topology. While PCIe switches may be used to expand the one PCIe connection to multiple downstream devices, a PCIe switch typically adds additional system complexity, latency, and bandwidth sharing. Additionally, since CXL runs on the same physical links as PCIe, the same constraints of limited connections and increased complexity may apply in CXL-based memory communication. Further, CXL strategies may require compatible computing devices, controllers, and / or memory devices such that compatibility issues arise.
[0006] In view of these deficiencies, among others, there exists a need for architectures which support memory access in disaggregated systems without PCIe or CXL communication. Accordingly, techniques are disclosed herein for a system to provide additional memory to computing devices. The systems and methods may comprise receiving, at a compute device, a plurality of memory requests that each correspond to a request to access a memory device of a plurality of memory devices. The compute device may be disposed on a first board and may be connected to a plurality of interconnects. Each interconnect of the plurality of interconnects may connect to at least one memory device of the plurality of memory devices. The systems and methods may further comprise applying a hash function (e.g., performed by a memory controller implemented on the compute device) to each memory request of the plurality of memory requests. The systems and methods may additionally identify a first memory device of the plurality of memory devices corresponding to a first memory request of the plurality of memory requests based on the hash function. The system may access the first memory device via a first interconnect of the plurality of interconnects, without using (Compute Express Link) CXL nor Peripheral Component Interconnect Express (PCIe). The first memory device may be disposed on a second board different than the first board comprising the compute device.000472-0005-W01Brief Description of the Drawings
[0007] The accompanying drawings provide additional details related to some embodiments of the disclosure described herein. The drawings are provided for purposes of illustration only and merely depict typical or example embodiments. For example, FIGS. 1-7 display one or more systems, portions of systems, appliances, portions of appliances, microelectronic devices, portions of microelectronic devices, connectors, portions of connectors, interconnects, and / or portions of interconnect, in accordance with embodiments of the disclosure. These drawings are provided to facilitate an understanding of the concepts disclosed herein and should not be considered limiting of the breadth, scope, or applicability of these concepts. It should be noted that for clarity and ease of illustration, these drawings are not necessarily made to scale.
[0008] FIG. 1 is an illustrative example of a compute appliance accessing disaggregated memory, in accordance with certain embodiments of the present disclosure.
[0009] FIG. 2 is an illustrative example of a compute appliance accessing disaggregated memory, in accordance with certain embodiments of the present disclosure.
[0010] FIG. 3 is an illustrative example of a compute appliance accessing disaggregated memory, in accordance with certain embodiments of the present disclosure.
[0011] FIG. 4 is a diagram depicting an example of a connection configuration for multiple appliances, in accordance with certain embodiments of the present disclosure.
[0012] FIG. 5 is an example architecture of compute and memory resources, in accordance with certain embodiments of the present disclosure.
[0013] FIG. 6 depicts an illustrative flowchart for a process of accessing disaggregated memory without CXL or PCIe, in accordance with certain embodiments of the present disclosure.
[0014] FIG. 7 depicts an illustrative flowchart for a process of transmitting and receiving memory and / or memory requests, in accordance with certain embodiments of the present disclosure.Detailed Description
[0015] FIG. 1 is an illustrative example of a compute appliance accessing disaggregated memory, in accordance with certain embodiment of the present disclosure. System 100 comprises a compute appliance 102 (e.g., compute board) connected to a plurality of memory appliances, including memory appliance 104 (e.g., memory sled). Compute appliance 102 comprises a plurality of compute devices, including artificial intelligence unit (AIU) 106. In some examples, compute appliance 102 may comprise only one AIU, e.g., AIU 106. In other examples, compute appliance 102 may comprise any number of AIUs. In some examples, AIU000472-0005-W01106 comprises memory controller 108 and Network on a Chip (NOC) 110. In some embodiments, for example when compute appliance 102 comprises multiple AIUs, each AIU may comprise its own respective memory controller and / or NOC. In some embodiments, the compute appliance 102 comprises a first PCB board and the memory appliance 104 comprises a second PCB board. In some embodiments, the compute appliance 102 and the memory appliance 104 are located on different racks in a data center.
[0016] In some examples, the plurality of compute devices of compute appliance 102 may comprise any one or more of a central processing unit (CPU), graphical processing unit (GPU), tensor processing unit (TPU), system-on-chip (SoC) device, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), neural processing unit (NPU), data processing unit (DPU), combinations of the same, or the like. The memory appliance 104 may comprise a plurality of memory modules (e.g., memory module 1, memory module 2 ... memory module N), which may be any one or more of a dual in-line memory modules (DIMMs), random access memory (RAM) chips, static RAM (SRAM), low power double data rate dynamic RAM (LPDDR DRAM), graphics double data rate DRAM (GDDR DRAM), high bandwidth memory (HBM), DDR-based memory, flash memory, combinations of the same, or the like.
[0017] In some embodiments, the plurality of memory modules and / or memory devices may be mounted to one or more PCBs of memory appliance 104. The memory modules may be disposed in sockets (e.g., DIMM sockets), soldered directly to a PCB, integrated into a multi-chip package, or mounted via interposers or package substrates. In some embodiments, memory devices may be arranged on one or both sides of a PCB, vertically stacked, or disposed in a three-dimensional (3D) configuration (e.g., HBM stacks). Electrical coupling between memory devices and other components (e.g., memory controllers or connectors) may be provided via PCB traces, package routing, interposers, flex circuits, cables, or other conductive interconnect structures. In some embodiments, memory modules may be removably coupled (e.g., via sockets) or permanently affixed (e.g., soldered or bonded) to the memory appliance.
[0018] In some embodiments, the memory controller 108 and / or NOC 110 are implemented within the compute device (e.g., AIU 106). For example, the memory controller 108 and / or NOC 110 may be implemented on one or more dies of the compute device. In some embodiments, the compute device (e.g., AIU 106) is mounted on a compute board (e.g., a PCB), such that the memory controller 108 and / or NOC 110 may be physically located within the compute device (e.g., AIU 106) while being electrically coupled to other components via the compute board.
[0019] As depicted in FIG. 1, compute appliance 102 comprises a plurality of connectors, including connector 112 and / or connector 122. Memory appliance 104 comprises a plurality of000472-0005-W01connectors, including connector 116 and / or connector 118. In some embodiments, one end of interconnect 114 is attached to connector 112 and the other end of interconnect 114 is attached to connector 116. In some embodiments, one end of interconnect 120 is attached to connector 122 and the other end of interconnect 120 is attached to connector 118. In some examples, connector 112 (e.g., and interconnects attached to connector 112) corresponds to transmitting signals from compute appliance 102 to memory appliances, and connector 122 (e.g., and interconnects attached to connector 122) corresponds to receiving signals at compute appliance 102 from memory appliances.
[0020] In some embodiments, AIU 106 generates a plurality of memory requests. Each memory request may correspond to a request for memory from a memory appliance, such as memory appliance 104. In some embodiments, the plurality of memory requests may be generated by one or more processing elements (e.g., cores, accelerators, execution units, etc.) of AIU 106. Each memory request may correspond to a physical address (e.g., a binary number). In some embodiments, each memory request may further comprise one or more attributes, such as a read or write indicator, a request identifier, and / or transaction metadata.
[0021] In some embodiments, the memory controller 108 may generate transformed memory requests. For example, the memory controller 108 may apply a first distributing function (e.g., hash function or an interleaving function) to each memory request of the plurality of memory requests. In some embodiments, the first distributing function may operate on one or more bits of the physical address associated with each memory request. The hash function or interleaving function may generate transformed memory requests that comprise destination data that corresponds to which memory appliance the request is directed to (e.g., memory appliance 104). In some embodiments, the destination data may comprise a device identifier, channel identifier, and / or port identifier. In some examples, the result of the hash function or interleaving function may be encoded into the memory request to generate the transformed memory request. In some embodiments, the transformed memory request may comprise the original memory request augmented with the destination data.
[0022] For example, in examples using a hash function, data corresponding to a memory request (e.g., address, key, token, parameter ID, etc.) may be input into a hash function (e.g., h(input) = 3, which may correspond to memory sled 3) and the output may correspond to which memory sled of the plurality of memory sleds comprises the memory indicated by the memory request. The hash function may facilitate distribution of data, load balancing, and / or efficient retrieval of stored parameters000472-0005-W01
[0023] In some embodiments, the NOC 110 may receive the plurality of transformed memory requests from memory controller 108. The plurality of transformed memory requests may be packetized prior to injection into the NOC 110, for example by an interface logic associated with memory controller 108. The NOC 110 may route each transformed memory request to the correct connector (e.g., through an egress port) which corresponds to the memory appliance that the request identifies, without using PCIe nor CXL. In some embodiments, the NOC 110 may route the transformed memory requests based on the destination data without recomputing the hash function.
[0024] In some embodiments, a second distributing function may be used to determine which memory module of the plurality of memory modules corresponding to memory appliance 104 is associated with a given memory request. For example, a hash function may be used at the memory appliance 104 to determine that a memory request corresponds to memory module 1 of memory appliance 104. The second function may comprise applying a second hash function or other mapping operation to an input associated with a memory access (e.g., a memory address, parameter identifier, token, key, and / or similar such identifier).
[0025] In some embodiments, the first distributing function (e.g., hash function or interleaving function) may be applied to determine a first mapping of the input to a first resource (e.g., a memory sled of a plurality of appliances), and the second distributing function may be applied to determine a second mapping of the input to a second resource within the first resource (e.g., a memory module, memory controller, channel, bank, partition, and / or similar such component within the memory appliance). The second distributing function may use the same input, a modified input, and / or an intermediate result generated by the first distributing function. For example, the second distributing function may take as input the same memory request as the first distributing function, a subset of the bits of the same memory request, or a derived value from the first distributing function (e.g., hash2 = h2(hashl (input) || input).
[0026] In some embodiments, the first and second distributing functions collectively distribute data across multiple levels of a memory hierarchy to improve load balancing, reduce contention, increase parallelism, combinations of the same, and / or the like. The second distributing function may further facilitate redistribution of data in response to servicing, removal, and / or failure of one or more resources, such that access requests may be remapped while maintaining a distributed allocation across available resources. In some embodiments, the distributing functions may comprise hash functions, modulo operations, bit selection operations, and / or other deterministic or pseudo-random mapping functions.000472-0005-W01
[0027] In some embodiments, the one or more connections allow the AIU 106 to access the memory appliance 104 without using CXL or PCIe cards. For example, a first memory request may be generated by AIU 106 and sent to memory controller 108. The first memory request may correspond to a physical address associated with data to be accessed. The first memory request is inputted into a function (e.g., a hash function or interleaving function) to generate a transformed first memory request that comprises destination data identifying that the request is for memory from memory appliance 104. The transformed first memory request is packetized (e.g., formatted into a packet comprising the destination data) and sent to NOC 110, where the NOC 110 routes the transformed first memory request to connector 112 (e.g., without interaction or coordination from a PCIe or a CXL interface). The transformed first memory request is transmitted from connector 112, through interconnect 114, to connector 116 of memory appliance 104. In some examples, memory appliance 104 transmits the memory back to AIU 106 based on the first memory request through connector 116, interconnect 114, and connector 112 again. In other examples, memory appliance 104 transmits the memory back to AIU 1 based on the first memory request from connector 118, through interconnect 120, to connector 122.
[0028] In some embodiments, each interconnect (e.g., interconnect 114 and / or interconnect 120) comprises a plurality of wires (e.g., 64 wires, 128 wires, etc.). In some embodiments, one or more interconnects comprise at least one optical wire. For example, the system may use copackaged optics (CPO) and at least one of the interconnects may comprise a plurality of optical wires. In some embodiments, one or more interconnects comprise at least one copper wire. In some embodiments, the system may use an optical substrate, or other techniques when using CPO. For example, the system may use co-packaged copper (CPC) and at least one interconnect may comprise a plurality of copper wires.
[0029] FIG. 2 shows a system 200 with a first architecture. Other architectures may incorporate one or more aspects described herein. FIG. 2 shows a first appliance 202 (e.g., which may correspond to compute appliance 102 of FIG. 1) comprising a plurality of processors, including processor 206, processor 208, processor 210, and / or processor 212 (e.g., AIU 106 of FIG. 1). First appliance 102 may be connected to a second appliance 204 (e.g., memory appliance 104 of FIG.1) comprising a plurality of memory modules / devices including memory module 214, memory module 216, memory module 218, memory module 220, memory module 222, and / or memory module 224 (e.g., dual in-line memory modules (DIMMs), random access memory (RAM) chips, etc.).
[0030] In some embodiments, the first appliance 202 comprises a first substrate (e.g., a PCB substrate) and a processor 206 placed on the first substrate. The processor 206 may comprise a000472-0005-W01first memory controller (e.g., memory controller 108 of FIG. 1). In some embodiments, the second appliance 204 comprises a second substrate 252 (e.g., a PCB substrate) and a plurality of memory modules such as memory module 224 placed on the second substrate 252. The plurality of memory modules may each comprise a memory controller. In some embodiments, the first appliance 202 and the second appliance 204 are located on different racks in a data center.
[0031] In some embodiments, there are one or more connections between the first appliance 202 and the second appliance 204. For example, there may be a first interconnect 234, a second interconnect 242, and a third interconnect 240, between the processor 206 and the memory module 224. A first side of the first interconnect 234 may be attached to a first connector 207 and a second side of the first interconnect 234 may be attached to a second connector 226 placed on the first substrate. A first side of the second interconnect 242 may be attached to second connector 226, and a second side of second connector 226 may be attached to a third connector 230. A first side of the third interconnect 240 may be attached to the third connector 230 and a second side of the third interconnect 240 may be attached to a fourth connector 223.
[0032] In some embodiments, the one or more connections allow the processor 206 to access the memory module 224 without using CXL or PCIe cards. For example, the processor 206 may use the first memory controller to transmit a memory space request to the memory module 224. The request may travel via the first connector 207, the first interconnect 234, the second connector 226, the second interconnect 242, the third connector 230, the third interconnect 240, the fourth connector 223, then arrive at the memory module 224. The memory module 224 may use the second memory controller to identify memory space associated with the request received from the processor 206. The memory module 224 may then transmit information about the requested memory space back to the processor 206. The information about the requested memory space may travel via additional interconnects and controllers and then arrive at the processor 206. Accordingly, the processor 206 is able access the memory module 224 without using CXL or PCIe cards. The first memory controller and / or the second memory controller may use a distributing function such as a hash function or interleaving function.
[0033] In some embodiments, each interconnect comprises a plurality of wires. For example, the first interconnect 234 may comprise sixty -four wires and the second interconnect 242 may comprise one hundred and twenty-eight wires. In some embodiments, one or more interconnects comprise at least one optical wire. For example, the system may use co-packaged optics (CPO) and the first interconnect 234 may comprises a plurality of optical wires. In some embodiments, one or more interconnects comprise at least one copper wire. In some embodiments, the system may use an optical substrate , or other techniques when using CPO. For example, the system may000472-0005-W01use co-packaged copper (CPC) and the first interconnect 234 may comprises a plurality of copper wires.
[0034] In some embodiments, the processor 206 comprises a plurality of additional microelectronic devices (e.g., chiplets, HBM, etc.). For example, the processor 206 may comprise four chiplets. In some embodiments, each chiplet of the plurality of chiplets uses one or more memory controllers to access the memory module 224. For example, the first chiplet may use a memory controller and a first wire of the plurality of wires within the first interconnect 234 to access the memory module 224 and the memory controller and a second wire of the plurality of wires within the first interconnect 234 may be used by a second chiplet to access the memory module 224. In some embodiments, one or more wires of the plurality of wires are dedicated to a connection between a single chiplet and a single memory device. In some embodiments, each chiplet of the plurality of chiplets has access to each memory device located on each memory appliance. In some embodiments, one or more chiplets of the plurality of chiplets may have access to different memory devices (e.g., memory module 224, memory module 222, etc.) using one or more wires of the plurality of wires within the first interconnect 234. For example, a first wire of the plurality of wires within the first interconnect 234 may be used by a first chiplet to access the memory module 224 and, a second wire of the plurality of wires within the first interconnect 234 may be used by the first chiplet to access memory module 222. In some embodiments, one memory controller may be able to access multiple memory devices. In some embodiments, the processor 206 may have different memory controllers assigned to different memory devices. For example, a first memory controller may provide access to a first memory device and a second memory controller may provide access to a second memory device physically separated from the first memory device.
[0035] In some embodiments, the first appliance may comprise any number of microelectronic devices, interconnects, and / or connectors. For example, the first appliance may comprise four microelectronic elements. In some embodiments, each microelectronic element comprises four chiplets. Accordingly, the first appliance 202 may comprise sixteen chiplets. In another example, the first appliance may comprise fourty connectors. Similarly, the second appliance 204 may comprise more microelectronic devices, interconnects, and / or connectors.
[0036] In some embodiments, pairs of connectors are associated with different appliances. For example, the second connector 226 may be associated with transmitting signals to the second appliance and the connector below the second connector 226 may be associated with receiving signals from the second appliance. In some embodiments, the additional connectors shown below the second connector 226 are dedicated to transmitting or receiving signals from additional000472-0005-W01appliances. In some embodiments, the first appliance may comprise any number of interconnects. In some embodiments, each chiplet is connected to at least two interconnects. For example, the first interconnect 234 may be connected to a first chiplet and may be dedicated to transmitting signals from the processor 206 to one or more memory devices / modules on the second appliance. An additional interconnect may be connected to the first chiplet and may be dedicated to receiving signals from one or more memory devices on the second appliance. In some embodiments, having one interconnect dedicated to transmitting signals and another interconnect dedicated to receiving signals reduces near-end cross talk allowing for faster processing.
[0037] In some embodiments, the first appliance is connected to a plurality of appliances as shown in FIGS. 3, 4, and 5. In some embodiments, the additional appliances comprise additional memory devices and memory controllers. Accordingly, the microelectronic devices of the first appliance may have access to memory devices on more than one board. In some embodiments, additional interconnects are used between the first appliance and the additional appliances. For example, the one or more additional interconnects may connect additional connectors of the first appliance with connectors on the additional appliances. Using the methodologies describe herein, one or more chiplets may have direct access to memory devices located on different PCBs without using CXL or PCIe cards.
[0038] FIG. 3 depicts an example 300 of first appliance 302 comprising connectors (e.g., including connector 312, which may correspond to connector 112 of FIG. 1 and / or connector 226 of FIG. 2), where each connector corresponds to a different appliance (e.g., second appliance 304). For example, all signals transmitted and received between first appliance 302 and second appliance 304 may travel through interconnect 306 (e.g., and no other signals from other appliances) and to / from connector 312. Signals traveling between other appliances may use other interconnects (e.g., such as interconnect 308 or interconnect 310) and connectors.
[0039] FIG. 4 is a diagram depicting an example of a connection configuration for multiple appliances, in accordance with certain embodiments. System 400 comprises a first appliance 402 (e.g., which may be the same as first appliance 102 of FIG. 1 and / or first appliance 202 of FIG. 2), second appliance 404 (e.g., which may be the same as second appliance 104 of FIG. 1 and / or second appliance 204 of FIG. 2), and third appliance 406. While FIG. 4 is depicted with three appliances and corresponding devices (e.g., microelectronic devices, memory devices), connectors, and interconnects, system 400 may comprise any number of components to implement the methods described herein.
[0040] In certain representative embodiments, first appliance 402 is a compute board, and second appliance 404 and third appliance 406 are memory appliances. First appliance 402 may comprise000472-0005-W01any number of devices, including device 408. Device 408 (e.g., which may be the same as device 106 of FIG. 1) may correspond to a microelectronic device or processor, such as a CPU, GPU, SoC device, ASIC, FPGA, TPU, NPU, DPU, combinations of the same, or the like. Second appliance 504 may comprise any number of devices, including device 410. Device 410 (e.g., which may be the same as device 224 of FIG. 2) may correspond to a memory device or resource, microelectronic device or processor, such as a DIMM, RAM, SRAM DDR-based memory, flash memory, combinations of the same, or the like. Device 408 and device 410 may comprise connectors and interconnects as described with respect to FIG. 1.
[0041] As depicted in FIG. 4, additional connectors and interconnects (e.g., four connectors and four interconnects) may be used to transmit and receive signals between two appliances (e.g., appliance 402 and appliance 404). In some embodiments, second appliance 404 and third appliance 406 each comprise connectors and interconnects (e.g., interconnect 412, which may be the same as interconnect 242 of FIG. 2) to connect appliances to first appliance 402. While FIG. 4 is depicted with four interconnects between first appliance 402 and second appliance 404, and four interconnects between first appliance 402 and third appliance 406, any number of interconnects may be used.
[0042] FIG. 5 shows a first appliance 501 (e.g., compute appliance 102 of FIG. 1, compute appliance 202 of FIG. 2) comprising a plurality of processors including processor 502 (e.g., AIU 106 of FIG. 1, processor 206 of FIG. 2) connected to a second appliance 504 (e.g., a memory appliance 104 of FIG. 1, memory appliance 204 of FIG. 2) comprising a plurality of memory devices (e.g., dual in-line memory modules (DIMMs), random access memory (RAM) chips, etc.). In some embodiments, the first appliance 501 comprises a first substrate (e.g., a PCB substrate) and four processors (e.g., GP-AIU) placed on the first substrate. In some embodiments, each processor comprises four chiplets. In some embodiments, each processor comprises a plurality of CPC connectors. For example, a first processor 502 may comprises sixteen CPC connectors. The first processor 502 may comprise sixteen CPC connectors so that each chiplet has access to each microelectronic device on each memory appliance. In FIG. 4, the first processor 502 comprises four chiplets and there are four memory appliances so the first processor 502 comprises sixteen CPC connectors.
[0043] In some embodiments, the first appliance 501 comprises interconnects between the CPC connectors, one or more high density (HD) connectors, and one or more memory controllers so that each chiplet of the plurality of chiplets may have access to each memory device on each memory appliance. In some embodiments, each processor has access to at least one HD connector per memory appliance. For example, the first processor 502 may have access to a first HD000472-0005-W01connector 516 associated with a first memory appliance (e.g., second appliance 504), a second HD connector 518 associated with a second memory appliance (e.g., third appliance 510), a third HD connector 520 associated with a third memory appliance (e.g., fourth appliance 512), and a fourth HD connector 522 associated with a fourth memory appliance (e.g., fifth appliance 514). In some embodiments, at least four interconnects are connected to each HD connector. For example, a first interconnect may be between a first CPC connector for a first chiplet of the first processor 502 and the first HD connector 516, a second interconnect may be between a second CPC connector for a second chiplet of the first processor 502 and the first HD connector 516, a third interconnect may be between a third CPC connector for a third chiplet of the first processor 502 and the first HD connector 516, and a fourth interconnect may be between a fourth CPC connector for a fourth chiplet of the first processor 502 and the first HD connector 516.
[0044] In some embodiments, the HD connectors of the first appliance 501 are connected to HD connectors of other appliances. For example, a first interconnect may connect the first HD connector 516 of the first appliance 501 to a fifth HD connector 550 of the second appliance 504. A second interconnect may connect the second HD connector 518 of the first appliance 501 to a sixth HD connector 552 of the third appliance 510. A third interconnect may connect the first HD connector 516 of the first appliance 501 to a seventh HD connector 554 of the fourth appliance 512. A fourth interconnect may connect the first HD connector 516 of the first appliance 501 to an eighth HD connector 556 of the fifth appliance 514. In some embodiments, each memory device on a memory appliance may be connected to each HD connector on said memory appliance. For example, a first plurality of interconnects may connect the fifth HD connector 550 of the second appliance 504 to each memory device located on the second appliance 504. A second plurality of interconnects may connect the sixth HD connector 552 of the third appliance 510 to each memory device located on the third appliance 510. A third plurality of interconnects may connect the seventh HD connector 554 of the fourth appliance 512 to each memory device located on the fourth appliance 414. A fourth plurality of interconnects may connect the eighth HD connector 556 of the fifth appliance 514 to each memory device located on the fifth appliance 514. In some embodiments, the described architecture allows each processor (e.g., the first processor 502) on the first appliance 501 to have access to each memory device on each memory appliance.
[0045] In some embodiments, each HD connector (e.g., 516, 518, 520, 522) attaches to a plurality of interconnects corresponding to devices of first appliance 501. For example, each interconnect of the plurality of interconnects of a HD connector may attach to a respective connector (e.g., connector 508) corresponding to a respective chiplet (e.g., chiplet 506) of a respective processor (e.g., device 502) of first appliance 501. For example, a first interconnect may attach to both the000472-0005-W01first HD connector 516, and a first CPC connector (e.g., connector 508) for a first chiplet (e.g., chiplet 506) of a first processor (e.g., device 502). Further, in some examples, a second interconnect may attach to both the first HD connector 516 and a second CPC connector for a second chiplet of the first processor (e.g., device 502). Further, in some examples, a third interconnect may attach to both the first HD connector 516 and a third CPC connector for a third chiplet of the first processor (e.g., device 502). Further, in some examples, a fourth interconnect may attach to both the first HD connector 516 and a fourth CPC connector for a fourth chiplet of the first processor (e.g., device 502).
[0046] In some embodiments, the first appliance 501 (e.g., a compute appliance) and the second appliance 504 (e.g., memory appliance) may be located in physically separate boxes. In some embodiments, the first appliance 501 and the second appliance 504 comprise connectors on the back plates and / or the front plates through which physical communication medium passes. In some embodiments, the first appliance 501 and the second appliance 504 do not comprise connectors on the back plates and / or the front plates through which physical communication medium passes. In some embodiments, the first appliance 501 and the second appliance 504 may be integrated into the same physical housing with one or more interconnects entirely housed within the housing. In some embodiments, the one or more interconnects do not pass through any paneling.
[0047] FIG. 6 depicts an illustrative flowchart for a process of accessing disaggregated memory without CXL or PCIe, in accordance with certain embodiments of the present disclosure. Process 600 may be performed by any one or more of the devices, modules, or systems described herein. For example, any one or more steps of process 600 may be performed by a compute appliance, such as compute appliance 102 of FIG. 1, compute appliance 202 of FIG. 2, and / or first appliance 501 of FIG. 5. In some embodiments, any one or more steps of process 600 is performed by an AIU, such as AIU 106 of FIG. 1, processor 206 of FIG. 2, and / or processor 502 of FIG. 5.
[0048] At step 602, a compute device (e.g., an AIU, processor, compute appliance, or the like) may receive a plurality of memory requests, each corresponding to a request to access a respective memory module of a respective memory device of a plurality of memory devices. In some examples, each memory request of the plurality of memory requests comprises at least one of a memory address, parameter identification, key, or token. In some embodiments, the compute device is connected to a plurality of interconnects, and each interconnect of the plurality of interconnects connects to at least one memory device of the plurality of memory devices. In some examples, a first memory controller (e.g., memory controller 108) implemented within the compute device receives the plurality of memory requests.000472-0005-W01
[0049] At step 604, the compute device may generate a hash function output for a first memory request. In some examples, the first memory controller generates the first hash function output using a first hash function.
[0050] At step 606, the compute device may identify a first memory device corresponding to the first memory request based on the first hash function output. In some embodiments, the compute device may be disposed on a first board and the first memory device may be disposed on a second board different from the first board. In some embodiments, the first memory device comprises a plurality of memory modules (e.g., DRAMs). In some embodiments, the identifying the first memory device is performed by a first NOC (e.g., NOC 110) implemented within the compute device.
[0051] At step 608, the compute device may transmit the first memory request to the first memory device (e.g., without using CXI or PCIe), based on the identifying the first memory request. In some embodiments, the first memory request is transmitted to the first memory device via a first interconnect of the plurality of interconnects. In some embodiments, the first memory request is transmitted without using (Compute Express Link) CXL or Peripheral Component Interconnect Express (PCIe).
[0052] In some embodiments, the transmitting the first memory request to the first memory device causes the first memory device to generate a second hash function output for the first memory request. In some embodiments, the transmitting causes the first memory device to identify a first memory module of the plurality of memory modules based, at least in part, on the second hash function output. In some embodiments, a second memory controller is implemented within the first memory device, and the second memory controller generates the second hash function output using a second hash function. In some examples, the first memory device identifying the first memory module is performed by a second NOC implemented within the first memory device.
[0053] In some embodiments, the first hash function uses, as input, every bit corresponding to the first memory request, and the second hash function uses, as input, a subset of bits of the first memory request. In other embodiments, the first hash function uses, as input, the first memory request, and the second hash function uses, as input, the first hash function output.
[0054] At step 610, the compute device may receive, from the first memory module of the first memory device, memory data corresponding to the first memory request. In some examples, the compute device receives the memory data from the first memory module via a second interconnect (different than the first interconnect) of the plurality of interconnects. In other examples, the compute device receives the memory data from the first memory module via the first interconnect.000472-0005-W01
[0055] FIG. 7 depicts an illustrative example flowchart and block diagram for a process of transmitting and receiving memory and / or memory requests, in accordance with certain embodiments of the present disclosure. System 700 comprises a compute device 702 which comprises memory controller 704 and NOC 706. System 700 further comprises memory device 708 comprising a memory controller 710 and a memory module 712. Compute device 702, memory controller 704, NOC 706, memory device 708, memory controller 710 and / or memory module 712 may correspond to any of the analogous devices described herein.
[0056] Steps 750-762 may be performed by any one or more of the devices, modules, or systems described herein. For example, any one or more steps of steps 750-762 may be performed by a compute appliance, such as compute appliance 102 of FIG. 1, compute appliance 202 of FIG. 2, and / or first appliance 501 of FIG. 5. In some embodiments, any one or more steps of the process of FIG. 7 is performed by an AIU, such as AIU 106 of FIG. 1, processor 206 of FIG. 2, and / or processor 502 of FIG. 5 (e.g., compute device 702).
[0057] At step 750, memory controller 704 may receive a plurality of memory requests, each corresponding to a request to access a respective memory module of a respective memory device (e.g., memory module 712 of memory device 708).
[0058] At step 752, memory controller 704 may generate (e.g., using a first hash function), a first hash function output for a first memory request.
[0059] At step 754, NOC 706 may determine a first memory device 708 corresponding to the first memory request based on the first hash function output. In some embodiments, NOC 706 routes the first memory request to the first memory device 708, based on the first hash function output.
[0060] At step 756, memory device 708 (or memory controller 710 implemented within memory device 708) may receive the first memory request. In some examples, memory device 708 receives at least a subset or version of the first memory request.
[0061] At step 758, memory controller 710 may generate a second hash function output for the first memory request. In some examples, the memory controller 710 applies a second hash function or other mapping operation to an input associated with the first memory request (e.g., subset of bits of the first memory request, the first hash function output, etc.)
[0062] At step 760, memory controller 710 may determine a first memory module 712 corresponding to the first memory request based on the second hash function output. In some examples, the memory controller 710 routes the first memory request (or a version / subset of bits of the first memory request) to the first memory module 712 based at least in part on the second hash function output.000472-0005-W01
[0063] At step 762, the first memory module 712 (e.g., after receiving the memory request) transmits, to the compute device 702, memory data corresponding to the first memory request. In some examples, the memory data is transmitted via a different interconnect as the interconnect used to transmit the first memory request.
[0064] The processes discussed above are intended to be illustrative and not limiting. One skilled in the art would appreciate that the steps of the processes discussed herein may be omitted, modified, combined and / or rearranged, and any additional steps may be performed without departing from the scope of the invention. More generally, the above disclosure is meant to be illustrative and not limiting. Only the claims that follow are meant to set bounds as to what the present invention includes. Furthermore, it should be noted that the features described in any one embodiment may be applied to any other embodiment herein, and flowcharts or examples relating to one embodiment may be combined with any other embodiment in a suitable manner, done in different orders, or done in parallel. In addition, the systems and methods described herein may be performed in real time. It should also be noted that the systems and / or methods described above may be applied to, or used in accordance with, other systems and / or methods.
Claims
000472-0005-W01What is claimed is:
1. A method for a compute device to access a plurality of memory devices, the method comprising:receiving a plurality of memory requests, each corresponding to a request to access a memory module of a memory device of the plurality of memory devices, wherein:the compute device is disposed on a first board;the compute device is connected to a plurality of interconnects; and each interconnect of the plurality of interconnects connects to at least one memory device of the plurality of memory devices;generating a first hash function output for a first memory request of the plurality of memory requests;identifying a first memory device of the plurality of memory devices corresponding to the first memory request of the plurality of memory requests, based, at the least in part, on the first hash function output, wherein the first memory device comprises a plurality of memory modules;transmitting the first memory request to the first memory device based, at least in part, on identifying the first memory device, wherein:the first memory request is transmitted to the first memory device via a first interconnect of the plurality of interconnects;the first memory device is disposed on a second board different than the first board;the first memory request is transmitted without using (Compute Express Link) CXL or Peripheral Component Interconnect Express (PCIe);the transmitting causes the first memory device to generate a second hash function output for the first memory request; andthe transmitting causes the first memory device to identify a first memory module of the plurality of memory modules based, at least in part, on the second hash function output; andreceiving, via a second interconnect of the plurality of interconnects, memory data corresponding to the first memory request from the first memory module.
2. The method of claim 1, wherein the identifying the first memory device is performed by a Network on a Chip (NOC) implemented within the compute device.000472-0005-W013. The method of claim 1, wherein the compute device is an artificial intelligence unit (AIU).
4. The method of claim 1, wherein the first memory device is a memory appliance and the first memory module is a Dynamic Random Access Memory (DRAM).
5. The method of claim 1, wherein each memory request of the plurality of memory requests comprises at least one of a memory address, parameter identification, key, or token.
6. The method of claim 1, wherein a first memory controller implemented within the compute device receives the plurality of memory requests.
7. The method of claim 6, wherein the first memory controller generates the first hash function output using a first hash function.
8. The method of claim 7, wherein a second memory controller is implemented within the first memory device, and wherein the second memory controller generates the second hash function output using a second hash function.
9. The method of claim 8, wherein the first hash function uses, as input, every bit corresponding to the first memory request, and wherein the second hash function uses, as input, a subset of bits of the first memory request.
10. The method of claim 8, wherein the first hash function uses, as input, the first memory request, and wherein the second hash function uses, as input, the first hash function output.
11. The method of claim 1, wherein the memory data corresponding to the first memory request is received without using CXL or PCIe.
12. A method comprising:receiving, by a compute device, a plurality of memory requests, each corresponding to a request to access a memory module of a memory device of the plurality of memory devices, wherein:the compute device is disposed on a first board;the compute device is connected to a plurality of interconnects; and000472-0005-W01each interconnect of the plurality of interconnects connects to at least one memory device of the plurality of memory devices;generating, by the compute device, a first hash function output for a first memory request of the plurality of memory requests;identifying, by the compute device, a first memory device of the plurality of memory devices corresponding to the first memory request of the plurality of memory requests based, at the least in part, on the first hash function output, wherein the first memory device comprises a plurality of memory modules;transmitting, by the compute device, the first memory request to the first memory device based, at least in part, on identifying the first memory device, wherein:the first memory request is transmitted to the first memory device via a first interconnect of the plurality of interconnects;the first memory device is disposed on a second board different than the first board; andthe first memory request is transmitted without using (Compute Express Link) CXL or Peripheral Component Interconnect Express (PCIe);generating, by the first memory device, a second hash function output for the first memory request;identifying, by the first memory device, a first memory module of the plurality of memory modules based, at least in part, on the second hash function output;receiving, by the first memory device, memory data corresponding to the first memory request from the first memory module based, at least in part, on identifying the first memory module;receiving, by the compute device, the memory data from the first memory device wherein the compute device receives the memory data via a second interconnect of the plurality of interconnects.
