System and method for storage
By using synchronous clock signals between memory nodes for data encoding and decoding, the problem of unpredictable delay in distributed memory systems is solved, and more efficient memory communication and computing performance is achieved.
Patent Information
- Application Number
- CN202311190294.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-06-16
- Filing Date
- 2023-09-14
- Publication Date
- 2024-11-12
AI Technical Summary
Existing distributed memory systems have great uncertainty and unpredictability in communication latency, especially in memory pools, resulting in limited computing speed and throughput.
By introducing a synchronous clock signal between memory nodes, the clock data recovery (CDR) process is avoided, thereby reducing communication delays and ensuring predictability of delays.
It effectively reduces the delay between memory nodes, improves the performance and reliability of the computing system, and reduces energy consumption and processing overhead.
Smart Images

Figure CN118939177A_ABST
Abstract
Description
[0001] This application claims priority to and the benefit of U.S. Provisional Application No. 63 / 465,794, filed on May 11, 2023, entitled “SYNCHRONOUS CELL / FLIT SWITCHING FABRIC FOR SCALABLE MEMORY POOING ARCHITECTURE,” the entire contents of which are incorporated herein by reference. Technical Field
[0002] One or more embodiments of the present disclosure relate to scalable memory, and more particularly, to coherent scalable memory. Background Art
[0003] Typically, a computer can perform various operations by accessing data stored in high-speed computer memory, as opposed to storage operations that permanently store data in slower non-volatile memory of a storage device. As such, the speed and performance of a computing system can depend on the amount of computer memory resources available to it, as well as the latency associated with communication between processing elements and computer memory resources.
[0004] The above information disclosed in this Background section is for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute prior art. Summary of the invention
[0005] Computer memory resources may be aggregated together so that the total memory may be considered a memory "pool". The memory pool may be used as a whole rather than as separate memory components. Thus, a processor may also interact with the memory pool as an integrated memory resource rather than as separate memory components. However, aggregating different memory resources into a memory pool may introduce significant latency.
[0006] One or more embodiments of the present disclosure are directed to systems and methods for reducing latency of a scalable memory connected to a host processor via a cache coherence protocol based interconnect, such as, for example, a Compute Express Link (CXL) interconnect.
[0007] According to one or more embodiments of the present disclosure, a system includes: a group of memory resources, including a first memory node and a second memory node, the first memory node is connected to the second memory node through a switching structure; and a synchronous clock source, connected to the first memory node and the second memory node, the synchronous clock source is configured to provide a synchronous clock signal to the first memory node and the second memory node to synchronize the first memory node with the second memory node. The first memory node and the second memory node are configured to encode memory data and decode the encoded memory data using the synchronous clock signal.
[0008] In one embodiment, a first memory node may include a first controller configured to encode memory data using a synchronous clock signal and send the encoded memory data to a second memory node; and a second memory node may include a second controller configured to receive the encoded memory data from the first memory node and decode the encoded memory data using the synchronous clock signal.
[0009] In one embodiment, the system may further include: a memory bank controller connected to the first memory node and the second memory node via a switch fabric, the switch fabric including an interconnect based on a cache coherence protocol; and an input / output (I / O) connection. The memory bank controller may be configured to allocate memory from at least one of the first memory node and the second memory node by enabling communication between the I / O connection and at least one of the first memory node and the second memory node.
[0010] In one embodiment, the system may further include: a first processor connected to a first memory node and a second memory node of the set of memory resources via a switching structure, and connected to a first network interface card (NIC); and a second processor connected to a third memory node and a fourth memory node of the set of memory resources via a second switching structure, and connected to a second NIC. The first processor may be configured to communicate with the second processor via the first NIC or the second NIC.
[0011] In one embodiment, the first NIC may be connected to the second NIC via a serial interface.
[0012] In one embodiment, the third memory node and the fourth memory node may be configured to encode the memory data and decode the encoded memory data using a synchronous clock signal.
[0013] In one embodiment, the first processor may include one of a system on chip (SoC), a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), a field programmable gate array (FPGA), and an application specific integrated circuit (ASIC).
[0014] In one embodiment, the first memory node may include a first one of a memory die, a memory module, a memory chute, a memory pod, and a memory rack, and the second memory node may include a second one of a memory die, a memory module, a memory chute, a memory pod, and a memory rack.
[0015] In one embodiment, the first memory node may include a first memory rack and the second memory node may include a second memory rack, the second memory rack being different from the first memory rack.
[0016] In one embodiment, a maximum latency between the first memory node and the second memory node may be less than a predetermined threshold.
[0017] In one embodiment, the system may further include a processor in communication with a processor memory storing instructions that, when executed by the processor, cause the processor to execute an application that stores information in the set of memory resources.
[0018] In one embodiment, the first memory node may also be connected to the second memory node through a second switch fabric.
[0019] In one embodiment, the set of memory resources may be cache coherent.
[0020] According to one or more embodiments of the present disclosure, a system includes: a group of memory resources, including: a first memory node, including a first memory controller, a first memory, and a first interconnect based on a cache consistency protocol; and a second memory node, including a second memory controller, a second memory, and a second interconnect based on a cache consistency protocol, the second memory node is connected to the first memory node through a switching structure, and the switching structure connects the first interconnect based on the cache consistency protocol with the second interconnect based on the cache consistency protocol; and a synchronization clock source, connected to the first memory node and the second memory node, the synchronization clock source is configured to provide a synchronization clock signal to the first memory node at the first interconnect based on the cache consistency protocol, and provide a synchronization clock signal to the second memory node at the second interconnect based on the cache consistency protocol, so that the first memory node is synchronized with the second memory node. The first memory node and the second memory node are configured to send memory data to each other and receive memory data from each other using the same synchronization clock signal.
[0021] In one embodiment, the first memory node and the second memory node may be cache coherent.
[0022] In one embodiment, the first memory node and the second memory node may be configured to encode and decode memory data sent and received from each other using a synchronous clock signal.
[0023] In one embodiment, the synchronous clock source may include: a clock synthesizer including a phase locked loop; and a clock buffer.
[0024] In one embodiment, the clock buffer may be connected to the first memory node at a first cache coherence protocol based interconnect and may be connected to the second memory node at a second cache coherence protocol based interconnect.
[0025] According to one or more embodiments of the present disclosure, a method for encoding and decoding data in a separate memory node of a group of memory resources includes: storing the memory data in a first memory node; encoding the memory data using a synchronous clock signal at the first memory node; sending the encoded memory data to a second memory node via a switching structure; receiving the encoded memory data at the second memory node via the switching structure; and decoding the encoded memory data at the second memory node using the synchronous clock signal.
[0026] In one embodiment, the switch fabric may include an interconnect based on a cache coherence protocol; the first memory node may include a first memory rack including a first plurality of memory devices; and the second memory node may include a second memory rack including a second plurality of memory devices. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] The above and other aspects and features of the present disclosure will be more clearly understood from the following detailed description of illustrative, non-limiting embodiments with reference to the accompanying drawings.
[0028] Figure 1 is a schematic block diagram of a scalable memory system according to one or more embodiments of the present disclosure.
[0029] Figure 2 is a schematic block diagram of a synchronous clock source connected to a first memory node and a second memory node according to one or more embodiments of the present disclosure.
[0030] Figure 3A is a schematic block diagram of a storage pool controller connected to storage through a switching fabric including a compute express link (CXL) interconnect according to one or more embodiments of the present disclosure.
[0031] Figure 3B is a schematic block diagram of a system including a plurality of storage pool controllers connected to storage according to one or more embodiments of the present disclosure.
[0032] Figure 4A is a schematic block diagram of a heterogeneous processor (XPU) connected to a memory through a switch fabric including a compute express link (CXL) interconnect according to one or more embodiments of the present disclosure.
[0033] Figure 4B is a schematic block diagram of a system including multiple heterogeneous processors (XPUs) connected to a memory according to one or more embodiments of the present disclosure.
[0034] Figure 5 is a schematic block diagram of an expandable memory pool in communication with a host processor according to one or more embodiments of the present disclosure.
[0035] Figure 6 is a schematic block diagram of a synchronous clock source connected to different forms of memory interfaces according to one or more embodiments of the present disclosure.
[0036] Figure 7 is a flowchart of a method of encoding and decoding data in a storage node of a storage pool according to one or more embodiments of the present disclosure. DETAILED DESCRIPTION
[0037] Hereinafter, embodiments will be described in more detail with reference to the accompanying drawings, in which the same reference numerals always refer to the same elements. However, the present disclosure may be implemented in various different forms and should not be construed as being limited to the embodiments shown herein. On the contrary, these embodiments are provided as examples so that the present disclosure will be comprehensive and complete, and will fully convey the aspects and features of the present disclosure to those skilled in the art. Therefore, the processing, elements and techniques that are not necessary for a complete understanding of the aspects and features of the present disclosure by those of ordinary skill in the art may not be described. Unless otherwise stated, throughout the drawings and written descriptions, the same reference numerals represent the same elements, and therefore, their redundant descriptions may not be repeated.
[0038] Compute Express Link (CXL) is a cache coherent interconnect designed to provide low-latency, high-bandwidth connections between host processors and other devices. CXL maintains unified, coherent memory between the host processor and any memory on a connected CXL device. The host processor is primarily responsible for coherency management.
[0039] Network computing and storage may face the problem of growing data demands. For example, hyperscale workloads may have growing memory requirements, which may be difficult to meet while maintaining low latency. In addition, the industry as a whole may be facing a shortage of viable distributed shared memory and large address space systems. Although there are some solutions to reduce latency in distributed memory systems (such as using CXL), it may be desirable to further reduce latency to increase overall computing speed and / or throughput.
[0040] In addition to relatively high latency, existing distributed memory systems may face the problem of unpredictable latency. For example, because distributed memory resources use clock data recovery (CDR) when communicating with each other, workloads may have variable latency when communicating with distributed memory resources. Clock data recovery (CDR) is used when decoding memory data. For example, a clock signal may be used to encode memory data (e.g., to parallelize memory data) before sending memory data (e.g., between memory nodes). However, sending a clock signal with memory data may be impractical or undesirable. Therefore, in order to decode the encoded memory data, the decoder may first perform clock data recovery (CDR) on the encoded data to obtain the original clock signal used to encode the data.
[0041] Clock data recovery (CDR) can be implemented by a controller or other device that can detect data transitions (e.g., between logic low and logic high) in a signal carrying encoded memory data, lock the data transition frequency, and use the resulting frequency to generate a recovered clock signal that is identical to the original clock signal. Once the recovered clock signal is generated, the decoder can use the recovered clock signal to decode the encoded data. CDR techniques can be implemented using digital signal processing algorithms or dedicated circuits. The additional processing / overhead required to recover the original clock signal introduces latency in the operation of transmitting memory data. When used in a memory pool that may include many memory nodes, the additional processing / overhead at each memory node can significantly increase latency.
[0042] In addition, depending on the memory being accessed, latency may be unpredictable. For example, more CDRs may be required when storing / accessing memory data in one portion of a memory pool compared to another portion of the memory pool. Having an uncertain amount of latency is disadvantageous because, for example, an agreement between a processing server owner and a memory server owner regarding maximum latency may need to take into account variable latency rather than, for example, known / approximately constant latency. Additional processing / overhead may also increase energy consumption, especially in data centers that include large memory pools implemented on many servers. Therefore, reducing the amount of latency and ensuring that latency is predictable is a desired aspect of a memory pool.
[0043] Peripheral Component Interconnect Express (PCIe) may refer to a computer interface that may have relatively high and variable latency, which may limit its usefulness in making connections to memory. CXL is an open industry standard for communicating over PCIe 5.0, which may provide a fixed, relatively short packet size, and therefore may be able to provide relatively high bandwidth and relatively low fixed latency. As such, CXL may be able to support cache coherency, and CXL may be well suited for making connections to memory. CXL may also be used to provide connectivity between accelerators, memory devices, and network interface circuits (or "network interface controllers" or "network interface cards" (NICs)) in hosts and servers.
[0044] The CXL transaction layer may include three multiplexed sub-protocols that run concurrently or substantially simultaneously on a single link, and may be referred to as CXL.io, CXL.cache, and CXL.memory. CXL.io may include I / O semantics similar to PCIe. CXL.cache may include cache semantics, and CXL.memory may include memory semantics. In addition, the Gen-Z protocol that works for memory semantics and the OpenCAPI protocol that works for cache semantics may be folded into CXL as part of the CXL protocol.
[0045] In some embodiments, the use of CXL connections to an aggregation or "pool" of memory (e.g., a quantity of memory including multiple memory units connected together) may provide various advantages, such as, for example, in a system including multiple servers connected together via a network. For example, a CXL switch having further capabilities in addition to providing packet switching functionality for CXL packets (referred to herein as an "enhanced capability CXL switch") may be used to connect the aggregation of memory to one or more central processing units (CPUs) (or "central processing circuits") and to one or more network interface circuits (the network interface circuits may have enhanced capabilities).
[0046] In some embodiments, a memory pool includes separate memory nodes connected together, each memory node including separate memory cells. The memory nodes may be aggregated into a memory pool using CXL connections. However, while using CXL connections may reduce latency between a central processor and a memory pool, latency between memory nodes of a memory pool may not be similarly reduced.
[0047] According to one or more embodiments of the present disclosure, a system may include a memory pool, the memory pool including a first memory node and a second memory node and a synchronous clock source, the synchronous clock source being connected to both the first memory node and the second memory node to reduce a latency in communication between the first memory node and the second memory node. For example, the first memory node and the second memory node may be configured to encode memory data and decode the encoded memory data using a synchronous clock signal generated by the synchronous clock source.
[0048] Although one or more embodiments of the present disclosure are described in more detail in the context of CXL memory and CXL interfaces and interconnects, the present disclosure is not limited thereto, and the memory may be any suitable type of scalable memory (e.g., such as NVLink / Nvidia-based memory pools) connected via any suitable type of interface or interconnect. The above and / or other aspects and features of the present disclosure will be described in more detail below with reference to the accompanying drawings.
[0049] Figure 1 1 is a schematic block diagram of an expandable memory system 100 according to one or more embodiments of the present disclosure. In brief overview, the expandable memory system 100 may include a synchronous clock source 102 connected to various suitable forms of memory that may constitute a memory pool 104. The memory pool 104 may include a group of memory resources.
[0050] Various forms of memory may include high-speed, volatile, computer memory generally referred to as "memory", for example, each of the various forms of memory may include dynamic random access memory (DRAM) or static random access memory (SRAM) technology. In contrast, slower, non-volatile computer storage devices are generally referred to as "storage devices". For example, storage devices may use storage technologies including, but not limited to, read-only memory (ROM), erasable programmable read-only memory (EPROM), hard disk drive (HDD), and flash memory. As will be described in more detail below, the synchronous clock source 102 may generate a synchronous clock signal to ensure lower latency communication (e.g., memory-to-memory communication) between the memory nodes of the memory pool 104. As will be understood by those of ordinary skill in the art, the memory nodes of the memory pool may include any suitable form of memory.
[0051] For example, one form of storage may include a “rack” of storage (such as, storage rack 106). Storage rack 106 may accommodate a large amount of storage, and may also include components other than storage. For example, storage rack 106 may include networking devices, power devices, persistent storage devices (e.g., hard disk drives, solid-state drives, etc.), processing devices, cooling devices, switching devices, suitable combinations thereof, and the like. Storage rack 106 may have different sizes and shapes for accommodating different amounts of storage. Storage rack 106 may be useful in applications where large amounts of data are required or desired (e.g., in a data center with distributed computing resources). In Figure 1 In the embodiment of the present invention, the storage rack 106 includes one or more storage pods 108a, 108b as another form of storage.
[0052] Similar to memory racks, memory bays (e.g., 108a, 108b) can hold a large amount of memory and can include components other than memory (such as, for example, one or more of those listed above with respect to memory racks). However, memory bays (e.g., 108a, 108b) typically hold less memory than memory racks (e.g., 106). For example, memory bays can be of different sizes and shapes, such that memory bay 108a is smaller and holds less memory than memory bay 108b. Figure 1 In the embodiment of the present invention, the storage compartments 108a, 108b include one or more storage sleds 110 as another form of storage.
[0053] A memory chute (e.g., 110) may hold a moderate amount of memory and may include components other than memory. For example, in addition to memory components, a memory chute may include a central processing unit (CPU) and networking components. Figure 1 In the embodiment, each memory chute 110 may include one or more memory modules 112 as another form of memory. The memory chute may include a printed circuit board on which one or more memory modules 112 and other components (eg, a CPU) are mounted.
[0054] Compared to other forms of memory described above, a memory module (e.g., a memory chip) can accommodate a relatively small amount of memory. Typically, a memory module can have a variety of form factors and can be a package including one or more memory dies (e.g., a package including a printed circuit board and components connected to the printed circuit board, or a housing including a printed circuit board), each memory die including a plurality of memory cells. Each memory die or each of the groups of multiple groups of memory dies can be in a package (e.g., an epoxy molding compound (EMC) package) soldered to a printed circuit board of the memory module (or connected to a printed circuit board of the memory module via a connector). Each memory module can have a CXL interface and can include a controller (e.g., an FPGA, an ASIC, a processor, etc.) for converting between a CXL package and a memory interface of the memory die (e.g., a signal suitable for the memory technology of the memory in the memory module). As used herein, a "memory interface" of a memory die is an interface native to the technology of the memory die, for example, in the case of a DRAM, the memory interface can be a word line and a bit line.
[0055] As mentioned above Figure 1 As described, a memory node may include more than one memory rack, a separate memory rack, a memory cabin, a memory chute, a memory module, a memory die, or a memory unit. Additionally, a synchronous clock source 102 may be connected to a memory node, whether the memory node includes multiple memory racks, a separate memory rack, a memory cabin, a memory chute, a memory module, a memory die, or a memory unit. In some embodiments, the synchronous clock source 102 is connected to multiple memory nodes. In some embodiments, the synchronous clock source 102 is connected to multiple different forms of memory (e.g., racks and modules).
[0056] Figure 2 2 is a schematic block diagram of a synchronous clock source 202 connected to a first memory node 210 and a second memory node 220 according to one or more embodiments of the present disclosure. The synchronous clock source 202 may include a clock synthesizer 232 and a clock buffer 234. In operation, the synchronous clock source 202 generates one or more synchronous clock signals that may be distributed to various devices including the first memory node 210 and the second memory node 220.
[0057] In the illustrated embodiment, the clock synthesizer 232 of the synchronous clock source 202 can generate one or more synchronous clock signals having a suitable (e.g., predetermined or specific) frequency and phase relationship. However, as will be appreciated by one of ordinary skill in the art, the synchronous clock signals generated by the clock synthesizer 232 are not limited to any one frequency, frequency range, period, phase, duration, polarity, duty cycle, amplitude, etc. The clock synthesizer 232 can include a reference clock signal, which the clock synthesizer 232 can use to generate one or more synchronous clock signals. In some embodiments, the reference clock signal can be generated internally in the clock synthesizer 232. For example, the reference clock signal can be generated by a crystal oscillator or an atomic clock. Additionally or alternatively, in some embodiments, the reference clock signal can be received by the clock synthesizer 232 from an external source.
[0058] In some example operations, the clock synthesizer 232 may use a reference clock to generate a desired output clock signal having a desired frequency and / or phase using a phase-locked loop control system. The phase-locked loop control system may include any suitable type of phase-locked loop control (including an analog or linear phase-locked loop, a digital phase-locked loop, a fully digital phase-locked loop, a software phase-locked loop, a charge pump phase-locked loop, etc.). In some embodiments, a delay-locked loop control system is used to generate the desired output clock. By using a phase-locked loop control system or a delay-locked loop control system, the clock synthesizer 232 can generate a more accurate clock that is phase-consistent with the phase of the reference clock.
[0059] In some embodiments, the clock synthesizer 232 does not use a phase-locked loop or delay-locked loop control system. Alternatively, the clock synthesizer 232 may use other suitable devices to generate a more accurate clock and maintain the accuracy of the clock. For example, in some embodiments, the clock synthesizer may use a crystal oscillator, an atomic clock, etc. The clock synthesizer 232 may also include a clock multiplier to generate a clock signal with a higher frequency than the reference clock signal or other generated clock signals.
[0060] exist Figure 2 In the embodiment shown in , the synchronous clock source 202 also includes a clock buffer 234. The clock buffer 234 can be configured to amplify the clock signal generated by the clock synthesizer 232 and / or distribute the clock signal generated by the clock synthesizer 232 to other components. Figure 2In some embodiments, the clock buffer 234 can enable the synchronous clock signal generated by the clock synthesizer 232 to reach both the first memory node 210 and the second memory node 220. In addition, in some embodiments, the synchronous clock signal generated by the clock synthesizer 232 can be amplified before being distributed to other devices (such as the first memory node and the second memory node). In some embodiments, additional clock buffer circuits separate from the synchronous clock source 202 can be used to amplify and / or distribute the synchronous clock signal. The additional clock buffer circuits can help ensure that the synchronous clock signal does not degrade over longer distances (e.g., between racks in a data center).
[0061] Still refer to Figure 2 , the first memory node 210 and the second memory node 220 are connected to or otherwise in communication with the synchronous clock source 202. In an example operation, the first memory node 210 and the second memory node 220 receive a synchronous clock signal from the synchronous clock source 202. The synchronous clock signal may be used by the first memory node 210 and the second memory node 220 to facilitate memory communications therebetween.
[0062] As mentioned above Figure 1 As described, the memory nodes (such as the first memory node 210 and the second memory node 220) may include any suitable form of memory (including but not limited to a memory rack, a memory cabin, a memory chute, a memory module, a memory die, or a memory cell). Figure 2 In an embodiment of the present invention, the first memory node 210 includes a first controller 212, a first link 214 for sending and receiving memory data, and a first memory 216. Similarly, the second memory node 220 includes a second controller 222, a second link 224 for sending and receiving memory data, and a second memory 226. The first memory 216 and the second memory 226 may include physical memories (such as circuits including DRAM and / or SRAM). However, the first controller 212, the second controller 222, the first link 214 and the second link 224 may include more complex circuit systems. For example, in some embodiments, the first controller 212 and the second controller 222 include memory controllers that can perform memory operations (including but not limited to address translation, data transfer, memory timing and control, caching, error detection and correction, memory configuration and management, etc.). In some embodiments, the first link 214 is integrated with the first controller 212, and the second link 224 is integrated with the second controller 222.
[0063] The first memory node 210 and the second memory node 220 can communicate with each other through the switch fabric 230. In general, the switch fabric 230 operates to connect one or more devices (such as memory nodes) with one or more other devices (such as other memory nodes) while reducing the latency between them. In some embodiments, the switch fabric 230 may include one or more CXL switches and associated devices. Therefore, in such embodiments, the switch fabric 230 may be defined as including a compute express link (CXL) interconnect. The CXL switch may (i) support memory and accelerator deaggregation through a single-stage switch, (ii) enable resources to be offline and online between domains, which may ensure time multiplexing across domains based on demand, and (iii) support virtualization of downstream ports. CXL may be employed to implement aggregated memory, which may ensure one-to-many and many-to-one switching. Various suitable switching topologies (such as typical tree topologies and mesh topologies) may be used to scale up / down memory pool sizes. In some embodiments, CXL memory pooling with a toroidal topology foundation may be used to reduce latency and provide latency manageability for memory nodes. CXL switches are further discussed in US 1 1 ,461 ,263, which is incorporated herein by reference in its entirety.
[0064] The switch fabric 230 may also include a fabric manager. The fabric manager may be employed to (i) perform device discovery and virtual CXL software creation, and (ii) bind virtual ports to physical ports. The fabric manager may be implemented in hardware, software, firmware, or a suitable combination thereof, and may reside, for example, in one of the memory nodes, or in a switch, or elsewhere in the network. The fabric manager may issue commands to the various components of the switch fabric 230 to perform the operations described above.
[0065] exist Figure 2 , a switch fabric 230 is configured to connect a first memory node 210 to a second memory node 220. By connecting the first memory node 210 to the second memory node 220, a first memory 216 of the first memory node 210 may be aggregated with a second memory 226 of the second memory node 220. For example, when connected via the switch fabric 230, the first memory 216 may send memory data to the second memory 226. Similarly, the second memory 226 may send memory data to the first memory 216. Transmitting memory data between the first memory node 210 and the second memory node 220 may also ensure that the first memory 216 and the second memory 226 maintain cache coherence.
[0066] In order to transmit memory data from the first memory node 210 to the second memory node 220, the first memory node 210 may include an encoder and the second memory node 220 may include a decoder. The encoder may encode the memory data to serialize the memory data before the memory data is transmitted, and the decoder may decode the received encoded memory data to deserialize / deserialize the memory data. Similarly, in order to transmit memory data from the second memory node 220 to the first memory node 210, the second memory node 220 may also include an encoder and the first memory node 210 may also include a decoder. Figure 2 In the example of, both the first controller 212 and the second controller 222 may include or may be configured to implement an encoder and a decoder. Additionally or alternatively, in some embodiments, the first link 214 and the second link 224 may include or may be configured to implement an encoder and a decoder.
[0067] According to one or more embodiments of the present disclosure, Figure 2 As shown in , CDR is not used, which can save energy and reduce latency. To avoid CDR, the first memory node 210 and the second memory node 220 can use a synchronous clock signal to both encode and decode the memory data. Figure 2 As shown in , the synchronous clock source 202 generates a synchronous clock signal, which is sent to both the first memory node 210 and the second memory node 220 for encoding and decoding memory data. In some embodiments, both the first controller 212 and the second controller 222 encode and decode the memory data. Additionally or alternatively, in some embodiments, both the first link 214 and the second link 224 encode and decode the memory data. The first memory node 210 can encode the memory data using the synchronous clock signal generated by the synchronous clock source 202. Once encoded, the first memory node 210 can send the encoded data to the second memory node 220 via the switching structure 230. Then, the second memory node 220 can receive the encoded memory data and decode the encoded memory data using the synchronous clock signal (e.g., the same synchronous clock signal) generated by the synchronous clock source 202. By having a synchronous clock connected to both the sending (e.g., encoding) side and the receiving (e.g., decoding) side, clock data recovery can be avoided due to the encoding and decoding sharing the same clock signal. Therefore, the clock signal used to encode the memory data may be the same as the clock signal used to decode the memory data.
[0068] Figure 3A3 is a schematic block diagram of a memory pool controller 336 connected to CXL memories 338a, 338b, 338c via a switch fabric 330 including a compute express link (CXL) interconnect according to one or more embodiments of the present disclosure. The memory pool controller 336 (e.g., also referred to as a memory bank controller) may include any suitable type of processor or processing circuit (including, but not limited to, a system on a chip (SoC), a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc.). The memory pool controller 336 may optionally be connected to one or more dual in-line memory modules (DIMMs) 342, which may include traditional memory modules (e.g., without CXL). The DIMMs may be used by the memory pool controller 336 as a cache to store instructions, etc., for execution by the memory pool controller 336.
[0069] In operation, the memory pool controller 336 may control aspects of the CXL memory 338a (including, but not limited to, address translation, data transfer, memory timing and control, caching, error detection and correction, memory configuration and management, etc.). For example, the memory pool controller 336 may allocate CXL memories connected to the memory pool controller 336 via the switch fabric 330. The switch fabric 330 includes an interconnect based on a cache coherence protocol (e.g., such as a CXL interconnect). In some embodiments, the CXL interconnect includes one or more CXL switches and one or more CXL ports 346 connected to the memory pool controller 336. The CXL memories 338a, 338b, 338c may include memory nodes (e.g., the first memory node 210, the second memory node 220, etc.). Any suitable number of CXL memories 338a, 338b, 338c may be connected to the memory pool controller 336 via the switch fabric 330. For example, each CXL port 346 connected to the memory pool controller 336 may be connected to a separate CXL memory via the switch fabric 330. Additionally or alternatively, more than one CXL port 446 may be connected to each CXL memory via switch fabric 330 .
[0070] exist Figure 3A In the embodiment, CXL memories 338a, 338b, 338c can be combined into a memory pool. Figure 3AThe illustrated embodiment includes a synchronous clock source 302 connected to each of the CXL memories 338a, 338b, 338c. In operation, the synchronous clock source 302 generates a synchronous clock signal that is used by each of the CXL memories 338a, 338b, 338c to reduce communication latency between them when they are aggregated together as a memory pool. Thus, the memory pool controller 336 can use the CXL memories 338a, 338b, 338c as a memory pool with an overall reduced latency when compared to using the CXL memories 338a, 338b, 338c as a memory pool without the synchronous clock source 302.
[0071] In some embodiments, the memory pool controller 336 may be configured to allocate CXL memories 338a, 338b, 338c. For example, a host processor connected to the memory pool controller 336 may request (e.g., may require) a specific amount of memory to run an application. In some such embodiments, the memory pool controller 336 may allocate a specific amount of CXL memory (e.g., 338a, 338b, 338c) by enabling a specific CXL port 346 associated with the specific amount of CXL memory. In one embodiment, the memory pool controller 336 may be configured to allocate memory from at least one of the CXL memories (e.g., 338a, 338b, 338c) by enabling an I / O connection (e.g., CXL port 346) to communicate with at least one of the CXL memories (e.g., 338a, 338b, 338c).
[0072] In some embodiments, the memory pool controller 336 may also be used to ensure cache coherence within a memory pool including CXL memories (e.g., 338a, 338b, 338c). For example, when there is an inconsistency within the CXL memory (e.g., between memory nodes of the CXL memory pool), the memory pool controller may be used to resolve the inconsistency or request updated memory data from a connected host processor.
[0073] Figure 3B is a schematic block diagram of a system 300 including a plurality of memory pool controllers 336a, 336b connected to memory according to one or more embodiments of the present disclosure. Figure 3B A first memory pool controller 336a is included which is connected to the first CXL memory 350a via the first switch fabric 330a. The first memory pool controller 336a is also connected to one or more first input / output connections (I / O) 328a. Figure 3BAlso included is a second memory pool controller 336b connected to the second CXL memory 350b via the second switch fabric 330b. The second memory pool controller 336b is also connected to one or more second input / output connections (I / O) 328b. In some embodiments, the memory pool controllers 336a, 336b communicate with the host processor via their respective I / Os 328a, 328b.
[0074] In some example operations, the first memory pool controller 336a may allocate and manage the first CXL memory 350a so that the host processor (or processors) can use a desired amount of the first CXL memory 350a. In such example operations, the host processor (or processors) may communicate with the first memory pool controller 336a through the first I / O 328a. For example, in some embodiments, the host processor (or processors) may request an amount of I / O to be used, wherein the number of I / Os determines how much CXL memory the host processor can access. Similarly, the second memory pool controller 336b may allocate and manage the second CXL memory 350b so that the host processor (or processors) can access and use a desired amount of the second CXL memory 350b. In such example operations, the host processor (or processors) may communicate with the second memory pool controller 336b through the second I / O 328b.
[0075] In some example operations, the first memory pool controller 336a and the second memory pool controller 336b may be configured as described above with reference to Figure 3A The first memory pool controller 336a can communicate with the second memory pool controller 336b via the first switch fabric 330a and the second switch fabric 330b. In some embodiments, the first switch fabric 330a and the second switch fabric 330b can be part of a single switch fabric. In such an embodiment, the first memory pool controller 336a can communicate with the second memory pool controller 336b via a single switch fabric.
[0076] In some embodiments, the first memory pool controller 336a may allocate / manage the first CXL memory 350a and / or the second CXL memory 350b. In some such embodiments, the first memory pool controller 336a may communicate with the second CXL memory 350b via the first switch fabric 330a and the second switch fabric 330b. Similarly, in some embodiments, the second memory pool controller 336b may allocate / manage the first CXL memory 350a and / or the second CXL memory 350b. In some such embodiments, the second memory pool controller 336b may communicate with the first CXL memory 350a via the first switch fabric 330a and the second switch fabric 330b.
[0077] Figure 3B The embodiment also includes a synchronization clock source 302 configured to generate and send a synchronization clock signal to the first CXL memory 350a and the second CXL memory 350b. The first CXL memory 350a and the second CXL memory 350b may include separate memory nodes, and each separate memory node may receive the synchronization clock signal. The synchronization clock signal may help reduce communication latency between memory nodes (such as between separate memory nodes constituting the first CXL memory 350a and / or between separate memory nodes constituting the second CXL memory 350b).
[0078] In some embodiments, the first CXL memory 350a and the second CXL memory 350b are aggregated into a single memory pool, which may be allocated / managed by the first memory pool controller 336a and / or the second memory pool controller 336b. The synchronous clock signal generated by the synchronous clock source 302 may be used to reduce communication latency between CXL memories (e.g., memory nodes) constituting the memory pool.
[0079] In some embodiments, the first CXL memory 350a may communicate with the second CXL memory 350b via one or more switch fabrics (eg, 330a, 330b) to maintain cache coherency between the first CXL memory 350a and the second CXL memory 350b.
[0080] Although Figure 3B The embodiments show an example of two memory pool controllers connected to a switch fabric, CXL memory, and I / O, but the disclosure is not limited thereto, and any suitable number of memory pool controllers, I / O, switch fabric, and CXL memory may be included as needed or desired.
[0081] Figure 4A 4 is a schematic block diagram of a heterogeneous processor (XPU) 444 connected to CXL memories 438a, 438b, 438c via a switch fabric 430 including a compute express link (CXL) interconnect according to one or more embodiments of the present disclosure. The heterogeneous processor (XPU) 444 may include any suitable type of processor or processing circuit, including but not limited to a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a neural processing unit (NPU), an accelerated processing unit (APU), a visual processing unit (VPU), a SoC, etc.
[0082] The XPU 444 may be connected to one or more dual in-line memory modules (DIMMs) 442, which may include conventional memory modules (e.g., without CXL). The DIMMs may be used as processor memory for the XPU and may store instructions that, when executed by the XPU, cause the XPU to run applications. In some embodiments, the DIMMs may be used to store information generated by the application. However, in some embodiments, the application may desire (e.g., may require) additional memory that exceeds the capacity of the DIMMs. Therefore, the XPU 444 is connected to the CXL memories 438a, 438b, 438c via a switching fabric 430 that includes an interconnect (such as a CXL interconnect) based on a cache coherence protocol. Although the CXL memories 438a, 438b, 438c may have a relatively higher latency than the DIMMs (the DIMMs may be physically closer to the XPU), the CXL memories 438a, 438b, 438c may include a relatively larger capacity (e.g., a much larger capacity) of memory.
[0083] In some embodiments, the CXL interconnect includes a connection to XPU 444 (or Figure 3A The XPU 444 may include one or more CXL switches (e.g., memory controllers in the memory controller 430) and one or more CXL ports 446. The CXL memories 438a, 438b, 438c may include memory nodes (e.g., the first memory node 210, the second memory node 220, etc.). Any suitable number of CXL memories 438a, 438b, 438c may be connected to the XPU 444 via the switch fabric 430. For example, each CXL port 446 connected to the XPU 444 may be connected to a separate CXL memory via the switch fabric 430. Additionally or alternatively, more than one CXL port 446 may be connected to each CXL memory via the switch fabric 430.
[0084] exist Figure 4A In the embodiment, CXL memories 438a, 438b, 438c can be combined into a memory pool. Figure 4A The illustrated embodiment of includes a synchronous clock source 402 connected to each CXL memory 438a, 438b, 438c. In operation, the synchronous clock source 402 generates a synchronous clock signal that is used by each of the CXL memories 438a, 438b, 438c to reduce communication latency between them when they are aggregated together as a memory pool. Thus, the XPU 444 can use the CXL memories 438a, 438b, 438c as a memory pool with an overall reduced latency when compared to using the CXL memories 438a, 438b, 438c as a memory pool without the synchronous clock source 402.
[0085] In some embodiments, XPU 444 may include additional functionality. For example, XPU 444 may be configured to allocate CXL memory 438a, 438b, 438c based on its need for memory or based on the need for memory of another connected processor. In some such embodiments, XPU 444 may allocate CXL memory 438a, 438b, 438c by enabling CXL port 446 associated with the desired amount of CXL memory. In this way, XPU 444 may dynamically allocate or compose an amount of CXL memory to be used by XPU 444 and / or by another connected processor.
[0086] Figure 4B is a schematic block diagram of a system 400 including multiple heterogeneous processors (XPUs) 444a, 444b connected to a memory according to one or more embodiments of the present disclosure. In general, Figure 4B The embodiment shown includes Figure 4A Multiple heterogeneous processors and Figure 4A A system in which heterogeneous processors are connected to CXL memory. Figure 4B The first XPU 444a is connected to a first CXL memory 450a through a first switch fabric 430a. The first XPU 444a is also connected to a first DIMM 442a and a first network interface card (NIC) 448a. Figure 4B Also included is a second XPU 444b connected to a second CXL memory 450b via a second switch fabric 430b. The second XPU 444b is also connected to a second DIMM 442b and a second network interface card (NIC) 448b.
[0087] In operation, the first XPU 444a and the second XPU 444b may communicate with each other to run applications and may be used as an aggregated computing resource. The first XPU 444a and the second XPU 444b may communicate with each other through their respective NICs 448a, 448b. Figure 4B In some embodiments, the first XPU 444a and the second XPU 444b may communicate with each other through their respective NICs 448a, 448b using a serial interface, such as Peripheral Component Interconnect Express (PCIe). However, in some embodiments, the first XPU 444a and the second XPU 444b may communicate with each other through their respective NICs 448a, 448b using another suitable interconnect, such as Ethernet. In some embodiments, the first NIC 448a may be integrated with the first XPU 444a, and the second NIC 448b may be integrated with the second XPU 444b.
[0088] In addition, Figure 4B, first switch fabric 430a may be connected to second switch fabric 430b. Alternatively, in some embodiments, first switch fabric 430a and second switch fabric 430b may include a single switch fabric. Thus, due to the connection between first switch fabric 430a and second switch fabric 430b, first CXL memory 450a may be used by both first XPU 444a and second XPU 444b. For example, in some embodiments, if one or both of the first XPU 444a and the second XPU 444b are running an application that desires (e.g., requires) to use CXL memory (e.g., due to large memory requirements), the first XPU 444a may store / access memory data in the first CXL memory 450a and / or the second CXL memory 450b, and the second XPU 444b may also store / access memory data in the first CXL memory 450a and / or the second CXL memory 450b. Furthermore, in some embodiments, the first CXL memory 450a may communicate with the second CXL memory 450b via one or more switch fabrics (e.g., 430a, 430b) to maintain cache coherence between the first CXL memory 450a and the second CXL memory 450b.
[0089] Figure 4B The embodiment also includes a synchronization clock source 402 configured to generate and send a synchronization clock signal to the first CXL memory 450a and the second CXL memory 450b. The first CXL memory 450a and the second CXL memory 450b may include separate memory nodes, wherein each separate memory node is capable of receiving the synchronization clock signal. The synchronization clock signal may help reduce communication latency between memory nodes (such as between separate memory nodes constituting the first CXL memory 450a and / or between separate memory nodes constituting the second CXL memory 450b).
[0090] In some embodiments, first CXL memory 450a and second CXL memory 450b are aggregated into a single memory pool accessible by first XPU 444a and / or second XPU 444b. The synchronous clock signal generated by synchronous clock source 402 may be used to reduce communication latency between CXL memories (eg, memory nodes) constituting the memory pool.
[0091] Although Figure 4BThe illustrated embodiment of includes an example of two XPUs connected to NICs, DIMMs, switch fabrics, and CXL memories, but the disclosure is not limited thereto and any suitable number of XPUs, NICs, DIMMs, switch fabrics, and CXL memories may be included.
[0092] In some embodiments, the dashed lines between CXL memory nodes shown in the figures indicate direct communication between CXL memory nodes using, for example, a cache coherence protocol-based interconnect, such as an inter-switch link (ISL) based on a CXL interface. Using ISLs can reduce latency to less than 1 microsecond (μs) (such as, for example, to about 600 nanoseconds (ns) to 900ns). However, the present disclosure is not limited thereto, and in some embodiments, remote direct memory access (RDMA) can be used for direct communication between CXL memory nodes, or other network-based protocols can be used for direct communication between CXL memory nodes if their longer latency can be tolerated.
[0093] Figure 5 is a schematic block diagram of an expandable memory pool 564 in communication with a host processor according to one or more embodiments of the present disclosure.
[0094] Reference Figure 5 , the host device 552 may include an operating system / kernel 554, a host processor 556, a host memory 558, and a storage device 560. The operating system / kernel 554 may include system software for providing an interface between the hardware and the user and between the software application and the hardware. For example, the operating system / kernel 554 may be configured for resource allocation, memory management, CPU management, file management, execution of processes, etc. of the host device 552. For example, in some embodiments, the operating system / kernel 554 may include a Linux operating system / kernel, but the present disclosure is not limited thereto, and the operating system / kernel 554 may include any suitable operating system / kernel as will be known to those skilled in the art (such as, Windows OS, Apple OS (e.g., macOS), Chrome OS, etc.).
[0095] The host processor 556 may be a processing circuit (e.g., a general purpose processor or central processing unit (CPU) core such as the host device 552). The host processor 556 may be connected to other components via an address bus, a control bus, a data bus, etc. The host processor 556 may execute instructions stored in the host memory 558 to perform various operations described herein. For example, the host processor 556 may execute one or more system processes and background processes (which are described in more detail below), which may be copied from a persistent storage device (e.g., storage device 560, read-only memory (ROM), etc.) to the host memory 558 as needed or desired (e.g., at startup, execution time, interrupt routines, etc.).
[0096] The host memory 558 may be considered as a high-performance main memory (e.g., main memory) of the host device 552. For example, in some embodiments, the host memory 558 may include (or may be) a volatile memory (such as a dynamic random access memory (DRAM) that may be directly connected to a memory slot of a motherboard of the host device 552 via the first memory interface 562). In this case, the first memory interface 562 (e.g., a connector and its protocol) may include (or may conform to) a dual in-line memory module (DIMM) to facilitate communication between the host memory 558 and the host processor 556 (e.g., via the host OS / kernel 554), so that the host memory 558 may be a DIMM memory connected to a DIMM slot of the host device 552. However, the present disclosure is not limited thereto, and the host memory 558 may include (or may be) any suitable high-performance main memory (e.g., main memory) alternative for the host device 552 as will be known to those skilled in the art. For example, in other embodiments, the host memory 558 may be a relatively high-performance non-volatile memory (such as, NAND flash memory, phase change memory (PCM), resistive RAM, spin transfer torque RAM (STTRAM), any suitable memory based on PCM technology, memristor technology, and / or resistive random access memory (ReRAM), and may include, for example, chalcogenides, etc.
[0097] The storage device 560 may be considered as an auxiliary memory (e.g., an auxiliary storage device) that can persistently store data accessible to the host device 552. In this context, the storage device 560 may include (or may be) a relatively slow memory when compared to the high-performance main memory of the host memory 558. For example, in some embodiments, the storage device 560 may be a solid-state drive (SSD). However, the present disclosure is not limited thereto, and in other embodiments, the storage device 560 may include (or may be) any suitable storage device (e.g., such as a magnetic storage device (e.g., a hard disk drive (HDD)), an optical storage device (e.g., a Blu-ray drive, a compact disk (CD) drive, a digital versatile disk (DVD) drive, etc.), other kinds of flash memory devices (e.g., a USB flash drive, etc.), etc.). In various embodiments, the storage device 560 may conform to a large form factor standard (e.g., a 3.5-inch hard disk drive form factor), a small form factor standard (e.g., a 2.5-inch hard disk drive form factor), an M.2 form factor, an E1.S form factor, etc. In other embodiments, the storage device 560 may conform to any suitable or desired derivative of these form factors.
[0098] The storage device 560 may be connected to the host processor 556 via a storage interface. The storage interface may facilitate communication between the host processor 556 and the storage device 560 (e.g., via the host OS / kernel 554) (e.g., using a connector and a protocol). In some embodiments, the storage interface may facilitate the exchange of storage requests and responses between the host processor 556 and the storage device 560. In some embodiments, the storage interface may facilitate data transfer from the storage device 560 to and from the host memory 558 of the host device 552. For example, in various embodiments, the storage interface (e.g., a connector and its protocol) may include (or may conform to) a small computer system interface (SCSI), non-volatile memory express (NVMe), peripheral component interconnect express (PCIe), remote direct memory access (RDMA) over Ethernet, serial advanced technology attachment (SATA), Fibre Channel, serial attached SCSI (SAS), NVMe over a network (NVMe-oF), etc. In other embodiments, the storage interface (e.g., a connector and its protocol) may include (or may conform to) various general interfaces (e.g., such as Ethernet, universal serial bus (USB), etc.).
[0099] Still refer to Figure 5, the host device 552 is connected to the scalable memory pool 564 via a second memory interface 566 different from the first memory interface 562. The scalable memory pool 564 may include one or more scalable memory devices 538a, 538b, 538c (e.g., such as one or more compute express link (CXL) memory devices 538a, 538b, 538c). In some embodiments, the scalable memory pool 564 may be a deaggregated CXL memory pool including multiple different types of CXL memory devices 538a, 538b, 538c, which may generally include volatile memory (such as, for example, DDR3, DDR4, DDR5, low power, high power, low profile, PMEM, HBM, DRAM among SSDs with DRAM, etc.). However, the present disclosure is not limited thereto, and similar to the example described above for the host memory 558, the scalable memory pool 564 may include (or may be) any suitable high-performance scalable memory for the host device 552 as will be known to those skilled in the art.
[0100] exist Figure 5 In the illustrated embodiment of , the second memory interface 566 (e.g., a connector and its protocol) may include (e.g., may conform to) an interconnect based on a cache coherence protocol (e.g., a CXL interconnect) as part of a switch fabric. In some embodiments, the scalable memory pool 564 is a network-attached scalable memory pool. In some such embodiments, the scalable memory pool 564 may be connected to the host device 552 via a processor / controller 570, which may include a network interface controller (NIC). The processor / controller 570 may include a memory pool controller (e.g., Figure 3A 4 ), heterogeneous processors (e.g., XPU 444 of FIG. 4 ), etc. In addition, the host device 552 can communicate with the expandable memory pool 564 using the processor / controller 570 and the NIC via a suitable communication network (e.g., the Internet, a wide area network, a local area network, a cellular network, etc.).
[0101] In some embodiments, the second memory interface 566 (e.g., a connector and its protocol) may include (e.g., may conform to) a cache coherence protocol-based interconnect (e.g., a CXL interconnect) built on a serial interface (e.g., Peripheral Component Interconnect Express (PCIe)) to facilitate communication between the host device 552 (e.g., via the host OS / kernel 554) and the CXL memory devices 538a, 538b, 538c of the expandable memory pool 564. In such embodiments, each of the CXL memory devices 538a, 538b, 538c may be connected to a PCIe slot of the host device 552 as a PCIe device.
[0102] In order to reduce latency between the CXL memory devices 538a, 538b, 538c and latency between the host device 552 and the CXL memory devices 538a, 538b, 538c, the system 500 may also include a synchronous clock source 502. As discussed above, the synchronous clock source 502 may generate a synchronous clock signal that is provided to each of the CXL memory devices 538a, 538b, 538c. The synchronous clock signal may ensure that the entire scalable memory pool 564 may be accessed as if the scalable memory pool 564 were a single memory device. In more detail, by using a synchronous clock signal, the need for clock data recovery (CDR) between the CXL memory devices 538a, 538b, 538c may be eliminated, and therefore, latency between the CXL memory devices may be significantly reduced, and the overall latency between the host device 552 and the scalable memory pool 564 may be reduced.
[0103] Although Figure 5 An example of one host device 552 connected to the expandable memory pool 564 is shown, but the present disclosure is not limited thereto, and a plurality of host devices 552 may be connected to the expandable memory pool 564 .
[0104] Figure 6 1 is a schematic block diagram of a synchronous clock source connected to a memory interface of different forms according to one or more embodiments of the present disclosure. Figure 1 and Figure 2 As described, the synchronous clock source 602 can be connected to multiple different forms of memory, including memory modules 612, memory chutes 610, memory bays 608, and memory racks 605, 606. In addition, the synchronous clock source can reduce the latency of accessing each form of memory when compared to not using a synchronous clock source. For example, when a synchronous clock source is not provided, each of the multiple different forms of memory can use or require clock data recovery in order to encode / decode data, which can introduce latency into the system.
[0105] Therefore, in some embodiments, because the delay associated with clock data recovery that varies depending on the form of the memory can be prevented or substantially reduced, each form of memory can have a relatively predictable maximum delay associated with writing / accessing memory data within the form of memory. In some embodiments, the predictable maximum delay can be within a predetermined threshold (e.g., such as 1 microsecond) (e.g., less than or equal to the predetermined threshold (e.g., such as 1 microsecond)). For example, in some embodiments, the maximum delay between the processor and the memory module 612 can be about 50 nanoseconds or less. In some embodiments, the maximum delay between the processor and the memory chute can be about 100 nanoseconds or less. In some embodiments, the maximum delay between the processor and the memory compartment can be about 200 nanoseconds or less. In some embodiments, the maximum delay between the processor and the memory rack can be about 300 nanoseconds or less. In some embodiments, the maximum delay between the processor and multiple racks can be about 900 nanoseconds or less.
[0106] As will be understood by those of ordinary skill in the art, as the capacity of memory increases in each form of memory, latency also increases. Therefore, in some embodiments, the form of memory may be grouped differently according to their predictable aggregate maximum latency and expected maximum latency (e.g., based on a service level agreement SLA). For example, a first group of memory may include various memory modules with relatively low latency. In addition, a second group of memory may include various memory chutes with higher latency than memory modules. A third group of memory may include various memory cabins with higher latency than memory chutes. A fourth group of memory may include a separate memory rack with a higher latency than a memory cabin. A fifth group of memory may include multiple memory racks with a higher latency than a separate memory rack. One or more host processors may (e.g., by service) request a desired group (e.g., a specific or predetermined group) of memory that meets their memory requirements. For example, a host processor may request a memory group with a maximum latency within a threshold limit, or may request a memory group with an expected (e.g., specific or predetermined) memory capacity.
[0107] Figure 7 is a flowchart of a method of encoding and decoding data in a storage node of a storage pool according to one or more embodiments of the present disclosure.
[0108] Method 700 may begin, and at block 705, memory data is stored in a first memory node. Figure 1As described, a memory node may include any suitable form of memory, including multiple memory racks, a single memory rack, one or more memory pods, one or more memory chutes, one or more memory modules, one or more memory dies, or one or more memory cells. Figure 2 As described, in addition to the physical memory, the memory node may include a memory controller and a link. At box 710, memory data is encoded at a first memory node using a synchronous clock signal. At box 715, the encoded memory data is sent to a second memory node via a switching structure. In some embodiments, the switching structure includes a compute express link (CXL) interconnect. At box 720, the encoded memory data is received at the second memory node via the switching structure. At box 725, the encoded memory data is decoded at the second memory node using a synchronous clock signal (e.g., the same synchronous clock signal), and method 700 may end. The decoded memory data may then be obtained (e.g., read) from the second memory node as needed or desired for further processing (e.g., by a host, a memory pool controller, a general processing unit, another memory node, etc.). In some embodiments, a system including at least a first memory node and a second memory node may execute Figure 7 method.
[0109] However, the present disclosure is not limited to Figure 7 The order or number of operations of the method 700 shown in the embodiment of the present invention may be changed to any desired order or number of operations as recognized by those of ordinary skill in the art. For example, in some embodiments, the order may be changed, or the method 700 may include fewer or additional operations.
[0110] According to one or more embodiments of the present disclosure described above, the memory pool may include a first memory node and a second memory node, wherein the first memory node is connected to the second memory node via a switching structure including a computing express line interconnect. According to one or more embodiments of the present disclosure described above, a synchronous clock source may be connected to the first memory node and the second memory node, wherein the synchronous clock source is configured to provide a synchronous clock signal to the first memory node and the second memory node. The first memory node and the second memory node may be configured to encode memory data and decode the encoded memory data using a synchronous clock signal (e.g., the same synchronous clock signal, or in other words, a synchronous clock signal provided from the same synchronous clock source).
[0111] According to one or more embodiments of the present disclosure, because the first memory node and the second memory node use the same synchronous clock signal for both encoding and decoding, any communication between the first memory node and the second memory node can avoid performing clock data recovery to obtain the original clock signal. Therefore, the latency of the communication between the first memory node and the second memory node can be reduced. In addition, computing resources and energy resources can be similarly reduced.
[0112] When a specific embodiment can be implemented differently, the specific processing order may be different from the described order. For example, two consecutively described processes may be performed simultaneously or substantially simultaneously, or may be performed in the reverse order of the described order.
[0113] It will be understood that although the terms "first", "second", "third", etc. may be used herein to describe various elements, components, regions, layers and / or parts, these elements, components, regions, layers and / or parts should not be limited by these terms. These terms are used to distinguish one element, component, region, layer or part from another element, component, region, layer or part. Therefore, without departing from the spirit and scope of the present disclosure, the first element, first component, first region, first layer or first part described below may be referred to as the second element, second component, second region, second layer or second part.
[0114] It will be understood that when an element or layer is referred to as being "on," "connected to," or "coupled to" another element or layer, it can be directly on, connected to, or coupled to the other element or layer, or one or more intervening elements or layers may be present. Similarly, when a layer, region, or element is referred to as being "electrically connected to" another layer, region, or element, it can be directly electrically connected to the other layer, region, or element, and / or can be indirectly electrically connected to one or more intervening layers, regions, or elements therebetween. Additionally, it will be understood that when an element or layer is referred to as being "between" two elements or layers, it can be the only element or layer between the two elements or layers, or one or more intervening elements or layers may also be present.
[0115] The terms used herein are for the purpose of describing a particular embodiment and are not intended to limit the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. It will also be understood that when the terms "include", "comprise", "have" are used in this specification, it is indicated that the features, integers, steps, operations, elements and / or components are present, but one or more other features, integers, steps, operations, elements, components and / or groups thereof are not excluded from being present or added. As used herein, the term "and / or" includes any and all combinations of one or more associated listed items. For example, the statement "A and / or B" means A, B or A and B. Statements such as "at least one of ...", when after a column of elements, modify the entire column of elements, rather than modifying a single element in the column. For example, the statements "at least one of a, b or c", "at least one of a, b and c" and "at least one of the group consisting of a, b and c" represent only a, only b, only c, both a and b, both a and c, both b and c, all of a, b and c or their variants.
[0116] As used herein, the terms "substantially," "about," and similar terms are used as terms of approximation rather than terms of degree, and are intended to take into account the inherent variations in measurements or calculations that one of ordinary skill in the art would recognize. Additionally, the use of "may" when describing embodiments of the present disclosure refers to "one or more embodiments of the present disclosure." As used herein, the term "use" may be considered synonymous with the term "utilizing." Additionally, the term "exemplary" is intended to refer to an example or illustration.
[0117] Any suitable hardware, firmware (e.g., application specific integrated circuit), software, or a combination of software, firmware, and hardware may be used to implement the electronic or electrical devices and / or any other related devices or components according to the embodiments of the present disclosure described herein. For example, the various components of these devices may be formed on an integrated circuit (IC) chip or on separate IC chips. In addition, the various components of these devices may be implemented on a flexible printed circuit film, a tape carrier package (TCP), a printed circuit board (PCB), or formed on a substrate. In addition, the various components of these devices may be processes or threads that run on one or more processors in one or more computing devices, execute computer program instructions, and interact with other system components for performing the various functions described herein. The computer program instructions are stored in a memory, which may be implemented in a computing device using a standard memory device (such as, for example, a random access memory (RAM)). The computer program instructions may also be stored in other non-temporary computer-readable media (such as, for example, a CD-ROM, a flash drive, etc.). In addition, those skilled in the art should recognize that, without departing from the spirit and scope of the exemplary embodiments of the present disclosure, the functions of various computing devices may be combined or integrated into a single computing device, or the functions of a particular computing device may be distributed on one or more other computing devices.
[0118] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those commonly understood by those of ordinary skill in the art to which the present disclosure belongs. It will also be understood that, unless explicitly defined as such herein, terms (such as those defined in general dictionaries) should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and / or in this specification, and should not be interpreted in an idealized or overly formal sense.
[0119] Although some embodiments have been described, it will be readily appreciated by those skilled in the art that various modifications may be made in the embodiments without departing from the spirit and scope of the present disclosure. It will be understood that, unless otherwise described, the description of features or aspects within each embodiment should generally be considered to be applicable to other similar features or aspects in other embodiments. Therefore, it will be clear to those of ordinary skill in the art that, unless otherwise specifically indicated, the features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with the features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, it should be understood that the foregoing is an illustration of various example embodiments and should not be construed as being limited to the specific embodiments disclosed herein, and that various modifications to the disclosed embodiments and other example embodiments are intended to be included within the spirit and scope of the present disclosure as defined in the appended claims and their equivalents.
Claims
1. A system for storing, include: a set of memory resources, including a first memory node and a second memory node, the first memory node being connected to the second memory node via a switch fabric; as well as a synchronous clock source connected to the first memory node and the second memory node, the synchronous clock source being configured to provide a synchronous clock signal to the first memory node and the second memory node to synchronize the first memory node with the second memory node, The first memory node and the second memory node are configured to encode memory data and decode the encoded memory data using a synchronous clock signal.
2. The system according to claim 1, in: The first memory node includes a first controller configured to encode memory data using a synchronous clock signal and send the encoded memory data to the second memory node; and The second memory node includes a second controller configured to receive the encoded memory data from the first memory node and decode the encoded memory data using the synchronous clock signal.
3. The system according to claim 1, further comprising: include: a memory bank controller connected to the first memory node and the second memory node via a switch fabric, the switch fabric including an interconnect based on a cache coherence protocol; as well as Input / output I / O connections, Wherein the memory bank controller is configured to allocate memory from at least one of the first memory node and the second memory node by enabling communication between an I / O connection and the at least one of the first memory node and the second memory node.
4. The system according to claim 1, further comprising: include: a first processor connected to a first memory node and a second memory node of the set of memory resources via a switch fabric and to a first network interface card NIC; as well as a second processor connected to a third memory node and a fourth memory node of the set of memory resources via a second switch fabric and to a second NIC, The first processor is configured to communicate with the second processor through the first NIC or the second NIC.
5. The system according to claim 4, in, The first NIC is connected to the second NIC via a serial interface.
6. The system according to claim 4, in, The third memory node and the fourth memory node are configured to encode memory data and decode the encoded memory data using the synchronous clock signal.
7. The system according to claim 4, in, The first processor includes one of a system on chip SoC, a central processing unit CPU, a graphics processing unit GPU, a tensor processing unit TPU, a field programmable gate array FPGA and an application specific integrated circuit ASIC.
8. The system according to claim 1, in, The first memory node includes one of a memory die, a memory module, a memory chute, a memory pod, and a memory rack, and the second memory node includes one of a memory die, a memory module, a memory chute, a memory pod, and a memory rack.
9. The system according to claim 8, in, The first storage node includes a first storage rack and the second storage node includes a second storage rack, the second storage rack being different from the first storage rack.
10. The system according to claim 1, in, The maximum latency between the first memory node and the second memory node is less than a predetermined threshold.
11. The system according to claim 1, further comprising: include: A processor is in communication with a processor memory storing instructions that, when executed by the processor, cause the processor to execute an application that stores information in the set of memory resources.
12. The system according to any one of claims 1 to 11, in, The first memory node is also connected to the second memory node through a second switch fabric.
13. The system according to any one of claims 1 to 11, in, The set of memory resources are cache coherent.
14. A system for storing, include: A set of memory resources, including: a first memory node comprising a first memory controller, a first memory, and a first cache coherence protocol-based interconnect; and a second memory node including a second memory controller, a second memory, and a second cache coherence protocol-based interconnect, the second memory node being connected to the first memory node via a switch fabric that connects the first cache coherence protocol-based interconnect to the second cache coherence protocol-based interconnect; and a synchronous clock source connected to the first memory node and the second memory node, the synchronous clock source being configured to provide a synchronous clock signal to the first memory node at a first cache coherence protocol-based interconnect and to provide a synchronous clock signal to the second memory node at a second cache coherence protocol-based interconnect to synchronize the first memory node with the second memory node, Wherein, the first memory node and the second memory node are configured to send and receive memory data to and from each other using a synchronous clock signal.
15. The system according to claim 14, in, The first memory node and the second memory node are cache coherent.
16. The system according to claim 14, in, The first memory node and the second memory node are configured to encode and decode memory data transmitted and received from each other using a synchronous clock signal.
17. A system according to any one of claims 14 to 16, in, Synchronous clock sources include: Clock synthesizers, including phase-locked loops; and Clock buffer.
18. The system according to claim 17, in, The clock buffer is connected to the first memory node at a first cache coherence protocol based interconnect and to the second memory node at a second cache coherence protocol based interconnect.
19. A method of encoding and decoding data in a separate memory node of a set of memory resources, include: storing the memory data in the first memory node; encoding memory data at a first memory node using a synchronous clock signal; sending the encoded memory data to a second memory node via a switch fabric; receiving the encoded memory data at a second memory node via a switch fabric; as well as The encoded memory data is decoded at the second memory node using the synchronous clock signal.
20. The method according to claim 19, in: The switch fabric includes an interconnect based on a cache coherence protocol; The first storage node comprises a first storage rack including a first plurality of storage devices; and The second storage node includes a second storage rack including a second plurality of storage devices.
Citation Information
Patent Citations
Node server system
CN101526935A
Time synchronization method, device and system
CN105656590A
Software-defined coherency caching for pooled memory
CN114461544A
Memory pool data placement technologies
US20220004330A1
Cited By
A multi-card NPU interconnection clock system, a multi-card NPU system and a clock control method thereof
CN122837579A
