Optimized command queue with low latency

Abstract

The invention relates to an optimized command queue with low latency. A system can include a memory device; and a processing device operably coupled with the memory device to perform operations including: monitoring a host command stream from a host system to the memory device; determining whether a stage value of a host command in the stream of host commands is valid; replicating the host command in response to determining that the phase value of the host command is valid; storing the replicated host command in a queue in a local memory; and executing the replicated host command in the queue in the local memory.

Description

Technical Field

[0001] Embodiments of the present disclosure relate generally to memory subsystems, and more specifically to implementing an optimized command queue with low latency for an accelerator or other near computing memory. Background Art

[0002] The memory subsystem may include one or more memory devices that store data. The memory devices may be, for example, non-volatile memory devices and volatile memory devices. In general, the host system may utilize the memory subsystem to store data at the memory devices and retrieve data from the memory devices. Summary of the invention

[0003] One aspect of the present disclosure relates to a system comprising: a memory device; and a processing device operably coupled to the memory device to perform operations including: monitoring a host command stream from a host system to the memory device; determining whether a phase value of a host command in the host command stream is valid; replicating the host command in response to determining that the phase value of the host command is valid; storing the replicated host command in a queue in a local memory; and executing the replicated host command in the queue in the local memory.

[0004] Another aspect of the present disclosure relates to a method, comprising: monitoring, by a processing device, a host command stream from a host system to a memory device; determining whether a phase value of a host command in the host command stream is valid; replicating the host command in response to determining that the phase value of the host command is valid; storing the replicated host command in a queue in a local memory; and executing the replicated host command in the queue in the local memory.

[0005] Yet another aspect of the present disclosure relates to a non-transitory computer-readable storage medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations comprising: monitoring a host command stream from a host system to a memory device; determining whether a phase value of a host command in the host command stream is valid; replicating the host command in response to determining that the phase value of the host command is valid; storing the replicated host command in a queue in a local memory; and executing the replicated host command in the queue in the local memory. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The present disclosure will be more fully understood from the detailed description given below and from the drawings of various embodiments of the present disclosure. However, the drawings should not be considered to limit the present disclosure to specific embodiments, but are only for explanation and understanding.

[0007] Figure 1An example computing system including a memory subsystem according to some embodiments of the present disclosure is described.

[0008] Figure 2 is a block diagram of an example system for implementing an optimized command queue with low latency according to some embodiments of the present disclosure.

[0009] Figure 3A An example of a command queue for implementing an optimized command queue with low latency according to some embodiments of the present disclosure is described.

[0010] Figure 3B An example of a descriptor ring for implementing an optimized command queue with low latency according to some embodiments of the present disclosure is described.

[0011] Figure 4 and 5 is a flow chart of an example method for implementing an optimized command queue with low latency according to some embodiments of the present disclosure.

[0012] Figure 6 is a block diagram of an example computer system in which embodiments of the present disclosure may operate. DETAILED DESCRIPTION

[0013] Aspects of the present disclosure relate to implementing an optimized command queue with low latency. The memory subsystem may be a storage device, a memory module, or a combination of a storage device and a memory module. Figure 1 Examples of storage devices and memory modules are described. In general, a host system may utilize a memory subsystem that includes one or more components, such as memory devices that store data. The host system may provide data to be stored at the memory subsystem and may request data to be retrieved from the memory subsystem.

[0014] The host system can initiate various data operations (e.g., write, read, erase, etc.) on the memory subsystem by generating commands. In some memory subsystems, the memory subsystem controller includes a command queue in which commands received from an external host system or generated locally by the memory subsystem controller are buffered and typically handled in a first-in, first-out order. Such commands include erase commands to erase physical blocks of memory, write commands to program certain data to one or more dies (or planes) of a memory device (e.g., one page at a time), or read commands to read certain data out of one or more dies (or planes) of a memory device (e.g., one page at a time).

[0015] As the size and volume of data sets continue to grow, the energy dedicated to system-level data movement also increases. Data flow—from different levels to the compute engine and back—becomes a source of system energy lost from computing, which has led to a trend to move selected computing to memory, which is called near memory computing (NMC). NMC may include computing in devices that employ Compute Express Link (CXL) technology. CXL technology introduces new memory interconnects and provides a path to NMC. CXL is an interface standard that can support several protocols that can run on Peripheral Component Interconnect Express (PCIe), including the CXL.io protocol and the CXL.mem protocol. The CXL.io protocol is a PCIe-like protocol that can be considered an "enhanced" PCIe protocol that can create managed memory, and can provide an interface for I / O devices. The CXL.mem protocol can enable a host to access the memory of an attached device using memory semantics (e.g., load and store commands), and can support both volatile memory architectures and persistent memory architectures.

[0016] An example of an NMC device that uses CXL technology is a Global Fabric Attached Memory (GFAM) device (GFD), which differs from traditional processors in that the memory is completely separated from the processing unit and a large shared memory pool is implemented. The memories in the large shared memory pool can be of the same type (e.g., DRAM) or different types (e.g., a mixture of DRAM, NAND flash memory). GFDs typically support the CXL.mem protocol, but do not support the CXL.io protocol for host system access.

[0017] Current command queue architectures are not optimized for commands to be handled with low latency in NMC processors. For example, in some command queue architectures, (several) doorbell bits are used, which need to be written separately and need to wait for the entire set of commands to be written into the queue, thereby causing latency. As another example, in some cases where cache coherence is required, transactions between the host system and the processor of the memory device may involve multiple rounds of communication, thereby causing latency. As another example, some command queue architectures use additional memory devices (e.g., memory mapped I / O (MMIO)) external to the system to reduce latency. However, in systems using GFD, the host system is generally unable to access additional memory devices external to the system because the CXL.io protocol is not applicable to systems that enable the host to access GFD devices. Therefore, the system cannot use the MMIO method to access additional memory devices to reduce latency. Other solutions may also require special instruction set architecture (ISA) commands to ensure that queue pushes are atomic (which may only be implemented in a subset of the host architecture) to provide low latency.

[0018] Aspects of the present disclosure address the above and other deficiencies by implementing an optimized command queue by including a phase value in each host command within the command queue and enabling a hardware component of the NMC processor (referred to as a command manager) to monitor the command queue and process the host commands based on the phase value, thereby achieving low latency. The phase value refers to a value stored in a reserved space of a host command to indicate whether the command manager should process or ignore the corresponding host command.

[0019] Specifically, a host system may generate a host command for a memory device and, as a last step in generating the host command, write a new stage value in the host command. The host system generates the stage value using a write index, wherein the stage value includes the last bit(s) of the result of dividing the write index by the descriptor ring size. The host system places the host command in a command queue (e.g., in the form of a descriptor ring) in a host cache memory and pushes (e.g., through a cache flush operation) the commands of the command queue through a CXL.mem link (e.g., according to a transaction specification defined in the protocol) in the order in which the commands were generated. A command manager monitors a stream of host commands from the CXL.mem link and, for each host command in the stream of host commands, determines whether the stage value of the host command is valid. To determine whether the stage value is valid, the command manager may compare the stage value of the host command with a value derived from a read index. When the stage value is equal to the last bit(s) (least significant bit) of the result of dividing the read index by the descriptor ring size, the stage value is valid. A determination that the stage value is valid indicates that the corresponding host command should be processed. When the stage value is not equal to the last bit(s) of the result of dividing the read index by the descriptor ring size, the stage value is invalid. A determination that the stage value is invalid indicates that the corresponding host command can be ignored. Thus, the host system can use the stage value as a trigger to switch between processing a command and ignoring a command because the host system knows whether the command should be processed when the command is generated. For example, if the host command has not yet fully completed writing data to the command but the command must be ended, the host system knows that the command should not be processed and therefore presents an invalid stage value to the command. If the host command generates the command without problems under normal circumstances, the host system knows that the command should be processed and thus presents a valid stage value to the command.

[0020] In some embodiments, the CXL.mem link that delivers write transactions with commands between the host system and the memory device may reorder transactions so that the commands may arrive at the memory device in an order different from the order in which the commands were issued from the host system. Although the command manager is monitoring the write stream within the command ring for the new phase, it can only process commands at the memory location referenced by the current read index and with a valid phase. Other commands with a valid phase are recorded (e.g., by associating the command with a bit in the received command bit vector) in the command manager, otherwise ignored and forwarded to the memory device. The received command bit vector may include a set of bits, each of which is associated with a command with a valid phase but not referenced by the current read index. In some embodiments, the received command bit vector may be included in the command manager. Once a command with a valid phase and referenced by the current read index is received and processed, the next read index is used to access the bit in the received command bit vector. If the bit is associated with a command, the command is retrieved from the memory device and processed by the command manager, and the bit is cleared for another use. This ensures that the commands are processed in the Command Manager in the same order as they were issued by the host system.

[0021] In response to determining that the host command has a valid phase value, the command manager copies the host command and stores the copied host command in a queue in a local memory (e.g., SRAM) within the command manager that is faster than a memory device (e.g., DRAM or non-volatile memory). The local memory may include SRAM that is capable of executing commands faster than DRAM or non-volatile memory. The local memory is "local" relative to the command manager compared to the memory device. The local memory may be located in the command manager. The local memory may be volatile or non-volatile memory. The command manager executes the copied host commands stored in the queue in the order in which the copied host commands are stored. If the command manager receives the host commands out of order, the host command is temporarily ignored by the command manager, advances to the memory device, and is read back from the memory device and placed in a queue (e.g., SRAM) in the command manager after the valid command received earlier is processed by the command manager. Although the original host command can still be transmitted to the memory device normally, such as through another communication link, the execution of the copied host command in the faster local memory may occur before the original host command reaches the memory device. Executing a replicated host command involves a single fabric latency, while receiving the original host command involves at least two fabric latencies, because of the additional time required to write the command and write the index (also known as a 'doorbell write' as a command valid indication). Since CXL technology implements variable latency for memory accesses, it potentially allows the command manager to access data and complete execution of a replicated host command before the memory returns a response to the original host write transaction.

[0022] In response to determining that the host command has an invalid phase value, the command manager ignores the host command and sends the host command to the memory device normally. Thus, by copying the commands that should be processed into the fast queue for execution, the commands that should be processed are executed (e.g., written) much faster than without the copying, and by letting the commands that should not be processed be written normally, the system behaves as a normal CXL memory device.

[0023] Advantages of the present disclosure include enhancing system performance by providing systems and methods for reducing latency in executing host commands for memory devices. Aspects of the present disclosure avoid using additional memory resources (e.g., MMIO) that are not included in the GFAM device. Moreover, using a phase value for a valid indication of a command can provide adaptability to executing an incomplete host command (e.g., an evicted cache line) because the phase value is written as the last step in generating the host command. Furthermore, using a phase value for a valid indication of a command can provide adaptability to a retry of a host command that has not been successfully transmitted to the memory device.

[0024] Figure 1 An example computing system 100 is illustrated that includes a memory subsystem 110 according to some embodiments of the present disclosure. Memory subsystem 110 may include media such as one or more volatile memory devices (e.g., memory device 140), one or more non-volatile memory devices (e.g., memory device 130), or a combination thereof.

[0025] The memory subsystem 110 may be a storage device, a memory module, or a combination of a storage device and a memory module. Examples of storage devices include solid state drives (SSDs), flash drives, universal serial bus (USB) flash drives, embedded multimedia controller (eMMC) drives, universal flash storage (UFS) drives, secure digital (SD) cards, and hard disk drives (HDDs). Examples of memory modules include dual in-line memory modules (DIMMs), small outline DIMMs (SO-DIMMs), and various types of non-volatile dual in-line memory modules (NVDIMMs).

[0026] Computing system 100 may be a computing device such as a desktop computer, a laptop computer, a network server, a mobile device, a vehicle (e.g., an airplane, drone, train, car, or other transportation vehicle), an Internet of Things (IoT)-enabled device, an embedded computer (e.g., a computer included in a vehicle, industrial equipment, or a networked business device), or such a computing device that includes a memory and a processing device.

[0027] The computing system 100 may include a host system 120 coupled to one or more memory subsystems 110. In some embodiments, the host system 120 is coupled to multiple memory subsystems 110 of different types. Figure 1 An example of a host system 120 coupled to one memory subsystem 110 is illustrated. As used herein, "coupled to" or "coupled with" generally refers to a connection between components, which may be an indirect communication connection or a direct communication connection (e.g., without intervening components), whether wired or wireless, including connections such as electrical connections, optical connections, magnetic connections, etc.

[0028] The host system 120 may include a processor chipset and a software stack executed by the processor chipset. The processor chipset may include one or more cores, one or more caches, a memory controller (e.g., an NVDIMM controller), and a storage protocol controller (e.g., a PCIe controller, a SATA controller). The host system 120 uses the memory subsystem 110, for example, to write data to the memory subsystem 110 and read data from the memory subsystem 110.

[0029] The host system 120 may be coupled to the memory subsystem 110 via a physical host interface. Examples of the physical host interface include, but are not limited to, a Serial Advanced Technology Attachment (SATA) interface, a Peripheral Component Interconnect Express (PCIe) interface, a Universal Serial Bus (USB) interface, a Fibre Channel, a Serial Attached SCSI (SAS), a Double Data Rate (DDR) memory bus, a Small Computer System Interface (SCSI), a Dual In-line Memory Module (DIMM) interface (e.g., a DIMM slot interface supporting Double Data Rate (DDR)), etc. The physical host interface may be used to transfer data between the host system 120 and the memory subsystem 110. When the memory subsystem 110 is coupled to the host system 120 through a physical host interface (e.g., a PCIe bus), the host system 120 may further utilize an NVM Express (NVMe) interface to access components (e.g., the memory device 130). The physical host interface may provide an interface for passing control, address, data, and other signals between the memory subsystem 110 and the host system 120. Figure 1 Memory subsystem 110 is illustrated. In general, host system 120 can access multiple memory subsystems via the same communication connection, multiple separate communication connections, and / or a combination of communication connections.

[0030] The NVMe interface is a communication interface / protocol developed for solid-state drives to operate on a host and memory device linked by a PCIe interface. The NVMe protocol provides a command queue and completion path for accessing data stored in a memory device by a host system 220. In some embodiments, the interface between the host system and the memory device may implement one or more alternative protocols supported by another interface standard. For example, the interface may implement one or more alternative protocols supported by PCIe (e.g., non-PCIe protocols). In some embodiments, the interface may be represented by a compute express link (CXL) interface or any communication link that allows cache line granularity updates and shares consistency control with a processing device.

[0031] The CXL system is a cache-coherent interconnect for processors, memory extensions, and accelerators. The CXL system maintains memory coherence between the CPU memory space and the memory on the attached device, which allows resource sharing to achieve higher performance, reduce software stack complexity, and reduce total system cost. In general, CXL is an interface standard that can support several protocols that can run on PCIe, including the CXL.io protocol, the CXL.mem protocol, and the CXL.cache protocol. The CXL.io protocol is a class PCIe protocol that can be considered an "enhanced" PCIe protocol that can create managed memory. CXL.io can be used for initialization, linking, device discovery and enumeration, register access, and can provide an interface for I / O devices. The CXL.mem protocol enables the host to access the memory of the attached device using memory semantics (e.g., load and store commands). This approach can support both volatile memory architectures and persistent memory architectures. The CXL.cache protocol can define host-device interactions to achieve efficient caching of host memory with low latency using a request and response method. Services (e.g., NVMe services) can be run over the CXL.io protocol, and the CXL.mem and CXL.cache protocols can share a common link layer and transaction layer. Thus, the CXL protocol can be multiplexed and transported over the PCIe physical layer.

[0032] Memory devices 130, 140 may include any combination of different types of non-volatile memory devices and / or volatile memory devices. Volatile memory devices (e.g., memory device 140) may be, but are not limited to, random access memory (RAM), such as dynamic random access memory (DRAM) and synchronous dynamic random access memory (SDRAM).

[0033] Some examples of non-volatile memory devices (e.g., memory device 130) include non-and (NAND) type flash memory and write-in-place memory, such as a three-dimensional cross-point ("3D cross-point") memory device, which is a cross-point array of non-volatile memory cells. The cross-point array of non-volatile memory cells can be combined with a stackable cross-grid data access array to perform bit storage based on changes in body resistance. In addition, compared to many flash-based memories, cross-point non-volatile memories can perform write-in-place operations, where non-volatile memory cells can be programmed without first erasing the non-volatile memory cells. NAND-type flash memory includes, for example, two-dimensional NAND (2D NAND) and three-dimensional NAND (3D NAND).

[0034] Each of the memory devices 130 may include one or more arrays of memory cells. One type of memory cell (e.g., a single level cell (SLC)) may store one bit per cell. Other types of memory cells (e.g., multi-level cells (MLC), three-level cells (TLC), four-level cells (QLC), and five-level cells (PLC)) may store multiple bits per cell. In some embodiments, each of the memory devices 130 may include one or more arrays of memory cells (e.g., SLC, MLC, TLC, QLC, PLC, or any combination thereof). In some embodiments, a particular memory device may include an SLC portion, an MLC portion, a TLC portion, a QLC portion, or a PLC portion of a memory cell. The memory cells of the memory device 130 may be grouped into pages, which may refer to a logical unit of a memory device used to store data. For some types of memory (e.g., NAND), pages may be grouped to form blocks. Some types of memory (e.g., 3D crosspoints) may group pages across dies and channels to form management units (MUs).

[0035] Although nonvolatile memory components such as a 3D cross-point array of nonvolatile memory cells and NAND-type flash memory (e.g., 2D NAND, 3D NAND) are described, the memory device 130 may be based on any other type of nonvolatile memory, such as read-only memory (ROM), phase-change memory (PCM), self-select memory, other chalcogenide-based memories, ferroelectric transistor random access memory (FeTRAM), ferroelectric random access memory (FeRAM), magnetic random access memory (MRAM), spin transfer torque (STT)-MRAM, conductive bridging RAM (CBRAM), resistive random access memory (RRAM), oxide-based RRAM (OxRAM), or non-(NOR) flash memory or electrically erasable programmable read-only memory (EEPROM).

[0036] The memory subsystem controller 115 (or simply controller 115) can communicate with the memory device 130 to perform operations such as reading data, writing data, or erasing data and other such operations at the memory device 130. The memory subsystem controller 115 may include hardware, such as one or more integrated circuits and / or discrete components, buffer memory, or a combination thereof. The hardware may include digital circuitry with dedicated (i.e., hard-coded) logic to perform the operations described herein. The memory subsystem controller 115 may be a microcontroller, dedicated logic circuitry (e.g., a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.), or other suitable processor.

[0037] The memory subsystem controller 115 may include a processing device including one or more processors (e.g., processing device 117) configured to execute instructions stored in a local memory 119. In the illustrated example, the local memory 119 of the memory subsystem controller 115 includes an embedded memory configured to store instructions for executing various processes, operations, logic flows, and routines that control the operation of the memory subsystem 110, including handling communications between the memory subsystem 110 and the host system 120.

[0038] In some embodiments, local memory 119 may include memory registers that store memory pointers, fetched data, etc. Local memory 119 may also include read-only memory (ROM) for storing microcode. Figure 1 The example memory subsystem 110 in FIG. 1 has been described as including a memory subsystem controller 115, but in another embodiment of the present disclosure, the memory subsystem 110 does not include a memory subsystem controller 115, but may rely on external control (e.g., provided by an external host, or provided by a processor or controller separate from the memory subsystem).

[0039] In general, the memory subsystem controller 115 may receive commands or operations from the host system 120 and may convert the commands or operations into instructions or appropriate commands to achieve the desired access to the memory device 130. The memory subsystem controller 115 may be responsible for other operations associated with the memory device 130, such as wear leveling operations, garbage collection operations, error detection and error correction code (ECC) operations, encryption operations, cache operations, and address translation between logical addresses (e.g., logical block addresses (LBA), namespaces) and physical addresses (e.g., physical MU addresses, physical block addresses). The memory subsystem controller 115 may further include a host interface circuit system for communicating with the host system 120 via a physical host interface. The host interface circuit system may convert commands received from the host system into command instructions for accessing the memory device 130, and convert responses associated with the memory device 130 into information for the host system 120.

[0040] The memory subsystem 110 may also include additional circuitry or components not illustrated. In some embodiments, the memory subsystem 110 may include a cache or buffer (e.g., DRAM) and address circuitry (e.g., row decoders and column decoders) that may receive addresses from the memory subsystem controller 115 and decode the addresses to access the memory device 130.

[0041] In some embodiments, the memory device 130 includes a local media controller 135 that operates in conjunction with the memory subsystem controller 115 to perform operations on one or more memory cells of the memory device 130. An external controller (e.g., the memory subsystem controller 115) can manage the memory device 130 externally (e.g., perform media management operations on the memory device 130). In some embodiments, the memory subsystem 110 is a managed memory device, which is a raw memory device 130 with control logic on the die (e.g., the local media controller 135) and a controller for media management (e.g., the memory subsystem controller 115) within the same memory device package. An example of a managed memory device is a managed NAND (MNAND) device.

[0042] According to an embodiment of the present disclosure, the memory subsystem 110 includes a command manager 113 that can be used to implement an optimized command queue with low latency. In some embodiments, the memory subsystem controller 115 includes at least a portion of the command manager 113. In some embodiments, the command manager 113 is part of the host system 110, an application, or an operating system. In other embodiments, the local media controller 135 includes at least a portion of the command manager 113 and is configured to perform the functionality described herein. Figures 2 to 6Further details regarding the operation of command manager 113 are described.

[0043] Those skilled in the art will appreciate that additional circuitry and signals may be provided and simplified. Figure 1 It should be recognized that the reference Figure 1 The functionality of the various block components described may not necessarily be separated into different components or component portions of an integrated circuit device. For example, a single component or component portion of an integrated circuit device may be adapted to perform Figure 1 Alternatively, one or more components or component parts of an integrated circuit device may be combined to perform Figure 1 The functionality of a single block component.

[0044] Figure 2 is a schematic block diagram of a system 200 implementing an optimized command queue with low latency. In various embodiments, the system 200 includes a host system 220, a compute express link (CXL) device 210, and a memory device 230. In some implementations, the CXL device 210 may be Figure 1 The memory subsystem controller 115 and the memory device 230 may be Figure 1 Memory devices 130, 140.

[0045] The host system 220 may include a central processing unit (CPU) 222 and a host memory 229. The CPU 222 executes computer-readable program code (e.g., software or firmware) executable instructions (referred to herein as "instructions"). The instructions may be executed by various components of the CPU 222 (e.g., processors, logic gates, switches, application specific integrated circuits (ASICs), programmable logic controllers, embedded microcontrollers, and other components of the CPU 222). Instructions executable by the CPU 222 for implementing the embodiments described herein are stored in a non-transitory computer-readable storage medium.

[0046] Host system 220 may provide spatial and temporal locality of memory access patterns to optimize performance, and may use higher performance and lower capacity media (referred to as cache) to store frequently accessed data (temporal locality) or data located in recently accessed memory regions (spatial locality). Host memory 229 may include such cache memory. Host memory 229 may be associated with a protocol that specifies the size of a management unit used by a memory component and / or a preferred size of a request to access data stored at a management unit. For example, a protocol for one memory component may specify a 512 kilobyte (KB) size request to be performed on the memory component. An application executing on the host system may initially request to read 512KB of data from the memory component, but due to the protocol of the bus used to communicate between the host system and the CXL device, the 512KB request is typically broken into smaller granularity requests (e.g., 8096 requests, each of which is 64B in size). The host system 220 may perform smaller granularity requests to obtain data from a memory component of the memory device 230, which may then be stored in a cache in the host memory 229 and / or returned to a requesting application in the host system 220. Performing smaller granularity requests on a memory component that is capable of handling larger granularity requests may result in faster wear and lower endurance of the memory component because more read operations will be performed at the memory component.

[0047] The system 200 may optionally include a CXL switch 250 coupled to a host system 220 via a CXL connection 215 or a communication link that allows high-speed communication and shares coherence control with a processing device. The CXL connection 215 may include a set of data transmission paths ("paths") for implementing the CXL protocol (including CXL.io, CXL.mem, and CXL.cache). According to embodiments described herein, the CXL connection 215 may include any suitable number of paths. For example, the CXL connection 215 may include 16 paths (i.e., CXL x16). The system 200 may further include a CXL connection 225 and a CXL device 210 operably coupled to the CXL switch 250 via the CXL connection 225. According to embodiments described herein, the CXL connection 225 may include any suitable number of paths. For example, the CXL connection 225 may include 4 paths (CXL x4).

[0048] CXL device 210 may include command manager 113, processor 217, and media controller 216. Figure 2, CXL device 210 may include a CXL link with CXL.mem protocol support. More specifically, CXL device 210 may allocate a portion of memory device 230 as a memory component that will be visible through CXL.mem. The memory component may be accessed through CXL.mem. For example, the memory component may have a size of 32MB. However, this size is purely exemplary. In the illustrative example, CXL device 210 is a GFD. A GFD is similar to a conventional CXL type 3 device, except that it can be accessed by multiple nodes (e.g., up to 4095) in a flexible manner using port-based routing. This architecture creates the possibility to build powerful systems that include compute and memory elements arranged to meet the needs of a specific workload.

[0049] The memory device 230 may include a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a ferroelectric random access memory (FeRAM), a magnetic random access memory (MRAM), or a resistive random access memory (RRAM) including several subcomponents. The memory component of the memory device 230 may refer to any portion of the memory device 230.

[0050] In some embodiments, the CPU 222 may create a descriptor ring for a command queue, wherein each host command in the command queue corresponds to an entry in the descriptor ring, and each entry in the descriptor ring includes space reserved for a phase indication of the corresponding host command. The value written into the space reserved for the phase indication (referred to as the phase value) indicates whether the corresponding host command is a command that should be processed. The phase value is written as the last step in generating a command. When the corresponding host command is a command that should be processed, the phase indication may be referred to as having a valid phase value. When the corresponding host command is a command that can be ignored and does not need to be processed, the phase indication may be referred to as having an invalid phase value. In one embodiment, the phase value may be written as a bit value, wherein the value of the last bit(s) equal to the result of dividing the write index by the descriptor ring size indicates that the phase value is valid, while the value of the last bit(s) not equal to the result of dividing the write index by the descriptor ring size indicates that the phase value is invalid. In some embodiments, the phase value may be written as a byte value, wherein some bytes indicate that the phase value is valid, while the remaining bytes indicate that the phase value is invalid. Different byte values ​​indicate that the phase value is valid during each traversal of the ring. Except Figure 2 In addition, the following also discusses Figures 3A to 3B Describes some details of a host command with a phase value.

[0051] refer to Figure 3A, the host command may be referred to as one or more messages 1-N. The message may be the size of a cache line. A cache line may refer to a sequence of memory bytes with a specific alignment and length. For example, in a CXL system, the alignment and length constraints may be 64 bytes. The command queue 300A may be a memory area containing a sequence of data entries and state information associated with the command queue. The data entry may save cache line metadata (e.g., the address of the contents of a cache line within a memory device) as well as the cache line contents. In some implementations, the state information includes a pointer to the "next" memory buffer, a write index, and a read index. The write index indicates the location to write data, and the read index indicates the location to read data.

[0052] refer to Figure 3B , the command queue may be viewed as a circular buffer where entries are reused after being released. The host system may write entries onto the descriptor ring 300B, increment a write index (e.g., Write_Index), and write a stage value, which may act as a doorbell to signal the CXL device that a new entry has been written. The CXL device reads from the ring, increments a read index (e.g., Read_Index) to indicate that the entry is now released, and performs the requested operation. The descriptor ring 300B is used as a memory buffer that contains naturally aligned 64-byte entries that instruct the CXL device to perform a given operation. In some implementations, each entry corresponds to a host command. The descriptor ring 300B is a circular ring that starts at a specified address (e.g., ds_ring_ptr). The descriptor ring 300B is contiguous at the translation level configured for the CXL device. The descriptor ring 300B is configured to contain a given number of descriptor ring entries (e.g., 8). The number of descriptor ring entries may be a power of 2 to simplify the algorithms required to generate the phase value. The number of bytes allocated in memory for the descriptor ring 300B may be the cache line size multiplied by the number of descriptor ring entries (eg, 8*64 bytes).

[0053] The descriptor ring 300B requires "start" and "end" indicators. Rather than using memory pointers to track these indicators, the descriptor ring 300B can use 64-bit unsigned logical indices to indicate the start (e.g., Read_Index) and end (e.g., Write_Index) of the descriptor ring. When writing or reading an entry at a given index, the logical index only needs to be mapped to the address of the entry in the descriptor ring 300B. Figure 3B The direction of the loop shown by the arrows indicates the direction in which the memory buffer is used. The read index should always be less than or equal to the write index, where the equality condition indicates that the queue is empty. The phase value indicating a valid command is the least significant bit of the write index divided by the descriptor ring size.

[0054] Return to reference Figure 2 , the CPU 222 may generate a host command to access (e.g., write or read) data stored in the memory device 230, and send the host command to be stored in the command queue 233 of the host memory 229. In some embodiments, the host command is a write command. The CPU 222 may generate the host command to the descriptor ring of the command queue and write the stage value. For example, the CPU may write bit 1 as the stage value, as a switch from bit 0, or as an update of the previous value bit 0. In some embodiments, the data of the command has not yet been completed, but for example, a cache eviction occurs, and the CPU 222 may keep the stage value as bit 0, because for the host command, the stage value needs to be written last, that is, all other data to be written in the host command needs to be written before the stage value is written. The CPU 222 will increment the write index (e.g., Write_Index) after updating the stage value. The CPU 222 may send the command queue containing the host command to the local memory 229. The local memory 229 stores the command queue 233. The command queue 233 may be stored in an in-memory cache, which is a portion of the host memory 229 (e.g., DRAM) and is used to speed up access to data stored on a slower memory storage device by temporarily storing portions of the data in a faster memory storage device that can be accessed more quickly than the slower memory storage device. Thus, data in the cache can be read and written faster than data that is not in the cache. Figure 3B In the example illustrated in FIG. 1 , the CPU 222 writes the host command onto the descriptor ring 300B at the entry pointed to by the write index (e.g., Write_Index), writes the phase value (e.g., bit 1) of the entry, and increments the write index (e.g., Write_Index+1 as the first bit in the descriptor ring 300B pointed to by the write index (e.g., Write_Index+1). Figure 3B A spatial movement in the direction of the cycle shown by the arrows).

[0055] Host system 220 may push command queue 233 to CXL device 210 over CXL.mem connections (e.g., CXL connections 215 and 225) using cache flush or other mechanisms to cause cache eviction. Host commands in command queue 233 may be generated according to the CXL.mem protocol. Command manager 113 may monitor CXL.mem write connections for writing to command queue 233. In some implementations, command manager 113 may include a queue address and a queue size (e.g., a descriptor ring size). Command manager 113 detects host commands belonging to command queue 233 by detecting a write command that includes a memory address equal to the queue address plus an offset address (e.g., the offset address may be the remainder of a read index divided by the queue size). In some implementations, command manager may monitor commands destined for memory device 230. In some implementations, simple logic may perform monitoring and forward commands destined for the queue offset address described above to the command manager.

[0056] The command manager 113 may determine whether the host command included in the command queue 233 has a valid phase value or an invalid phase value. Figure 3B In the example illustrated in , each entry in the descriptor ring 300B includes a phase value (represented by P). Based on the number of entries (e.g., N), the valid phase value P may be equal to the bit indication of the write index (e.g., P = (Write_Index / N) & 0xff). Using a number of entry values ​​N that is a power of 2 simplifies the hardware necessary to determine the valid phase value. The command manager 113 may determine whether the entry pointed to by the read index has a valid phase value or an invalid phase value by comparing the phase value in the command pointed to by the read index (which may be derived as the last bit of the result of dividing the read index by the descriptor ring size) with the valid phase value (which is equal to the last bit of the result of dividing the write index by the descriptor ring size). The valid phase value indicates that the write index used when the entry was generated is equal to the read index used to access the entry.

[0057] In some embodiments, when the stage value of the entry pointed to by the read index is equal to the last bit of the result of dividing the write index by the descriptor ring size, the command manager 113 can determine that the entry (i.e., the host command) has a valid stage value, which indicates that the entry should be processed. In response to determining that the host command has a valid stage value, the command manager 113 copies the host command and stores the copied host command in the SRAM queue 213 for processing. The command manager 113 also increments the read index (e.g., Read_Index). It is not necessary to change the stage value to an invalid stage value because different values ​​will indicate a valid command during subsequent traversals of the ring and the host writes a new value to the stage when generating a command. In some embodiments, the SRAM queue 213 is a first-in, first-out queue. In some embodiments, the command manager 113 includes the SRAM queue 213. In some embodiments, the multiple copied host commands stored in the queue in the faster local memory are in the order in which the multiple corresponding host commands are created. If the command manager receives a host command with a valid stage out of order, the host command can be temporarily ignored by the command manager and advance to the memory device. This host command may be read back from the memory device and placed in the SRAM queue in the command manager after the valid command received earlier is processed by the command manager. This may ensure that the multiple host commands are executed in the order in which the multiple corresponding host commands were created in the command manager. The command manager 113 may execute the replicated host commands according to the order in which the host commands were stored in the SRAM queue 213. Figure 3B In the example illustrated in FIG. 1 , the command manager 113 reads the entry pointed to by the read index (e.g., Read_Index) from the descriptor ring 300B, increments the read index (e.g., Read_Index+1 as the entry pointed to by Figure 3B ) and performs the requested operation as requested in the host command. In some implementations, the command manager 113 may execute the replicated host command before the original host command reaches the memory device 230. Thus, the host command is processed with low latency.

[0058] In some implementations, when the phase value of the entry pointed to by the read index is not equal to the last bit of the result of dividing the write index by the descriptor ring size, command manager 113 may determine that the entry (i.e., the host command) has an invalid phase value, indicating that the entry should be ignored. In response to determining that command queue 233 includes a host command with an invalid phase value, command manager 113 ignores the host command. Processor 217 (e.g., an on-chip network processor) may process the host command as usual and communicate with media controller 216 to access the memory components of memory device 230 as indicated in the host command. CXL device 210 may then access the memory components of memory device 230 via connection 235. In some implementations, the memory components of memory device 230 store the entire command queue 233. Since the memory component of memory device 230 stores the entire command queue 233 regardless of whether the host command has a valid stage value or an invalid stage value, and command manager 113 only stores and processes host commands with valid stage values, host commands with valid stage values ​​will be executed with low latency because they are processed directly in faster local memory without requiring an additional connection (e.g., connection 235) from the CXL device to the memory device.

[0059] Figure 4 and 5 4 is a flowchart of example methods 400 and 500 for implementing an optimized command queue with low latency according to some embodiments of the present disclosure. Methods 400 and 500 may be performed by processing logic, which may include hardware (e.g., a processing device, a circuit system, a dedicated logic, a programmable logic, a microcode, hardware of a device, an integrated circuit, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, methods 400 and 500 are performed by Figure 1 and Figure 2 The commands of the command manager 113 are executed. Although shown in a particular order or sequence, the order of the processes may be modified unless otherwise specified. Therefore, the illustrated embodiments should be understood as examples only, and the illustrated processes may be performed in a different order, and some processes may be performed in parallel. In addition, one or more processes may be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are possible.

[0060] refer to Figure 4 , the method 400 may be performed by a host system (e.g., Figure 1 The host system 120 and Figure 2The processing device (e.g., CPU 222) at a host system 220 of the embodiment of the present invention is executed. At operation 410, the processing device may create a descriptor ring of a command queue (e.g., command queue 233), wherein the descriptor ring includes a plurality of entries and each entry in the descriptor ring includes a space reserved for storing a stage value. In some embodiments, the processing device may generate the command queue in another form as long as each host command included in the command queue includes a space reserved for storing a stage value.

[0061] At operation 420, the processing device may create a host command to access (e.g., write or read) data. In some embodiments, the host command specifies a queue address. At operation 430, the processing device may write the host command onto the descriptor ring. In some embodiments, the processing device may place the host command onto the descriptor ring based on the queue address. In some embodiments, the processing device may place the host command onto another form of command queue based on the queue address.

[0062] At operation 440, the processing device may write a stage value for the host command. As a final step in generating the host command, the processing device writes the stage value for the host command. In some implementations, the processing device may write a bit value (e.g., bit 1) to the stage value. In some implementations, the processing device writes a value to the stage value as long as the value can indicate whether the host command should be processed or ignored. In some implementations, a value indicating that the host command should be processed represents a valid stage value, while a value indicating that the host command should be ignored represents an invalid stage value. The valid stage value may be the least significant bit(s) of the result of writing the index divided by the descriptor ring size. At operation 450, the processing device may send the host command to the NMC device (e.g., CXL device 210) via a CXL.mem link (e.g., connections 215 and / or 225). In some implementations, the NMC device is a CXL GFD.

[0063] refer to Figure 5 , method 500 may be performed by a memory subsystem (eg, Figure 1 The memory subsystem 110 or Figure 2 A processing device (e.g., Figure 1 and 2 At operation 510, the processing device monitors a host command stream from the host system to the memory device. In some implementations, the host command stream includes a descriptor ring, and the descriptor ring includes a plurality of entries and each entry corresponds to a host command.

[0064] At operation 520, the processing device determines whether a stage value of a host command in a host command stream is valid. In some embodiments, the host command is placed in one of a plurality of entries in a descriptor ring. In some embodiments, to determine whether a stage value of a host command in a host command stream is valid, the processing device determines whether the stage value is updated by the host system. In some embodiments, to determine whether a stage value of a host command in a host command stream is valid, the processing device determines whether a stage value of a command pointed to by a read index is equal to the least significant bit(s) of a result of dividing a write index by a descriptor ring size.

[0065] At operation 530, in response to determining that the phase value of the host command in the host command stream is valid, the processing device copies the host command. For example, the processing device may determine Figure 3A , wherein the message N includes a message payload and metadata; the processing device may replicate the message N by making a duplicate message N including the message payload and metadata. At operation 540, the processing device stores the replicated host command in a queue in a faster local memory. In some embodiments, the faster local memory is SRAM. For example, the queue may be Figure 2 In some embodiments, the queue in the faster local memory is a first-in, first-out queue. At operation 550, the processing device executes the queue in the faster local memory (e.g., Figure 2 In some embodiments, the processing device receives the copied host command in the SRAM queue 213 of the host device after the host command arrives at the memory device (e.g., Figure 2 memory device 230) before executing the faster queue in the local memory (e.g., Figure 2 The copied host commands in the SRAM queue 213).

[0066] Figure 6 An example machine illustrating a computer system 600 in which a set of instructions for causing the machine to perform any one or more of the methodologies discussed herein may be executed. In some embodiments, the computer system 600 may correspond to a host system (e.g., Figure 1 1) a host system 120 that includes, is coupled to, or utilizes a memory subsystem (e.g., Figure 1 The memory subsystem 110 of the controller may be used to execute the operation of the controller (for example, to execute the operating system to execute the corresponding Figure 1 and Figure 2In some embodiments, the machine may be connected (e.g., using a network) to other machines in a LAN, an intranet, an extranet, and / or the Internet. The machine may operate in the capacity of a server or a client user machine in a client-server network environment, as a peer machine in a peer-to-peer (or distributed) network environment, or as a server or a client user machine in a cloud computing infrastructure or environment.

[0067] The machine may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a network appliance, a server, a network router, a switch or a bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine. Furthermore, while a single machine is described, the term "machine" shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

[0068] The example computer system 600 includes a processing device 602, a main memory 604 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) (e.g., synchronous DRAM (SDRAM) or RDRAM), etc.), a static memory 606 (e.g., flash memory, static random access memory (SRAM), etc.), and a data storage system 618, which communicate with each other via a bus 630.

[0069] The processing device 602 represents one or more general purpose processing devices, such as a microprocessor, a central processing unit, or the like. More specifically, the processing device may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor implementing other instruction sets or multiple processors implementing a combination of instruction sets. The processing device 602 may also be one or more special purpose processing devices, such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, or the like. The processing device 602 is configured to execute instructions 626 for performing the operations and steps discussed herein. The computer system 600 may further include a network interface device 608 for communicating over a network 620.

[0070] The data storage system 618 may include a machine-readable storage medium 624 (also referred to as a computer-readable medium) having stored thereon one or more sets of instructions 626 or software embodying any one or more of the methodologies or functionality described herein. The instructions 626 may also reside, completely or at least partially, within the main memory 604 and / or the processing device 602 during execution by the computer system 600, the main memory 604 and the processing device 602 also constituting machine-readable storage media. The machine-readable storage medium 624, the data storage system 618, and / or the main memory 604 may correspond to Figure 1 Memory subsystem 110.

[0071] In one embodiment, instructions 626 include instructions for implementing instructions corresponding to a memory management component (e.g., Figure 1 and Figure 2 The machine-readable storage medium 624 is shown as a single medium in the example embodiment, but the term "machine-readable storage medium" should be considered to include a single medium or multiple media storing one or more sets of instructions. The term "machine-readable storage medium" should also be considered to include any medium capable of storing or encoding a set of instructions for machine execution and causing the machine to perform any one or more methodologies of the present disclosure. Therefore, the term "machine-readable storage medium" should be considered to include, but not limited to, solid-state memory, optical media, and magnetic media.

[0072] Some portions of the foregoing detailed description have been presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is herein and generally conceived to be a self-consistent sequence of operations leading to a desired result. The operations are those requiring physical manipulation of physical quantities. Typically, but not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, combined, compared, and otherwise manipulated. It has proven convenient at times, primarily for common usage reasons, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

[0073] It should be remembered, however, that all of these and similar terms are associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. The present disclosure may involve the actions and processes of a computer system or similar electronic computing device that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system's memories or registers or other such information storage systems.

[0074] The present disclosure also relates to an apparatus for performing the operations described herein. This apparatus may be specially constructed for the intended purpose, or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer. This computer program may be stored in a computer-readable storage medium, such as any type of disk, including floppy disks, optical disks, CD-ROMs and magneto-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, or any type of medium suitable for storing electronic instructions, each coupled to a computer system bus.

[0075] The algorithms and displays presented herein are not inherently related to any particular computer or other device. Various general purpose systems may be used with programs according to the teachings herein, or it may prove convenient to build more specialized equipment to perform the methods. The structures of a variety of these systems will appear as described in the following description. In addition, the present disclosure is not described with reference to any particular programming language. It will be appreciated that a variety of programming languages ​​may be used to implement the teachings of the present disclosure as described herein.

[0076] The present disclosure may be provided as a computer program product or software comprising a machine-readable medium having instructions stored thereon, which instructions may be used to program a computer system (or other electronic device) to perform a process according to the present disclosure. A machine-readable medium includes any mechanism for storing information in a form readable by a machine (e.g., a computer). In some embodiments, a machine-readable (e.g., computer-readable) medium includes a machine (e.g., computer) readable storage medium, such as a read-only memory ("ROM"), a random access memory ("RAM"), a magnetic disk storage medium, an optical storage medium, a flash memory component, etc.

[0077] In the foregoing description, embodiments of the present disclosure have been described with reference to specific example embodiments thereof. It will be apparent that various modifications may be made to the present disclosure without departing from the broader spirit and scope of the embodiments of the present disclosure as set forth in the appended claims. Accordingly, the description and drawings should be regarded in an illustrative rather than a restrictive sense.

Claims

1. A system comprising: Memory device; and A processing device operatively coupled to the memory device to perform operations comprising: monitoring a host command flow from a host system to the memory device; determining whether a phase value of a host command in the host command stream is valid; replicating the host command in response to determining that the phase value of the host command is valid; storing the replicated host commands in a queue in local memory; and The replicated host commands in the queue in the local memory are executed.

2. The system of claim 1, wherein executing the replicated host commands in the queue in the local memory is performed before the host commands reach the memory device.

3. The system of claim 1, wherein the host command stream comprises a descriptor ring, wherein the descriptor ring comprises a plurality of entries, and wherein the host command is placed in one of the plurality of entries.

4. The system of claim 1 , wherein determining whether the phase value of the host command is valid comprises: A determination is made whether the phase value derived from the write index is equal to the value derived from the read index.

5. The system of claim 1, wherein the operations further comprise: The host command is ignored by the processing device in response to determining that the phase value of the host command is invalid.

6. The system of claim 1, wherein the operations further comprise: The host command is stored in the memory device.

7. The system of claim 1, wherein the operations further comprise: A plurality of replicated host commands are stored in the queue in the local memory in the order in which a plurality of corresponding host commands were created.

8. The system of claim 1, wherein the host command is generated according to the CXL.mem protocol.

9. The system of claim 1, wherein the host command comprises data to be written to the memory device.

10. The system of claim 1, wherein the processing device comprises a near memory computing (NMC) device.

11. The system of claim 1, wherein the local memory comprises a static random access memory (SRAM).

12. The system of claim 1, wherein the processing device includes the local memory.

13. A method comprising: monitoring, by the processing device, a host command flow from the host system to the memory device; determining whether a phase value of a host command in the host command stream is valid; replicating the host command in response to determining that the phase value of the host command is valid; storing the replicated host commands in a queue in local memory; and The replicated host commands in the queue in the local memory are executed.

14. The method of claim 13, wherein executing the replicated host commands in the queue in the local memory is performed before the host commands reach the memory device.

15. The method of claim 13, wherein the host command stream comprises a descriptor ring, wherein the descriptor ring comprises a plurality of entries, and wherein the host command is placed in one of the plurality of entries.

16. The method of claim 13, wherein determining whether the phase value of the host command is valid comprises: A determination is made whether the phase value derived from the write index is equal to the value derived from the read index.

17. A non-transitory computer-readable storage medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations comprising: monitoring a host command flow from a host system to a memory device; determining whether a phase value of a host command in the host command stream is valid; replicating the host command in response to determining that the phase value of the host command is valid; storing the replicated host commands in a queue in local memory; and The replicated host commands in the queue in the local memory are executed.

18. The non-transitory computer-readable storage medium of claim 17, wherein executing the replicated host commands in the queue in the local memory is performed before the host commands reach the memory device.

19. The non-transitory computer-readable storage medium of claim 17, wherein the host command stream comprises a descriptor ring, wherein the descriptor ring comprises a plurality of entries, and wherein the host command is placed in one of the plurality of entries.

20. The non-transitory computer-readable storage medium of claim 17, wherein determining whether the phase value of the host command is valid comprises: A determination is made whether the phase value derived from the write index is equal to the value derived from the read index.

Images