Accelerating command latency using hybrid compute fast link type 3 SSD memory devices

Abstract

The invention relates to accelerating command latency using a hybrid compute fast link type 3 SSD memory device. A memory device, method, and system are provided, the system including: a host in communication with a system memory having a driver that creates commands for writing and reading data; and a memory device in communication with the host, the memory device comprising: a memory array comprising a plurality of memory components; a device attached memory comprising a commit queue and a completion queue for receiving commands from the driver; and a device controller configured to communicate with the device attached memory, the host, and the plurality of memory components such that the device controller receives an interface or link from the driver indicating that a command is placed in the commit queue, and automatically executes any pending command in the commit queue for completion.

Description

[0001] CROSS REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 517,328, filed on August 2, 2023, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] The present disclosure relates generally to memory systems and methods, and more particularly to hybrid CXL Type 3 solid-state (SSD) memory devices and accelerated command latency. Background Art

[0004] Memory devices (also referred to as "storage media devices") are widely used to store information in a variety of electronic devices, such as computers, user devices, wireless communication devices, cameras, digital displays, and the like. Memory devices typically include volatile or nonvolatile memory. Nonvolatile memory (e.g., flash memory) allows information to be stored even when the nonvolatile memory is not connected to a power source.

[0005] Memory devices are used to store data provided by a host (e.g., a computer) that includes a central processing unit (CPU) that communicates with system memory (e.g., dynamic random access memory (DRAM)) to store and retrieve data and instructions, where the data is read back to the host. The instructions may include operating systems, drivers, and application programs. Drivers operate and control a particular type of device. The operating system uses drivers to provision resources and services provided by the device.

[0006] Sometimes a non-volatile memory specification is utilized, namely, a non-volatile memory express (NVMe) type specification, which specifies a logical device interface protocol for accessing non-volatile memory through a peripheral component interconnect express (PCIe) bus via a host device. Typically, NVMe-based protocols work by utilizing a submission queue (SQ) and a completion queue (CQ) to submit NVMe commands to a memory device and obtain command completion notifications through the CQ. The SQ and CQ are currently allocated in system memory with the assistance of an NVMe driver.

[0007] The host writes to the submission queue (i.e., input / output (I / O) command queue) and rings the doorbell (i.e., I / O command ready signal), and the NVMe controller of the memory device picks up the I / O command queue from the system memory, executes the I / O command queue, and sends the I / O completion queue. The host records the I / O completion queue and clears the doorbell with the I / O command completion signal. Therefore, this conventional process requires the memory device to jump to the system memory to retrieve the command, thereby causing undesirable command latency.

[0008] It is desirable to have a driver for creating submission queues and completion queues in local device attached memory of a Hybrid Compute Express Link (CXL) Type 3 SSD memory device that is memory mapped to a host for faster access and execution of commands, thereby speeding up command latency. Summary of the invention

[0009] One aspect of the present disclosure is directed to a system comprising a host in communication with a system memory and including a driver configured to create commands for writing and reading data; and a memory device in communication with the host and configured to store data therein. The memory device comprises: a memory array including a plurality of memory components for storing memory therein; a device attached memory including a submission queue and a completion queue for receiving commands from the driver; and a device controller configured to communicate with the device attached memory, the host, and the plurality of memory components, wherein the device controller receives an interface or link from the driver indicating that a command is placed in the submission queue and automatically executes any pending commands in the submission queue for completion.

[0010] In some embodiments, a completion status of the write command is sent to the device controller, and the device controller is configured to instruct to move the write command to the completion queue and clear the buffer space occupied by the write command.

[0011] In some embodiments, once the data is read, the status of the read command is retrieved by the device controller and then the read command is placed in the completion queue.

[0012] Another aspect of the present disclosure relates to a method for performing a write operation of data in a memory device including a device-attached memory, a device controller, and a plurality of memory components. The method includes: creating commands for writing and reading data via a driver of a host; receiving the commands in a submission queue at the device-attached memory; communicating with the device-attached memory, the host, and a plurality of memory components via the device controller, including receiving an interface or link from the driver via the device controller indicating that a command is placed in the submission queue, and automatically executing any pending commands in the submission queue for completion.

[0013] In some embodiments, the device attached memory is memory mapped to the host, and the driver comprises a non-volatile memory express (NVMe) type driver and is configured for creating NVMe commands.

[0014] In some embodiments, wherein the device controller comprises a hybrid Compute Express Link (CXL) / NVMe device controller, the method further comprises determining a type of CXL packet received from the drive via the interface or link and an NVMe command wrapped within the CXL packet.

[0015] In some embodiments, after creating the NVMe command via the driver, the method further includes ringing a doorbell for the device controller via the driver at the host by placing the NVMe command inside the submission queue at the device attached storage, and automatically consuming any pending commands in the submission queue via the device controller.

[0016] In some embodiments, once the corresponding NVMe command is completed, the method further includes moving the completed NVMe command to a completion queue in the device attached memory via the device controller, and then instructing to clear the buffer space for the corresponding NVMe command.

[0017] In some embodiments, when the NVMe command includes a write command, the method further includes: confirming via the device controller whether the packet received at the submission queue includes the write command; and automatically executing the write command, wherein associated data is written to a memory component among the multiple memory components.

[0018] In some embodiments, the method further includes sending a completion status of the write command to the device controller; and instructing, via the device controller, to move the write command to a completion queue of the device attached memory and clear the buffer space occupied by the write command.

[0019] In some embodiments, when the NVMe is a read command, the method further includes: preparing the packet having the read command therein at the drive, and transmitting the read command to the submission queue; identifying the read command in the submission queue via the device controller, and polling a logical to physical (L2P) mapping table to determine a location of the data to be read stored at a memory component in the plurality of memory components; and moving the data corresponding to the read command to the device attached memory to be accessed by the host.

[0020] In some embodiments, upon reading the data, the method includes retrieving a status of the read command at the device controller and placing the read command in a completion queue of the device attached memory.

[0021] Additional aspects of the disclosure relate to a memory device that communicates with a host, the host including a driver for creating commands for writing and reading data. The memory device includes: a memory array including a plurality of memory components for storing memory therein; a device-attached memory including a submission queue and a completion queue for receiving commands from the driver; and a device controller configured to communicate with the device-attached memory, the host, and the plurality of memory components, wherein the device controller receives an interface or link from the driver indicating that a command is placed in the submission queue, and automatically executes any pending commands in the submission queue for completion. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a block diagram illustrating a system including a host and a hybrid memory device according to one or more exemplary embodiments of the present disclosure.

[0023] Figure 2 is a method for describing one or more exemplary embodiments of the present disclosure. Figure 1 A flowchart of an example method for an NVMe write operation of a system.

[0024] Figure 3 is a method for describing one or more exemplary embodiments of the present disclosure. Figure 1 A flowchart of an example method for an NVMe read operation of a system.

[0025] Figure 4 is a method for describing one or more exemplary embodiments of the present disclosure. Figure 1 Flowchart of an example method for CXL memory write / read operations of a system.

[0026] The drawings are used only for the purpose of illustrating the preferred embodiments and should not be construed as limiting the present disclosure. In view of the following illustrative description of the drawings, novel aspects of the presently described technology should become apparent to one of ordinary skill in the art. This detailed description uses numerical and letter names to refer to features in the drawings. Like or similar names in the drawings and description have been used to refer to like or similar parts of embodiments of the presently described technology. DETAILED DESCRIPTION

[0027] In addition, the embodiments may be implemented by hardware, software, firmware, middleware, microcode, hardware description language or any combination thereof. When implemented in software, firmware, middleware or microcode, program code or code segments to perform necessary tasks may be stored in a machine-readable medium. The processor may perform necessary tasks.

[0028] Embodiments of the present disclosure implement NVMe-based protocols and improve the performance of NVMe SSDs by utilizing CXL Type 3 devices with local device attached memory (DAM), where SQs and CQs are placed on the device attached memory instead of being installed on the system memory on the system platform or host. This ensures that whenever the host submits NVMe commands, these commands already exist locally inside the device attached memory that is memory mapped to the host.

[0029] Figure 1 is a block diagram illustrating a system 10. The system 10 includes a memory device 20 that interfaces with a host 30 (e.g., a computer) having a system memory 32 (e.g., DRAM) for storing data and instructions and a drive 34. The host 30 and the system memory 32 may or may not be physically co-located. The host 30 may be located remotely from the system memory 32, or they may be integrated.

[0030] Host 30 may be a host system including, for example, a personal laptop computer, a desktop computer, a mobile device (eg, a cellular phone), a network server, a memory card reader, a camera, or any other suitable type of host system or device.

[0031] According to an embodiment of the present disclosure, the memory device 20 is a single hybrid CXL Type 3 SSD that includes a device controller 22, a memory array including a plurality of memory components (e.g., flash memory devices 24) integrated therein, a local device attached memory (DAM) 26, and a plurality of memory devices 29 (e.g., double data rate (DDR) memory).

[0032] As shown, the host 30 communicates with the system memory 32 and the memory device 20 to transfer and receive data therebetween. The driver 34 is configured to create a submission queue (SQ) 27 and a completion queue (CQ) 28, and transfer data to and from the device attached memory 26.

[0033] Driver 34 is an NVMe driver that is configured to create NVMe commands to be sent to memory device 20 over the PCIe bus using a CXL.io interface or link 40. There are two different types of memory protocols (CXL.io and CXL.mem).

[0034] Device controller 22 comprises a hybrid CXL / NVMe device controller and is configured to distinguish between CXL.io, CXL.mem, and NVMe packets wrapped in CXL.io packets. It can then process the packets accordingly. For example, device controller 22 is configured to determine when a CXL.io packet is being transmitted to it and whether an NVMe packet is inside a CXL.io packet.

[0035] If the device controller 22 determines that there is an NVMe packet, the device controller will fetch the information and operate on the NVMe packet by sending a message to the submission queue 27. However, if the device controller determines that the NVMe packet is strictly a CXL.io packet, the device controller will process the NVMe packet accordingly, and if the NVMe packet is a CXL.mem packet, the device controller 22 will transfer the data to the DDR memory device 29 of the memory device 20 that has been memory mapped to the host 30. According to an embodiment of the present disclosure, the device controller 22 and its components may be implemented in hardware, firmware, and software, or any combination thereof.

[0036] Further in Figure 1 As shown in FIG. 1 , the flash memory device 24 may include NAND flash nonvolatile memory, persistent byte addressable memory, NOR flash nonvolatile memory, or any of these persistent memories or other suitable memories for the purposes described herein. The flash memory device 24 retrieves and stores data therein.

[0037] Memory device 29 may include a DDR DRAM device, such as DDR5, or any other persistent byte-addressable memory, or byte-addressable volatile or non-volatile memory, or any other suitable memory device for the purposes described herein.

[0038] The local device attached memory 26 is memory mapped to the host 30 so that the host 30 can use the device attached memory 26 to create an NVMe submission queue 27 and a completion queue 28 therein.

[0039] Embodiments of the present disclosure implement a producer-consumer model in which the host 30 fills the I / O requests (.NVMe read / write commands) of each CPU core via the driver 34 and rings the doorbell by placing these commands inside the submission queue 27 placed locally in the device-attached memory 26 of the memory device 20; and the memory device 20 automatically consumes any pending commands in the submission queue 27.

[0040] According to an embodiment, there is one submission queue 27 and corresponding completion queue 28 per CPU core. For example, for a 128 core CPU, there will be 128 SQ / CQ pairs inside the device attached memory 26. According to an embodiment, each of these submission queues 27, completion queues 28 is a circular buffer space created in the device attached memory 26 by the driver 34 via the device controller 22 during device enumeration. In some embodiments, for example, the driver 34 can generate up to 64,000 queues and up to 64,000 NVMe commands per queue.

[0041] After a command is completed, it is placed in a completion queue 28 in the device attached memory 26, and then the device controller 22 instructs the buffer space for the command to be cleared. The internal controller logic of the device controller 22 monitors and polls the submission queue 27 in the device attached memory 26 to ensure that all commands are processed and consumed immediately, thereby speeding up the command latency that exists in conventional memory systems.

[0042] In order to minimize latency, it is necessary to make the application data that needs to be stored on the flash memory reside inside the DDR memory device 29 (e.g., DDR5) of the memory device 20. Second, compared to the conventional process in which the SQ / CQ exists on the system memory 32 (e.g., DRAM), in the present disclosure, when an NVMe command is submitted to the device attached memory 26, the device controller 22 can quickly access the local device attached memory 26 and pick up the NVMe command instead of performing DMA access to the system memory.

[0043] Now in reference Figure 2 and 3 Write and read operations of the memory device 20 according to one or more embodiments of the present disclosure are described below.

[0044] Figure 2 It is used for Figure 1 Flowchart of an example method for an NVMe write operation of a system. Figure 2 , a host 30 includes a driver 34 and communicates with a memory device 20 through a bus (eg, Peripheral Component Interconnect Express (PCLe) Gen5) via a CXL interface or link 40.

[0045] As mentioned above, the driver 34 is responsible for creating the submission queue 27 and the completion queue 28 , as well as moving data to and from the device-attached storage 26 of the memory device 20 .

[0046] In operation, the NVMe write command 52 wrapped inside the CXL.io packet 50 is submitted by the host 30 via the driver 34 to the submission queue 27 of the device attached storage 26. That is, the driver 34 utilizes the submission queue 27 and the completion queue 28 in the device attached storage 26 of the memory device 20 to submit the NVMe command.

[0047] It should be noted that based on the application's memory assignment, the data to be written to the flash memory device 24 may reside in the device attached memory 26 (eg, a CXL Type 3 device), or it may reside in the system memory 32 (eg, DRAM).

[0048] The driver 34 is configured to prepare an NVMe command 52 (e.g., an NVMe write command) and a scatter-gather list with physical region pages (PRPs) 54 to enable completion of writing data from memory (e.g., device attached memory 26 or system memory 32) into the flash memory device 24. In some cases, there is all data to be written to the flash memory device 24 to be present in the device attached memory 26. And in other cases, the data may also be present in the system memory 32 (e.g., DRAM).

[0049] According to an embodiment of the present disclosure, the memory device 20 is aware of the address ranges available locally in the device attached memory 26 and the address ranges in the system memory 32. This information can be advertised to the host 30 and the driver 34 by the memory device 20 during device enumeration via PCIe with designated vendor specific extended capabilities (DVSEC). This provides the host 30 and the driver 34 with the address ranges of the device attached memory 26 and the DDR memory device 29. This information can be passed to the device attached memory 26.

[0050] As described above, the device controller 22 is configured to determine whether the CXL.io packet 50 contains an NVMe write command 52. The driver 34 sends the write command 52 wrapped inside the CXL.io packet 50 to the submission queue 27. The device controller 22 is also configured to monitor and identify the presence of any new commands on the submission queue 27, and begin automatically loading / executing those commands. That is, the device controller 22 is configured to poll the submission queue 27 for any new entries, and immediately know and pick up new commands when an entry exists. Alternatively, according to an embodiment, when an entry arrives at the submission queue 27, the driver 34 may ring a doorbell to the device controller 22 to pick up the entry from the submission queue 27 and begin processing it. The command is then processed by the device controller 22, and depending on whether it is a write command 52 or a read command 56, it will follow the respective Figure 2 Or the process shown in 3.

[0051] All commands submitted by the driver 34 are first processed by the device controller 22. The front-end interface logic of the device controller 22 then determines whether the CXL.io packet has NVMe wrapped inside it. If so, the device controller 22 will forward the command directly to the submission queue 27 of the device attached storage.

[0052] The data is then written to the flash memory device 24 (e.g., a NAND flash memory device) from the system memory 32 or the device attached memory 26. However, the completion status of this write command 52 is sent to the device controller 22, and the device controller 22 then instructs the write command 52 to be moved to the completion queue 28 and clears the buffer space associated with the write command 52. The driver 34 is able to access the completion queue to determine the completion status of the write command 52.

[0053] Figure 3 is a method for describing one or more exemplary embodiments of the present disclosure. Figure 1 Flowchart of an example method for NVMe read operations of a system. As shown, similar to Figure 2 The write operation, Figure 3 In the read operation shown in FIG. 5 , the driver 34 prepares a CXL.io packet 50 with an NVMe read command 56 in the CXL.io packet 50 and transmits it to the submission queue 27 of the memory device 20. The device controller 22 also recognizes the NVMe read command 56 in the submission queue 27.

[0054] The device controller 22 then polls a stored logical to physical (L2P) mapping table that associates each data with a logical address. The L2P mapping table stores a mapping of logical addresses to physical addresses that indicate the location of the data stored in the flash memory device 24. The device attached memory 26 moves the data corresponding to the read command 56 into the device attached memory 26 that can then be accessed by the host 30.

[0055] Since the device attached memory 26 is memory mapped to the host 30, the host 30 can read data directly from the device attached memory 26. Therefore, no direct memory access (DMA) transfer to the system memory 32 is required if the SQ / CQ exists in RAM instead of the device attached memory 26. Once the data is read, the status of the read command 56 is retrieved by the device controller 22, and the read command 56 is then placed in the completion queue 28.

[0056] The present disclosure provides the benefit of creating data buffers for write commands 52 and read commands 56 on device attached memory local to the memory device 20. For example, since the read command 56 is placed in the command (submission queue 27 or completion queue 28), the memory device 20 will also create data buffers for the read command 56. Therefore, in the present disclosure, everything is created on the local device attached memory, and the host 30 is notified when the command is completed. The host 30 can still read data from the device attached memory 26 via load storage transactions.

[0057] Figure 4is a method for describing one or more exemplary embodiments of the present disclosure. Figure 1 Flowchart of an example method of CXL memory write / read operations for a system of FIG. 10 . As shown, existing data may also be processed directly between the host 30 and the device attached memory 26 via CXL memory write commands 58 and CXL memory read commands 60. The driver 34 sends a CXL memory write command 58 or a memory read command 60 of a CXL.mem packet 62 to the device controller 22, and the device controller 22 processes the CXL write command 58 or CXL read command 60 associated data requested as part of a load / store directly to a corresponding address on the DDR memory device 29, retrieves or writes the data, and, if retrieved, sends the data back to the host 30. Thus, the data is written or read from the DDR memory device 29.

[0058] Several embodiments of the present disclosure provide the benefits of such a single hybrid device that can be used as a CXL Type 3 device and an SSD device, resulting in an improved optimized SSD with less command latency and capable of operating as both an SSD and a memory expander or accelerator more cost effectively.

[0059] Although specific embodiments have been illustrated and described herein, it will be appreciated by those of ordinary skill in the art that arrangements calculated to achieve the same results may be substituted for the specific embodiments shown. The present disclosure is intended to cover adaptations or variations of several embodiments of the present disclosure. It should be understood that the above description has been made in an illustrative manner and not in a restrictive manner.

[0060] A person of ordinary skill in the art will understand the combination of the above embodiments and other embodiments not explicitly described herein after reviewing the above description. The scope of several embodiments of the present disclosure includes other applications using the above structures and methods. Therefore, the scope of several embodiments of the present disclosure should be determined with reference to the attached claims and the full range of equivalents to which such claims are entitled.

[0061] In the foregoing detailed description, some features are grouped in a single embodiment for the purpose of simplifying the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the disclosed embodiments of the disclosure must use more features than those explicitly recited in each claim. Rather, as the appended claims reflect, the inventive subject matter resides in less than all the features of a single disclosed embodiment. Therefore, the appended claims are hereby incorporated into the detailed description, with each claim standing on its own as a separate embodiment.

Claims

1. A system comprising: a host in communication with the system memory and including a driver configured to create commands for writing and reading data; as well as a memory device in communication with the host and configured to store data therein, the memory device comprising: a memory array comprising a plurality of memory components for storing memory therein, a device attached memory including a submission queue and a completion queue for receiving commands from the driver, and a device controller configured to communicate with the device attached storage, the host, and the plurality of storage components, wherein the device controller receives an interface or link from the driver indicating that commands are placed in the submission queue and automatically executes any pending commands in the submission queue for completion.

2. The system of claim 1, wherein the device attached memory is memory mapped to the host.

3. The system of claim 1, wherein the driver comprises a Non-Volatile Memory Express (NVMe) type driver and is configured to create NVMe commands.

4. The system of claim 3, wherein the device controller comprises a hybrid compute express link (CXL) / NVMe device controller, wherein the device controller is configured to determine a type of CXL packet received from the drive via the interface or link and an NVMe command wrapped inside the CXL packet.

5. The system of claim 4, wherein the memory device is a CXL Type 3 solid-state memory device and the plurality of memory components each comprise flash memory.

6. The system of claim 5, wherein upon creation of the NVMe command via the driver, the host is configured to ring a doorbell by placing the NVMe command inside the submission queue at the device attached storage, and the storage device automatically consumes any pending commands in the submission queue.

7. The system of claim 6, wherein once a corresponding NVMe command is completed, the device controller is configured to move the completed NVMe command to be placed in the completion queue in the device attached memory, and then instruct to clear the buffer space for the corresponding NVMe command.

8. The system of claim 5, wherein when the NVMe command comprises a write command, the device controller is configured to confirm whether the packet received at the submission queue includes the write command, and the device controller then begins automatically executing the write command, wherein associated data is written to a memory component of the plurality of memory components.

9. The system of claim 5, wherein when the NVMe is a read command, the driver is configured to prepare the packet having the read command therein and transmit the read command to the submission queue, and the device controller is configured to identify the read command in the submission queue and poll a logical-to-physical L2P mapping table to determine a location of the data stored at a memory component in the plurality of memory components to be read, and the device attached memory is then configured to move the data corresponding to the read command to the device attached memory which is then accessible by the host.

10. A method for performing a write operation of data in a memory device including a device-attached memory, a device controller, and a plurality of memory components, the method comprising: Creating commands for writing and reading data via a host's driver; receiving the command in a submission queue at the device attached memory; Communicating with the device attached storage, the host, and a plurality of storage components via the device controller, including receiving an interface or link from the driver via the device controller indicating that commands are placed in the submission queue, and automatically executing any pending commands in the submission queue for completion.

Images