CMB CASHING USING HYBRID SRAM / DRAM DATAPATH

Abstract

To increase CMB performance, a controller memory buffer (CMB) caching mechanism can be used. Instead of reading and writing data from static random access memory (SRAM), data is read from SRAM. Reading data from CMB to SRAM increases performance, but leaves little space to process both read and write commands. Using dynamic random access memory (DRAM) for write commands and CMB in SRAM for read commands enables higher performance. Due to the limited space in SRAM, commands are flushed when they are read by the host. This allows relevant data stored in SRAM to be used for the next command but then flushed for processing the next command. The performance increase is possible without using additional SRAM or DRAM.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Non-Provisional Application No. 18 / 359,159, entitled "CMB CACHING USING HYBRID SRAM / DRAM DATA PATH," filed on July 26, 2023, in the U.S. Patent and Trademark Office, which claims priority to U.S. Provisional Application No. 63 / 497,784, filed on April 24, 2023, and hereby incorporates the entire contents of this application by reference for all purposes. BACKGROUND OF REVELATIONArea of Revelation

[0002] Embodiments of the present disclosure generally relate to controller memory buffer (CMB) caching to enhance CMB performance. Description of the state of the art

[0003] Non-Volatile Memory Express (NVMe) is based on a paired submission and completion queue mechanism. Commands are placed in a submission queue by the host software. Completions are placed in the associated completion queue by a controller. Generally, submission and completion queues are allocated in host memory, with each queue being physically contiguous or non-contiguous in host memory. However, CMB features allow the host to place submission queues, completion queues, physical region page (PRP) lists, scatter gather list (SGL) segments, and data buffers in controller memory.

[0004] The Persistent Memory Region (PMR) is an optional area of general-purpose persistent read / write memory that can be used for a variety of purposes. The PMR address space is defined by a Peripheral Interconnect Base Address Register (PCI BAR) and occupies the entire address space shared by the BAR. PMR supports the required features of the PCI Express (PCIe) programming model (i.e., PMR does not in any way restrict what PCI Express otherwise allows). The contents of the PMR persist even after PMR is disabled, after controller and NVM subsystem resets, and after power cycles.

[0005] There are several different types of read / write accesses that can occur with CMB. Sector data is read or written using a dedicated address range within the CMB address space. An NVMe submit / completion queue reads or writes using a dedicated address range within the CMB address space. Furthermore, an NVMe PRP list or SGL segment reads or writes using a dedicated address range within the CMB address space.

[0006] CMB performance varies depending on whether the processor uses static random-access memory (SRAM) in the controller or dynamic random-access memory (DRAM) attached to the controller to store the CMB accesses. The normal data path is through SRAM (for best performance and power consumption), so DRAM is designed to store metadata rather than as part of the data path. This means that the DRAM interface on the controllers is small (e.g., 32-bit bus width), and the DRAM is used for millions of small random metadata reads and writes (significantly reducing DRAM efficiency). Adding CMB traffic to this metadata-optimized DRAM path limits host performance to a much lower level.

[0007] In the previous approach, a CMB can be integrated into either DRAM or SRAM, but without cache management. Therefore, the main disadvantage is CMB performance and latency. Cache algorithms used are traditional cache mechanisms (e.g., Least Recently Used (LRU)) that are not adapted to CMB, resulting in a low hit rate. Dedicating a very large amount of SRAM in the controller's SSD adds significant cost. Using a wider DRAM interface (e.g., 64-bit bus width) to provide more DRAM significantly reduces raw bandwidth efficiency. A wider DRAM bus increases controller cost and drives up DRAM costs for smaller drives. Using two separate DRAM interfaces, one for metadata and one for the data path including CMB, also adds significant controller cost.

[0008] Therefore, there is a need in the art for a CMB caching mechanism to enhance CMB performance. SUMMARY OF REVELATION

[0009] To increase CMB performance, a controller memory buffer (CMB) caching mechanism can be used. Instead of reading and writing data from static random access memory (SRAM), data is read from SRAM. Reading data from CMB to SRAM increases performance, but leaves little space to process both read and write commands. Using dynamic random access memory (DRAM) for write commands and CMB in SRAM for read commands enables higher performance. Due to the limited space in SRAM, commands are flushed when they are read by the host. This allows relevant data stored in SRAM to be used for the next command but then flushed for processing the next command. The performance increase is possible without using additional SRAM or DRAM.

[0010] In one embodiment, a data storage device comprises: a memory device; and a controller coupled to the memory device, the controller having a controller memory buffer (CMB), the controller configured to: receive a read command from a host device; retrieve data from the memory device, the data associated with the read command; write the retrieved data to a CMB cache of the CMB; inform the host device that the read command is complete; and delete the retrieved data from the CMB cache after the host device reads the retrieved data from the CMB cache.

[0011] In another embodiment, a data storage device comprises: a memory device; and a controller coupled to the memory device, the controller configured to: receive a read command from a host device; read data from the memory device, wherein the read data corresponds to the read command; place the read data into the controller memory buffer (CMB) cache; determine that the host device has read the read data from the CMB cache; find, in the mapping table, the corresponding page physical region (PRP) pointer or scatter-gather list (SGL) pointer; delete the read data from the CMB cache; and delete the PRP pointer or the SGL pointer from the mapping table.

[0012] In another embodiment, a data storage device comprises: means for storing data; and a controller coupled to the means for storing data, the controller configured to: retrieve data from the means for storing data; write the retrieved data to static random access memory (SRAM); detect that the retrieved data was received from a host device; and delete the retrieved data from the DRAM based on the detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order that the above-mentioned features of the present disclosure may be understood in detail, a more particular description of the disclosure, briefly summarized above, follows with reference to embodiments, some of which are illustrated in the accompanying drawings. It should be noted, however, that the accompanying drawings illustrate only typical embodiments of this disclosure and are therefore not to be considered limiting of its scope, for the disclosure may admit to other equally effective embodiments. Fig. 1 is a schematic block diagram illustrating a storage system in which a data storage device may function as a storage device for a host device, according to certain embodiments. Fig. 2 is a block diagram illustrating a method of operating a storage device to execute a read or write command according to certain embodiments. Fig. 3A is a block diagram illustrating a method for processing data over a data path without CMB according to one embodiment. Fig. 3B is a block diagram illustrating a method for processing data over a data path with a CMB according to one embodiment. Fig. 4A is a block diagram illustrating a method for processing data with a CMB / PMR in SRAM according to one embodiment. Fig. 4B is a block diagram illustrating a method for processing data with a CMB in DRAM according to one embodiment. Fig. 5 is a block diagram illustrating a method of an asymmetric power CMB according to an embodiment. Fig. 6 is a schematic block diagram illustrating a memory system in which CMB is located in DRAM while a CMB cache is located in SRAM, according to certain embodiments. Fig. 7 is a schematic block diagram illustrating a method of a CMB caching mechanism according to certain embodiments. Fig. 8 is a flowchart illustrating a method for a CMB caching mechanism according to certain embodiments. Fig. 9 is a schematic block diagram illustrating a memory system with a PRP / SGL-based caching algorithm according to certain embodiments. Fig. 10 is a flowchart illustrating a method for a CMB read / write process in SRAM according to certain embodiments.

[0014] To facilitate understanding, identical reference numerals have been used, wherever possible, to designate identical elements present throughout the figures. It is understood that elements disclosed in one embodiment may be advantageously utilized in other embodiments without specific mention. DETAILED DESCRIPTION

[0015] Reference is made below to embodiments of the disclosure. However, it should be understood that the disclosure is not limited to the specific embodiments described. Rather, any combination of the following features and elements, whether or not related to different embodiments, is intended to implement and practice the disclosure. Furthermore, although embodiments of the disclosure may achieve advantages over other possible solutions and / or over the prior art, whether or not a particular advantage is achieved by a given embodiment does not constitute a limitation of the disclosure.Thus, the following aspects, features, embodiments, and advantages are merely illustrative and are not to be considered elements or limitations of the appended claims unless expressly recited in one or more claims. Likewise, a reference to "the disclosure" is not to be construed as a generalization of any subject matter disclosed herein and is not to be considered elements or limitations of the appended claims unless expressly recited in one or more claims.

[0016] To increase CMB performance, a controller memory buffer (CMB) caching mechanism can be used. Instead of reading and writing data from static random access memory (SRAM), data is read from SRAM. Reading data from CMB to SRAM increases performance, but leaves little space to process both read and write commands. Using dynamic random access memory (DRAM) for write commands and CMB in SRAM for read commands enables higher performance. Due to the limited space in SRAM, commands are flushed when they are read by the host. This allows relevant data stored in SRAM to be used for the next command but then flushed for processing the next command. The performance increase is possible without using additional SRAM or DRAM.

[0017] Fig. 1 is a schematic block diagram illustrating a storage system 100 including a data storage device 106 that may function as a storage device for a host device 104, according to certain embodiments. For example, the host device 104 may utilize a non-volatile memory (NVM) 110 included in the data storage device 106 to store and retrieve data. The host device 104 includes a host dynamic random access memory (DRAM) 138. In some examples, the storage system 100 may include a plurality of storage devices, such as the data storage device 106, that may function as a storage array. For example, the storage system 100 may include a plurality of data storage devices 106 configured as a redundant array of low-cost / independent disks (RAID) that collectively function as a mass storage device for the host device 104.

[0018] The host device 104 may store and / or retrieve data on one or more storage devices, such as the data storage device 106. As in Fig. 1, the host device 104 may communicate with the data storage device 106 via an interface 114. The host device 104 may include a wide variety of devices, including computer servers, network attached storage (NAS) units, desktop computers, notebook computers (i.e., laptops), tablet computers, set-top boxes, telephone handsets such as so-called "smartphones," so-called "smart pads," televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, or other devices capable of sending or receiving data to or from a data storage device.

[0019] The host DRAM 138 may optionally include a host memory buffer (HMB) 150. The HMB 150 is a portion of the host DRAM 138 dedicated to the data storage device 106 for exclusive use by a controller 108 of the data storage device 106. For example, the controller 108 may store mapping data, buffered instructions, logical-to-physical (L2P) tables, metadata, and the like in the HMB 150. In other words, the HMB 150 may be used by the controller 108 to store data that would normally be stored in volatile memory 112, a buffer 116, internal memory of the controller 108, such as static random access memory (SRAM), and the like.

[0020] The data storage device 106 includes the controller 108, NVM 110, a power supply 111, a volatile memory 112, the interface 114, a write buffer 116, and an optional DRAM 118. In some examples, the data storage device 106 may include additional components, which are not shown for clarity. Fig. 1. For example, the data storage device 106 may include a printed circuit board (PCB) to which components of the data storage device 106 are mechanically attached and which includes electrically conductive traces that electrically interconnect components of the data storage device 106, or the like. In some examples, the physical dimensions and connection configurations of the data storage device 106 may conform to one or more standard form factors. Some examples of standard form factors include, but are not limited to, 3.5-inch data storage devices (e.g., an HDD or SSD), 2.5-inch data storage devices, 1.8-inch data storage devices, Peripheral Component Interconnect (PCI), PCI-Extended (PCI-X), PCI Express (PCIe) (e.g., PCIe x1, x4, x8, x16, PCIe Mini Card, MiniPCI, etc.).In some examples, the data storage device 106 may be directly coupled to a motherboard of the host device 104 (e.g., directly soldered or plugged into a connector).

[0021] The interface 114 may include a data bus for exchanging data with the host device 104 and / or a control bus for exchanging commands with the host device 104. The interface 114 may operate according to any suitable protocol. For example, the interface 114 may operate according to one or more of the following protocols: PCIe, Non-Volatile Memory Express (NVMe), OpenCAPI, GenZ, Cache Coherent Interface Accelerator (CCIX), Open Channel SSD (OCSSD), or the like. The interface 114 (e.g., the data bus, the control bus, or both) is electrically connected to the controller 108 and provides an electrical connection between the host device 104 and the controller 108 such that data can be exchanged between the host device 104 and the controller 108.In some examples, the electrical connection of the interface 114 may also enable the data storage device 106 to receive power from the host device 104. For example, as shown in FIG. Fig. 1, the power supply 111 may receive power from the host device 104 via interface 114.

[0022] The NVM 110 may include a plurality of storage devices or storage units. NVM 110 may be configured to store and / or retrieve data. For example, a storage unit of the NVM 110 may receive data and a message from controller 108 instructing the storage unit to store the data. Similarly, the storage unit may receive a message from controller 108 instructing the storage unit to retrieve data. In some examples, each of the storage units may be referred to as a die. In some examples, the NVM 110 may include a plurality of dies (i.e., a plurality of storage units). In some examples, each storage unit may be configured to store relatively large amounts of data (e.g., 128 MB, 256 MB, 512 MB, 1 GB, 2 GB, 4 GB, 8 GB, 16 GB, 32 GB, 64 GB, 128 GB, 256 GB, 512 GB, 1 TB, etc.).

[0023] In some examples, each memory unit may include any type of non-volatile memory device, such as flash memory devices, phase change memory (PCM) devices, resistive random access memory (ReRAM) devices, magnetoresistive random access memory (MRAM) devices, ferroelectric random access memory (F-RAM), holographic memory devices, and any other type of non-volatile memory device.

[0024] The NVM 110 may include a plurality of flash memory devices or memory units. NVM flash memory devices may include NAND- or NOR-based flash memory devices and may store data based on a charge contained in a floating gate of a transistor for each flash memory cell. In NVM flash memory devices, the flash memory device may be divided into a plurality of dies, where each die of the plurality of dies includes a plurality of physical or logical blocks, which may be further divided into a plurality of pages. Each block of the plurality of blocks within a particular memory device may include a plurality of NVM cells. Rows of NVM cells may be electrically connected using a wordline to define one of a plurality of pages.Respective cells in each of the plurality of pages may be electrically connected to respective bitlines. Furthermore, NVM flash memory devices may be 2D or 3D devices and may be of the single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), or quad-level cell (QLC) type. Controller 108 may write and read data to and from NVM flash memory devices at the page level and erase data from NVM flash memory devices at the block level.

[0025] The power supply 111 may provide power to one or more components of the data storage device 106. When operating in default mode, the power supply 111 may provide power to one or more components using power provided by an external device, such as the host device 104. For example, the power supply 111 may provide power to the one or more components using power received from the host device 104 via interface 114. In some examples, the power supply 111 may include one or more power storage components configured to provide power to the one or more components when in shutdown mode, such as when power is no longer received from the external device.In this way, the power supply 111 may function as an integrated backup power source. Some examples of the one or more power storage components include, but are not limited to, capacitors, supercapacitors, batteries, and the like. In some examples, the amount of power that can be stored by the one or more power storage components may be a function of the cost and / or size (e.g., area / volume) of the one or more power storage components. In other words, as the amount of power stored by one or more power storage components increases, the cost and / or size of the one or more power storage components also increases.

[0026] The volatile memory 112 may be used by the controller 108 to store information. The volatile memory 112 may include one or more volatile storage devices. In some examples, the controller 108 may use the volatile memory 112 as a cache. For example, the controller 108 may store cached information in the volatile memory 112 until the cached information is written to the NVM 110. As shown in Fig. 1, the volatile memory 112 may consume power received from the power supply 111. Examples of volatile memory 112 include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static RAM (SRAM), and synchronous dynamic RAM (SDRAM (e.g., DDR1, DDR2, DDR3, DDR3L, LPDDR3, DDR4, LPDDR4, and the like)). Likewise, the optional DRAM 118 may be utilized to store mapping data, buffered instructions, logical-to-physical (L2P) tables, metadata, cached data, and the like in the optional DRAM 118. In some examples, the data storage device 106 does not include the optional DRAM 118, so the data storage device 106 does not include DRAM. In other examples, the data storage device 106 includes the optional DRAM 118.

[0027] The controller 108 may manage one or more operations of the data storage device 106. For example, the controller 108 may manage reading data from and / or writing data to the NVM 110. In some embodiments, when the data storage device 106 receives a write command from the host device 104, the controller 108 may initiate a data store command to store data in the NVM 110 and monitor the progress of the data store command. The controller 108 may determine at least one operational characteristic of the storage system 100 and store at least one operational characteristic in the NVM 110. In some embodiments, when the data storage device 106 receives a write command from the host device 104, the controller 108 temporarily stores the data associated with the write command in the internal memory or write buffer 116 before sending the data to the NVM 110.

[0028] The controller 108 may include an optional second volatile memory 120. The optional second volatile memory 120 may be similar to the volatile memory 112. For example, the optional second volatile memory 120 may be SRAM. The controller 108 may allocate a portion of the optional second volatile memory to the host device 104 as a controller memory buffer (CMB) 122. The CMB 122 may be directly accessed by the host device 104. For example, instead of maintaining one or more submission queues in the host device 104, the host device 104 may utilize the CMB 122 to store the one or more submission queues normally maintained in the host device 104.In other words, the host device 104 may generate instructions and store the generated instructions with or without the associated data in the CMB 122, with the controller 108 accessing the CMB 122 to retrieve the stored generated instructions and / or the associated data.

[0029] Fig. 2 is a block diagram illustrating a method 200 for operating a storage device to execute a read or write command according to one embodiment. The method 200 may be implemented with the storage system 100 of Fig. 1, which includes the host device 104 and the data storage device 106, wherein the data storage device includes the controller 108.

[0030] Method 200 begins with operation 250, in which the host device writes a command as an entry to an SQ. The host device may write one or more commands to the SQ at operation 250. The commands may be read or write commands. The host device may include one or more SQs.

[0031] In operation 252, the host device writes one or more updated SQ tail pointers and rings the doorbell or sends an interrupt signal to notify or signal the storage device of the new instruction ready for execution. The host may write an updated SQ tail pointer and send a doorbell or interrupt signal for each of the SQs if more than one SQ is present. In operation 254, in response to receiving the doorbell or interrupt signal, a controller of the storage device retrieves the instruction from one or more SQs and receives the instruction.

[0032] In operation 256, the controller processes the command and writes or transfers data associated with the command to the host device's memory. The controller can process more than one command at a time. The controller can process one or more commands in submission order or in sequential order.

[0033] Once the instruction has been fully processed in operation 258, the controller writes a completion entry corresponding to the executed instruction to a completion queue (CQ) of the host device and moves or updates the CQ head pointer to point to the newly written completion entry.

[0034] In operation 260, the controller generates and sends an interrupt signal, or doorbell, to the host device. The interrupt signal indicates that the instruction has been executed and the data associated with the instruction is available in the storage device. The interrupt signal also notifies the host device that the CQ is ready to be read or processed.

[0035] In operation 262, the host device processes the completion entry. In operation 264, the host device writes an updated CQ head pointer to the storage device and rings the doorbell or sends an interrupt signal to the storage device to release the completion entry.

[0036] Fig. 3A is a block diagram illustrating a method 300 for processing data over a non-CMB data path according to one embodiment. The primary use is for a drive behind a bridge (e.g., NVMe-OF bridge, Analytics Engine), where the SSD is connected to the backend of the bridge (i.e., the drive does not directly connect to a host).

[0037] Without CMB in the SSD drives, all host data, such as Host 104 from Fig. 1, first provisioned in the bridge DRAM and then sent to the SSD, resulting in all host data going in and out of the bridge DRAM. Bridge DRAM has high bandwidth and is not scalable. When many SSDs are connected to the bridge, all host traffic goes through the bridge's DRAM. All processing by NVMe is based on memory addresses. When the host wants to write or read data, the host communicates with the bridge. To read data, the host sends an NVMe command over NVMe Ethernet, requesting a piece of data at an address in the SSD. The DRAM then proceeds with the command over the PCIe NVMe interface to read the address in the SSD from the host. To write data, the host sends an NVMe command over NVMe Ethernet, requesting a piece of data in the host DRAM. The bridge then stores the data from the host DRAM.Data continues to be written from the bridge DRAM to the SSD via the PCIe NVMe interface. Placing many SSDs behind the bridge can cause performance bottlenecks in the bridge.

[0038] To avoid performance bottlenecks, some or all of the host data bypasses the bridge DRAM and goes directly to a CMB in an SSD. Fig. 3B is a block diagram illustrating a method 350 for processing data over a data path using a CMB, according to one embodiment. For example, the CMB may be used to store host data (rather than storing host data in the bridge DRAM). Adding a storage element to the SSD avoids the performance bottleneck issue associated with using the bridge DRAM for read and write commands.

[0039] When the host requests that a data element be written to the host DRAM, the data is read from the NAND. Once the data is read from the NAND, it is inserted into the SSD's CMB. The data is then passed to the bridge DRAM via the PCIe NVMe interface. The bridge DRAM does not store the data but instead passes it to the host DRAM via NVMe Ethernet. The bridge DRAM is bypassed because the SSD creates a storage element that can be used as a memory buffer for reading from and writing to the host.

[0040] The CMB / PMR size is a crucial factor in determining where the CMB / PMR data can be stored on the SSD. Fig. 4A is a block diagram illustrating a method 400 for processing data with a CMB / PMR in SRAM according to one embodiment. There is minimal internal contention because the SRAM bandwidth is designed to match the PCIe bandwidth. Therefore, line-rate CMB / PMR is possible. If the size is small enough, CMB / PMR can be stored in spare SRAM (e.g., SRAM that would be used for other advanced features that are disabled). The small buffer size allows a PMR SRAM buffer to be protected during power failures (pFail). However, due to the small SRAM size, not much SRAM remains.

[0041] If the CMB size is larger than the maximum CMB SRAM size, the CMB is kept in DRAM. Fig. 4B is a block diagram illustrating a method 450 for processing data using a CMB in DRAM, according to one embodiment. There is significant internal contention because the DRAM bandwidth would be used by both CMB data and FTL metadata (e.g., L2P table). Even with 100% CMB traffic (no sector traffic), the DRAM bandwidth is unlikely to allow line-rate performance. With mixed CMB and sector traffic, either CMB performance or sector data performance is reduced. This is especially true for write workloads, where the highest metadata traffic (random small reads / writes) is to DRAM. Therefore, using CMB in DRAM results in a significant speed reduction compared to that achieved when placing the CMB in SRAM.

[0042] To avoid space losses due to a CMB in SRAM and speed losses due to a CMB in DRAM, asymmetric performance is proposed. Fig. 5 is a block diagram illustrating a method 500 of a CMB with asymmetric performance according to one embodiment. For asymmetric performance, the CMB host's reads must be performed via SRAM (full bandwidth), while the CMB host's writes must be performed via DRAM (reduced bandwidth). For asymmetric performance, it is also necessary for the controller, such as controller 108 of Fig. 1, a CMB address mapping table manages the CMB address space into DRAM and SRAM addresses. The CMB address mapping table is similar to a logical-physical mapping table for flash. Maintaining a CMB address mapping table enables a CMB PCIe read from the CMB sector data in SRAM. Once the SRAM is read, the data in SRAM is erased, allowing subsequent read commands to be stored in SRAM for processing.

[0043] To enable basic support for only CMB sector data / metadata (no CMB queues or PRP / SGL lists), the CMB address mapping table would need to implement a hybrid mode. In hybrid mode, PCIe writes to CMB (sector data / metadata) could initially be placed in the elastic SRAM buffer, and while in the elastic SRAM buffer, the CMB address mapping would reflect that SRAM location. However, when PCIe writes move from the elastic buffer to DRAM, the CMB address mapping must be updated to point to the DRAM location. Alternatively, if the elastic buffer is not needed, CMB writes (sector data / metadata) are written directly to a DRAM location, and the CMB address mapping points to the DRAM location.PCIe reads of CMB (sector data / metadata) would always look up the CMB address mapping table to learn where to obtain the requested data. If an NVMe read command has the CMB as the destination address, the data read from NAND would be placed in SRAM. The CMB address mapping for the destination address would be updated to point to data in SRAM. Once data is read from SRAM, instead of reading data from SRAM and sending the read data to DRAM, the data is erased, and DRAM is used exclusively for write commands. Thus, a CMB read of the results of an NVMe read command would produce maximum performance (e.g., line rate).Alternatively, if the host writes sector data to CMB and then reads that sector data directly (using CMB as a scratchpad), reading that data back will occur with reduced DRAM performance if the write data has been moved to DRAM.

[0044] CMB / PMR can support the following structures. CMB / PMR is becoming a hot topic in the next-generation enterprise market because of the feature's direct impact on performance, especially in a PCIe fabric topology. Additionally, CMB / PMR reduces the amount of memory implemented in host DRAM. Admin or I / O queues can be placed in the CMB, and for a given queue, all queue-associated memory must reside either in the CMB or in host memory.

[0045] The controller may support physical region pages (PRPs) and scatter gather lists (SGLs) in the CMB. For a given PRP list or SGL associated with a single instruction, all memory associated with the PRP list or SGLs must reside either in the CMB or in host memory. The PRPs and SGLs for an instruction may only be placed in the CMB if the associated instruction is present in a submission queue in the CMB.

[0046] The controller can support data and metadata in the CMB. All data or metadata associated with a particular command must be located either in the CMB or in host memory.

[0047] The system discussed herein integrates CMB into DRAM while also having a CMB cache in SRAM to improve performance and reduce host latency.

[0048] To avoid reduced DRAM performance, a CMB is stored in the DRAM and the controller. Fig. 6 is a schematic block diagram illustrating a memory system 600 in which CMB is located in DRAM while a CMB cache is located in SRAM, according to certain embodiments. For the system to be feasible, there should be a strong and unique caching algorithm adapted to CMB. Otherwise, the cache buffer will not help. As discussed herein, a unique, intelligent caching algorithm is adapted to CMB accesses, focusing on the data buffers more relevant to the compute segment. Previously, it was possible to integrate CMB into DRAM while simultaneously providing a cache buffer for user data in SRAM, but the caching algorithms are conventional mechanisms not adapted to CMB, resulting in a low hit rate.

[0049] The storage system 600 includes a host, such as host 104 of Fig. 1, a controller, such as controller 108 from Fig. 1, a DRAM CMB 604, and a NAND 606. The controller further includes a CMB cache 602. The DRAM CMB 604 is a DRAM comprising a CMB. As discussed herein, the CMB caching mechanism is adapted to CMB, which exhibits very high hit rates. With the proposed approach, it would be practical to have a CMB cache, which has not been implemented today. High-speed CMB caching mechanisms increase overall system performance and quality of service (QoS), enabling a practical solution for CMB.

[0050] Fig. Figure 7 is a schematic block diagram illustrating a method 700 of a CMB caching mechanism according to certain embodiments. The basic assumption of the CMB caching mechanism is that the host, such as host 104 of Fig. 1, retrieves the data sequentially from a CMB per instruction.

[0051] In previous approaches, the corresponding PRP / SGL pointers were deleted upon completion of an instruction because they were no longer needed. The current approach proposes to retain the pointers until the host reads the data from the CMB. The controller, such as controller 108 from Fig. 1, uses these pointers to manage the CMB cache, such as the CMB cache 602 from Fig. 6, which results in a very high hit rate. After the host has read the data from the CMB in SRAM, the data can be erased. Unlike sending the data to DRAM, the corresponding PRP / SGL pointers are erased after reading. Erasing the pointers enables subsequent read commands in the CMB cache, which results in the aforementioned high hit rate.

[0052] The device controller maintains previously issued PRP / SGL pointers in an internal database. When the host accesses the CMB to retrieve data, the device finds the corresponding pointer in the internal database. Then, the next pointer associated with the same previously issued host command is used to populate a new entry in the cache. When populating the new entry, it assumes that the next CMB access associated with the command would be the next pointer. The corresponding PRP / SGL is removed from the internal database. Later, the host issues the next CMB access associated with the command while keeping the data in the CMB cache. Again, the next PRP / SGL pointer is used to populate the next entry in the cache.

[0053] Fig. 8 is a flowchart illustrating a method 800 for a CMB caching mechanism according to certain embodiments. The host, such as host 104 of Fig. 1, issues a read command at 802, requesting data from NAND, such as NAND 606 from Fig. 6, into the CMB cache, such as CMB cache 602. The device retrieves the data from the NAND at 804 and stores the data in the CMB, while at 806 a first block of data is placed into the CMB cache 602. The controller, such as controller 108 of Fig. 1, completes the instruction, but internally holds the PRP / SGL pointers at 808. The host then begins reading the data from the CMB cache 602 at 810. The controller then finds the relevant PRP / SGL pointer in the internal database at 812. The next PRP / SGL pointer (if any) is used to insert new data into the CMB. Finally, the data from the CMB cache is returned to the host.

[0054] More specifically, method 800 begins at block 802. In block 802, the host issues a read command from NAND to the CMB. In block 804, the controller reads the data from NAND into the CMB. In block 806, the controller places the first data (e.g., 4 KB) into the CMB cache. In block 808, the controller completes the command but keeps the PRP / SGL buffers internally. In block 810, the host reads the data from the CMB. In block 812, the controller finds the relevant PRP / SGL pointer in the internally maintained database. In block 814, the controller determines whether the data is in the CMB cache. If the controller determines that the data is not in the CMB cache, method 800 proceeds to block 816. If the controller determines that the data is in the CMB cache, method 800 proceeds to block 818. In both block 816 and block 818, the controller provides the data back from the CMB or cache and removes the relevant PRP / SGL.The removal of relevant PRP / SGL is not limited to after the host has read the command. The removal can occur during the host's reading of a command or after the host has completely read all read commands. The removal of data is not limited to the execution of the read command. Furthermore, after completing block 816 and block 818, the method 800 proceeds to block 820. In block 820, the controller determines whether the transfer is the last transfer for the command. If the controller determines that the transfer is not the last transfer for the command, the method 800 returns to block 812. If the controller determines that the transfer is the last transfer for the command, the method 800 proceeds to block 822 to terminate the method 800.

[0055] Fig. 9 is a schematic block diagram illustrating a memory system 900 with a PRP / SGL-based cache algorithm according to certain embodiments. The controller, such as controller 108 of Fig. 1, integrates the CMB cache 602, while the cache algorithm is based on the previous PRPs / SGLs provided by the host, such as host 104 from Fig. 1, for which relevant legacy host commands were provided. The PRP / SGL buffers are maintained in a dedicated buffer even after the command completes, until the host retrieves all data from the CMB.

[0056] The storage system 900 includes a host, a controller, a DRAM CMB 604, and a NAND 606. The controller includes one or more processors 904, a flash interface module (FIM) 908 for connecting to a storage device, a host interface module (HIM) 912 for connecting to the host, a command scheduler 906 coupled between FIM 908 and HIM 912, an encryption / decryption module 916 disposed between FIM 908 and HIM 912, and a data path, ECC and RAID 914, between the encryption / decryption module 916 and FIM 908. The HIM 912 decides when to execute the write command. The FIM 908 brings data from the NAND 606. The controller also includes a CMB manager 902 that holds the PRP / SGL buffers 910 and the CMB cache 602.

[0057] Fig. 10 is a flowchart illustrating a method 1000 for a CMB read / write process in SRAM according to certain embodiments. The method 1000 begins at block 1002. At block 1002, the controller, like controller 108 of Fig. 1, a read command. In block 1004, the controller retrieves data for the read command from a storage device. In block 1006, the controller writes the retrieved data to the SRAM in the CMB cache, as described in the CMB cache 602 of Fig. 6. The controller may optionally detect an additional read command from the NAND before block 1006 is completed. In block 1008, the controller updates the mapping table for the CMB. At block 1010, the controller writes the completion to the completion queue, as in the completion queue of Fig.2. In block 1012, the controller detects that the host device has read / retrieved data from the SRAM. In block 1014, the controller clears the retrieved data from the SRAM and updates the mapping table.

[0058] The disclosure assumes that the host reads the CMD data sequentially per command, which is a valid assumption, and several hosts behave this way. In one embodiment, the device controller may detect that the specific host uses a different approach to fetch ordering. In such a scenario, cache management adapts to the host behavior, and the cache buffers are filled accordingly.

[0059] Overall performance is increased and latency is reduced because the SRAM is used as a "cache" buffer, which is adapted to NVMe transactions and has a very high hit rate with the current approach. Other advantages include the device being able to advertise support for very large CMBs and PMRs, since the CMB / PMR is integrated in DRAM rather than SRAM, which is especially important when multiple or even many CMBs / PMRs are present, such as in a virtualization environment. Additionally, a huge area for application-specific integrated circuits (ASICs) is not required to support a huge CMB / PMR size.

[0060] In one embodiment, a data storage device comprises: a memory device; and a controller coupled to the memory device, the controller having a controller memory buffer (CMB), the controller configured to: receive a read command from a host device; retrieve data from the memory device, the data associated with the read command; write the retrieved data to a CMB cache of the CMB; inform the host device that the read command is complete; and flush the retrieved data from the CMB cache after the host device reads the retrieved data from the CMB cache. The CMB cache is a static random access memory (SRAM). The controller is further configured to flush the retrieved data from the SRAM to the dynamic random access memory (DRAM) after a predetermined time has elapsed before the host device reads the retrieved data.The CMB includes the CMB cache and dynamic random access memory (DRAM). The controller is configured to maintain a CMB address mapping table. The CMB address mapping table contains entries that indicate whether fetched data associated with read commands is located in static random access memory (SRAM) or dynamic random access memory (DRAM). The controller is configured to delete physical region page (PRP) or scatter gather list (SGL) pointers from the CMB address mapping table after the host device reads the fetched data from the CMB cache. The controller is configured to write additional data associated with the read command to the CMB cache after the fetched data is deleted. The controller is configured to detect when the host device has read the fetched data from the CMB cache.

[0061] In another embodiment, a data storage device comprises: a memory device; and a controller coupled to the memory device, the controller configured to: receive a read command from a host device; read data from the memory device, wherein the read data corresponds to the read command; place the read data into the controller memory buffer (CMB) cache; determine that the host device has read the read data from the CMB cache; find, in the mapping table, the corresponding page physical region (PRP) pointer or scatter-gather list (SGL) pointer; delete the read data from the CMB cache; and delete the PRP pointer or the SGL pointer from the mapping table. The controller includes a CMB manager, and wherein the CMB manager includes the CMB cache and the PRP / SGL buffers.The controller is configured to determine whether the CMB cache is static random access memory (SRAM) or dynamic random access memory (DRAM), and where the controller includes both SRAM and DRAM. The controller is configured to determine whether additional data is to be retrieved for the read command. The controller is configured to update a completion queue before locating it. The controller is configured to detect that the host device is reading data in order from the CMB cache. The controller is configured to detect that the host device is reading data from the CMB cache out of order. The controller is configured to adjust the processing of future read commands to retrieve data from the storage device out of order.

[0062] In another embodiment, a data storage device comprises: means for storing data; and a controller coupled to the means for storing data, the controller configured to: retrieve data from the means for storing data; write the retrieved data to the static random access memory (SRAM); detect that the retrieved data was received from a host device; and delete the retrieved data from the DRAM based on the detection. Data for write commands passes from the host device through the dynamic random access memory (DRAM) and the retrieved data through the SRAM, and wherein the SRAM and the DRAM are distinct from the means for storing the data. The controller is configured to update a controller memory buffer (CMB) mapping table with the location of the retrieved data in the SRAM.

[0063] While the foregoing refers to embodiments of the present disclosure, other and further embodiments of the disclosure may be devised without departing from the basic scope thereof, and the scope thereof is determined by the following claims. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] US 18 / 359,159

[0001] US 63 / 497,784

[0001]

Claims

[1] Data storage device comprising: a storage device; and a controller coupled to the memory device, the controller having a controller memory buffer (CMB), the controller configured to: Receiving a read command from a host device; Retrieving data from the storage device, the data being associated with the read command; Writing the retrieved data to a CMB cache of the CMB; Informing the host device that the read command is completed; and Flushing the retrieved data from the CMB cache after the host device has read the retrieved data from the CMB cache. [2] The data storage device of claim 1, wherein the CMB cache is a static random access memory (SRAM). [3] The data storage device of claim 2, wherein the controller is further configured to flush the retrieved data from the SRAM to the dynamic random access memory (DRAM) after a predetermined time has elapsed before the host device reads the retrieved data. [4] The data storage device of claim 2, wherein the CMB includes the CMB cache and the dynamic random access memory (DRAM). [5] The data storage device of claim 1, wherein the controller is configured to maintain a CMB address mapping table. [6] The data storage device of claim 5, wherein the CMB address mapping table includes entries indicating whether retrieved data associated with read commands is located in the static random access memory (SRAM) or the dynamic random access memory (DRAM). [7] The data storage device of claim 6, wherein the controller is configured to delete physical region page (PRP) pointers or scatter gather list (SGL) pointers from the CMB address mapping table after the host device reads the retrieved data from the CMB cache. [8] The data storage device of claim 1, wherein the controller is configured to write additional data associated with the read command to the CMB cache after deleting the retrieved data. [9] The data storage device of claim 1, wherein the controller is configured to detect when the host device has read the retrieved data from the CMB cache. [10] Data storage device comprising: a storage device; and a controller coupled to the storage device, the controller configured to: Receiving a read command from a host device; Reading data from the storage device, wherein the read data corresponds to the read command; Placing the read data into the controller memory buffer (CMB) cache; Determining that the host device has read the read data from the CMB cache; Find, in the mapping table, the corresponding pointer for the physical region (PRP) page or the scatter-gather list (SGL) pointer; Deleting the read data from the CMB cache; and Deleting the PRP pointer or the SGL pointer from the mapping table. [11] The data storage device of claim 10, wherein the controller includes a CMB manager, and wherein the CMB manager includes the CMB cache and the PRP / SGL buffers. [12] The data storage device of claim 10, wherein the controller is configured to determine whether the CMB cache is a static random access memory (SRAM) or a dynamic random access memory (DRAM), and wherein the controller includes both SRAM and DRAM. [13] The data storage device of claim 10, wherein the controller is configured to determine whether additional data is to be retrieved for the read command. [14] The data storage device of claim 10, wherein the controller is configured to update a completion queue prior to the retrieval. [15] The data storage device of claim 10, wherein the controller is configured to detect that the host device reads the data sequentially from the CMB cache. [16] The data storage device of claim 10, wherein the controller is configured to detect that the host device reads the data from the CMB cache in the wrong order. [17] The data storage device of claim 16, wherein the controller is configured to adapt the processing of future read commands to retrieve data from the storage device in the wrong order. [18] Data storage device comprising: means for storing data; and a controller coupled to the means for storing data, where the controller is configured to: Retrieving data from the data storage means; Writing the retrieved data to the static random access memory (SRAM); Detecting that the retrieved data was received by a host device; and Delete the retrieved data from DRAM based on the detection. [19] The data storage device of claim 18, wherein data for write commands from the host device passes through the dynamic random access memory (DRAM) and the retrieved data passes through the SRAM, and wherein the SRAM and the DRAM are different from the means for storing the data. [20] The data storage device of claim 18, wherein the controller is configured to update a controller memory buffer (CMB) mapping table with the location of the retrieved data in the SRAM.

Images