CMB caching using hybrid SRAM / DRAM datapath

Abstract

A controller memory buffer (CMB) caching mechanism can be used to improve CMB performance. Rather than reading data from static random access memory (SRAM) and writing data, data is read from SRAM. When data is read from the CMB in SRAM, performance is improved, but there is little space to process both read and write commands. Using dynamic random access memory (DRAM) for write commands and using the CMB in SRAM for read commands allows for improved performance. Due to the limited space in SRAM, when a read command is read from the host, the command is deleted. This allows the associated data stored in SRAM to be used for the next command, but is deleted for the next command to be processed. Performance can be improved without using extra SRAM or DRAM.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of U.S. Non-Provisional Patent Application No. 18 / 359,159, entitled "CMB CACHING USING HYBRID SRAM / DRAM DATA PATH," filed with the U.S. Patent and Trademark Office on July 26, 2023, which claims priority to U.S. Provisional Patent Application No. 63 / 497,784, filed on April 24, 2023, the entire contents of which are incorporated herein by reference for all purposes.

[0002] FIELD OF THE INVENTION FIELD Embodiments of the present disclosure generally relate to Controller Memory Buffer (CMB) caching to improve CMB performance. [Background technology]

[0003] Non-volatile memory express (NVMe) is based on a pair of transmit and completion queue mechanisms. Commands are queued in a transmit queue by the host software. Completions are queued in the associated completion queue by the controller. Typically, the transmit queue and completion queue are allocated in host memory, but each queue can be located physically contiguous or non-contiguous in host memory. However, the CMB feature allows the host to place the transmit queue, completion queue, Physical Region Page (PRP) list, Scatter Gather List (SGL) segments, and data buffers in the controller memory.

[0004] A Persistent Memory Region (PMR) is an optional region of general-purpose read / write persistent memory that can be used for a variety of purposes. The address range of a PMR is defined by a peripheral component interconnect (PCI) Base Address Register (BAR) and consumes the entire address space exposed by the BAR. PMRs support the required features of the PCI express (PCIe) programming model (i.e., PMRs do not restrict what is allowed by PCI Express). The contents of a PMR persist across PMR disable, controller, and NVM subsystem resets and power cycles.

[0005] There are several different types of read / write accesses that can occur in the CMB: Sector data is read or written at dedicated address ranges within the CMB address space, NVMe send / completion queues are read or written using dedicated address ranges within the CMB address space, and NVMe PRP lists or SGL segments are read or written using dedicated address ranges within the CMB address space.

[0006] CMB performance varies depending on whether the performance uses static random access memory (SRAM) in the controller or dynamic random access memory (DRAM) attached to the controller to store CMB accesses. Because the normal data path goes through SRAM (for best performance and power), DRAM is designed for metadata storage rather than as part of the data path. This means that the DRAM interface on the controller is small (e.g., 32-bit bus width) and the DRAM is used for random reads and writes of millions of small pieces of metadata (which significantly reduces DRAM efficiency). Adding CMB data traffic to this metadata-optimized DRAM path limits host performance to a much lower level.

[0007] In previous approaches, the CMB can be built into either DRAM or SRAM, but no cache management is provided. However, the main drawbacks are measured in CMB performance and latency. Using caching algorithms is a traditional cache mechanism (e.g., least recently used (LRU)) that is not adapted to the CMB, resulting in a low hit rate. Dedicating a very large amount of SRAM to an SSD in the controller adds significant cost. While using a wider DRAM interface (e.g., 64-bit bus width) provides more DRAM raw bandwidth, efficiency is significantly reduced. Having a wider DRAM bus adds controller cost and increases DRAM cost on smaller drives. Using two separate DRAM interfaces, one for metadata and the other for the data path containing the CMB, also adds significant controller cost.

[0008] Therefore, there is a need in the art for a CMB caching mechanism to improve CMB performance. Summary of the Invention

[0009] A controller memory buffer (CMB) caching mechanism can be used to improve CMB performance. Rather than reading data from static random access memory (SRAM) and writing data, data is read from SRAM. When data is read from the CMB in SRAM, performance is improved, but there is little space to process both read and write commands. Using dynamic random access memory (DRAM) for write commands and using the CMB in SRAM for read commands allows for improved performance. Due to the limited space in SRAM, when a read command is read from the host, the command is deleted. This allows the associated data stored in SRAM to be used for the next command, but is deleted for the next command to be processed. Performance can be improved without using extra SRAM or DRAM.

[0010] In one embodiment, a data storage device includes a memory device and a controller coupled to the memory device, the controller having a controller memory buffer (CMB), the controller configured to receive a read command from a host device, retrieve data associated with the read command from the memory device, write the retrieved data to a CMB cache of the CMB, notify the host device that the read command is completed, and delete the retrieved data from the CMB cache after the host device reads the retrieved data from the CMB cache.

[0011] In another embodiment, a data storage device includes a memory device and a controller coupled to the memory device, wherein the controller is configured to receive a read command from a host device, read data corresponding to the read command from the memory device, place the read data in a controller memory buffer (CMB) cache, determine that the host device has read the read data from the CMB cache, find an associated physical region page (PRP) pointer or scatter gather list (SGL) pointer in a mapping table, remove the read data from the CMB cache, and remove the PRP pointer or SGL pointer from the mapping table.

[0012] In another embodiment, a data storage device comprises means for storing data and a controller coupled to the means for storing data, the controller configured to retrieve data from the means for storing data, write the retrieved data to a static random access memory (SRAM), detect that the retrieved data has been received by a host device, and, based on the detection, delete the retrieved data from the SRAM. [Brief explanation of the drawings]

[0013] So that the above-mentioned features of the present disclosure may be understood in detail, a more particular description of the present disclosure briefly summarized above may be had by reference to embodiments, some of which are illustrated in the accompanying drawings. It should be noted, however, that the accompanying drawings illustrate only typical embodiments of the present disclosure and therefore should not be considered as limiting its scope, as the present disclosure may admit of other equally effective embodiments. [Figure 1] 1 is a schematic block diagram illustrating a storage system in which a data storage device may function as a storage device for a host device, according to certain embodiments. [Figure 2] 1 is a block diagram illustrating a method of operating a storage device to execute a read or write command, according to certain embodiments. [Figure 3A] FIG. 1 is a block diagram illustrating a method for processing data through a data path without a CMB according to an example embodiment. [Figure 3B] FIG. 2 is a block diagram illustrating a method for processing data through a data path using a CMB, according to an example embodiment. [Figure 4A] FIG. 2 is a block diagram illustrating a method for processing data using CMB / PMR in an SRAM according to an example embodiment. [Figure 4B] FIG. 2 is a block diagram illustrating a method for processing data using CMBs in a DRAM according to an example embodiment. [Figure 5] FIG. 1 is a block diagram illustrating a method for CMB using asymmetric performance, according to an example embodiment. [Figure 6] 1 is a schematic block diagram illustrating a storage system in which the CMB resides in DRAM and the CMB cache resides in SRAM, according to a particular embodiment. [Figure 7] 1 is a schematic block diagram illustrating a method for a CMB caching mechanism, according to a particular embodiment. [Figure 8] 1 is a flowchart illustrating a method for a CMB caching mechanism, according to certain embodiments. [Figure 9] FIG. 1 is a schematic block diagram illustrating a storage system in which a caching algorithm is based on PRP / SGL, according to a particular embodiment. [Figure 10] 1 is a flowchart illustrating a method for a read / write process using CMB in an SRAM, according to a particular embodiment.

[0014] For ease of understanding, wherever possible, the same reference numbers have been used to designate identical elements common to the figures. It is contemplated that elements disclosed in one embodiment may be beneficially utilized in other embodiments without specific recitation. DETAILED DESCRIPTION OF THE INVENTION

[0015] Reference will be made below to embodiments of the present disclosure. However, it should be understood that the present disclosure is not limited to the specific described embodiments. Instead, any combination of the following features and elements, whether associated with different embodiments or not, is contemplated for implementing and practicing the present disclosure. Furthermore, embodiments of the present disclosure may achieve other possible solutions and / or advantages over the prior art, but whether or not a particular advantage is achieved by a given embodiment does not limit the present disclosure. Accordingly, the following aspects, features, embodiments, and advantages are merely exemplary and should not be considered elements or limitations of the appended claims unless expressly recited in the claims. Similarly, references to "the present disclosure" should not be construed as a generalization of any inventive subject matter disclosed herein, and should not be considered elements or limitations of the appended claims unless expressly recited in the claims.

[0016] A controller memory buffer (CMB) caching mechanism can be used to improve CMB performance. Rather than reading data from static random access memory (SRAM) and writing data, data is read from SRAM. When data is read from the CMB in SRAM, performance is improved, but there is little space to process both read and write commands. Using dynamic random access memory (DRAM) for write commands and using the CMB in SRAM for read commands allows for improved performance. Due to the limited space in SRAM, when a read command is read from the host, the command is deleted. This allows the associated data stored in SRAM to be used for the next command, but is deleted for the next command to be processed. Performance can be improved without using extra SRAM or DRAM.

[0017] 1 is a schematic block diagram illustrating a storage system 100 having a data storage device 106 that may function as a storage device for a host device 104, according to certain embodiments. For example, the host device 104 may store and retrieve data using non-volatile memory (NVM) 110 included in the data storage device 106. The host device 104 includes host dynamic random access memory (DRAM) 138. In some examples, the storage system 100 may include multiple storage devices, such as the data storage device 106, that may operate as a storage array. For example, the storage system 100 may include multiple data storage devices 106 configured as a redundant array of inexpensive / independent disks (RAID) that collectively function as a mass storage device for the host device 104.

[0018] 1, host device 104 may store data on and / or retrieve data from one or more storage devices, such as data storage device 106. As shown in FIG. 1, host device 104 may communicate with data storage device 106 via interface 114. Host device 104 may include any of a wide range of devices, including a computer server, a network-attached storage (NAS) unit, a desktop computer, a notebook (i.e., laptop) computer, a tablet computer, a set-top box, a telephone handset such as a so-called "smart" phone, a so-called "smart" pad, a television, a camera, a display device, a digital media player, a video game console, a video streaming device, or any other device capable of sending or receiving data from a data storage device.

[0019] Host DRAM 138 may optionally include a host memory buffer (HMB) 150. HMB 150 is a portion of host DRAM 138 allocated to data storage device 106 for exclusive use by controller 108 of data storage device 106. For example, controller 108 may store mapping data, buffered commands, logical to physical (L2P) tables, metadata, etc. in HMB 150. In other words, HMB 150 may be used by controller 108 to store data that would normally be stored in volatile memory 112, buffer 116, internal memory of controller 108 such as static random access memory (SRAM), etc.

[0020] Data storage device 106 includes controller 108, NVM 110, power supply 111, volatile memory 112, interface 114, write buffer 116, and optional DRAM 118. In some examples, data storage device 106 may include additional components not shown in FIG. 1 for clarity. For example, data storage device 106 may include a printed circuit board (PCB) to which the components of data storage device 106 are mechanically attached and which includes conductive traces that electrically interconnect components such as data storage device 106. In some examples, the physical dimensions and connector configuration of data storage device 106 may conform to one or more standard form factors. Some exemplary standard form factors include, but are not limited to, a 3.5-inch data storage device (e.g., HDD or SSD), a 2.5-inch data storage device, a 1.8-inch data storage device, a Peripheral Component Interconnect (PCI), a PCI-extended (PCI-X), a PCI Express (PCIe) (e.g., PCIe x1, x4, x8, x16, PCIe Mini Card, MiniPCI, etc.). In some examples, the data storage device 106 may be directly coupled to the motherboard of the host device 104 (e.g., soldered directly or plugged into a connector).

[0021] The interface 114 may include one or both of a data bus for exchanging data with the host device 104 and a control bus for exchanging commands with the host device 104. The interface 114 may operate according to any suitable protocol. For example, the interface 114 may operate according to one or more of the following protocols: PCIe, Non-Volatile Memory Express (NVMe), OpenCAPI, GenZ, Cache Coherent Interface Accelerator (CCIX), Open Channel SSD (OCSSD), etc. The interface 114 (e.g., the data bus, the control bus, or both) is electrically connected to the controller 108 and provides an electrical connection between the host device 104 and the controller 108, allowing data to be exchanged between the host device 104 and the controller 108. In some examples, the electrical connection of the interface 114 may also allow the data storage device 106 to receive power from the host device 104. For example, as shown in FIG. 1, power supply 111 may receive power from host device 104 via interface 114 .

[0022] The NVM 110 may include multiple memory devices or memory units. The NVM 110 may be configured to store and / or retrieve data. For example, a memory unit of the NVM 110 may receive data and a message from the controller 108 instructing the memory unit to store the data. Similarly, the memory unit may receive a message from the controller 108 instructing the memory unit to retrieve the data. In some examples, each of the memory units may be referred to as a die. In some examples, the NVM 110 may include multiple dies (i.e., multiple memory units). In some examples, each memory unit may be configured to store a relatively large amount of data (e.g., 128 MB, 256 MB, 512 MB, 1 GB, 2 GB, 4 GB, 8 GB, 16 GB, 32 GB, 64 GB, 128 GB, 256 GB, 512 GB, 1 TB, etc.).

[0023] In some examples, each memory unit may include any type of non-volatile memory device, such as a flash memory device, a phase-change memory (PCM) device, a resistive random-access memory (ReRAM) device, a magneto-resistive random-access memory (MRAM) device, a ferroelectric random-access memory (F-RAM), a holographic memory device, and any other type of non-volatile memory device.

[0024] The NVM 110 may include multiple flash memory devices or memory units. The NVM flash memory devices may include NAND or NOR-based flash memory devices and may store data based on the charge contained in the floating gate of the transistor in each flash memory cell. In an NVM flash memory device, the flash memory device may be divided into multiple dies, each of which may include multiple physical or logical blocks, and the multiple physical or logical blocks may be further divided into multiple pages. Each of the multiple blocks within a particular memory device may include multiple NVM cells. Rows of NVM cells may be electrically connected using word lines to define one of multiple pages. Each cell in each of the multiple pages may be electrically connected to a respective bit line. Furthermore, the NVM flash memory device may be a 2D or 3D device and may be a single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), or quad-level cell (QLC) device. The controller 108 may write data to the NVM flash memory device at the page level, read data from the NVM flash memory device, and erase data from the NVM flash memory device at the block level.

[0025] The power supply 111 may provide power to one or more components of the data storage device 106. When operating in a standard mode, the power supply 111 may provide power to one or more components using power provided by an external device, such as the host device 104. For example, the power supply 111 may provide power to one or more components using power received from the host device 104 via the interface 114. In some examples, the power supply 111 may include one or more power storage components configured to provide power to one or more components when operating in a shutdown mode, such as when power is no longer received from an external device. In this manner, the power supply 111 may function as an on-board backup power source. Some examples of the one or more power storage components include, but are not limited to, capacitors, supercapacitors, batteries, etc. In some examples, the amount of power that can be stored by the one or more power storage components may be a function of the cost and / or size (e.g., area / volume) of the one or more power storage components. In other words, as the amount of power stored by the one or more power storage components increases, the cost and / or size of the one or more power storage components also increase.

[0026] The volatile memory 112 may be used by the controller 108 to store information. The volatile memory 112 may include one or more volatile memory devices. In some examples, the controller 108 may use the volatile memory 112 as a cache. For example, the controller 108 may store cached information in the volatile memory 112 until the cached information is written to the NVM 110. As shown in FIG. 1 , the volatile memory 112 may consume power received from the power supply 111. Examples of the volatile memory 112 include, but are not limited to, random-access memory (RAM), dynamic random access memory (DRAM), static RAM (SRAM), and synchronous dynamic RAM (SDRAM (e.g., DDR1, DDR2, DDR3, DDR3L, LPDDR3, DDR4, LPDDR4, etc.)). Similarly, the optional DRAM 118 may be utilized to store mapping data, buffered commands, logical-to-physical (L2P) tables, metadata, cached data, and the like. In some examples, the data storage device 106 is DRAM-less, as it does not include the optional DRAM 118. In other examples, the data storage device 106 includes the optional DRAM 118.

[0027] The controller 108 may manage one or more operations of the data storage device 106. For example, the controller 108 may manage reading data from and / or writing data to the NVM 110. In some embodiments, when the data storage device 106 receives a write command from the host device 104, the controller 108 may initiate the data storage command to store the data in the NVM 110 and monitor the progress of the data storage command. The controller 108 may determine at least one operating characteristic of the storage system 100 and store the at least one operating characteristic in the NVM 110. In some embodiments, when the data storage device 106 receives a write command from the host device 104, the controller 108 temporarily stores data associated with the write command in an internal memory or write buffer 116 before sending the data to the NVM 110.

[0028] The controller 108 may include an optional second volatile memory 120. The optional second volatile memory 120 may be similar to the volatile memory 112. For example, the optional second volatile memory 120 may be an SRAM. The controller 108 may allocate a portion of the optional second volatile memory to the host device 104 as a controller memory buffer (CMB) 122. The CMB 122 may be directly accessed by the host device 104. For example, rather than maintaining one or more transmit queues within the host device 104, the host device 104 may utilize the CMB 122 to store one or more transmit queues that are normally maintained within the host device 104. In other words, the host device 104 may generate commands and store the generated commands, with or without associated data, in the CMB 122, and the controller 108 accesses the CMB 122 to retrieve the stored generated commands and / or associated data.

[0029] 2 is a block diagram illustrating a method 200 for operating a storage device to execute a read or write command, according to one embodiment. Method 200 may be used with storage system 100 of FIG. 1 having host device 104 and data storage device 106, which includes controller 108.

[0030] Method 200 begins at operation 250, where a host device writes a command as an entry into an SQ. The host device can write one or more commands into an SQ in operation 250. The command can be a read command or a write command. The host device can have one or more SQs.

[0031] In operation 252, the host device writes one or more updated SQ tail pointers and rings a doorbell or sends an interrupt signal to notify or signal the storage device of a new command that is ready to be executed. The host can write updated SQ tail pointers and send a doorbell or interrupt signal for each of the SQs if there is more than one SQ. In operation 254, in response to receiving the doorbell or interrupt signal, the controller of the storage device fetches the command from one or more SQs, and the controller receives the command.

[0032] In operation 256, the controller processes the command and writes or transfers data associated with the command to host device memory. The controller can process more than one command at a time. The controller can process one or more commands in the order they were sent or in sequential order.

[0033] In operation 258, once the command has been completely processed, the controller writes a completion entry corresponding to the executed command to the host device's completion queue (CQ) and moves or updates the CQ head pointer to point to the newly written completion entry.

[0034] In operation 260, the controller generates and sends an interrupt signal or doorbell to the host device. The interrupt signal indicates that the command has been executed and that data associated with the command is available in the memory device. The interrupt signal also notifies the host device that the CQ is ready to be read or processed.

[0035] The host device processes the completion entry in operation 262. In operation 264, the host device writes an updated CQ head pointer to the storage device and rings the doorbell or sends an interrupt signal to the storage device to release the completion entry.

[0036] 3A is a block diagram illustrating a method 300 for processing data through a data path without a CMB, according to an example embodiment. The primary application is for drives behind a bridge (e.g., an NVMe-OF bridge, an Analytics Engine), with SSDs connected to the backend of the bridge (i.e., the drives are not directly connected to the host).

[0037] Without a CMB in the SSD drive, all host data, such as that of the host 104 in FIG. 1, must first be staged in the bridge DRAM and then sent to the SSD, resulting in all host data moving in and out of the bridge DRAM. The bridge DRAM has a large bandwidth and is not scalable. When there are multiple SSDs connected to the bridge, all host traffic passes through the bridge DRAM. All NVMe processing is based on memory addresses. When the host wants to write data or read data, it communicates with the bridge. To read data, the host sends an NVMe command over the NVMe Ethernet requesting a piece of data at an address in the SSD. The DRAM then continues the command over the PCIe NVMe interface so that the address in the SSD can be read by the host. To write data, the host sends an NVMe command over the NVMe Ethernet requesting a piece of data in the host DRAM. The bridge then stores the data from the host DRAM. Data continues to be written from the bridge DRAM to the SSD over the PCIe NVMe interface. If a large number of SSDs are placed behind a bridge, the bridge will suffer from bottleneck performance.

[0038] To avoid bottleneck performance issues, some or all of the host data bypasses the bridge DRAM and goes directly to the CMB in the SSD. Figure 3B is a block diagram illustrating a method 350 for processing data through a data path using the CMB, according to an example embodiment. For example, the CMB can be used to store host data (instead of storing the host data in the bridge DRAM). By adding one memory on the SSD, the bottleneck performance issues of using the bridge DRAM for read and write commands are avoided.

[0039] When the host requests to write a piece of data to the host DRAM, the data is read from the NAND. Once the data is read from the NAND, it is placed in the CMB in the SSD. The data is then passed to the bridge DRAM via the PCIe NVME interface. The bridge DRAM does not store the data, but passes it to the host DRAM via the NVMe Ethernet. The bridge DRAM is bypassed because the SSD creates a memory buffer to use as a memory for reads and writes to the host.

[0040] CMB / PMR size is an important factor in where CMB / PMR data can be stored within an SSD. FIG. 4A is a block diagram illustrating a method 400 for processing data using CMB / PMR in SRAM, according to an exemplary embodiment. Because the SRAM bandwidth is designed around the PCIe bandwidth, there is minimal internal contention. Therefore, CMB / PMR at line rate is possible. If the size is small enough, the CMB / PMR can be stored in spare SRAM (e.g., SRAM used for other extended features that are disabled). The small buffer size allows the PMR SRAM buffer to be protected during a performance fail (pFail). Although the SRAM size is small, there is not much spare SRAM.

[0041] If the CMB size is larger than the maximum CMB SRAM size, the CMB is kept in DRAM. FIG. 4B is a block diagram illustrating a method 450 for processing data using CMBs in DRAM, according to an exemplary embodiment. Because DRAM bandwidth is used by both CMB data and FTL metadata (e.g., L2P tables), significant internal contention exists. Even with 100% CMB traffic (no sector data traffic), the DRAM bandwidth is unlikely to allow line-rate performance. With mixed CMB and sector data traffic, either CMB performance or sector data performance is reduced. In particular, during write workloads, DRAM has the highest metadata traffic (random small reads / writes). Therefore, using CMBs in DRAM significantly deviates from the speeds found when putting CMBs in SRAM.

[0042] To avoid losing space with CMB in SRAM and losing speed with CMB in DRAM, asymmetric performance is proposed. FIG. 5 is a block diagram illustrating a method 500 of CMB using asymmetric performance, according to an exemplary embodiment. Asymmetric performance requires CMB host reads to go through SRAM (full bandwidth) while CMB host writes go through DRAM (reduced bandwidth). Asymmetric performance also requires a controller, such as controller 108 of FIG. 1, to maintain a CMB address mapping table from CMB address space to DRAM addresses and SRAM addresses. The CMB address mapping table is similar to a flash logical-to-physical mapping table. Maintaining the CMB address mapping table enables CMB PCIe reads from CMB sector data in SRAM. When the SRAM is read, the data in SRAM is deleted, allowing more subsequent read commands to be stored in the SRAM for processing.

[0043] To enable base support for only CMB sector data / metadata (no CMB queue or PRP / SGL list), the CMB address mapping table must implement hybrid mode. In hybrid mode, PCIe writes to CMB (sector data / metadata) can be initially placed in the SRAM elastic buffer, and while in the SRAM elastic buffer, the CMB address mapping reflects its SRAM location. However, when the PCIe write is moved from the elastic buffer to DRAM, the CMB address mapping must be updated to point to the DRAM location. Alternatively, if an elastic buffer is not required, writes to CMB (sector data / metadata) are written directly to the DRAM location, and the CMB address mapping points to the DRAM location. PCIe reads from CMB (sector data / metadata) always look up the CMB address mapping table to know where to get the requested data. When an NVMe read command has CMB as the destination address, the data read from NAND is placed in SRAM. The CMB address mapping for the destination address is updated to point to the data in SRAM. When the data is read from SRAM, it is discarded, as opposed to reading the data in SRAM and sending it to DRAM. The data is discarded, and the DRAM is used strictly for write commands. Therefore, the resulting CMB read of an NVMe read command yields maximum performance (e.g., line rate). Alternatively, if the host writes sector data to the CMB and then returns and reads that sector data directly again (using the CMB as a scratchpad), the return read of that data will result in degraded DRAM performance if the write data has been moved to DRAM.

[0044] The CMB / PMR may have the following structure: CMB / PMR is a hot topic in the next-generation enterprise market because its features have a direct impact on performance, especially in PCIe fabric topologies. Additionally, CMB / PMR reduces the amount of memory implemented in host DRAM. Management queues or I / O queues may be placed in the CMB, and for a particular queue, all memory associated with the queue shall reside either in the CMB or in host memory.

[0045] The controller may support physical region pages (PRPs) and scatter gather lists (SGLs) within the CMB. For a particular PRP list or SGL associated with a single command, all memory associated with the PRP list or SGL resides in either the CMB or host memory. The PRP and SGL for a command may be placed in the CMB only if the associated command is presented in a send queue within the CMB.

[0046] The controller may support data and metadata in the CMB. All data or metadata associated with a particular command shall be located either in the CMB or in host memory.

[0047] The system described herein incorporates the CMB in DRAM while having a CMB cache in SRAM to improve performance and reduce host latency.

[0048] To avoid degradation of DRAM performance, the CMB is stored in the DRAM and the controller. FIG. 6 is a schematic block diagram illustrating a storage system 600 in which the CMB resides in DRAM and the CMB cache resides in SRAM, according to a specific embodiment. For the system to be practical, there must be a powerful and unique caching algorithm adapted to the CMB. Otherwise, the cache buffer is useless. As described herein, a unique smart caching algorithm is adapted to CMB accesses that focus on data buffers more relevant to the computational segment. Previously, it was possible to incorporate the CMB in DRAM while having a cache buffer in SRAM for user data, but the caching algorithm was a conventional mechanism not adapted to the CMB, resulting in a low hit rate.

[0049] The storage system 600 includes a host, such as the host 104 of FIG. 1, a controller, such as the controller 108 of FIG. 1, a DRAM CMB 604, and a NAND 606. The controller further includes a CMB cache 602. The DRAM CMB 604 is a DRAM with a CMB. As described herein, the CMB caching mechanism is adapted for CMBs that exhibit very high hit rates. Using the proposed approach, it is practical to have a CMB cache that is not implemented today. A high-rate CMB caching mechanism improves overall system performance and quality of service (QoS), making it possible to have a practical solution for CMBs.

[0050] 7 is a schematic block diagram illustrating a method 700 of a CMB caching mechanism, according to a particular embodiment. A basic assumption of the CMB caching mechanism is that for each command, a host, such as host 104 of FIG. 1, fetches data sequentially from the CMB.

[0051] In previous approaches, when completing a command, the associated PRP / SGL pointers are deleted because they are no longer needed. The current approach proposes maintaining the pointers until the host reads the data from the CMB. A controller, such as controller 108 in FIG. 1, uses these pointers to manage a CMB cache, such as CMB cache 602 in FIG. 6, resulting in a very high hit rate. After the host reads the data from the CMB in SRAM, the data can be deleted. As opposed to sending the data to DRAM after reading the data, the associated PRP / SGL pointers are deleted. Deleting the pointers allows subsequent read commands to fall into the CMB cache, thereby resulting in a high hit rate.

[0052] The device controller maintains previously issued PRP / SGL pointers in an internal database. When the host accesses the CMB to retrieve data, the device finds the associated pointer in the internal database. The next pointer associated with the same previously issued host command is then used to populate a new entry into the cache. Populating the new entry is under the assumption that the next CMB access associated with the command is the next pointer. The associated PRP / SGL is evicted from the internal database. The host then issues the next CMB access associated with the command while the data is held in the CMB cache. Again, the next PRP / SGL pointer is used to populate the next entry into the cache.

[0053] FIG. 8 is a flowchart illustrating a method 800 for a CMB caching mechanism, according to a particular embodiment. At 802, a host, such as host 104 of FIG. 1, issues a read command requesting that data be fetched from a NAND, such as NAND 606 of FIG. 6, to a CMB cache, such as CMB cache 602. The device fetches the data from the NAND at 804 and places the data into the CMB at 806, while the first chunk of data is placed into CMB cache 602. A controller, such as controller 108 of FIG. 1, completes the command, but the controller retains the PRP / SGL pointers internally at 808. Next, at 810, the host begins reading the data from CMB cache 602. Next, the controller finds the associated PRP / SGL pointers in an internal database at 812. The next PRP / SGL pointer (if any) is used to place the new data into the CMB. Finally, the data is returned from the CMB cache to the host.

[0054] More specifically, method 800 begins at block 802. In block 802, the host issues a read command from the NAND to the CMB. In block 804, the controller reads the data from the NAND to the CMB. In block 806, the controller caches the first data (e.g., 4 KB). In block 808, the controller completes the command to internally maintain the PRP / SGL buffer. In block 810, the host reads the data from the CMB. In block 812, the controller finds the associated PRP / SGL pointers in an internally maintained database. In block 814, the controller determines whether the data is in the CMB cache. If the controller determines that the data is not in the CMB cache, method 800 proceeds to block 816. If the controller determines that the data is in the CMB cache, method 800 proceeds to block 818. In both blocks 816 and 818, the controller returns the data from the CMB or cache and removes the associated PRP / SGL. Removing associated PRP / SGL is not limited to after the host has read the command. This removal may occur while the command is being read by the host, or may occur after all read commands have been completely read by the host. Removal of data is not limited to read command conflicts. Furthermore, if both blocks 816 and 818 conflict, method 800 continues to block 820. In block 820, the controller determines whether the transfer is the last transfer of the command. If the controller determines that the transfer is not the last transfer of the command, method 800 returns to block 812. If the controller determines that the transfer is the last transfer of the command, method 800 proceeds to block 822, where method 800 ends.

[0055] 9 is a schematic block diagram illustrating a storage system 900 in which the caching algorithm is based on the PRP / SGL, according to a particular embodiment. A controller, such as controller 108 of FIG. 1, incorporates a CMB cache 602, but the caching algorithm is based on a previous PRP / SGL provided by a host, such as host 104 of FIG. 1, for an associated older host command. The PRP / SGL buffer is maintained in a dedicated buffer until the host fetches all data from the CMB, even after completing the command.

[0056] The storage system 900 includes a host, a controller, a DRAM CMB 604, and a NAND 606. The controller includes one or more processors 904, a flash interface module (FIM) 908 for interfacing with a memory device, a host interface module (HIM) 912 for interfacing with a host, a command scheduler 906 coupled between the FIM 908 and the HIM 912, an encryption / decryption module 916 disposed between the FIM 908 and the HIM 912, and a data path, ECC, and RAID 914 disposed between the encryption / decryption module 916 and the FIM 908. The HIM 912 determines when to execute a write command. The FIM 908 retrieves data from the NAND 606. The controller also includes a CMB manager 902 that maintains a PRP / SGL buffer 910 and a CMB cache 602.

[0057] FIG. 10 is a flowchart illustrating a method 1000 for a read / write process using a CMB in SRAM, according to a particular embodiment. Method 1000 begins at block 1002. In block 1002, a controller, such as controller 108 of FIG. 1, receives a read command. In block 1004, the controller retrieves data for the read command from a memory device. In block 1006, the controller writes the retrieved data to an SRAM in a CMB cache, such as CMB cache 602 of FIG. 6. The controller can optionally detect additional read commands from the NAND before block 1006 completes. In block 1008, the controller updates the mapping table of the CMB. In block 1010, the controller posts a completion to a completion queue, such as the completion queue of FIG. 2. In block 1012, the controller detects that the host device has read / retrieved data from the SRAM. In block 1014, the controller deletes the retrieved data from the SRAM and updates the mapping table.

[0058] This disclosure assumes that for each command, the host reads the CMD data in order, which is a valid assumption and most hosts behave this way. In one embodiment, the device controller can detect that a particular host uses a different approach to fetch order. In such a scenario, cache management adapts to the host behavior and populates the cache buffer accordingly.

[0059] SRAM is used as a "cache" buffer that is adapted to NVMe transactions while still having a very high hit rate using current techniques, improving overall performance and reducing latency. Other advantages include the ability for devices to advertise support for very large CMB and PMR sizes, since the CMB / PMR are embedded in DRAM rather than SRAM, which is particularly important when having several or many CMB / PMRs, such as in a virtualized environment. Additionally, supporting large CMB / PMR sizes does not require large application specific integrated circuit (ASIC) areas.

[0060] In one embodiment, the data storage device includes a memory device and a controller coupled to the memory device, the controller having a controller memory buffer (CMB), the controller configured to receive a read command from a host device, retrieve data associated with the read command from the memory device, write the retrieved data to a CMB cache of the CMB, notify the host device that the read command is completed, and delete the retrieved data from the CMB cache after the host device reads the retrieved data from the CMB cache. The CMB cache is static random access memory (SRAM). The controller is further configured to flush the retrieved data from the SRAM to dynamic random access memory (DRAM) after a predetermined period of time has elapsed and before the host device reads the retrieved data. The CMB includes the CMB cache and the dynamic random access memory (DRAM). The controller is configured to maintain a CMB address mapping table. The CMB address mapping table includes entries indicating whether the retrieved data associated with the read command is located in the static random access memory (SRAM) or the dynamic random access memory (DRAM). The controller is configured to delete a physical region page (PRP) pointer or a scatter gather list (SGL) pointer from the CMB address mapping table after the host device reads the retrieved data from the CMB cache. The controller is configured to write additional data associated with the read command to the CMB cache after deleting the retrieved data. The controller is configured to detect when the host device reads the retrieved data from the CMB cache.

[0061] In another embodiment, a data storage device includes a memory device and a controller coupled to the memory device, wherein the controller is configured to receive a read command from a host device, read data corresponding to the read command from the memory device, place the read data in a controller memory buffer (CMB) cache, determine that the host device has read the read data from the CMB cache, find an associated physical region page (PRP) pointer or scatter gather list (SGL) pointer in a mapping table, remove the read data from the CMB cache, and remove the PRP pointer or SGL pointer from the mapping table. The controller includes a CMB manager, which includes the CMB cache and a PRP / SGL buffer. The controller is configured to determine whether the CMB cache is static random access memory (SRAM) or dynamic random access memory (DRAM), and the controller includes both SRAM and DRAM. The controller is configured to determine whether there is additional data to retrieve for the read command. The controller is configured to update a completion queue before the discovery. The controller is configured to detect that the host device reads data in order from the CMB cache. The controller is configured to detect when a host device reads data out of order from the CMB cache, and to adjust future read command processing to retrieve data out of order from the memory device.

[0062] In another embodiment, a data storage device includes a means for storing data and a controller coupled to the means for storing data, the controller configured to retrieve data from the means for storing data, write the retrieved data to a static random access memory (SRAM), detect that the retrieved data has been received by a host device, and delete the retrieved data from the SRAM based on the detection. Data for a write command from the host device passes through a dynamic random access memory (DRAM), and the retrieved data passes through the SRAM, the SRAM and the DRAM being separate from the means for storing data. The controller is configured to update a controller memory buffer (CMB) mapping table with the location of the retrieved data in the SRAM.

[0063] While the foregoing is directed to embodiments of the present disclosure, other and further embodiments of the disclosure may be devised without departing from the basic scope thereof, the scope of which is determined by the claims that follow.

Claims

1. 1. A data storage device comprising: a memory device; a controller coupled to the memory device, the controller having a controller memory buffer (CMB), and the controller Receives a read command from the host device, Retrieving data associated with the read command from the memory device; writing the retrieved data to a CMB cache of the CMB; notifying the host device that the read command has been completed; The data storage device is configured to delete the retrieved data from the CMB cache after the host device reads the retrieved data from the CMB cache.

2. 10. The data storage device of claim 1, wherein the CMB cache is a static random access memory (SRAM).

3. 3. The data storage device of claim 2, wherein the controller is further configured to flush the retrieved data from SRAM to dynamic random access memory (DRAM) after a predetermined period of time has elapsed and before the host device reads the retrieved data.

4. The data storage device of claim 2 , wherein the CMB includes the CMB cache and dynamic random access memory (DRAM).

5. The data storage device of claim 1 , wherein the controller is configured to maintain a CMB address mapping table.

6. 6. The data storage device of claim 5, wherein the CMB address mapping table includes an entry that indicates whether retrieved data associated with a read command is located in static random access memory (SRAM) or dynamic random access memory (DRAM).

7. 7. The data storage device of claim 6, wherein the controller is configured to delete a physical region page (PRP) pointer or a scatter gather list (SGL) pointer from the CMB address mapping table after the host device reads the retrieved data from the CMB cache.

8. 2. The data storage device of claim 1, wherein the controller is configured to write additional data associated with the read command to the CMB cache after deleting the retrieved data.

9. 2. The data storage device of claim 1, wherein the controller is configured to detect when the host device reads the retrieved data from a CMB cache.

10. 1. A data storage device comprising: a memory device; a controller coupled to the memory device, the controller comprising: receiving a read command from a host device; reading data from the memory device, the data read corresponding to the read command; placing the read data in a controller memory buffer (CMB) cache; determining that the host device read the read data from a CMB cache; Finding the associated physical region page (PRP) pointer or scatter gather list (SGL) pointer in a mapping table; Removing the read data from the CMB cache; and deleting the PRP pointer or the SGL pointer from the mapping table.

11. 11. The data storage device of claim 10, wherein the controller includes a CMB manager, the CMB manager including the CMB cache and PRP / SGL buffers.

12. 11. The data storage device of claim 10, wherein the controller is configured to determine whether the CMB cache is static random access memory (SRAM) or dynamic random access memory (DRAM), and the controller includes both SRAM and DRAM.

13. The data storage device of claim 10 , wherein the controller is configured to determine whether there is additional data to retrieve for the read command.

14. The data storage device of claim 10 , wherein the controller is configured to update a completion queue before the discovery.

15. 11. The data storage device of claim 10, wherein the controller is configured to detect that the host device reads the data in order from a CMB cache.

16. 11. The data storage device of claim 10, wherein the controller is configured to detect that the host device reads the data out of order from a CMB cache.

17. 17. The data storage device of claim 16, wherein the controller is configured to regulate future read command processing to retrieve data from the memory device out of order.

18. 1. A data storage device comprising: means for storing data; a controller coupled to the means for storing the data, the controller comprising: Retrieving data from the means for storing data; writing the retrieved data into a static random access memory (SRAM); Detecting that the retrieved data has been received by a host device; The data storage device is configured to delete the retrieved data from the SRAM based on the detection.

19. 20. The data storage device of claim 18, wherein data for a write command from the host device passes through a dynamic random access memory (DRAM), and the retrieved data passes through the SRAM, and the SRAM and the DRAM are separate from the means for storing the data.

20. 20. The data storage device of claim 18, wherein the controller is configured to update a controller memory buffer (CMB) mapping table with the location of the retrieved data in SRAM.

Images