HMB write coalescing for optimized write performance

Abstract

The present disclosure generally relates to improved processing of write commands. A host memory buffer (HMB) or other storage space can be utilized to delay the execution of host write commands, which improves write performance in different use cases and enables having more concurrent streams than open blocks without affecting write or read performance. Generally, when a write command is received, the write command is modified as a new write command that is logically equivalent to the original write command. The modified write command is moved to the HMB along with the data. By doing so, the write commands are coalesced and the processing of the write commands is improved.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Non-Provisional Patent Application No. 17 / 885,265, filed August 10, 2022, entitled "Write Coalescing Via HMB To Optimize Write Performance," the entire contents of which are incorporated herein by reference for all purposes. [Background technology]

[0002] FIELD Embodiments of the present disclosure generally relate to improved write command processing.

[0003] FIELD OF THE INVENTION Write performance is important for both client and server storage applications and is one of the most important metrics for clients. Write performance can be hindered in different use cases limited by system resources. For example, a data storage device typically maintains an open block for each stream ID currently in use by the host, and data written from a stream ID is written to the corresponding open block.

[0004] If the number of open blocks in a data storage device is smaller than the number of different streams, excessive writing and reading may occur, resulting in a problem of reduced write performance. When a host writes data to a data storage device, it may be useful to temporarily store the data in an intermediate space to perform some operations before actually writing the data to the storage medium. This intermediate storage space may be a dedicated buffer in the storage space of the data storage device, or alternatively, to save space, it may be a host memory buffer (HMB) located in the host. The HMB is a host storage space dedicated to the needs of the storage controller. Typically, the HMB stores management tables or other management data according to the configuration of the controller.

[0005] In the existing architecture, once the corresponding write command is interpreted, the written data is passed directly from the host's data buffer to the data storage device's controller by a hardware (HW) module. Information about the command interpretation is passed to a completion queue. The write data itself does not necessarily need to be written to the memory device immediately, but may be stored in a cache. The cache may be in the data storage device or on the host side in the HMB.

[0006] If the number of concurrent streams is greater than the number of open blocks, data from the streams may be written to the same open blocks, later impairing read throughput. Data from these extra streams can also be rearranged later to induce write amplification. Alternatively, the host may be limited to a number of streams determined by the number of open blocks.

[0007] Therefore, there is a need in the art for improved processing of write commands. Summary of the Invention

[0008] The present disclosure generally relates to improved processing of write commands. A host memory buffer (HMB) or other storage space can be utilized to delay the execution of host write commands, which improves write performance in different use cases and allows for more concurrent streams than open blocks without impacting write or read performance. Generally, when a write command is received, the write command is modified as a new write command that is logically equivalent to the original write command. The modified write command, along with the data, is moved to the HMB. In this way, the write commands are coalesced and the processing of the write commands is improved.

[0009] In one embodiment, a data storage device includes a first memory device and a controller coupled to the memory device, where the controller is configured to receive a write command, determine that the write command can be write-coalesced, allocate one or more buffers in a second memory device separate from the first memory device, generate a modified write command from the write command, and move data associated with the write command to at least one of the allocated one or more buffers.

[0010] In another embodiment, a data storage device includes a first memory device and a controller coupled to the memory device, wherein the controller is configured to receive a write command, modify the write command to create a modified write command, store the modified write command in a second memory device different from the first memory device, determine that a predetermined period of time has elapsed or a trigger mechanism has occurred, and execute the modified write command.

[0011] In another embodiment, a data storage device comprises a memory means and a controller coupled to the memory means, the controller configured to translate a write command from a Physical Region Page (PRP) entry to a Scatter-Gather List (SGL) entry, store data associated with the PRP entry and the translated write command in a host device, notify the host device that the write command is completed, and after the notification, execute the translated write command, wherein the execution includes writing the stored data to the memory means. [Brief explanation of the drawings]

[0012] So that the above-mentioned features of the present disclosure can be understood in detail, a more particular description of the present disclosure briefly summarized above can be had by reference to embodiments, some of which are illustrated in the accompanying drawings. It should be noted, however, that the accompanying drawings illustrate only typical embodiments of the present disclosure and therefore should not be considered as limiting its scope, as the present disclosure may admit of other equally effective embodiments. [Figure 1] 1 is a schematic block diagram illustrating a storage system in which a data storage device can function as a storage device for a host device, according to certain embodiments. [Figure 2] 1 is a block diagram illustrating a method of operating a storage device to execute a read or write command, according to certain embodiments. [Figure 3] 10 is a flowchart illustrating the execution of a write command. [Figure 4] FIG. 1 is a schematic diagram of an architecture incorporating write command coalescing, according to one embodiment. [Figure 5] 10 is a flowchart illustrating write command coalescing according to one embodiment. [Figure 6] 10 is a flowchart illustrating delayed command flow execution according to one embodiment. [Figure 7] FIG. 1 is a schematic diagram of a system incorporating write command coalescing, according to one embodiment. [Figure 8] 10 is a flowchart illustrating the coalescing and execution of a write command according to one embodiment.

[0013] To facilitate understanding, the same reference numbers have been used, whenever possible, to designate identical elements common to the figures. It is contemplated that elements disclosed in one embodiment may be beneficially utilized on other embodiments without specific mention thereof. DETAILED DESCRIPTION OF THE INVENTION

[0014] Reference will be made below to embodiments of the present disclosure. However, it should be understood that the present disclosure is not limited to the specifically described embodiments. Instead, any combination of the following features and elements, whether associated with different embodiments or not, is contemplated to implement and practice the present disclosure. Furthermore, although embodiments of the present disclosure may achieve advantages over other possible solutions and / or over the prior art, whether or not a particular advantage is achieved by a given embodiment does not limit the present disclosure. Accordingly, the following aspects, features, embodiments, and advantages are merely exemplary and are not considered elements or limitations of the appended claims unless expressly recited in the claim(s). Similarly, references to "the present disclosure" should not be construed as a generalization of any inventive subject matter disclosed herein, and should not be considered elements or limitations of the appended claims unless expressly recited in the claims.

[0015] The present disclosure generally relates to improved processing of write commands. A host memory buffer (HMB) or other storage space can be utilized to delay the execution of host write commands, which improves write performance in different use cases and allows for more concurrent streams than open blocks without impacting write or read performance. Generally, when a write command is received, the write command is modified as a new write command that is logically equivalent to the original write command. The modified write command, along with the data, is moved to the HMB. In this way, the write commands are coalesced and the processing of the write commands is improved.

[0016] 1 is a schematic block diagram illustrating a storage system 100 in which a host device 104 communicates with a data storage device 106, according to certain embodiments. For example, the host device 104 may store and retrieve data using non-volatile memory (NVM) 110 included in the data storage device 106. The host device 104 includes a host DRAM 138 and, optionally, a host memory buffer 150. In some implementations, the storage system 100 may include multiple storage devices, such as the data storage device 106, that may operate as a storage array. For example, the storage system 100 may include multiple data storage devices 106 configured as a redundant array of inexpensive / independent disks (RAID) that collectively function as a mass storage device for the host device 104.

[0017] Host device 104 may store and / or retrieve data to and / or from one or more storage devices, such as data storage device 106. As illustrated in Figure 1, host device 104 may communicate with data storage device 106 via interface 114. Host device 104 may comprise any of a wide range of devices, including a computer server, a network-attached storage (NAS) unit, a desktop computer, a notebook (i.e., laptop) computer, a tablet computer, a set-top box, a telephone such as a so-called "smart" phone, a so-called "smart" pad, a television, a camera, a display device, a digital media player, a video game console, a video streaming device, or any other device capable of transmitting or receiving data from a data storage device.

[0018] Data storage device 106 includes controller 108, NVM 110, power supply 111, volatile memory 112, interface 114, and write buffer 116. In some embodiments, data storage device 106 may include additional components not shown in FIG. 1 for clarity. Controller 108 may include volatile memory, such as DRAM 152, and a Controller Memory Buffer (CMB) dedicated for use by host device 104. For example, data storage device 106 may include a printed circuit board (PCB) to which components, such as data storage device 106, are mechanically attached and which includes conductive traces that electrically interconnect the components of data storage device 106. In some embodiments, the physical dimensions and connector configuration of data storage device 106 may conform to one or more standard form factors. Some exemplary standard form factors include, but are not limited to, a 3.5" data storage device (e.g., HDD or SSD), a 2.5" data storage device, a 1.8" data storage device, a Peripheral Component Interconnect (PCI), a PCI Expansion (PCI-X), a PCI Express (PCIe) (e.g., PCIe x1, x4, x8, x16, PCIe Mini Card, Mini PCI, etc.). In some embodiments, the data storage device 106 may be directly coupled to the motherboard of the host device 104 (e.g., soldered or plugged directly into a connector).

[0019] The interface 114 may include one or both of a data bus for exchanging data with the host device 104 and a control bus for exchanging commands with the host device 104. The interface 114 may operate according to any suitable protocol. For example, the interface 114 may operate according to one or more of the following protocols: Advanced Technology Attachment (ATA) (e.g., Serial ATA (SATA) and Parallel ATA (PATA)), Fibre Channel Protocol (FCP), Serial Attached SCSI (SAS), PCI and PCIe, Small Computer System Interface (SCSI), Non-Volatile Memory Express (NVMe), OpenCAPI, GenZ, Cache Coherent Interface Accelerator (CCIX), Open Channel SSD (OCSSD), etc. The interface 114 (e.g., the data bus, the control bus, or both) is electrically connected to the controller 108 and provides an electrical connection between the host device 104 and the controller 108, enabling data to be exchanged between the host device 104 and the controller 108. In some embodiments, the electrical connection of interface 114 may also allow data storage device 106 to receive power from host device 104. For example, as illustrated in FIG. 1, power supply 111 may receive power from host device 104 via interface 114.

[0020] NVM 110 may include multiple memory devices or memory units. NVM 110 may be configured to store and / or retrieve data. For example, a memory unit of NVM 110 may receive data and a message from controller 108 instructing the memory unit to store data. Similarly, a memory unit may receive a message from controller 108 instructing the memory unit to retrieve data. In some embodiments, each of the memory units may be referred to as a die. In some embodiments, NVM 110 may include multiple dies (i.e., multiple memory units). In some embodiments, each memory unit may be configured to store a relatively large amount of data (e.g., 128 MB, 256 MB, 512 MB, 1 GB, 2 GB, 4 GB, 8 GB, 16 GB, 32 GB, 64 GB, 128 GB, 256 GB, 512 GB, 1 TB, etc.).

[0021] In some embodiments, each memory unit may include any type of non-volatile memory device, such as a flash memory device, a phase change memory (PCM) device, a resistive random access memory (ReRAM) device, a magnetoresistive random access memory (MRAM) device, a ferroelectric random access memory (F-RAM), a holographic memory device, and any other type of non-volatile memory device.

[0022] The NVM 110 may include multiple flash memory devices or memory units. The NVM flash memory devices may include NAND- or NOR-based flash memory devices and may store data based on the charge contained in the floating gate of the transistor of each flash memory cell. In an NVM flash memory device, the flash memory device may be divided into multiple dies, each of which may include multiple physical or logical blocks, and the multiple physical or logical blocks may be further divided into multiple pages. Each of the multiple blocks within a particular memory device may include multiple NVM cells. Rows of NVM cells may be electrically connected using word lines to define each of multiple pages. Each cell in each of the multiple pages may be electrically connected to a respective bit line. Furthermore, the NVM flash memory device may be a 2D or 3D device and may be a single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), or quad-level cell (QLC) device. The controller 108 may write data to and read data from the NVM flash memory device at the page level and erase data from the NVM flash memory device at the block level.

[0023] The power supply 111 may provide power to one or more components of the data storage device 106. When operating in a standard mode, the power supply 111 may provide power to one or more components using power provided by an external device, such as the host device 104. For example, the power supply 111 may provide power to one or more components using power received from the host device 104 via the interface 114. In some embodiments, the power supply 111 may include one or more power storage components configured to provide power to one or more components when operating in a shutdown mode, such as when the power supply 111 stops receiving power from an external device. In this manner, the power supply 111 may function as an on-board power source. Some examples of the one or more power storage components include, but are not limited to, capacitors, supercapacitors, batteries, etc. In some embodiments, the amount of power that can be stored by the one or more power storage components may be a function of the cost and / or size (e.g., area / volume) of the one or more power storage components. In other words, as the amount of power stored by the one or more power storage components increases, the cost and / or size of the one or more power storage components also increase.

[0024] Volatile memory 112 may be used by controller 108 to store information. Volatile memory 112 may include one or more volatile memory devices. In some embodiments, controller 108 may use volatile memory 112 as a cache. For example, controller 108 may store cached information in volatile memory 112 until the cached information is written to NVM 110. As illustrated in FIG. 1 , volatile memory 112 may consume power received from power supply 111. Examples of volatile memory 112 include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static RAM (SRAM), and synchronous dynamic RAM (SDRAM (e.g., DDR1, DDR2, DDR3, DDR3L, LPDDR3, DDR4, LPDDR4, etc.)).

[0025] The controller 108 may manage one or more operations of the data storage device 106. For example, the controller 108 may manage reading data from and / or writing data to the NVM 110. In some embodiments, when the data storage device 106 receives a write command from the host device 104, the controller 108 may initiate the data storage command, store the data in the NVM 110, and monitor the progress of the data storage command. The controller 108 may determine at least one operating characteristic of the storage system 100 and store the at least one operating characteristic in the NVM 110. In some embodiments, when the data storage device 106 receives a write command from the host device 104, the controller 108 temporarily stores data associated with the write command in its internal memory (i.e., a second volatile memory) before sending the data to the NVM 110. This memory may be an SRAM memory.

[0026] 2 is a block diagram illustrating a method 200 for operating a storage device to execute a read or write command, according to one embodiment. Method 200 may be used with storage system 100 of FIG. 1 having host device 104 and data storage device 106, which includes controller 108.

[0027] Method 200 begins at operation 250, where a host device writes a command as an entry into a Submission Queue (SQ). The host device may write one or more commands to the SQ at operation 250. The commands may be read commands or write commands. The host device may include one or more SQs.

[0028] In operation 252, the host device writes one or more updated SQ tail pointers and rings a doorbell or sends an interrupt signal to notify or signal the storage device of a new command that is ready to be executed. The host may write the updated SQ tail pointers and send a doorbell or interrupt signal for each of the SQs if there is more than one SQ. In operation 254, in response to receiving the doorbell or interrupt signal, the controller of the storage device fetches the command from one or more SQs, and the controller receives the command.

[0029] In operation 256, the controller processes the command and writes or transfers data associated with the command to host device memory. The controller may process more than one command at a time. The controller may process one or more commands in the order they are sent or in sequential order. Processing a write command may include identifying a zone to write the data associated with the command, writing the data to one or more Logical Block Addresses (LBAs) in the zone, and advancing a write pointer for the zone to identify the next available LBA in the zone.

[0030] In operation 258, once the command has been completely processed, the controller writes a completion entry corresponding to the executed command to the host device's Completion Queue (CQ) and moves or updates the CQ head pointer to point to the newly written completion entry.

[0031] In operation 260, the controller generates and sends an interrupt signal or doorbell to the host device. The interrupt signal indicates that the command has been executed and that data associated with the command is available in the memory device. The interrupt signal also notifies the host device that the CQ is ready to be read or processed.

[0032] The host device processes the completion entry in operation 262. In operation 264, the host device writes an updated CQ head pointer to the storage device and rings the doorbell or sends an interrupt signal to the storage device to release the completion entry.

[0033] 3 is a flowchart 300 illustrating the execution of a write command. First, a write command is received. Then, at 302, the write command is processed. At 304, data corresponding to the write command is fetched from the host device. The data is typically in the host device data buffer. The data is then processed and stored in a cache. The cache may be the HMB, CMB, controller DRAM, or NVM of the data storage device. Once the data is in the cache, a completion queue is updated at 306.

[0034] As described herein, the write sequence is modified so that data can be stored in the HMB (or any similar host-based memory space) or CMB to delay execution of host write commands and improve the overall write performance of the system. Thus, more simultaneous streams than open blocks are possible without adversely affecting write performance.

[0035] The method is based on analyzing the write workflow and determining that temporarily delaying the actual writes to the memory device will benefit overall write performance. The determination may be based on host-specified or device-specified write alignment / granularity considerations. For example, writes that do not meet stream write granularity or namespace write granularity guidance may be coalesced. In another example, if emergency foreground maintenance is required to free up open blocks, writes may be temporarily coalesced, allowing the host to free up write buffers while the device cannot simultaneously write new data to the memory device (i.e., NAND).

[0036] The write coalescing function involves a hardware (HW) modification such that the host device pointer is modified and written to point to HMB space. The HW automatically Direct Memory Access (DMA) the original payload to the new HMB location before auto-completing the command. Following the DMA, the firmware (FW) or HW can release the write, and the write is then resubmitted as if it were a new write command, without additional completion.

[0037] FIG. 4 is a schematic diagram of an architecture incorporating write command coalescing according to one embodiment, illustrating the concept using an example. An original NVMe write command 406 in host memory 402 is queued in a send queue 404 by the host. The command 406 holds a pointer to a data buffer. PRP1 holds a pointer to the first buffer. PRP2 holds pointers to a PRP list 408, each of which is a pointer to a data buffer 410. The device controller first fetches the original write command, classifies the original write command, and determines whether to execute the write command now or postpone it. This decision is based on the classification result (e.g., stream ID) and the current state of open blocks in the device. If the write command is postponed, the contents of the write command are copied to the HMB 412 with some adaptation. In FIG. 4, a modified command 416 uses a scattered-aggregated list (SGL) method and does not use a physical region page (PRP). The modified command 416, including the SGL segment, is stored in an internal send queue 404 implemented in the HMB 412. The device prepares an SGL list 418 in the HMB 412, with each entry pointing to a data buffer 420 in the HMB. The data from the original data buffer 410 is copied to the allocated data buffer 420 in the HMB 412. A completion entry is posted to the host device. Later, the modified command 416 is executed by the data storage device while fetching all relevant information from the HMB 412. The completion entry is not posted to the host device because it was already posted when copying the original data from the data buffer 410 to the data buffer 420.

[0038] After command interpretation is performed, a new "write coalescing module" can analyze the current write, taking into account the state of the last command, the HMB contents, and memory-related characteristics. The module can then determine whether to delay the command using the HMB, write the data directly to the storage device (normal write), or execute a previously delayed command from the HMB and write the delayed command data to the data storage device along with the current write data.

[0039] When a host device issues a flush command instructing the data storage device to write all unwritten data to the memory device, the data storage device executes all previously delayed commands and writes the data to the memory device. In one embodiment, a data storage device typically has a limited number of reserved open blocks. An open block is a block that has not been completely written and is subject to different protection mechanisms due to reduced resistance to different factors. Once a block is completely written, it is closed and another block is opened in its place. In a host device that supports streams, an open block may be associated with a stream so that consecutively written data is written to the same physical block (metablock). However, the number of streams supported by a host device may be greater than the number of available open blocks in the storage device. As described herein, it is proposed to use write coalescing to stack data associated with a particular stream (not supported by open blocks in the storage controller) within the HMB. HMB-based data may be written to the memory device when a complete block is coalesced or at another time determined by the controller, allowing for effective support of any number of simultaneous streams without the need to dedicate space in the controller to the task.

[0040] 5 is a flowchart 500 illustrating write command coalescing according to one embodiment illustrating a write process. First, a write command is received, and classification of the write command (e.g., stream ID) and the current state of the system (e.g., open block count) is performed in the write coalescing module 502, and a determination is made at 504 whether coalescing should be performed on the command. In a positive result, HMB space is allocated at 506. At 508, a modified write command is created in the HMB along with a list of pointers to the HMB. At 510, the write command data content is copied to the allocated location, and at 512, a completion entry is notified to the host device. If write coalescing should not be applied to the write command at 504, the corresponding data is fetched from the data buffer, processed, and stored in the cache at 514, after which the completion queue is updated at 512.

[0041] 6 is a flowchart 600 illustrating delayed command flow execution, according to one embodiment, illustrating a decision to write the contents of a delayed write command from the HMB to a data storage device. The decision to execute a delayed command may be triggered by either a "flush" command sent by the host device or the number of open blocks changed by the data storage device (some blocks closed). Once the decision to write the delayed write command from the HMB is made, the data is written to the storage device and no additional overhead associated with a completion entry notification to the host device is required.

[0042] In one embodiment, the device controller implements reserved SQs that are invisible to the host device. The data storage device queues modified write commands in the reserved SQs. If there is a trigger to execute some of the modified commands, the data storage device simply rings the doorbell of the associated reserved SQ. The modified write commands in the reserved SQs are then executed normally, except that there is no notification to the CQ. In other words, while a reserved SQ exists, there is no corresponding reserved CQ, and therefore no place to notify completion. Completion does not need to be notified to the regular CQ, as it should have been notified previously. The number of reserved SQs may be a function of the streams and open blocks supported.

[0043] When cached operations are invalidated by the host device either globally or command-specific, the same flow works, the only difference being that the completion is sent only after completing the execution of the modified command and only after the data has been stored in the memory device (e.g., NAND).

[0044] In flowchart 600, once a trigger event occurs or a predetermined time has passed, a determination is made at 602 as to whether it is time to execute the delayed command. The trigger event may be, for example, a flush command. If it is not time to execute the delayed write command, the process continues to wait. If it is time to execute the delayed write command, at 604, the HW is activated to execute the delayed write command, and at 606, the delayed write command is executed. The delayed write command is executed using the HMB as a data buffer, rather than the host data buffer as with the original write command. When the delayed write command is completed, at 608, the completion queue is not updated because it was previously updated when the delayed write command was created and the data was stored in the HMB buffer.

[0045] Another embodiment may use a CMB-based write delay for coalescing small write commands. The CMB is a memory buffer located within the data storage device controller that may be used for different host device applications. In this context, the CMB may be used as a memory space (albeit typically a much smaller memory space) for delaying shorter write commands.

[0046] 7 is a schematic diagram of a system 700 incorporating write command coalescing, showing a high-level block diagram of a system for write command coalescing, according to one embodiment. System 700 includes a host device 702 having a DRAM 704 and an HMB 706. System 700 also includes a memory device 710 and a data storage device controller 708.

[0047] The controller 708 includes one or more processors 724, a Flash Interface Module (FIM) 726 for interfacing with the memory device 710, a Host Interface Module (HIM) 712 for interfacing with the host device 702, a command scheduler 722 coupled between the FIM 726 and the HIM 712, an encryption / decryption module 718 disposed between the FIM 726 and the HIM 712, and a data path and ECC and RAID disposed between the encryption / decryption module 718 and the FIM 726. The HIM 712 includes an HMB write command coalescing module 714, which determines when to execute a write command while activating a DMA copy engine 716 to copy the command and all data structures, with any modifications, from the host device 702 memory to the HMB 706.

[0048] In one embodiment, the modified command held in the HMB may be a write scatter command that includes several LBA ranges, rather than just one. In a write scatter command embodiment, several host device write commands may be compressed into a single modified write scatter command stored in the HMB. In another embodiment, the grouping of delayed write commands may be based on a context attribute identifier or on a region / zone within a DPM / ZNS device.

[0049] 8 is a flowchart 800 illustrating the coalescing and execution of a write command according to one embodiment. The process begins at 802 when a write command is received; more specifically, the write command is placed in a transmit queue in the host and a doorbell is rung to notify the data storage device that there is a command to fetch in the transmit queue. At 804, a determination is made as to whether the write command can be coalesced. If the write command cannot be coalesced at 804, the write command is executed normally and the completion queue is updated to indicate that the write command has been executed.

[0050] If the write command can be coalesced at 804, then at 808 the write command is modified and stored with the data in a new location. In one embodiment, the new location is the HMB. In another embodiment, the new location is the CMB. In yet another embodiment, the new location is in host controller volatile memory. In either embodiment, the new location is not the NVM where the data of the original write command will ultimately be written. After modifying the write command and storing the associated data in the new location, the Completion Queue is updated at 810 by posting completion to the Completion Queue and ringing a doorbell to notify the host that the write command has completed, even though the write command has not actually completed. At this point, the write command has been modified and the data has been moved to the new location. The modified write command is executed at a later point in time when the original write command will technically eventually complete.

[0051] The modified write command can be executed after a predetermined period of time has elapsed or after a trigger event occurs. If the time threshold is exceeded at 812, the modified command is executed at 814 and completion is not notified to the completion queue at 816. If the time threshold is not exceeded at 812, a determination is made as to whether a trigger event has occurred at 818. If the trigger event occurs at 818, the modified command is executed at 814 and completion is not notified to the host at 816. If the trigger event does not occur at 818, the modified write command is still not executed but simply waits to be executed. Note that the checks for the trigger event and whether the time threshold has been exceeded may occur in any order or simultaneously. An example of a trigger event is the receipt of a flush command from the host. In any case, when the modified write command is executed, the host is not notified because the host believes the write command has already completed and is unaware of the existence of the modified write command.

[0052] In either case, the host is notified that the original write command has completed before the modified write command has completed. In other words, the host device perceives the original write command as completed even though it has not actually completed and the data to be written to NVM has not yet been written to NVM. Completing the modified write command is technically completing the original write command by writing the data to NVM, but this occurs later than the host perceives, and therefore the write command is processed more effectively and efficiently. The location for storing the modified write command and associated data waiting to be executed may be in a memory location such as the HMB, CMB, or other memory location that is not the final destination memory location (i.e., NVM or NAND) indicated by the original write command.

[0053] Utilizing write command coalescing in the HMB (or CMB or other storage area) results in improved write performance for different scenarios, such as having more simultaneous streams than open blocks, and therefore write commands are processed more effectively and efficiently.

[0054] In one embodiment, a data storage device includes a first memory device and a controller coupled to the memory device, wherein the controller is configured to receive a write command, determine that the write command can be write-coalesced, allocate one or more buffers in a second memory device separate from the first memory device, generate a modified write command from the write command, and move data associated with the write command to at least one of the allocated one or more buffers. The controller is further configured to generate a list of pointers to the at least one buffer. The controller is configured to update a completion queue for the write command. The update occurs before completing the write command. The update occurs after executing the modified write command. The second memory device is a host memory buffer (HMB). The second memory device is a controller memory buffer (CMB). The write command is a non-volatile memory express (NVMe) write command that uses one or more physical region page (PRP) entries. The modified write command uses a scatter-aggregate list (SGL) element. The controller is further configured to execute the modified write command in response to either a trigger mechanism or a predetermined period of time having elapsed.

[0055] In another embodiment, a data storage device includes a first memory device and a controller coupled to the memory device, wherein the controller is configured to receive a write command, modify the write command to create a modified write command, store the modified write command in a second memory device different from the first memory device, determine that a predetermined period of time has elapsed or a trigger mechanism has occurred, and execute the modified write command. The controller is further configured to activate hardware (HW) to execute the modified write command. Executing the modified write command includes retrieving data associated with the modified write command from the second memory device. Storing the modified write command in the second memory device includes storing the modified write command and data associated with the write command in the second memory device. Executing the modified write command includes retrieving data associated with the write command from the second memory device and writing the data to the first memory device. A completion queue in the host device is updated before executing the modified write command. The completion queue is not updated after executing the modified write command.

[0056] In another embodiment, a data storage device comprises a memory means and a controller coupled to the memory means, the controller configured to translate a write command from a physical region page (PRP) entry to a scatter-aggregate list (SGL) entry, store data associated with the PRP entry and the translated write command in a host device, notify the host device that the write command is completed, and after the notification, execute the translated write command, the execution including writing the stored data to the memory means. The controller is configured not to notify the host device that the translated write command is executed. The controller is configured to execute the translated write command after receiving a flash command from the host device.

[0057] While the foregoing is directed to embodiments of the present disclosure, other and further embodiments of the present disclosure may be devised without departing from the basic scope thereof, which scope is determined by the following claims.

Claims

1. a first memory device; a controller coupled to the first memory device; wherein the controller: receiving a write command using one or more physical region page (PRP) entries; determining that the write commands can be write coalesced; allocating one or more buffers in a second memory device separate from the first memory device; generating a modified write command from the write command that uses a scatter-aggregate list (SGL) entry; moving data associated with the write command to at least one of the allocated one or more buffers; A data storage device configured to:

2. The data storage device of claim 1 , wherein the controller is further configured to generate a list of pointers to the at least one buffer.

3. The data storage device of claim 1 , wherein the controller is configured to update a completion queue for the write command.

4. The data storage device of claim 3 , wherein the updating occurs before completing the write command.

5. The data storage device of claim 1 , wherein the second memory device is a host memory buffer (HMB).

6. The data storage device of claim 1 , wherein the second memory device is a controller memory buffer (CMB).

7. 10. The data storage device of claim 1, wherein the write command is a Non-Volatile Memory Express (NVMe) write command.

8. 10. The data storage device of claim 1, wherein the controller is further configured to execute the modified write command in response to either a trigger mechanism or a predetermined period of time having passed.

9. a first memory device; a controller coupled to the first memory device; wherein the controller: receiving a write command using one or more physical region page (PRP) entries; modifying the write command to create a modified write command that uses a scatter-aggregate list (SGL) entry; storing the modified write command in a second memory device different from the first memory device; determining that a predetermined period of time has elapsed or that a trigger mechanism has occurred; executing the modified write command; A data storage device configured to:

10. The data storage device of claim 9 , wherein the controller is further configured to activate hardware (HW) to execute the modified write command.

11. 11. The data storage device of claim 10, wherein executing the modified write command includes retrieving data associated with the modified write command from the second memory device.

12. 12. The data storage device of claim 11, wherein storing the modified write command in the second memory device comprises storing the modified write command and data associated with the write command in the second memory device.

13. 10. The data storage device of claim 9, wherein executing the modified write command includes retrieving data associated with the write command from the second memory device and writing the data to the first memory device.

14. 14. The data storage device of claim 13, wherein a completion queue in a host device is updated prior to executing the modified write command.

15. 15. The data storage device of claim 14, wherein the completion queue is not updated after executing the modified write command.

16. The write command is an original write command, and the modified write command is logically equivalent to the original write command; 10. The data storage device of claim 1 or 9, wherein the controller is configured to perform write coalescing to stack data associated with a particular stream in a host memory buffer (HMB) and to write the data to the first memory device without dedicating space in the controller for the write coalescing to support a number of simultaneous streams greater than the number of available open blocks in the data storage device.

17. memory means; a controller coupled to said memory means; wherein the controller: Translating the write command from a physical region page (PRP) entry to a scatter-aggregate list (SGL) entry; storing the data associated with the PRP entry and the translated write command in a host device; notifying the host device that the write command has been completed; a data storage device configured to, after said notification, execute said translated write command, said execution including writing said stored data to said memory means;

18. 20. The data storage device of claim 17, wherein the controller is configured not to notify the host device that the translated write command is being executed.

19. 20. The data storage device of claim 17, wherein the controller is configured to execute the translated write command after receiving a flush command from the host device.

20. A first memory device; a controller coupled to the first memory device; wherein the controller: Receives a write command, determining that the write commands can be write coalesced; allocating one or more buffers in a second memory device separate from the first memory device; generating a modified write command from the write command; moving data associated with the write command to at least one of the allocated one or more buffers; It is structured as follows: The data storage device, wherein the controller is further configured to generate a list of pointers to the at least one buffer.

Images