Controller memory buffer (CMB) doorbell synchronization

Abstract

To provide a data storage device that ensures that data is in a controller memory buffer (CMB) before retrieving the data after a doorbell rings.SOLUTION: In the storage system 100, the data storage device comprises a memory device and a controller coupled to the memory device. The controller detects that the number of entries in the submission queue (SQ) doorbell is greater than the number of entries in the CMB doorbell, fetches a number of commands that is less than or equal to the number of entries in the CMB doorbell, and decreases the number of entries in the SQ doorbell and the number of entries in the CMB doorbell by the number of commands fetched.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] FIELD OF THE INVENTION TECHNICAL FIELD Embodiments of the present disclosure generally relate to efficient controller memory buffer (CMB) utilization.

[0002] Description of Related Art The Controller Memory Buffer (CMB) is a non-volatile memory (NVM) express (NVMe) solid state drive (SSD) feature in which the data storage device maintains some local memory for host device use. Due to the CMB size, the CMB resides in local dynamic random access memory (DRAM). The host device can use that memory for user data related to read or write commands. The host device can use the CMB for physical region page (PRP) / scatter gather list (SGL) lists, or for completion queues (CQ) and submission queues (SQ). The NVMe standard provides the following text:

[0003] The address space allocated to the CMB shall be 4KiB aligned. It is recommended that the controller allocate the CMB on an 8KiB boundary. The controller shall support burst transactions up to the maximum payload size, support byte enables, and support any byte alignment. The host shall ensure that all writes to the CMB required for the command have been sent before updating the SQ tail doorbell property. Memory write requests to the SQ tail doorbell property shall not have the relaxed ordering bit set to '1' to ensure that previous writes to the CMB have completed.

[0004] The last sentence of the previous paragraph suggests a possible race condition between the write command and the doorbell. In the case of a write command, the host device writes the command to the CMB to initiate a write to DRAM. In the case of a doorbell, the host device writes the doorbell to the controller, which is observed by the controller, and the controller begins reading the command from DRAM. If the write command completes before the controller begins reading from DRAM, all is well. However, if the controller begins reading from DRAM before the write command completes, the controller reads "stale" data from DRAM and not the correct data. The "relaxed ordering bits" protect the data until it reaches the device peripheral component express interconnect (PCIe) boundary, but not after that. The doorbell and the command travel separate paths through the data storage device, leading to a race scenario.

[0005] Therefore, there is a need in the art to ensure that the data is in the CMB before retrieving it after the doorbell rings. Summary of the Invention

[0006] The use of controller memory buffers (CMBs) is increasing. Host devices often place a transmission queue (SQ) in the CMB to utilize the memory provided by a data storage device. The host device places an entry in the SQ and rings the doorbell associated with the SQ. The controller checks the SQ when the doorbell rings. Even if the doorbell rings, the data may not be in the SQ due to the fact that access to the CMB is slow. To ensure that the data is in the SQ, the controller can utilize two doorbells, one that the host device rings and a second doorbell that is invisible to the host device. Once the data is in the SQ, the second doorbell rings. The controller then retrieves the data from the SQ only if both doorbells are ringing.

[0007] In one embodiment, the data storage device comprises a memory device and a controller coupled to the memory device, wherein the controller is configured to detect that the number of entries in the SQ doorbell is greater than the number of entries in the CMB doorbell, fetch a number of commands, where the number of commands is less than or equal to the number of entries in the CMB doorbell, and reduce the number of entries in the SQ doorbell and the number of entries in the CMB doorbell by the number of commands fetched.

[0008] In another embodiment, a data storage device includes a memory device and a controller coupled to the memory device, wherein the controller is configured to receive an indication that a host has issued a doorbell for an SQ, determine that the SQ is managed in a CMB, select a minimum value between the number of commands written by the host device and the number of commands having a doorbell issued for the SQ, reduce the minimum value from both the number of commands written by the host device and the number of commands having a doorbell issued for the SQ, and fetch commands from the SQ.

[0009] In another embodiment, a data storage device comprises means for storing data and a controller coupled to the means for storing data, wherein the controller is configured to maintain a first doorbell corresponding to SQ, maintain a second doorbell corresponding to CMB, and fetch a number of commands from CMB, wherein the number of commands is less than or equal to the number of entries in the second doorbell. [Brief explanation of the drawings]

[0010] So that the above-mentioned features of the present disclosure may be understood in detail, a more particular description of the present disclosure briefly summarized above may be had by reference to embodiments, some of which are illustrated in the accompanying drawings. It should be noted, however, that the accompanying drawings illustrate only typical embodiments of the present disclosure and therefore should not be considered as limiting its scope, as the present disclosure may admit of other equally effective embodiments. [Figure 1] 1 is a schematic block diagram illustrating a storage system in which a data storage device may function as a storage device for a host device, according to certain embodiments. [Figure 2] FIG. 1 is a schematic diagram of a controller having a controller memory buffer (CMB). [Figure 3A] 10 is a flow chart illustrating a doorbell ringing according to one embodiment. [Figure 3B] 10 is a flowchart illustrating doorbell processing, according to one embodiment. [Figure 4] 1 is a schematic diagram of a data storage system, according to one embodiment. [Figure 5] 1 is a flowchart illustrating data processing, according to one embodiment.

[0011] For ease of understanding, wherever possible, the same reference numbers have been used to designate identical elements common to the figures. It is contemplated that elements disclosed in one embodiment may be beneficially utilized in other embodiments without specific recitation. DETAILED DESCRIPTION OF THE INVENTION

[0012] Reference will be made below to embodiments of the present disclosure. However, it should be understood that the present disclosure is not limited to the specific described embodiments. Instead, any combination of the following features and elements, whether associated with different embodiments or not, is contemplated for implementing and practicing the present disclosure. Furthermore, embodiments of the present disclosure may achieve other possible solutions and / or advantages over the prior art, but whether or not a particular advantage is achieved by a given embodiment does not limit the present disclosure. Accordingly, the following aspects, features, embodiments, and advantages are merely exemplary and should not be considered elements or limitations of the appended claims unless expressly recited in the claims. Similarly, references to "the present disclosure" should not be construed as a generalization of any inventive subject matter disclosed herein, and should not be considered elements or limitations of the appended claims unless expressly recited in the claims.

[0013] The use of controller memory buffers (CMBs) is increasing. Host devices often place a transmission queue (SQ) in the CMB to utilize the memory provided by a data storage device. The host device places an entry in the SQ and rings the doorbell associated with the SQ. The controller checks the SQ when the doorbell rings. Even if the doorbell rings, the data may not be in the SQ due to the fact that access to the CMB is slow. To ensure that the data is in the SQ, the controller can utilize two doorbells, one that the host device rings and a second doorbell that is invisible to the host device. Once the data is in the SQ, the second doorbell rings. The controller then retrieves the data from the SQ only if both doorbells are ringing.

[0014] 1 is a schematic block diagram illustrating a storage system 100 having a data storage device 106 that may function as a storage device for a host device 104, according to certain embodiments. For example, the host device 104 may store and retrieve data utilizing non-volatile memory (NVM) 110 included in the data storage device 106. The host device 104 includes host dynamic random access memory (DRAM) 138. In some examples, the storage system 100 may include multiple storage devices, such as the data storage device 106, that may operate as a storage array. For example, the storage system 100 may include multiple data storage devices 106 configured as a redundant array of inexpensive / independent disks (RAID) that collectively function as a mass storage device for the host device 104.

[0015] 1, host device 104 may store data on and / or retrieve data from one or more storage devices, such as data storage device 106. As shown in FIG. 1, host device 104 may communicate with data storage device 106 via interface 114. Host device 104 may include any of a wide range of devices, including a computer server, a network-attached storage (NAS) unit, a desktop computer, a notebook (i.e., laptop) computer, a tablet computer, a set-top box, a telephone handset such as a so-called "smart" phone, a so-called "smart" pad, a television, a camera, a display device, a digital media player, a video game console, a video streaming device, or any other device capable of sending or receiving data from a data storage device.

[0016] The host DRAM 138 may optionally include a host memory buffer (HMB) 150. The HMB 150 is a portion of the host DRAM 138 allocated to the data storage device 106 for exclusive use by the controller 108 of the data storage device 106. For example, the controller 108 may store mapping data, buffered commands, logical to physical (L2P) tables, metadata, etc. in the HMB 150. In other words, the HMB 150 may be used by the controller 108 to store data that would normally be stored in the controller 108's internal memory, such as the volatile memory 112, the buffer 116, static random access memory (SRAM), etc. In examples where the data storage device 106 does not include DRAM (i.e., the optional DRAM 118), the controller 108 may utilize the HMB 150 as the DRAM of the data storage device 106.

[0017] Data storage device 106 includes controller 108, NVM 110, power supply 111, volatile memory 112, interface 114, write buffer 116, and optional DRAM 118. In some examples, data storage device 106 may include additional components not shown in FIG. 1 for clarity. For example, data storage device 106 may include a printed circuit board (PCB) to which the components of data storage device 106 are mechanically attached and which includes conductive traces that electrically interconnect components such as data storage device 106. In some examples, the physical dimensions and connector configuration of data storage device 106 may conform to one or more standard form factors. Some exemplary standard form factors include, but are not limited to, a 3.5-inch data storage device (e.g., HDD or SSD), a 2.5-inch data storage device, a 1.8-inch data storage device, a peripheral component interconnect (PCI), a PCI-extended (PCI-X), a PCI Express (PCIe) (e.g., PCIe x1, x4, x8, x16, PCIe Mini Card, MiniPCI, etc.). In some examples, the data storage device 106 may be directly coupled to the motherboard of the host device 104 (e.g., soldered directly or plugged into a connector).

[0018] The interface 114 may include one or both of a data bus for exchanging data with the host device 104 and a control bus for exchanging commands with the host device 104. The interface 114 may operate according to any suitable protocol. For example, the interface 114 may operate according to one or more of the following protocols: advanced technology attachment (ATA) (e.g., serial-ATA (SATA) and parallel-ATA (PATA)), Fibre Channel Protocol (FCP), small computer system interface (SCSI), serially attached SCSI (SAS), PCI, PCIe, Non-Volatile Memory Express (NVMe), OpenCAPI, GenZ, Cache Coherent Interface Accelerator (CCIX), Open Channel SSD (OCSSD), etc. The interface 114 (e.g., a data bus, a control bus, or both) is electrically connected to the controller 108 and provides an electrical connection between the host device 104 and the controller 108, allowing data to be exchanged between the host device 104 and the controller 108. In some examples, the electrical connection of the interface 114 may also allow the data storage device 106 to receive power from the host device 104. For example, as shown in FIG. 1, the power supply 111 may receive power from the host device 104 via the interface 114.

[0019] The NVM 110 may include multiple memory devices or memory units. The NVM 110 may be configured to store and / or retrieve data. For example, a memory unit of the NVM 110 may receive data and a message from the controller 108 instructing the memory unit to store the data. Similarly, the memory unit may receive a message from the controller 108 instructing the memory unit to retrieve the data. In some examples, each of the memory units may be referred to as a die. In some examples, the NVM 110 may include multiple dies (i.e., multiple memory units). In some examples, each memory unit may be configured to store a relatively large amount of data (e.g., 128 MB, 256 MB, 512 MB, 1 GB, 2 GB, 4 GB, 8 GB, 16 GB, 32 GB, 64 GB, 128 GB, 256 GB, 512 GB, 1 TB, etc.).

[0020] In some examples, each memory unit may include any type of non-volatile memory device, such as a flash memory device, a phase-change memory (PCM) device, a resistive random-access memory (ReRAM) device, a magneto-resistive random-access memory (MRAM) device, a ferroelectric random-access memory (F-RAM), a holographic memory device, and any other type of non-volatile memory device.

[0021] The NVM 110 may include multiple flash memory devices or memory units. The NVM flash memory devices may include NAND or NOR-based flash memory devices and may store data based on the charge contained in the floating gate of the transistor in each flash memory cell. In an NVM flash memory device, the flash memory device may be divided into multiple dies, each of which may include multiple physical or logical blocks, and the multiple physical or logical blocks may be further divided into multiple pages. Each of the multiple blocks within a particular memory device may include multiple NVM cells. Rows of NVM cells may be electrically connected using word lines to define one of multiple pages. Each cell in each of the multiple pages may be electrically connected to a respective bit line. Furthermore, the NVM flash memory device may be a 2D or 3D device and may be a single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), or quad-level cell (QLC) device. The controller 108 may write data to the NVM flash memory device at the page level, read data from the NVM flash memory device, and erase data from the NVM flash memory device at the block level.

[0022] The power supply 111 may provide power to one or more components of the data storage device 106. When operating in a standard mode, the power supply 111 may provide power to one or more components using power provided by an external device, such as the host device 104. For example, the power supply 111 may provide power to one or more components using power received from the host device 104 via the interface 114. In some examples, the power supply 111 may include one or more power storage components configured to provide power to one or more components when operating in a shutdown mode, such as when power is no longer received from an external device. In this manner, the power supply 111 may function as an on-board backup power source. Some examples of the one or more power storage components include, but are not limited to, capacitors, supercapacitors, batteries, etc. In some examples, the amount of power that can be stored by the one or more power storage components may be a function of the cost and / or size (e.g., area / volume) of the one or more power storage components. In other words, as the amount of power stored by the one or more power storage components increases, the cost and / or size of the one or more power storage components also increase.

[0023] The volatile memory 112 may be used by the controller 108 to store information. The volatile memory 112 may include one or more volatile memory devices. In some examples, the controller 108 may use the volatile memory 112 as a cache. For example, the controller 108 may store cached information in the volatile memory 112 until the cached information is written to the NVM 110. As shown in FIG. 1 , the volatile memory 112 may consume power received from the power supply 111. Examples of the volatile memory 112 include, but are not limited to, random-access memory (RAM), dynamic random access memory (DRAM), static RAM (SRAM), and synchronous dynamic RAM (SDRAM (e.g., DDR1, DDR2, DDR3, DDR3L, LPDDR3, DDR4, LPDDR4, etc.)). Similarly, the optional DRAM 118 may be utilized to store mapping data, buffered commands, logical-to-physical (L2P) tables, metadata, cached data, and the like. In some examples, the data storage device 106 is DRAM-less, as it does not include the optional DRAM 118. In other examples, the data storage device 106 includes the optional DRAM 118.

[0024] The controller 108 may manage one or more operations of the data storage device 106. For example, the controller 108 may manage reading data from and / or writing data to the NVM 110. In some embodiments, when the data storage device 106 receives a write command from the host device 104, the controller 108 may initiate the data storage command to store the data in the NVM 110 and monitor the progress of the data storage command. The controller 108 may determine at least one operating characteristic of the storage system 100 and store the at least one operating characteristic in the NVM 110. In some embodiments, when the data storage device 106 receives a write command from the host device 104, the controller 108 temporarily stores data associated with the write command in an internal memory or write buffer 116 before sending the data to the NVM 110. The controller 108 may include a circuit or processor configured to execute a program for operating the data storage device 106.

[0025] The controller 108 may include an optional second volatile memory 120. The optional second volatile memory 120 may be similar to the volatile memory 112. For example, the optional second volatile memory 120 may be an SRAM. The controller 108 may allocate a portion of the optional second volatile memory to the host device 104 as a controller memory buffer (CMB) 122. The CMB 122 may be directly accessed by the host device 104. For example, rather than maintaining one or more transmit queues within the host device 104, the host device 104 may utilize the CMB 122 to store one or more transmit queues that are normally maintained within the host device 104. In other words, the host device 104 may generate commands and store the generated commands, with or without associated data, in the CMB 122, and the controller 108 accesses the CMB 122 to retrieve the stored generated commands and / or associated data.

[0026] FIG. 2 is a schematic diagram of a controller 200 with a CMB. A host device interacts with the PCIe of a data storage device, and the host device may write data or commands, which in FIG. 2 is the CMB. Commands sent to the CMB are transmitted on the inbound path. The inbound path is where the host device writes data to either the CMB or the doorbell (i.e., BAR0), where the doorbell resides. Transactions coming from the host device are detected by an inbound (IB) address decoder to determine the target of the transaction, such as the CMB. These transactions are forwarded to the CMB.

[0027] The host device may also write other transactions, such as a doorbell to notify the data storage device that one of the SQs has a pending command. The doorbell is a register implemented in the NVMe layer. To access the register, the inbound path is used, and the IB address decoder determines that BAR0 should be accessed instead of CMB. Therefore, the transaction for BAR0 is forwarded to BAR0.

[0028] The doorbell and write commands go on separate paths to BAR0 and CMB respectively. Because the CMB resides in DRAM, writing to the CMB is slow, so the doorbell may be accepted before the data is in the CMB, which may lead to the controller reading the data from the CMB too early and therefore getting stale data.

[0029] To send a command to the data storage device, the host needs to write the command to the SQ using the inbound path and then go directly to the CMB, which takes time. The host device doesn't actually care whether the data is actually written to the CMB or not. The host device just posts write transactions and doesn't wait for any response from the data storage device for those transactions. The host device writes the command to the CMB and then immediately rings the doorbell by accessing the BAR0 area.

[0030] The outbound path is the path along which the data storage device is writing or reading from the CMB or host device. There are several interconnects between PCIe and NVMe that travel from the NVMe towards the outbound (OB) address decoder and CMB, and towards the PCIe.

[0031] There may be scenarios where the host device writes the command again. Write commands still take time, and if the write occurs immediately after ringing the doorbell, there may be a scenario where the doorbell transaction is accepted by the NVMe layer before the data is written to the CMB. In such a scenario, the data storage device attempts to access the CMB and obtains incorrect data. This scenario is a valid conflict scenario that can occur when the CMB is in DRAM and BAR0 is in a register.

[0032] One approach to resolving the contention scenario is to ensure the CMB write pipe is clear before performing a command fetch due to the doorbell, but there is a latency penalty. When the doorbell rings, the write path needs to be flushed to ensure the write pipe is empty so that all previous transactions issued by the host device are actually completed. This approach has a read performance problem because every read transaction needs to flush the write pipe, which takes time and therefore increases latency.

[0033] Dual doorbells may be beneficial as described herein. When using dual doorbells, the controller fetches commands only if a host-issued doorbell occurs and the command has been written to DRAM. The host device is unaware of the additional doorbell. An example of the use of the additional doorbell will now be described.

[0034] First, the host device writes a command to the CMB, and then the host device issues a doorbell. An additional doorbell is triggered by the CMB. The CMB rings the additional doorbell after the data or command is written to DRAM. In this scenario, the data storage device will not start fetching commands from the CMB until both doorbells ring. The first doorbell is the doorbell ringed by the host device. The other doorbell indicates that data is available in the CMB. The NVMe layer will not access the command stored in the CMB unless two doorbell registers indicate that the data is valid or that a command is pending in the CMB and ready to be fetched. Adding the additional doorbell confirms that the command is actually stored in the CMB.

[0035] FIG. 3A is a flowchart 300 illustrating the doorbell ringing, according to one embodiment. When the host writes to the CMB, the data is written to DRAM. If the write was for one (or more) command entries to an SQ, the counter "CMB_DOOR_BELL" for that particular SQ is incremented accordingly. In FIG. 3A, the host device first sends a command to write data to the CMB in block 302, and the data is written to DRAM in block 304. In block 306, a determination is made as to whether the write command is for an SQ. If yes, then in 308, CMB_DOOR_BELL[N] is incremented by the number of write commands. If no, then the process ends in block 310. In either case, if writing to an SQ in block 306, then after incrementing CMB_DOOR_BELL[N] in block 308, the process ends in block 310.

[0036] 3B is a flowchart 350 illustrating doorbell processing, according to one embodiment. The flow begins in block 352 when the host issues a doorbell to send queue [M], which asks the controller to fetch a command (or [C] command). First, the controller checks whether SQ[M] is managed in the CMB or in host DRAM in block 354. If the SQ is not managed in DRAM, then the data storage device fetches the command from the host in block 362 and the flow is complete. If SQ[M] is managed in the CMB, then the data storage device does the following: Select the minimum value (call it [X]) between the number of commands written by the host (from FIG. 3A: CMB_DOOR_BELL[M]) and the number of commands [C] that have been doorbelled for the send queue [M], reduce the minimum value [X] found from both values ​​([C] and CMB_DOOR_BELL[M]), and fetch [X] commands at block 356. If the host device requested to fetch more commands ([C] > 0) than were available when the fetch was made at block 358, the process repeats at block 356 until all commands have been written to DRAM (FIG. 3A) and fetched by the controller. Otherwise, the process ends at block 360.

[0037] Considering Figures 3A and 3B together, after receiving the first command to write data, the data storage device detects whether the write command is to write to SQ in the CMB. If it is not writing to SQ, nothing special is done. If it is writing to SQ, a write command is available and the data storage device can go ahead and fetch the write command. However, if a write command is present in the CMB, the doorbell from the host device indicates that the command is available, but the command is not fetched. Instead, the data storage device waits for a second doorbell. Only when the second doorbell transaction occurs is the write command fetched. Instead of having a single doorbell indication coming from the host device, there is an additional doorbell indication created within the controller. The second doorbell indication does not come from the host device. The additional doorbell is implemented inside the data storage device to verify that the write command is actually in DRAM.

[0038] Figure 4 is a schematic diagram 400 of a data storage system, according to one embodiment. Figure 4 includes a device controller with a host interface module (HIM), which includes an SQ write monitor, an SQ command fetcher, and two NVMe doorbells. The device controller also includes multiple processors and a DDR controller for the volatile memory (i.e., DRAM), which in Figure 4 includes SQ in the CMB. A flash interface module (FIM) exists for each flash (e.g., NAND) module. There is also a command scheduler, an encryption / decryption module, and a data path that includes ECC and RAID.

[0039] The SQ write monitor is an engine responsible for monitoring write accesses to SQs implemented in the CMB DRAM. When the host device writes a new command, a new doorbell register is updated. The NVMe doorbell x2 is an engine that implements two doorbell registers per SQ. For every SQ implemented in the CMB, there are two doorbells: one for the host device and one for internal use. The first doorbell register is defined in the NVMe standard, and the second doorbell register is based on write and read accesses of the associated SQ. Write transactions are issued by the host device, and read transactions are issued by the device controller. The SQ command fetcher is an engine responsible for command fetching, but fetching occurs only when both doorbell registers indicate that a command is pending in the associated SQ.

[0040] The SQL write monitor is used to monitor whether the host device is writing commands to the SQ implemented in the CMB. Detecting the write location is required to ensure that a second doorbell register is used to ensure that the data storage device waits for two doorbell indications before fetching from the CMB.

[0041] 5 is a flowchart 500 illustrating data processing, according to one embodiment. Initially, at block 502, the data storage device detects that the host device has written a command to an SQ located in the CMB. Accordingly, at block 504, the CMB doorbell is incremented by one. At block 506, a determination is made as to whether any additional CMB commands have been detected. If there are more CMB commands, the process repeats at block 504; however, if there are no additional CMB commands, at block 508, a determination is made as to whether the SQ doorbell is greater than zero. If the SQ doorbell is not greater than zero, the process repeats at block 506. If the SQ doorbell is greater than zero, at block 510, a determination is made as to whether the SQ doorbell is greater than the CMB doorbell.

[0042] At block 512, the SQ doorbell is detected to be ringing, and at block 514, the SQ doorbell is incremented by 1. At block 516, a determination is made as to whether any additional doorbells have been detected. If yes, the process repeats as block 514. If no, the process moves to block 510.

[0043] At block 510, a determination is made as to whether SQ doorbell is numerically greater than CMB doorbell. If SQ doorbell is greater than CMB doorbell, then the data storage device may fetch only a number of commands equal to the number of CMB doorbells at block 518 and decrement CMB doorbell and SQ doorbell by the number of commands fetched at block 522. If SQ doorbell is not greater than CMB doorbell at block 510, then the data storage device fetches a number of commands equal to SQ doorbell at block 520 and proceeds to block 522. After block 522, the process repeats at blocks 502 and 512.

[0044] By tracking how many commands have been posted by the host device per SQ in the CMB and how many of those commands were triggered by doorbells, the data storage device can fetch commands without risking race conditions. Doorbells do not backpressure CMB writes and therefore do not cause bandwidth degradation.

[0045] In one embodiment, the data storage device comprises a memory device and a controller coupled to the memory device, wherein the controller is configured to detect that the number of entries in the SQ doorbell is greater than the number of entries in the CMB doorbell, fetch a number of commands, the number of commands being less than or equal to the number of entries in the CMB doorbell, and decrease the number of entries in the SQ doorbell and the number of entries in the CMB doorbell by the number of fetched commands. The controller is further configured to increase the number of entries in the CMB doorbell by one each time a new command is written to the CMB. The controller is further configured to increase the number of entries in the SQ doorbell by one each time a new doorbell is written to the SQ doorbell register. The controller is configured to determine whether an SQ exists in the CMB. The controller is configured not to fetch a number of commands greater than the number of entries in the CMB doorbell. The controller includes a host interface module (HIM) including an SQ write monitor, an SQ command fetcher, and two NVMe doorbells. The two NVMe doorbells include an SQ doorbell and a CMB doorbell. The SQ write monitor is configured to monitor write accesses to the SQ implemented in the CMB. When a host device writes a new command, an entry is made in the CMB doorbell. The SQ command fetcher is configured to fetch a command when both doorbells indicate that a command is pending in the associated SQ.

[0046] In another embodiment, the data storage device includes a memory device and a controller coupled to the memory device, wherein the controller is configured to receive an indication that a host has issued a doorbell for an SQ, determine that the SQ is managed within a CMB, select a minimum value between the number of commands written by the host device and the number of commands having a doorbell issued for the SQ, reduce the minimum value from both the number of commands written by the host device and the number of commands having a doorbell issued for the SQ, and fetch commands from the SQ. The number of commands fetched is equal to the minimum value. The number of commands written by the host device is equal to the number of command doorbells. The controller is further configured to maintain two doorbells. One doorbell of the two doorbells is for the number of commands written by the host device. The second doorbell of the two doorbells is for the number of commands doorbelled for the SQ by the host device. One of the two doorbells is defined by the NVMe standard, and the second of the two doorbells is based on SQ write and read access.

[0047] In another embodiment, a data storage device comprises a means for storing data and a controller coupled to the means for storing data, the controller being configured to maintain a first doorbell corresponding to an SQ, maintain a second doorbell corresponding to a CMB, and fetch a number of commands from the CMB, the number of commands being less than or equal to the number of entries in the second doorbell. The SQ is allocated within the CMB. The controller is configured to monitor write accesses to the SQ.

[0048] While the foregoing is directed to embodiments of the present disclosure, other and further embodiments of the disclosure may be devised without departing from the basic scope thereof, the scope of which is determined by the claims that follow.

Claims

1. 1. A data storage device comprising: a memory device; a controller coupled to the memory device, the controller comprising: Detecting that the number of entries in the transmit queue (SQ) doorbell is greater than the number of entries in the controller memory buffer (CMB) doorbell; fetching a number of commands, the number of commands being less than or equal to the number of entries in the CMB doorbell; a data storage device configured to decrease the number of entries in the SQ doorbell and the number of entries in the CMB doorbell by the number of commands fetched.

2. 2. The data storage device of claim 1, wherein the controller is further configured to increase the number of entries in the CMB doorbell by one each time a new command is written to the CMB.

3. 2. The data storage device of claim 1, wherein the controller is further configured to increase the number of entries in the SQ doorbell register by one each time a new doorbell is written to the SQ doorbell register.

4. The data storage device of claim 1 , wherein the controller is configured to determine whether an SQ exists in a CMB.

5. 2. The data storage device of claim 1, wherein the controller is configured to not fetch a number of commands greater than the number of entries in the CMB doorbell.

6. 10. The data storage device of claim 1, wherein the controller includes a host interface module (HIM) including an SQ write monitor, an SQ command fetcher, and two non-volatile memory express (NVMe) doorbells.

7. 7. The data storage device of claim 6, wherein the two NVMe doorbells include the SQ doorbell and the CMB doorbell.

8. The data storage device of claim 6 , wherein the SQ write monitor is configured to monitor write accesses to an SQ implemented in a CMB.

9. 9. The data storage device of claim 8, wherein an entry is made to the CMB doorbell when a host device writes a new command.

10. 7. The data storage device of claim 6, wherein the SQ command fetcher is configured to fetch a command when both doorbells indicate that a command is pending in the associated SQ.

11. 1. A data storage device comprising: a memory device; a controller coupled to the memory device, the controller comprising: receiving an indication that the host has issued a doorbell for the send queue (SQ); determining that the SQ is managed in a controller memory buffer (CMB); Selecting the minimum value between the number of commands written by the host device and the number of commands with doorbells issued for said SQ; Reducing the minimum from both the number of commands written by the host device and the number of commands with doorbells issued for the SQ; A data storage device configured to fetch commands from the SQ.

12. 12. The data storage device of claim 11, wherein the number of commands fetched is equal to the minimum value.

13. 12. The data storage device of claim 11, wherein the number of commands written by the host device is equal to the number of command doorbells.

14. The data storage device of claim 11 , wherein the controller is further configured to maintain two doorbells.

15. 15. The data storage device of claim 14, wherein one of the two doorbells corresponds to the number of the command written by the host device.

16. 16. The data storage device of claim 15, wherein a second of the two doorbells is for the number of the command to ring the doorbell for the SQ by the host device.

17. 15. The data storage device of claim 14, wherein one of the two doorbells is defined in the Non-Volatile Memory Express (NVMe) standard and a second of the two doorbells is based on the write and read access of the SQ.

18. 1. A data storage device comprising: means for storing data; a controller coupled to the means for storing the data, the controller comprising: Maintaining a first doorbell corresponding to a transmission queue (SQ); maintaining a corresponding second doorbell in a controller memory buffer (CMB); a data storage device configured to fetch a number of commands from the CMB, the number of commands being less than or equal to a number of entries in the second doorbell;

19. The data storage device of claim 18 , wherein the SQs are allocated within the CMBs.

20. 20. The data storage device of claim 18, wherein the controller is configured to monitor write accesses to the SQ.

Images