Data storage devices, systems, and related methods for grouping doorbell transaction commands from host devices.
By grouping commands from the host device and executing them in different operational sequences, the problems of resource waste and complexity caused by disordered operations are solved, achieving simplified and efficient device operation.
Patent Information
- Application Number
- CN202280048405.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2021-10-20
- Filing Date
- 2022-05-10
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2042-05-10
AI Technical Summary
In existing technologies, disordered operations between host devices and data storage devices lead to problems such as large batch filling, resource waste, and increased complexity in host device operations.
The controller of the data storage device groups multiple commands from the host device into multiple groups with a grouping order, and executes these commands in different operation orders, managing these groups by receiving, scanning and sending to complete doorbell transactions.
It simplifies the operation process of the host equipment, reduces resource consumption and waste, and improves equipment performance.
Smart Images

Figure CN117616374B_ABST
Abstract
Description
[0001] Cross-references to related applications
[0002] This application claims the benefit of the entire contents of U.S. Nonprovisional Application No. 17 / 506,212, filed October 20, 2021, entitled “Data storage device, system, and associated method for grouping doorbell transaction commands from a host device,” and the entire contents are hereby incorporated by reference. Background Technology Technical Field
[0003] This disclosure relates in general to data storage devices, systems, and related methods for grouping commands from a host device for doorbell transactions into multiple packets. These commands can be write and / or read operations on one or more memory devices, such as solid-state drives (SSDs).
[0004] Description of related fields
[0005] The purpose of communication between the host device and the data storage device is to access the memory to support the operation of the host device. The data storage device can execute commands in an unordered manner relative to the order in which commands are sent by the host device.
[0006] Out-of-order operations can cause problems, such as requiring large batches of data to be populated on both the host and data storage sides. Out-of-order operations can also lead to wasted power and resources, as these operations may be performed by the data storage device before being effectively canceled by the host device. Furthermore, out-of-order operations can complicate the operation of the host device.
[0007] Therefore, there is a need in the field for data storage devices that practically, simply, and effectively facilitate disordered operations. Summary of the Invention
[0008] This disclosure relates in general to a data storage device, system, and related method for grouping commands from doorbell transactions originating from a host device into multiple packets. The controller of the data storage device is configured to receive multiple submitted doorbell transactions from the host device, each comprising multiple commands. Each submitted doorbell transaction identifies a subset of one or more commands in a first command sequence.
[0009] The controller is configured to group multiple commands from multiple submitted doorbell transactions into multiple groups having a grouping order. Each group in the multiple groups corresponds to a single doorbell transaction in the multiple submitted doorbell transactions. The controller is configured to send one or more completed doorbell transactions to a host device. Each of the one or more completed doorbell transactions identifies a completed group in the multiple groups. In one aspect, each of the one or more completed doorbell transactions identifies the completed group in a second command order identical to the first command order, and the controller is configured to execute the multiple commands in a first operational order different from the first and second command orders. In one aspect, the controller is configured to execute the multiple commands in a second operational order identical to the first and second command orders. In one aspect, the controller is configured to receive a second plurality of submitted doorbell transactions, scan the second plurality of submitted doorbell transactions to obtain an abort command, and execute the multiple commands after scanning the second plurality of submitted doorbell transactions.
[0010] In one embodiment, the data storage device includes one or more memory devices and a controller coupled to the one or more memory devices. The controller is configured to receive multiple submit doorbell transactions from a host device, each containing a plurality of commands in a first overall order. Each submit doorbell transaction identifies a subset of one or more commands in the first command order. The controller is configured to group the commands of the multiple submit doorbell transactions into multiple groups in a grouping order. Each group corresponds to a single doorbell transaction in the multiple submit doorbell transactions. The controller is configured to send one or more complete doorbell transactions to the host device. Each complete doorbell transaction identifies a completed group within the multiple groups. A completed group indicates the completion of a subset of one or more commands corresponding to a completed group.
[0011] In one embodiment, the data storage device includes one or more memory devices and a controller coupled to the one or more memory devices. The controller is configured to receive multiple submit doorbell transactions from a host device, each containing a plurality of commands in a first overall order. Each submit doorbell transaction identifies a subset of one or more commands in the first command order. The controller is configured to group the commands of the multiple submit doorbell transactions into multiple groups in a grouping order. Each group corresponds to a single doorbell transaction in the multiple submit doorbell transactions. The controller is configured to receive a second plurality of submit doorbell transactions and scan the second plurality of submit doorbell transactions for an abort command. The controller is configured to execute multiple commands after scanning the second plurality of submit doorbell transactions. The controller is configured to send one or more complete doorbell transactions to the host device. Each complete doorbell transaction identifies a completed group within the multiple groups. A completed group indicates the completion of a subset of one or more commands corresponding to the completed group.
[0012] In one embodiment, the data storage device includes one or more memory storage devices and a controller coupled to the one or more memory storage devices. The controller is configured to receive from a host device a plurality of submitted doorbell transactions having multiple commands. Each submitted doorbell transaction identifies a subset of one or more commands in a first command order. The controller is configured to group the multiple commands of the multiple submitted doorbell transactions into multiple groups having a grouping order. Each group corresponds to a single doorbell transaction in the plurality of submitted doorbell transactions. The controller is configured to send one or more completed doorbell transactions to the host device. Each completed doorbell transaction identifies a completed group among the multiple groups. The completed group indicates the completion of a subset of one or more commands corresponding to the completed group. Attached Figure Description
[0013] Therefore, a detailed understanding of the foregoing features of this disclosure, a more specific description of this disclosure, and the foregoing brief overview can be obtained by referring to the embodiments, some of which are shown in the accompanying drawings. However, it should be noted that the drawings only illustrate typical embodiments of this disclosure and should therefore not be considered as limiting its scope, as this disclosure allows for other equally effective embodiments.
[0014] Figure 1 It is a schematic block diagram of a storage system in which a data storage device can be used as a storage device for a host device.
[0015] Figure 2 It is based on a specific implementation. Figure 1 The schematic block diagram view of the controller shown is shown.
[0016] Figure 3 It is based on a specific implementation. Figure 1 and Figure 2 The diagram shows a schematic block view of the two cores of the host device.
[0017] Figure 4 It is based on a specific implementation and Figure 1 and Figure 2 The diagram shows a schematic block view of the doorbell operation process related to the storage system.
[0018] Figure 5 This is a schematic block diagram view of a method for determining the write fragmentation level of a memory device according to a specific implementation.
[0019] Figure 6 It is a schematic block diagram view based on a specific implementation of a method of using a host device.
[0020] Figure 7 It is a schematic block diagram view based on a specific implementation of a method of using a data storage device.
[0021] Figure 8 It is a schematic block diagram view of a doorbell operation process related to a specific implementation of a storage system.
[0022] Figure 9 It is a schematic block diagram view based on a specific implementation of a method of using a host device.
[0023] For ease of understanding, the same reference numerals are used where possible to denote the same elements common in the accompanying drawings. It is conceivable that elements disclosed in one embodiment may be advantageously used in other embodiments without specific description. Detailed Implementation
[0024] In the following text, reference is made to embodiments of this disclosure. However, it should be understood that this disclosure is not limited to the specifically described embodiments. Rather, consider any combination of the following features and elements (whether or not related to different embodiments) to achieve and practice this disclosure. Furthermore, while embodiments of this disclosure may achieve advantages over other possible solutions and / or over the prior art, achieving a particular advantage through a given embodiment is not a limitation of this disclosure. Therefore, the following aspects, features, embodiments, and advantages are illustrative only and should not be considered elements or limitations of the appended claims unless expressly stated in the claims. Similarly, reference to “this disclosure” should not be construed as a generalization of any inventive subject matter disclosed herein and should not be considered elements or limitations of the appended claims unless expressly stated in the claims.
[0025] This disclosure relates in general to a data storage device, system, and related method for grouping commands from a host device into multiple packets for a doorbell transaction. The controller of the data storage device is configured to receive multiple submitted doorbell transactions from the host device, each comprising multiple commands. Each submitted doorbell transaction identifies a subset of one or more commands in a first command sequence.
[0026] The controller is configured to group multiple commands from multiple submitted doorbell transactions into multiple groups having a grouping order. Each group in the multiple groups corresponds to a single doorbell transaction in the multiple submitted doorbell transactions. The controller is configured to send one or more completed doorbell transactions to a host device. Each of the one or more completed doorbell transactions identifies a completed group in the multiple groups. In one aspect, each of the one or more completed doorbell transactions identifies a completed group in a second command order identical to the first command order, and the controller is configured to execute the multiple commands in a first operational order different from the first and second command orders. In one aspect, the controller is configured to execute the multiple commands in a second operational order identical to the first and second command orders. In one aspect, the controller is configured to receive a second plurality of submitted doorbell transactions, scan the second plurality of submitted doorbell transactions to obtain an abort command, and execute the multiple commands after scanning the second plurality of submitted doorbell transactions.
[0027] Figure 1 This is a schematic block diagram view of a storage system 100 in which data storage device 106 can be used as a storage device for host device 104, according to a specific implementation.
[0028] Host device 104 communicates with data storage device 106. For example, host device 104 may use non-volatile memory (NVM) 110 included in data storage device 106 to store and retrieve data. Host device 104 includes host dynamic random access memory (DRAM) 138. In one embodiment that can be combined with other embodiments, storage system 100 may include multiple storage devices that can operate as a storage array, such as one or more additional data storage devices 106.
[0029] Host device 104 can store data to and / or retrieve data from one or more storage devices (such as data storage device 106). Figure 1As shown, host device 104 can communicate with data storage device 106. The controller 108 of data storage device 106 can communicate with the host memory buffer (HMB) 139 of host device 104. Host device 104 can include any of a variety of devices, including computer servers, network attached storage (NAS) units, desktop computers, laptops (e.g., laptop computers), tablets, set-top boxes, mobile phones such as so-called "smart" phones, so-called "smart" tablets, televisions, cameras, display devices, digital media players, video game consoles, live video streaming equipment, or other devices capable of sending or receiving data from data storage devices.
[0030] Data storage device 106 includes a controller 108, an NVM 110, a power supply 111, volatile memory 112, and a write buffer 116. In one embodiment that can be combined with other embodiments, the write buffer 116 is temporary memory. The controller 108 is coupled to one or more memory devices and / or volatile memory 112 of the NVM 110 via the write buffer 116. For clarity, data storage device 106 may include Figure 1 Additional components are not shown. For example, data storage device 106 may include a printed circuit board (PCB) to which components of data storage device 106 are mechanically attached. The PCB includes conductive traces for electrically interconnecting components such as data storage device 106. In one embodiment that can be combined with other embodiments, the physical dimensions and connector configuration of data storage device 106 conform to one or more standard form factors. Some exemplary standard form factors include, but are not limited to, 3.5” data storage devices (e.g., hard disk drives (HDDs) or solid-state drives (SSDs)), 2.5” data storage devices, 1.8” data storage devices, peripheral component interconnects (PCI), PCI expansion (PCI-X) and PCI fast lanes (PCIe) (e.g., PCIe x1, x4, x8, x16, PCIe Mini cards, MiniPCI, etc.). In one embodiment that can be combined with other embodiments, data storage device 106 may be directly coupled (e.g., directly soldered or inserted into a connector) to the motherboard of host device 104.
[0031] NVM 110 includes one or more memory devices or memory cells. NVM 110 can be configured to store and / or retrieve data. For example, the memory device of NVM 110 can receive data and messages from controller 108 instructing the memory device to store data. Similarly, the memory cell can receive messages from controller 108 instructing the memory device to retrieve data. Each memory device may be referred to as a die. In one embodiment that can be combined with other embodiments, NVM 110 includes multiple dies (e.g., multiple memory devices). Each memory cell can be configured to store a relatively large amount of data (e.g., 128MB, 256MB, 512MB, 1GB, 2GB, 4GB, 8GB, 16GB, 32GB, 64GB, 128GB, 256GB, 512GB, 1TB, etc.).
[0032] Each memory cell may include any type of non-volatile memory device, such as flash memory device, phase-change memory (PCM) device, resistive random access memory (ReRAM) device, magnetoresistive random access memory (MRAM) device, ferroelectric random access memory (F-RAM), holographic memory device, and any other type of non-volatile memory device.
[0033] NVM 110 may include multiple flash memory devices or memory cells. The flash memory devices of NVM 110 may include NAND- or NOR-based flash memory devices and may store data based on the charge in the floating gate of the transistors included in each flash memory cell. In the NVM flash memory device, the flash memory device may be divided into multiple dies, wherein each of the multiple dies includes multiple physical or logical blocks, which may be further divided into multiple pages. Each of the multiple blocks within a particular memory device may include multiple NVM cells. Rows of NVM cells may be electrically connected using word lines to define pages within the multiple pages. A corresponding cell in each of the multiple pages may be electrically connected to a corresponding bit line. Furthermore, the NVM flash memory device may be a 2D or 3D device and may be a single-level cell (SLC), multi-level cell (MLC), three-level cell (TLC), or four-level cell (QLC). Controller 108 may write data to and read data from the NVM flash memory device of NVM 110 at the page level and delete data from the NVM flash memory device of NVM 110 at the block level.
[0034] Each of one or more storage devices in the NVM 110 includes multiple blocks therein capable of storing (e.g., writing) data. The controller 108 is configured to write data to and / or read data from the multiple blocks of the NVM 110 in response to a message received by the controller 108 from the host device 104.
[0035] Power supply 111 provides power to one or more components of data storage device 106. When operating in standard mode, power supply 111 can use power provided by external devices such as host device 104 to power one or more components. Power supply 111 can utilize HIM 114 (such as...) Figure 2 (As shown) Power received from host device 104 supplies power to one or more components. In one embodiment that can be combined with other embodiments, power supply 111 includes one or more power storage components configured to supply power to one or more components when operating in a shutdown mode, such as when power reception from external devices is stopped. In this way, power supply 111 can be used as an onboard backup power source. Some examples of one or more power storage components include, but are not limited to, capacitors, supercapacitors, batteries, etc. The amount of electricity that can be stored by one or more power storage components can be a function of the cost and / or size (e.g., area / volume) of one or more power storage components. In other words, as the amount of electricity stored by one or more power storage components increases, the cost and / or size of one or more power storage components also increases.
[0036] Volatile memory 112 can be used by controller 108 to store information. Volatile memory 112 may include one or more memory devices. In one embodiment that can be combined with other embodiments, controller 108 uses volatile memory 112 as a cache. Controller 108 may store cached information in volatile memory 112 until the cached information is written to NVM 110. Figure 1 As shown, volatile memory 112 can consume power received from power supply 111. Examples of volatile memory 112 include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static RAM (SRAM), and synchronous dynamic RAM (SDRAM (e.g., DDR1, DDR2, DDR3, DDR3L, LPDDR3, DDR4, LPDDR4, etc.)).
[0037] Controller 108 can manage one or more operations of data storage device 106. For example, controller 108 can manage reading data from NVM 110 and / or writing data to NVM. In one embodiment that can be combined with other embodiments, when data storage device 106 receives a write command from host device 104, controller 108 initiates a data storage command to store data in NVM 110 and monitors the progress of the data storage command. Controller 108 can determine at least one operational characteristic of storage system 100 and store that at least one operational characteristic in NVM 110.
[0038] Figure 2 It is based on a specific implementation. Figure 1 The diagram shows a schematic block view of the controller 108. The data storage device 106 communicates with the host device 104 using a host interface module (HIM) 114 that communicates with the host device 104. The HIM 114 can communicate with the host device 104's HMB 139.
[0039] HIM 114 may include one or both of a data bus for exchanging data with host device 104 and a control bus for exchanging commands with host device 104. HIM 114 may operate according to any suitable protocol. For example, HIM 114 may operate according to one or more of the following protocols: Advanced Technology Attachment (ATA) (e.g., Serial ATA (SATA) and Parallel ATA (PATA)), Fibre Channel Protocol (FCP), Small Computer System Interface (SCSI), Serial Attached SCSI (SAS), PCI and PCIe, Non-Volatile Memory Fast Channel (NVMe), OpenCAPI, GenZ, Cache Coherent Interface Accelerator (CCIX), Open Channel SSD (OCSSD), etc. HIM 114 (e.g., data bus, control bus, or both) is electrically connected to controller 108, thereby providing an electrical connection between host device 104 and controller 108 and allowing data exchange between host device 104 and controller 108. The electrical connection of HIM 114 may also allow data storage device 106 to receive power from host device 104. Power supply 111 (e.g.) Figure 1 (As shown) can receive power from host device 104 using HIM 114.
[0040] HIM 114 includes a submission database 141 and completion sorting logic 142 that communicates with the submission database 141. HIM 114 is configured to receive multiple submitted doorbell transactions from host device 104 and send one or more completed doorbell transactions to host device 104. Controller 108 includes a processor 121, an encryption / decryption module 125, and a redundant array of independent disks (RAID) 127 with redundancy information. Controller 108 includes a flash interface module (FIM) 131 that communicates with NVM 110, and a command scheduler 123 that communicates with FIM 131 to execute multiple commands via FIM 131. FIM 131 is the logic of controller 108. The multiple commands include read operations and / or write operations that read from and / or write to NVM 110. When executed, the multiple commands can indicate the status and / or configuration of data storage device 106.
[0041] Controller 108 includes encoder / decoder 118. In one example that can be combined with other examples, encoder / decoder 118 is a low-density parity-check code (LDPC) encoder / decoder. In another example that can be combined with other examples, encoder / decoder 118 is an XOR encoder / decoder. Encoder / decoder 118 may include internal memory to hold several matrices including multiple error correction codes (ECCs). One or more matrices in the internal memory of encoder / decoder 118 can allow fast switching between matrices to avoid any firmware (FW) intervention or performance loss due to matrix switching time. In one embodiment that can be combined with other embodiments, when controller 108 of data storage device 106 receives a write command from host device 104, controller 108 temporarily stores the data associated with the write command in the internal memory of encoder / decoder 118, volatile memory 112, or write buffer 116 before sending the data associated with the write command to NVM 110 for writing.
[0042] In one implementation that can be combined with other implementations, the controller 108 is configured according to the Non-Volatile Memory Fast Channel (NVMe) protocol.
[0043] Figure 3 It is based on a specific implementation. Figure 1 and Figure 2 The diagram shows a schematic block view of the two cores 301 and 303 of the host device 104. Cores 301 and 303 can be part of DRAM 138.
[0044] The first core 301 includes a submission queue 310 and a completion queue 330, shown in a 1:1 mapping. Each of the submission queue 310 and the completion queue 330 may include one or more commands 321-324. If a corresponding command is sent to the controller 108, the corresponding command is written to a block in the submission queue 310. If the corresponding command is part of a completion doorbell sent from the controller 108 to the host device 104, then the corresponding command is written to a block in the completion queue 330, which indicates that the corresponding command has been completed. Each of the submission queue 310 and the completion queue 330 corresponds to the same queue for commands 321-324.
[0045] The second core 303 includes a first commit queue 340, a second commit queue 350, and a completion queue 360, showing a 2:1 mapping. Each of the first commit queue 340 and the completion queue 360 corresponds to the first queue of commands 341-344. The second commit queue 350 corresponds to the second queue of commands 351-354.
[0046] Figure 4 It is based on a specific implementation and Figure 1 and Figure 2 The diagram shows a schematic block view of the doorbell operation flow related to the storage system 100. The controller 108 receives multiple submitted doorbell transactions 410, 420, 430, and 440 from the host device 104, each containing commands 411, 412, 421-423, 431-435, 441, and 442. Commands 411, 412, 421-423, 431-435, 441, and 442 are sent to the controller in a first overall order, such that command 411 is the first and command 442 is the last. The multiple submitted doorbell transactions 410, 420, 430, and 440 correspond to a submission queue of the command queue.
[0047] Each of the multiple doorbell submission transactions 410, 420, 430, and 440 identifies a subset of one or more commands in the first command order. As an example, the third doorbell submission transaction 430 identifies a third subset of five commands 431-435. These five commands are in the first command order such that in the third doorbell submission transaction 430, command 431 is the first and command 435 is the last. Doorbell submission transactions 410, 420, 430, and 440 are sent to controller 108, and controller 108 receives doorbell submission transactions 410, 420, 430, and 440 in grouping order, such that doorbell submission transaction 410 is the first and doorbell submission transaction 440 is the last. Controller 108 groups the multiple commands 411, 412, 421-423, 431-435, 441, and 442 into multiple groups according to the grouping order. Each of the multiple groups corresponds to a single doorbell transaction in multiple committed doorbell transactions 410, 420, 430, and 440.
[0048] Controller 108 sends one or more doorbell completion transactions 460, 470, 480, and 490 (four are shown) to host device 104. Doorbell completion transactions 460, 470, 480, and 490 correspond to the completion queue of the command queue. Each doorbell completion transaction 460, 470, 480, and 490 identifies a completed group within a plurality of groups with a second command order identical to the first command order of the group. This completed group indicates the completion of a subset of one or more commands corresponding to the completed group. Commands 411, 412, 421-423, 431-435, 441, and 442 sent to host device 104 are held in the same groups as commands 411, 412, 421-423, 431-435, 441, and 442 were in when they were sent to controller 108. Figure 4 In the specific implementation shown, completed packets are sent in a first completed packet order that differs from the packet order. As an example, according to... Figure 4 The first completion grouping order shown indicates that the first completed doorbell transaction 460 (corresponding to the same group as the second submitted doorbell transaction 420) is the first, and the fourth completed doorbell transaction 490 (corresponding to the same group as the third submitted doorbell transaction 430) is the last. In an embodiment that can be combined with other embodiments, the completion sorting logic 142 is configured to, upon determining the completion of each of the plurality of groups, follow a second completion grouping order that is identical to the order of those groups. Figure 4 (Not shown in the image) Schedule the sending of one or more doorbell transactions 460-490.
[0049] As an example, the fourth completion of doorbell transaction 490 indicates completion of the third subset of commands 431-435. When the third subset of commands 431-435 is sent to controller 108, the third subset of commands 431-435 sent to host device 104 is in the second command sequence, which is the same as the first command sequence. Commands 411, 412, 421-423, 431-435, 441, and 442 are sent to host device 104 in a second overall sequence, different from the first overall sequence.
[0050] Submission database 141 is configured to store multiple submitted doorbell transactions 410, 420, 430, and 440, as well as commands 411, 412, 421-423, 431-435, 441, and 442. Submission database 141 is also configured to store multiple packets. Completion sorting logic 142 is configured to monitor the multiple packets in submission database 141 and schedule the transmission of one or more completed doorbell transactions 460-490 to host device 104 when at least one of the multiple packets is determined to be completed.
[0051] Controller 108 is configured to execute a plurality of commands 411, 412, 421-423, 431-435, 441, and 442 in an unordered manner and in a first operational order different from the first overall order. In an embodiment that can be combined with other embodiments, controller 108 is configured to execute the plurality of commands in an ordered manner and in a second operational order identical to the first overall order. In such an embodiment, completion sequencing logic 142 is configured to schedule the transmission of completed doorbell transactions 460-490 to host device 104 upon determining that each of the plurality of packets is completed. HIM 114 includes command grouping logic configured to identify a single doorbell transaction 410, 420, 430, or 440 receiving each of a plurality of commands 411, 412, 421-423, 431-435, 441, and 442, and to group the corresponding commands into appropriate groups corresponding to the single doorbell transactions 410, 420, 430, or 440.
[0052] Controller 108 is configured to receive a second or more submitted doorbell transactions. Figure 4(Not shown in the diagram) and scans a second plurality of submitted doorbell transactions for an abort command. Controller 108 is configured to execute a plurality of previously received commands 411, 412, 421-423, 431-435, 441, and 442 after scanning the second plurality of submitted doorbell transactions. The abort command corresponds to one of the plurality of commands 411, 412, 421-423, 431-435, 441, and 442, and includes instructions for aborting the command. The abort command includes a unique identifier identifying the command. Controller 108 is configured to remove the command from one or more completed doorbell transactions 460-490 if an abort command is detected. Removing the command from the plurality of groups ensures that the command is not executed by controller 108 and that the command is not part of completed doorbell transactions 460, 470, 480, and 490. After removing the command, complete all other commands (except those that were aborted) and the group to which the command previously belonged.
[0053] Controller 108 is configured to store commands in commit database 141, and command scheduler 123 will not schedule a command if an abort command is stored in commit database 141. Completion sorting logic 142 is configured to monitor multiple groups in commit database 141, and remove a command from one or more complete doorbell transactions 460-490 if an abort command is stored in commit database 141.
[0054] The controller 108 is configured to, for each of the received multiple submitted doorbell transactions, wait to scan subsequent multiple submitted doorbell transactions for an abort command before executing the command for the initially received multiple submitted doorbell transactions.
[0055] Figure 5 This is a schematic block diagram view of a method 500 for determining the write fragmentation level of a memory device according to a specific implementation. For clarity, operations 502-518 of method 500 are described herein with respect to storage system 100. This disclosure contemplates that operations 502-518 of method 500 can be performed with respect to other storage systems. In one embodiment that can be combined with other embodiments, controller 108 is configured to perform operations 502-518 of method 500. In one embodiment that can be combined with other embodiments, controller 108 is configured to instruct other components of data storage device 106 (such as NVM 110, command scheduler 123, FIM 131, commit database 141 of HIM 114, completion sorting logic 142 of HIM 114, and / or other parts of HIM 114) to perform operations 502-518.
[0056] Operation 502 includes receiving multiple submit doorbell transactions from a host device, comprising a plurality of commands in a first overall order. Each submit doorbell transaction in the plurality of submit doorbell transactions identifies a subset of one or more commands in the first command order.
[0057] Operation 504 involves grouping multiple commands from multiple submitted doorbell transactions into multiple groups with a grouping order. Each of the multiple groups corresponds to a single doorbell transaction within the multiple submitted doorbell transactions. Each group includes an identifier for the single doorbell transaction under which each command was received.
[0058] Operation 505 involves storing multiple commands and multiple groups. These commands and groups are stored in the commit database 141.
[0059] Optional operation 506 includes receiving a second or more committed doorbell transactions.
[0060] Optional operation 508 includes scanning a second plurality of committed doorbell transactions to obtain abort commands. The second plurality of committed doorbell transactions may be stored in the committed database 141, and completion sorting logic 142 monitors multiple groups in the committed database 141, and if an abort command is stored in the committed database 141, removes that command from one or more completed doorbell transactions.
[0061] Optional operation 509 includes removing the command corresponding to the abort command if the abort command is detected. If detected, the abort command can be stored in the commit database 141. The command is removed from multiple groups and one or more completed doorbell transactions.
[0062] Operation 510 includes executing multiple commands. Controller 108 can retrieve the multiple commands from host device 104 to execute them. In one embodiment that can be combined with other embodiments, the multiple commands are executed after scanning a second plurality of submitted doorbell transactions. In one embodiment that can be combined with other embodiments, the multiple commands are executed in a first operational order different from the first overall order. In one embodiment that can be combined with other embodiments, the multiple commands are executed in a second operational order identical to the first overall order.
[0063] Operation 514 involves monitoring multiple stored groups and multiple commands. The completion of multiple groups is monitored to determine if all commands within a group have been completed. Completed commands are detected and held until every command within a group (among the multiple groups) has been completed.
[0064] Operation 516 includes scheduling the transmission of one or more completed doorbell transactions to host device 104 when it is determined that at least one of the plurality of packets has been completed. When it is determined that each command in a packet has been completed, the packet is scheduled to be sent to host device 104 in the next single interrupt notification. In one embodiment that can be combined with other embodiments, method 500 waits for the determination that each of the plurality of packets has been completed, and upon determination, schedules the transmission of one or more completed doorbell transactions in a second completed packet order, the same as the packet order.
[0065] Operation 518 includes sending one or more completed doorbell transactions to host device 104. The one or more completed doorbell transactions are sent to host device 104 as part of a single interrupt notification. Each completed doorbell transaction in the one or more completed doorbell transactions identifies a completed packet among multiple packets. A completed packet indicates the completion of a subset of one or more commands corresponding to the completed packet. For each completed packet, a subset of one or more commands corresponding to the completed packet is sent to host device 104 in the same interrupt notification. Each completed doorbell transaction in the one or more completed doorbell transactions identifies a completed packet in a second command order identical to the first command order. In one embodiment that can be combined with other embodiments, one or more completed doorbell transactions are sent in a first completed packet order different from the packet order. In one embodiment that can be combined with other embodiments, one or more completed doorbell transactions are sent in a second completed packet order identical to the packet order.
[0066] Figure 6 This is a schematic block diagram view of a method 600 using host device 104 according to a specific implementation. Method 600 can be used with... Figure 5 The method shown is used in conjunction with 500.
[0067] Operation 602 includes the host device 104 receiving one or more completed doorbell transactions 460-490 from the controller 108 as part of a single interrupt notification.
[0068] Operation 604 involves host device 104 pulling multiple completed commands from a single interrupt notification.
[0069] Operation 606 includes host device 104 using multiple completed commands by writing completed commands to one or more cores (such as...). Figure 3 The completion queues of cores 301 and 303 shown (such as...) Figure 3The completion queues (330, 360) shown are used to update one or more cores. Updating the completion queue can update subsequent commit doorbell transactions sent to controller 108. As an example, subsequent commit doorbell transactions can be updated to remove completed commands. In such an example, when a completed command is written to the completion queue, the corresponding command in the commit queue (such as commit queues 310, 340 and / or 350) can be removed and / or rewritten.
[0070] Method 600 is completed at operation 608. This disclosure envisions that method 600 can be repeated.
[0071] Figure 6 The method 600 for host device 104 shown is relatively simple compared to the method 900 described below, thereby facilitating disordered operation in a simple and efficient manner.
[0072] Figure 7 This is a schematic block diagram view of a method 700 using data storage device 106 according to a specific implementation. Method 700 can be used with... Figure 5 The method shown is used in conjunction with 500.
[0073] Operation 702 includes executing a command to complete a command that is already completed.
[0074] Operation 704 includes holding the completed command at controller 108.
[0075] Operation 706 includes determining whether at least one of a plurality of groups (such as one of a plurality of doorbell transactions 410, 420, 430, 440) is the group that completes each command.
[0076] If the response to operation 706 is yes, resulting in at least one of the multiple packets being completed, then method 700 proceeds to operations 708 and 710. Operation 708 includes scheduling the transmission of at least one completed packet to host device 104 in the next single interrupt notification as completion of the doorbell transaction. Operation 710 includes controller 108 sending a single interrupt notification to host device 104. Method 700 then completes at block 712.
[0077] If the response to operation 706 is no, such that none of the multiple groups is completed, then method 700 completes at box 712, and method 700 continues to hold the completed command.
[0078] This disclosure envisions a repeatable method 700.
[0079] Figure 8 It is a schematic block diagram view of a doorbell operation process related to a specific implementation of a storage system. Figure 8The doorbell operation process in the middle is similar to Figure 4 The doorbell operation process, and includes one or more of its aspects, features, components and / or attributes.
[0080] Multiple commands 411, 412, 421-423, 431-435, 441, and 442 are sent to host device 104 in a second overall order, different from the first overall order, as part of completing a doorbell transaction. Multiple commands 411, 412, 421-423, 431-435, 441, and 442 are also sent to host device 104 in a second command order, different from the first command order, and in a first completion group order, different from the group order, as part of completing a doorbell transaction.
[0081] Figure 9 This is a schematic block diagram view of a method 900 using a host device according to a specific implementation. Method 900 can be used with... Figure 8 The doorbell operation process shown is used in conjunction with this.
[0082] Operation 902 includes the host device receiving an interrupt notification from the data storage device.
[0083] Operation 904 includes scanning for interruption notifications to obtain multiple completed commands.
[0084] Operation 906 involves updating one or more completion queues of the host device's internal database.
[0085] Operation 908 includes determining whether a completed command has been received for each command in the group.
[0086] If the answer to operation 908 is yes, and grouping is completed, then method 900 continues to operations 910 and 912.
[0087] Operation 910 includes scanning for interrupt notifications to obtain completed commands and writing the completed commands to one or more completion queues on the host device. Operation 912 includes the host device raising one or more events to the calling application. These one or more events correspond to the completed commands. Method 900 then completes at box 914.
[0088] If the answer to operation 908 is no, causing the grouping to be incomplete, then method 900 is completed at box 900.
[0089] This disclosure envisions a repeatable method 900. For example... Figure 6 As shown in method 600, the operation of the host device is simplified and streamlined using the aspects described herein (and...). Figure 9 Compared to method 900 shown, it also facilitates disordered operation of commands in data storage devices.
[0090] The benefits of this disclosure include simplicity and usability, efficiency, device performance, reduced waste, and reduced resource consumption. As an example, controller 108 groups multiple commands submitting doorbell transactions into multiple groups to facilitate out-of-order operation of controller 108. Out-of-order operation improves device performance by prioritizing commands used for operation and enhancing pipeline utilization, and improves efficiency and reduces latency (e.g., improved quality of service) by reducing the time between host device 104 sending a command and receiving notification of its completion. As another example, controller 108 facilitates out-of-order operation of controller 108 by identifying completed groups of commands with the same second command order as the first command order, while also helping to reduce the complexity and resource consumption of host device 104. It is believed that by using the aspects described herein, the complexity of host device 104 is significantly reduced while maintaining out-of-order operation of data storage device 106.
[0091] Additionally, as an example, scanning a second or more committed doorbell transactions for abort commands before executing commands for multiple doorbell transactions helps improve efficiency and reduce waste of electricity and resources. Using the aspects described herein, controller 108 can cancel such commands before the abort command is executed by controller 108.
[0092] For some applications (such as games), such benefits can be amplified, where the unordered delivery of completed commands to the completion queue can be limited by operational constraints.
[0093] It is envisioned that one or more aspects disclosed herein may be combined. Furthermore, it is envisioned that one or more aspects disclosed herein may include some or all of the foregoing benefits. As an example, the operations, aspects, components, features, and / or attributes of storage system 100, cores 301, 303, doorbell transaction submission 410, 420, 430 and / or 440, doorbell transaction completion 460-490, and / or method 500 may be combined. In one embodiment that can be combined with other embodiments, regarding... Figure 4 The described one or more operations and about Figure 5 The method 500 shown describes one or more combinations of operations.
[0094] In one embodiment, the data storage device includes one or more memory devices and a controller coupled to the one or more memory devices. The controller is configured to receive multiple submitted doorbell transactions from a host device, each containing multiple commands in a first overall order. Each submitted doorbell transaction identifies a subset of one or more commands in the first command order. The controller is configured to group the multiple commands of the multiple submitted doorbell transactions into multiple groups in a grouping order. Each group corresponds to a single doorbell transaction in the multiple submitted doorbell transactions. The controller is configured to send one or more completed doorbell transactions to the host device. Each completed doorbell transaction identifies a completed group in the multiple groups. A completed group indicates the completion of a subset of one or more commands corresponding to a completed group. Each completed doorbell transaction identifies a completed group in a second command order identical to the first command order. The controller is also configured to execute the multiple commands in a first operational order different from the first overall order. The controller includes a host interface module (HIM). The HIM includes a submission database configured to store the multiple commands and completion sorting logic communicating with the submission database. The controller is also configured to store multiple packets in a submission database. Completion sorting logic is configured to monitor the multiple packets in the submission database and schedule the transmission of one or more completed doorbell transactions to the host device when at least one of the multiple packets is completed. The controller is also configured to execute multiple commands in a second operational order identical to the first overall order. The completion sorting logic is configured to schedule the transmission of one or more completed doorbell transactions to the host device when each of the multiple packets is completed. The completion sorting logic is configured to schedule the transmission of one or more completed doorbell transactions in completion packet order when each of the multiple packets is completed. The completion packet order is identical to the packet order. The HIM includes command packet logic configured to identify a single doorbell transaction receiving each command for each of the multiple commands. The controller includes a processor, a flash interface module (FIM) communicating with one or more memory devices, and a command scheduler communicating with the FIM to execute multiple commands via the FIM. The multiple commands include one or more read or write operations.
[0095] In one embodiment, the data storage device includes one or more memory devices and a controller coupled to the one or more memory devices. The controller is configured to receive multiple submitted doorbell transactions from a host device, each containing a plurality of commands in a first overall order. Each submitted doorbell transaction identifies a subset of one or more commands in the first command order. The controller is configured to group the commands of the multiple submitted doorbell transactions into multiple groups in a grouping order. Each group corresponds to a single doorbell transaction in the multiple submitted doorbell transactions. The controller is configured to receive a second plurality of submitted doorbell transactions and scan the second plurality of submitted doorbell transactions to obtain an abort command. The controller is configured to execute multiple commands after scanning the second plurality of submitted doorbell transactions. The controller is configured to send one or more completed doorbell transactions to the host device. Each completed doorbell transaction identifies a completed group in the multiple groups. A completed group indicates the completion of a subset of one or more commands corresponding to a completed group. An abort command corresponds to a command in the multiple commands of the multiple submitted doorbell transactions. An abort command includes an instruction to abort the command. An abort command includes a unique identifier identifying the command. The controller is also configured to remove a stop command from one or more completed doorbell transactions if a stop command is detected. These multiple commands are executed in a first operational order different from the first overall order. A second set of multiple submitted doorbell transactions includes a second set of commands, and the controller includes a Host Interface Module (HIM). The HIM includes a submission database configured to store the multiple commands and the second set of commands, and completion ordering logic communicating with the submission database. The controller includes a command scheduler communicating with the HIM. If a stop command is stored in the submission database, the command scheduler does not schedule the command. The completion ordering logic is configured to monitor multiple groups in the submission database and remove a stop command from one or more completed doorbell transactions if it is stored in the submission database.
[0096] In one embodiment, the data storage device includes one or more memory storage devices and a controller coupled to the one or more memory storage devices. The controller is configured to receive multiple submit doorbell transactions having multiple commands from a host device. Each submit doorbell transaction identifies a subset of one or more commands in a first command order. The controller is configured to group the multiple commands of the multiple submit doorbell transactions into multiple groups having a grouping order. Each group corresponds to a single doorbell transaction in the multiple submit doorbell transactions. The controller is configured to send one or more complete doorbell transactions to the host device. Each complete doorbell transaction identifies a completed group in the multiple groups. A completed group indicates the completion of a subset of one or more commands corresponding to a completed group.
[0097] While the foregoing describes embodiments of this disclosure, other and additional embodiments of this disclosure may be contemplated without departing from the basic scope of this disclosure, the scope of which is defined by the appended claims.
Claims
1. A data storage device, comprising: One or more memory devices; and A controller, coupled to the one or more memory devices, wherein the controller is configured to: Multiple submitted doorbell transactions are received from a host device, comprising multiple commands in a first overall order, each of the multiple submitted doorbell transactions identifying a subset of one or more commands in the first command order, wherein the multiple submitted doorbell transactions originate from a first submission queue. The multiple commands of the multiple doorbell submission transactions are grouped into multiple groups with a grouping order, and each of the multiple groups corresponds to a single doorbell transaction in the multiple doorbell submission transactions. Receive a second plurality of submitted doorbell transactions, wherein the second plurality of submitted doorbell transactions are set in a second submission queue, wherein the second submission queue is different from the first submission queue. Scan the second plurality of committed doorbell transactions to obtain an abort command, wherein the abort command is in the second committed queue, and One or more complete doorbell transactions are sent to the host device, each of the one or more complete doorbell transactions identifying a completed group among the plurality of groups, the completed group indicating the completion of the subset of one or more commands corresponding to the completed group, wherein the subset of one or more commands includes a plurality of commands, wherein the plurality of groups are sent in a complete group order different from the group order.
2. The data storage device of claim 1, wherein each completed doorbell transaction in the one or more completed doorbell transactions identifies the completed group of the second command order which is the same as the first command order.
3. The data storage device of claim 2, wherein the controller is further configured to execute the plurality of commands in a first operation order different from the first overall order.
4. The data storage device according to claim 1, wherein the controller includes a host interface module (HIM), and the HIM includes: A database configured to store the submissions of the multiple commands; and The completion sorting logic communicates with the submitted database.
5. The data storage device of claim 4, wherein the controller is further configured to store the plurality of groups in the submission database.
6. The data storage device according to claim 5, wherein the sorting logic is configured as follows: Monitor the multiple groups in the submission database; and When at least one of the plurality of packets is determined to be complete, the one or more completed doorbell transactions are scheduled to be sent to the host device, wherein for each completed packet, a subset of one or more commands corresponding to the completed packet is sent to the host device in the same interrupt notification.
7. The data storage device of claim 6, wherein the controller is further configured to execute the plurality of commands in a second operational order identical to the first overall order.
8. The data storage device according to claim 7, wherein the sorting logic is configured as follows: Upon determination that each of the plurality of packets is complete, the sending of the one or more completed doorbell transactions is scheduled to the host device.
9. The data storage device according to claim 8, wherein the sorting logic is configured as follows: Upon determining that each of the plurality of packets is complete, the sending of the one or more completed doorbell transactions is scheduled in the order of the completed packets.
10. The data storage device of claim 4, wherein the HIM is further configured to identify, for each of the plurality of commands, the individual doorbell transaction under which each command is received.
11. The data storage device of claim 10, wherein the controller further comprises: processor; The flash memory interface module, or FIM, communicates with the one or more memory devices. and A command scheduler that communicates with the FIM to execute the plurality of commands through the FIM, the plurality of commands including one or more read operations or write operations.
12. A data storage device, comprising: One or more memory devices; and A controller, coupled to the one or more memory devices, wherein the controller is configured to: Multiple submitted doorbell transactions are received from a host device, comprising multiple commands in a first overall order, each of the multiple submitted doorbell transactions identifying a subset of one or more commands in the first command order, wherein the multiple submitted doorbell transactions originate from a first submission queue. The multiple commands for submitting doorbell transactions are grouped into multiple groups with a grouping order, and each group corresponds to a single doorbell transaction in the multiple doorbell submission transactions. Receive a second plurality of submitted doorbell transactions, wherein the second plurality of submitted doorbell transactions are set in a second submission queue, wherein the second submission queue is different from the first submission queue. Scan the second or more submitted doorbell transactions to obtain the abort command. After scanning the second plurality of committed doorbell transactions, the plurality of commands are executed, wherein the abort command is in the second commit queue, and One or more complete doorbell transactions are sent to the host device, each of the one or more complete doorbell transactions identifying a completed group among the plurality of groups, the completed group indicating the completion of a subset of one or more commands corresponding to the completed group, wherein the subset of one or more commands includes a plurality of commands, wherein the plurality of groups are sent in a first complete group order different from the group order.
13. The data storage device of claim 12, wherein the abort command corresponds to one of the plurality of commands submitting doorbell transactions, and the abort command includes an instruction to abort the command.
14. The data storage device of claim 13, wherein the abort command includes a unique identifier that identifies the command.
15. The data storage device of claim 13, wherein the controller is further configured to remove the command from the one or more completed doorbell transactions if the abort command is detected.
16. The data storage device of claim 15, wherein the plurality of commands are executed in a first operation order different from the first overall order.
17. The data storage device of claim 13, wherein the second plurality of submitted doorbell transactions includes a second plurality of commands, and the controller includes a host interface module, i.e., a HIM, the HIM comprising: A database configured to store the plurality of commands and the submissions of the second plurality of commands; and The completion sorting logic communicates with the submitted database.
18. The data storage device according to claim 17, wherein: The controller also includes a command scheduler that communicates with the HIM; If the abort command is stored in the commit database, the command scheduler does not schedule the command; and The completion sorting logic is configured to monitor the plurality of groups in the commit database and remove the command from the one or more complete doorbell transactions if the abort command is stored in the commit database.
19. A data storage device, comprising: One or more memory devices; and A controller, coupled to the one or more memory devices, wherein the controller is configured to: The host device receives multiple submitted doorbell transactions, each of which identifies a subset of one or more commands in a first command sequence, wherein the multiple submitted doorbell transactions originate from a first submission queue. The multiple commands for submitting doorbell transactions are grouped into multiple groups with a grouping order, and each group corresponds to a single doorbell transaction in the multiple doorbell submission transactions. Receive a second plurality of submitted doorbell transactions, wherein the second plurality of submitted doorbell transactions originate from a second submitted queue, wherein the second submitted queue is different from the first submitted queue. Scan the second plurality of committed doorbell transactions to obtain abort commands, wherein the abort commands are in the second committed queue, and One or more complete doorbell transactions are sent to the host device, each of the one or more complete doorbell transactions identifying a completed group among the plurality of groups, the completed group indicating the completion of the subset of one or more commands corresponding to the completed group, wherein the subset of one or more commands includes a plurality of commands, wherein the plurality of groups are sent in a first complete group order different from the group order.
20. The data storage device of claim 19, wherein the first submission queue and the second submission queue are located in the same core.
Citation Information
Patent Citations
Multiple command queues having separate interrupts
US20100262740A1
Managed fetching and execution of commands from submission queues
US20190278523A1
Method and apparatus to abort a command
US9996262B1
