Multi-host bandwidth management taking into account external and internal traffic

Abstract

Without using a bandwidth limiter for bandwidth allocation in the SSD, a dummy virtual function (VF) is used to communicate internal operations. Centralized logic, such as the bandwidth limiter, is incorporated into the device controller. This logic is responsible for controlling bandwidth between hosts. The logic is responsible for not only data transmissions triggered by the host, but also data transmissions triggered by the device in internal operations such as garbage collection. To control traffic triggered by internal operations, a dummy VF is created along with a dummy commit queue. The internal operation is queued in the dummy commit queue, while the bandwidth limiter is responsible for performance rates. Using this approach, bandwidth allocation is balanced between the host and the SSD.

Description

[0001] Cross - Reference to Related Applications

[0002] This application claims the benefit of the entire content of U.S. Non - Provisional Application No. 18 / 363,131, entitled "MULTI - HOST BANDWIDTH MANAGEMENT CONSIDERING EXTERNAL AND INTERNAL TRAFFIC", filed on August 1, 2023, with the United States Patent and Trademark Office, and hereby incorporates it by reference in its entirety for all purposes. Background Art Field of the Invention

[0003] Embodiments of the present disclosure generally relate to improving logic for controlling bandwidth allocation in solid - state drives (SSDs).

[0004] Description of the Related Art

[0005] The Single - Root Input / Output Virtualization (SR - IOV) interface is an extension to the PCI Express (PCIe) specification. SR - IOV allows a device such as a network adapter to separate access to resources among various PCIe hardware functions. These functions consist of a PCIe Physical Function (PF) and one or more PCIe Virtual Functions (VFs).

[0006] The PF is the primary function of the device and advertises the SR - IOV capabilities of the device. The PF is associated with the hypervisor parent partition in a virtualization environment. Each VF is associated with the PF of the device. The VF shares one or more physical resources of the device, such as memory and network ports, with the PF and other VFs on the device. Each VF is associated with a hypervisor child partition in the virtualization environment.

[0007] Each PF and VF is assigned a unique PCI Express Requester ID (RID), which allows the I / O Memory Management Unit (IOMMU) to distinguish between different traffic flows and apply memory and interrupt translations between the PF and VFs. This allows traffic flows to be delivered directly to the appropriate hypervisor parent or child partition. Thus, there is an unprivileged data traffic flow from the PF to the VF without affecting other VFs.

[0008] SR - IOV enables network traffic to bypass the software switch layer of the hypervisor virtualization stack. Since the VFs are assigned to child partitions, network traffic flows directly between the VFs and the child partitions. As a result, the I / O overhead in the software emulation layer is reduced, and network performance almost identical to that in a non - virtualized environment is achieved.

[0009] There is a need in the art to improve bandwidth allocation for VFs in SSDs. Summary of the Invention

[0010] Instead of using a bandwidth limiter for bandwidth allocation in an SSD, a dummy virtual function (VF) is used to transfer internal operations. Centralized logic such as a bandwidth limiter is incorporated into the device controller. This logic is responsible for controlling the bandwidth between hosts. The logic is responsible not only for data transfers triggered by the host, but also for data transfers triggered by the device in internal operations such as garbage collection. To control the traffic triggered by internal operations, a dummy VF is created along with a dummy submission queue. Internal operations are queued in the dummy submission queue, and the bandwidth limiter is responsible for the performance rate. Using this method, bandwidth allocation is balanced between the host and the SSD.

[0011] In one embodiment, a data storage device includes: a memory device; and a controller coupled to the memory device, where the controller is configured to: create a dummy virtual function (VF), where the dummy VF has one or more queues; configure a bandwidth limiter, where the bandwidth limiter is configured to allocate bandwidth to the dummy VF and one or more other VFs; queue data storage device internal operations in one or more queues; and control the rate of all data traffic from the dummy VF and one or more other VFs.

[0012] In another embodiment, a data storage device includes: a memory device; and a controller coupled to the memory device, where the controller is configured to: allocate performance bandwidth to one or more virtual functions (VFs) associated with read and write commands from one or more host devices; allocate performance bandwidth to a dummy VF associated with internal data storage device operations; determine that the data storage device cannot provide 100% performance; and proportionally reduce the performance of one or more VFs and the dummy VF.

[0013] In another embodiment, a data storage device includes: a component for storing data; and a controller coupled to the component for storing data, where the controller is configured to: allocate performance bandwidth to a first virtual function (VF) and a second VF associated with read and write commands from one or more host devices; allocate performance bandwidth to a dummy VF associated with internal data storage device operations; determine that the first VF has a bandwidth guarantee; determine that the data storage device cannot provide 100% performance to the first VF; and reduce the performance of the second VF and the dummy VF. Brief Description of the Drawings

[0014] To understand the above features of the present disclosure in detail, the present disclosure briefly outlined above may be described in more specific terms by reference to the embodiments, some of which are illustrated in the accompanying drawings. It should be noted, however, that the drawings merely illustrate typical embodiments of the present disclosure and should not be considered as limiting the scope of the present disclosure, as the present disclosure may admit other equivalent embodiments.

[0015] Figure 1 FIG. 1 is a schematic block diagram showing a storage system in which a data storage device can be used as a storage device of a host device according to certain embodiments.

[0016] Figure 2 FIG. 2 is a schematic diagram showing a system according to certain embodiments.

[0017] Figure 3 FIG. 3 is a schematic diagram showing a virtualized NVMe device according to certain embodiments.

[0018] Figure 4 FIG. 4 is a schematic diagram showing a bandwidth limiter according to certain embodiments.

[0019] Figure 5 FIG. 5 is a schematic diagram showing a bandwidth limiter considering internal operations according to certain embodiments.

[0020] Figure 6 FIG. 6 is a flowchart showing a method for a bandwidth limiter considering internal operations according to certain embodiments.

[0021] Figure 7 FIG. 7 is a flowchart showing a method for a bandwidth limiter considering internal operations and VF performance according to certain embodiments.

[0022] For ease of understanding, wherever possible, the same reference numerals are used to denote the same elements common to the drawings. It is contemplated that the elements disclosed in one embodiment may be beneficially utilized in other embodiments without specific recitation. DETAILED DESCRIPTION

[0023] In the following, reference is made to embodiments of the present disclosure. However, it should be understood that the present disclosure is not limited to the specifically described embodiments. Instead, any combination of the following features and elements, whether or not relating to different embodiments, is contemplated for implementing and practicing the present disclosure. Further, although embodiments of the present disclosure may achieve advantages over other possible solutions and / or over the prior art, whether a particular advantage is achieved by a given embodiment is not limiting of the present disclosure. Thus, the following aspects, features, embodiments, and advantages are merely illustrative and are not to be considered elements or limitations of the appended claims unless expressly recited therein. Likewise, references to "the present disclosure" should not be construed as a generalization of any inventive subject matter disclosed herein and should not be considered an element or limitation of the appended claims unless expressly recited therein.

[0024] Instead of using a bandwidth limiter for bandwidth allocation in an SSD, a dummy virtual function (VF) is used to transfer internal operations. A centralized logic such as a bandwidth limiter is incorporated into the device controller. This logic is responsible for controlling the bandwidth between hosts. The logic is responsible not only for data transfers triggered by the host but also for data transfers triggered by the device in internal operations such as garbage collection. To control the traffic triggered by internal operations, a dummy VF is created along with a dummy submission queue. Internal operations are queued in the dummy submission queue, and the bandwidth limiter is responsible for the performance rate. Using this method, the bandwidth allocation is balanced between the host and the SSD.

[0025] Figure 1 FIG. 1 is a schematic block diagram of a storage system 100 having a data storage device 106 that can serve as a host device 104 according to some embodiments. For example, the host device 104 can utilize a non-volatile memory (NVM) 110 included in the data storage device 106 to store and retrieve data. The host device 104 includes a host dynamic random access memory (DRAM) 138. In some examples, the storage system 100 can include multiple storage devices that can operate as a storage array, such as the data storage device 106. For example, the storage system 100 can include multiple data storage devices 106 configured as a redundant array of inexpensive / independent disks (RAID) that together serve as a mass storage device for the host device 104.

[0026] The host device 104 can store data to and / or retrieve data from one or more storage devices such as the data storage device 106. As Figure 1As shown, the host device 104 can communicate with the data storage device 106 via the interface 114. The host device 104 can include any of a wide range of devices, including: computer servers, network-attached storage (NAS) units, desktop computers, notebooks (i.e., laptops) computers, tablets, set-top boxes, telephone handsets (such as so-called "smart" phones, so-called "smart" tablets), televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, or other devices capable of sending or receiving data from the data storage device.

[0027] The host DRAM 138 can optionally include a host memory buffer (HMB) 150. The HMB 150 is a portion of the host DRAM 138 that is allocated to the data storage device 106 for exclusive use by the controller 108 of the data storage device 106. For example, the controller 108 can store mapping data, buffered commands, logical-to-physical (L2P) tables, metadata, etc. in the HMB 150. In other words, the HMB 150 can be used by the controller 108 to store data that would typically be stored in volatile memory 112, buffer 116, the internal memory of the controller 108 such as static random access memory (SRAM), etc. In an example where the data storage device 106 does not include DRAM (i.e., optional DRAM 118), the controller 108 can utilize the HMB 150 as the DRAM of the data storage device 106.

[0028] The data storage device 106 includes a controller 108, NVM 110, power supply 111, volatile memory 112, interface 114, write buffer 116, and optional DRAM 118. In some examples, the data storage device 106 can include additional components, which are not shown for clarity. Figure 1 For example, the data storage device 106 can include a printed circuit board (PCB) to which the components of the data storage device 106 are mechanically attached, and the printed circuit board includes conductive traces that electrically interconnect the components of the data storage device 106, etc. In some examples, the physical size and connector configuration of the data storage device 106 can conform to one or more standard form factors. Some example standard form factors include, but are not limited to, 3.5-inch data storage devices (e.g., HDD or SSD), 2.5-inch data storage devices, 1.8-inch data storage devices, peripheral component interconnect (PCI), extended PCI (PCI-X), express PCI (PCIe) (e.g., PCIe x1, x4, x8, x16, PCIe mini card, mini PCI, etc.). In some examples, the data storage device 106 can be directly coupled (e.g., directly soldered or inserted into a connector) to the motherboard of the host device 104.

[0029] Interface 114 may include one or both of a data bus for exchanging data with host device 104 and a control bus for exchanging commands with host device 104. Interface 114 may operate according to any suitable protocol. For example, interface 114 may operate according to one or more of the following protocols: Advanced Technology Attachment (ATA) (e.g., Serial ATA (SATA) and Parallel ATA (PATA)), Fibre Channel Protocol (FCP), Small Computer System Interface (SCSI), Serial Attached SCSI (SAS), PCI and PCIe, Non-Volatile Memory Express (NVMe), OpenCAPI, GenZ, Cache Coherent Interface eXtensions (CCIX), Open Channel Solid State Drive (OCSSD), etc. Interface 114 (e.g., the data bus, the control bus, or both) is electrically connected to controller 108, providing an electrical connection between host device 104 and controller 108, enabling data to be exchanged between host device 104 and controller 108. In some examples, the electrical connection of interface 114 may also allow data storage device 106 to receive power from host device 104. For example, as Figure 1 shown, power supply 111 may receive power from host device 104 via interface 114.

[0030] NVM 110 may include a plurality of memory devices or memory cells. NVM 110 may be configured to store and / or retrieve data. For example, the memory cells of NVM 110 may receive data and a message indicating that the memory cell stores the data from controller 108. Similarly, the memory cell may receive a message from controller 108 indicating that the memory cell retrieves data. In some examples, each of the memory cells may be referred to as a die. In some examples, NVM 110 may include a plurality of dies (i.e., a plurality of memory cells). In some examples, each memory cell may be configured to store a relatively large amount of data (e.g., 128 MB, 256 MB, 512 MB, 1 GB, 2 GB, 4 GB, 8 GB, 16 GB, 32 GB, 64 GB, 128 GB, 256 GB, 512 GB, 1 TB, etc.).

[0031] In some examples, each memory cell may include any type of non-volatile memory device such as: flash memory device, phase change memory (PCM) device, resistive random access memory (ReRAM) device, magnetoresistive random access memory (MRAM) device, ferroelectric random access memory (F-RAM), holographic memory device, and any other type of non-volatile memory device.

[0032] The NVM 110 may include multiple flash memory devices or memory cells. The NVM flash memory devices may include NAND or NOR-based flash memory devices, and may store data based on the charge contained in the floating gate of the transistor of each flash memory cell. In the NVM flash memory devices, the flash memory devices may be divided into multiple dies, where each die of the multiple dies includes multiple physical blocks or logical blocks, and the multiple physical blocks or logical blocks may be further divided into multiple pages. Each of the multiple blocks within a particular memory device may include multiple NVM cells. The rows of the NVM cells may be electrically connected using word lines to define the pages within the multiple pages. The corresponding cells within each page of the multiple pages may be electrically connected to corresponding bit lines. Additionally, the NVM flash memory devices may be 2D or 3D devices, and may be single-level cells (SLC), multi-level cells (MLC), triple-level cells (TLC), or quad-level cells (QLC). The controller 108 may write data to and read data from the NVM flash memory devices at the page level, and erase data from the NVM flash memory devices at the block level.

[0033] The power supply 111 may supply power to one or more components of the data storage device 106. When operating in the standard mode, the power supply 111 may use the power provided by an external device such as the host device 104 to supply power to one or a component. For example, the power supply 111 may use the power received from the host device 104 via the interface 114 to supply power to one or more components. In some examples, the power supply 111 may include one or more power storage components configured to supply power to one or more components when operating in the off mode, such as in the case of stopping receiving power from an external device. In this way, the power supply 111 may be used as an on-vehicle backup power source. Some examples of the one or more power storage components include, but are not limited to, capacitors, supercapacitors, batteries, etc. In some examples, the amount of electric power that may be stored by the one or more power storage components may be a function of the cost and / or size (e.g., area / volume) of the one or more power storage components. In other words, as the amount of electric power stored by the one or more power storage components increases, the cost and / or size of the one or more power storage components also increases.

[0034] The controller 108 may use the volatile memory 112 to store information. The volatile memory 112 may include one or more volatile memory devices. In some examples, the controller 108 may use the volatile memory 112 as a cache. For example, before the information in the cache is written to the NVM 110, the controller 108 may store the information in the cache in the volatile memory 112. As Figure 1As shown, the volatile memory 112 can consume the power received from the power supply 111. Examples of the volatile memory 112 include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static RAM (SRAM), and synchronous dynamic RAM (SDRAM (e.g., DDR1, DDR2, DDR3, DDR3L, LPDDR3, DDR4, LPDDR4, etc.)). Similarly, the optional DRAM 118 can be used to store mapping data, buffered commands, logical-to-physical (L2P) tables, metadata, cached data, etc. in the optional DRAM 118. In some examples, the data storage device 106 does not include the optional DRAM 118, such that the data storage device 106 is DRAM-less. In other examples, the data storage device 106 includes the optional DRAM 118.

[0035] The controller 108 can manage one or more operations of the data storage device 106. For example, the controller 108 can manage reading data from and / or writing data to the NVM 110. In some embodiments, when the data storage device 106 receives a write command from the host device 104, the controller 108 can initiate a data storage command to store the data into the NVM 110 and monitor the progress of the data storage command. The controller 108 can determine at least one operating characteristic of the storage system 100 and store the at least one operating characteristic in the NVM 110. In some embodiments, when the data storage device 106 receives a write command from the host device 104, the controller 108 temporarily stores the data in an internal memory or a write buffer 116 before sending the data associated with the write command to the NVM 110.

[0036] The controller 108 can include an optional second volatile memory 120. The optional second volatile memory 120 can be similar to the volatile memory 112. For example, the optional second volatile memory 120 can be SRAM. The controller 108 can allocate a portion of the optional second volatile memory to the host device 104 as a controller memory buffer (CMB) 122. The CMB 122 can be directly accessed by the host device 104. For example, the host device 104 can utilize the CMB 122 to store one or more submission queues that are typically maintained in the host device 104, rather than maintaining the one or more submission queues in the host device 104. In other words, the host device 104 can generate commands and store the generated commands with or without associated data in the CMB 122, where the controller 108 accesses the CMB 122 to retrieve the stored generated commands and / or associated data.

[0037] Figure 2FIG. 0 is a schematic diagram showing a system 200 according to certain embodiments. The system 200 includes a host server. In the host server, there is a hypervisor and multiple virtual machines (VMs). The hypervisor is a manager and is connected to VF0 (parent) in the SSD. All other VFs (VF1 - VF63) are connected to corresponding VMs (VM1 - VM63) in the host server. The other VFs (VF1 - VF63) are also considered as children.

[0038] NVMe virtualization provisions allow up to 100% of the SSD capacity and performance across VFs. Each VF will have dedicated resources assigned, and the hypervisor can only allocate spare capacity or performance that has not been allocated to other VFs. There are several types of resource types, such as but not limited to capacity resources, queue resources, and performance resources. Capacity resources can be namespace IDs. Queue resources can be NVMe queues and interrupt vectors.

[0039] Figure 3 FIG. 7 is a schematic diagram showing a virtualized NVMe device 300 according to certain embodiments. The VFs are shown with associated NVMe queues and namespaces. Each VF has performance resources. These numbers represent the committed performance numbers advertised to the associated clients.

[0040] VF0 (parent) only has a management queue and no I / O queues because there is no data transfer as part of VF0, but only management. For the other VFs (VF1 and VF2), there are other resource allocations. One of them is a management queue and several other I / O queues. There are namespaces assigned to VF1 and VF2. VF1 and VF2 each have a bandwidth allocation. The bandwidth is allocated for sequential reads and writes along with random reads and writes. The virtual NVMe device 300 is responsible for the read requirements of the bandwidth for both VF1 and VF2. There are several internal operations running through the NVMe device 300, and these internal operations will consume some of the allocated bandwidth. Since some bandwidth is obtained from internal operations such as garbage collection, bandwidth balancing is required.

[0041] In previous methods, a bandwidth limiter block was implemented. The logic can be implemented in firmware (FW) or hardware (HW) and is optimized for NVMe virtualization. The bandwidth limiter is responsible for quality of service (QoS) among various NVMe users. NVMe users can be physical submission queues, VFs, supported namespace IDs, or NVMe command streams. QoS is achieved by controlling the submission queue (SQ) extraction operation. An internal database is implemented to track the current bandwidth of each bandwidth limiter vector.

[0042] Figure 4is a schematic diagram illustrating a bandwidth limiter 400 according to some embodiments. In the initialization phase, each user (VF) is initialized with the maximum bandwidth allocated to the user. The algorithm ensures that each user does not exceed the pre-allocated bandwidth by controlling the SQ extraction operation.

[0043] Whenever a command is received from the host, the command parameters are checked. Figure 1 A controller such as the controller 108 of the UE determines the amount of bandwidth required to complete the command. The associated bandwidth limiter vector and the current bandwidth limiter counter are decremented based on the size of the command. When the low threshold of the counter is crossed, the associated submission queue ID is disabled for the next extraction operation until the bandwidth allows the extraction to continue.

[0044] On a periodic basis, the logic scans the bandwidth limiter set (FW registers and current BW count) and allocates bandwidth to each of them. When the high threshold of the counter is crossed, all the previous SQIDs that were disabled by that particular vector will now be enabled.

[0045] The main disadvantage of this approach is that the bandwidth requirements are managed only for the host interface and are only met when there are no internal operations (such as garbage collection). When these operations are added, noise is added to the logic. Without considering the noise, the device will not be able to control the noise. As will be discussed below, internal SSD operations affect bandwidth and bandwidth management should be considered.

[0046] As discussed herein, a centralized logic or bandwidth limiter incorporated into the device controller is beneficial. This logic is responsible for controlling the bandwidth toward the memory device interface. The logic is responsible not only for data transfers triggered by the host device, but also for data transfers triggered by the device in SSD internal operations such as garbage collection. In order to control the traffic triggered by internal SSD operations, a dummy PF / VF is created internally along with several SQs. The internal operations are queued in the dummy SQ, and the bandwidth limiter is responsible for the consumption rate. Under this approach, QoS is improved while taking into account internal SSD operations.

[0047] Figure 5 is a schematic diagram showing an SSD 500 with a bandwidth limiter that takes internal operations into account according to certain embodiments. All internal operations that require some bandwidth from a memory device (e.g., NAND) go through the bandwidth limiter. A dummy VF is created along with a dummy SQ. These requests are queued in the SQ. The bandwidth limiter also takes internal operations into account to control the bandwidth on the drive. By using the same technique, the bandwidth of the VF can be balanced when the dummy VF emulates the host for all internal operations.

[0048] The SSD 500 includes a bandwidth limiter that takes into account the bandwidth limitations of all internal operations. There is also a bandwidth monitor that monitors the bandwidth of both Host A and Host B. It should be understood that although this example only shows two external hosts, more external hosts are conceivable. The bandwidth monitor then sends feedback to the bandwidth limiter. The SSD 500 also includes a dummy host, NVMe arbitration logic, and a bandwidth limiter database. Both the NVMe arbitration logic and the bandwidth limiter database receive information from the bandwidth limiter. The NVMe arbitration logic includes SQ extraction enabling. The SQ extraction enabling controls the queue and determines when to extract new commands and when to stop extracting new commands from the device.

[0049] The dummy host is a newly added addition in conjunction with the use of the bandwidth limiter. The dummy host is not visible to the external hosts (Host A and Host B). The external hosts are unaware of the existence of the dummy host. Although the added host is a dummy host, the bandwidth limiter database will operate as if the dummy host were a regular external host. For example, if the system has 48 hosts, the dummy host will be treated as the 49th host, and the logic of the bandwidth limiter database will operate as if all 49 hosts were external hosts. This will allow all internal operations to run through the dummy host. Additionally, a dummy SQ will be created, and all internal operations will be queued in the same way as if the dummy host were an external host. The bandwidth limiter and the bandwidth monitor will continue this logic with the dummy SQ. During the initialization phase, the FW will be defined, and the bandwidth allocated to the dummy host will also be defined. The benefit of this method is to use the bandwidth limiter while being able to control the bandwidth of all internal operations without restricting the bandwidth from external hosts.

[0050] Each of the external host and the dummy host can be configured in one of two modes. The first mode is the base isolation mode. The base isolation mode is configured as if the SSD 500 cannot provide 100% performance for any reason. In the base isolation mode, the VF (sub) will have its corresponding write performance decreased proportionally. The read performance of the VF (sub) may also be decreased proportionally. For example, instead of all hosts having 100% performance, all hosts only have 95% performance. In other words, in the base isolation mode, when the performance cannot be 100%, all VFs are affected. In one embodiment, all VFs are affected by the exact same amount. For example, the total bandwidth is reduced by 5%, and all VFs are reduced by the exact same amount to achieve a 5% bandwidth reduction. In this case, some VFs will have a reduction greater than 5%, while other VFs will have a reduction less than 5%, resulting in a total 5% reduction. In another embodiment, all VFs are affected proportionally based on the amount of allocated bandwidth. For example, for a 5% reduction, each VF is reduced by 5%.

[0051] The second mode is the full isolation mode. The full isolation mode is configured such that if the SSD 500 cannot provide 100% performance for any reason, the VFs (subs) in the full isolation mode will be prioritized so that they can continue to receive their allocated performance. The prioritized VFs (subs) will be determined based on the highest demand during the operation of the SSD 500. For example, if host A has the highest priority and is allocated 1 GB / s, then all VFs (subs) associated with host A will continue to use the allocated 1 GB / s bandwidth. In other words, if a VF has a guaranteed bandwidth, when the overall performance degrades, a VF must maintain the bandwidth guarantee, and thus, all other VFs are reduced. The reduction of the remaining VFs may follow the base isolation mode distribution discussed above.

[0052] Figure 6 is a flowchart showing a method 600 for a bandwidth limiter that takes into account bandwidth limitations for internal operations. Bandwidth allocation can be changed dynamically. When extensive background operations are needed, the bandwidth allocation to the dummy VF is configured to serve those operations while only starving other regular VFs representing host traffic for a short period of time.

[0053] Method 600 begins at block 602. At block 602, initialization begins in an SSD (such as Figure 5 the SSD 500). At block 604, a controller such as Figure 1 controller 108 creates a dummy VF and a dummy SQ. At block 606, a bandwidth limiter (such as Figure 5 the bandwidth limiter) is configured to an operation mode. The operation mode can be one of the base isolation mode or the full isolation mode as described above. At block 608, all internal operations are queued in the dummy SQ. At block 610, the bandwidth limiter controls the rate of all traffic including internal operations. At block 612, the controller determines whether recalibration is needed. If the controller determines that recalibration is needed, method 600 returns to block 606. If the controller determines that calibration is not needed, method 600 returns to block 608.

[0054] Figure 7 is a flowchart showing a method 700 for a bandwidth limiter that takes into account internal operations and VF performance. VF performance determines the amount of allocated bandwidth received by each VF.

[0055] Method 700 begins at block 702. At block 702, a controller (such as Figure 1The controller 108) creates dummy VFs and one or more dummy SQs. At block 704, bandwidth is allocated to the dummy VFs and other VFs. At block 706, the controller determines that 100% is not possible for at least one of the VFs. At block 708, the controller determines whether a VF has a performance guarantee. If the controller determines that a VF has a performance guarantee, method 700 proceeds to block 710. If the controller determines that a VF does not have a performance guarantee, method 700 proceeds to block 712. At block 710, the bandwidth of the VF is increased while the bandwidth of all other remaining VFs is decreased. When block 710 is complete, method 700 proceeds to block 722.

[0056] At block 712, the controller determines whether the VF is a dummy VF. If the controller determines that the VF is not a dummy VF, method 700 proceeds to block 714. If the controller determines that the VF is a dummy VF, method 700 proceeds to block 716. At block 714, the performance of all VFs will be reduced proportionally. At block 716, the SSD determines whether the background operation is extensive. If the controller determines that the background operation is not extensive, method 700 proceeds to block 718. If the controller determines that the background operation is extensive, method 700 proceeds to block 720. At block 718, there will be no bandwidth change. When block 718 is complete, method 700 proceeds to block 722. At block 720, the performance of all VFs except the dummy VF will be reduced. When block 720 is complete, method 700 proceeds to block 722. At block 772, the controller re - evaluates the bandwidth allocation for all VFs.

[0057] The main advantage of the present disclosure can be measured in better QoS results, which is achieved because the logic treats internal operations as part of bandwidth management.

[0058] In one embodiment, a data storage device includes: a memory device; and a controller coupled to the memory device, wherein the controller is configured to: create a dummy virtual function (VF), wherein the dummy VF has one or more queues; configure a bandwidth limiter, wherein the bandwidth limiter is configured to allocate bandwidth to the dummy VF and one or more other VFs; queue data storage device internal operations in one or more queues; and control the rate of all data traffic from the dummy VF and one or more other VFs. The dummy VF is configured to control internal data storage device operations. The controller is configured to dynamically adjust the bandwidth allocation. The controller is further configured to determine whether recalibration of the allocated bandwidth should occur. The controller is configured to determine that the data storage device cannot provide 100% performance. The controller is configured to proportionally reduce the performance of one or more VFs and the dummy VF in response to the determination. The controller is configured to determine that a first VF among the one or more VFs has a bandwidth guarantee. The controller is configured to reduce the performance of a second VF among the one or more VFs while maintaining the performance of the first VF in response to the determination. The controller is configured to reduce the performance of one or more VFs while increasing the performance of the dummy VF. The reduction is in response to determining that a higher than threshold amount of background operations are needed. The reduction continues for a predetermined period of time.

[0059] In another embodiment, a data storage device includes: a memory device; and a controller coupled to the memory device, wherein the controller is configured to: allocate performance bandwidth to one or more virtual functions (VFs) associated with read and write commands from one or more host devices; allocate performance bandwidth to a dummy VF associated with internal data storage device operations; determine that the data storage device cannot provide 100% performance; and proportionally reduce the performance of one or more VFs and the dummy VF. The dummy VF is invisible to the host device. The dummy VF includes a plurality of submission queues. The controller is configured to queue internal data storage device operation requests in the plurality of submission queues.

[0060] In another embodiment, a data storage device includes: a component for storing data; and a controller coupled to the component for storing data, wherein the controller is configured to: allocate performance bandwidth to a first virtual function (VF) and a second VF associated with read and write commands from one or more host devices; allocate performance bandwidth to a dummy VF associated with internal data storage device operations; determine that the first VF has a bandwidth guarantee; determine that the data storage device cannot provide 100% performance to the first VF; and reduce the performance of the second VF and the dummy VF. Reducing the performance includes reducing the bandwidth of the second VF and the dummy VF. The reduction of the bandwidth is proportional. The controller is configured to increase the bandwidth of the first VF in response to the reduction. The controller is further configured to recalibrate the allocated bandwidth.

[0061] While the foregoing is directed to embodiments of the present disclosure, other and additional embodiments of the present disclosure can be devised without departing from the basic scope thereof, and the scope of the present disclosure is determined by the appended claims.

Claims

1. A data storage device, the data storage device comprising: A memory device; And A controller, the controller being coupled to the memory device, wherein the controller is configured to: Create a dummy virtual function (VF), wherein the dummy VF has one or more queues; Configure a bandwidth limiter, wherein the bandwidth limiter is configured to allocate bandwidth to the dummy VF and one or more other VFs; Queue data storage device internal operations in the one or more queues; And Control the rate of all data traffic from the dummy VF and the one or more other VFs.

2. The data storage device according to claim 1, wherein the dummy VF is configured to control internal data storage device operations.

3. The data storage device according to claim 1, wherein the controller is configured to dynamically adjust bandwidth allocation.

4. The data storage device according to claim 1, wherein the controller is further configured to determine whether recalibration of the allocated bandwidth should occur.

5. The data storage device according to claim 1, wherein the controller is configured to determine that the data storage device cannot provide 100% performance.

6. The data storage device according to claim 5, wherein the controller is configured to proportionally reduce the performance of one or more VFs and the dummy VF in response to the determination.

7. The data storage device according to claim 5, wherein the controller is configured to determine that a first VF among the one or more VFs has a bandwidth guarantee.

8. The data storage device according to claim 7, wherein the controller is configured to reduce the performance of a second VF among the one or more VFs while maintaining the performance of the first VF in response to the determination.

9. The data storage device according to claim 1, wherein the controller is configured to reduce the performance of the one or more VFs while increasing the performance of the dummy VF.

10. The data storage device according to claim 9, wherein the reduction is in response to determining that a higher than threshold amount of background operations is required.

11. The data storage device according to claim 10, wherein the reduction persists for a predetermined period of time.

12. A data storage device, the data storage device comprising: A memory device; And A controller, the controller being coupled to the memory device, wherein the controller is configured to: Allocate performance bandwidth to one or more virtual functions (VFs) associated with read and write commands from one or more host devices; Allocate performance bandwidth to a dummy VF associated with internal data storage device operations; Determine that the data storage device cannot provide 100% performance; and Proportionally reduce the performance of the one or more VFs and the dummy VF.

13. The data storage device according to claim 12, wherein the dummy VF is invisible to the host device.

14. The data storage device according to claim 12, wherein the dummy VF includes a plurality of submission queues.

15. The data storage device according to claim 14, wherein the controller is configured to queue internal data storage device operation requests in the plurality of submission queues.

16. A data storage device, the data storage device comprising: a component for storing data; and a controller coupled to the component for storing data, wherein the controller is configured to: allocate performance bandwidth to a first virtual function (VF) and a second VF associated with read and write commands from one or more host devices; allocate performance bandwidth to a dummy VF associated with internal data storage device operations; determine that the first VF has a bandwidth guarantee; determine that the data storage device cannot provide 100% performance to the first VF; and reduce the performance of the second VF and the dummy VF.

17. The data storage device according to claim 16, wherein reducing performance includes reducing the bandwidth of the second VF and the dummy VF.

18. The data storage device according to claim 17, wherein the reduction in bandwidth is proportional.

19. The data storage device according to claim 16, wherein the controller is configured to increase the bandwidth of the first VF in response to the reduction.

20. The data storage device according to claim 16, wherein the controller is further configured to re - calibrate the allocated bandwidth.

Images