BALANCED TRANSMITTS AT INTERFACES

Abstract

Indiscriminately sending transactions across an interface can lead to unused interfaces and generally inefficient operations. Reordering transactions or data packets may change the order in which packets are transmitted to host devices. Reordering ensures that each interface between a host device and the data storage device is fully utilized. Full utilization is achieved by reordering in a way that balances packet transmission across interfaces. This balancing can be based on any number of factors, such as namespace identification (namespace ID), zone ID in the case of a zoned namespace drive (ZNS drive), transmit and execute IDs, physical and virtual functions, and host addresses, to name a few.Reordering and thus balancing packets achieves fairness at the packet level and enables the integration of asymmetric systems. This optimizes performance, maintains reliability, and supports scalability.

Description

BACKGROUND OF REVELATION Area of ​​Revelation

[0001] Embodiments of the present disclosure generally relate to the improvement of traffic balancing between interfaces. Description of the state of the art

[0002] Non-Volatile Memory (NVM) express solid-state drives (NVMe SSDs) are connected to host devices via a Peripheral Component Interconnect (PCI) express interface (PCIe interface). This interface is used to comply with the NVMe protocol while maximizing performance. To process host commands, NVMe must utilize the interface for various tasks: reading commands, reading pointers, reading data, and, in some products, reading mapping tables.

[0003] Traffic balancing in the context of storage devices using PCIe and NVMe interfaces optimizes performance, improves efficiency, and ensures fair resource allocation. Key reasons for the importance of traffic balancing include optimizing throughput, avoiding bottlenecks, efficiently utilizing multiple lanes, improving scalability, enhancing reliability, reducing latency, maximizing NVMe parallelism, supporting dynamic workloads, and meeting application requirements.

[0004] In summary, traffic balancing in PCIe and NVMe-based storage systems optimizes performance, maintains reliability, and supports scalability. Traffic balancing ensures that the full capabilities of high-speed interfaces are efficiently utilized, contributing to an overall responsive and reliable storage infrastructure. However, achieving traffic balancing is always a challenge.

[0005] Therefore, there is a general need for improved traffic balancing between interfaces. SUMMARY OF THE REVELATION

[0006] Indiscriminately sending transactions across an interface can lead to unused interfaces and generally inefficient operations. Reordering transactions or data packets may change the order in which packets are transmitted to host devices. Reordering ensures that each interface between a host device and the data storage device is fully utilized. Full utilization is achieved by reordering in a way that balances packet transmission across interfaces. This balancing can be based on any number of factors, such as namespace identification (namespace ID), zone ID in the case of a zoned namespace drive (ZNS drive), transmit and execute IDs, physical and virtual functions, and host addresses, to name a few.Reordering and thus balancing packets achieves fairness at the packet level and enables the integration of asymmetric systems. This optimizes performance, maintains reliability, and supports scalability.

[0007] In one embodiment, a data storage device comprises: a storage device; and a controller coupled to the storage device, the controller being configured to: classify transactions to be sent to one or more host devices, the transactions being performed at the packet level; reorder the transactions based on the classifications and transmit the reordered transactions to the one or more host devices.

[0008] In another embodiment, a data storage device comprises: a storage device; and a controller coupled to the storage device, the controller comprising: a host interface module (HIM) comprising a reorder buffer and an execution and interrupt synchronization module, wherein the HIM is configured to support a first interface between the controller and a first host device, and wherein the HIM is configured to support a second interface between the controller and a second host device; a flash interface module (FIM) coupled to the storage device;and a command scheduler coupled between the HIM and the FIM, the controller being configured to: balance traffic between the first host device and the second host device, the balancing including ensuring that the first interface and the second interface are fully utilized.

[0009] In another embodiment, a data storage device comprises: means for storing data; and a controller coupled to the means for storing data, wherein the controller is configured to: receive a transaction packet from the means for storing data; classify the transaction packet; place the transaction packet in one of a plurality of queues; send the transaction packet to a host device; and maintain full utilization at an interface between the controller and the host device, as well as at interfaces between the controller and other host devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to understand in detail the nature of the aforementioned features of the present disclosure, a more precise description of the disclosure, which has been briefly summarized above, can be given with reference to embodiments, some of which are illustrated in the accompanying drawings. It should be noted, however, that the accompanying drawings only illustrate typical embodiments of this disclosure and are therefore not to be considered as limiting in its scope of protection, since the disclosure also permits other, equally effective embodiments. Fig. Figure 1 is a schematic block diagram illustrating a storage system in which a data storage device can function as a storage device for a host device according to certain embodiments. Fig.Figure 2 is a schematic illustration of a Non-Volatile Memory (NVM) express solid-state drive (NVMe SSD system) system according to one embodiment. Fig. Figure 3 is a schematic illustration of a multi-tenant system according to one embodiment. Fig. Figure 4 is a schematic illustration of a data storage system according to one embodiment. Fig. Figure 5 is a schematic illustration of a reordering logic arrangement according to one embodiment. Fig. Figure 6 is a schematic illustration of a system comprising two hosts according to one embodiment. Fig. Figure 7 is a flowchart illustrating a transaction balancing process according to one embodiment.

[0011] To facilitate understanding, identical reference numerals have been used wherever possible to denote identical elements present in all figures. It is intended that elements disclosed in one embodiment may also be used effectively in other embodiments without specific mention. DETAILED DESCRIPTION

[0012] The following refers to embodiments of the disclosure. However, it is understood that the disclosure is not limited to the specific embodiments described. Instead, any combination of the following features and elements, regardless of whether they relate to different embodiments or not, is intended for the implementation and practical application of the disclosure. Furthermore, although embodiments of the disclosure may offer advantages over other possible solutions and / or over the prior art, the fact that a particular embodiment achieves a particular advantage or not does not constitute a limitation of the disclosure.Therefore, the following aspects, features, embodiments, and advantages serve only for illustration and are not considered elements or limitations of the appended claims unless expressly stated in one or more claims. Likewise, a reference to "the disclosure" is not to be construed as a generalization of any inventive subject matter disclosed herein and is not to be considered an element or limitation of the appended claims unless expressly stated in one or more claims.

[0013] Indiscriminately sending transactions across an interface can lead to unused interfaces and generally inefficient operations. Reordering transactions or data packets may change the order in which packets are transmitted to host devices. Reordering ensures that each interface between a host device and the data storage device is fully utilized. Full utilization is achieved by reordering in a way that balances packet transmission across interfaces. This balancing can be based on any number of factors, such as namespace identification (namespace ID), zone ID in the case of a zoned namespace drive (ZNS drive), transmit and execute IDs, physical and virtual functions, and host addresses, to name a few.Reordering and thus balancing packets achieves fairness at the packet level and enables the integration of asymmetric systems. This optimizes performance, maintains reliability, and supports scalability.

[0014] Fig.Figure 1 is a schematic block diagram illustrating a storage system 100 comprising a data storage device 106, which, according to certain embodiments, can function as a storage device for a host device 104. For example, the host device 104 can use non-volatile memory (NVM) 110 enclosed in the data storage device 106 for storing and retrieving data. The host device 104 includes dynamic random-access memory (DRAM) 138. In some examples, the storage system 100 can include a variety of storage devices, such as the data storage device 106, which can function as a storage array.For example, the storage system 100 can include a variety of data storage devices 106 configured as a redundant array of inexpensive / independent disks (RAID) and acting collectively as a mass storage device for the host device 104.

[0015] The host device 104 can store data on and / or retrieve data from one or more storage devices, such as the data storage device 106. As described in Fig.As illustrated in Figure 1, the host device 104 can communicate with the data storage device 106 via an interface 114. The host device 104 can include a wide range of devices, including computer servers, network-attached storage (NAS) units, desktop computers, notebooks (i.e., laptops), tablet computers, set-top boxes, mobile phones such as smartphones or smart tablets, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, or other devices that can send data to or receive data from a data storage device.

[0016] The host DRAM 138 can optionally include a host memory buffer (HMB) 150. The HMB 150 is a section of the host DRAM 138 allocated to the data storage device 106 for the exclusive use of a controller 108 of the data storage device 106. For example, the controller 108 can store mapping data, buffered instructions, logic-physical (L2P) tables, metadata, and the like in the HMB 150. In other words, the HMB 150 can be used by the controller 108 to store data that would normally be stored in volatile memory 112, a buffer 116, internal memory of the controller 108 such as static random-access memory (SRAM), and the like. In examples where the data storage device 106 does not include DRAM (i.e., optional DRAM 118), the controller 108 can use the HMB 150 as the DRAM of the data storage device 106.

[0017] The data storage device 106 includes a controller 108, an NVM 110, a power supply 111, volatile memory 112, an interface 114, a write buffer 116, and an optional DRAM 118. In some examples, the data storage device 106 may include additional components, which for clarity are shown in Fig.Figure 1 is not shown. For example, the data storage device 106 may include a printed circuit board (PCB) to which components of the data storage device 106 are mechanically attached and which includes electrically conductive traces that electrically connect components of the data storage device 106 or the like. In some examples, the physical dimensions and connection configurations of the data storage device 106 may conform to one or more standard form factors. Some examples of standard form factors include, but are not limited to, 3.5-inch data storage devices (e.g., an HDD or SSD), 2.5-inch data storage devices, 1.8-inch data storage devices, Peripheral Component Interconnect (PCI), PCI-Extended (PCI-X), and PCI Express (PCIe) (e.g., PCIe x1, x4, x8, x16, PCIe Mini Card, MiniPCI, etc.).In some examples, the data storage device 106 can be directly coupled to a mainboard of the host device 104 (e.g., directly soldered or plugged into a connector).

[0018] Interface 114 can include a data bus for data exchange with the host device 104 and / or a control bus for exchanging commands with the host device 104. Interface 114 can operate according to any suitable protocol. For example, interface 114 can operate according to one or more of the following protocols: Advanced Technology Attachment (ATA) (e.g., Serial ATA (SATA) and Parallel ATA (PATA)), Fibre Channel Protocol (FCP), Small Computer System Interface (SCSI), Serially Attached SCSI (SAS), PCI and PCIe, Non-Volatile Memory Express (NVMe), OpenCAPI, GenZ, Cache Coherent Interface Accelerator (CCIX), Open Channel SSD (OCSSD), or the like. Interface 114 (e.g.,The data bus, the control bus, or both) is electrically connected to the controller 108 and provides an electrical connection between the host device 104 and the controller 108, allowing data to be exchanged between the host device 104 and the controller 108. In some examples, the electrical connection of the interface 114 can also allow the data storage device 106 to receive power from the host device 104. For example, in... Fig. As illustrated in Figure 1, the power supply 111 can receive power from the host device 104 via the interface 114.

[0019] The NVM 110 can include a variety of storage devices or storage units. The NVM 110 can be configured to store and / or retrieve data. For example, a storage unit of the NVM 110 can receive data and a message from the controller 108 instructing the storage unit to store the data. Likewise, the storage unit can receive a message from the controller 108 instructing the storage unit to retrieve data. In some examples, each of the storage units can be referred to as a chip. In some examples, the NVM 110 can include a variety of chips (i.e., a variety of storage units). In some examples, each storage unit can be configured to store relatively large amounts of data (e.g., 128 MB, 256 MB, 512 MB, 1 GB, 2 GB, 4 GB, 8 GB, 16 GB, 32 GB, 64 GB, 128 GB, 256 GB, 512 GB, 1 TB, etc.).

[0020] In some examples, each storage unit can include any type of non-volatile storage device, such as flash memory devices, phase-change memory (PCM), resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), holographic storage devices, and any other type of non-volatile storage device.

[0021] The NVM 110 can incorporate a variety of flash memory devices or storage units. NVM flash memory devices can include NAND- or NOR-based flash memory devices and can store data based on a charge contained in a floating gate of a transistor for each flash memory cell. In NVM flash memory devices, the flash memory device can be subdivided into a variety of dies, with each die containing a variety of physical or logical blocks, which can be further subdivided into a variety of pages. Each block within a given memory device can contain a variety of NVM cells. Rows of NVM cells can be electrically connected using a word line to define a page within a variety of pages.The individual cells in each of the multitude of pages can be electrically connected to the respective bit lines. Furthermore, NVM flash storage devices can be 2D or 3D devices and can be of the single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), or quad-level cell (QLC) type. The controller 108 can write and read data to and from NVM flash storage devices at the page level and erase data from NVM flash storage devices at the block level.

[0022] The power supply 111 can power one or more components of the data storage device 106. In standard mode, the power supply 111 can power one or more components by drawing power from an external device, such as the host device 104. For example, the power supply 111 can power the one or more components by using power received from the host device 104 via interface 114. In some examples, the power supply 111 can include one or more power storage components configured to power the one or more components when they are in shutdown mode, such as when no power is being received from the external device. In this way, the power supply 111 can function as an integrated backup power source.Examples of one or more energy storage components include capacitors, supercapacitors, batteries, and the like. In some cases, the amount of electricity that can be stored by one or more energy storage components may depend on the cost and / or size (e.g., area / volume) of the one or more energy storage components. In other words, as the amount of electricity stored by one or more energy storage components increases, so do the cost and / or size of the one or more energy storage components.

[0023] The volatile memory 112 can be used by the controller 108 to store information. The volatile memory 112 can include one or more volatile storage devices. In some examples, the controller 108 can use the volatile memory 112 as a cache. For example, the controller 108 can store cached information in the volatile memory 112 until the cached information is written to the NVM 110. As in Fig.As illustrated in Figure 1, the volatile memory 112 can consume power received from the power supply 111. Examples of volatile memory 112 include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static RAM (SRAM), and synchronous dynamic RAM (SDRAM (e.g., DDR1, DDR2, DDR3, DDR3L, LPDDR3, DDR4, LPDDR4, and the like)). Likewise, the optional DRAM 118 can be used to store mapping data, buffered instructions, logic-physical (L2P) tables, metadata, cached data, and the like. In some examples, the data storage device 106 does not include the optional DRAM 118, so the data storage device 106 has no DRAM. In other examples, the data storage device 106 includes the optional DRAM 118.

[0024] The controller 108 can manage one or more operations of the data storage device 106. For example, the controller 108 can manage reading data from and / or writing data to the NVM 110. In some embodiments, when the data storage device 106 receives a write command from the host device 104, the controller 108 can initiate a data storage command to store data in the NVM 110 and monitor the progress of the data storage command. The controller 108 can determine at least one operating property of the storage system 100 and store at least one operating property in the NVM 110. In some embodiments, when the data storage device 106 receives a write command from the host device 104, the controller 108 temporarily stores the data associated with the write command in internal memory or write buffer 116 before sending the data to the NVM 110.The controller 108 can include switching logic or processors configured to run programs to operate the data storage device 106.

[0025] The controller 108 can include an optional second volatile memory 120. The optional second volatile memory 120 can be similar to the volatile memory 112. For example, the optional second volatile memory 120 can be SRAM. The controller 108 can allocate a portion of the optional second volatile memory to the host device 104 as a control memory buffer (CMB) 122. The CMB 122 can be accessed directly by the host device 104. For example, instead of managing one or more transmission queues in the host device 104, the host device 104 can use the CMB 122 to store the one or more transmission queues that are normally managed in the host device 104.In other words, the host device 104 can generate commands and store the generated commands with or without the associated data in the CMB 122, with the controller 108 accessing the CMB 122 to retrieve the stored generated commands and / or the associated data.

[0026] As stated above, traffic balancing offers numerous advantages. For example, to optimize throughput, PCIe and NVMe interfaces are designed to provide high-speed data transfer between the data storage device and the host device. Traffic balancing helps to distribute data traffic evenly across multiple lanes or channels, thereby maximizing the overall throughput of the data storage system.

[0027] Regarding bottleneck prevention, an uneven distribution of data traffic can lead to congestion in certain lanes or channels. Traffic balancing helps avoid overloads and ensures that no single lane becomes a limiting factor, allowing the data storage device to operate at its full potential.

[0028] To efficiently utilize multiple lanes, PCIe-based data storage devices often feature multiple lanes or channels that support parallel data transmission. Traffic balancing ensures that each lane is used efficiently, thus preventing situations where some lanes are underutilized while others are congested.

[0029] Regarding improved scalability, traffic balancing is becoming increasingly important as storage systems scale in terms of capacity and performance. Traffic balancing enables horizontal scaling of the storage infrastructure while simultaneously utilizing multiple PCIe / NVMe devices and maintaining optimal performance.

[0030] Regarding improved reliability, traffic balancing contributes to the reliability of the storage system by preventing uneven wear on different components. A balanced traffic distribution reduces the likelihood of hotspots where certain components might be subjected to excessive stress.

[0031] Regarding latency reduction, minimizing latency is beneficial for responsiveness in data storage systems. Traffic balancing ensures that data is distributed evenly, thus avoiding scenarios where some data paths experience longer latency due to congestion.

[0032] Regarding maximizing NVMe parallelism, NVMe is designed to utilize the inherent parallelism of storage, particularly NAND flash memory. Through appropriate traffic balancing, NVMe queues and parallelism can be used efficiently, ensuring that multiple input / output (I / O) operations can be processed concurrently.

[0033] Regarding support for dynamic workloads, storage workloads can vary dynamically, and traffic balancing allows the data storage system to adapt to changing conditions. Traffic balancing ensures that the data storage device can handle varying amounts of read and write requests without experiencing performance fluctuations.

[0034] Regarding the fulfillment of application requirements, different applications and workloads place different demands on storage performance. Traffic balancing helps to meet the specific requirements of various applications, enabling the storage infrastructure to support a wide range of use cases.

[0035] The present disclosure addresses the problem of achieving packet-level balanced transmissions, which is advantageous for some specific applications. For example, traffic balancing is beneficial in asymmetric multi-host systems when each host device has a different maximum performance.

[0036] Fig. Figure 2 is a schematic illustration of an NVMe SSD system 200 according to one embodiment. In general, the system 200 includes a host device and a data storage device. The host device comprises local DRAM with HMB. The data storage device includes a controller with a PCIe interface. The interface has a buffer for data storage. The controller is responsible, among other things, for retrieving instructions, pointers, data, and tables.

[0037] More specifically, the data storage device uses NVMe over a PCIe interface. This interface is located between the host device and the data storage device, and generally, the NVMe protocol involves several types of transfers across this interface, such as fetching commands or data pointers. This also includes the transfer of user data itself and potentially L2P tables or other tables stored in the HMB. The disclosure discusses achieving balanced transfers across the host interface between command retrieval, pointer retrieval, data and tables, and other transfers to maximize performance.

[0038] If the interface is used solely for retrieving data, a scenario could arise where there are not enough instructions available in the data storage device, and therefore the interface to the storage device (e.g., NAND) is not fully utilized. Since the interface is constantly engaged in data transmission, a balance must be struck, and bottlenecks must be avoided. The goal is to determine how to better balance the various types of transmissions occurring at the interface to maximize performance. Generally, this disclosure is more relevant for multi-host interfaces, with a focus on lower-level packets. However, it is understood that this disclosure is also relevant for a single-host interface, particularly if the single host includes one or more virtual and / or physical functions.

[0039] The disclosure discusses a reordering logic responsible for balancing data traffic across the interface and ensuring that each interface is fully utilized. Balancing could be based on the namespace ID, the zone ID in the case of a CNS drive, transmit and execute IDs, physical and virtual PCIe functions, and / or host addresses. It is understood that the balancing could also be based on other criteria. Special logic is used to synchronize the execute and interrupt messages to the host device. The disclosure enables fairness in packet-level processing. Balancing allows for the integration of asymmetric systems, is beneficial for performance optimization, maintains reliability, and supports scalability. Fig. Section 3 presents the concept of such a system.

[0040] Fig.Figure 3 is a schematic illustration of a multi-tenant system according to one embodiment. A single storage device is connected to multiple host systems via a switch. The storage device is connected to the switch via a single-port interface. The maximum throughput at the interface between the storage device and the switch is greater than the throughput at the interface between a specific host device and the switch. In a balanced system, the interface between the storage device and the switch equals the total throughput at the interfaces between all host devices and the switch. The data storage device must ensure that the interface of each host device is fully utilized.

[0041] In the example of Fig.There are four host devices. A switch is located between the host devices and the data storage device. For example, there could be a scenario where the interface between the storage device and the switch is a fifth-generation interface with four lanes, and the interface between Host A and the switch is something else. The interface could also be a fifth-generation interface with four lanes, or, just as an example, a fifth-generation interface with one lane, or something else entirely. The interface could be configured for any desired speed. When maximizing performance, the different interfaces should be taken into account. Therefore, the data storage device must also ensure that there is sufficient activity or transmission across all interfaces.It is not enough to simply ensure that one interface is fully utilized, as a scenario could occur where one host interface is fully utilized while all other host devices are idle, which is not ideal. For efficient operation, all interfaces should be saturated, not just one or a few.

[0042] Fig. Figure 4 illustrates a high-level block diagram of the data storage system that implements the revelation. Fig. Figure 4 is a schematic illustration of a data storage system according to one embodiment. Fig.Figure 4 shows the reordering logic within the controller's Host Interface Module (HIM). This reordering logic is responsible for classifying transactions and sending them to the host devices in a different order, thus balancing traffic across all host device interfaces. The reordering logic performs this balancing using the reorder buffer, rearranging transmissions to achieve interface balance and maximize performance. Additionally, the execution and interrupt synchronization logic handles internal execution and interrupt messages, holding them internally and sending them to the host devices only after the associated data transfers are complete.

[0043] More precisely, in relation to Fig.4. On the one hand, there are numerous host devices in a multi-host system, and on the other hand, the data storage device with a storage device (e.g., NAND or DRAM) and the device controller, which is responsible for the interaction between the host device and the storage device. The HIM includes reorder logic, which comprises a reorder buffer and an execution and interrupt synchronization module.

[0044] Regarding balancing, there are two main parts to discuss. The first part is the reorder buffer, and the other part is execution and interrupt synchronization, which relates to... Fig.Section 5 will discuss this in more detail. Balancing could be based on several elements, such as the namespace ID, so that the data storage device ensures fairness per namespace. Another possibility is to perform balancing per zone ID in the storage device, per delivery queue, or per execution queue. Balancing could be based on physical or virtual functions. The balance could be based on LBAs. These are merely examples, as other bases for balancing are conceivable. To achieve fairness, balancing must have a basis.

[0045] Fig. Section 5 presents the reorganization logic in detail. Fig.Figure 5 is a schematic illustration of a reorder logic arrangement according to one embodiment. All device control components remain the same. These include encryption and decryption, data path management, direct memory access (DMA), flash interface logic, and so on. The NVMe component sends data transactions to the reorder logic. The reorder logic parses the transactions, classifies them, and places them in the appropriate queues. The round-robin logic is responsible for extracting entries from the queues and sending them to the host devices in a fair manner.

[0046] The classifications could be based on the following parameters: namespace ID, zone ID for a CNS drive, transmission and execution IDs, physical and virtual PCIe functions, and / or addresses of host devices. Other parameters are also taken into account.

[0047] Based on the classification, the data is placed in a specific queue. For example, one queue could be a namespace ID queue, another could be based on a delivery queue. There can be a queue for each transmission type, and the classifier is responsible for classifying the packet type and then placing it in the appropriate queue. Only then is the round-robin algorithm or a similar method used for balancing. Weighting could be applied within the round-robin algorithm.

[0048] The other part of the logic is for execution and includes the execution queue and the interrupt message to the host device. Special care must be taken with these transmissions because the logic reorders them, and therefore it cannot be assumed that a packet has been sent and an execution can be sent to the host device. This is not permissible in this case, because there could be a scenario where the sequencing logic decides to reorder the transmission, and it takes some time for the logic to decide to send the packet to the host device; therefore, the execution message logic should be aware of this.

[0049] The data transfer associated with the command is delayed, and therefore the execution message associated with the command should also be delayed. Otherwise, a race condition could occur, and therefore the logic is responsible for synchronization. The reorder buffer adds a delay before the data is sent to a host device, whereas the NVMe logic assumes the data has already been transferred. Therefore, the NVMe logic may be able to send execution and interrupt messages, but the reorder logic cannot immediately send the messages to the host device. The logic tracks the relevant data, and only when the data transfer associated with a command is complete are the execution / interrupt messages sent to the host device.

[0050] An example will now be described where fairness is based on host device addresses. It should be understood that the host device addresses are merely an example and other criteria can be used to determine fairness. Additionally, the host's memory area is 32 bits, with half allocated to Host A and the other half to Host B. The memory device is responsible for balancing packets between these host devices at packet granularity. Fig. Figure 6 shows the example. Fig. Figure 6 is a schematic illustration of a system comprising two hosts according to one embodiment. In conjunction with Tables I and II, it provides an example of fairness based on the LBA.

[0051] In this example, Host A is responsible for the higher addresses and Host B for the lower addresses. In this example, 1 K needs to be transferred from one address to Host A, and Table I shows the transfer sequence implemented by the internal logic and which the device controller intends to send to the host device. Thus, Table I shows the internal sequence prior to the reordering logic. Table 1 address Size Goal 0xA000 1 KB Host A 0xA400 1 KB Host A 0xA800 1 KB Host A 0xAC00 1 KB Host A 0x8000_0000 1 KB Host B 0x8000_0400 1 KB Host B 0x8000_0800 1 KB Host B 0x8000_0C00 1 KB Host B

[0052] Table I shows the traffic that the NVMe component sends to the reorder buffer. Four addresses are allocated to Host A, followed by four addresses allocated to Host B. In this scenario, all these packets are sent to Host A in the order shown in Table I, and Host A becomes fully utilized while Host B remains idle, which is undesirable. Table I shows that the packet-level transfer is unbalanced, as 4 KB of transactions are sent to Host A before 4 KB of transactions are sent to Host B. Previously, Table I would have represented the traffic that would have been monitored at the host device interface.

[0053] A better approach is to reorder the packets as shown in Table II. Table II shows the balanced traffic visible to the host device after it has passed through the reordering logic. The logic is responsible for this reordering. After reordering, the first packet would go to Host A, then the second packet would go to Host B, and so on. This process ensures that the interfaces are used in a way that maximizes performance. As can be seen, Table II achieves fairness in packet granularity, whereas Table I does not. Table II address Size Goal 0xA000 1 KB Host A 0x8000_0000 1 KB Host B 0xA400 1 KB Host A 0x8000_0400 1 KB Host B 0xA800 1 KB Host A 0x8000_0800 1 KB Host B 0xAC00 1 KB Host A 0x8000_0C00 1 KB Host B

[0054] In yet another embodiment, the fairness granularity could be configured, for example, as packet granularity, transmission size, etc. The transmission size of 1 KB is merely an example.

[0055] Fig.Figure 7 is a Flowchart 700 illustrating a transaction balancing process according to one embodiment. The process involves receiving a transaction packet from the storage device in block 702, followed by classifying the transaction packet in block 704 and placing the transaction packet in a queue in block 706. Blocks 702, 704, and 706 repeat continuously while more transaction packets are received from the storage device. Once packets are in a queue, the data storage device determines in block 708 whether the controller is ready to send a packet to a host device. This involves determining whether an interface is available between a host device and the controller for transmitting a packet. If no interfaces are available, block 708 continues in a loop.If at least one interface is ready, the controller selects the ready interface by specifying in block 710 which host device a packet will be sent to. Then, in block 712, the appropriate queue for the selected host device is chosen, and in block 714, the selected packet is sent from the selected queue to the selected host device.

[0056] Balancing achieves packet-level fairness, enabling the integration of asymmetric systems. Balancing is accomplished by saturating the host interfaces. By balancing transmissions across an interface, performance is optimized, reliability is maintained, and scalability is supported.

[0057] In one embodiment, a data storage device comprises: a storage device; and a controller coupled to the storage device, the controller being configured to: classify transactions to be sent to one or more host devices, the transactions being sent at the packet level; reorder the transactions based on the classifications; and transmit the reordered transactions to the one or more host devices. Classification is based on one or more of: namespace identification (namespace ID), zone ID, transmission ID, execution ID, a physical function of Peripheral Component Interconnect (PCI) express (PCIe), a virtual function of PCIe, and addresses of host devices. Reordering includes placing the classified transactions into different queues.The controller is configured to select a queue from the various queues for transmission. The controller is configured to delay reporting the execution of a command corresponding to a transaction until the transaction for the command has been transmitted. The one or more host devices are a plurality of host devices, and the controller is configured to ensure that interfaces between the controller and the plurality of host devices are fully utilized. The controller includes a host interface module (HIM), which comprises a reorder buffer and an execution and interrupt synchronization module. The reorder buffer uses a round-robin technique to determine when the reordered transactions are transmitted.The reorder buffer is configured to add a delay before a reordered transaction is transferred to a host device. The one or more host devices comprise a plurality of host devices, where the first host device in the plurality has a different full utilization level than the second host device in the plurality.

[0058] In another embodiment, a data storage device comprises: a storage device; and a controller coupled to the storage device, the controller comprising: a host interface module (HIM) comprising a reorder buffer and an execution and interrupt synchronization module, the HIM being configured to support a first interface between the controller and a first host device, and the HIM being configured to support a second interface between the controller and a second host device; a flash interface module (FIM) coupled to the storage device; and a command scheduler coupled between the HIM and the FIM, the controller being configured to: balance traffic between the first host device and the second host device, the balancing ensuring that,The first and second interfaces are fully utilized. The first and second interfaces are asymmetric. The controller is configured to synchronize execution and interrupt messages to the first and second host devices. The first host device is a physical function, and the second host device is a virtual function. The reorder buffer is configured to manage multiple queues for setting classified transactions. The reorder buffer is configured to add a delay before transferring data to either the first or second host device. The execution and interrupt synchronization module is configured to track data transfers across the first and second interfaces to prevent race conditions.

[0059] In another embodiment, a data storage device comprises: means for storing data; and a controller coupled to the means for storing data, the controller being configured to: receive a transaction packet from the means for storing data; classify the transaction packet; place the transaction packet in one of a plurality of queues; send the transaction packet to a host device; and maintain full utilization at an interface between the controller and the host device, as well as at interfaces between the controller and other host devices. The controller is configured to determine to which host device data should be sent from the host device and other host devices. The controller is configured to select from which of the plurality of queues data should be sent.

[0060] While the foregoing relates to embodiments of the present disclosure, other and further embodiments of the disclosure may be conceived without deviating from its basic scope, the scope of which is determined by the following claims.

Claims

[1] Data storage device comprising: a storage device; and a controller coupled to the storage device, wherein the controller is configured to: Classifying transactions to be sent to one or more host devices, with the transactions occurring at a packet level; Reordering transactions based on the classifications and Transferring the reordered transactions to one or more host devices. [2] Data storage device according to claim 1, wherein the classification is based on one or more of: namespace identification (namespace ID), zone ID, transmission ID, execution ID, a physical function of Peripheral Component Interconnect (PCI) express (PCIe), a virtual function of PCIe and addresses of host devices. [3] Data storage device according to claim 1, wherein the reordering comprises placing the classified transactions into different queues. [4] Data storage device according to claim 3, wherein the controller is configured to select a queue from the various queues for transfer. [5] Data storage device according to claim 4, wherein the controller is configured to delay the reporting of an execution of a command corresponding to a transaction until the transaction for the command has been transmitted. [6] Data storage device according to claim 1, wherein the one or more host device(s) is a plurality of host devices and wherein the controller is configured to ensure that interfaces between the controller and the plurality of host devices are fully utilized. [7] Data storage device according to claim 1, wherein the controller comprises a host interface module (HIM) and wherein the HIM comprises a reorder buffer and an execution and interrupt synchronization module. [8] Data storage device according to claim 7, wherein the reorder buffer uses a round-robin technique to determine when the reordered transactions are transferred. [9] Data storage device according to claim 8, wherein the reorder buffer is configured to add a delay before a reordered transaction is transferred to a host device. [10] Data storage device according to claim 1, wherein the one or more host devices comprise a plurality of host devices and wherein a first host device of the plurality of host devices has a different full utilization level than a second host device of the plurality of host devices. [11] Data storage device comprising: a storage device; and a controller coupled to the storage device, the controller comprising: a host interface module (HIM) comprising a reorder buffer and an execution and interrupt synchronization module, wherein the HIM is configured to support a first interface between the controller and a first host device, and wherein the HIM is configured to support a second interface between the controller and a second host device; a flash interface module (FIM) coupled to the storage device; and a command scheduler that is coupled between the HIM and the FIM, with the control configured to: Balancing data traffic between the first host device and the second host device, wherein balancing includes ensuring that the first interface and the second interface are fully utilized. [12] Data storage device according to claim 11, wherein the first interface and the second interface are asymmetric. [13] Data storage device according to claim 11, wherein the controller is configured to synchronize execute and interrupt messages to the first host device and the second host device. [14] Data storage device according to claim 11, wherein the first host device is a physical function and the second host device is a virtual function. [15] Data storage device according to claim 11, wherein the reorder buffer is configured to manage a plurality of queues for setting classified transactions. [16] Data storage device according to claim 11, wherein the reorder buffer is configured to add a delay before data is transferred to the first host device or the second host device. [17] Data storage device according to claim 11, wherein the execution and interrupt synchronization module is configured to track data transmissions over the first interface and the second interface to avoid race conditions. [18] Data storage device comprising: Means for storing data; and a controller coupled with the means for storing data, wherein the controller is configured to: Receiving a transaction packet from the means of storing data; Classifying the transaction package; Placing the transaction package in one of a multitude of queues; Sending the transaction packet to a host device and Maintaining full utilization at an interface between the controller and the host device, as well as at interfaces between the controller and other host devices. [19] Data storage device according to claim 18, wherein the controller is configured to determine to which host device data should be sent from the host device and other host devices. [20] Data storage device according to claim 19, wherein the controller is configured to select from which queue of the plurality of queues data should be sent.

Images