Systems and methods to facilitate efficient load balancing in network interface controllers (NICs)

Abstract

This disclosure relates to systems and methods for facilitating efficient load balancing in a network interface controller (NIC). A network interface controller (NIC) capable of efficient load balancing across hardware engines is provided. The NIC may be equipped with multiple ordering control units (OCUs), queues, selection logic blocks, and allocation logic blocks. The selection logic block can determine an OCU from the multiple OCUs for commands from the queues, which may store one or more commands. The allocation logic block then determines selection settings for the OCU, selects an egress queue based on the selection settings for the command, and sends the command to the egress queue.

Description

[0001] This application is a divisional application of the invention patent application filed on March 23, 2020, with application number 202080028945.8 and entitled "System and method for promoting efficient load balancing in network interface controller (NIC)". Technical Field

[0002] This disclosure generally relates to the field of networking technology. More specifically, this disclosure relates to systems and methods for facilitating efficient load balancing among hardware engines in a network interface controller (NIC). Background Technology

[0003] As network-enabled devices and applications become increasingly prevalent, the diverse types of traffic and ever-increasing network loads continue to demand higher performance from the underlying network architecture. For example, applications such as high-performance computing (HPC), media streaming, and the Internet of Things (IoT) can generate different types of traffic with distinct characteristics. Therefore, in addition to traditional network performance metrics such as bandwidth and latency, network architecture continues to face challenges such as scalability, versatility, and efficiency. Summary of the Invention

[0004] A network interface controller (NIC) is provided that enables efficient load balancing among hardware engines. The NIC may be equipped with multiple ordering control units (OCUs), queues, selection logic blocks, and allocation logic blocks. The selection logic block can determine an OCU from among the multiple OCUs for commands from the queues, where the queues can store one or more commands. The allocation logic block then determines selection settings for the OCUs, selects a flow queue based on the selection settings for the command, and sends the command to the flow queue. Attached Figure Description

[0005] Figure 1 An exemplary network is shown.

[0006] Figure 2A An exemplary NIC chip with multiple NICs is shown.

[0007] Figure 2B An exemplary architecture for a NIC is shown.

[0008] Figure 3 This demonstrates efficient load balancing between hardware engines within the NIC.

[0009] Figure 4A A flowchart illustrating the process of efficiently load balancing out-of-order commands in a NIC is shown.

[0010] Figure 4BA flowchart illustrating the process of efficiently load balancing ordered commands in a NIC is shown.

[0011] Figure 5 An exemplary computer system is shown, equipped with a NIC that facilitates efficient load balancing among hardware engines.

[0012] In these accompanying drawings, the same reference numerals refer to the same elements. Detailed Implementation

[0013] Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of this disclosure. Therefore, the invention is not limited to the embodiments shown.

[0014] Overview

[0015] This disclosure describes systems and methods for facilitating efficient load balancing among hardware engines within a network interface controller (NIC). The NIC allows hosts to communicate with data-driven networks.

[0016] The embodiments described herein address the problem of efficiently distributing load in a NIC for combinations of ordered and unordered commands by: (i) distributing load among a set of ordering control units (OCUs); and (ii) dynamically mapping the corresponding OCUs to the corresponding forwarding hardware units.

[0017] During operation, the NIC can process commands that can perform operations on the host device's memory (e.g., "GET" or "PUT" commands for Remote Direct Memory Access (RDMA)). The NIC's host interface couples the NIC to the host device and facilitates communication between the host device's memory and the NIC. Applications running on the host device can submit commands to the NIC via the host interface. These commands can then be executed simultaneously by multiple hardware engines within the NIC. These hardware engines can be dedicated accelerators for processing Direct Memory Access (DMA) commands. Commands can be retrieved from their respective command queues and distributed among the hardware engines for processing.

[0018] A subset of commands can be ordered. The NIC should execute these ordered commands sequentially. The remaining commands can be unordered and therefore can be executed out of order. Typically, the NIC can apply a hash function to the commands to determine a hash value and assign the command to the hardware engine corresponding to that hash value. Applying a hash function to ordered commands can produce the same hash value. Therefore, if a hash function is used to select from a small set of hardware engines, the NIC may repeatedly select a particular hardware engine while another hardware engine may still be underutilized. As a result, concurrency levels are adversely affected, leading to performance degradation of the NIC and the switch architecture coupled to the NIC.

[0019] To address this issue, the NIC can be equipped with a set of OCUs that can serve as targets for hash functions. The OCUs allow the NIC to facilitate dynamic load balancing mechanisms, ensuring that ordered commands remain ordered while being evenly distributed across hardware engines. Each hardware engine can be associated with one or more flow queues and forwarding hardware. The forwarding hardware may include egress queues (e.g., egress buffers for sending packets). Each hardware engine can obtain commands from one or more flow queues assigned to it for processing. The OCUs can form an intermediate layer between the command queues and the flow queues.

[0020] The NIC can use a hash function to distribute commands to OCUs. However, since the number of OCUs can be significantly greater than the number of hardware engines, command distribution among OCUs can be balanced. The NIC can then dynamically select a stream queue based on its load. For example, the NIC can select a stream queue with a load below a threshold or the stream queue with the lowest load. In this way, OCUs can be dynamically mapped to hardware engines by selecting the hardware engine with the lowest load. Once selected, the same hardware engine can be used for a set of ordered commands.

[0021] Since different applications may generate different ordered command streams, the NIC can allocate the corresponding ordered command streams to different flow queues. Once the execution of a specific ordered command stream is complete, the NIC can release the flow queue for subsequent allocation and remapping. Applying hash functions to a large number of OCUs and dynamically selecting flow queues based on load can facilitate efficient load distribution among corresponding hardware engines. In this way, the NIC can facilitate efficient load balancing among forwarding engines and improve the performance of the NIC and the switch architecture coupled to the NIC.

[0022] One embodiment of the present invention provides a NIC that may be equipped with multiple ordering control units (OCUs), queues, selection logic blocks, and allocation logic blocks. The selection logic block can determine an OCU from among the multiple OCUs for a command from the queues, where the queues may store one or more commands. Then, the allocation logic block can determine a selection setting for the OCU, select an exit queue based on the selection setting for the command, and send the command to the exit queue.

[0023] In a variant of this embodiment, the exit queue may be a stream queue. The allocation logic block can then determine whether the selection setting indicates a static stream queue allocation for the OCU. If the selection setting indicates a static stream queue allocation, the allocation logic block can select the stream queue based on the static allocation.

[0024] In a further variant, if the selection setting indicates dynamic flow queue allocation, the allocation logic block can dynamically select a flow queue from a set of flow queues associated with the OCU based on the load on the respective flow queue.

[0025] In a further variant, the OCU may be in a set of OCUs, and the set of flow queues may be associated with the set of OCUs.

[0026] In a further variant, the allocation logic block can reallocate the stream queue if the corresponding command in the stream queue has been processed.

[0027] In a variant of this embodiment, the selection logic block may determine the OCU by one or more of the following: (i) determining the OCU based on an indicator in the command; and (ii) determining a hash value by applying a hash function to the command and identifying the OCU based on the hash value.

[0028] In a variant of this embodiment, the allocation logic block can determine that the command is an ordered command.

[0029] In a further variant, the allocation logic block can determine whether the command is the initial command in an ordered command stream. If the command is not the initial command, the allocation logic block can select the exit queue based on a previous selection of another command in the ordered command stream.

[0030] In a variant of this embodiment, if the command is an unordered command, the allocation logic block can set a predefined value as the identifier of the OCU. The predefined value can indicate that the command is unordered.

[0031] In a variant of this embodiment, the allocation logic block can maintain the number of commands associated with the OCU.

[0032] In this disclosure, combined with Figure 1 The description is related to network architecture and combines Figure 2A The following descriptions provide further details about the architecture and operations associated with NICs that support efficient management of idempotent operations.

[0033] Figure 1 An exemplary network is illustrated. In this example, switch network 100 (also referred to as a "switch fabric") may include switches 102, 104, 106, 108, and 110. Each switch may have a unique address or ID within switch fabric 100. Various types of devices and networks can be coupled to the switch fabric. For example, storage array 112 may be coupled to switch fabric 100 via switch 110; unlimited bandwidth (IB) based HPC network 114 may be coupled to switch fabric 100 via switch 108; multiple end hosts, such as host 116, may be coupled to switch fabric 100 via switch 104; and IP / Ethernet network 118 may be coupled to switch fabric 100 via switch 102. Typically, switches may have edge ports and fabric ports. Edge ports may be coupled to devices outside the fabric. Fabric ports may be coupled to another switch within the fabric via fabric links. Typically, traffic may be injected into switch fabric 100 via the ingress port of an edge switch and leave switch fabric 100 via the egress port of another (or the same) edge switch. An ingress link can couple the NIC of an edge device (e.g., an HPC terminal host) to an ingress edge port of an edge switch. The switch architecture 100 can then transmit traffic to an egress edge switch, which in turn can transmit the traffic to a destination edge device via another NIC.

[0034] Exemplary NIC Architecture

[0035] Figure 2A An exemplary NIC chip with multiple NICs is shown. (Reference) Figure 1 In the example, NIC chip 200 may be a custom application-specific integrated circuit (ASIC) designed for host 116 to work with switch architecture 100. In this example, chip 200 may provide two separate NICs 202 and 204. Each NIC of chip 200 may be equipped with a host interface (HI) (e.g., an interface for connecting to a host processor) and a high-speed network interface (HNI) for use with devices coupled to... Figure 1The switch architecture 100 communicates with the links. For example, NIC 202 may include HI 210 and HNI 220, and NIC 204 may include HI 211 and HNI 221.

[0036] In some embodiments, HI 210 may be a Peripheral Component Interconnect (PCI) interface, a Fast Peripheral Component Interconnect (PCIe) interface, or a Compute Fast Link (CXL) interface. HI 210 may be coupled to a host via host connection 201, which may include N (e.g., N may be 16 in some chips) PCIe Gen 4 lanes capable of operating at signaling rates up to 25 Gbps per lane. HI 210 may facilitate high-speed network connectivity 203, which can be connected to… Figure 1 The HNI 210 communicates via links in the switch architecture 100. It can operate at aggregated rates of 100 Gbps or 200 Gbps using M (e.g., M could be 4 in some chips) full-duplex serial channels. Each of the M channels can operate at rates of 25 Gbps or 50 Gbps, respectively, based on non-return-to-zero (NRZ) modulation or pulse amplitude modulation 4 (PAM4). The HNI 220 supports the IEEE 802.3 Ethernet-based protocol and enhanced frame formats to support higher-speed small messages.

[0037] The NIC 202 can support one or more of the following: point-to-point messaging via the Message Passing Interface (MPI), remote memory access (RMA) operations, offloading and scheduling of bulk data operations, and Ethernet packet processing. When a host sends an MPI message, the NIC 202 can match the corresponding message type. Furthermore, the NIC 202 can implement both urgent and conventional protocols for MPI to offload the corresponding operation from the host.

[0038] Furthermore, the RMA operations supported by NIC 202 can include PUT, GET, and atomic memory operations (AMO). NIC 202 can provide reliable transmission. For example, if NIC 202 is the source NIC, it can provide a retry mechanism for idempotent operations. Additionally, connection-based error detection and retry mechanisms can be used for ordered operations that may manipulate the target state. The hardware of NIC 202 can maintain the state required for the retry mechanism. In this way, NIC 202 can eliminate the burden on the host (e.g., software). The strategy for determining the retry mechanism can be specified by the host through driver software, thus ensuring the flexibility of NIC 202.

[0039] Furthermore, NIC 202 can facilitate the progress of triggered operations, general unloading mechanisms, and sequences of dependent operations (such as batch data collections). NIC 202 can support application programming interfaces (APIs) (e.g., the libfabric API), which facilitate the... Figure 1 The switch architecture 100 provides infrastructure communication services to applications running on host 116. NIC 202 can also support low-level network programming interfaces, such as the Portals API. Additionally, NIC 202 can provide efficient Ethernet packet processing, which can include efficient transmission when NIC 202 is the sender, and flow manipulation and checksum calculation when NIC 202 is the destination. Furthermore, NIC 202 can support virtualization (e.g., using containers or virtual machines).

[0040] Figure 2B An exemplary architecture of the NIC is shown. In NIC 202, the port macros of HNI 220 can facilitate low-level Ethernet operations such as Physical Coding Sublayer (PCS) and Media Access Control (MAC). Additionally, NIC 202 can provide support for Link Layer Retry (LLR). Incoming packets can be parsed by parser 228 and stored in buffer 229. Buffer 229 can be a PFC buffer, which is supplied with a latency bandwidth for buffering a threshold amount (e.g., one microsecond). HNI 220 may also include a control sending unit 224 and a control receiving unit 226 for managing outgoing and incoming packets, respectively.

[0041] NIC 202 may include a command queuing (CQ) unit 230. CQ unit 230 is responsible for acquiring and issuing host-side commands. CQ unit 230 may include a command queue 232 and a scheduler 234. Command queue 232 may include two separate queues for initiator commands (PUT, GET, etc.) and target commands (Append, Search, etc.). Command queue 232 may be implemented as a circular buffer maintained in the memory of NIC 202. Applications running on the host can write directly to command queue 232. Scheduler 234 may include two separate schedulers for initiator commands and target commands. Initiator commands are categorized into stream queue 236 based on a hash function. One of the stream queues 236 can be assigned to a unique stream. Furthermore, CQ unit 230 may further include a trigger operation module 238 responsible for queuing and dispatching trigger commands.

[0042] Outbound transmission engine (OXE) 240 can pull commands from flow queue 236 for processing and dispatch. OXE 240 may include Address Translation Request Unit (ATRU) 244, which can send address translation requests to Address Translation Unit (ATU) 212. ATU 212 may provide virtual-to-physical address translation on behalf of different engines such as OXE 240, Inbound Transmission Engine (IXE) 250, and Event Engine (EE) 216. ATU 212 may maintain a large translation cache 214. ATU 212 may perform translation itself or use a host-based Address Translation Service (ATS). OXE 240 may also include Message Segmentation Unit (MCU) 246, which can segment large messages into packets corresponding to the size of the Maximum Transmit Unit (MTU). MCU 246 may include multiple MCU modules. When an MCU module is available, it can obtain the next command from the assigned flow queue. The received data can be written to data buffer 242. Then, the MCU module can send the packet header, corresponding traffic category, and packet size to traffic shaper 248. Shaper 248 can determine which requests initiated by MCU 246 can enter the network.

[0043] The selected data packets can then be sent to Packet and Connection Tracking (PCT) 270. PCT 270 can store the data packets in queue 274. PCT 270 can also maintain the status information of outbound commands and update the status information upon receiving a response. PCT 270 can also maintain packet status information (e.g., allowing matching of responses with requests), message status information (e.g., tracking the progress of multi-packet messages), initiator completion status information, and retry status information (e.g., maintaining information needed to retry commands if a request or response is lost). If no response is returned within a threshold time, the corresponding command can be stored in retry buffer 272. PCT 270 can facilitate connection management for initiator commands and target commands based on source table 276 and destination table 278, respectively. For example, PCT 270 can update its source table 276 to track the status required for reliable delivery of data packets and message completion notifications. PCT 270 can forward outbound data packets to HNI 220, which stores the data packets in outbound queue 222.

[0044] NIC 202 may also include IXE 250, which provides packet processing when NIC 202 is a destination or target. IXE 250 may receive incoming packets from HNI 220. Parser 256 may parse incoming packets and pass the corresponding packet information to List Processing Engine (LPE) 264 or Message Status Table (MST) 266 for matching. LPE 264 may match incoming messages against buffers. LPE 264 may determine the buffer and starting address to use for each message. LPE 264 may also manage a pool of list entries 262 used to represent buffers and unexpected messages. MST 266 may store matching results and information needed to generate target-side completion events. MST 266 may be used by unrestricted operations, including multi-packet PUT commands and single-packet and multi-packet GET commands.

[0045] Subsequently, parser 256 can store the data packet in packet buffer 254. IXE 250 can obtain the matching result for conflict checking. Then, DMA write and AMO module 252 can issue an update generated by the write and AMO operation to memory. If the data packet includes a command to generate a target-side memory read operation (e.g., a GET response), the data packet can be passed to OXE 240. NIC 202 may also include EE 216, which can receive requests to generate event notifications from other modules or units in NIC 202. The event notification can specify the generation of a fill event or a count event. EE 216 can manage an event queue located in host processor memory, which writes complete events to the host processor memory. EE 216 can forward count events to CQ unit 230.

[0046] High-efficiency load balancing in NICs

[0047] Figure 3Efficient load balancing between hardware engines in a NIC is illustrated. In this example, host device 300 may include NIC 320. HI 322 of NIC 320 can couple NIC 320 to device 300 and facilitate communication between device 300 and NIC 320. Device 300 may include memory device 302 (e.g., a dynamic random access memory (DRAM) module). Application 308 may issue commands for memory access (e.g., DMA GET or PUT). Host device 300 may store commands in a command queue 306 in memory device 302. NIC 320 may retrieve commands from command queue 306 via HI 322 and distribute the commands among the hardware engines of NIC 320 for processing. Multiple operating units across NIC 320 may be distributed with corresponding hardware engines, such as OXE 330 (e.g., MCU and flow shaper). NIC 320 can use the hardware engines to execute these commands simultaneously.

[0048] A subset of commands in command queue 306 can be ordered. NIC 320 should execute these ordered commands sequentially. The remaining commands in command queue 306 can be unordered and therefore can be executed out of order. Typically, NIC 320 can apply a hash function to the commands to determine a hash value and assign the command to the hardware engine corresponding to that hash value. Applying a hash function to ordered commands 342, 344, and 346 can produce the same hash value. Therefore, if a hash function is used to select from a small set of hardware engines, NIC 320 may repeatedly select a particular hardware engine while another hardware engine may still be underutilized. As a result, concurrency levels are adversely affected, leading to a decrease in the performance of NIC 320.

[0049] To address this issue, the NIC 320 may be equipped with an OCU module 310, which may include multiple OCUs 312, 314, and 316. The NIC 320 may also include an OCU mapping unit (OMU) 326, which facilitates the selection of OCUs from the OCU module 310 by a hash function. The OCU module 310 allows the NIC 320 to facilitate a dynamic load balancing mechanism, ensuring that ordered commands 342, 344, and 346 remain ordered while achieving uniform distribution across hardware engines. The NIC 320 may also include a set of flow queues 328. A corresponding hardware engine may be associated with one or more flow queues and forwarding hardware. The forwarding hardware may include an egress queue. A corresponding hardware engine may obtain commands from one or more flow queues assigned to it for processing. The OCU module 310 may form an intermediate layer between the command queue 306 and the flow queues 328. The NIC 320 may obtain commands 342, 344, and 346 from the command queue 306 and store these commands in a prefetch queue 324.

[0050] Then, OMU 326 can apply a hash function to command 342 in prefetch queue 324. After obtaining the hash value, OMU 326 can determine which OCU the hash value corresponds to. Command 342 can also specify the OCU to which it should be assigned. If the hash value corresponds to OCU 312, OMU 326 can assign command 342 to OCU 312. However, since the number of OCUs can be significantly greater than the number of hardware engines, the distribution of commands among OCUs in OCU module 310 can be balanced. Subsequently, NIC 320 can dynamically select stream queue 340 for command 342 based on the load of stream queue 328. Stream queue 340 can have a load below a threshold or minimum load. In this way, OCU 312 can be dynamically mapped to the hardware engine associated with stream queue 340. Once selected, the same hardware engine can be used for commands 344 and 346, because the hash function of OMU 326 can select OCU 312 for these commands.

[0051] Since different applications may generate different ordered command streams, NIC 320 can allocate the corresponding ordered command streams to different stream queues. For example, a stream including commands 342, 344, and 346 can be allocated to stream queue 340, while another command stream can be allocated to another stream queue. Once the execution of commands 342, 344, and 346 is complete, OMU 326 can release stream queue 340 for subsequent allocation and remapping. Furthermore, when commands 342 and 344 are completed and command 346 is being processed, OMU 326 can remap OCU 312. This allows OMU 326 to release and reallocate OCU 312 when the last command in the command stream has been processed (e.g., entered into stream queue 328 or processed by OXE 330). Applying hash functions to a large number of OCUs in the OCU module 310 and dynamically selecting stream queues from the stream queue 328 based on load can facilitate efficient load distribution among corresponding hardware engines in the NIC 320, thereby improving the performance of the NIC 320.

[0052] The OCUs in the OCU module 310 can be divided into multiple OCU sets (OCUSETs). Resources associated with the NIC 320 (such as command queue 306, prefetch queue 324, and flow queue 328) can be allocated to OCUUSETs. Corresponding OCUs in an OCUSET can share resources associated with the OCUUSET. For example, if OCUs 312 and 314 belong to the same OCUUSET, then OCUs 312 and 314 can share resources belonging to the OCUUSET, such as flow queue 340. The OCUUSET mapping table in the NIC 320 can indicate which OCU belongs to which OCUUSET. Each OCUUSET can include a different set of OCUs. Furthermore, the flow queue mapping table can determine which flow queues are allocated to an OCUUSET based on their respective flow queue identifiers. Multiple OCUUSETs can be mapped to the same flow queue.

[0053] A corresponding OCU (such as OCU 312) can represent a stream of ordered commands 342, 344, and 346. All ordered commands from a particular command queue should be assigned to the same OCU. It should be noted that commands from different command queues within command queue 306 can be assigned to the same OCU. However, when commands from multiple command queues are assigned to the same OCU, NIC 320 can treat these commands as multiple parts of the same command stream. Therefore, each OCU is mapped to a stream queue, and the ordered commands associated with that OCU should enter the same stream queue. For example, since commands 342, 344, and 346 are all mapped to OCU 312, each of these commands can be forwarded to stream queue 340.

[0054] If there are no pending commands for OCU 312 in flow queue 340, NIC 320 can map OCU 312 to different flow queues based on load. Out-of-order command 348 can be assigned to any flow queue belonging to the OCUSET associated with the command. The association between command 348 and its OCUSET can be determined based on the command queue of command 348. In other words, command 348 can be associated with the OCUSET associated with the same command queue as command 348. The selection of flow queues can be determined by selection settings. Based on selection settings, the NIC can select flows by dynamically allocating flow queues based on load or statically allocating flow queues by mapping OCUs to flow queues (e.g., based on user configuration).

[0055] Commands associated with the OCU can only be forwarded to flow queues within its OCUET. Therefore, for each OCUET, the NIC 320 can maintain the number or count of flow queues available to the OCUET, as well as the current minimum load value in the flow queues assigned to the OCUET. The OMU 326 can calculate load metrics when assigning command 342 to flow queue 340. Load metrics can be calculated based on the command forwarded to the OXE 330 and the command load determined by the OXE 330. The NIC 320 can maintain multiple values ​​associated with the corresponding traffic category, such as header, scale, and shift. These values ​​can be stored in corresponding registers (e.g., the Control and Status Register (CSR)). The NIC 320 can determine the load using the command length and the values ​​in the CSR.

[0056] Figure 4A A flowchart illustrating the process of efficiently load balancing out-of-order commands in a NIC is shown. During operation, the NIC can determine the OCU based on the hash of the out-of-order command (operation 402) and determine whether the flow queue allocation for the OCU is static (operation 404). If the flow queue allocation for the OCU is static, the NIC can select the flow queue to be allocated to the OCU (operation 406). On the other hand, if the flow queue allocation is not static (i.e., dynamic), the NIC can select the minimum load flow queue associated with the OCU (operation 408).

[0057] After selecting a flow queue (operation 406 or 408), the NIC can set the OCU identifier to indicate that the command is an out-of-order command (operation 410). It should be noted that since out-of-order commands can be assigned to any flow queue associated with the OCU, a predefined value (e.g., N-1, where N is the number of OCUs in the NIC) can be set to the OCU identifier. Setting the predefined value informs the flow queue that commands can be delivered out of order. The NIC can then increment a counter (e.g., a message counter) associated with the OCU identifier (operation 412) and send the command to the selected flow queue and update the load associated with the flow queue (operation 414).

[0058] Figure 4B A flowchart illustrating the process of efficiently load balancing ordered commands in a NIC is shown. During operation, the NIC can determine the OCU based on the hash of the ordered command (operation 452) and determine whether the flow queue allocation for the OCU is static (operation 454). If the flow queue allocation for the OCU is static, the NIC can select a flow queue to be allocated to the OCU (operation 456). On the other hand, if the flow queue allocation is not static (i.e., dynamic), the NIC can determine whether the received message is an initial message associated with a command (operation 458). If the received message is not an initial message, the NIC can select an already mapped flow queue (operation 460).

[0059] On the other hand, if the received message is an initial message, the NIC can select the minimum load flow queue associated with OCUSET (operation 462). The NIC can then update the flow queue mapping using the selected flow queue (operation 464). The NIC can then set the OCU identifier to indicate that the command is an ordered command (operation 466). After selecting a static flow queue (operation 456), mapping a flow queue (operation 460), or setting the OCU identifier (operation 466), the NIC can increment a counter (e.g., a message counter) associated with the OCU identifier (operation 468) and send the command to the selected flow queue and update the load associated with the flow queue (operation 470).

[0060] Exemplary computer system

[0061] Figure 5An exemplary computer system equipped with a NIC that facilitates efficient packet forwarding is illustrated. Computer system 550 includes a processor 552, a memory device 554, and a storage device 556. Memory device 554 may include a volatile memory device (e.g., a dual in-line memory module (DIMM)). Furthermore, computer system 550 may be coupled to a keyboard 562, a pointing device 564, and a display device 566. Storage device 556 may store an operating system 570. Application program 572 may operate on operating system 570.

[0062] Computer system 550 may be equipped with a host interface coupled to NIC 520 to facilitate efficient data request management. NIC 520 may provide one or more HNIs to computer system 550. NIC 520 may be coupled to switch 502 via one of the HNIs. NIC 520 may include OCU logic block 530 (e.g., in the CQ unit of NIC 520). OCU logic block 530 can manage the OCU of NIC 520 and may include selection logic block 532, mapping logic block 534, and allocation logic block 536.

[0063] Selection logic block 532 can select an OCU for a command from OCU module 540 based on a hash function. Mapping logic block 534 can update the OCUSET mapping table and the flow queue mapping table. Allocation logic block 536 can allocate flow queues to OCUs based on static or dynamic allocation. Dynamic allocation can be based on a dynamic load balancing mechanism.

[0064] In summary, this disclosure describes a NIC that facilitates efficient load balancing among hardware engines within a NIC. The NIC may be equipped with multiple ordering control units (OCUs), queues, selection logic blocks, and allocation logic blocks. The selection logic block can determine an OCU from among the multiple OCUs for commands from the queues, which may store one or more commands. The allocation logic block can then determine selection settings for the OCUs, select a flow queue based on the selection settings for the command, and send the command to the flow queue.

[0065] The methods and processes described above can be executed by hardware logic blocks, modules, logic blocks, or devices. Hardware logic blocks, modules, logic blocks, or devices may include, but are not limited to, application-specific integrated circuit (ASIC) chips, field-programmable gate arrays (FPGAs), dedicated or shared processors that execute blocks of code at specific times, and other programmable logic devices now known or developed in the future. When activated, a hardware logic block, module, or device executes the methods and processes contained therein.

[0066] The methods and processes described herein can also be embodied in code or data, which can be stored in a storage device or computer-readable storage medium. When a processor reads and executes the stored code or data, the processor can execute these methods and processes.

[0067] The foregoing description of embodiments of the present invention is presented for illustrative and descriptive purposes only. The description is not intended to be exhaustive or to limit the invention to the forms disclosed. Accordingly, many modifications and variations will be apparent to those skilled in the art. Furthermore, the foregoing disclosure is not intended to limit the invention. The scope of the invention is defined by the appended claims.

Claims

1. A network interface controller (NIC) comprising: a plurality of ordering control units (OCUs); a queue to store one or more commands; a selection logic block to determine an OCU for a command from the queue from among the plurality of OCUs; and an assignment logic block to: determine a selection setting for the OCU; select an egress queue for the command based on the selection setting and a load of the egress queue; determine that the command is an in-order command; determine whether the command is an initial command in an in-order command stream; in response to the command not being the initial command, select the egress queue based on a previous selection of another command in the in-order command stream; and send the command to the egress queue.

2. The network interface controller of claim 1, wherein, the egress queue is a stream queue, and wherein the assignment logic block is further to: determine whether the selection setting indicates a static stream queue assignment for the OCU; and in response to the selection setting indicating a static stream queue assignment, select the stream queue based on the static stream queue assignment.

3. The network interface controller of claim 2, wherein, in response to the selection setting indicating a dynamic stream queue assignment, the assignment logic block is further to dynamically select the stream queue from a set of stream queues associated with the OCU based on a load on a respective stream queue.

4. The network interface controller of claim 3, wherein, the OCU is in a set of OCUs, and wherein the set of stream queues is associated with the set of OCUs.

5. The network interface controller of claim 2, wherein, the assignment logic block is further to reassign the stream queue in response to determining that a respective command in the stream queue has been processed.

6. The network interface controller of claim 1, wherein, the selection logic block is further to determine the OCU by one or more of: determining the OCU based on an indicator in the command; and determining a hash value by applying a hash function to the command, and identifying the OCU based on the hash value.

7. The network interface controller of claim 1, wherein, the assignment logic block is further to maintain a number of commands associated with the OCU.

8. A method for a network interface controller (NIC) comprising: obtaining a command from a queue in the NIC, wherein the NIC comprises a plurality of ordering control units (OCUs); determining an OCU for the command obtained from the queue from among the plurality of OCUs; determining a selection setting for the OCU; selecting an egress queue for the command based on the selection setting and a load of the egress queue; in response to determining that the command is an out-of-order command, setting a predefined value as an identifier of the OCU, wherein the predefined value indicates that the command is out-of-order; and sending the command to the egress queue.

9. The method of claim 8, wherein, the egress queue is a stream queue, and wherein the method further comprises: determining whether the selection setting indicates a static stream queue assignment for the OCU; and in response to the selection setting indicating a static stream queue assignment, selecting the stream queue based on the static stream queue assignment.

10. The method of claim 9, further comprising, in response to the selection setting indicating a dynamic stream queue assignment, dynamically selecting the stream queue from a set of stream queues associated with the OCU based on a load on a respective stream queue.

11. The method of claim 10, wherein, The OCU is in a set of OCUs, and wherein the set of flow queues is associated with the set of OCUs.

12. The method of claim 9, further comprising reassigning the flow queue in response to determining that a respective command in the flow queue has been processed.

13. The method of claim 8, wherein, Determining the OCU further comprises one or more of: determining the OCU based on an indicator in the command; and determining a hash value by applying a hash function to the command, and identifying the OCU based on the hash value.

14. The method of claim 8, further comprising maintaining a number of commands associated with the OCU.

Images