System and method for facilitating efficient load balancing in network interface controller (NIC)

Abstract

The disclosure relates to systems and methods for facilitating efficient load balancing in a network interface controller (NIC). A network interface controller (NIC) capable of efficient load balancing between hardware engines is provided. The NIC may be equipped with a plurality of ordering control units (OCUs), queues, selection logic blocks, and allocation logic blocks. The selection logic block may determine an OCU from the plurality of OCUs for a command from the queue, which may store one or more commands. The allocation logic block may then determine a selection setting for the OCU, select an egress queue for the command based on the selection setting, and send the command to the egress queue.

Description

[0001] This application is a divisional application of an invention patent application with an application date of March 23, 2020, application number 202080028945.8, and invention name “System and method for promoting efficient load balancing in a network interface controller (NIC)”. Technical Field

[0002] The present disclosure relates generally to the field of networking technology. More specifically, the present disclosure relates to systems and methods for facilitating efficient load balancing between hardware engines in a network interface controller (NIC). Background Art

[0003] As network-enabled devices and applications become more ubiquitous, various types of traffic and increasing network loads continue to require higher performance from the underlying network architecture. For example, applications such as high-performance computing (HPC), media streaming, and the Internet of Things (IoT) can generate different types of traffic with distinct characteristics. Therefore, in addition to traditional network performance indicators such as bandwidth and latency, network architecture continues to face challenges such as scalability, versatility, and efficiency. Summary of the invention

[0004] A network interface controller (NIC) capable of efficient load balancing between hardware engines is provided. The NIC may be equipped with a plurality of ordering control units (OCUs), queues, a selection logic block, and an allocation logic block. The selection logic block may determine an OCU from the plurality of OCUs for a command from the queue, the queue may store one or more commands. The allocation logic block may then determine a selection setting for the OCU, select a stream queue for the command based on the selection setting, and send the command to the stream queue. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 An exemplary network is shown.

[0006] Figure 2A An exemplary NIC chip having multiple NICs is shown.

[0007] Figure 2B An exemplary architecture of a NIC is shown.

[0008] Figure 3 Efficient load balancing between hardware engines in the NIC is shown.

[0009] Figure 4A A flow chart of a process for efficiently load balancing out-of-order commands in a NIC is shown.

[0010] Figure 4BA flow chart showing a process for efficiently load balancing ordered commands in a NIC.

[0011] Figure 5 An exemplary computer system equipped with a NIC that facilitates efficient load balancing among hardware engines is shown.

[0012] In the drawings, like reference numerals refer to like drawing elements. DETAILED DESCRIPTION

[0013] Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the present disclosure. Therefore, the present invention is not limited to the embodiments shown.

[0014] Overview

[0015] The present disclosure describes systems and methods that facilitate efficient load balancing between hardware engines in a network interface controller (NIC). The NIC allows a host to communicate with a data-driven network.

[0016] The embodiments described herein solve the problem of efficiently distributing the load for a combination of in-order and out-of-order commands in a NIC by: (i) distributing the load among a set of order control units (OCUs); and (ii) dynamically mapping the respective OCUs to corresponding forwarding hardware units.

[0017] During operation, the NIC can process commands that can perform operations on the memory of the host device (e.g., a "GET" or "PUT" command of a remote direct memory access (RDMA)). The host interface of the NIC can couple the NIC to the host device and facilitate communication between the memory of the host device and the NIC. Applications running on the host device can submit commands to the NIC via the host interface. These commands can then be executed simultaneously by multiple hardware engines of the NIC. These hardware engines can be dedicated accelerators for processing direct memory access (DMA) commands. Commands can be obtained from respective command queues and distributed among the hardware engines for processing.

[0018] A subset of commands may be ordered. The NIC should execute these ordered commands in order. The remaining commands may be unordered and therefore may not be executed in order. Typically, the NIC may apply a hash function to the command to determine a hash value and assign the command to a hardware engine corresponding to the hash value. Applying a hash function to ordered commands may generate the same hash value. Therefore, if a hash function is used to select from a small group of hardware engines, the NIC may repeatedly select a specific hardware engine, while another hardware engine may remain underutilized. As a result, the level of concurrency may be adversely affected, resulting in reduced performance of the NIC and the switch structure coupled to the NIC.

[0019] To address this problem, the NIC can be equipped with a set of OCUs that can be targets of hash functions. The OCU can allow the NIC to facilitate a dynamic load balancing mechanism so that ordered commands remain ordered while being evenly balanced between hardware engines. The corresponding hardware engine can be associated with one or more flow queues and forwarding hardware. The forwarding hardware may include an egress queue (e.g., an outgoing buffer for sending data packets). The corresponding hardware engine can obtain commands from one or more flow queues assigned to the hardware engine for processing. The OCU can form an intermediate layer between the command queue and the flow queue.

[0020] The NIC can use a hash function to assign commands to OCUs. However, since the number of OCUs can be significantly greater than the number of hardware engines, the distribution of commands between OCUs can be balanced. Subsequently, the NIC can dynamically select a flow queue based on the load of the flow queue. For example, the NIC can select a flow queue with a load below a threshold or a flow queue with the smallest load. In this way, the OCU can be dynamically mapped to the hardware engine by selecting the hardware engine with the smallest load. Once selected, the same hardware engine can be used for a set of ordered commands.

[0021] Since different applications may generate different ordered command streams, the NIC can assign the corresponding ordered command streams to different flow queues. Once the execution of a particular ordered command stream is completed, the NIC can release the flow queue for subsequent allocation and remapping. Applying hash functions on a large number of OCUs and performing dynamic load-based selection of flow queues can facilitate efficient load distribution between corresponding hardware engines. In this way, the NIC can facilitate efficient load balancing between forwarding engines and can improve the performance of the NIC and the switch structure coupling the NIC.

[0022] One embodiment of the present invention provides a NIC, which may be equipped with a plurality of order control units (OCUs), queues, a selection logic block, and an allocation logic block. The selection logic block may determine an OCU for a command from the queue from the plurality of OCUs, and the queue may store one or more commands. Then, the allocation logic block may determine a selection setting for the OCU, select an egress queue for the command based on the selection setting, and send the command to the egress queue.

[0023] In a variation of this embodiment, the egress queue may be a flow queue. The allocation logic block may then determine whether the selection setting indicates a static flow queue allocation for the OCU. If the selection setting indicates a static flow queue allocation, the allocation logic block may select the flow queue based on the static allocation.

[0024] In a further variation, if the selection setting indicates dynamic flow queue allocation, the allocation logic block may dynamically select the flow queue from a set of flow queues associated with the OCU based on the load on the corresponding flow queue.

[0025] In a further variant, the OCU may be in a group of OCUs, and the group of flow queues may be associated with the group of OCUs.

[0026] In a further variation, the allocation logic block may reallocate the flow queue if a corresponding command in the flow queue has been processed.

[0027] In a variation of this embodiment, the selection logic block may determine the OCU by one or more of: (i) determining the OCU based on an indicator in the command; and (ii) determining a hash value by applying a hash function to the command and identifying the OCU based on the hash value.

[0028] In a variation of this embodiment, the allocation logic block may determine that the command is an ordered command.

[0029] In a further variation, the allocation logic block may determine whether the command is an initial command in an ordered command stream. If the command is not the initial command, the allocation logic block may select the egress queue based on a previous selection of another command in the ordered command stream.

[0030] In a variation of this embodiment, if the command is an out-of-order command, the allocation logic block may set a predefined value as the identifier of the OCU. The predefined value may indicate that the command is out-of-order.

[0031] In a variation of this embodiment, the allocation logic block may maintain the number of commands associated with the OCU.

[0032] In the present disclosure, Figure 1 The description is related to the network architecture and combined with Figure 2A The description that follows provides more details regarding the architecture and operation associated with a NIC that supports efficient management of idempotent operations.

[0033] Figure 1 An exemplary network is shown. In this example, a switch network 100 (which may also be referred to as a "switch fabric") may include switches 102, 104, 106, 108, and 110. Each switch may have a unique address or ID within the switch fabric 100. Various types of devices and networks may be coupled to the switch fabric. For example, a storage array 112 may be coupled to the switch fabric 100 via switch 110; an InfiniBand (IB)-based HPC network 114 may be coupled to the switch fabric 100 via switch 108; multiple end hosts such as host 116 may be coupled to the switch fabric 100 via switch 104; and an IP / Ethernet network 118 may be coupled to the switch fabric 100 via switch 102. Typically, a switch may have edge ports and fabric ports. An edge port may be coupled to a device external to the fabric. A fabric port may be coupled to another switch within the fabric via a fabric link. Typically, traffic may be injected into the switch fabric 100 via an ingress port of an edge switch and leave the switch fabric 100 via an egress port of another (or the same) edge switch. The ingress link can couple the NIC of the edge device (e.g., HPC end host) to the ingress edge port of the edge switch. The switch fabric 100 can then transmit the traffic to the egress edge switch, which in turn can transmit the traffic to the destination edge device via another NIC.

[0034] Exemplary NIC Architecture

[0035] Figure 2A An exemplary NIC chip having multiple NICs is shown. Figure 1 In the example of FIG. 1 , NIC chip 200 may be a custom application specific integrated circuit (ASIC) designed for host 116 to work with switch fabric 100. In this example, chip 200 may provide two independent NICs 202 and 204. Each NIC of chip 200 may be equipped with a host interface (HI) (e.g., an interface for connecting to a host processor) and a high-speed network interface (HNI) for connecting to a host processor. Figure 1For example, the NIC 202 may include the HI 210 and the HNI 220, and the NIC 204 may include the HI 211 and the HNI 221.

[0036] In some embodiments, HI 210 may be a peripheral component interconnect (PCI) interface, a peripheral component interconnect express (PCIe) interface, or a compute express link (CXL) interface. HI 210 may be coupled to a host via a host connection 201, which may include N (e.g., N may be 16 in some chips) PCIe Gen 4 lanes capable of operating at a signaling rate of up to 25 Gbps per lane. HNI 210 may facilitate a high-speed network connection 203, which may be coupled to a host computer via a host connection 201. Figure 1 The HNI 210 can communicate with the links in the switch structure 100. The HNI 210 can operate at an aggregate rate of 100 Gbps or 200 Gbps using M (for example, M can be 4 in some chips) full-duplex serial channels. Each of the M channels can operate at a rate of 25 Gbps or 50 Gbps based on non-return to zero (NRZ) modulation or pulse amplitude modulation 4 (PAM4), respectively. The HNI 220 can support the Institute of Electrical and Electronics Engineers (IEEE) 802.3 Ethernet-based protocols and enhanced frame formats that provide support for small messages at higher rates.

[0037] NIC 202 can support one or more of the following: point-to-point message passing based on message passing interface (MPI), remote memory access (RMA) operations, offloading and scheduling of bulk data collective operations, and Ethernet packet processing. When the host sends an MPI message, NIC 202 can match the corresponding message type. In addition, NIC 202 can implement both the urgent protocol and the agreed protocol for MPI, thereby offloading the corresponding operation from the host.

[0038] In addition, the RMA operations supported by NIC 202 may include PUT, GET, and atomic memory operations (AMO). NIC 202 may provide reliable transmission. For example, if NIC 202 is a source NIC, NIC 202 may provide a retry mechanism for idempotent operations. In addition, connection-based error detection and retry mechanisms may be used for ordered operations that may manipulate target states. The hardware of NIC 202 may maintain the state required for the retry mechanism. In this way, NIC 202 may eliminate the burden on the host (e.g., software). The strategy for determining the retry mechanism may be specified by the host through the driver software, thereby ensuring the flexibility of NIC 202.

[0039] In addition, the NIC 202 can facilitate the progress of triggered operations, general offload mechanisms, and dependent operation sequences (such as bulk data collection). The NIC 202 can support an application programming interface (API) (e.g., libfabric API) that facilitates Figure 1 The switch fabric 100 provides fabric communication services to applications running on the host 116. The NIC 202 may also support low-level network programming interfaces, such as the Portals API. In addition, the NIC 202 may provide efficient Ethernet packet processing, which may include efficient transmission when the NIC 202 is a sender, flow manipulation when the NIC 202 is a target, and checksum calculation. In addition, the NIC 202 may support virtualization (e.g., using containers or virtual machines).

[0040] Figure 2B An exemplary architecture of a NIC is shown. In NIC 202, the port macros of HNI 220 can facilitate low-level Ethernet operations, such as physical coding sublayer (PCS) and media access control (MAC). In addition, NIC 202 can provide support for link layer retries (LLR). Incoming data packets can be parsed by parser 228 and stored in buffer 229. Buffer 229 can be a PFC buffer that is supplied with a delay bandwidth for buffering a threshold amount (e.g., one microsecond). HNI 220 can also include a control sending unit 224 and a control receiving unit 226 for managing outgoing data packets and incoming data packets, respectively.

[0041] NIC 202 may include a command queue (CQ) unit 230. CQ unit 230 may be responsible for obtaining and issuing host-side commands. CQ unit 230 may include a command queue 232 and a scheduler 234. Command queue 232 may include two independent queues for initiator commands (PUT, GET, etc.) and target commands (Append, Search, etc.), respectively. Command queue 232 may be implemented as a circular buffer maintained in the memory of NIC 202. Applications running on the host may directly write to command queue 232. Scheduler 234 may include two separate schedulers for initiator commands and target commands, respectively. Initiator commands are classified into flow queues 236 based on hash functions. One of flow queues 236 may be assigned to a unique flow. In addition, CQ unit 230 may further include a trigger operation module 238, which is responsible for queuing and dispatching trigger commands.

[0042] Outbound transmission engine (OXE) 240 can pull commands from stream queue 236 to process them for dispatch. OXE 240 can include address translation request unit (ATRU) 244, which can send address translation request to address translation unit (ATU) 212. ATU 212 can provide virtual to physical address translation on behalf of different engines, such as OXE 240, inbound transmission engine (IXE) 250 and event engine (EE) 216. ATU 212 can maintain a large translation cache 214. ATU 212 can perform translation itself or use host-based address translation service (ATS) to perform translation. OXE 240 can also include message segmentation unit (MCU) 246, which can split large messages into packets of size corresponding to maximum transmission unit (MTU). MCU 246 can include multiple MCU modules. When MCU module is available, MCU module can get the next command from the assigned stream queue. The received data may be written into the data buffer 242. The MCU module may then send the packet header, the corresponding traffic class, and the packet size to the traffic shaper 248. The shaper 248 may determine which requests made by the MCU 246 may enter the network.

[0043] Subsequently, the selected data packet can be sent to a data packet and connection tracking (PCT) 270. PCT 270 can store the data packet in a queue 274. PCT 270 can also maintain state information of the outbound command and update the state information when a response is returned. PCT 270 can also maintain data packet state information (e.g., allowing a response to be matched with a request), message state information (e.g., tracking the progress of a multi-data packet message), initiator completion state information, and retry state information (e.g., maintaining information required for retrying a command when a request or response is lost). If a response is not returned within a threshold time, the corresponding command can be stored in a retry buffer 272. PCT 270 can facilitate connection management for initiator commands and target commands based on a source table 276 and a target table 278, respectively. For example, PCT 270 can update its source table 276 to track the state required for reliably transmitting data packets and message completion notifications. PCT 270 can forward outgoing data packets to HNI 220, which stores the data packets in an outbound queue 222.

[0044] NIC 202 may also include IXE 250, which provides packet processing when NIC 202 is a target or destination. IXE 250 may obtain incoming packets from HNI 220. Parser 256 may parse incoming packets and pass corresponding packet information to list processing engine (LPE) 264 or message status table (MST) 266 for matching. LPE 264 may match incoming messages with buffers. LPE 264 may determine the buffer and start address to be used for each message. LPE 264 may also manage a pool of list entries 262 for representing buffers and unexpected messages. MST 266 may store matching results and information required to generate target side completion events. MST 266 may be used by unrestricted operations, including multi-packet PUT commands and single-packet and multi-packet GET commands.

[0045] Subsequently, the parser 256 may store the data packet in the data packet buffer 254. The IXE 250 may obtain the matching result for conflict checking. Then, the DMA write and AMO module 252 may issue updates generated by the write and AMO operations to the memory. If the data packet includes a command (e.g., a GET response) to generate a target side memory read operation, the data packet may be passed to the OXE 240. The NIC 202 may also include an EE 216, which may receive a request to generate an event notification from other modules or units in the NIC 202. The event notification may specify the generation of a fill event or a count event. The EE 216 may manage an event queue located within the host processor memory, which writes a complete event to the host processor memory. The EE 216 may forward the count event to the CQ unit 230.

[0046] Efficient load balancing in NIC

[0047] Figure 3Efficient load balancing between hardware engines in a NIC is shown. In this example, a host device 300 may include a NIC 320. A HI 322 of the NIC 320 may couple the NIC 320 to the device 300 and facilitate communication between the device 300 and the NIC 320. The device 300 may include a memory device 302 (e.g., a dynamic random access memory (DRAM) module). An application 308 may issue a command for memory access (e.g., a DMA GET or PUT). The host device 300 may store the command in one of the command queues 306 in the memory device 302. The NIC 320 may obtain the command from the command queue 306 via the HI 322 and distribute the command between the hardware engines of the NIC 320 for processing. Corresponding hardware engines, such as OXE 330 (e.g., an MCU and a traffic shaper) may be distributed across multiple operating units of the NIC 320. The NIC 320 may use the hardware engines to execute these commands simultaneously.

[0048] A subset of the commands in the command queue 306 may be ordered. The NIC 320 should execute these ordered commands in order. The remaining commands in the command queue 306 may be unordered and therefore may not be executed in order. Typically, the NIC 320 may apply a hash function to the command to determine a hash value and distribute the command to a hardware engine corresponding to the hash value. Applying a hash function to the ordered commands 342, 344, and 346 may generate the same hash value. Therefore, if a hash function is used to select in a small group of hardware engines, the NIC 320 may repeatedly select a specific hardware engine, while another hardware engine may still not be fully utilized. As a result, the concurrency level may be adversely affected, resulting in reduced performance of the NIC 320.

[0049] To address this problem, the NIC 320 may be equipped with an OCU module 310, which may include a plurality of OCUs 312, 314, and 316. The NIC 320 may also include an OCU mapping unit (OMU) 326, which may facilitate a hash function to select an OCU from the OCU module 310. The OCU module 310 may allow the NIC 320 to facilitate a dynamic load balancing mechanism so that the ordered commands 342, 344, and 346 remain in order while being evenly balanced between the hardware engines. The NIC 320 may also include a set of flow queues 328. The corresponding hardware engine may be associated with one or more flow queues and forwarding hardware. The forwarding hardware may include an egress queue. The corresponding hardware engine may obtain commands from one or more flow queues assigned to the hardware engine for processing. The OCU module 310 may form an intermediate layer between the command queue 306 and the flow queue 328. The NIC 320 may obtain commands 342, 344, and 346 from the command queue 306 and store them in the pre-fetch queue 324.

[0050] Then, OMU 326 can apply a hash function to command 342 in pre-fetch queue 324. After obtaining the hash value, OMU 326 can determine which OCU the hash value corresponds to. Command 342 can also specify the OCU to which it should be assigned. If the hash value corresponds to OCU 312, OMU 326 can assign command 342 to OCU 312. However, since the number of OCUs can be significantly greater than the number of hardware engines, the distribution of commands between OCUs in OCU module 310 can be balanced. Subsequently, NIC 320 can dynamically select flow queue 340 for command 342 based on the load of flow queue 328. Flow queue 340 can have a load below a threshold or a minimum load amount. In this way, OCU 312 can be dynamically mapped to the hardware engine associated with flow queue 340. Once selected, the same hardware engine can be used for commands 344 and 346 because the hash function of OMU326 can select OCU 312 for these commands.

[0051] Since different applications may generate different ordered command streams, NIC 320 can allocate corresponding ordered command streams to different stream queues. For example, a stream including commands 342, 344, and 346 can be allocated to stream queue 340, while another command stream can be allocated to another stream queue. Once the execution of commands 342, 344, and 346 is completed, OMU 326 can release stream queue 340 for subsequent allocation and remapping. In addition, when commands 342 and 344 are completed and command 346 is being processed, OMU 326 can remap OCU 312. This allows OMU 326 to release and reallocate OCU 312 when the last command in the command stream has been processed (e.g., entered stream queue 328 or processed by OXE 330). Applying a hash function across a large number of OCUs in the OCU module 310 and performing dynamic load-based selection of flow queues from the flow queues 328 may facilitate efficient load distribution among corresponding hardware engines in the NIC 320 , thereby improving the performance of the NIC 320 .

[0052] The OCUs in the OCU module 310 may be divided into a plurality of OCU sets (OCUSETs). Resources associated with the NIC 320, such as the command queue 306, the pre-fetch queue 324, and the stream queue 328, may be allocated to the OCUSET. The corresponding OCUs in the OCUSET may share resources associated with the OCUSET. For example, if OCUs 312 and 314 belong to the same OCUSET, OCUs 312 and 314 may share resources belonging to the OCUSET, such as the stream queue 340. The OCUSET mapping table in the NIC 320 may indicate which OCU belongs to which OCUSET. Each OCUSET may include a set of different OCUs. In addition, the stream queue mapping table may determine which stream queues are allocated to the OCUSET based on respective stream queue identifiers. Multiple OCUSETs may be mapped to the same stream queue.

[0053] The corresponding OCU (such as OCU 312) can represent the stream of ordered commands 342, 344 and 346. All ordered commands from a particular command queue should be assigned to the same OCU. It should be noted that commands from different command queues in command queue 306 can be assigned to the same OCU. However, when commands from multiple command queues are assigned to the same OCU, NIC 320 can treat these commands as multiple parts of the same command stream. Therefore, each OCU is mapped to a stream queue, and the ordered commands associated with the OCU should enter the same stream queue. For example, since commands 342, 344 and 346 are all mapped to OCU 312, each of these commands can be forwarded to stream queue 340.

[0054] If there are no outstanding commands for OCU 312 in flow queue 340, NIC 320 can map OCU 312 to different flow queues based on load. Out-of-order command 348 can be assigned to any flow queue belonging to the OCUSET associated with the command. The association between command 348 and its OCUSET can be determined based on the command queue of command 348. In other words, command 348 can be associated with the OCUSET associated with the command queue of command 348. The selection of flow queues can be determined by a selection setting. Based on the selection setting, the NIC can select a flow by dynamically allocating flow queues based on load or statically allocating flow queues by mapping OCUs to flow queues (e.g., based on user configuration).

[0055] Commands associated with OCU can only be forwarded to the flow queues within its OCUSET. Therefore, for each OCUSET, NIC320 can maintain the number or count of flow queues available to OCUSET and the current minimum load value in the flow queue assigned to OCUSET. OMU 326 can calculate the load index when used to assign command 342 to flow queue 340. The load index can be calculated based on the command forwarded to OXE 330 and the command load determined by OXE 330. NIC 320 can maintain multiple values ​​associated with the corresponding class of traffic, such as headers, ratios, and shifts. These values ​​can be stored in corresponding registers (e.g., control and status registers (CSRs)). NIC 320 can use the length of the command and the value in the CSR to determine the load.

[0056] Figure 4A A flow chart of a process for efficiently load balancing out-of-order commands in a NIC is shown. During operation, the NIC may determine an OCU based on hashing out-of-order commands (operation 402), and determine whether a flow queue assignment for the OCU is static (operation 404). If the flow queue assignment for the OCU is static, the NIC may select a flow queue assigned to the OCU (operation 406). On the other hand, if the flow queue assignment is not static (i.e., dynamic), the NIC may select a least loaded flow queue associated with the OCUSET (operation 408).

[0057] After selecting a flow queue (operation 406 or 408), the NIC may set the OCU identifier to indicate that the command is an out-of-order command (operation 410). It should be noted that since out-of-order commands can be assigned to any flow queue associated with the OCUSET, a predefined value (e.g., N-1, where N is the number of OCUs in the NIC) may be set to the OCU identifier. Setting the predefined value may notify the flow queue that commands may not be transmitted in order. The NIC may then increment a counter (e.g., a message counter) associated with the OCU identifier (operation 412), and send the command to the selected flow queue and update the load associated with the flow queue (operation 414).

[0058] Figure 4B A flow chart of a process for efficiently load balancing ordered commands in a NIC is shown. During operation, the NIC may determine an OCU for an ordered command based on a hash (operation 452), and determine whether a flow queue assignment for the OCU is static (operation 454). If the flow queue assignment for the OCU is static, the NIC may select a flow queue assigned to the OCU (operation 456). On the other hand, if the flow queue assignment is not static (i.e., dynamic), the NIC may determine whether the received message is an initial message associated with a command (operation 458). If the received message is not an initial message, the NIC may select a flow queue that has already been mapped (operation 460).

[0059] On the other hand, if the received message is an initial message, the NIC may select the least loaded flow queue associated with OCUSET (operation 462). The NIC may then update the flow queue mapping using the selected flow queue (operation 464). The NIC may then set the OCU identifier to indicate that the command is an ordered command (operation 466). After selecting a static flow queue (operation 456) or mapping a flow queue (operation 460) or setting the OCU identifier (operation 466), the NIC may increment a counter (e.g., a message counter) associated with the OCU identifier (operation 468), and send the command to the selected flow queue and update the load associated with the flow queue (operation 470).

[0060] Exemplary Computer System

[0061] Figure 5An exemplary computer system equipped with a NIC that facilitates efficient packet forwarding is shown. Computer system 550 includes a processor 552, a memory device 554, and a storage device 556. The memory device 554 may include a volatile memory device (e.g., a dual in-line memory module (DIMM)). In addition, computer system 550 may be coupled to a keyboard 562, a directional device 564, and a display device 566. The storage device 556 may store an operating system 570. Applications 572 may operate on the operating system 570.

[0062] The computer system 550 may be equipped with a host interface coupled to the NIC 520 that facilitates efficient data request management. The NIC 520 may provide one or more HNIs to the computer system 550. The NIC 520 may be coupled to the switch 502 via one of the HNIs. The NIC 520 may include an OCU logic block 530 (e.g., in a CQ unit of the NIC 520). The OCU logic block 530 may manage the OCU of the NIC 520 and may include a selection logic block 532, a mapping logic block 534, and an allocation logic block 536.

[0063] The selection logic block 532 may select an OCU for the command from the OCU module 540 based on a hash function. The mapping logic block 534 may update the OCUSET mapping table and the flow queue mapping table. The allocation logic block 536 may allocate the flow queue to the OCU based on static allocation or dynamic allocation. The dynamic allocation may be based on a dynamic load balancing mechanism.

[0064] In summary, the present disclosure describes a NIC that facilitates efficient load balancing between hardware engines in a NIC. The NIC may be equipped with a plurality of ordering control units (OCUs), queues, a selection logic block, and an allocation logic block. The selection logic block may determine an OCU from the plurality of OCUs for a command from the queue, the queue may store one or more commands. The allocation logic block may then determine a selection setting for the OCU, select a flow queue for the command based on the selection setting, and send the command to the flow queue.

[0065] The above methods and processes may be performed by hardware logic blocks, modules, logic blocks or devices. Hardware logic blocks, modules, logic blocks or devices may include, but are not limited to, application specific integrated circuit (ASIC) chips, field programmable gate arrays (FPGAs), dedicated or shared processors that execute code blocks at specific times, and other programmable logic devices now known or later developed. The hardware logic blocks, modules or devices execute the methods and processes contained therein when activated.

[0066] The methods and processes described herein may also be embodied as code or data, which may be stored in a storage device or a computer-readable storage medium. When a processor reads and executes the stored code or data, the processor may perform the methods and processes.

[0067] The foregoing description of the embodiments of the present invention is presented for the purpose of illustration and description only. The description is not intended to be exhaustive or to limit the present invention to the disclosed forms. Accordingly, many modifications and variations will be apparent to those of ordinary skill in the art. In addition, the above disclosure is not intended to limit the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A network interface controller (NIC), comprising: Multiple order control units (OCUs); A queue, the queue being used to store one or more commands; a selection logic block for determining an OCU from among the plurality of OCUs for a command from the queue; and An allocation logic block is used to: determining selection settings for the OCU; selecting an egress queue for the command based on the selection setting; determining that the command is an ordered command; determining whether the command is an initial command in an ordered command stream; In response to the command not being the initial command, selecting the egress queue based on a previous selection of another command in the ordered command stream; and Send the command to the egress queue.

2. The network interface controller according to claim 1, wherein: The egress queue is a flow queue, and wherein the allocation logic block is further configured to: determining whether the selection setting indicates a static flow queue assignment for the OCU; and In response to the selection setting indicating a static flow queue assignment, the flow queue is selected based on the static assignment.

3. The network interface controller according to claim 2, wherein: In response to the selection setting indicating dynamic flow queue allocation, the allocation logic block is further configured to dynamically select the flow queue from a set of flow queues associated with the OCU based on a load on the corresponding flow queue.

4. The network interface controller of claim 3, wherein: The OCU is in a group of OCUs, and wherein the group of flow queues is associated with the group of OCUs.

5. The network interface controller according to claim 2, wherein: The allocation logic block is further configured to reallocate the flow queue in response to determining that a corresponding command in the flow queue has been processed.

6. The network interface controller of claim 1, wherein: The selection logic block is further configured to determine the OCU by one or more of the following: determining the OCU based on an indicator in the command; and A hash value is determined by applying a hash function to the command, and the OCU is identified based on the hash value.

7. The network interface controller of claim 1, wherein: The allocation logic block is further configured to maintain a number of commands associated with the OCU.

8. A method comprising: Obtaining a command from a queue in a network interface controller (NIC), wherein the NIC includes a plurality of order control units (OCUs); determining an OCU from among the plurality of OCUs for the command obtained from the queue; determining selection settings for the OCU; selecting an egress queue for the command based on the selection setting; In response to determining that the command is an out-of-order command, setting a predefined value as an identifier of the OCU, wherein the predefined value indicates that the command is out-of-order; and Send the command to the egress queue.

9. The method of claim 8, wherein: The egress queue is a flow queue, and wherein the method further comprises: determining whether the selection setting indicates a static flow queue assignment for the OCU; and In response to the selection setting indicating a static flow queue assignment, the flow queue is selected based on the static assignment.

10. The method of claim 9, further comprising, in response to the selection setting indicating dynamic flow queue allocation, dynamically selecting the flow queue from a set of flow queues associated with the OCU based on loads on corresponding flow queues.

11. The method of claim 10, wherein: The OCU is in a group of OCUs, and wherein the group of flow queues is associated with the group of OCUs.

12. The method of claim 9, further comprising reallocating the flow queue in response to determining that a corresponding command in the flow queue has been processed.

13. The method of claim 8, wherein: Determining the OCU further includes one or more of the following: determining the OCU based on an indicator in the command; and A hash value is determined by applying a hash function to the command, and the OCU is identified based on the hash value.

14. The method of claim 8, further comprising maintaining a number of commands associated with the OCU.

Images