Method for injecting packets into an output buffer in a network interface controller NIC and NIC

Abstract

A network interface controller (NIC) is provided that can efficiently inject packets into an output buffer. The NIC can be equipped with an output buffer, a plurality of injectors, a prioritization logic block, and a selection logic block. The plurality of injectors can share the output buffer. The prioritization logic block can determine the priority associated with the injector based on the high water level and the low water level associated with the corresponding injector. Then, the selection logic block can determine the injector subset associated with the buffer class from the plurality of injectors and determine whether the injector subset includes a high priority injector. After identifying that the high priority injector is included in the injector subset, the selection logic block can select the high priority injector for injecting the packet into the output buffer.

Description

Technical Field

[0001] The present disclosure relates generally to the field of networking technology. More specifically, the present disclosure relates to systems and methods for facilitating efficient injection of packets into output buffers in a network interface controller (NIC). Background Art

[0002] As network-enabled devices and applications become more ubiquitous, the various types of traffic and the increasing network load continue to require higher performance from the underlying network architecture. For example, applications such as high-performance computing (HPC), streaming media, and the Internet of Things (IOT) can generate different types of traffic with distinct characteristics. Therefore, in addition to traditional network performance indicators such as bandwidth and latency, network architects continue to face challenges such as scalability, versatility, and efficiency. Summary of the invention

[0003] A network interface controller (NIC) is provided that can efficiently inject packets into an output buffer. The NIC can be equipped with an output buffer, a plurality of injectors, a prioritization logic block, and a selection logic block. The plurality of injectors can share the output buffer. The prioritization logic block can determine the priority associated with the injector based on the high water level and the low water level associated with the corresponding injector. Then, the selection logic block can determine the injector subset associated with the buffer class from the plurality of injectors and determine whether the injector subset includes a high priority injector. After identifying that the high priority injector is included in the injector subset, the selection logic block can select the high priority injector for injecting the packet into the output buffer. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Figure 1 An exemplary network is shown.

[0005] Figure 2A An exemplary NIC chip having multiple NICs is shown.

[0006] Figure 2B An exemplary architecture of a NIC is shown.

[0007] Figure 3A An exemplary packet injection in an output buffer in a NIC is shown.

[0008] Figure 3B An exemplary arbitration process for injecting packets into an output buffer in a NIC is shown.

[0009] Figure 4A A flow chart of a priority assignment process for injecting packets into an output buffer in a NIC is shown.

[0010] Figure 4B A flow chart showing an arbitration process for injecting packets into an output buffer in a NIC.

[0011] Figure 5 An exemplary computer system equipped with a NIC that facilitates efficient injection of packets into an output buffer is shown.

[0012] In the drawings, like reference numerals refer to like drawing elements. DETAILED DESCRIPTION

[0013] Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the present disclosure. Therefore, the present invention is not limited to the embodiments shown.

[0014] Overview

[0015] The present disclosure describes systems and methods that facilitate efficient injection of packets into output buffers in a network interface controller (NIC). The NIC allows a host to communicate with a data-driven network. The network can adapt to dynamic data traffic with fast and effective congestion control by maintaining state information for individual packet flows. More specifically, packets injected into a switch network can be classified into flows that can be mapped to their layer 2, layer 3, or other protocol-specific header information. Each flow can be marked with a different identifier local to the input port of the switch and provided with a flow-specific input buffer so that each flow can be flow-controlled individually. In addition, packets in the corresponding flow can be confirmed when they arrive at the exit point of the network, and the confirmed packets can be sent back to the entry point of the flow in the opposite direction along the same data path. Therefore, each switch can obtain state information of the active packet flow it is forwarding and can perform highly responsive, flow-specific flow control. This flow control can allow the network to operate at a higher capacity while providing general traffic engineering capabilities.

[0016] Embodiments described herein solve the problem of efficiently allocating packets from multiple injectors to a shared output buffer of a NIC by: (i) determining the priority of the injectors based on their occupancy of the buffer; and (ii) arbitrating between the injectors based on the class and the determined priority. An injector can be any element of a NIC that can inject traffic into a buffer.

[0017] During operation, the NIC may receive commands from a host device of the NIC. The host interface of the NIC may couple the NIC to the host device and facilitate communication between the host device and the NIC. The command may be an immediate data command (IDC) or a direct memory access (DMA) command. A command that carries data associated with a command may be an IDC. On the other hand, a command with a pointer to associated data is referred to as a DMA command (DMAC) (e.g., a "GET" or "PUT" command for a remote DMA (RDMA)). In addition, traffic generated based on commands may be assigned to different classes, such as a traffic shaping class or a buffer class. Each buffer class may be associated with one or more injectors. On the other hand, an injector may be assigned to a buffer class. Therefore, for the same buffer class, multiple injectors may send packets in parallel.

[0018] However, injectors can share common output buffers. Therefore, if a large number of injectors share a buffer, one injector may consume a large portion of the buffer due to the uneven and random nature of the traffic. In contrast, another injector may not get enough buffer capacity. As a result, the buffer may cause some injectors to be underutilized, causing other injectors to experience bottlenecks. Furthermore, the buffer may be shared unfairly between injectors and injectors' buffer classes.

[0019] To address this problem, the NIC can arbitrate between injectors so that the capacity of the buffer is fairly distributed. The buffer can be divided into multiple cells. The corresponding cells can have a fixed size (e.g., 2048 bytes). The injector can insert traffic into the buffer at a granularity of the cell size. In order to ensure that the buffer capacity is fairly distributed to the injectors, the NIC can select an injector based on one or more selection criteria to insert the traffic into the next available cell. The selection criteria can allow the NIC to select an injector that is undersubscribed (or underutilized) and avoid selecting an injector that is oversubscribed (or overutilized). In addition, the NIC can also distinguish between IDC and DMAC. In addition, the NIC can select an injector to ensure fair distribution of buffer capacity between buffer classes. In some embodiments, the injector can be a message segmentation unit (MCU) module, which can segment messages into packets of a size corresponding to a maximum transmission unit (MTU).

[0020] During operation, the NIC can assign priorities to respective injectors based on restrictions associated with the injectors (e.g., the maximum buffer capacity that can be allocated to the injectors). The NIC can determine the command type associated with the injector. Since the DMAC typically stays in the buffer for a longer time (e.g., due to the presence of additional memory accesses for obtaining associated data), the NIC can use restrictions for each injector for the DMAC and an overall restriction for the IDC command. Based on the restrictions, the NIC can determine a high water mark and a low water mark for the injector, which can be used to determine oversubscription and undersubscription, respectively. A high water mark can indicate that the capacity may be close to the limit. On the other hand, a low water mark can indicate that the capacity may be significantly less than the limit.

[0021] The NIC may then determine the number of cells in the buffer that are currently occupied by data from the injector. If the capacity represented by the occupied cells is greater than or equal to the high water mark, the NIC may assign a low priority to the injector. On the other hand, if the capacity represented by the occupied cells is less than or equal to the low water mark, the NIC may assign a high priority to the injector. The NIC may assign a priority to each injector by repeating this process. The NIC may then perform a two-stage arbitration process to select an injector for the next available cell in the buffer. In the first stage, the NIC may select a buffer class (e.g., based on a weighted round-robin selection).

[0022] After selecting a buffer class, the NIC may identify corresponding injectors associated with the buffer class. The NIC may then determine whether the identified injectors include an injector with a high priority. If the identified injectors include at least one injector with a high priority, the NIC may select the injector. However, in order to select from a plurality of such injectors, the NIC may select one of the injectors using selection strategies such as round-robin selection and first available selection. On the other hand, if the identified injectors include only injectors with a low priority, the NIC may select one of the injectors based on a selection strategy. In order to ensure that a priority is not assigned to an injector for a long time, the NIC may periodically perturb the priority. In this way, the NIC may facilitate allocating packets to a shared output buffer in an efficient manner.

[0023] One embodiment of the present invention provides a NIC, which can be equipped with an output buffer, multiple injectors, a prioritization logic block and a selection logic block. The multiple injectors can share the output buffer. The prioritization logic block can determine the priority associated with the injector based on the high water level and the low water level associated with the corresponding injector. Then, the selection logic block can determine an injector subset associated with a buffer class from the multiple injectors and determine whether the injector subset includes a high priority injector. After identifying the high priority injector in the injector subset, the selection logic block can select the high priority injector for injecting packets into the output buffer.

[0024] In a variation of this embodiment, if the subset of injectors does not include a high priority injector, the selection logic block may select a low priority injector for injecting packets into the output buffer.

[0025] In a variation of this embodiment, the prioritization logic block may determine a command type associated with a corresponding injector. If the command type is an immediate data command (IDC), the prioritization logic block may determine the high water mark and the low water mark based on global restrictions. On the other hand, if the command type is a direct memory access (DMA) command, the prioritization logic block may determine the high water mark and the low water mark based on restrictions specific to the injector.

[0026] In a further variation, the command is issued to the NIC via a Peripheral Component Interconnect Express (PCIe) interface.

[0027] In a variation of this embodiment, the prioritization logic block may obtain the number of cells in the buffer that are occupied by data from the corresponding injector.

[0028] In a further variation, the prioritization logic block may assign a high priority to the injector if the number of cells is less than or equal to the low water mark. On the other hand, the prioritization logic block may assign a low priority to the injector if the number of cells is greater than or equal to the high water mark.

[0029] In a variation of this embodiment, upon detecting that the priority is reset, the prioritization logic block may assign a high priority to the injector. In addition, if a timer associated with the injector has expired, the prioritization logic block may assign a low priority to the injector.

[0030] In a variation of this embodiment, the selection logic block may select the buffer class from a set of buffer classes enabled for the NIC.

[0031] In a variation of this embodiment, the injector may be a message segmentation unit (MCU) configured to generate packets based on commands issued to the NIC.

[0032] In a variation of this embodiment, the output buffer is divided into a plurality of cells. Injecting the packet may then include injecting the packet into the next available cell.

[0033] In the present disclosure, Figure 1 The description is related to the network architecture and combined with Figure 2A The description that follows provides more details regarding the architecture and operation associated with a NIC that supports efficient management of idempotent operations.

[0034] Figure 1 An exemplary network is shown. In this example, a switch network 100 (which may also be referred to as a "switch fabric") may include switches 102, 104, 106, 108, and 110. Each switch may have a unique address or ID within the switch fabric 100. Various types of devices and networks may be coupled to the switch fabric. For example, a storage array 112 may be coupled to the switch fabric 100 via switch 110; an InfiniBand (IB)-based HPC network 114 may be coupled to the switch fabric 100 via switch 108; multiple end hosts such as host 116 may be coupled to the switch fabric 100 via switch 104; and an IP / Ethernet network 118 may be coupled to the switch fabric 100 via switch 102. Typically, a switch may have edge ports and fabric ports. An edge port may be coupled to a device external to the fabric. A fabric port may be coupled to another switch within the fabric via a fabric link. Typically, traffic may be injected into the switch fabric 100 via an ingress port of an edge switch and leave the switch fabric 100 via an egress port of another (or the same) edge switch. The ingress link can couple the NIC of the edge device (e.g., HPC end host) to the ingress edge port of the edge switch. The switch fabric 100 can then transmit the traffic to the egress edge switch, which in turn can pass the traffic to the destination edge device via another NIC.

[0035] Exemplary NIC Architecture

[0036] Figure 2A An exemplary NIC chip having multiple NICs is shown. Figure 1In the example of FIG. 1 , the NIC chip 200 may be a custom application specific integrated circuit (ASIC) designed for the host 116 to work with the switch fabric 100. In this example, the chip 200 may provide two independent NICs 202 and 204. Each NIC of the chip 200 may be equipped with a host interface (HI) (e.g., an interface for connecting to a host processor) and a high-speed network interface (HNI) for connecting to a host processor. Figure 1 For example, the NIC 202 may include the HI 210 and the HNI 220, and the NIC 204 may include the HI 211 and the HNI 221.

[0037] In some embodiments, HI 210 may be a peripheral component interconnect (PCI) interface or a peripheral component interconnect express (PCIe) interface. HI 210 may be coupled to a host via a host connection 201, which may include N (e.g., N may be 16 in some chips) PCIe Gen 4 lanes capable of operating at a signaling rate of up to 25 Gbps per lane. HNI 210 may facilitate a high-speed network connection 203, which may be coupled to a host computer. Figure 1 The HNI 210 can communicate with the links in the switch structure 100. The HNI 210 can operate at a total rate of 100 Gbps or 200 Gbps using M (for example, M can be 4 in some chips) full-duplex serial channels. Each of the M channels can operate at a rate of 25 Gbps or 50 Gbps based on non-return to zero (NRZ) modulation or pulse amplitude modulation 4 (PAM4), respectively. The HNI 220 can support the Institute of Electrical and Electronics Engineers (IEEE) 802.3 Ethernet-based protocols and enhanced frame formats that provide support for small messages at higher rates.

[0038] NIC 202 can support one or more of the following: point-to-point message passing based on message passing interface (MPI), remote memory access (RMA) operations, offloading and scheduling of bulk data collective operations, and Ethernet packet processing. When the host sends an MPI message, NIC 202 can match the corresponding message type. In addition, NIC 202 can implement both the urgent protocol and the agreed protocol for MPI, thereby offloading the corresponding operation from the host.

[0039] In addition, the RMA operations supported by NIC 202 may include PUT, GET, and atomic memory operations (AMO). NIC 202 may provide reliable transmission. For example, if NIC 202 is a source NIC, NIC 202 may provide a retry mechanism for idempotent operations. In addition, connection-based error detection and retry mechanisms may be used for ordered operations that may manipulate target states. The hardware of NIC 202 may maintain the state required for the retry mechanism. In this way, NIC 202 may reduce the burden on the host (e.g., software). The strategy for determining the retry mechanism may be specified by the host through the driver software, thereby ensuring the flexibility of NIC 202.

[0040] In addition, the NIC 202 can facilitate the progress of triggered operations, general offload mechanisms, and dependent operation sequences (such as bulk data collection). The NIC 202 can support an application programming interface (API) (e.g., libfabric API) to facilitate Figure 1 The switch fabric 100 provides fabric communication services to applications running on the host 116. The NIC 202 may also support low-level network programming interfaces, such as the Portals API. In addition, the NIC 202 may provide efficient Ethernet packet processing, which may include efficient transmission when the NIC 202 is a sender, flow manipulation when the NIC 202 is a target, and checksum calculation. In addition, the NIC 202 may support virtualization (e.g., using containers or virtual machines).

[0041] Figure 2B An exemplary architecture of a NIC is shown. In NIC 202, the port macros of HNI 220 can facilitate low-level Ethernet operations, such as physical coding sublayer (PCS) and media access control (MAC). In addition, NIC 202 can provide support for link layer retries (LLR). Incoming packets can be parsed by parser 228 and stored in buffer 229. Buffer 229 can be a PFC buffer that is supplied with a delay bandwidth for buffering a threshold amount (e.g., one microsecond). HNI 220 can also include a control transmitting unit 224 and a control receiving unit 226 for managing outgoing packets and incoming packets, respectively.

[0042] NIC 202 may include a command queue (CQ) unit 230. CQ unit 230 may be responsible for obtaining and issuing host-side commands. CQ unit 230 may include a command queue 232 and a scheduler 234. Command queue 232 may include two independent queues for initiator commands (PUT, GET, etc.) and target commands (Append, Search, etc.), respectively. Command queue 232 may be implemented as a circular buffer maintained in the memory of NIC 202. Applications running on the host may directly write to command queue 232. Scheduler 234 may include two separate schedulers for initiator commands and target commands, respectively. Initiator commands are classified into flow queues 236 based on hash functions. One of flow queues 236 may be assigned to a unique flow. In addition, CQ unit 230 may further include a trigger operation module 238, which is responsible for queuing and dispatching trigger commands.

[0043] Outbound transmission engine (OXE) 240 can pull commands from stream queue 236 to process them for dispatch. OXE 240 can include address translation request unit (ATRU) 244, which can send address translation request to address translation unit (ATU) 212. ATU 212 can provide virtual to physical address conversion on behalf of different engines, such as OXE 240, inbound transmission engine (IXE) 250 and event engine (EE) 216. ATU 212 can maintain a large conversion buffer 214. ATU 212 can perform conversion by itself or use host-based address translation service (ATS) to perform conversion. OXE 240 can also include message segmentation unit (MCU) 246, which can split large messages into packets of size corresponding to maximum transmission unit (MTU). MCU 246 can include multiple MCU modules. When MCU module is available, MCU module can get the next command from the assigned stream queue. The received data may be written into the data buffer 242. The MCU module may then send the packet header, the corresponding traffic classification, and the packet size to the traffic shaper 248. The shaper 248 may determine which requests made by the MCU 246 may enter the network.

[0044] Subsequently, the selected packet can be sent to a packet and connection tracking (PCT) 270. PCT 270 can store the packet in a queue 274. PCT 270 can also maintain the state information of the outbound command and update the state information when a response is returned. PCT 270 can also maintain packet state information (e.g., allowing the response to be matched with the request), message state information (e.g., tracking the progress of multi-packet messages), initiator completion state information, and retry state information (e.g., maintaining the information required for retrying the command when the request or response is lost). If a response is not returned within a threshold time, the corresponding command can be stored in a retry buffer 272. PCT 270 can facilitate connection management of initiator commands and target commands based on source table 276 and target table 278, respectively. For example, PCT 270 can update its source table 276 to track the state required for reliably transmitting packets and message completion notifications. PCT 270 can forward the outgoing packet to HNI 220, which stores the packet in an outbound queue 222.

[0045] NIC 202 may also include IXE 250, which provides packet processing when NIC 202 is a target or destination. IXE 250 may obtain incoming packets from HNI 220. Parser 256 may parse incoming packets and pass corresponding packet information to list processing engine (LPE) 264 or message status table (MST) 266 for matching. LPE 264 may match incoming messages with buffers. LPE 264 may determine the buffer and start address to be used for each message. LPE 264 may also manage list entry pool 262, and unexpected messages, for representing buffers. MST 266 may store matching results and information required for generating target end completion events. MST 266 may be used by unrestricted operations, including multi-packet PUT commands and single-packet and multi-packet GET commands.

[0046] Subsequently, the parser 256 may store the packet in the packet buffer 254. The IXE 250 may obtain a matching result for conflict checking. Then, the DMA write and AMO module 252 may issue an update generated by the write and AMO operation to the memory. If the packet includes a command (e.g., a GET response) to generate a target end memory read operation, the packet may be passed to the OXE 240. The NIC 202 may also include an EE 216, which may receive a request to generate an event notification from other modules or units in the NIC 202. The event notification may specify the generation of a fill event or a count event. The EE 216 may manage an event queue located in a host processor memory, which writes a complete event to the host processor memory. The EE 216 may forward the count event to the CQ unit 230.

[0047] Efficient Packet Injection in NIC

[0048] Figure 3A An exemplary packet injection in an output buffer in a NIC is shown. In this example, a host device 300 may include a NIC 320. A host interface 322 of the NIC 320 may couple the NIC 320 to the device 300 and facilitate communication between the device 300 and the NIC 320. The NIC 320 may include an MCU 324, which may include a plurality of MCU modules 312, 314, 416, and 318. An MCU module in the MCU 324 may inject traffic into a shared output buffer 328. Therefore, an MCU module in the MCU 324 may be an injector of the buffer 328. The traffic injected by the MCU 324 may belong to different buffer classes. Each buffer class may be associated with one or more MCU modules. On the other hand, an MCU module may be assigned to a buffer class. Therefore, for the same buffer class, a plurality of MCU modules may inject packets in parallel.

[0049] However, since MCU modules can share buffer 328, one MCU module may occupy a large portion of buffer 328 due to the unevenness and randomness of traffic. In contrast, another MCU module may not be able to obtain sufficient buffer capacity. Therefore, buffer 328 may cause underutilization of some MCU modules and cause bottlenecks for other MCU modules. In addition, buffer 328 may be unfairly shared between MCU modules and buffer classes of MCU modules.

[0050] To address this problem, the NIC 320 may be equipped with an arbitrator 326 that may arbitrate between the MCU modules in the MCU 324 so that the capacity of the buffer 328 is fairly distributed. The buffer 328 may be divided into a plurality of cells. The respective cells may have a fixed size. The respective MCU modules (such as the MCU module 312) may insert traffic into the buffer 328 at a granularity of the cell size. To ensure that the capacity of the buffer 328 is fairly distributed to the MCU modules, the arbitrator 326 may select an MCU module for inserting traffic into the next available cell 330 based on one or more selection criteria. The selection criteria may allow the arbitrator 326 to select an under-subscribed MCU module and avoid selecting an over-subscribed MCU module.

[0051] In addition, the arbitrator 326 can also distinguish between IDC and DMAC. Assume that the IDC 342 is assigned to the MCU module 312 and the DMAC 344 is assigned to the MCU module 318. Since the DMAC 344 can stay in the buffer 328 for a long time, the arbitrator 326 can use the restrictions specific to the MCU module 312 (e.g., the maximum number of cells of the buffer 328 that can be allocated to the MCU module) to determine whether to select the MCU module 312. On the other hand, the arbitrator 326 can use the global restrictions associated with the IDC to determine whether to select the MCU module 318. In addition, the arbitrator 326 can select the MCU module to ensure fair distribution of buffer capacity between buffer classes.

[0052] Figure 3B An exemplary arbitration process for injecting packets into an output buffer in a NIC is shown. In order to facilitate efficient injection of packets into a buffer 328, an arbitrator 326 may assign priorities to the MCU modules based on restrictions associated with the corresponding MCU modules in MCU 324. The arbitrator 326 may determine that the MCU module 312 is associated with an IDC 342. Therefore, the arbitrator 326 may determine the high water mark and low water mark of the MCU module 312 based on the global restrictions associated with the IDC. Then, the arbitrator 326 may determine the number of cells currently occupied by data from the MCU module 312 in the buffer. The number (or count) of occupied cells may be referred to as an occupied cell count (OCC). If the OCC of the MCU module 312 is greater than or equal to the high water mark, the arbitrator 326 may assign a low priority to the MCU 312. On the other hand, if the OCC of the MCU module 312 is less than or equal to the low water mark, the arbitrator 326 may assign a high priority to the MCU module 312.

[0053] Similarly, the arbitrator 326 can assign priorities to the MCU modules 314, 316, and 318 by repeating this process. For example, the arbitrator 326 can determine that the MCU module 318 is associated with the DMAC 344. Therefore, the arbitrator 326 can determine the high water mark and the low water mark of the MCU module 318 based on the restrictions associated with the MCU module 318. Then, the arbitrator 326 can determine the OCC of the MCU module 318. If the OCC of the MCU module 318 is greater than or equal to the high water mark, the arbitrator 326 can assign a low priority to the MCU 318. On the other hand, if the OCC of the MCU module 318 is less than or equal to the low water mark, the arbitrator 326 can assign a high priority to the MCU module 318.

[0054] The arbitrator 326 may then perform a two-phase arbitration process 360 to select an MCU module for the next available cell 330 in the buffer 328. The arbitration process 360 may include a first phase arbitration 362 and a second phase arbitration 364. In arbitration 362, the arbitrator 326 may select a buffer class among the buffer classes enabled in the NIC 320 (e.g., based on a weighted round-robin selection). In some embodiments, the NIC 320 may support N predefined buffer classes (e.g., 10 classes), each of which may correspond to a buffer class that is associated with the NIC 320. Figure 2B The traffic shaping class associated with the traffic shaper 248 in . The buffer class can be enabled when sufficient resources are available for the buffer class. Such resources may include Figure 2B The retry buffer 272 and the source table 276 associated with the transmission credits and the availability of the buffer 328 of the buffer class. The source table 276 may include one or more of the following: a source packet table (SPT), a source message table (SMT), and a source connection table (SCT).

[0055] NIC 320 may enable buffer classes 352, 354, and 356. Buffer class 352 may include MCU modules 312 and 314; buffer class 354 may include MCU module 316; and buffer class 356 may include MCU module 318. By applying arbitration 362 to buffer classes 352, 354, and 356, arbitrator 326 may select buffer class 352. Arbitrator 326 may then identify that MCU modules 312 and 314 are associated with buffer class 352. Arbitrator 326 may then apply arbitration 364 to MCU modules 312 and 314. Arbitration 364 may select an MCU module with a high priority, if applicable. Otherwise, arbitration 364 may select an MCU module with a low priority.

[0056] Accordingly, the arbitrator 326 can determine whether the MCU modules 312 and 314 include an MCU module with a high priority. For example, if the MCU module 312 has a high priority, the arbitrator 326 can select the MCU module 312. On the other hand, if the MCU modules 312 and 314 have a low priority, the arbitrator 326 can select one of the MCU modules 312 and 314 based on the selection strategy. If the MCU module 312 is selected, the MCU module 312 can inject the packet associated with the command 342 into the cell 330. It should be noted that if the command 342 is larger than the MTU (e.g., a PUT command with a large amount of data), the MCU module 312 can generate multiple packets based on the command 342. After injecting the packet, the MCU module 312 can obtain the next packet associated with the command 342. Subsequently, the MCU module 312 can accept the arbitration 360 again.

[0057] In order to ensure that the priority is not assigned to an MCU module for a long time, the arbitrator 326 can periodically disturb the priority. For example, the priority can be reset (e.g., periodically or based on the NIC being reset). Once reset, a high priority can be assigned to the corresponding MCU module. On the other hand, the corresponding MCU module can be associated with a priority timer. If the timer expires, a low priority can be assigned to the corresponding MCU module. This disturbance can alleviate the effect of the MCU module maintaining a high priority for a considerable period of time and adversely affecting the fairness of the arbitration 360. By ensuring the fairness of the arbitration 360, the NIC 320 can facilitate the injection of packets into the buffer 328 in an efficient manner.

[0058] Figure 4A A flow chart of a priority assignment process for injecting packets into an output buffer in a NIC is shown. During operation, the NIC may determine a command associated with an injector (operation 402). The NIC may then determine whether the command is a DMA command (operation 404). If the command is a DMA command, the NIC may determine a high water mark and a low water mark based on injector-specific limits (operation 416). On the other hand, if the command is not a DMA command (e.g., IDC), the NIC may determine a high water mark and a low water mark based on global limits (operation 406). Subsequently, the NIC may determine whether a reset has been triggered (operation 408). If a reset has not been triggered, the NIC may also determine whether a timer associated with the injector has expired (operation 410).

[0059] If the timer has not expired, the NIC may determine whether the OCC associated with the injector is greater than or equal to the high water level (operation 412). If the OCC is not greater than or equal to the high water level, the NIC may determine whether the OCC is less than or equal to the low water level (operation 414). If a reset has been triggered (operation 408) or the OCC is less than or equal to the low water level (operation 414), the NIC may assign a high priority to the injector (operation 420). On the other hand, if the associated timer has expired (operation 410) or the OCC is greater than or equal to the high water level (operation 412), the NIC may assign a low priority to the injector (operation 418).

[0060] Figure 4BA flow chart of an arbitration process for injecting a packet into an output buffer in a NIC is shown. During operation, the NIC may determine an enabled buffer class (operation 452) and select a buffer class based on a class selection policy (operation 454). The NIC may then identify an injector associated with the buffer class (operation 456) and determine whether there are any high priority injectors among the selected injectors (operation 458). If there is at least one high priority injector among the selected injectors, the NIC may select an injector from the high priority injectors based on the injector selection policy (operation 460). Otherwise, the NIC may select an injector from the low priority injectors based on the injector selection policy (operation 462).

[0061] Exemplary Computer System

[0062] Figure 5 An exemplary computer system equipped with a NIC that facilitates efficient packet forwarding is shown. Computer system 550 includes a processor 552, a memory device 554, and a storage device 556. The memory device 554 may include a volatile memory device (e.g., a dual in-line memory module (DIMM)). In addition, computer system 550 may be coupled to a keyboard 562, a pointing device 564, and a display device 566. The storage device 556 may store an operating system 570. Applications 572 may operate on the operating system 570.

[0063] The computer system 550 may be equipped with a host interface to which the NIC 520 is coupled to facilitate efficient data request management. The NIC 520 may provide one or more HNIs to the computer system 550. The NIC 520 may be coupled to the switch 502 via one of the HNIs. The NIC 520 may include an arbitration logic block 530, such as in conjunction with Figure 3A and Figure 3B The arbiter logic block 530 may include a tracking logic block 532 , a priority logic block 534 , and a selection logic block 536 .

[0064] Tracking logic block 532 can track the OCC of a corresponding injector (e.g., MCU module) of NIC 520. Priority logic block 534 can determine the high water mark and low water mark of the corresponding injector based on the command type associated with the corresponding injector. Priority logic block 534 can then determine the priority and assign the priority to the corresponding injector, such as in conjunction with Figure 4A The selection logic block 536 may select an injector for injecting the packet into the shared output buffer based on the priority and the buffer class of the corresponding injector, such as in conjunction with Figure 4B as described.

[0065] In summary, the present disclosure describes a NIC that facilitates efficient injection of packets into an output buffer. The NIC can be equipped with an output buffer, multiple injectors, a prioritization logic block, and a selection logic block. The multiple injectors can share the output buffer. The prioritization logic block can determine the priority associated with the injector based on the high water mark and low water mark associated with the corresponding injector. The selection logic block can then determine a subset of injectors associated with a buffer class from the multiple injectors and determine whether the subset of injectors includes a high priority injector. After identifying a high priority injector in the injector subset, the selection logic block can select the high priority injector for injecting packets into the output buffer.

[0066] The above methods and processes may be performed by hardware logic blocks, modules, logic blocks or devices. Hardware logic blocks, modules, logic blocks or devices may include, but are not limited to, application specific integrated circuit (ASIC) chips, field programmable gate arrays (FPGAs), dedicated or shared processors that execute code at specific times, and other programmable logic devices now known or later developed. The hardware logic blocks, modules or devices execute the methods and processes contained therein when activated.

[0067] The methods and processes described herein may also be embodied as code or data, which may be stored in a storage device or a computer-readable storage medium. When a processor reads and executes the stored code or data, the processor may perform the methods and processes.

[0068] The foregoing description of the embodiments of the present invention is presented for the purpose of illustration and description only. The description is not intended to be exhaustive or to limit the present invention to the disclosed forms. Accordingly, many modifications and variations will be apparent to those of ordinary skill in the art. In addition, the above disclosure is not intended to limit the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A network interface controller NIC, comprising: Output buffer; A plurality of injectors, the plurality of injectors sharing the output buffer; a prioritization logic block for determining a priority associated with a respective injector based on a high water level and a low water level associated with the respective injector; as well as A selection logic block is used to: determining, from the plurality of injectors, a subset of injectors associated with a buffer class; determining whether the subset of injectors includes a high priority injector; as well as In response to identifying a high priority injector in the subset of injectors, selecting the high priority injector for injecting packets into the output buffer, Wherein, the prioritization logic block is further used for: obtaining the number of cells in the output buffer occupied by data from the corresponding injector; assigning a high priority to the corresponding injector in response to the number of cells being less than or equal to the low water level; and In response to the number of cells being greater than or equal to the high water mark, the corresponding injector is assigned a low priority.

2. The network interface controller according to claim 1, wherein: In response to determining that the subset of injectors does not include a high priority injector, the selection logic block is further for selecting a low priority injector for injecting the packet into the output buffer.

3. The network interface controller according to claim 1, wherein: The prioritization logic block is further configured to: determining a command type associated with a corresponding injector; In response to the command type being an immediate data command IDC, determining the high water mark and the low water mark based on a global limit; as well as In response to the command type being a direct memory access (DMA) command, the high water level and the low water level are determined based on limitations specific to the corresponding injector.

4. The network interface controller of claim 3, wherein: The command is issued to the network interface controller via a peripheral component interconnect express (PCIe) interface.

5. The network interface controller of claim 1, wherein: The prioritization logic block is further configured to: assigning a high priority to the corresponding injector in response to detecting a priority reset; and The respective injector is assigned a low priority in response to detecting expiration of a timer associated with the respective injector.

6. The network interface controller of claim 1, wherein: The selection logic block is further configured to select the buffer class from a set of buffer classes enabled for the network interface controller.

7. The network interface controller of claim 1, wherein: The injector is a message segmentation unit MCU, which is used to generate packets according to commands issued to the network interface controller.

8. The network interface controller of claim 1, wherein: The output buffer is divided into a plurality of cells; and Wherein, injecting the packet includes injecting the packet into the next available cell.

9. A method for injecting a packet into an output buffer in a network interface controller NIC, comprising: identifying a plurality of injectors in the NIC that share the output buffer; determining a priority associated with a respective injector based on a high water level and a low water level associated with the respective injector; determining, from the plurality of injectors, a subset of injectors associated with a buffer class; determining whether the subset of injectors includes a high priority injector; as well as In response to identifying a high priority injector in the subset of injectors, selecting the high priority injector for injecting packets into the output buffer, Wherein, the method further comprises: obtaining the number of cells in the output buffer occupied by data from the corresponding injector; assigning a high priority to the corresponding injector in response to the number of cells being less than or equal to the low water level; and In response to the number of cells being greater than or equal to the high water mark, the corresponding injector is assigned a low priority.

10. The method of claim 9, in response to determining that the subset of injectors does not include a high priority injector, the method further comprising selecting a low priority injector for injecting packets into the output buffer.

11. The method of claim 9, further comprising: determining a command type associated with a corresponding injector; In response to the command type being an immediate data command IDC, determining the high water mark and the low water mark based on a global limit; as well as In response to the command type being a direct memory access (DMA) command, the high water level and the low water level are determined based on limitations specific to the corresponding injector.

12. The method of claim 11, wherein: Commands are issued to the NIC via a Peripheral Component Interconnect Express (PCIe) interface.

13. The method of claim 9, further comprising: assigning a high priority to the corresponding injector in response to detecting a priority reset; as well as The respective injector is assigned a low priority in response to detecting expiration of a timer associated with the respective injector.

14. The method of claim 9, further comprising selecting the buffer class from a set of buffer classes enabled for the NIC.

15. The method of claim 9, wherein: The injector is a message segmentation unit MCU, which is used to generate packets according to commands issued to the NIC.

16. The method of claim 9, wherein: The output buffer is divided into a plurality of cells; and Wherein, injecting the packet includes injecting the packet into the next available cell.

Images