Implementing load balancing between network devices using queue sharing

Abstract

The invention relates to implementing load balancing between network devices using queue sharing. A system includes one or more processors, and a plurality of network devices for connecting the one or more processors to a network. The one or more processors are configured to issue work requests to a plurality of network devices by issuing work descriptors to one or more shared queues, each shared queue being accessible by the plurality of network devices. The network device is used for extracting the work descriptors from the one or more shared queues and executing the work request in response to the work descriptors.

Description

TECHNICAL FIELD

[0001] The present specification generally relates to network communications, and particularly relates to methods and systems for implementing load balancing among network devices. BACKGROUND

[0002] In some communication systems, a processor or a group of processors can connect to a network using multiple network devices. One example of such a system is a graphics processing unit (GPU) that connects to a network using two network interface controllers (NICs) or data processing units (DPUs). SUMMARY

[0003] One embodiment described herein provides a system comprising (i) one or more processors and (ii) a plurality of network devices for connecting the one or more processors to a network. The one or more processors are configured to issue work requests to the plurality of network devices by publishing work descriptors to one or more shared queues, where each queue is accessible to the plurality of network devices. The network devices extract work descriptors from the one or more shared queues and perform the work requests in response to the work descriptors.

[0004] Generally, when publishing a work descriptor to a shared queue, the processor is configured to make the work descriptor available to any of the plurality of network devices. In some embodiments, after publishing a work descriptor to a shared queue, the processor is configured to issue a "doorbell" to the plurality of network devices, notifying the plurality of network devices that the work descriptor has been published. Generally, the network devices are configured to extract work descriptors from the shared queues without being assigned work descriptors by the one or more processors.

[0005] In one embodiment, the network devices are configured to estimate a communication load experienced by the network devices and extract work descriptors according to defined criteria in response to finding that the estimated communication load is sufficiently low.

[0006] In disclosed embodiments, the one or more shared queues are a plurality of shared queues, each of which is accessible to the plurality of network devices; the one or more processors are configured to issue work requests to the plurality of network devices by publishing work descriptors to the plurality of shared queues; and the network devices are configured to select a queue from at least the plurality of shared queues according to quality of service (QoS) criteria and extract work descriptors from the selected queue.

[0007] In some embodiments, the shared queue is associated with a shared read pointer that indicates a head of the shared queue, and the network devices are configured to increment the read pointer using an atomic fetch-and-add command when attempting to extract a work descriptor from the shared queue. In one embodiment, the atomic fetch-and-add command specifies a limit that prevents the incremented read pointer from exceeding a write pointer of the shared queue. In one example embodiment, the one or more processors and the plurality of network devices communicate over one or more peripheral buses using a bus communication protocol; and the atomic fetch-and-add command is implemented as an extension of the bus communication protocol.

[0008] In certain embodiments, the read pointer is stored in memory of one of the network devices, and the network devices are configured to execute the atomic fetch-and-add command in memory of the one of the network devices. In one example embodiment, the network devices are configured to execute the atomic fetch-and-add command over a dedicated peer-to-peer connection between the network devices.

[0009] In an alternative embodiment, the read pointer is stored in memory of one of the processors, and the network devices are configured to execute the atomic fetch-and-add command in memory of the one of the processors. In one embodiment, in response to discovering that the read pointer exceeds the write pointer of the shared queue after executing the atomic fetch-and-add command, the network devices are configured to roll back the incremented read pointer. In other embodiments, in response to discovering that the read pointer exceeds the write pointer of the shared queue after executing the atomic fetch-and-add command, the network devices are configured to wait for the processor to issue a new work descriptor, and then extract the new work descriptor.

[0010] According to embodiments described herein, there is also provided a method comprising issuing work requests from one or more processors to a plurality of network devices that connect the one or more processors to a network by issuing work descriptors to one or more shared queues, each of which is accessible by the plurality of network devices. In the plurality of network devices, work descriptors are extracted from the one or more shared queues, and work requests are performed in response to the work descriptors.

[0011] The present description will become more fully understood from the detailed description given herein below, and accompanied by the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 For illustrative block diagrams, a computing system employing load balancing between two network devices using queue sharing is shown according to embodiments described herein;

[0013] Figure 2 For illustrative block diagrams, a computing system employing load balancing between four network devices using queue sharing is shown according to alternative embodiments described herein;

[0014] Figure 3 A diagram for a method of load balancing between two network devices using queue sharing according to embodiments described herein;

[0015] Figure 4 A diagram for a consumer index (CI) overrun scenario during queue sharing according to embodiments described herein;

[0016] Figure 5 A diagram for a joint load balancing and quality of service (QoS) method according to embodiments described herein. DETAILED DESCRIPTION

[0017] SUMMARY

[0018] Various existing and emerging computing system configurations contain multiple network devices, such as network adapters or data processing units (DPUs), that collectively serve one processor or a group of processors. As communication rates increase, it becomes critical to efficiently utilize the resources of the network devices. It is particularly important to balance the communication load among the network devices. A well-balanced group of network devices can provide superior performance, such as high throughput, low latency, low jitter, and fast completion of jobs involving multiple network operations.

[0019] Embodiments described herein provide methods and systems for balancing communication load among multiple network devices. The disclosed technology balances the load by using (i) shared queues that are accessible to each of the network devices, and (ii) a“work-stealing” scheme in which the network devices pull work requests from the shared queues, rather than being pushed work requests. The following description primarily uses network adapters as an example, but the disclosed technology is applicable to any other suitable type of network device.

[0020] The disclosed technology is also applicable to various types of accelerators (e.g., compression accelerators, cryptography accelerators, etc.) that are connected to a processor. In general, the disclosed technology utilizes“work-stealing” from shared queues and can be used to achieve load balancing among various peripheral devices that serve one or more processors. The peripheral devices can include, for example, network devices and accelerators.

[0021] In a typical embodiment, a system includes one or more processors that are connected to a network through multiple network adapters. The processors issue work requests to the network adapters by posting work descriptors onto one or more shared queues, each of which is accessible to multiple network adapters. The network devices pull work descriptors from the shared queues and perform the corresponding work requests.

[0022] In this context, the term “work descriptor” refers to a data item posted to a work queue in response to a work request. In one typical (but not limiting) example, a work request is generated by an application program and a corresponding work descriptor is posted to a work queue by a software driver associated with a network device. The work descriptor can include or point to any suitable information related to the work request to be performed. Such information can include, for example, the type of operation to be performed, related addresses, data, metadata, and / or any other suitable information. Fetching a work descriptor from a queue by a network device typically involves reading that work descriptor.

[0023] In one example embodiment, each network adapter continuously estimates its current or expected communication load. When the estimated load is low enough (e.g., below a defined threshold), the network adapter will fetch one or more work descriptors from one of the shared queues and perform the corresponding work requests. An example technique for load estimation that can be used for this purpose is described in U.S. Patent Application 18 / 638,756, entitled “Load balancing between network devices based on communication load,” which is assigned to the assignee of the present patent application, the disclosure of which is incorporated herein by reference.

[0024] In this way, each network adapter is responsible for ensuring that its resources are used efficiently. The processor does not pre-assign work requests and work descriptors to any particular network adapter. The processor can not even know the identity of the network adapter that performs a particular work request, or even that the work is being distributed among multiple network adapters.

[0025] In some embodiments, a given shared queue has a write pointer (also referred to as a producer index - PI) and a read pointer (also referred to as a consumer index - CI) stored in memory. The write pointer points to the next location in the queue where the processor is to write. The read pointer is accessible to multiple network adapters and points to the next location in the queue where the network adapters are to read. Upon reading a work descriptor from a shared queue, a network adapter typically uses an atomic fetch-and-add command to increment the queue’s read counter. Several techniques for preventing the read pointer from exceeding the write pointer (e.g., due to race conditions among the network adapters) are presented herein.

[0026] In a typical implementation, the system includes multiple shared queues, each of which can have pending work descriptors. In some embodiments, the network adapter selects a queue to service according to defined quality of service (QoS) criteria. In other words, the network adapter services the shared queues while ensuring QoS and load balancing. In one example embodiment, the network adapter first uses QoS criteria to select one of the shared queues that has pending work descriptors, and then verifies that its communication load is low enough to service the selected queue. When this order of operations (QoS decision first, then load balancing decision) is employed, the network adapter has visibility into all pending work, and work is not committed to the network adapter until the network adapter makes a QoS decision (i.e., selects which queue to service). As a result, the network adapter can make better QoS decisions.

[0027] System Description

[0028] Figure 1 is a block diagram schematically illustrating a computing system 20 that uses queue sharing to employ load balancing between two network devices in accordance with embodiments described herein. The network devices can include network adapters, such as Ethernet network interface controllers (NICs) or InfiniBand (IB) host channel adapters (HCAs). Alternatively, the techniques of the present disclosure can be used with other suitable types of network devices, such as data processing units (DPUs, also known as “smart NICs”), or with suitable peripheral devices, such as accelerators (compression accelerators, cryptography accelerators, etc.). TM

[0029] In Figure 1 embodiments, the system 20 includes a processor 24 and two NICs 28. The processor 24 uses the NICs 28 to send and receive communication traffic over a network 32. The processor 24 can include, for example, a central processing unit (CPU), a graphics processing unit (GPU), or any other suitable type of processor. To simplify the illustration, the following description is primarily directed to a single processor. In alternative embodiments, the disclosed techniques can be used with a group of processors that collectively communicate through the network adapters 28. In the present example, the system 20 includes two NICs, but any other suitable number of NICs can also be used.

[0030] ​Each NIC 28 communicates with the processor 24 over a peripheral bus 36. In the present example, the bus 36 is a Peripheral Component Interconnect Express (PCIe) bus. Alternatively, any other suitable peripheral bus can be used, such as NVLINK or Compute Express Link (CXL). Each NIC communicates with the network 32 using one or more network ports 40. Additionally, alternatively, any of the NICs 28 can be connected to the processor 24 by a direct connection, i.e., not via a peripheral bus.

[0031] A given NIC typically includes a host interface for communicating with the processor 24 over the bus 36, one or more network interfaces for communicating with the network 32, and circuitry to perform various processing tasks of a network adapter.

[0032] The system 20 also includes a memory 44, typically a random access memory (RAM). The memory 44 is accessible by the processor 24 and the NICs 28. The processor 24 maintains one or more shared work queues (WQs) 48 and one or more shared completion queues (CQs) 52 in the memory 44. Another embodiment using different completion notification semantics in place of shared CQs will be presented below. In the present example, the WQs 48 and CQs 52 are accessible by NIC1 and NIC2. When a given NIC supports multiple ports, physical functions (PFs), and / or virtual functions (VFs), the shared WQs are typically shared among the various ports, PFs, and / or VFs.

[0033] For clarity, only a single shared WQ 48 and a single CQ 52 are shown in the figure. In many practical implementations, multiple shared WQs 48 and multiple CQs 52 can be used. Certain aspects of manipulating multiple sharded work queues (WQs) in the context of quality of service (QoS) will be discussed further below. In the present example, the shared WQ 48 is stored in the memory 44 of the processor 24. Alternatively, the shared WQ 48 can also be stored in any other suitable location, e.g., in the memory of one of the NICs 28.

[0034] When the shared WQ 48 is stored in the memory of one of the NICs, the other NICs 28 typically access the shared WQ using peer-to-peer communication. In one embodiment, the peer-to-peer communication is over the bus 36. In another embodiment, the peer-to-peer communication is over a dedicated peer-to-peer connection between the NICs 28, independent of the bus 36.

[0035] When the system includes multiple processors 24, each processor 24 typically has its own set of (one or more) shared WQs 48 and CQs 52. In other words, the term "shared" refers to the fact that the queues are accessible to multiple network adapters, not to multiple processors. More generally, however, the disclosed technology can also be used in a scheme in which different processors 24 can post work descriptors on a given queue. In some embodiments, the system can also include one or more WQs that are not shared, i.e., that are associated with a particular NIC.

[0036] In a typical embodiment, a processor 24 issues a work request (WR) to a NIC 28 by posting a work queue element (WQE) 56 on a shared WQ 48. For example, the WQE can request the NIC to perform a remote direct memory access (RDMA) write (WRITE) transaction that writes particular data to a remote memory over the network 32. As another example, the WQE can request the NIC to perform an RDMA read (READ) transaction that fetches particular data from a remote memory over the network 32. Other suitable types of WQE (e.g., send (SEND)) can also be used.

[0037] The NIC 28 extracts the WQE from the shared WQ and performs the corresponding WR. When a NIC completes a WR, the NIC returns a completion notification to the processor, e.g., by posting a completion queue element (CQE) on a CQ 52. In another typical embodiment, the completion notification is implemented in the form of incrementing a counter by a value. The counter address or index and the value can be defined in the WR. This mechanism is sometimes referred to as a "count event," e.g., as described in "Portals 4.0 Network Programming Interface" (Sandia National Laboratories, November 2012). See Section 3.14 in particular.

[0038] In certain embodiments, the CQs 52 are also shared among the NICs 28. In one such embodiment, before posting a CQE to a shared CQ, a NIC first reserves ("steals") a location in the shared CQ for the CQE to be posted. The NIC can reserve a location, e.g., by performing an atomic fetch-and-add on the CQ's producer index (PI). The CQE can then be posted at the reserved location. In alternative embodiments, the system 20 can use completion notification semantics (e.g., the "count event" mechanism described above) instead of using shared CQs. In such embodiments, once a NIC completes processing of a given WQE, the NIC increments a count event by a value. The count event and the increment value are both specified in the WQE.

[0039] The embodiments described herein relate primarily to WQs, CQs, WQEs, and CQEs, by way of non-limiting example only. The disclosed technology can be used with any other suitable type of queue, work descriptor, and completion notification. Thus, in this context, the terms "WQ" and "WQE" are considered to be examples of a queue and a work descriptor, respectively. Although certain terms in the following description are commonly used in InfiniBand (IB) networks, the disclosed technology is by no means limited to any particular communication protocol or network type. TM (IB) networks, the disclosed technology is by no means limited to any particular communication protocol or network type.

[0040] As shown in Figure 1 , the shared WQ 48 stores one or more valid WQEs 56. The WQ 48 has a read pointer, shown in the figure as a consumer index (CI), that points to the head of the WQ (the next location from which the NIC 28 is to read). The WQ 48 also has a write pointer, shown in the figure as a producer index (PI), that points to the tail of the WQ (the next location to which the processor 24 is to write). In some embodiments, both the CI and the PI are stored in the memory 44. In other embodiments, at least the CI is stored in the memory of one of the NICs 28 - this embodiment will be described in further detail below.

[0041] Figure 2 As a block diagram, the figure schematically shows a computing system 60 that employs load balancing among four network adapters using queue sharing, according to alternative embodiments described herein. The system 60 includes one CPU 64, two GPUs 68 (labeled GPU1 and GPU2, respectively), and four NICs 28 (labeled NIC1-NIC4, respectively).

[0042] In this example, the CPU 64 is connected to the GPU1 and GPU2 through suitable communication interfaces. The NIC1 is connected to the GPU1 through a PCIe link 36, the NIC2 and NIC3 are connected to the CPU 64 through two respective PCIe links 36, and the NIC4 is connected to the GPU2 through a fourth PCIe link 36. Given this physical connection, the CPU 64 is able to exchange communication traffic with the network 32 through any of the four NICs 28 (NIC1-NIC4). For clarity, the memory and shared queues are not shown in the figure.

[0043] As shown in Figure 1 and Figure 2 , the phrase "processor exchanges communication traffic through a network adapter" is used in various grammatical forms to refer to both a direct physical connection between the processor and the network adapter, as well as an indirect physical connection. In the example of Figure 2 , the CPU 64 can use the disclosed technology to balance the load of communication traffic exchanged through the NIC1-NIC4.

[0044] As Figure 1 and Figure 2 illustrated, the configurations of systems 20 and 60 are merely example configurations, selected for conceptual clarity. In alternative embodiments, any other suitable configuration can be used. Elements unnecessary for understanding the principles of the disclosed solution have been omitted from the figures for clarity.

[0045] The various elements of systems 20 and 60, including the disclosed various processors and network adapters, can be implemented in hardware (e.g., one or more application specific integrated circuits (ASICs) or FPGAs), in software, or using a combination of hardware and software elements. In some embodiments, certain elements of the disclosed processors and network adapters can be implemented, in part or in whole, using one or more general purpose processors programmed with software to perform the functions described herein. The software can be downloaded to any of the processors in electronic form (over the internet, for example) or alternatively or additionally it can be provided and / or stored on non-transitory tangible media (e.g., magnetic, optical or electronic memory).

[0046] Implementing load balancing between network devices using queue sharing

[0047] Figure 3 For a flowchart, the flowchart schematically illustrates a method of implementing load balancing between two network adapters using queue sharing, according to embodiments described herein. In this example, the method is performed by processor 24 and NIC 28 of system 20 Figure 1 ).

[0048] The method begins with processor 24 having a new work request (WR) to be performed by NIC 28. In a WQE posting stage 70, processor 24 posts a WQE on shared WQ 48, the WQE describing the new WR. In a doorbell sounding stage 74, processor 24 sounds a doorbell to each of NIC1 and NIC2, informing the NICs that a new WQE has been posted on shared WQ 48. The doorbell can specify any relevant information, such as the current value of the PI of the shared WQ.

[0049] NIC1 and NIC2 each respond to the doorbell and independently of the other NIC perform the following stages (78-102). In a load estimation stage 78, the NIC estimates its current or expected communication load. Any suitable method can be used for this purpose, such as the method described in U.S. Patent Application 18 / 638,756, referenced above. Example types of communication load that can be estimated and used for load balancing include:

[0050] "Outbound load" - the total amount of data provided to the NIC to send to the network but not yet completed. In one example embodiment, the outbound load can be estimated as the cumulative total number of bytes on all WRITE WQEs fetched by the NIC but not yet completed. Within the total number of bytes of incomplete WRITE WQEs, a more accurate estimate would exclude the number of bytes that have already been sent to the network.

[0051] "Inbound load" - the total amount of data that the NIC has requested to read through the network but not yet completed. An example estimate of the inbound load includes the cumulative total number of bytes on all READ WQEs fetched by the NIC but not yet completed. Again, within the total number of bytes of incomplete READ WQEs, a more accurate estimate would exclude the number of bytes that have already been acquired through the network.

[0052] At the load check stage 82, the NIC checks whether the estimated load is low enough to warrant obtaining additional work from the shared WQ 48. For example, the NIC can compare the estimated load to a defined threshold (which can be fixed or adaptive).

[0053] If the estimated load is not low enough, the NIC will wait for the load to drop at the wait stage 86, and then loop back to stage 78. On the other hand, if the estimated load is low enough, the NIC will fetch a new WQE from the head of the shared WQ 48 at the fetch stage 90. To fetch the new WQE, the NIC reads the value of the CI, and then reads the WQE from the location pointed to by the CI.

[0054] After fetching the WQE, the NIC increments the CI to point to the next WQE. In some embodiments, the NIC uses an atomic get-and-add command to atomically increment the CI. The term "atomic" here means that no other entity can modify the CI until the get-and-add command is completed. The command typically specifies an "add value" by which the value of the CI will be incremented.

[0055] In one embodiment, the NIC can use a single atomic get-and-add command to fetch multiple WQEs from the shared WQ. In this embodiment, the "add value" in the command indicates the number of WQEs to fetch. As a result of executing the get-and-add command, the NIC receives the CI. The NIC fetches the WQEs from this CI.

[0056] At the CI overrun check stage 94, the NIC checks whether the incremented CI (read pointer) overruns the PI (write pointer) of the shared WQ. This overrun can occur when the NIC loses a race to extract the same WQE from the shared WQ, but still increments the CI. If an overrun is detected, the NIC initiates a corrective action at the corrective action stage 98. Examples of overrun conditions and possible corrective actions are discussed further below.

[0057] If no CI overrun is detected at stage 94, the NIC performs a new WR according to the newly extracted WQE at the WR execution stage 102.

[0058] Preventing / correcting CI overrun

[0059] Figure 4 A consumer index (CI) overrun scenario that occurs during queue sharing according to embodiments described herein is schematically illustrated.

[0060] The left side of the figure (labeled 106) shows the state of the shared WQ 48 storing a single WQE 56. In this state, CI = 2, PI = 3.

[0061] The right side of the figure (labeled 110) shows the state of the shared WQ after the NIC attempts to extract the WQE 56, fails due to another NIC extracting the same WQE, but still increments the CI (incorrectly). In this case, the CI has been incremented twice (once by the other NIC that successfully extracted the WQE, and once by the NIC that failed in the WQE race). As a result, PI = 3, CI = 4, which overruns the PI.

[0062] It can be appreciated that the situation where CI overruns PI is an undesirable situation. In this document, the term "overrun condition" refers to any relationship between CI and PI that needs to be prevented or corrected. In some embodiments, CI overrunning PI is not allowed, i.e., CI > PI is defined as an overrun condition. A more stringent overrun condition is CI > PI, i.e., CI equal to PI is also not allowed. In other embodiments, a more relaxed overrun condition can be CI > PI + N, where N is a predefined integer, e.g., 1, 2, or 3 (i.e., CI is allowed to overrun PI by a certain predefined integer N). Further, other suitable overrun conditions can also be defined instead.

[0063] In various embodiments, the NIC 28 can take various preventive or corrective measures to prevent or correct CI overrun.

[0064] In some embodiments, after extracting a WQE from the shared WQ 48, the NIC 28 increments the CI using a new atomic fetch-and-add command, referred to herein as "fetch-and-add with limit value." This command specifies (i) an "add value" by which the CI is to be incremented, and (ii) a limit value against which the incremented CI is to be checked. For example, if the current value of the PI is 3 (as shown in the example of FIG. 3), and the applicable overlimit condition is CI > PI, then the NIC can use the "fetch-and-add with limit value" to increment the CI, with the limit value set to 3. If the expected value of the CI exceeds the limit value specified in the command, then the CI is not incremented, and the command returns an indication that the CI was not incremented. Figure 4

[0065] In one example embodiment, the "fetch-and-add with limit value" command is formatted as an extension of a peripheral bus protocol (e.g., PCIe or NVLINK) used for communication over the bus 36. In alternative embodiments, any other suitable format can be used.

[0066] When using the "fetch-and-add with limit value" command, it is preferable that the CI be stored in memory of one of the NICs 28. In this configuration, the "fetch-and-add with limit value" command is issued in a peer-to-peer manner between the NICs, and does not require that the processor 24 support it. Alternatively, the CI can be stored in the memory 44 of the processor 24, or any other suitable location. Furthermore, alternatively, the CI can also be stored in any suitable location when using the conventional atomic fetch-and-add command (without a limit value).

[0067] In alternative embodiments, a NIC that causes the CI to overlimit can correct the overlimit by rolling back the CI (i.e., decrementing the CI back to its previous value).

[0068] In another embodiment, a NIC that causes the CI to overlimit takes ownership of the erroneously incremented CI (which now points to a free slot in the shared WQ), and waits for the processor 24 to issue a new WQE. The NIC then extracts the new WQE and performs the corresponding WR.

[0069] Furthermore, alternatively, the NICs 28 can use any other suitable technique to prevent or correct CI overlimit conditions.

[0070] Joint Load Balancing and Quality of Service (QOS)

[0071] ​As noted above, in some embodiments, system 20 includes multiple shared WQs 48. For example, a given processor 24 can maintain multiple shared WQs 48. As another example, a system can include multiple processors 24, each of which maintains one or more shared WQs 48. Thus, at a given time, a given NIC 28 can observe multiple shared WQs 48 that have pending WQEs.

[0072] In various embodiments, NIC 28 serves multiple shared WQs 48, as well as possibly one or more non-shared WQs, according to defined quality of service (QoS) criteria. Any suitable QoS criteria can be used. Example criteria include:

[0073] Ensuring uniform fairness among shared WQs.

[0074] Ensuring that traffic load is allocated among shared WQs in a specified manner (not necessarily uniformly).

[0075] Ensuring that a given shared WQ is allocated at least a specified percentage of traffic bandwidth.

[0076] Maintaining specified relative priorities among shared WQs.

[0077] In various embodiments, NIC 28 can combine load balancing with QoS in different ways. In one example embodiment, QoS decisions are made first, and load balancing decisions second. In other words, a given NIC 28 first selects a WQ from among multiple WQs (shared WQs and non-shared WQs, if any) to service. The NIC then checks whether its estimated communication load is small enough to fetch a new WQE. If so, the NIC fetches a WQE from the selected WQ.

[0078] Figure 5 is a flowchart that schematically illustrates a method of joint load balancing and quality of service (QoS), in accordance with an embodiment described herein. The method is typically performed by each NIC 28 independently of the other NICs. For simplicity, the following description refers only to shared WQs. In general, the method can be applied jointly to multiple WQs, some of which are shared and others of which are non-shared.

[0079] The method begins with a pending work identification phase 114, in which the NIC identifies which of the multiple shared work queues currently have pending WQEs. In a QoS selection phase 118, the NIC selects one of the shared WQs that have pending WQEs. The selection can be performed according to any suitable QoS criteria (e.g., the criteria listed above).

[0080] In some embodiments, after selecting a shared WQ, the NIC checks the current congestion state of the selected WQ to see if transmission from that WQ is allowed. If not, the NIC repeats stage 118, i.e. selects a different shared WQ to service using the QoS criteria. In alternative embodiments, the NIC considers the congestion state of each WQ as part of the QoS selection of stage 118 (i.e. applies the QoS criteria only to the WQ from which transmission is first allowed).

[0081] In a load estimation stage 122, the NIC estimates its current or expected communication load. The load calculation can depend on the selected WQ. In a load check stage 126, the NIC checks whether the estimated load is low enough to warrant obtaining additional work from the shared WQ 48. If the estimated load is not low enough, the NIC waits for the load to drop in a wait stage 130, and then loops back to stage 122. Stages 122-130 are parallel to stages 78-86 in the above Figure 3

[0082] If the estimated load is low enough, in an extract stage 134, the NIC extracts a WQE from the selected shared WQ 48 (i.e. the shared WQ selected in stage 118 above). In an execute stage 138, the NIC executes the WR described by the new WQE.

[0083] It is understood that the above-described embodiments are intended to be illustrative only and in no way limit the scope of the present application. Rather, the scope of the present application encompasses all features and sub-combinations thereof as well as modifications and variations thereof which would occur to those skilled in the art upon reading the above description. The references cited in this patent application are to be considered part of the disclosure of this application but, if the definition of any term in these references conflicts with the explicit or implicit definition in the present specification, only the definition in the present specification is to be considered.​

Claims

1. A system comprising (i) one or more processors, and (ii) a plurality of network devices for connecting the one or more processors to a network, wherein: the one or more processors are for issuing work requests to the plurality of network devices by publishing work descriptors onto one or more shared queues, each shared queue being accessible by the plurality of network devices; and the network devices are for extracting the work descriptors from the one or more shared queues and performing the work requests in response to the work descriptors.

2. The system of claim 1, wherein, In publishing a work descriptor onto the shared queue, the processor is for making the work descriptor obtainable by any of the plurality of network devices.

3. The system of claim 1, wherein, In publishing a work descriptor onto a shared queue, the processor is for issuing a doorbell to the plurality of network devices, informing the plurality of network devices that the work descriptor has been published.

4. The system of claim 1, wherein, The network devices do not extract work descriptors from the shared queues in response to the work descriptors being assigned to the network devices by the one or more processors.

5. The system of claim 1, wherein, The network devices are for estimating a communication load experienced by the network devices and for extracting work descriptors in response to finding that the estimated communication load is sufficiently low according to a defined criterion.

6. The system of claim 1, wherein: the one or more shared queues are a plurality of shared queues, each shared queue being accessible by the plurality of network devices; the one or more processors are for issuing the work requests to the plurality of network devices by publishing the work descriptors onto the plurality of shared queues; and the network devices are for selecting a queue from at least the plurality of shared queues according to a quality of service, QoS, criterion and for extracting work descriptors from the selected queue.

7. The system of claim 1, wherein, A shared queue is associated with a shared read pointer indicating a head of the shared queue, and wherein, in attempting to extract a work descriptor from the shared queue, the network devices are for incrementing the read pointer using an atomic fetch-and-add command.

8. The system of claim 7, wherein, The atomic fetch-and-add command specifies a limit value that prevents the incremented read pointer from exceeding a write pointer of the shared queue.

9. The system of claim 8, wherein: the one or more processors and the plurality of network devices communicate over one or more peripheral buses using a bus communication protocol; and the atomic fetch-and-add command is implemented as an extension to the bus communication protocol.

10. The system of claim 7, wherein, The read pointer is stored in a memory of one of the network devices, and wherein the network devices are for executing the atomic fetch-and-add command in the memory of the one of the network devices.

11. The system of claim 10, wherein, The network devices are for executing the atomic fetch-and-add command over a dedicated peer-to-peer connection between the network devices.

12. The system of claim 7, wherein, The read pointer is stored in a memory of one of the processors, and wherein the network devices are for executing the atomic fetch-and-add command in the memory of the one of the processors.

13. The system of claim 7, wherein, The network device is to roll back the incremented read pointer in response to discovering that the read pointer exceeds the write pointer of the shared queue after the atomic fetch-and-add command.

14. The system of claim 7, wherein, In response to discovering that the read pointer exceeds the write pointer of the shared queue after the atomic fetch-and-add command, the network device is to wait for a processor to issue a new work descriptor and then fetch the new work descriptor.

15. A method comprising: issuing work requests from one or more processors to a plurality of network devices connecting the one or more processors to a network by issuing work descriptors onto one or more shared queues, wherein each shared queue is accessible by the plurality of network devices; and in the plurality of network devices, fetching the work descriptors from the one or more shared queues and performing the work requests in response to the work descriptors.

16. The method of claim 15, wherein, Issuing a work descriptor onto a shared queue comprises making the work descriptor available to any of the plurality of network devices.

17. The method of claim 15, further comprising: After issuing a work descriptor onto a shared queue, ringing a doorbell to the plurality of network devices to inform the plurality of network devices that the work descriptor has been issued.

18. The method of claim 15, wherein, Fetching a work descriptor from a shared queue by a network device is not performed in response to the work descriptor being assigned to the network device by the one or more processors.

19. The method of claim 15, wherein: the one or more shared queues are a plurality of shared queues, each shared queue being accessible by the plurality of network devices; issuing the work requests to the plurality of network devices comprises issuing the work descriptors onto the plurality of shared queues; and fetching the work descriptors comprises selecting a queue from at least the plurality of shared queues according to a quality of service, QoS, criteria and fetching a work descriptor from the selected queue.

20. The method of claim 15, wherein, A shared queue is associated with a shared read pointer indicating a head of the shared queue, and the method comprises incrementing the read pointer using an atomic fetch-and-add command after attempting to fetch a work descriptor from the shared queue.

Images