Peripheral device with loose sequence bus interface

Abstract

The invention discloses a peripheral device with a loose sequence bus interface. Peripheral devices include a bus interface and circuitry. The bus interface is to exchange bus transactions through a peripheral bus that allows disordered transfer of at least a portion of the bus transactions. The circuitry is to generate a plurality of streams of bus transactions, select one or more streams from the plurality of streams that require transaction ordering, enforce transaction ordering in the bus transactions of the selected streams, and send the bus transactions to the peripheral bus over the bus interface.

Description

Technical Field

[0001] The present invention relates generally to computing systems communicating via a peripheral bus, and more particularly to methods and systems for applying ordering between peripheral bus transactions. Background Art

[0002] A computing system typically includes peripheral devices connected to a host or other processing device via a peripheral bus. Peripheral devices may include, for example, network adapters, storage devices, accelerators, and graphics processing units (GPUs). A peripheral bus, also known as a system bus, may include, for example, Peripheral Component Interconnect Express (PCIe), Advanced Extensible Interface (AXI), Compute Express Link (CXL), NVLink, or NVLink chip-to-chip (NVLink-C2C). Summary of the Invention

[0003] Embodiments of the invention described herein provide a peripheral device including a bus interface and circuitry. The bus interface is configured to exchange bus transactions over a peripheral bus that allows out-of-order transmission of at least some bus transactions. The circuitry is configured to generate a plurality of bus transaction streams, select one or more streams requiring transaction ordering from the plurality of streams, enforce transaction ordering on the bus transactions of the selected streams, and send the bus transactions to the peripheral bus via the bus interface.

[0004] In some embodiments, circuitry is configured to enforce transaction ordering within a selected stream by inserting one or more isolate commands into bus transactions of the selected stream. In a disclosed embodiment, the one or more isolate commands are stream-specific isolate commands applicable only to the selected stream. In another embodiment, in response to detecting that the number of currently processed isolate commands exceeds a defined maximum number, circuitry is configured to halt processing of any stream requiring insertion of a new isolate command. In an alternative embodiment, circuitry is configured to enforce transaction ordering within the selected stream by marking bus transactions of the selected stream with a unique tag, and pausing new bus transactions marked with the unique tag at a selected time until a previous transaction marked with the unique tag is acknowledged.

[0005] In some embodiments, circuitry is configured to assign flows to two or more groups and enforce transaction ordering among bus transactions of a given group among the plurality of groups. In disclosed embodiments, circuitry is configured to enforce transaction ordering within a given group by inserting one or more isolate commands into bus transactions of flows belonging to the given group. In an example embodiment, the one or more isolate commands are group-specific isolate commands applicable only to the given group. In disclosed embodiments, in response to detecting that the number of currently processed isolate commands exceeds a defined maximum number, circuitry is configured to halt processing of any group requiring insertion of a new isolate command.

[0006] In other embodiments, circuitry is provided to enforce transaction ordering within a given group by tagging bus transactions of the given group with a unique tag, and to pause new bus transactions tagged with the unique tag at selected times until prior transactions tagged with the unique tag are acknowledged.

[0007] In one embodiment, the circuitry assigns flows to groups by queuing bus transactions into two or more queues, respectively, associated with the two or more groups. In another embodiment, the circuitry enforces transaction ordering by applying one or more intra-group ordering rules to bus transactions belonging to a given group.

[0008] In yet another embodiment, in addition to transaction ordering within each group, the circuitry is further configured to enforce selective transaction ordering between bus transactions belonging to different groups according to one or more inter-group ordering rules. In another embodiment, the circuitry is configured to assign flows to groups according to one or more group assignment rules.

[0009] In a disclosed embodiment, circuitry is used to assign bus transactions to flows according to one or more flow assignment rules. In one embodiment, a peripheral bus that allows out-of-order transfers includes two or more separate bus links, each requiring in-order transfer of at least some bus transactions.

[0010] In some embodiments, the peripheral device is a network adapter configured to communicate with the processing device via a peripheral bus for coupling the processing device to a network. In an example embodiment, the circuitry is configured to send or receive communication packets over the network in accordance with a Remote Direct Memory Access (RDMA) protocol. In some embodiments, the bus interface is configured to communicate with the processing device via the peripheral bus, and the circuitry is configured to enforce transaction ordering in the selected flow by applying one or more ordering rules defined between the peripheral device and a device driver running in the processing device.

[0011] According to one embodiment of the present invention, a method is also provided, comprising: generating, in a peripheral device, a plurality of bus transaction streams for transmission via a peripheral bus that allows at least some bus transactions to be transmitted out of order; selecting, from the plurality of streams, one or more streams requiring transaction ordering; enforcing, in the peripheral device, transaction ordering among the bus transactions of the selected streams; and sending, from the peripheral device, the bus transactions to the peripheral bus.

[0012] The present invention will be more fully understood through the following detailed description of the embodiments of the present invention in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a block diagram schematically illustrating a computing system using relaxed-order peripheral buses according to one embodiment of the present invention;

[0014] Figure 2 is a schematic diagram showing an embodiment of the present invention Figure 1 a block diagram of a requester side of a computing system;

[0015] Figure 3 is a schematic diagram showing an embodiment of the present invention Figure 1 a block diagram of a responder side of a computing system;

[0016] Figure 4 is a block diagram schematically illustrating a network adapter according to one embodiment of the present invention;

[0017] Figure 5 is a schematic diagram showing an embodiment of the present invention Figure 4 A block diagram of the sequencing logic in a network adapter;

[0018] Figure 6A and Figure 6B is a schematic diagram showing an embodiment of the present invention. Figure 5 The sequencing logic is executed in a flow-specific isolated message graph;

[0019] Figure 7A and Figure 7B is a diagram schematically illustrating an example allocation of bus transactions to flows according to an embodiment of the present invention;

[0020] Figure 8 is a diagram schematically illustrating dividing a flow queue into sub-queues according to an embodiment of the present invention; and

[0021] Figure 9 is a message diagram schematically illustrating data read / write sequencing according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] Overview

[0023] When a peripheral device communicates with a processing device (such as a host) over a peripheral bus, certain sequences of bus transactions must be delivered in order from the peripheral device to the processing device. For example, consider a network adapter that connects a host to a network. In a simple example, while performing a job for the host, the network adapter writes data to the host memory and then sends a completion notification to the host. To ensure data integrity, the completion notification must not be delivered to the host until the data is fully observable in the host memory. In many other scenarios, there are requirements for sequential delivery over the peripheral bus.

[0024] However, peripheral buses do not always guarantee unconditional in-order delivery of all bus transactions. Some types of peripheral buses can be considered "strict-order" buses, while other types of peripheral buses can be considered "relaxed-order" buses. For example, PCIe is relatively strict. Non-relaxed-order PCIe does not allow read or write transactions to bypass earlier write transactions. On the other hand, Nvlink, as an example of a memory interface, is much more relaxed in terms of transaction ordering. Some bus types guarantee that requests addressed to the same address will be executed in order, while other bus types do not have such a guarantee.

[0025] In practice, there are many advantages to using a relaxed-order peripheral bus. For example, a relaxed-order bus can offer lower latency, higher bandwidth, lower cost, and easier multi-channel expansion. However, to use a relaxed-order bus, the in-order transfer requirements imposed by the peripheral devices and / or processing device must be enforced externally on the bus.

[0026] A simple solution is to apply a "fence" or "barrier" between every two bus transactions that need to be delivered in order. However, this solution means that only one outstanding bus transaction is allowed. Because the round-trip bus latency is non-negligible, this solution reduces the achievable throughput in real systems. Another possible solution is to mark individual bus transactions as "loosely ordered" or "strictly ordered" and enforce in-order delivery only for strictly ordered transactions. This solution also has a significant impact on throughput.

[0027] Embodiments of the invention described herein provide improved methods and apparatus for using a relaxed order peripheral bus while satisfying in-order transfer requirements not otherwise supported by the bus. The disclosed techniques are typically implemented in a peripheral device, enabling it to service legacy or conventional processing devices.

[0028] In some embodiments described herein, a peripheral device generates multiple bus transaction streams for transmission to a processing device via a relaxed order peripheral bus. The peripheral device selectively enforces transaction ordering for each stream or group of streams. In other words, the peripheral device selects one or more streams from the multiple streams that require transaction ordering and enforces transaction ordering on the bus transactions of the selected streams. The peripheral device may exclude other streams from transaction ordering.

[0029] In some embodiments, the peripheral device enforces strict ordering in a given stream (or group of streams) by inserting an Isolate command between consecutive transactions in the given stream (or group). The Isolate command can be a stream-specific Isolate command applied to a given stream or a group-specific Isolate command applied to a group of streams. In an alternative embodiment, the Isolate command is a global Isolate command applied to all transactions of all streams (but inserted only between transactions requiring strict ordering). In other embodiments, the peripheral device enforces strict ordering in a given stream (or group) by marking the transactions of the given stream (or group) with a unique tag and pausing new bus transactions marked with the unique tag at selected times until the previous transaction marked with the unique tag is acknowledged.

[0030] In some embodiments, the peripheral device enforces strict ordering for each flow group by enqueuing each group's transactions in a respective queue and applying strict ordering within each queue.

[0031] The techniques described herein enable peripheral devices to apply various sorting rules, such as intra-stream sorting rules, inter-stream sorting rules, intra-group sorting rules, and inter-group sorting rules. Various assignment rules can also be used to select how to assign streams to groups. Several examples are described.

[0032] When using the disclosed technology, the performance loss associated with strict ordering is incurred only for the flows or groups of flows that actually need to be delivered in order. For other flows, the advantages of relaxed order processing can be fully obtained.

[0033] System Description

[0034] The embodiments described below primarily relate to network devices, particularly network adapters. However, the disclosed technology can be used with various other types of peripheral devices, such as network switches, storage devices, storage controllers, graphics processing units (GPUs), and accelerators. A network adapter is considered herein as a non-limiting example of a peripheral device.

[0035] Figure 1 FIG2 is a block diagram schematically illustrating a computing system 20 using a relaxed order peripheral bus according to one embodiment of the present invention. In this context, the term "relaxed order peripheral bus" refers to a peripheral bus that allows at least some bus transactions to be transferred out of order. Generally, a relaxed order bus has the following characteristics:

[0036] ■ Allows transactions addressed to different addresses to bypass each other.

[0037] ■ The bus supports a fence / flush mechanism to ensure that write transactions have reached the target ordering domain. The bus can support one or more concurrent outstanding fence / flush operations. Example configurations include:

[0038] o All DMA transactions on the bus are non-posted (i.e., each DMA transaction sends an implicit or explicit acknowledgement to the source).

[0039] ○ Special transactions with the refresh attribute can be issued to guarantee that all previous transactions have reached the target sort range.

[0040] ■ In some embodiments, but not necessarily, a relaxed sequential bus may already enforce same-address ordering, i.e., guaranteeing that transactions addressed to the same address are not allowed to bypass each other. In various embodiments, same-address ordering may guarantee that reads and / or writes from addresses with full and / or partial address matches adhere to the following rule: writes may not bypass writes, and reads may not bypass writes. For example, when the bus enforces same-address ordering with full address matches, a write to address 0x4000 of size 0x8 may not bypass a write to another address 0x4000 of size 0x8. When the bus enforces same-address ordering with partial address matches, a read from address 0x1004 of size 0x8 may not bypass a write to address 0x1000 of size 0x40.

[0041] In this example, Figure 1 The system 20 includes two network adapters 24A and 24B, which communicate with each other through a network 28. The network adapters 24A and 24B and the network 28 can be configured to communicate with each other using any suitable network protocol (e.g., Ethernet or InfiniBand). TM The thick arrows in the figure represent the various data paths within and between network adapters, as described below.

[0042] In this example, network adapters 24A and 24B execute remote direct memory access (RDMA) commands, with network adapter 24A acting as a "requester network adapter" (also referred to herein as the "requester") and network adapter 24B acting as a "responder network adapter" (also referred to herein as the "responder"). During typical system operation, each of the two network adapters may act as a requester for some RDMA commands and a responder for others, and may even act as both a requester and a responder in the same transaction (a "loopback transaction"). The thick arrows in the figure represent the various paths taken by the various messages and transactions that make up the RDMA commands executed by network adapters 24A and 24B, as described below.

[0043] The network adapter 24A (requester) serves one or more processing devices 36A and is coupled to one or more memories 40A. A given processing device 36A typically issues and manages jobs to be executed by the network adapter 24A. Jobs may include, for example, sending data over the network 28 and / or preparing data to arrive over the network 28. The network adapter 24A communicates with the processing devices 36A and accesses the memories 40A via the peripheral bus 32A. Each processing device 36A may include, for example, a host, such as a central processing unit (CPU), a GPU, or a peer device, such as a field programmable gate array (FPGA).

[0044] The network adapter 24B, also referred to as a "responder network adapter," is coupled to one or more memories 40B. The network adapter 24B accesses the memories 40B via the peripheral bus 32B. In some embodiments, although not necessarily, the network adapter 24B also communicates with one or more processing devices 36B via the bus 32B. On the responder side, the processing devices are considered optional. Each processing device 36B may include, for example, a host (e.g., a CPU, a GPU) or a peer device (e.g., an FPGA).

[0045] In this example, memories 40A and 40B may include, for example, random access memory (RAM) devices. Peripheral buses 32A and 32B may include loosely ordered buses (e.g., Nvlink, Nvlink C2C, or PCIe Unordered IO (PCIe UIO)). Alternatively, bus 32A or bus 32B may include a strictly ordered bus (e.g., PCIe or CXL). In yet other embodiments, a given network adapter may communicate with a given processing device via one or more loosely ordered buses, one or more strictly ordered buses, or a mixture of one or more loosely ordered buses and one or more strictly ordered buses.

[0046] In one embodiment, a loosely ordered bus is created when a network adapter uses two (or more) separate bus links to communicate with a processing device or memory, each of which requires at least some bus transactions to be transmitted in order. For example, consider a network adapter that uses two separate PCIe links to communicate with a host. Even though each individual PCIe link is relatively strictly ordered, there is no ordering enforced between the PCIe links, so the two links together exhibit loosely ordered behavior.

[0047] Example Requester Operation

[0048] Figure 2 is a schematic diagram showing an embodiment of the present invention Figure 1 FIG. 2 is a block diagram of the requester side of system 20. An example operational flow on the requester side begins with network adapter 24A receiving a work request from a processing device 36. The work request describes a job to be performed.

[0049] The network adapter 24A performs the following tasks:

[0050] ■ local operations (e.g., memory registers, local memory to local memory operations, loopback packets); or

[0051] ■ Convert remote operations into network packets; and

[0052] ■ For reliable protocols, wait for confirmation; and

[0053] ■ For READ / ATOMIC operations—scatter the response data to the memory 40A.

[0054] After execution, the network adapter 24A sends a completion indication to the processing device 36 that requested the job, notifying the processing device that execution has been completed. As described below, the completion notification may include multiple writes, such as a completion entry, an event entry, and an MSI-X write or another interrupt.

[0055] The network adapter 24A is used for local operations in a variety of scenarios. For example, the network adapter may include one or more accelerators (e.g., replication, compression, etc.). As another example, the network adapter may be connected to multiple disjoint memories (e.g., in a "multi-host" configuration or when a host accesses the network adapter's memory (if present)). As yet another example, the network adapter may be used to send network packets internally (in loopback), in which case the internal processing resembles network operations, but the network packets are not visible on the external interface.

[0056] Figure 2 The bold arrow in FIG shows an example of network adapter 24A executing a work request. The operation includes the following stages:

[0057] A work request is received (arrow 1a).

[0058] The corresponding descriptor and / or data are read from the memory 40A (arrow 2).

[0059] If necessary, local operations are performed (arrow 3a).

[0060] A network packet containing data is generated and sent (arrow 3bi).

[0061] An acknowledgement is received from the responder (arrow 3bii).

[0062] The response data is written to the memory 40A (arrow 4).

[0063] Write completion indication (arrow 5).

[0064] Example Responder Action

[0065] Figure 3 is a schematic diagram according to an embodiment of the present invention Figure 1 A block diagram of the responder side of the system 20 is shown.

[0066] In some embodiments, for an original data packet or message, the responder process is performed by network adapter 24B and includes:

[0067] The data is distributed to the memory 40B based on the receive descriptor queue.

[0068] The completion indication is updated to notify the processing device 36B that the operation is complete. As described above, the completion indication may include multiple writes, such as a completion entry, an event entry, and an MSI-X write or other interrupt.

[0069] For RDMA read / write / atomic (READ / WRITE / ATOMIC) or similar zero-copy operations, the responder process includes writing to memory 40B based on the data packet instruction. For reliable transmission, the responder process includes sending an acknowledgment or read / atomic (READ / ATOMIC) response back to the requester device (network adapter 24A).

[0070] Figure 3 The thick arrow in FIG shows an example of a responder process performed by network adapter 24B. The process includes the following stages:

[0071] Receiving network data packets via network 28 (arrow 1)

[0072] Data is written and / or read in the memory 40B (arrow 2 ).

[0073] Write completion indication (arrow 3 - optional, only applicable to certain RDMA transactions).

[0074] The data response / acknowledgement is sent over the network 28 (arrow 4).

[0075] Completion mechanisms

[0076] As shown above, the various requester and responder processes typically involve some kind of completion notification, wherein the network adapter notifies the processing device that it has completed the execution of the requested operation. In various embodiments, network adapters 24A and 24B may use various mechanisms to determine completion and completion notification. Several non-limiting examples of completion mechanisms and completion notification mechanisms are listed below.

[0077] Completion Counter: In some embodiments, one or more work queues are associated with a counter, called a "completion counter". The counter counts the number of completed messages or bytes associated with these queues. This mechanism is sufficient, for example, when the completion does not need to convey additional metadata. This is the case, for example, in requester completions, and in some responder completions (e.g., if the inbound message size is fixed, or if the message contains its own "message length" indication).

[0078] Completion Queue: One or more work queues (and / or work queue entries) are associated with a completion queue. A work request from an associated requester or responder triggers the writing of a Completion Queue Element (CQE) to a Completion Queue (CQ). Typically, each CQE defines a "valid" or "ownership" indication. Other implementations may involve a counter indicating the number of valid CQEs.

[0079] Completion Events: In some cases, it is useful to send a single event to indicate that several completions are pending. This notification can be performed using a similar mechanism as described above, for example, using an event counter or an event queue (EQ) to which an event queue element (EQE) is posted.

[0080] Interrupts: The completion or event mechanism can in turn be connected to an interrupt (e.g., MSI or MSI-X) mechanism. MSI and MSI-X interrupts are typically implemented as write transactions on the peripheral bus.

[0081] In some embodiments, completion notification is implemented using an event hierarchy: the output of one or more CQs triggers MSI-X via the EQ mechanism. Alternatively, a CQ (or completion counter) may trigger MSI-X directly.

[0082] Optional enforcement of transaction ordering

[0083] Various requester-side and responder-side flows (such as the example flows described above) involve a sequence of bus transactions sent from a network adapter to a processing device over a relaxed-order peripheral bus. In many scenarios, the transactions in the sequence must be visible to the processing device in the same order that they were sent to the bus by the network adapter.

[0084] In some cases, the ordering requirements are derived from rules defined between the network adapter and a driver for the network adapter running in the processing device. Additionally or alternatively, the ordering requirements may be derived from requirements of the communication protocol being implemented, for example, when implementing RDMA on a loosely ordered peripheral bus.

[0085] For example, consider the process by which a requester network adapter performs and reports completion of a work request. This process might include up to four write transactions from the network adapter to the processing device over the peripheral bus:

[0086] I. Data write (eg, a READ response or a local memory to memory operation).

[0087] II. Completion Notice.

[0088] III. Event queue writing (optional).

[0089] IV. Interrupt (eg, MSI-X write) (optional).

[0090] These four write transactions are addressed to different addresses but are still expected to be visible to the processing device in sequence. The semantics of a relaxed sequential bus may not guarantee that this sequence will be delivered in order.

[0091] As another example, it may be desirable to deliver some completion notifications in order relative to one another. When using a completion counter mechanism as described above (or a counter indicating the number of valid CQE entries), completion notifications are sent to the same address, so some loose-order buses will guarantee the desired in-order delivery. However, when using queue-based completions, a processing device (e.g., a device driver running on the processing device) may wish to rely on the valid bit of a later completion to conclude that a previous completion was already valid. With some loose-order buses, this may not be possible.

[0092] As another example, for an RDMA read or atomic response, data is written by the network adapter to the processing device's memory, and a completion notification indicates the arrival of the data. This imposes an additional ordering constraint: the completion notification must not be visible on the processing device until the data is completely written.

[0093] In some embodiments of the present invention, a network adapter employs a novel mechanism to ensure that a selected sequence of bus transactions is delivered in order to overcome the relaxed ordering semantics of a peripheral bus.

[0094] Figure 4 FIG. 1 is a schematic diagram illustrating a network adapter 24 (eg, Figure 1 FIG2 is a block diagram of the internal configuration of a network adapter 24A or 24B (e.g., a network adapter 24A or 24B). Network adapter 24 includes a bus interface 44 for transmitting bus transactions over a loosely ordered peripheral bus 32, a network interface 48 for transmitting network packets over network 28, and processing circuitry 52 that performs various processing tasks for the network adapter. Processing circuitry 52 includes, among other components, sequencing logic 56. Logic 56 enforces transaction ordering among at least some bus transactions transmitted over bus 32 using methods described in detail below.

[0095] In some embodiments, as part of ongoing operation of network adapter 24, processing circuitry 52 generates bus transaction streams and sends them to peripheral bus 32 using bus interface 44. Ordering logic 56 selects one or more bus transaction streams and enforces transaction ordering among the transactions in the selected streams.

[0096] The execution divides transactions into streams so that no sorting is required between transactions belonging to different streams. For example, different host processes (on processing devices) may have independent and isolated clusters of queue pairs (QPs), completion queues (CQs), etc. In this way, the logic 56 does not indiscriminately apply strict sorting to all bus transactions, but rather aims to exclude streams that do not require sorting. This selective sorting reduces the performance degradation (e.g., latency and throughput) caused by strict sorting.

[0097] Figure 5 is a schematic diagram showing an embodiment of the present invention Figure 4 FIG. 5 is a block diagram of the internal structure of sequencing logic 56 in network adapter 24. In this example, logic 56 enforces transaction ordering by inserting isolation commands between selected transactions.

[0098] Figure 5 The ordering logic 56 in receives N bus transaction streams to send to the bus 32. In an example implementation, each bus transaction processed by the ordering logic 56 is accompanied by metadata that specifies (i) a stream ID indicating the stream (from the N streams) to which the transaction belongs, and (ii) an explicit indication of the type of transaction ordering required. The type of transaction ordering can be (i) strict ordering, (ii) relaxed ordering with same-address ordering (described in detail below), (iii) relaxed ordering with same-address ordering for write-before-read only (not for write-before-write), or (iv) relaxed ordering with no address ordering.

[0099] The logic 56 includes M queues 60 (also referred to as "stream queues") for queuing bus transactions, where typically M ≤ N. Each queue 60 is assigned to queue transactions for one or more corresponding streams. The logic 56 also includes a queue selector 64 that routes each incoming bus transaction to the appropriate queue 60. The sequencing logic 56 enforces transaction ordering within each queue 60. A multiplexer (MUX) 68 sends queued transactions in the queues 60 to the bus 32 (typically to the bus interface 44).

[0100] The one or more flows assigned to a given queue 60 are also referred to as a "group" of flows. Thus, assigning N bus transaction flows to M queues is also considered herein to be dividing the N flows into M groups. Similarly, enforcing transaction ordering within a queue is considered to be enforcing transaction ordering within a flow group. The terms "queue" and "group" are sometimes used interchangeably, although queue 60 is only one example implementation of partitioning into groups and enforcing order within each group. In alternative embodiments, sequencing logic 56 may use any other suitable mechanism or data structure to enforce transaction ordering for each flow group.

[0101] As described above, the ordering logic 56 enforces transaction ordering within each queue 60. Transactions assigned to different queues 60 are considered independent, so no ordering is enforced between them. The efficiency of this queue-specific ordering mechanism depends on the ratio between M (the number of queues) and N (the number of flows). When M=N, that is, when each flow is assigned a dedicated queue, efficiency is highest because each flow is ordered independently of other flows.

[0102] However, in practice, the number of flows is often too large to allow M = N, in which case each queue is assigned multiple flows. When a given queue queues transactions from multiple flows, enforcing strict ordering within the queue may introduce some unnecessary ordering, resulting in a degree of unnecessary latency. Therefore, setting the ratio between M and N is a design trade-off.

[0103] In one embodiment, the logic 56 enforces ordering between transactions of a given queue by inserting quarantine commands between selected transactions on that queue. In various embodiments, quarantine commands can be flow-specific, queue-specific, or global. A flow-specific quarantine command applies to a specific flow and has no effect on other flows. A queue-specific quarantine command (also known as a group-specific quarantine command) applies to flows belonging to a specific flow queue 60 and has no effect on other flows. A global quarantine command applies to all flows.

[0104] When servicing the various queues 60, upon encountering an isolate command, the logic 56 stalls all subsequent bus transactions for the flows affected by the command until all previous bus transactions for the flows affected by the command have completed. Depending on the type of isolate command used, the stalled bus transactions may belong to a specific flow, a group of flows assigned to a specific queue, or all flows.

[0105] In some embodiments, the sequencing logic 56 includes multiple isolation engines 76. Each isolation engine 76 supports a single corresponding isolation session, for example, it can stop, trace, and resume bus transactions in response to a single corresponding isolation command. If the number of isolation engines 76 is large enough, the logic 56 can maintain complete independence between different queues 60.

[0106] However, in practice, the number of active quarantine commands at a given time may exceed the number of quarantine engines 76. In this case, a requested quarantine command cannot be executed until the previous quarantine command completes and the corresponding quarantine engine 76 becomes available. This scenario introduces some dependencies between queues 60.

[0107] The dependency on the availability of the isolation engine 76 and the policy of the allocation engine 76 can be viewed as applying “inter-queue rules” or “inter-group rules” through the sequencing logic 56. An example of an inter-queue rule set is as follows:

[0108] While stream B is in isolation, relaxed-order transactions from stream A can continue.

[0109] If an isolation engine 76 is available, a flow queue with strictly ordered transactions can either start a new isolation session immediately (initiated by the available engine 76) or wait for a period of time T for other flow queues to share the same isolation session (handled by an existing engine 76).

[0110] If all isolation engines 76 are in use, all stream queues that need isolation will stop, i.e., be in a head-of-line blocking state. Once an isolation engine is available, all stream queues waiting for isolation can share the same isolation engine.

[0111] Within a given flow queue 60 (i.e., within a given flow group), the ordering logic 56 applies appropriate "in-queue rules" or "in-group rules." An example set of in-queue rules is as follows:

[0112] No isolation is required between relaxed ordering transactions on the same queue 60. An exception may be transactions that specify same address ordering, as described below.

[0113] Each strictly sequential transaction requires the entire queue 60 to be quarantined, ie all previously incomplete transactions to be flushed.

[0114] The configuration of the system 20, including the internal configuration of the various network adapters 24, including the internal configuration of the processing circuit 52 and the sequencing logic 56, such as Figures 1 to 5 The illustrated example configuration is chosen purely for conceptual clarity. In alternative embodiments, any other suitable configuration may be used.

[0115] For example, in some embodiments, the peripheral device is a data processing unit (DPU), also known as a "smart NIC." Such a DPU typically includes one or more processing devices (e.g., ARM cores), a NIC, and a peripheral bus connecting the processing devices and the NIC, all integrated into an integrated circuit (IC). In such embodiments, the peripheral device in the IC can be a loosely ordered bus, and the disclosed techniques can be applied to bus transactions sent between the NIC and the processing device within the IC.

[0116] For the sake of clarity, elements that are not necessary for understanding the principles of the present invention are omitted in the figure. The various elements of system 20 (including elements of network adapter 24) can be implemented in hardware (e.g., one or more application-specific integrated circuits (ASICs) or FPGAs), software, or a combination of hardware and software elements. In some embodiments, some elements of network adapter 24 can be implemented in part or in whole using one or more general-purpose processors that are programmed with software to perform the functions described herein. The software can be downloaded to any processor via a network in electronic form, or can be provided and / or stored in a non-transitory tangible medium, such as a magnetic, optical, or electronic memory, alternatively or additionally.

[0117] Figure 6A and Figure 6B is a schematic diagram showing the embodiment of the present invention. Figure 5 The flow-specific isolated message diagram is performed by the sequencing logic 56. In both embodiments, the logic 56 processes two bus transaction streams, numbered "0" and "1." Each stream includes several data transactions followed by a marker (in this example, a completion, such as a CQE). The sequencing requirement is that subsequent data must be delivered to the processing device only after the previous completion has been delivered.

[0118] exist Figure 6A In the embodiment of the present invention, both streams are assigned to the same queue 60. In this embodiment, the isolation command applied to one stream will affect both streams. As shown in the figure, the "fence0" command causes a stop in the transaction of stream 1.

[0119] On the other hand, Figure 6BIn the embodiment of FIG1 , each flow is assigned to a separate queue 60. In this embodiment, a fence command applied to one flow does not affect another flow. In this implementation, a "fence0" command does not stop transactions for flow 1, and a "fence1" command does not stop transactions for flow 0. A comparison of the two figures demonstrates the advantages in latency and throughput achieved by using flow-specific isolation with multiple queues.

[0120] The use of the Isolate command is only one non-limiting example of a mechanism for enforcing transaction ordering within a selected stream or stream group. In other embodiments, any other suitable mechanism may be used. For example, the logic 56 may enforce strict ordering within a given stream (or stream group) by marking the transactions of the given stream (or stream group) with a unique tag. At selected times, the logic 56 may pause new bus transactions marked with a unique tag until a previous transaction marked with a unique tag is confirmed, rather than inserting an explicit Isolate command.

[0121] In various embodiments, queue selector 64 may use various techniques to assign flows to queues 60 (or otherwise assign flows to groups). In some embodiments, queue selector 64 assigns flows to queues (or otherwise assigns flows to groups) by applying a suitable hash function to the flow ID. In other embodiments, queue selector 64 assigns flows to queues (or otherwise assigns flows to groups) by applying one or more group assignment rules.

[0122] Sorting by same address

[0123] When the peripheral bus 32 already maintains same-address ordering, the logic 56 generally does not need additional same-address execution. Otherwise, in some embodiments, the ordering logic 56 applies same-address ordering between bus transactions (i.e., ensuring that transactions addressed to the same address are delivered in order). Same-address ordering can be applied within a stream and / or between streams (i.e., applied to transactions belonging to the same stream and / or to transactions belonging to different streams). In some embodiments, the logic 56 includes a same-address ordering transaction database 72 for this purpose.

[0124] For example, consider an implementation where the peripheral bus 32 is completely relaxed, i.e., same-address ordering is not maintained at all. In one embodiment, the database 72 includes a tracking table for each outstanding non-isolated transaction. If a given stream queue 60 is requested to send a transaction addressed to the same address as a previously outstanding transaction, additional isolation is required. A single isolation can indicate that all previous transactions have committed, allowing transactions addressed to the same address to be safely issued to the bus 32.

[0125] In one embodiment, a special case is "counted writes"—see below. These transactions consist of a pair of addresses—one for data and one for flags. The native ordering of bus 32 can enforce ordering for both addresses, one of the addresses, or neither address. If bus 32 only maintains native ordering for one of the addresses, the mechanism described above can be used to enforce ordering for the other address.

[0126] Stream selection – assigning bus transactions to streams

[0127] In various embodiments, the processing circuitry 52 ( Figure 4 ) applies various flow assignment rules to define bus transaction flows, i.e., assigns bus transactions to flows. In the exemplary embodiment, processing circuitry 52 assigns bus transactions to flows based on disjoint contexts. The term "context" refers to the various bus transactions associated with inbound messages associated with a given QP, including the inbound messages themselves and the various transactions associated with the completion of those messages.

[0128] The discovery and configuration of contexts are considered beyond the scope of this disclosure. These tasks can be accomplished, for example, using an in-band relaxed order configuration mechanism, using an out-of-band configuration mechanism (e.g., via a separate strictly ordered peripheral bus, or using a tunneled configuration mechanism (e.g., tunneling a strictly ordered bus through a relaxed order bus).

[0129] As described above, the process of executing and reporting the completion of an inbound message can include the following bus transactions: (i) receiving the message, (ii) issuing a work request on the QP (or shared receive queue SRQ), (iii) issuing a completion queue element on the CQ, (iv) issuing a completion event on the EQ, and (v) issuing an interrupt, such as MSI-X. In some cases, the process is shorter. For example, in some cases, the QP does not issue a completion. In other cases, the CQ does not issue an event. In other cases, the EQ may not issue an MSI-X.

[0130] In many implementations, there are many-to-one relationships between these elements, for example, (i) multiple QPs may be associated with the same CQ, (ii) multiple CQs may be associated with the same EQ, and / or (iii) multiple EQs may be associated with the same MSI-X. Using these one-to-many relationships, contexts can be logically arranged in groups (e.g., a tree) that share common elements. Contexts that have no common elements (and therefore belong to different groups) are said to be disjoint.

[0131] In some embodiments, the ordering logic 56 assigns various bus transactions belonging to a context group to corresponding flows. This allocation heuristic is useful because different CPU processes typically hold disjoint contexts (QPs, CQs, etc.). In this case, there is no reason to enforce ordering between transactions targeting different CPU processes. On the other hand, EQs and MSI-Xs can be assigned per CPU core rather than per process. This can result in inter-process ordering if more than one process is bound to a single CPU core.

[0132] Figure 7A FIG. 1 is a diagram schematically illustrating an example of allocating bus transactions to flows based on disjoint contexts according to an embodiment of the present invention. This example illustrates the following:

[0133] Since QP0, QP1, and QP2 are all indirectly connected to EQ0, logic 56 assigns them to a single stream ("Stream0").

[0134] QP3 is connected to EQ1 and therefore assigned to a different stream ("Stream 1").

[0135] QP4 and QP5 are connected to CQ3, which is not connected to EQ. Therefore, these QPs are assigned to a separate stream ("Stream 2").

[0136] QP6 and QP7 do not issue CQEs at all, only RDMA transactions. Therefore, logic 56 assigns each of these QPs to a corresponding individual stream ("stream 3" and "stream 4" respectively).

[0137] QP8 and QP9 are Extended Reliable Connection (XRC) QPs and are therefore not bound to a single QP. Therefore, all QPs in the same XRC domain can be assigned to a single stream ("stream 5" in this example).

[0138] In some embodiments, the logic 56 adaptively modifies the assignment of bus transactions to flows based on the actual traffic handled by the network adapter. For example, in some cases, a QP may receive a sequence of transactions that does not require a CQE. As another example, a sequence of CQEs issued may not require an EQE. In this case, a static assignment based on the context topology may be too restrictive. In one embodiment, the logic 56 initially assigns each QP to a separate flow, and similarly assigns each CQ to a separate flow. The initial assignment can then be refined based on actual traffic patterns. For example, if a QP requires a CQE, the logic 56 can combine the QP and CQ into a common flow. Prior to reallocation, the previous flow should typically be flushed. Following the same argument, if a period without CQEs is encountered, the logic 56 can revert to the initial configuration and split the QP and CQ into separate flows. A similar approach can be applied between CQs and EQs.

[0139] In an alternative embodiment, logic 56 assigns bus transactions to flows based on disjoint transfer transactions. For example, some transfer definitions (e.g., OpenShmem and NVShmem) include out-of-order semantics, meaning that no ordering is required between different transactions unless explicit isolation is issued.

[0140] In this case, for one-sided RDMA (no CQE), the logic 56 may treat each inbound message as a separate stream. The logic 56 may associate bus transactions with transport transactions (e.g., RDMA messages) using any suitable method. For example, in some implementations, a bus transaction may include an explicit "message identifier" that indicates the RDMA message to which the bus transaction belongs. For example, see U.S. patent application Ser. No. 17 / 902,150, filed on September 2, 2022, entitled "Payload Direct Memory Storing (PDMS) for Remote Direct Memory Access (RDMA)."

[0141] Figure 7B FIG2 is a diagram schematically illustrating an example of assigning bus transactions to streams based on disjoint RDMA transactions according to one embodiment of the present invention. In this example, bus transactions "data0," "flag0," "data1," and "flag1" arrive at QP0; bus transactions "data2" and "data3" arrive at QP1; and bus transactions "data6" and "data7" arrive at QP2. Although "data0," "flag0," "data1," and "flag1" all arrive at the same QP, they can be separated into separate streams ("Stream0" and "Stream1") for the transfer semantics used.

[0142] Improving flow queue operations using subqueues

[0143] On top Figure 5 In the embodiment of FIG, bus transactions of one or more flows are queued in a given flow queue 60. In some cases, even if a separate flow queue 60 can be allocated for each individual flow, such allocation may not be optimal.

[0144] For example, despite belonging to the same flow, data generally does not need to wait for CQE or certain types of flags (e.g., while a write transaction needs to wait for a subsequent atomic transaction, the write transaction does not need to wait for new types of transactions, such as "atomic write" or CQE). Similarly, a CQE generally does not need to wait for an EQE.

[0145] Therefore, in some embodiments, the logic 56 divides a given flow queue 60 into multiple sub-queues. A given sub-queue depends on the progress of another sub-queue to reach a particular transaction. This implementation retains the independent progress of each flow queue 60 as described above, but further optimizes the progress within a flow queue.

[0146] Figure 8 Schematically illustrates the division of a flow queue into sub-queues according to an embodiment of the present invention. Figure 5 Any queue 60 in is implemented using three sub-queues denoted as “data,” “CQE / flag,” and “EQE.” Each sub-queue is used to queue bus transactions of a corresponding type (data transactions, CQE / flag, and EQE).

[0147] Logic 56 services bus transactions at the head of a subqueue. Any suitable arbitration scheme may be used between subqueues, for example, servicing the oldest transaction that can proceed (i.e., the oldest transaction that does not wait for isolation of a previous dependent transaction). In this way, even if the transaction at the head of a stream queue is stalled due to isolation, bus transactions that do not depend on previous bus transactions within the stream queue can proceed.

[0148] In some embodiments, a flow is divided into multiple sub-flows, and explicit dependencies are specified between the sub-flows. This structure enables the sequencing logic 56 to process the flow more efficiently using the sub-queues.

[0149] In the exemplary embodiment, in addition to specifying the stream ID, the logic 56 ( Figure 5 ) also specifies the following:

[0150] Transaction ID.

[0151] The subflow ID that identifies the subflow (within the current transaction).

[0152] Subflow Dependency – Indicates which previous transaction the current transaction depends on.

[0153] For example, reference Figure 8 , data can be assigned to sub-flow 0, CQE to sub-flow 1, and EQE to sub-flow 2. In this example, the data transactions queued in sub-queue 0 (the leftmost sub-queue, denoted as "Data") are numbered 801-805, the CQE / flags queued in sub-queue 1 (the middle sub-queue, denoted as "CQE / Flag") are numbered 811-812, and the EQE queued in sub-queue 2 (the rightmost sub-queue, denoted as "EQE") is numbered 821.

[0154] When the light shaded CQE (811) is pushed to the middle subqueue, the CQE (811) explicitly references the bottom-most light shaded data transaction (803) on substream 0, etc. These references enable the sequencing logic 56 to resolve dependencies between the progress of substreams (or subqueues).

[0155] Since the ordering logic 56 typically processes subqueues rather than subflows, in some embodiments, the transaction ordering number is reassigned from each subflow domain to each subqueue domain. Figure 8 , even though the light shaded transactions and dark shaded transactions in the middle subqueue (subqueue 1, denoted as “CQE / flag”) may belong to different flows, the light shaded CQE (811) at the head of the middle subqueue is waiting for {subqueue 0, transaction 2} (the bottom-most light shaded data transaction (803) posted on the left subqueue), and the dark shaded CQE (812) posted on the middle subqueue is waiting for {subqueue 0, transaction 1} (the bottom-most dark shaded data transaction (805) posted on the left subqueue), since both are being pushed to the same subqueue, the transactions that CQE / flag is waiting for can be reallocated to {subqueue 0, transaction 3} and {subqueue 0, transaction 5} respectively.

[0156] Because subsequent transactions depend on prior transaction ordering, in some embodiments, ordering logic 56 maintains a table that converts between client numbers (e.g., {flow 18, sub-flow 0, transaction 0}) and ordering logic numbers (e.g., {flow queue 7, sub-queue 0, transaction 1}).

[0157] Transaction Ordering Using Counted Writes

[0158] In some embodiments, a relaxed-order peripheral bus supports "counted write" bus transactions (also known as "flagged write" transactions). In this context, a "counted write" transaction is a transaction that atomically (i) writes data to a certain address and (ii) writes a flag (e.g., "data valid") or increments a write counter at another address. When using counted write transactions, the peripheral bus ensures that the flag is written (or the write counter is incremented) only after the data has been written, even if the two writes are addressed to different addresses. However, in general, there is no ordering guarantee between separate counted write transactions.

[0159] Consider a loosely ordered peripheral bus that supports counted write transactions and ensures same-address ordering. When using a completion counter mechanism on this bus, transaction ordering (e.g., isolation) is still required between the data (e.g., read / atomic response) and the write completion counter. This requirement incurs a latency penalty. In some embodiments, ordering logic 56 optimizes this mechanism using counted write transactions.

[0160] For example, for small data sizes (smaller than the maximum payload size of the peripheral bus), counted writes can be used instead of writes. While this scheme does not avoid the need to use isolation between completions, it avoids the need to use isolation between data and completions, thereby reducing latency.

[0161] As described above, counted write transactions involve writes with two addresses: a data address and a write counter address. For these transactions, same-address ordering should be preserved for the write counter address. If a relaxed-ordering bus natively maintains same-address ordering only for data addresses, the same-address ordering mechanism described above can be used exclusively for counter writes. Even for implementations that don't support same-address ordering, the latency penalty for the first transaction can still be avoided due to the isolation between data and counter writes.

[0162] Furthermore, assuming that a counter counts the number of bytes completed, a counted write mechanism can be used to completely avoid the need for isolation between data and CQE. For read responses, this can be called a "counted read" or "counted get" operation because, from the software's perspective, the read response is associated with a counter. For example, see U.S. patent application Ser. No. 17 / 977,910, filed Oct. 31, 2022, and entitled "Remote Direct Memory Access Operations with Integrated Data Arrival Indication."

[0163] Completion Queue and Completion Counter Use Cases

[0164] As another example, consider the use of a completion queue mechanism on a loosely ordered peripheral bus. Unlike completion counters, completion queues involve writing individual entries. In this scenario, several optimizations can be employed. For example, the network adapter interface protocol can be configured so that completion entries must be consumed sequentially. This avoids the need to sequence CQE writes. Where applicable, isolation between data and CQE writes, as well as between CQE and EQE writes, is still required. As another example, peripheral buses typically have a degree of atomicity, for example, aligned writes of up to one cache line (e.g., 128 bytes) are atomically written to memory. This property can be used to atomically write a large group of completions, rather than requiring isolation between each completion. This solution holds true even if software consumes completions out of order. As yet another example, a model can be defined where each completion entry can indicate multiple previous completions (e.g., completion entries for m requests, indexed from work_request (n) to (n+m-1)). This definition allows for a merging mechanism (e.g., if subsequent writes are pending, wait for the previous data to complete and send a single completion indicating multiple requests). Since completions may contain data or metadata, rather than just an indication that work was completed, the merge session can be terminated as soon as the metadata needs to be written.

[0165] In an alternative embodiment, the sequencing logic 56 uses a "completion counter per operation" mechanism. This solution can be viewed as a hybrid between a completion counter and a completion queue entry. The completion counter for each operation (or group of operations) can be software-defined. For example, the work request itself can dictate the counter position. Alternatively, the completion counter for each operation can include a predefined array of counters, where each counter corresponds to a single work request. Since each transaction is completed using a separate (fixed or user-defined) counter, no sequencing is required between completions.

[0166] Data read / write sequencing

[0167] An RDMA READ or ATOMIC response (WRITE) can be read by a subsequent work request. In some cases, the read is expected to be delivered out-of-order by the peripheral bus relative to the subsequent write. However, as long as the network adapter issues a peripheral bus read transaction (associated with the RDMA WRITE source data) after issuing a peripheral bus write transaction (associated with the RDMA READ response data), this is required for the above operation to work regardless of the peripheral bus ordering semantics, as there is no race condition due to same-address ordering.

[0168] Figure 9 is a message diagram schematically illustrating data read / write sequencing according to an embodiment of the present invention.

[0169] As shown in the figure, the network adapter issues a data write after a data read operation. As long as this condition is maintained, same-address ordering ensures consistency.

[0170] In some embodiments, the above conditions are enforced using appropriate software-defined mechanisms, such as an IB spec work request "fence" indication that explicitly requests a write (WRITE) to wait for a previous READ. In other embodiments, an appropriate cross-channel mechanism (such as "core direct") can be used to enforce the order. Such a mechanism is described, for example, in the following document: "CORE-Direct - The Most Advanced Technology for MPI / SHMEM Collectives Offloads", Mellanox Technologies Technology Brief, May 2010. In still other embodiments, explicit memory synchronization can be used to enforce the order, as described, for example, in U.S. Patent Application Publication No. 2021 / 0406179.

[0171] Responder-side transaction ordering

[0172] The examples described above primarily relate to the operation of the requester network adapter (e.g., Figure 1 However, the above techniques are not limited to the requester side and may also be used in a similar manner for responder network adapters (e.g., network adapter 24B communicating via loose order bus 32B).

[0173] For example, the responder network adapter also typically needs to guarantee in-order delivery of data and completion notifications (i.e., ensure that completions are seen on the peripheral bus only after the corresponding data is fully visible). For example, the RDMA specification requires that completions (generated for an RDMA WRITE with an IMMEDIATE operation, or for a SEND operation that may be sent after an RDMA WRITE operation) guarantee visibility of the previous data. Many of the techniques described above for in-order delivery of completion notifications for the requester network adapter can also be used for the responder network adapter. One difference is that, unlike the requester network adapter, the responder network adapter typically has no information about the number of inbound bytes. Solutions applicable when the size of the inbound data is known are described further below.

[0174] Some of the requester-side solutions described above can also be used to maintain write / atomic ordering on the responder side. One way to signal data arrival is to issue an RDMA atomic operation after the RDMA WRITE. In an example implementation, the responder network adapter sends an atomic add (ADD) operation that increments a software-defined counter similar to the completion counter described for the requester network adapter. In the responder network device, there is also an ordering issue between a data write and subsequent data writes. The solution to this issue is similar to the work completion ordering technique described for the requester network adapter.

[0175] In other embodiments, the "per-operation completion counter" mechanism described for the requester network adapter may also be used.

[0176] In an alternative embodiment, if the relaxed order peripheral bus supports counted write operations, this can be exploited, for example, as described in U.S. patent application Ser. No. 17 / 977,894, entitled "Remote Direct Memory Access Operations with Integrated Data Arrival Indication," filed Oct. 31, 2022. In this solution, a write transaction includes a data address and a counter address, where the counter is incremented by the number of bytes written to the data address. The target will not consume the data until the counter reaches the specified value.

[0177] Since counter incrementing is a commutative operation, the goal is not to depend on the order in which data is written to the peripheral bus (the peripheral bus itself will ensure that the counter update is committed only after the updated data is visible).

[0178] Each data transaction is initiated by a work request, converted into one or more network packets, and finally written to the target memory via the peripheral bus.

[0179] End-to-end counted write operations mean that work requests, network packets, and peripheral bus transactions can all express counted writes. Because transactions do not require any ordering to complete correctly, no fencing or other expensive operations are required by the peripheral.

[0180] Some of the requester solutions described above can also be used to maintain write / write ordering on the responder side. The most complex case is two regular RDMA WRITE messages, as there is nothing to distinguish the first as "data" and the second as "flags." In some embodiments, the responder network adapter supports full write-after-write ordering by using the stream-specific isolation techniques described above. In an example embodiment, configuration is added to each queue that requires this semantics.

[0181] Some of the above requester-side solutions can also be used to maintain write / read ordering on the responder side. For example, you may need to ensure that an RDMA READ (or ATOMIC) is ordered after an RDMA WRITE. However, this ordering is usually resolved when the peripheral bus supports same-address ordering.

[0182] Although the embodiments described herein are primarily directed to peripheral devices connected to a loose order peripheral bus, the methods and systems described herein can also be used in mixed configurations including loose order peripheral buses and strict order peripheral buses. For example, a single peripheral bus can support both strict order and loose order semantics (e.g., some PCIe configurations can support ordered and unordered transfers on separate virtual channels). As another example, a single peripheral device can be connected to multiple peripheral buses of different types, such as PCIe and Nvlink. In some configurations, the bandwidth of a strict order peripheral bus may be lower than that of a loose order peripheral bus.

[0183] In some embodiments, peripheral devices can effectively use this hybrid configuration in various ways. In one embodiment, writes originating from write / write-ordered queues can utilize a strictly ordered bus due to the high serialization penalty. In another embodiment, CQE / EQE / MSI-X, which typically uses less bandwidth than the data, can be sent on a strictly ordered peripheral bus. In this implementation, data is typically sent on a loosely ordered peripheral bus using the disclosed flow-aware isolation techniques. However, CQE, EQE, and MSI-X can be issued to a strictly ordered bus without the additional isolation overhead.

[0184] Therefore, it should be understood that the above embodiments are cited by way of example, and the present invention is not limited to what is specifically shown and described above. On the contrary, the scope of the present invention includes combinations and sub-combinations of the various features described above, as well as variations and modifications that will occur to a person skilled in the art upon reading the above description and that are not disclosed in the prior art. The documents incorporated by reference into this patent application should be considered an integral part of this application, but whenever the definition of any term in these incorporated documents conflicts with the definition explicitly or implicitly in this specification, only the definition in this specification should be considered.

Claims

1. A peripheral device comprising: a bus interface for exchanging bus transactions via a peripheral bus, the peripheral bus allowing out-of-order transmission of at least some of the bus transactions; as well as Circuit for: generating a plurality of streams of the bus transactions; Selecting one or more flows requiring transaction ordering from the plurality of flows; enforcing said transaction ordering among said bus transactions of a selected flow; as well as The bus transaction is sent to the peripheral bus through the bus interface.

2. The peripheral device according to claim 1, wherein The circuitry is for enforcing the transaction ordering in the selected stream by inserting one or more isolate commands into the bus transactions of the selected stream.

3. The peripheral device according to claim 2, wherein: One or more of the isolation commands are stream-specific isolation commands that apply only to selected streams.

4. The peripheral device according to claim 2, wherein: In response to detecting that the number of currently processed sequestered commands exceeds a defined maximum number, the circuitry is operable to cease processing of any stream requiring insertion of a new sequestered command.

5. The peripheral device according to claim 1, wherein The circuit is configured to enforce the transaction ordering in the selected stream by tagging the bus transactions of the selected stream with a unique tag, and to suspend new bus transactions tagged with the unique tag at a selected time until a previous transaction tagged with the unique tag is acknowledged.

6. The peripheral device according to claim 1, wherein The circuitry is for distributing the streams to two or more groups and enforcing the transaction ordering among the bus transactions of a given one of the groups.

7. The peripheral device according to claim 6, wherein: The circuitry is for enforcing the transaction ordering within the given group by inserting one or more isolate commands into the bus transactions of the stream that belong to the given group.

8. The peripheral device according to claim 7, wherein: One or more of the quarantine commands are group-specific quarantine commands that apply only to the given group.

9. The peripheral device according to claim 7, wherein: In response to detecting that the number of currently processed segregated commands exceeds a defined maximum number, the circuitry is operable to cease processing of any group requiring insertion of a new segregated command.

10. The peripheral device according to claim 6, wherein: The circuit is configured to enforce the transaction ordering within the given group by tagging the bus transactions of the given group with a unique tag, and to suspend new bus transactions tagged with the unique tag at selected times until a previous transaction tagged with the unique tag is acknowledged.

11. The peripheral device according to claim 6, wherein: The circuitry is for assigning the flows to the groups by queuing the bus transactions in two or more queues respectively associated with the two or more groups.

12. The peripheral device according to claim 6, wherein: The circuitry is configured to enforce the transaction ordering by applying one or more intra-group ordering rules to the bus transactions belonging to the given group.

13. The peripheral device according to claim 6, wherein: In addition to the transaction ordering within each of the groups, the circuitry is further configured to enforce selective transaction ordering among bus transactions belonging to different groups according to one or more inter-group ordering rules.

14. The peripheral device according to claim 6, wherein: The circuitry is configured to assign the flows to the groups based on one or more group assignment rules.

15. The peripheral device according to claim 1, wherein The circuit is configured to allocate the bus transactions to the flows according to one or more flow allocation rules.

16. The peripheral device according to claim 1, wherein The peripheral bus that enables the out-of-order transfers includes two or more separate bus links, each bus link requiring in-order transfer of at least some of the bus transactions.

17. The peripheral device according to claim 1, wherein The peripheral device is a network adapter configured to communicate with the processing device via the peripheral bus, so as to couple the processing device to a network.

18. The peripheral device according to claim 17, wherein: The circuit is used to send or receive communication data packets through a network according to a Remote Direct Memory Access (RDMA) protocol.

19. The peripheral device according to claim 1, wherein The bus interface is for communicating with a processing device over the peripheral bus, and wherein the circuitry is for enforcing transaction ordering in a selected stream by applying one or more ordering rules defined between the peripheral device and a device driver executing in the processing device.

20. A method comprising: generating, in the peripheral device, a plurality of streams of bus transactions for transmission over the peripheral bus that allows out-of-order transmission of at least some of the bus transactions; Selecting one or more flows requiring transaction ordering from the plurality of flows; enforcing, in the peripheral device, the transaction ordering between the bus transactions of the selected stream; as well as The bus transaction is sent from the peripheral device to the peripheral bus.

Images