Relaxed refresh on out-of-order interconnect

Abstract

The present disclosure relates to relaxed flushes on an out-of-order interconnect. In one embodiment, an apparatus comprises a network interface to receive a network stream over a network, a data bus interface to share data with connected devices via a data bus, and processing circuitry to send a given relaxed flush transaction to a globally visible point of the connected devices via the data bus in response to receiving at least one first acknowledgement that all previously sent relaxed ordering transactions are globally observed by the consumers, and before receiving a second acknowledgement that a previously sent strong ordering transaction or a given previously sent relaxed flush transaction is globally observed by the consumers.

Description

Technical Field

[0001] This disclosure relates to computer systems, and more particularly, but not exclusively, to transaction ordering. Background Technology

[0002] In traditional PCIe systems, data is published to a global visibility point via an ordered structure to ensure transactions follow a specific order. When a producer publishes data and its subsequent flags to a global visibility point (e.g., a device such as a processor or memory) via a PCIe interconnect, the system ensures that a consumer can only observe the flag after the data has been globally observed by the consumer. This process relies on the PCIe specification's enforcement of transaction ordering, where a flag is only visible after the corresponding data is globally visible to the consumer.

[0003] PCIe uses two main ordering methods for write transactions: Loose Order (RO) and Strong Order (SO). With RO, transactions can be published without enforcing an order, allowing for greater flexibility and parallelism. However, with SO transactions, subsequent transactions must wait until all previous transactions are globally visible to the consumer to ensure strict ordering. This approach guarantees data integrity but can lead to inefficiency when high bandwidth is required.

[0004] In a System-on-Chip (SoC) environment, the CHI protocol is used to introduce an out-of-order consistency architecture with full cache coherency. In this scenario, the Consistency Request Node (RN-F) interfaces with the Network Interface Controller (NIC) to translate PCIe transactions into CHI protocol, enforcing PCIe ordering rules across the interconnect. For RO transactions, the RN-F directly sends data to memory, while for SO transactions, the RN-F requests ownership of the transaction and enforces cache coherency until the transactions are completed in order.

[0005] An alternative approach, introduced in PCIe specification revision 6.2, uses an unordered structure with unordered inputs / outputs (UIOs), allowing transactions to be sent to a globally visible point without waiting for strict ordering guarantees. This achieves higher parallelism and reduced latency. However, UIOs introduce a different challenge: producers must wait for acknowledgments (ACKs) from the globally visible point before publishing flags. This process, known as source ordering, requires producers to ensure all data transactions are globally visible to consumers before flags are set, and introduces additional latency for flag visualization.

[0006] In high-performance computing environments, maintaining data integrity while achieving low latency and high bandwidth remains an ongoing area of ​​research and development. As systems continue to evolve, new methods for transaction sequencing and data visualization are being explored to meet the growing demands of modern applications and workloads. Summary of the Invention

[0007] According to embodiments of the present disclosure, an apparatus is provided, comprising: a network interface for receiving network streams over a network; a data bus interface for sharing data with a connected device via a data bus; and a processing circuitry system configured to: in response to receiving at least one first acknowledgment that all previously sent relaxed ordering transactions have been globally observed by a consumer, and before receiving a second acknowledgment that a previously sent strong ordering transaction or a given previously sent relaxed refresh transaction has been globally observed by the consumer, send a given relaxed refresh transaction via the data bus to a globally visible point of the connected device.

[0008] Further according to embodiments of this disclosure, the processing circuitry is configured to: wait for at least one first acknowledgment before sending a given relaxation refresh transaction to a globally visible point of the connected device via a data bus.

[0009] Furthermore, according to embodiments of the present disclosure, the device further includes a network interface controller, the network interface controller including a packet processing circuitry system, the packet processing circuitry system including a processing circuitry system.

[0010] Additionally, according to embodiments of this disclosure, the processing circuitry is configured to: send a given relaxed refresh transaction, a previously sent relaxed ordered transaction, and a previously sent strongly ordered transaction using an out-of-order input-output protocol.

[0011] Furthermore, according to embodiments of this disclosure, the processing circuitry is configured to: send a given relaxation refresh transaction to a globally visible point via a data bus before receiving a second acknowledgment that a previously sent strongly ordered transaction has been globally observed by the consumer.

[0012] Further according to embodiments of the present disclosure, the processing circuitry is configured to: send a given relaxation refresh transaction to a globally visible point via a data bus before receiving a second acknowledgment that a given previously sent relaxation refresh transaction is globally observed by the consumer.

[0013] According to a further embodiment of the present disclosure, the processing circuitry is configured to: wait for a third acknowledgment that another relaxation refresh transaction is also globally observed by the consumer at the given memory address before sending the given relaxation refresh transaction to the globally visible point at the given memory address; and in response to receiving the third acknowledgment, send the given relaxation refresh transaction to the globally visible point via the data bus.

[0014] Additionally, according to embodiments of this disclosure, the processing circuitry is configured to: send relaxed-ordered transactions to globally visible points via the data bus while any previously sent transactions have not yet been globally observed by the consumer.

[0015] Furthermore, according to embodiments of this disclosure, the processing circuitry is configured to: wait for at least one acknowledgment that all previously sent transactions have been globally observed by the consumer before sending a strongly ordered transaction to a globally visible point; and, in response to receiving at least one acknowledgment that all previously sent transactions have been globally observed by the consumer, send the strongly ordered transaction to the globally visible point via a data bus.

[0016] Further according to embodiments of this disclosure, the previously sent relaxed ordering transaction includes data, and the given relaxed refresh transaction includes a flag or completion notification, and the previously sent strong ordering transaction includes a flag or completion notification.

[0017] Furthermore, according to embodiments of this disclosure, the processing circuitry is configured to: send multiple relaxed refresh transactions, including corresponding completion notifications, to globally visible points via a data bus in geographical order rather than chronological order.

[0018] Additionally, according to embodiments of this disclosure, the processing circuitry is configured to: operate according to a hierarchy of relaxation refresh levels; wait for at least one third acknowledgment that the following transactions are globally observed by the consumer before sending a given relaxation refresh transaction to a global visibility point: all previously sent relaxation refresh transactions with a relaxation refresh level lower than that of the given relaxation refresh transaction, or another previously sent relaxation refresh transaction with the same refresh level as that of the given relaxation refresh transaction and sent to the same memory address as that given relaxation refresh transaction; and, in response to receiving the at least one third acknowledgment, send the given relaxation refresh transaction to the global visibility point via a data bus.

[0019] Furthermore, according to embodiments of this disclosure, a given previously sent relaxation refresh transaction has the same relaxation refresh level as or a higher relaxation refresh level than the given relaxation refresh transaction; and the processing circuitry is configured to: send the given relaxation refresh transaction to a globally visible point via a data bus before receiving a second acknowledgment that the given previously sent relaxation refresh transaction is globally observed by the consumer.

[0020] Further according to embodiments of the present disclosure, a given previously sent relaxation refresh transaction has the same relaxation refresh level as a given relaxation refresh transaction; the given previously sent relaxation refresh transaction and the given relaxation refresh transaction are being sent to different memory addresses in a globally visible point; and the processing circuitry is configured to: send the given relaxation refresh transaction to the globally visible point via a data bus before receiving a second acknowledgment that the given previously sent relaxation refresh transaction is globally observed by the consumer.

[0021] Furthermore, according to embodiments of this disclosure, the hierarchy of relaxation refresh levels corresponds to the hierarchy of different completion notification types.

[0022] According to another embodiment of this disclosure, an apparatus is provided, comprising: a data bus interface for sharing data with a connected device via the data bus; and a processing circuitry system configured to: send transactions via the data bus to a global visibility point of the connected device, each transaction having a transaction type selected from one of the following transaction types: a relaxed ordering transaction type, a relaxed refresh transaction type, or a strongly ordered transaction type; send a given relaxed ordering transaction to the global visibility point via the data bus regardless of the transaction type of a previously sent transaction that has not yet been globally observed by a consumer; and send a given relaxed refresh transaction to the global visibility point via the data bus in response to no previously sent relaxed ordering transaction waiting to be globally observed by a consumer, and regardless of a previously sent relaxed refresh transaction sent to a different memory address than the given relaxed refresh transaction waiting to be globally observed by a consumer.

[0023] Further according to embodiments of the present disclosure, the processing circuitry is configured to: send a given strongly ordered transaction to a globally visible point via a data bus in response to the absence of any previously sent transactions waiting to be globally observed by the consumer; and send a given relaxed refresh transaction to a globally visible point via a data bus regardless of whether any previously sent strongly ordered transactions are waiting to be globally observed by the consumer.

[0024] Furthermore, according to embodiments of the present disclosure, the processing circuitry is configured to: operate according to a hierarchy of relaxation refresh levels; and send a given relaxation refresh transaction to a globally visible point via a data bus, regardless of previously sent relaxation refresh transactions having the same or higher relaxation refresh levels as the given relaxation refresh transaction that are waiting to be globally observed by the consumer.

[0025] Additionally, according to embodiments of this disclosure, the processing circuitry is configured to: in response to the absence of any previously sent lower-level relaxation refresh transaction with a lower relaxation refresh level than the given relaxation refresh transaction waiting to be globally observed by the consumer, send the given relaxation refresh transaction to a globally visible point via a data bus.

[0026] According to yet another embodiment of this disclosure, a method is also provided, the method comprising: receiving a network stream over a network; and, in response to receiving at least one first acknowledgment that all previously sent relaxed ordering transactions are globally observed by a consumer, and prior to receiving a second acknowledgment that a previously sent strong ordering transaction or a given previously sent relaxed refresh transaction is globally observed by a consumer, sending a given relaxed refresh transaction to a globally visible point of a connected device via a data bus.

[0027] According to yet another embodiment of this disclosure, a method is also provided. The method includes: sending transactions via a data bus to a globally visible point of a connected device, each transaction having a transaction type selected from one of the following transaction types: a relaxed ordering transaction type, a relaxed refresh transaction type, or a strongly ordered transaction type; sending a given relaxed ordering transaction to the globally visible point via the data bus, regardless of the transaction type of previously sent transactions that have not yet been globally observed by a consumer; and sending a given relaxed refresh transaction to the globally visible point via the data bus in response to no previously sent relaxed ordering transactions waiting to be globally observed by a consumer, and regardless of whether previously sent relaxed refresh transactions sent to a different memory address than the given relaxed refresh transaction are waiting to be globally observed by a consumer. Attached Figure Description

[0028] This disclosure will be understood from the following detailed description taken in conjunction with the accompanying drawings, in which: Figure 1 This is a simplified block diagram of a system constructed and operated according to embodiments of the present disclosure; Figure 2 It's a diagram. Figure 1 A detailed sequence diagram of transaction flows in the system; Figure 3 It is used for Figure 1 The transaction sorting rule table in the system; Figure 4 It includes Figure 1 A flowchart of the steps in the system's operation method; and Figure 5 The block diagram illustrates a computing system (e.g., a data center or a high-performance computing (HPC) cluster) according to embodiments of the present disclosure. Detailed Implementation

[0029] definition As used herein, the term “producer” can refer to a component or entity in a computing system that generates or sends data, transactions or other information for processing or consumption by other parts of the system.

[0030] As used herein, the term "consumer" can refer to a component or entity in a computing system that receives, processes, or utilizes data, transactions, or other information generated by a producer.

[0031] As used herein, the term “network flow” can refer to a sequence of related data packets or transactions transmitted over a network, typically associated with a specific communication session or data flow between network endpoints.

[0032] As used in this article, the term "global visibility point" can refer to the location or state in a system where data or transactions become visible or accessible to all relevant components or consumers within the system.

[0033] As used herein, the term “globally visible to consumers” can refer to a state or condition in which data or transactions reach a point in the system that can be observed or accessed by all relevant consumers or components that may need to process or utilize that information.

[0034] Overview of Example Implementations As previously mentioned, traditional PCIe systems enforce strict transaction ordering, which can limit scalability and efficiency, especially in high-bandwidth scenarios. This strict ordering ensures data integrity but leads to inefficiency when high bandwidth is required. In out-of-order I / O (UIO) systems, producers must wait for acknowledgments (ACKs) before sending flags, causing latency and hindering the system from efficiently sending subsequent data. This process, known as source ordering, requires producers to ensure all data transactions are globally visible to consumers before sending flags, which can significantly impact system performance in high-throughput environments.

[0035] One potential solution is to send the flags as a relaxed-order (RO) transaction after the data is globally visible to consumers. However, this approach introduces new challenges: sending flags sequentially reduces bandwidth. The problem aims to alleviate the latency and bandwidth limitations caused by waiting for ACKs and sending flags sequentially in high-performance systems while maintaining data integrity and consistency.

[0036] Embodiments of this disclosure address at least some of the aforementioned drawbacks by providing a system and method for introducing a new UIO transaction called a relaxed flusher transaction. This new transaction type can be used in conjunction with relaxed-ordered transactions and strongly-ordered transactions, thus providing an intermediate level of ordering flexibility. This allows for more efficient transaction processing while maintaining the necessary ordering constraints.

[0037] In some embodiments, a relaxation refresh transaction can override a strongly ordered transaction and another relaxation refresh (assuming they have different memory addresses), but cannot override a relaxed ordered transaction. In some cases, the restriction of different memory addresses can be lifted.

[0038] As used in this paper, the term "overtaking" can refer to the ability of a transaction to be processed and globally visible to consumers before another transaction that was initiated earlier in the sequence. When a transaction "overtakes" another transaction, it may reach a point of global visibility and become observable to consumers before the transaction it overtakes, potentially altering the original order of operations. This overtaking behavior allows for more flexible and efficient transaction processing in systems with unordered inputs / outputs, while still maintaining certain ordering constraints to ensure data consistency and integrity.

[0039] In some cases, the system can operate according to a hierarchy of relaxation refresh levels, providing finer-grained control over transaction ordering. The rules used for relaxation refresh transactions may vary depending on their relative levels. Relaxation refresh transactions cannot override lower-level relaxation refresh transactions, relaxation refresh transactions of the same level at the same memory address (in some cases), or relaxed-ordered transactions. Relaxation refresh transactions can override higher-level relaxation refresh transactions, relaxation refresh transactions of the same level at different memory addresses, and strongly ordered transactions. This hierarchy allows for fine-tuning of transaction ordering, enabling optimized handling of various transaction types and completion notifications. The system can adjust the override rules based on the relative levels of relaxation refresh transactions, potentially improving overall system performance and efficiency.

[0040] The advantages of this solution include reduced latency, improved bandwidth utilization, and enhanced flexibility in transaction ordering. By allowing certain transactions to bypass others under specific conditions, the system can achieve higher throughput while maintaining data consistency and integrity.

[0041] System Description Now for reference Figure 1 This is a block diagram illustrating a system 10 for managing transaction ordering in network communications. System 10 may include a network device 12 and a connected device 14. In some cases, the network device 12 may be a peripheral device, while the connected device 14 may be a host device or a peer device.

[0042] System 10 can enable communication between one or more senders 16 and one or more consumers 18 via network stream 20 (or any suitable data stream). Network device 12 may include network interface 30 (or any suitable connection) configured to receive network stream 20 from the one or more senders 16 via network 22.

[0043] In some cases, network device 12 may include network interface controller 22. Network interface controller 22 may include packet processing circuitry 34 and network interface 30. Packet processing circuitry 34 may be configured to process packets of network flow 20.

[0044] Network device 12 may also include processing circuitry 26, memory 28, and data bus interface 32. Data bus interface 32 may be configured to share data with connected device 14 via data bus 36. In some embodiments, the processing circuitry may be included in the packet processing circuitry 34.

[0045] In operation, network device 12 can receive network streams 20 from one or more senders 16 via network interface 30. As a producer, network device 12 can process these network streams 20, generate transactions 40, and send the transactions 40 to a globally visible point 48 in the connected device 14 via data bus 36, making them globally visible to one or more consumers 18. The transaction can be generated by a processing circuitry based on the network streams 20.

[0046] In some cases, each transaction 40 may have a transaction type selected from the following transaction types: relaxed order transaction type, relaxed refresh transaction type, or strong order transaction type. The processing circuitry system 26 may be configured to send a given relaxed refresh transaction to the global visibility point 48 of the connected device 14 via the data bus 36.

[0047] The connected device 14 may include a data bus interface 42, one or more processors 44, and memory 46. A globally visible point 48 in the connected device 14 may be a location where a transaction can be observed by the one or more consumers 18. Figure 1 A global visibility point 48 in memory 46 is shown. In some embodiments, the global visibility point 48 may be set in any suitable location, such as in one or more processors 44.

[0048] The data bus interface of the connected device 14 is configured to generate an acknowledgment 38 for transaction 40 and send the acknowledgment 38 to the network device 12 via the data bus 36. The acknowledgment 38 can be used to confirm the global visibility of transaction 40 to one or more consumers 18.

[0049] Now for reference Figure 2 This is a sequence diagram 200 illustrating an example transaction flow and acknowledgments between network device 12 and connected device 14. Sequence diagram 200 demonstrates the ordering and management of different transaction types to maintain data integrity while improving efficiency.

[0050] In some cases, the processing circuitry 26 may send a Level 1 (first) relaxation refresh transaction 202 to the connected device 14. Following this, the processing circuitry 26 may send a first relaxation sorting transaction 204. The connected device 14 can then respond with a first acknowledgment 206 to the Level 1 (first) relaxation refresh transaction 202.

[0051] The processing circuitry 26 can continue by sending a second relaxation ordering transaction 208. In some cases, the connected device 14 can send an acknowledgment 210 for the first relaxation ordering transaction 204 and an acknowledgment 214 for the second relaxation ordering transaction 208. This sequence illustrates that the processing circuitry 26 can send relaxation ordering transactions (which may include data transactions) to a globally visible point 48 via the data bus 36 without waiting for acknowledgments of previously sent transactions (including relaxation refresh transactions).

[0052] In some cases, the processing circuitry 26 may send a Level 2 (second) relaxation refresh transaction 216. The connected device 14 may respond with an acknowledgment 220 of this Level 2 (second) relaxation refresh transaction 216. Subsequently, the processing circuitry 26 may send a Level 1 (third) relaxation refresh transaction 222, and the connected device 14 may respond with an acknowledgment 224 of this Level 1 (third) relaxation refresh transaction 222. As explained above, all previously sent relaxation ordering transactions have been acknowledged before any relaxation refresh transaction is sent.

[0053] Sequence diagram 200 illustrates that the processing circuitry 26 can send a given relaxation refresh transaction (e.g., a level 1 third relaxation refresh transaction 222) before receiving an acknowledgment 220 of a previously sent relaxation refresh transaction (e.g., an acknowledgment 220 for a level 2 second relaxation refresh transaction 216). This can occur when the level of a subsequent relaxation refresh transaction is lower than one or more previous relaxation refresh transactions, and / or when those one or more previous relaxation refresh transactions are sent to different memory addresses. Sequence diagram 200 also illustrates that the processing circuitry 26 can send another relaxation refresh transaction 230 (e.g., another level 1 fourth refresh transaction RF4) before receiving an acknowledgment 224 for that level 1 (third) relaxation refresh transaction 222. In other words, two relaxation refreshes at the same level can be sent without waiting (in some cases, the same memory address requirement needs to be met). Acknowledgment for relaxation refresh transaction 230 is acknowledgment 232. The relaxation refresh transaction may include a flag or a completion notification.

[0054] In some cases, the processing circuitry 26 may send a strongly ordered transaction 226. The connected device 14 may respond with an acknowledgment of the strongly ordered transaction 226. The strongly ordered transaction 226 may include a flag or a completion notification. Figure 2 It is shown that strongly ordered transaction 228 is sent after all pending transactions have been acknowledged.

[0055] Sequence diagram 200 illustrates the processing circuit system 26, which can be referenced. Figure 3 The sorting rules are described in more detail, using an unordered input-output protocol to send relaxed flush transactions, relaxed sorted transactions, and strongly ordered transactions.

[0056] In some cases, the processing circuitry 26 can send relaxed ordering transactions to the globally visible point 48 via the data bus 36, regardless of the transaction type of previously sent transactions that have not yet been globally observed by the consumer 18.

[0057] Before sending a relaxation refresh transaction to the global visibility point 48 of the connected device 14 via the data bus 36, the processing circuitry system 26 may wait for the following confirmations: one or more confirmations that all previously sent relaxation ordering transactions are globally observed by the consumer 18, or one or more confirmations of previously sent relaxation refresh transactions with a lower or equal level and the same memory address. The processing circuitry system 26 may be configured to send the relaxation refresh transaction to the global visibility point 48 of the connected device 14 via the data bus 36 in response to receiving one or more confirmations that all previously sent relaxation ordering transactions are globally observed by the consumer 18, and / or one or more confirmations of previously sent relaxation refresh transactions with a lower or equal level and the same memory address. In some embodiments, the processing circuitry system 26 may be configured to send the given relaxation refresh transaction to the global visibility point 48 via the data bus 36 in response to no previously sent relaxation refresh transactions with a lower relaxation refresh level than the given relaxation refresh transaction waiting to be globally observed by the consumer 18, and / or in response to no previously sent relaxation ordering transactions of the same level waiting to be globally observed by the consumer 18 at the same memory address.

[0058] In some cases, the processing circuitry 26 may send a relaxed refresh transaction to the globally visible point 48 via the data bus 36 before receiving confirmation that a previously sent strong ordering transaction or a previously sent (e.g., a higher-level or the same-level but different memory address) relaxed refresh transaction has been globally observed by the consumer 18.

[0059] In some cases, the processing circuitry system 26 can send a Completion Queue Entry (CQE) as a relaxed refresh transaction to refresh all relaxed-ordered writes preceding it. The processing circuitry system 26 can also send an Event Queue Entry (EQE) as a higher-level relaxed refresh transaction (above the CQE level) to refresh all CQEs and data preceding it. Additionally, the processing circuitry system 26 can send an MSI-X interrupt as a strong-ordered transaction to push all content preceding it.

[0060] In some cases, processing circuitry 26 may send a given strongly ordered transaction to global visibility point 48 via data bus 36 in response to the absence of any previously sent transactions waiting to be globally observed by consumer 18. Processing circuitry 26 may also send a given relaxed refresh transaction to global visibility point 48 via data bus 36, regardless of whether previously sent strongly ordered transactions are waiting to be globally observed by consumer 18.

[0061] Sequence diagram 200 shows that the processing circuit system 26 can manage different transaction types (relaxed sorting, relaxed refresh, and strong sorting) to maintain data integrity while improving the efficiency of high-performance systems.

[0062] Now for reference Figure 3 This is a transaction sorting rule table 300, which illustrates the relationship between different types of transactions and their transcendence rules. The transaction sorting rule table 300 includes a transaction type column 302, a "transactions that can be transcended" column 304, and a "transactions that cannot be transcended" column 306.

[0063] Transaction sorting rule table 300 defines the sorting rules for different transaction types, including relaxed order (RO), relaxed refresh (RF) at various levels, and strong order (SO) transactions. These rules manage how transactions can be sent and processed by the processing circuit system 26.

[0064] For RO transactions, the "can be surpassed" transaction column 304 indicates that any transaction can be surpassed by an RO transaction. This means that the processing circuitry 26 can send a relaxed ordering transaction to the globally visible point 48 via the data bus 36 before any previously sent transaction has been globally observed by the consumer 18.

[0065] RF transactions are organized in a hierarchical structure, such as RF level 1, RF level 2, RF level N, and so on. The processing circuitry system 26 can operate according to this hierarchical structure of relaxation refresh levels. For each RF level, rules specify which transactions can be skipped and which cannot.

[0066] For RF Level 1 transactions, column 304, "Transactions that can be surpassed," indicates that an RF Level 1 transaction can surpass RF Level 2 and higher transactions, RF Level 1 transactions with different memory addresses, and any SO transactions. Column 306, "Transactions that cannot be surpassed," indicates that an RF Level 1 transaction cannot surpass any RO and RF Level 1 transactions with the same memory address.

[0067] Therefore, the processing circuitry 26 can be configured to wait for confirmation that another relaxation refresh transaction is also globally observed by the consumer 18 at the same memory address before sending the relaxation refresh transaction to the globally visible point 48 at the memory address. The processing circuitry 26 can then, in response to receiving this confirmation, send the relaxation refresh transaction to the globally visible point 48 via the data bus 36.

[0068] For RF Level 2 and higher, similar rules apply, but with additional restrictions. Before sending a given relaxation refresh transaction to global visibility point 48, the processing circuitry system 26 may wait for one or more acknowledgments of the following transactions to be globally observed by consumer 18: all previously sent relaxation refresh transactions with a relaxation refresh level lower than that of the given relaxation refresh transaction; or another previously sent relaxation refresh transaction with the same refresh level as the given relaxation refresh transaction and sent to the same memory address as the given relaxation refresh transaction.

[0069] In some cases, the processing circuitry 26 can send a given relaxation refresh transaction to the global visibility point 48 via the data bus 36, regardless of previously sent relaxation refresh transactions with the same or higher relaxation refresh level waiting to be globally observed by the consumer 18. In some cases, this may occur when a previously sent relaxation refresh transaction and the given relaxation refresh transaction are being sent to different memory addresses in the global visibility point 48.

[0070] Because relaxation refreshes of the same level can bypass each other (and in some cases follow the same memory sorting criteria), the processing circuitry 26 can send multiple relaxation refresh transactions, including corresponding completion notifications, to the globally visible point 48 via the data bus 36 in geographical order rather than chronological order. This allows for efficient transaction processing while maintaining the necessary sorting rules.

[0071] For SO transactions, the unsurpassable transaction column 306 indicates that they cannot surpass any other transaction. Before sending the strongly ordered transaction to the globally visible point 48, the processing circuitry system 26 may wait for one or more acknowledgments of all previously sent transactions to be globally observed by the consumer 18. The processing circuitry system 26 may then, in response to receiving one or more acknowledgments of all previously sent transactions being globally observed by the consumer 18, send the strongly ordered transaction to the globally visible point 48 via the data bus 36.

[0072] These transaction ordering rules can efficiently and flexibly handle different transaction types while maintaining necessary ordering constraints and ensuring appropriate visibility to consumer 18.

[0073] Now for reference Figure 4 It is illustrated in the diagram for managing in Figure 1 The flowchart shows the method 400 for sending transactions in System 10. Method 400 includes various steps for handling different transaction types and their confirmation.

[0074] Step 402 involves receiving network streams from one or more senders. Following this, step 404 manages the transmission of transactions based on the received network streams. In step 406, method 400 sends relaxed-ordered transactions even when acknowledgments (ACKs) for other transaction types have expired.

[0075] At step 408, a relaxation refresh transaction is generated. In step 410, method 400 waits for one or more ACKs from one or more previously sent RO transactions. Determination step 412 determines whether one or more ACKs from one or more previously sent RO transactions have been received. If one or more ACKs have not been received, method 400 returns to step 410 to continue waiting. If all outstanding one or more ACKs have been received, method 400 proceeds to step 414, where a relaxation refresh transaction is sent.

[0076] In optional step 416, method 400 generates a strongly ordered transaction. Step 418 involves waiting for one or more ACKs from all previously sent transactions. Method 400 then proceeds to decision step 420, where it determines whether one or more ACKs have been received for all previously sent transactions. If not all pending ACKs have been received, method 400 returns to step 418 to continue waiting. If all pending ACKs have been received, method 400 proceeds to step 422, where it sends the strongly ordered transaction.

[0077] In some cases, the processing circuitry can operate according to a hierarchy of relaxation refresh levels. This hierarchy can correspond to a hierarchy of different completion notification types. For example, the hierarchy may include data event queue entries, completion queue entries, and MSIX (Message Signaling Interruption Extension).

[0078] Method 400 ensures that transactions are sent in an order that optimizes performance while maintaining data consistency. By allowing relaxed-ordered transactions to be sent without waiting for confirmation from other transactions, and by implementing a hierarchy of relaxed refresh levels, Method 400 provides flexibility in transaction ordering while still maintaining the necessary ordering constraints for different transaction types and completion notifications.

[0079] In some implementations, the ordering rules for specific use cases can be modified. For example, in high-throughput scenarios, a relaxation refresh transaction can be allowed to override other relaxation refresh transactions of the same level, even if they are being sent to the same memory address. In some cases, this modification can improve overall system performance.

[0080] The system can also support dynamically adjusting transaction sorting rules based on runtime conditions. For example, rules can be automatically adjusted based on network congestion, processing load, or other performance metrics.

[0081] In some cases, a hybrid approach can be implemented where some streams or data types use the described relaxed refresh mechanism, while others use traditional strongly ordered transactions. This hybrid approach is particularly beneficial in systems where different data types have different performance requirements.

[0082] For reference Figure 5 This is a block diagram schematically illustrating a computing system 500 (e.g., a data center or a high-performance computing (HPC) cluster) according to embodiments of the present disclosure.

[0083] According to at least one embodiment, system 500 includes multiple subsystems, such as multiple processing devices, multiple network devices, and multiple networks coupled to each other. The computing system 500 is designed to have multiple integrated circuits (referred to as processing devices), wherein each integrated circuit may include one or more CPUs and GPUs, thereby forming a powerful and flexible architecture.

[0084] The various processing devices are interconnected via NVLink or other high-speed interconnects to enable high-speed communication between subsystems, and are also connected via NICs or DPUs to ensure efficient data transmission across computing system 500 and to one or more external networks 530, 536. In this example, system 500 includes packet switch 548 connecting NIC / DPU 528 to network 530, and packet switch 550 connecting NIC / DPU 532 to network 536.

[0085] The processing device, coupled via NVLink, allows for seamless data exchange and parallel processing, enhancing overall computing performance. Connecting to multiple networks via one or more Network Interface Cards (NICs) or Data Processing Units (DPUs) enables the system to handle complex multi-network tasks with high bandwidth and low latency. This configuration is ideal for demanding applications requiring significant processing power, such as artificial intelligence (AI), machine learning (ML), and data-intensive computing, while ensuring robust connectivity and scalability across diverse network environments. The integrated circuits of the computing system 500 may include one or more CPUs and one or more GPUs.

[0086] Figure 5 An example architecture for a multi-GPU architecture is also demonstrated. As shown in the figure, the computing system 500 includes a processing device 502 with a multi-GPU architecture. Specifically, the processing device 502 may be a system-on-a-chip and includes multiple subsystems, such as a CPU 506, a GPU 508, and a GPU 510. The CPU 506 may be coupled to the GPU 508 via die-to-die (D2D) or chip-to-chip (C2C) interconnects 512, such as a ground reference signal interconnect (GRS interconnect). The CPU 506 may be coupled to the GPU 510 via a D2D or C2C interconnect 514. The CPU 506 may also be coupled to the GPU 508 and GPU 510 via a PCIe interconnect.

[0087] The CPU 506 can be coupled to one or more NICs or DPUs, which are then coupled to one or more networks. For example, as Figure 5 As shown, CPU 506 is coupled to a first NIC / DPU 526, which is coupled to network 530. CPU 506 is also coupled to a second NIC / DPU 528, which is coupled to network 530 via switch 548. For example, NIC / DPU 526 and NIC / DPU 528 can be coupled to network 530 via Ethernet (ETH), NVLINK, or InfiniBand (IB) connections.

[0088] The computing system 500 also includes a processing device 504 with a multi-GPU architecture. Specifically, the processing device 504 includes multiple subsystems, including a CPU 516, a GPU 518, and a GPU 520. The CPU 516 can be coupled to the GPU 518 via a D2D or C2C interconnect 522. The CPU 516 can be coupled to the GPU 520 via a D2D or C2C interconnect 524. The CPU 516 can also be coupled to the GPU 518 and GPU 520 via a PCIe interconnect. The CPU 516 can be coupled to one or more NICs or DPUs, which are coupled to one or more networks. For example, as... Figure 5 As shown, CPU 516 is coupled to a first NIC / DPU 534, which is coupled to network 536. CPU 516 is also coupled to a second NIC / DPU 532, which is coupled to network 536 via switch 550. NIC / DPU 532 and NIC / DPU 534 can be coupled to network 536 via Ethernet (ETH), NVLINK, or InfiniBand (IB) connections.

[0089] In at least one embodiment, processing device 502 and processing device 504 may communicate with each other via NIC / DPU 538, for example, via PCIe interconnect. Processing device 502 and processing device 504 may also communicate with each other via high-bandwidth communication interconnect 540, such as NVLink interconnect or other high-speed interconnect. Figure 5 The packet switches in the diagram may include, for example, Nvidia Quantum-2 switches. The NICs / DPUs in the diagram may include, for example, Nvidia Bluefield DPUs.

[0090] The device can be any suitable device, such as: an accelerator device; a processing device including a central processing unit (CPU) and / or a graphics processing unit (GPU); a host device connected to one or more peripheral devices; a peripheral device connected to another peripheral device and / or one or more host devices; a network device, such as a network interface controller (NIC) device, a data processing unit (DPU), or a smart NIC including a NIC and one or more processing cores, or a network switch. One or more processing steps described herein can be performed by a CPU, GPU, DPU, NIC, or any suitable combination thereof.

[0091] A NIC may include any of the following: an Ethernet port (RJ45 connector), which is the physical interface through which a network cable (typically an Ethernet cable) connects to the NIC for wired network connectivity; packet processing hardware or circuitry responsible for handling network communications, processing incoming and outgoing data packets, and managing network interface functions; memory (e.g., RAM or ROM) for storing temporary data, such as network packet buffers, configuration settings, and firmware, and for helping to accelerate data transmission and processing; firmware, which is software programmed into the NIC's memory that controls hardware operation and can perform firmware updates to improve performance or add new features to the NIC; LED indicators that provide visual indication of network status, with common indicators including power status, network activity, and link speed; a bus interface (e.g., PCI or PCIe) for connecting the NIC to the host computer's motherboard; a processor for handling network processing tasks and other processing tasks to reduce the workload of the host device's main CPU and improve network performance; a heatsink or cooling system (e.g., for high-performance NICs), particularly those for servers, to prevent overheating; power management circuitry to ensure the NIC receives adequate power and manages power consumption effectively; and / or connector pins and circuitry, including internal connections and paths for routing signals between NIC components.

[0092] Packet processing hardware or circuitry is the core component of the NIC and handles network communications. It may include several key components that work together to manage and process network data, such as one or more of the following: a MAC (Media Access Control) layer, which handles the data link layer of the OSI model and manages how packets are formatted, addressed, and transmitted over the network; a MAC address register, which stores the NIC's unique hardware address (MAC address); a frame buffer, which temporarily holds data frames during processing; a PHY (Physical Layer) interface, which interfaces with the physical medium (e.g., Ethernet cable) and is responsible for the actual sending and receiving of data bits over the network; a transceiver, which converts data between the digital signals used at the MAC layer and the analog signals used for transmission over the network medium; a DMA (Direct Memory Access) controller, which manages data transfer between the NIC and computer memory without CPU intervention, helping to offload CPU processing tasks and improve data transfer efficiency; and a packet processing engine, which handles the encapsulation and decapsulation of network packets and processes incoming and outgoing packets, managing... The NIC is equipped with several components: a transmission buffer for temporarily storing packets (e.g., a transmit buffer for storing packets being sent from the computer to the network, and a receive buffer for storing packets received from the network before being processed by the system); an interrupt controller for managing and generating interrupts to notify the CPU of events such as packet reception or transmission completion, and for facilitating efficient handling of network events; a clock generator for providing timing signals to the various components of the NIC to synchronize their operation; a power management unit for regulating power consumption and managing the energy-saving features of the NIC chip to improve energy efficiency; error handling and correction logic for detecting and correcting errors in data transmission and reception, and may include features for error-checking protocols such as CRC (Cyclic Redundancy Check); configuration registers for storing configuration settings and parameters that control NIC operation, such as speed settings, interrupt configuration, and buffer size; and firmware / ROM containing embedded software that controls NIC operation and manages network protocols.

[0093] A network switch may include any of the following: ports for network cable connections; a switching structure for data transmission between management ports; a MAC address table that stores device addresses and port information; a forwarding engine that directs packets to the correct port; a buffer that temporarily stores data to manage traffic; a management processor that handles configuration and monitoring in a managed switch; a power supply that provides electrical power; a cooling system that prevents the switch from overheating; firmware that controls the switch; LED indicators that display status and activity; and network modules (in modular switches) that allow for additional ports or features.

[0094] In practice, some or all of these functions may be combined in a single physical component, or alternatively, implemented using multiple physical components. These physical components may include hardwired or programmable devices, or a combination of both. In some embodiments, at least some functions of the processing circuitry may be executed by a programmable processor under the control of suitable software. This software may be downloaded to the device electronically, for example, via a network. Alternatively, or additionally, the software may be stored in a tangible, non-transitory computer-readable storage medium, such as optical memory, magnetic memory, or electronic memory.

[0095] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various examples of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, code segment, or code portion, comprising one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in a block may occur in a non-concurrent order as shown in the drawings. For example, two blocks shown consecutively may actually be executed substantially concurrently, or these blocks may sometimes be executed in reverse order. The block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented by a dedicated hardware system, or a combination of dedicated hardware and computer instructions, that performs the specified function or action. The description of the various examples of this disclosure is presented for illustrative purposes and is not intended to be exhaustive or limited to the disclosed examples. Many modifications and variations will occur to those skilled in the art without departing from the scope and spirit of the described examples.

[0096] As used herein, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” include plural references.

[0097] For clarity, the various features of this disclosure described in the context of individual embodiments may also be provided in combination in a single embodiment. Conversely, for brevity, the various features of this disclosure described in the context of a single embodiment may also be provided individually or in any suitable sub-combination.

[0098] The above embodiments are cited by way of example, and this disclosure is not limited to the content already specifically shown and described above. Rather, the scope of this disclosure includes combinations and sub-combinations of the various features described above, as well as variations and modifications thereto that would occur to those skilled in the art upon reading the above description and that are not disclosed in the prior art.

Claims

1. An apparatus, the apparatus comprising: A network interface, wherein the network interface is used to receive network streams over a network; A data bus interface, wherein the data bus interface is used to share data with connected devices via a data bus; and A processing circuitry system is configured to: send a given relaxation refresh transaction via the data bus to a global visibility point of the connected device in response to the following conditions: Receive at least one first acknowledgment of all previously sent relaxed-ordered transactions that are globally observed by the consumer; and Before receiving a second acknowledgment globally observed by the consumer for either a previously sent strongly ordered transaction or a given previously sent relaxed refresh transaction.

2. The device of claim 1, wherein the processing circuitry is configured to: wait for the at least one first acknowledgment before sending the given relaxation refresh transaction to the global visibility point of the connected device via the data bus.

3. The device according to claim 1 further includes a network interface controller, the network interface controller including a packet processing circuit system, the packet processing circuit system including the processing circuit system.

4. The apparatus of claim 1, wherein the processing circuitry is configured to: send the given relaxed refresh transaction, the previously sent relaxed ordered transaction, and the previously sent strong ordered transaction using an out-of-order input-output protocol.

5. The apparatus of claim 1, wherein the processing circuitry is configured to: send the given relaxed refresh transaction to the globally visible point via the data bus before receiving a second acknowledgment that the previously sent strongly ordered transaction is globally observed by the consumer.

6. The apparatus of claim 1, wherein the processing circuitry is configured to: send the given relaxation refresh transaction to the globally visible point via the data bus before receiving a second acknowledgment that the given previously sent relaxation refresh transaction is globally observed by the consumer.

7. The apparatus of claim 1, wherein the processing circuitry is used for: Before sending the given relaxed refresh transaction to the globally visible point at the given memory address, wait for a third acknowledgment that another relaxed refresh transaction is also globally observed by the consumer at the given memory address; and In response to receiving the third confirmation, the given relaxation refresh transaction is sent to the global visibility point via the data bus.

8. The apparatus of claim 1, wherein the processing circuitry is configured to: send a relaxed ordering transaction to the globally visible point via the data bus while any previously sent transaction has not yet been globally observed by the consumer.

9. The apparatus of claim 8, wherein the processing circuitry is used for: Before sending a strongly ordered transaction to the globally visible point, wait for at least one acknowledgment of all previously sent transactions to be globally observed by the consumer; and In response to receiving at least one acknowledgment that all previously sent transactions are globally observed by the consumer, the strongly ordered transaction is sent to the globally visible point via the data bus.

10. The device according to claim 1, wherein: The previously sent relaxed sorting transaction includes data; and The given relaxation refresh transaction includes a flag or completion notification; and The previously sent strongly ordered transactions include flags or completion notifications.

11. The apparatus of claim 1, wherein the processing circuitry is configured to: send a plurality of relaxed refresh transactions, including corresponding completion notifications, to the globally visible point via the data bus in geographical order rather than chronological order.

12. The apparatus of claim 1, wherein the processing circuitry is used for: Operate according to the hierarchy of relaxation refresh levels; Before sending the given relaxation refresh transaction to the global visibility point, wait for at least one third confirmation of the following transactions to be globally observed by the consumer: all previously sent relaxation refresh transactions with a relaxation refresh level lower than that of the given relaxation refresh transaction. Alternatively, another previously sent relaxed refresh transaction with the same relaxed refresh level as the given relaxed refresh transaction and sent to the same memory address as the given relaxed refresh transaction. as well as In response to receiving the at least one third confirmation, the given relaxation refresh transaction is sent to the global visibility point via the data bus.

13. The device according to claim 12, wherein: The given previously sent relaxation refresh transaction has the same relaxation refresh level as or a higher relaxation refresh level than the given relaxation refresh transaction; and The processing circuitry is configured to: send the given relaxation refresh transaction to the globally visible point via the data bus before receiving a second acknowledgment that the given previously sent relaxation refresh transaction has been globally observed by the consumer.

14. The device according to claim 13, wherein: The given previously sent relaxation refresh transaction has the same relaxation refresh level as the given relaxation refresh transaction. The given previously sent relaxation refresh transaction and the given relaxation refresh transaction are being sent to different memory addresses in the globally visible point; and The processing circuitry is configured to: send the given relaxation refresh transaction to the globally visible point via the data bus before receiving a second acknowledgment that the given previously sent relaxation refresh transaction has been globally observed by the consumer.

15. The device of claim 13, wherein the hierarchy of relaxation refresh levels corresponds to the hierarchy of different completion notification types.

16. An apparatus, the apparatus comprising: A data bus interface, wherein the data bus interface is used to share data with connected devices via a data bus; and a processing circuit system, the processing circuit system being used for: Transactions are sent to the global visibility point of the connected device via the data bus, and each transaction has a transaction type selected from one of the following transaction types: relaxed sorted transaction type; Relax refresh transaction type; Or strongly ordered transaction types; Regardless of the transaction type of previously sent transactions that have not yet been globally observed by the consumer, a given relaxed ordering transaction is sent to the globally visible point via the data bus; as well as A given relaxation refresh transaction is sent to the globally visible point via the data bus under the following conditions: In response to the absence of previously sent relaxed-ordered transactions waiting to be globally observed by the consumer; and Regardless of whether a previously sent relaxation refresh transaction is sent to a different memory address than the given relaxation refresh transaction, the consumer will observe the relaxation refresh transaction globally.

17. The apparatus of claim 16, wherein the processing circuitry is configured to: In response to the absence of previously sent transactions waiting to be globally observed by the consumer, a given strongly ordered transaction is sent to the globally visible point via the data bus; and The given relaxed refresh transaction is sent to the globally visible point via the data bus, regardless of any previously sent strongly ordered transactions waiting to be globally observed by the consumer.

18. The apparatus of claim 16, wherein the processing circuitry is configured to: Operate according to the hierarchy of relaxation refresh levels; and The given relaxation refresh transaction is sent to the globally visible point via the data bus, regardless of whether a previously sent relaxation refresh transaction with the same or higher relaxation refresh level as the given relaxation refresh transaction is waiting to be globally observed by the consumer.

19. The apparatus of claim 18, wherein the processing circuitry is configured to: send the given relaxation refresh transaction to the globally visible point via the data bus in response to the absence of a previously sent lower-level relaxation refresh transaction having a lower relaxation refresh level than the given relaxation refresh transaction waiting to be globally observed by the consumer.

20. A method comprising: Receive network streams over the network; as well as In response to the following conditions, a given relaxation refresh transaction is sent to the global visibility point of the connected device via the data bus: Receive at least one first acknowledgment that all previously sent relaxed ordering transactions are globally observed by the consumer; and Before receiving a second acknowledgment globally observed by the consumer for either a previously sent strongly ordered transaction or a given previously sent relaxed refresh transaction.

Images