Multi-socket NIC Transaction Ordering via Segmented Queues
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
NICs with multiple bus interfaces face challenges in maintaining proper ordering of DMA transactions and optimizing bus resource use, leading to potential data corruption and inefficiencies in high-speed network communications, especially in NUMA architectures.
Innovation Solution
The implementation of packet processing logic that writes data concurrently via multiple NIC bus interfaces and ensures completion reports are written only after data transfer is complete, using DMA transactions and memory mappings to balance communication load and maintain transaction ordering, while presenting a unified network interface to the host system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data are written to host memory concurrently via multiple NIC bus interfaces, then data communication speed is improved, but transaction ordering may be compromised leading to data corruption
Solution Approach 1:
The patent segments DMA transactions by associating each work queue with a specific bus interface. This segmentation ensures that transactions for a given work queue are processed through a single interface, maintaining ordering while allowing concurrent processing across different queues. The completion queue is further segmented into per-bus interfaces, isolating completion reports to their respective buses.
Solution Approach 2:
The patent introduces bus-specific completion buffers as intermediaries between the NIC and host memory. These buffers act as mediators that ensure completion reports are written to memory after data transactions are complete, while allowing concurrent operations across multiple buses. The completion buffer serves as a synchronization point that maintains ordering without serializing all transactions.
2Productivity
If multiple bus interfaces are used for concurrent DMA transactions, then productivity is improved, but device complexity increases
Solution Approach 1:
The patent implements a universal work queue structure that can be associated with any bus interface. The same work queue management logic applies across all interfaces, reducing configuration complexity. The packet processing logic uses identical procedures for initiating DMA transactions and writing completion reports, regardless of which bus interface is used, enabling scalable multi-interface operation without proportionally increasing complexity.
Data Source
AI summary
Computing apparatus includes a host computer, including multiple non-uniform memory access (NUMA) nodes, including at least first and second NUMA nodes, which include first and second local memories and first and second host bus interfaces for connection to first and second peripheral component buses, respectively. A network interface controller (NIC) is to receive a definition of a memory region extending over respective first and second parts of the first and second local memories and to receive a memory mapping with respect to the memory region that is applicable to both the first and second local memories, and to apply the memory mapping in writing data to the memory region via first and second NIC bus interfaces in a sequence of direct memory access (DMA) transactions to the respective first and second parts of the first and second local memories in response to packets received through a network port.