Network I/O Packet Steering with RX/TX Queue Pinning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hashing algorithms for Virtual Network Functions (VNFs) fail to guarantee that initiator and reverse flows are processed by the same CPU core, leading to flow asymmetries and increased packet processing overhead due to synchronization mechanisms and data cache misses.
Innovation Solution
Implementing flow-based lookup in a network I/O device to maintain initiator and reverse flow tables, pinning RX queue IDs to TX queue IDs, ensuring that packets are steered to the same CPU core, even in the presence of NAT or NAPT, thereby eliminating the need for synchronization locks and thread handoffs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If hashing algorithms (e.g., RSS) are used to shard traffic across multiple CPU cores, then traffic distribution is improved, but flow symmetry is disrupted causing initiator and reverse flows to be assigned to different CPU cores
Solution Approach 1:
The patent performs flow-based lookup in initiator and reverse flow tables before packet processing to determine the appropriate CPU core assignment. By pre-establishing flow entries with pinned RX queue IDs to specific CPU cores, the system ensures that both initiator and reverse flows are directed to the same CPU core before any processing occurs, preventing flow symmetry disruption.
Solution Approach 2:
The patent introduces flow tables as an intermediary mechanism between the hashing algorithm and CPU core assignment. The flow tables store mappings between N-tuple values and pinned RX queue IDs, acting as a mediator that overrides the asymmetric behavior of hashing algorithms and ensures symmetric flow handling on the same CPU core.
2Reliability
If synchronization mechanisms (e.g., locks) are used across CPU cores to correct flow asymmetries, then flow symmetry is improved, but packet processing overhead increases
Solution Approach 1:
The patent extracts the flow symmetry enforcement mechanism from the CPU core level and relocates it to the network I/O device level. By performing flow-based lookup and RX queue ID pinning in the network I/O device's flow tables, the system removes the need for synchronization locks and inter-CPU communication mechanisms, thereby eliminating the associated overhead and complexity.
Solution Approach 2:
The network I/O device autonomously performs flow-based lookup and determines appropriate CPU core assignments without requiring synchronization coordination between CPU cores. Each packet is independently routed to the correct CPU core based on pre-established flow entries, eliminating the need for lock-based synchronization mechanisms.
3Reliability
If packet handoffs between CPU cores are implemented, then flow symmetry is improved, but data cache misses increase resulting in decreased network device performance
Solution Approach 1:
The patent performs CPU core assignment through flow-based lookup in advance, before packets arrive. By pre-pinning RX queue IDs to specific CPU cores in the flow tables, the system ensures that packets are directed to the correct CPU core from the beginning, eliminating the need for subsequent packet handoffs between cores and preventing cache misses that would degrade performance.
Solution Approach 2:
The flow tables serve as an intermediary that directly maps packets to the appropriate CPU core through RX queue ID pinning. This intermediary mechanism eliminates the need for packet handoffs between CPU cores by ensuring that both initiator and reverse flows are assigned to the same CPU core from the start, thereby maintaining cache efficiency and network device performance.
Data Source
AI summary
A method performed by a network Input/Output (I/O) device includes, when an outgoing packet is transmitted on a TX queue by a CPU core to the network I/O device, performing a flow-based lookup in an initiator flow table and a reverse flow table stored in the network I/O device to match N-tuples of the outgoing packet. When a matching flow entry is found based on the values of the N-tuples, the method determines whether an RX queue ID stored in the flow entry is pinned to a TX queue ID of the TX queue on which the outgoing packet is transmitted. When the RX queue ID is not pinned to the TX queue ID, the method updates the RX queue ID and the TX queue ID in the initiator and reverse flow tables such that the updated RX queue ID is pinned to the TX queue ID.