NUMA-Aware TEP Selection for Cross-Node Flow Routing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Overlay workloads experience network performance degradation due to data message flows crossing non-uniform memory access (NUMA) nodes when sent through a network interface card not associated with the running workload, which traditional link aggregation groups cannot address.
Innovation Solution
A method for forwarding data messages between host computers by identifying a particular NUMA node associated with a data message flow and selecting a destination tunnel endpoint (TEP) within that node, using virtual switches to encapsulate messages with specific TEP IDs, thereby avoiding cross-NUMA node transfers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional link aggregation groups are used for data message forwarding, then device complexity is reduced, but network performance degrades due to cross-NUMA node transfers
Solution Approach 1:
The system performs preliminary action by pre-establishing NUMA node to TEP mapping relationships and pre-configuring forwarding decisions before data message flows occur. The virtual switch maintains mapping information that associates NUMA nodes with their corresponding TEPs, enabling proactive routing decisions that avoid cross-NUMA node transfers without requiring complex real-time analysis during data forwarding.
Solution Approach 2:
The patent introduces an intermediary mechanism through the virtual switch that mediates between the data message flow and the physical network infrastructure. The virtual switch acts as an intermediary by translating overlay network destinations into appropriate TEPs based on NUMA node associations, thereby controlling and optimizing the forwarding path without requiring changes to the underlying physical network or complex decisions at lower layers.
2Productivity
If data message flows are routed through any available TEP, then ease of operation is improved, but network performance deteriorates due to inter-NUMA node transfers
Solution Approach 1:
The patent applies local quality by making forwarding decisions that are locally optimized for each NUMA node. Instead of using a uniform forwarding policy across all TEPs, the system selectively routes data messages through TEPs associated with the local NUMA node where the destination machine resides. This creates locally adapted forwarding paths that maximize bandwidth by avoiding cross-NUMA node transfers, while the virtual switch handles the complexity of determining the appropriate local TEPs.
Data Source
AI summary
Some embodiments provide a novel method for forwarding data messages between first and second host computers. To send, to a first machine of the first host, a second flow from a second machine of the second host in response to a first flow from the first machine, the method identifies from a set of tunnel endpoints (TEPs) of the first host a TEP that is a source TEP of the first flow. The method uses the identified TEP to identify one non-uniform memory access (NUMA) node of a set of NUMA nodes of the first host as the NUMA node associated with the first flow. The method selects, from a subset of TEPs of the first host that is associated with the identified NUMA node, one TEP as a destination TEP of the second flow. The method sends the second flow to the selected TEP of the first host.