NUMA Node Smart NIC Segmentation for Packet Routing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current container networking solutions face challenges in efficiently managing and routing packets between separate NUMA nodes on a single compute node, particularly when multiple smart NICs are involved, leading to configuration failures and performance issues.
Innovation Solution
The implementation of a configuration that includes instances of datapath components, such as virtual router agents and DPDK virtual routers, instantiated for each NUMA node, where each node has a single smart NIC, allowing each NUMA node to operate as a separate bare metal server and enabling efficient packet routing between nodes without crossing NUMA boundaries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If multiple smart NICs are deployed on a single compute node, then network capacity and functionality are improved, but configuration complexity and packet routing efficiency deteriorate due to NUMA boundary crossings
Solution Approach 1:
The compute node is segmented into separate NUMA domains, with each smart NIC bound to a specific NUMA node. This segmentation isolates network devices into distinct memory and CPU regions, preventing cross-NUMA traffic and simplifying configuration by eliminating the need to manage complex inter-NUMA routing paths.
Solution Approach 2:
A virtual switch or routing layer is introduced as an intermediary between workloads on different NUMA nodes. This intermediary handles packet forwarding between NUMA domains through standardized interfaces, abstracting the complexity of multi-NIC configuration and routing from individual applications while maintaining efficient local packet processing within each NUMA domain.
2Quantity of substance
If multiple smart NICs are deployed on a single compute node, then network capacity is improved, but packet routing performance deteriorates due to NUMA boundary crossings
Solution Approach 1:
The compute node is segmented into separate NUMA domains, with each smart NIC bound to a specific NUMA node. This segmentation isolates network devices into distinct memory and CPU regions, preventing cross-NUMA traffic and simplifying configuration by eliminating the need to manage complex inter-NUMA routing paths.
Solution Approach 2:
Each NUMA domain is configured with local packet processing capabilities through virtual switches or routing instances that reside within the same NUMA node as the smart NIC. This ensures that packet processing operations access local memory and CPU resources, maximizing processing speed and minimizing latency by avoiding cross-NUMA memory accesses.
3Speed
If instances of datapath components are instantiated for each NUMA node, then packet routing efficiency is improved by avoiding NUMA boundary crossings, but system complexity increases
Solution Approach 1:
The virtual switch or routing instance is designed as a universal component that can be instantiated on any NUMA node with standardized configuration and behavior. This multi-functionality allows the same component to handle packet processing across different NUMA domains without requiring custom implementations, reducing overall system complexity despite the proliferation of instances.
Solution Approach 2:
Instead of creating a single complex centralized routing system, the patent deploys simplified copies of the datapath component (virtual switch or routing instance) on each NUMA node. Each copy handles local packet processing independently, and the standardized copying approach reduces complexity by eliminating the need for complex inter-component communication protocols required by centralized systems.
Data Source
AI summary
Example techniques and computing devices are disclosed. An example computing device includes a first non-uniform memory access (NUMA) node and a second NUMA nod. The first NUMA node includes a first network interface card, a first virtual router for one or more virtual networks, the first virtual router comprising first processing circuitry and configured with a first virtual host interface having a first Internet Protocol (IP) address, and a first workload executing on the first NUMA node. The second NUMA node includes a second network interface card, a second virtual router for the one or more virtual networks, the second virtual router comprising second processing circuitry and configured with a second virtual host interface having a second IP address, and a second workload executing on the second NUMA node.