Disaggregated Load Balancing With Hardware Flow Cache Offload
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing load balancing solutions force cloud platform providers to choose between software-only solutions that provide maximum flexibility in policy evaluation and flow table management but are inefficient in packet transform operations, and hardware-only solutions that are optimized for high-throughput packet transform operations but limited in table management capabilities and routing policy flexibility.
Innovation Solution
A hybrid load balancing system that disaggregates load balancing tasks into software-implemented logic and specially-purposed load balancing hardware, where software performs stateful load balancing decisions and hardware accelerates packet transform operations, allowing independent scalability and efficient resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If software-based load balancing is used, then flexibility in routing logic and policy evaluation is improved, but packet processing throughput and efficiency deteriorate
Solution Approach 1:
The load balancing system is segmented into two distinct components: a software-based control plane that handles flow admission, policy evaluation, and routing decisions, and a hardware-based data plane that performs high-speed packet transformation and forwarding. This segmentation allows each component to specialize in its strengths while working together as an integrated system.
Solution Approach 2:
A hardware accelerator acts as an intermediary between the software load balancer and the network interface. The software component generates flow entries and routing policies, which are then offloaded to the hardware accelerator that maintains a flow cache and performs rapid packet matching and transformation without involving the CPU for each packet.
2Productivity
If dedicated load balancing hardware is used, then packet processing efficiency is improved, but adaptability to customer-specific routing policies deteriorates
Solution Approach 1:
The hardware accelerator is designed with reconfigurable capabilities that allow it to dynamically adapt to different routing policies and customer requirements. The flow cache structure and matching rules can be programmed and updated via software without requiring hardware redesign, enabling the system to evolve with changing customer needs.
Solution Approach 2:
The system allows dynamic modification of operational parameters such as flow cache size, matching criteria, and transformation rules through software control. This enables the hardware to maintain high processing efficiency while adapting its behavior to match specific customer routing requirements without sacrificing performance.
3Productivity
If smart NIC ports are increased to handle more traffic, then packet processing capacity is improved, but device complexity and cost increase
Solution Approach 1:
Instead of increasing the number of physical smart NIC ports, the system creates virtual copies of flow entries in the hardware accelerator's flow cache. This allows a single physical port to handle multiple logical flows simultaneously through hardware-based flow table lookups, effectively multiplying processing capacity without adding physical hardware.
Solution Approach 2:
The system transitions from scaling in the physical dimension (adding more ports) to scaling in the virtual dimension (increasing flow cache capacity and flow entry multiplexing). This dimensional shift allows the system to achieve higher packet processing capacity by utilizing available bandwidth more efficiently through hardware acceleration rather than adding more physical interfaces.
Data Source
AI summary
A method of load balancing in disaggregated load balancing system includes receiving, at a hardware accelerator, a data packet; performing a lookup-operation in a flow cache of the hardware accelerator; and transmitting the data packet from the hardware accelerator to a flow admission service executed by a software-based load balancing component in response to determining that the flow cache of the hardware accelerator does not yet include a flow entry that matches packet header information of the data packet. The method further includes receiving, from the software-based load balancing component, a new flow entry associated with the data packet that defines a first packet transformation for the data packet; updating the flow cache stored on the hardware accelerator to include the new flow entry; and processing the data packet on the hardware accelerator by applying the first packet transformation.