RDMA Load Balancer Hardware Packet Handlers

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing load balancing techniques are unsuitable for handling RDMA traffic due to high latency and processor usage, as they require software processing of all packets, leading to potential packet loss and congestion, and differ significantly from TCP/IP semantics, making parsing and translation challenging.

Innovation Solution

Implementing a load balancer that only handles the first packet of an RDMA connection and uses packet handlers with hardware circuitry to process subsequent packets directly between the originating and destination servers, reducing software processing and latency, and employing a scale-out design with commodity servers for high availability and scalability.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If a load balancer processes all packets using software, then it can handle TCP/IP traffic with proper semantics, but it incurs high latency and processor usage for RDMA traffic

Engineering Contradiction:

Improveability to handle different traffic typesVSAvoidpacket processing latency

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The packet processing function is segmented into two paths: software processing for control packets and TCP/IP traffic, and hardware processing for data packets. The packet handler circuitry automatically routes RDMA data packets through hardware for low-latency processing while software handles less time-sensitive traffic types.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Software-based packet processing is replaced with hardware-based packet handler circuitry for RDMA data packets. This substitution of mechanical (software) processing with electronic (hardware) processing dramatically reduces latency and processor utilization for high-speed RDMA traffic.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Reliability

If a load balancer processes all packets in software, then it can maintain connection state, but it causes packet loss and congestion when processing rate cannot keep up with arriving packets

Engineering Contradiction:

Improveconnection management reliabilityVSAvoidpacket processing throughput

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent replaces software-based packet processing with hardware-based packet handler circuitry that can process packets at line rate. This hardware acceleration eliminates the bottleneck where software processing speed couldn't keep up with incoming packet rates, preventing queue buildup and packet loss.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

A packet handler circuitry acts as an intermediary between the network interface and the software load balancer. It pre-processes RDMA data packets in hardware, extracting essential information and forwarding only critical control packets to software, thereby distributing the processing load and preventing software overload.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Adaptability or versatility

If RDMA packets are parsed and translated into TCP/IP packets, then they can be handled by standard load balancers, but it exacerbates latency and processing overhead

Engineering Contradiction:

Improvecompatibility with standard load balancersVSAvoidpacket translation time

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The patent extracts the packet translation step from the processing pipeline. Instead of translating RDMA packets to TCP/IP format, the system preserves the original RDMA packet format and processes it directly in hardware, eliminating the time-consuming translation step while maintaining compatibility through protocol awareness.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

Rather than converting RDMA packets to TCP/IP format to fit existing infrastructure, the approach is inverted: the load balancer is made RDMA-aware and processes RDMA packets natively in hardware, eliminating the need for format conversion while achieving the same load balancing objectives.

Inventive Principle:

Principle #13The other way round (Inversion)

4Loss of time

If HCAs are installed to implement RDMA, then RDMA traffic can be handled with low latency, but it increases device complexity and capital costs

Engineering Contradiction:

ImproveRDMA communication latencyVSAvoidnetwork adapter complexity

Core Design Contradiction:

Loss of timeVSDevice complexity

Solution Approach 1:

The load balancer's packet handler circuitry is designed to handle multiple protocols including both RDMA and TCP/IP traffic. This multi-functional capability allows the system to process RDMA packets with low latency using existing load balancer hardware, eliminating the need for separate specialized RDMA network adapters.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent merges RDMA packet handling capability into the standard load balancer infrastructure. By integrating RDMA processing functions into the existing load balancer's hardware circuitry, the system achieves low-latency RDMA communication without requiring additional specialized components like HCAs.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS11218537B2Load balancing in distributed computing systems

Publication Date: 2022.01.04 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

Techniques for facilitating load balancing in distributed computing systems are disclosed herein. In one embodiment, a method includes receiving, at a destination server, a request packet from a load balancer via the computer network requesting a remote direct memory access (“RDMA”) connection between an originating server and one or more other servers selectable by the load balancer. The method can also include configuring, at the destination server, a rule for processing additional packets transmittable to the originating server via the RDMA connection based on the received reply packet. The rule is configured to encapsulate an outgoing packet transmittable to the originating server with an outer header having a destination field containing a network address of the originating server and a source field containing another network address of the destination server.