Admission Control for Latency-Critical RPCs in Datacenters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Datacenter networks face challenges in providing predictable performance and meeting latency service-level objectives (SLOs) for Remote Procedure Calls (RPCs), especially during network overloads, as existing congestion control schemes fail to guarantee latency SLOs and prioritize critical traffic effectively.
Innovation Solution
The Admission Control System (ACS) leverages weighted fair queuing to manage RPC traffic by mapping priority classes to network Quality of Service (QoS) levels, allowing for adaptive QoS code point adjustments to ensure latency-sensitive traffic meets SLOs, even under high network loads, by downgrading out-of-profile traffic and making local admission decisions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If congestion control schemes are used to maximize link capacity, then network throughput is improved, but latency SLO guarantees cannot be provided and all RPCs experience slowdown
Solution Approach 1:
The patent segments RPC traffic into multiple QoS classes (e.g., latency-sensitive, throughput-intensive, best-effort) and applies differentiated admission control and congestion control policies to each class. This allows latency-sensitive traffic to receive prioritized treatment while maintaining overall network throughput, resolving the contradiction between maximizing capacity and guaranteeing latency SLOs.
Solution Approach 2:
The patent implements local admission control decisions at each host based on measured RPC latency and offered load per QoS class. Each host independently determines whether to admit or downgrade incoming RPCs based on local conditions, providing localized quality adjustment that maintains latency SLOs without requiring global coordination, thus preserving both throughput and reliability guarantees.
2Productivity
If Weighted-Fair Queuing is used to assign priority, then bandwidth allocation is improved, but mapping application traffic to QoS queues becomes complex and arbitrary mappings yield no guarantees
Solution Approach 1:
The patent enables applications to self-manage their traffic by marking RPCs with priority classes that directly correspond to desired QoS levels. The system automatically translates these markings into appropriate QoS queue assignments without requiring complex centralized mapping configurations. Applications effectively self-service their QoS needs by simply indicating priority preferences, eliminating the complexity of arbitrary mappings while maintaining bandwidth allocation efficiency.
3Loss of time
If small RPCs are prioritized via strict priority, then small RPC latency is improved, but RPC size no longer reflects application priorities across multiple tenants
Solution Approach 1:
The patent replaces static size-based priority with dynamic priority marking where applications can explicitly mark RPCs with priority classes based on their actual importance. The admission control system dynamically adjusts admission decisions based on measured latency and offered load per QoS class, allowing priority to be assigned according to application needs rather than fixed RPC size criteria. This makes the system adaptable to different tenant requirements while maintaining low latency for critical traffic.
4Quantity of substance
If network is over-subscribed for statistical multiplexing, then cost is reduced, but network overloads are inevitable and critical traffic experiences degraded latency
Solution Approach 1:
The patent implements preliminary admission control decisions at host level before traffic enters the network core. By measuring offered load and current latency per QoS class and making advance decisions on whether to admit or downgrade RPCs, the system prevents overload conditions from propagating through the network. This preliminary action allows the network to operate at high utilization while ensuring critical traffic maintains latency SLOs by filtering out excess traffic before it causes congestion.
Data Source
AI summary
A distributed sender driven Admission Control System (ACS) is described herein, leveraging Weighted-Fair Quality of Service (QoS) queues, found in standard NICs and switches, to guarantee RPC level latency service level objectives (SLOs) by a judicious selection of QoS weights and traffic-mix across QoS queues. ACS installs cluster-wide RPC latency SLOs by mapping LS RPCs to higher weight QoS queues, and coping with overloads by adaptively apportioning LS RPCs amongst QoS queues based on measured completion times for each queue. When the network demand spikes unexpectedly to predetermined threshold percentage of provisioned capacity, ACS achieves a latency SLO that is significantly lower than the state-of-art congestion control at the 99.9th-p and admits significantly more RPCs meeting SLO target when RPC sizes are not aligned with priorities.