Load Balancer Metadata Forwarding via TCP Options
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional load balancers face challenges in reliably forwarding metadata and detecting application-level issues, leading to inefficient request routing and potential service outages due to reliance on unreliable health probes and parsing mechanisms.
Innovation Solution
Implementing a load balancing algorithm that uses an unused feature of the transport layer security protocol to forward metadata and a resource-aware ping response mechanism to simulate host unavailability based on resource thresholds, reducing the load on hosts with unexpectedly fast response times and preventing service outages.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If conventional parsing mechanisms are used to forward metadata, then the implementation is simple, but the reliability of metadata forwarding deteriorates due to parsing failures
Solution Approach 1:
The patent introduces a TCP option as an intermediary mechanism to forward metadata. Instead of parsing application-layer protocols directly, the load balancer embeds metadata in TCP options, which are reliably transmitted through the TCP stack. This intermediary approach maintains implementation simplicity while significantly improving metadata forwarding reliability, as TCP options are handled at the transport layer and do not depend on application-layer parsing success.
2Ease of manufacture
If TCP ping probes are used to detect host availability, then the implementation is simple, but the detection capability deteriorates as they cannot detect application-level issues
Solution Approach 1:
The patent segments the health probe functionality into multiple layers: TCP connectivity checks at the transport layer and application-level probes (HTTP, gRPC) at the application layer. Each layer detects different types of issues - TCP probes detect network connectivity while application-level probes detect service availability. This segmentation allows the system to maintain simple TCP ping implementation while adding comprehensive application-level detection capabilities without requiring a complete probe redesign.
3Productivity
If load balancers route requests to hosts with fast response times, then request routing efficiency improves, but service outage risk increases due to hosts being removed from rotation prematurely
Solution Approach 1:
The patent implements dynamic host rotation weights that adjust based on response time performance. Instead of statically removing fast hosts from rotation, the system dynamically modifies their rotation probability - fast hosts receive higher weights and are selected more frequently, while slow hosts receive lower weights. This dynamic adjustment maintains service availability by keeping all hosts in rotation while optimizing request routing efficiency toward healthier hosts, preventing premature removal and potential service outages.
Data Source
AI summary
Network devices, such as load balancers may be configured to forward client metadata to back-end nodes using defined fields of a security protocol. For example, client metadata may be inserted into an extension field or certificate defined by a security protocol that is used for a secure connection between the load balancer and the back-end node. In some instances, a source IP address based on a received request may be inserted into the extension field or certificate defined by the security protocol before the request is forwarded to the back-end node. The back-end node may extract the client metadata and use the client metadata for any of a number of processes (e.g., billing, tracking, security, logging, etc.).