Dynamic Load Balancer Queues for Multi-Core Webserver Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Commercial webservers using HTTP2/HTTP3 protocols experience uneven core utilization, leading to performance degradation due to inefficient software load balancers that fail to distribute workloads effectively across multi-core processor circuitry, especially when handling multi-object requests.
Innovation Solution
Implementing dynamic load balancer (DLB) circuitry for hardware-based queue management to improve performance and throughput by dynamically balancing workloads across processor cores, reducing latency and eliminating direct core-to-core interactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If software load balancers are used to distribute workloads, then workload distribution is attempted, but core utilization becomes uneven and performance degrades
Solution Approach 1:
The patent replaces software-based load balancing with hardware-based load balancer circuitry integrated into the network interface controller. This hardware implementation directly distributes incoming network packets to processor cores based on packet characteristics, eliminating the software overhead and inefficiencies that caused uneven core utilization and performance degradation.
Solution Approach 2:
The load balancer circuitry acts as an intermediary between the network interface and processor cores. It intercepts incoming packets, determines the appropriate target core based on packet characteristics (such as destination address or flow identification), and redirects packets to the selected core before they reach the main processing queue, thereby achieving balanced core utilization.
2Productivity
If direct core-to-core interactions are implemented, then workload balancing occurs, but latency increases
Solution Approach 1:
The load balancer circuitry performs workload distribution in advance, at the network interface level, before packets reach the processor cores. By pre-distributing packets to the appropriate core queues based on packet characteristics, the system eliminates the need for real-time core-to-core interactions and reduces processing latency.
Solution Approach 2:
The patent segments the packet processing workload by creating separate queues for different processor cores within the network interface controller. Each queue is dedicated to a specific core, allowing packets to be processed independently without requiring coordination or communication between cores, thereby reducing latency.
3Productivity
If more processor cores are utilized, then throughput increases, but core availability for high-value tasks decreases
Solution Approach 1:
The patent applies different processing qualities to different packet types by using packet characteristics (such as protocol type, destination address, or flow identification) to determine the target core. This allows routine packets to be handled by dedicated load-balanced queues while preserving core availability for high-value tasks that require direct core access.
Solution Approach 2:
The load balancer circuitry handles only the necessary portion of packet distribution at the hardware level, directing routine traffic to balanced queues while leaving high-value or special-purpose packets to be processed directly by available cores. This partial action approach maintains core availability for critical tasks while still achieving throughput improvement through hardware-based load balancing.
Data Source
AI summary
Methods, apparatus, systems, and articles of manufacture are disclosed to improve webservers using dynamic load balancers. An example method includes identifying a first and second data object type associated with media and with first and second data objects of the media. The example method also includes enqueuing first and second event data associated with the first and second data object in a first and second queue in first circuitry in a die of programmable circuitry. The example method further includes dequeuing the first and second event data into a third and fourth queue associated with a first and second core of the programmable circuitry, the first circuitry separate from the first core and the second core. The example method additionally includes causing the first and second core to execute a first and second computing operation based on the first and second event data in the third and fourth queues.