Hardware Load Balancing via Real-Time Memory Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current load balancing solutions in server systems rely on software-driven methods that use stale load data, leading to inefficient routing of requests as the server load changes, causing delays in servicing requests.
Innovation Solution
A server system with hardware acceleration devices that maintain load data in memory and route requests to target hardware acceleration devices with lower loads, implementing a hardware acceleration plane that collaboratively provides services with a software plane, ensuring real-time load balancing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If software-driven load balancing is used, then implementation complexity is reduced, but load balancing efficiency deteriorates due to stale load data
Solution Approach 1:
The patent replaces software-driven load balancing with hardware-based load balancing using dedicated hardware circuits. The hardware load balancer directly processes routing decisions based on real-time load data stored in hardware memory, eliminating the software processing bottleneck and achieving both low complexity and high efficiency.
Solution Approach 2:
The hardware load balancer proactively maintains real-time load data in hardware memory before routing decisions are needed. This preliminary action ensures that when requests arrive, the load information is already ready for immediate use, eliminating delays associated with software-based real-time queries.
2Loss of time
If real-time load data is used for routing, then request servicing timeliness is improved, but data freshness requirement increases
Solution Approach 1:
The hardware load balancer automatically updates and maintains its own load data in hardware memory without external intervention. Each hardware component self-reportes its load state, and the hardware load balancer autonomously processes this information to generate routing decisions, ensuring continuous data freshness with minimal overhead.
Solution Approach 2:
The hardware memory acts as an intermediary buffer between the distributed hardware components and the routing decision logic. This intermediary storage maintains real-time load data locally at the hardware level, enabling fast routing decisions without requiring constant external data updates or complex synchronization protocols.
3Speed
If hardware acceleration plane is introduced, then processing speed is improved, but system architecture complexity increases
Solution Approach 1:
The system is segmented into distinct hardware planes: a hardware acceleration plane for compute-intensive operations and a separate hardware load balancing plane for routing decisions. This segmentation allows each plane to be optimized independently, with the load balancer focusing solely on routing while the acceleration plane handles processing, thereby managing overall system complexity through modular architecture.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A server system is provided that includes a plurality of servers, each server including at least one hardware acceleration device and at least one processor communicatively coupled to the hardware acceleration device by an internal data bus and executing a host server instance, the host server instances of the plurality of servers collectively providing a software plane, and the hardware acceleration devices of the plurality of servers collectively providing a hardware acceleration plane that implements a plurality of hardware accelerated services, wherein each hardware acceleration device maintains in memory a data structure that contains load data indicating a load of each of a plurality of target hardware acceleration devices, and wherein a requesting hardware acceleration device routes the request to a target hardware acceleration device that is indicated by the load data in the data structure to have a lower load than other of the target hardware acceleration devices.