Load Balancer Circuitry Dynamic Core Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current load balancing techniques in packet processing applications face challenges in efficiently managing worker cores, leading to suboptimal power usage and latency, as the number of worker cores provisioned is often based on maximum predicted throughput, which can result in underutilization or overutilization during varying traffic loads.
Innovation Solution
A load balancer circuitry that includes a hardware scheduling unit and credit-based arbiter to dynamically allocate work among worker cores, allowing for power-saving by putting underutilized cores into a low power state and reallocating them as needed based on traffic load, while maintaining low latency through efficient scheduling and queue management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the number of worker cores is provisioned based on maximum predicted throughput, then the system can handle peak traffic loads, but power consumption increases and cores remain underutilized during low traffic periods
Solution Approach 1:
The system dynamically adjusts the number of active worker cores based on real-time traffic load conditions. During peak traffic, all provisioned cores are activated to handle maximum throughput. During low traffic periods, excess cores are deactivated or redirected to other tasks, reducing power consumption while maintaining the ability to quickly scale up when needed.
Solution Approach 2:
The system changes the operational state parameter of worker cores from static (always active) to dynamic (actively managed). By monitoring traffic patterns and adjusting the number of active cores accordingly, the system optimizes the balance between handling peak loads and reducing power consumption during low-utilization periods.
2Loss of time
If worker cores are kept active to handle sudden traffic increases, then processing latency is reduced, but power consumption increases during low traffic periods
Solution Approach 1:
The system performs preliminary provisioning of worker cores based on predicted peak traffic requirements, but implements dynamic activation. Cores are pre-configured and can be rapidly activated when traffic increases, minimizing latency while avoiding continuous operation during low traffic periods. This allows the system to be ready to act without permanently maintaining all cores in an active state.
Solution Approach 2:
The load balancer autonomously monitors traffic patterns and dynamically activates or deactivates worker cores based on actual demand. This self-managed approach ensures that cores are available when needed (maintaining low latency) while automatically reducing power consumption during low traffic periods without requiring manual intervention.
3Use of energy by moving object
If worker cores are redirected to other tasks during low traffic, then power savings are achieved, but latency increases when cores need to be recalled for packet processing
Solution Approach 1:
The system implements dynamic core management where worker cores can be redirected to alternative tasks during low traffic periods to save power. When traffic load increases, the load balancer dynamically recalls and reactivates these cores, balancing power savings with acceptable latency recovery time based on traffic patterns and core availability.
Solution Approach 2:
Worker cores are designed to be multi-functional, capable of performing both packet processing and alternative tasks. During low traffic periods, cores are redirected to other useful tasks (such as maintenance, monitoring, or handling non-time-critical operations), maximizing resource utilization while saving power. When packet processing demand increases, these same cores can be quickly reallocated back to their primary function.
Data Source
AI summary
Examples described herein relate to a load balancer that is configured to selectively perform ordering of requests from the one or more cores, allocate the requests into queue elements prior to allocation to one or more receiver cores of the one or more cores to process the requests, and perform two or more operations of: adjust a number of queues associated with a core of the one or more cores by changing a number of consumer queues (CQs) allocated to a single domain, adjust a number of target cores in a group of target cores to be load balanced, and order memory space writes from multiple caching agents (CAs).