Tenant Affinity Load Balancing for Multitenant CPU Utilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing load balancing mechanisms in multitenant environments often result in uneven CPU resource utilization across nodes, leading to disparate user experiences due to nodes being primarily assigned less or highly CPU-intensive services.
Innovation Solution
Implementing a load balancer with tenant affinity, where user requests are directed to nodes with the highest affinity for the requesting tenant, based on the percentage of users from that tenant already assigned to the node, while ensuring the node has available CPU resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If load balancer assigns user requests based on node with least number of assigned users, then user request distribution is simplified, but CPU resource utilization becomes uneven across nodes
Solution Approach 1:
The patent changes the load balancing parameter from simple user count to a composite metric that includes CPU utilization percentage. The load balancer now considers both the number of assigned users and the CPU usage of each node, dynamically adjusting request distribution based on current system state to optimize resource utilization while maintaining operational simplicity
Solution Approach 2:
The system implements feedback by continuously monitoring CPU utilization metrics from each node and using this information to adjust load balancing decisions. The load balancer receives real-time or periodic updates on node CPU usage and incorporates this feedback into its request routing algorithm, creating a closed-loop control system that adapts to changing conditions
2Ease of operation
If load balancer distributes requests to minimize user count per node, then user assignment is simplified, but service performance becomes inconsistent across different nodes
Solution Approach 1:
The patent modifies the request assignment criteria by incorporating CPU utilization thresholds and performance metrics into the load balancing decision process. Instead of solely relying on user count, the system now evaluates multiple parameters including node performance characteristics, ensuring that requests are routed to nodes that can handle them effectively while maintaining consistent user experience across the distributed system
Solution Approach 2:
The system performs preliminary evaluation of node capacity and performance characteristics before assigning user requests. By pre-assessing node readiness and matching request requirements with appropriate node capabilities, the system ensures consistent service delivery while maintaining simplified operational procedures through automated decision-making
Data Source
AI summary
Embodiments provide systems and methods to load balance using tenant affinities. A method includes configuring a load balancer with a set of tenant affinities, the set of tenant affinities including a tenant affinity for each of a plurality of tenants with respect to each of a plurality of computing nodes that provide services to users in a multitenant environment. Load balancing may include receiving a user request, associated with a tenant, for a service provided by the computing nodes, determining a highest tenant affinity computing node for the tenant using the set of tenant affinities and directing the user request to the highest tenant affinity computing node for the tenant.