Priority-Based Load Shedding for Overloaded Request Queues
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Servers face performance degradation and user experience issues due to sudden load increases, with existing load shedding methods affecting many users and causing tasks to fail.
Innovation Solution
A computing system prioritizes load shedding by associating requests with tiers and cohorts based on type and user, rejecting requests below a determined threshold tier and cohort to manage overloads efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the server increases computing and network resources to handle sudden load increases, then the system capacity is improved, but the response time to handle load changes becomes slower
Solution Approach 1:
The system performs preliminary classification of requests into tiers and cohorts before load shedding occurs. This pre-organization allows the system to quickly identify and reject low-priority requests during overload conditions without needing to increase resources or spend time analyzing request priorities during the crisis moment.
2Productivity
If the system rejects certain requests to improve system performance during overload, then system performance is improved, but user experience deteriorates
Solution Approach 1:
The system applies different quality levels of service to different requests based on their tier and cohort classification. High-priority requests receive full processing resources while low-priority requests are rejected, allowing the system to maintain good user experience for critical users while preserving overall system performance during overload conditions.
Solution Approach 2:
The request population is segmented into multiple tiers and cohorts based on priority and user characteristics. This segmentation allows the system to selectively reject only low-priority requests while preserving high-priority user experience, rather than uniformly rejecting all requests during overload.
3Reliability
If the system rejects requests during overload, then system recovery is accelerated, but task completion rate deteriorates
Solution Approach 1:
The system dynamically changes the threshold parameters for request rejection based on current load conditions. By adjusting which tiers and cohorts are rejected based on real-time system state, the system can accelerate recovery when necessary while minimizing impact on task completion rates when the system is operating normally.
Data Source
AI summary
A system and method for dynamic load shedding in computing environments experiencing high request volumes. The system monitors a request queue storing unprocessed requests from client devices and determines both a current queue size and an aggregate historical queue size. During a defined time interval, the system identifies the number of requests enqueued and dequeued. Based on these metrics, the system determines a percentage or number of requests to be rejected to mitigate overload. Each request is ranked according to its type and the time it was received. A subset of unprocessed requests is then selected for rejection based on their rankings and the determined rejection threshold. This targeted rejection strategy enables prioritized load shedding that maintains system responsiveness while minimizing user impact.