Dynamic Request Scheduling via External QoS Mediator
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data processing systems, particularly back-end servers, face challenges in providing optimal response times due to limited resources and peak loads, necessitating improved request prioritization and scheduling techniques to differentiate service levels effectively.
Innovation Solution
A dynamic request scheduling technique that assigns quality-of-service (QoS) classes to clients based on response targets and estimated server response times, using an external scheduler to manage request submission and throughput, ensuring that higher-priority requests meet their response targets while balancing server load and response times.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the system provisions resources for peak loads, then response times during peak loads are improved, but resource utilization during normal load conditions deteriorates (resources go under-utilized)
Solution Approach 1:
The patent implements dynamic resource provisioning through load-based scaling. The system automatically adjusts the number of server instances based on real-time load conditions - scaling up during peak loads to maintain response times, and scaling down during normal loads to optimize resource utilization. This dynamic adaptation resolves the contradiction by making the system flexible rather than static.
Solution Approach 2:
The system changes operational parameters (number of active server instances) based on load conditions. During peak loads, the system increases the parameter of active instances to ensure response time targets are met. During normal loads, it reduces this parameter to improve resource utilization efficiency, thus resolving the contradiction through parameter adjustment.
2Ease of operation
If request priority implementation is integrated into back-end server software, then scheduling control is improved, but system complexity deteriorates (requires changing complex server code)
Solution Approach 1:
The patent introduces an external scheduler as an intermediary component between clients and the back-end server. This mediator handles all request prioritization and scheduling logic externally, allowing the server code to remain unchanged. The intermediary captures and processes priority information from incoming requests, then queues them appropriately without requiring modifications to the complex server software.
Solution Approach 2:
The scheduling functionality is extracted from the back-end server software and placed in an external scheduler component. This extraction removes the complexity of integrating scheduling logic into the server code, while still providing the necessary scheduling control. The external scheduler operates independently, managing request queues based on priority without touching the server's internal code structure.
3Ease of operation
If all requests are serviced with equal priority, then fairness is improved, but response time for critical requests deteriorates
Solution Approach 1:
The patent applies local quality by differentiating service treatment based on request characteristics. Critical requests (e.g., from high-value customers or time-sensitive operations) are assigned to high-priority queues with faster processing, while non-critical requests go to lower-priority queues. This localized differentiation ensures that specific critical requests receive the fast service they need without completely compromising overall system fairness.
Solution Approach 2:
The system segments the request stream into multiple priority-based queues. Instead of treating all requests uniformly, it divides them into categories (high priority, normal priority, low priority) based on client value, request type, and other factors. This segmentation allows critical requests to be serviced faster while maintaining a structured approach to handling all requests, balancing fairness with responsiveness to critical needs.
Data Source
AI summary
Techniques are provided for dynamically scheduling requests in data processing systems in accordance with differentiated service levels. In a first aspect of the invention, a technique for processing a request to at least one server comprises the following steps. A request is received. Then, submission of the request to the at least one server is scheduled based on: (i) a quality-of-service (QoS) class assigned to a client from which the request originated; (ii) a response target associated with the QoS class; and (iii) an estimated response time associated with the at least one server. The technique may further comprise the step of withholding the request from submission to the at least one server when the request originated from a client assigned to a first QoS class to allow a request that originated from a client assigned to a second QoS class to meet a response target associated therewith.