Accelerator Rate Limiting for Per-Flow SLA Resource Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing accelerator resource allocation methods, such as per-VF/VSI rate limiting, lead to inefficient and disproportionate resource allocation among data flows with varying service level agreements (SLAs) and service level objectives (SLOs), resulting in suboptimal performance and resource wastage.
Innovation Solution
Implementing per-flow rate limiting based on priority and characteristics of data flows, using a data structure to set and enforce processing rate limits for each data flow, allowing dynamic resource allocation and adjustment based on flow state and priority.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If per-VF/VSI rate limiting is implemented to share accelerator resources, then resource sharing is achieved, but resource allocation becomes inefficient and disproportionate among data flows with different SLAs/SLOs
Solution Approach 1:
The patent segments the rate limiting granularity from the traditional per-VF/VSI level to a per-data-flow level. Each data flow is independently tracked and rate limited, allowing fine-grained resource allocation that respects individual SLA/SLO requirements rather than treating all flows within a VF/VSI uniformly.
Solution Approach 2:
The patent applies different rate limiting characteristics to different data flows based on their individual SLA/SLO requirements. High-priority flows with stricter SLOs receive preferential treatment in resource allocation, while lower-priority flows receive proportionally less resources, creating locally optimized quality of service for each flow.
2Device complexity
If aggregated rate limiting is applied per VF/VSI, then implementation simplicity is maintained, but resource competition among data flows leads to wasteful and disproportional allocation
Solution Approach 1:
The patent introduces dynamic rate limiting where the accelerator continuously monitors and adjusts the processing rate of each data flow based on real-time conditions and SLA/SLO requirements. This dynamic adaptation allows the system to allocate resources efficiently without waste, unlike static aggregated limiting.
Solution Approach 2:
The patent implements feedback mechanisms where the accelerator monitors the actual resource consumption and processing status of each data flow, and uses this information to adjust rate limiting decisions. This feedback loop enables the system to prevent resource wastage by identifying and throttling flows that are consuming resources inefficiently or exceeding their SLA guarantees.
3Productivity
If per-flow rate limiting is implemented to optimize resource allocation, then resource utilization efficiency improves, but system complexity increases
Solution Approach 1:
The patent designs a universal rate limiting framework that can handle multiple data flows with different SLA/SLO requirements using a unified approach. The same per-flow limiting mechanism applies to all flows, simplifying the overall system architecture despite the increased granularity of control.
Data Source
AI summary
Methods, apparatus, and computer programs are disclosed for accelerator rate limiting. In one embodiment, a method is disclosed to comprise setting one or more processing rate limits at an accelerator of the computing system for a respective one of a set of data flows based on a priority within a plurality of priorities, the respective one of the set of data flows to be processed by the accelerator and a processor of the computing system. The method further comprises upon receiving data of a data flow, determining whether to process the data at the accelerator based on the one or more processing rate limits for the data flow and a processing rate of the data flow in the accelerator; and responsive to a determination to process the data at the accelerator, causing a processing rate update of the data flow based on resources consumed in the accelerator.