CPU QoS Throttling for Accelerator Service Requests
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern system on chips (SoCs) face significant performance and energy efficiency degradation due to the interference caused by accelerator system service requests (SSRs) that need to be serviced by the central processing unit (CPU), leading to destructive CPU application interference.
Innovation Solution
Implementing a mechanism where the CPU monitors and adjusts the processing of SSRs by adding an adjustable delay based on CPU overhead, using a kernel worker thread to enforce quality of service (QoS) guarantees, thereby throttling SSRs when CPU performance is impacted.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If accelerator SSRs are serviced immediately by the CPU, then accelerator functionality is maintained, but CPU performance and energy efficiency degrade significantly
Solution Approach 1:
The patent implements periodic sampling of CPU performance metrics (cycles spent servicing SSRs, application performance) at intervals defined by a sliding window parameter. Based on this periodic measurement, the system dynamically adjusts the SSR service rate using exponential backoff, creating a rhythmic control cycle that balances accelerator functionality with CPU performance protection
Solution Approach 2:
The system dynamically adjusts the SSR service delay using exponential backoff based on measured CPU overhead. The service rate is not fixed but adapts in real-time according to CPU performance conditions, transitioning between aggressive SSR servicing (when CPU is underutilized) and throttling (when CPU performance degrades)
2Productivity
If accelerator SSRs are serviced frequently, then accelerator operations are supported, but CPU energy consumption increases
Solution Approach 1:
The system continuously monitors CPU energy consumption indicators (cycles spent in kernel mode servicing SSRs versus user mode application execution) and uses this feedback to adjust SSR service frequency. When feedback indicates excessive CPU energy consumption, the system increases service delay exponentially, creating a closed-loop control that optimizes energy efficiency
3Productivity
If CPU time spent servicing SSRs is increased, then accelerator functionality is improved, but CPU application performance deteriorates
Solution Approach 1:
The system performs preliminary measurements of CPU performance characteristics (baseline performance, sliding window averages) before making SSR service rate adjustments. This preliminary characterization allows the system to predict the impact of SSR servicing on application performance and adjust service timing to minimize disruption to CPU applications
Data Source
AI summary
Systems, apparatuses, and methods for enforcing processor quality of service guarantees when servicing system service requests (SSRs) are disclosed. A system includes a first processor executing an operating system and a second processor executing an application which generates SSRs for the first processor to service. The first processor monitors the number of cycles spent servicing SSRs over a previous time interval, and if this number of cycles is above a threshold, the first processor starts delaying the servicing of subsequent SSRs. In one implementation, if the previous delay was non-zero, the first processor increases the delay used in the servicing of subsequent SSRs. If the number of cycles is less than or equal to the threshold, then the first processor services SSRs without delay. As the delay is increased, the second processor begins to stall and its SSR generation rate falls, reducing the load on the first processor.