Graphics Command Queue Utilization Normalization

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing graphics processing systems face sub-optimal hardware utilization due to inadequate granularity in resource allocation, leading to 'starvation' of lower priority processes and unfair treatment of command queues, resulting in inefficient use of graphics hardware resources.

Innovation Solution

A method that involves obtaining command queues for executing applications, normalizing Quality of Service (QoS) settings to ensure equitable resource allocation, constructing a utilization table to track current and target resource utilization, and dynamically adjusting the status of command queues to prevent overutilization, allowing for fair and efficient distribution of graphics hardware resources across applications.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If multiple processes are simultaneously executed in parallel to increase hardware utilization, then graphics hardware utilization is improved, but resource allocation fairness deteriorates causing starvation of lower priority processes

Engineering Contradiction:

Improvegraphics hardware utilizationVSAvoidresource allocation fairness

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system dynamically adjusts the priority and resource allocation to command queues based on current hardware utilization state. The fairness controller continuously monitors utilization metrics and modifies scheduling decisions in real-time, transforming the static resource allocation into a dynamic system that adapts to changing conditions while maintaining fairness guarantees

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The fairness controller implements a feedback mechanism that monitors command queue utilization metrics and uses this information to adjust resource allocation decisions. The system measures actual hardware utilization and feeds this information back to the scheduler, which then modifies its behavior to prevent starvation and maintain fairness across multiple parallel processes

Inventive Principle:

Principle #23Feedback

2Measurement precision

If fine-grained resource allocation is implemented to match requested division, then resource allocation precision is improved, but system complexity increases

Engineering Contradiction:

Improveresource allocation granularityVSAvoidallocation system complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The fairness controller acts as an intermediary layer between the high-level process scheduling and the low-level hardware resource allocation. This intermediary component translates coarse-grained process requests into fine-grained command queue allocations, achieving precise resource distribution without requiring complex modifications to either the process level or hardware level

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system segments resource allocation into hierarchical levels: processes are divided into command queues, which are further divided into individual commands. This segmentation allows the fairness controller to operate at the command queue level, providing fine-grained control over resource allocation while keeping the overall system architecture manageable and avoid overwhelming complexity

Inventive Principle:

Principle #1Segmentation

3Reliability

If equitable treatment is provided to all command queues, then fairness is improved, but hardware utilization efficiency deteriorates due to underutilization of available resources

Engineering Contradiction:

Improvecommand queue fairnessVSAvoidhardware utilization efficiency

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The system implements dynamic fairness adjustment where the degree of equitable treatment applied to command queues varies based on current hardware utilization conditions. When resources are abundant, more equitable distribution is applied; when resources are constrained, the system dynamically adjusts allocation to maximize utilization while maintaining fairness guarantees, preventing the static equity approach from causing underutilization

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS11593175B2Normalizing target utilization rates of a cross-application table of concurrently executing applications to schedule work on a command queue of a graphics processors

Publication Date: 2023.02.28 APPLE INC

AI summary

In general, embodiments are disclosed herein for tracking and allocating graphics hardware resources. In one embodiment, a software and/or firmware process constructs a cross-application command queue utilization table based on one or more specified command queue quality of service (QoS) settings, in order to track the target and current utilization rates of each command queue on the graphics hardware over a given frame and to load work onto the graphics hardware in accordance with the utilization table. Based on the constructed utilization table for a given frame, any command queues that have exceed their respective target utilization value may be moved to an “inactive” status for the duration of the current frame. For any command queues that remain in an “active” status for the current frame, work from those command queues may be loaded on to slots of the appropriate data masters of the graphics hardware in any desired order.