Distributed Workload Parser Circuitry for GPU Compute Kernels

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Distributed architectures for GPUs face challenges in efficiently distributing compute workloads to processing elements, leading to suboptimal performance and power consumption due to reduced information availability and potential overburdening of communication fabrics.

Innovation Solution

A distributed workload parser architecture that includes a global workload parser and multiple distributed workload parsers, using a communications fabric to manage kernel state and batch iteration, reducing data transmission and enabling efficient load balancing by adjusting batch sizes and tracking shader resources.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of information

If a centralized workload parser is used to distribute compute workloads, then information availability is complete, but device complexity and communication fabric load increase significantly

Engineering Contradiction:

Improveinformation availabilityVSAvoidparser architecture complexity

Core Design Contradiction:

Loss of informationVSDevice complexity

Solution Approach 1:

The centralized workload parser is segmented into multiple distributed workload parsers, each responsible for a subset of processing elements. This segmentation distributes the information management burden across multiple nodes, reducing the complexity and communication load on any single parser while maintaining complete information availability through localized knowledge at each parser node.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system transitions from a single-dimension centralized parser model to a multi-dimensional distributed parser architecture. Each distributed parser operates in its own dimensional space, managing workloads for specific processing element groups independently, which reduces the overall system complexity while preserving information availability through hierarchical coordination.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Ease of operation

If a centralized workload parser is used to distribute compute workloads, then workload distribution control is centralized, but communication fabric becomes overburdened

Engineering Contradiction:

Improveworkload distribution controlVSAvoidcommunication fabric load

Core Design Contradiction:

Ease of operationVSLoss of energy

Solution Approach 1:

The communication fabric load is segmented by distributing workload management functions across multiple parsers. Each distributed parser handles communication with its assigned processing elements locally, reducing the aggregate communication traffic on the fabric while maintaining centralized coordination capabilities through hierarchical control mechanisms.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Distributed workload parsers act as intermediaries between the centralized control logic and processing elements. These intermediary parsers localize communication transactions, reducing direct fabric traffic between the central controller and processing elements, thereby reducing overall communication fabric load and energy consumption.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Productivity

If compute workloads are distributed to processing elements, then parallel processing capability is utilized, but power consumption increases

Engineering Contradiction:

Improveparallel processing capabilityVSAvoidpower consumption

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The workload distribution system dynamically adjusts the allocation of compute tasks to processing elements based on real-time resource availability and workload characteristics. This dynamic adaptation enables efficient parallel processing while optimizing power consumption by activating only the necessary processing elements and adjusting their operational states according to current demands.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system changes operational parameters such as batch sizes, workload granularity, and processing element activation states to optimize the balance between parallel processing throughput and power consumption. By dynamically adjusting these parameters based on workload characteristics and system state, the system maximizes productivity while minimizing energy usage.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS20200098160A1Distributed Compute Work Parser Circuitry using Communications Fabric

Publication Date: 2020.03.26 APPLE INC

AI summary

Techniques are disclosed relating to distributing work from compute kernels using a distributed hierarchical parser architecture. In some embodiments, an apparatus includes a plurality of shader units configured to perform operations for compute workgroups included in compute kernels processed by the apparatus, a plurality of distributed workload parser circuits, and a communications fabric connected to the plurality of distributed workload parser circuits and a master workload parser circuit. In some embodiments, the master workload parser circuit is configured to iteratively determine a next position in multiple dimensions for a next batch of workgroups from the kernel and send batch information to the distributed workload parser circuits via the communications fabric to assign the batch of workgroups. In some embodiments, the distributed parsers maintain coordinate information for the kernel and update the coordinate information in response to the batch information, even when the distributed parsers are not assigned to execute the batch.