GPU Local Memory Architecture for Shader Efficiency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing graphics processing units (GPUs) face inefficiencies in executing shader programs due to repeated calculations of common expressions across multiple threads, leading to increased bandwidth and power consumption.

Innovation Solution

Incorporating a programmable execution unit with a local register memory and a further local memory that stores data accessible to plural threads, allowing output data to be stored in the further local memory instead of external memory, reducing the need for repeated calculations and improving efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If common expressions are recalculated for each thread independently, then calculation accuracy is maintained, but processing time and power consumption increase

Engineering Contradiction:

Improveshader program execution efficiencyVSAvoidpower consumption

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The patent applies preliminary action by executing common expressions once before thread execution and storing results in a further local memory. The compiler identifies common expressions and generates pilot threads that execute these expressions beforehand, so subsequent worker threads can directly access pre-computed results instead of recalculating, thereby reducing power consumption while maintaining accuracy.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent uses copying by creating pilot threads that execute common expressions and store results in a shared further local memory. Worker threads then copy these pre-computed results from the further local memory rather than performing independent calculations, reducing redundant computational operations and power usage while preserving calculation accuracy.

Inventive Principle:

Principle #26Copying

2Quantity of substance

If output data is stored in external memory, then memory capacity is sufficient, but bandwidth consumption and access time increase

Engineering Contradiction:

Improvememory capacityVSAvoidmemory access time

Core Design Contradiction:

Quantity of substanceVSLoss of time

Solution Approach 1:

The patent segments memory into three distinct levels: register memory for individual thread data, further local memory for common expression results accessible by multiple threads, and external memory for bulk data storage. This segmentation allows frequently accessed common data to be stored closer to the execution unit, reducing access time for critical operations while external memory retains sufficient total capacity.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The further local memory acts as an intermediary between external memory and the execution unit. It stores pre-computed common expression results that are frequently accessed by multiple worker threads, serving as a buffer that reduces the need for repeated external memory accesses and thereby decreases bandwidth consumption and access time.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Productivity

If a further local memory is added to store common data, then processing efficiency improves, but device complexity increases

Engineering Contradiction:

Improvethread execution efficiencyVSAvoidmemory structure complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The further local memory is designed with multi-functionality to justify its addition. It serves multiple purposes: storing common expression results for worker threads, caching frequently accessed data, and reducing external memory bandwidth requirements. This universal utility across multiple functions helps offset the increased device complexity by providing substantial performance benefits in various operational scenarios.

Inventive Principle:

Principle #6Universality (Multi-functionality)

4Ease of operation

If common expressions are executed by each thread independently, then thread autonomy is maintained, but redundant calculations increase power consumption

Engineering Contradiction:

Improvethread independenceVSAvoidenergy waste from redundant calculations

Core Design Contradiction:

Ease of operationVSLoss of energy

Solution Approach 1:

The system applies self-service by having worker threads access pre-computed results from the further local memory rather than independently recalculating common expressions. Each thread serves itself by retrieving already-validated results, maintaining operational independence while eliminating redundant computational energy waste. The pilot threads also serve themselves by executing common expressions once and making results available to all.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS10115222B2Data processing systems

Publication Date: 2018.10.30 ARM LTD

AI summary

A graphics processing unit comprises a programmable execution unit executing graphics processing programs for execution threads to perform graphics processing operations, a local register memory comprising one or more registers, where registers of the register memory are assignable to store data associated with an individual execution thread that is being executed by the execution unit, and where the register(s) assigned to an individual execution thread are accessible only to that associated individual execution thread, and a further local memory that is operable to store data for use in common by plural execution threads, where the data stored in the further local memory is accessible to plural execution threads as they execute. The programmable execution unit is operable to selectively store output data for an execution thread in a register(s) of the local register memory assigned to the execution thread, and the further local memory.