GPU Scalar Execution via Operand Vector Compression

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

GPU architectures face power and thermal constraints due to high consumption by execution units and register files, particularly in SIMT execution where operand vectors often have similarities that are not leveraged for energy savings.

Innovation Solution

Implement a GPU architecture that monitors and exploits similarities in operand vectors across execution units, allowing for scalar execution on a single unit and reducing register file memory activation, using existing crossbar switches for routing and compression to achieve power savings.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If multiple execution units are activated for SIMT execution, then computational throughput is improved, but power consumption increases

Engineering Contradiction:

Improvecomputational throughputVSAvoidpower consumption

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

Multiple execution units are merged into a single active execution unit when operand vectors are identical. The scalar execution unit combines the work of multiple threads by executing a single instruction on a single operand vector and broadcasting the result to all execution units, thereby reducing overall power consumption while maintaining computational throughput.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The system dynamically adjusts the number of active execution units based on operand vector similarity. When all operand vectors are identical, the system transitions to scalar execution mode with a single active execution unit. When operand vectors differ, the system activates multiple execution units for full SIMT execution, creating a dynamic adaptation between extreme cases.

Inventive Principle:

Principle #15Dynamics

2Reliability

If all register file memory circuits are activated to store operand vectors, then data availability is improved, but power consumption increases

Engineering Contradiction:

Improvedata availabilityVSAvoidpower consumption

Core Design Contradiction:

ReliabilityVSUse of energy by stationary object

Solution Approach 1:

The system extracts and stores only a single representative operand vector in the register file when all threads have identical operands. This single stored vector is then broadcast to all execution units, eliminating the need to activate and store multiple duplicate vectors in the register file, thereby reducing power consumption while maintaining data availability.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The single stored operand vector in the register file serves multiple execution units simultaneously. By storing only one universal vector that can be broadcast to all execution units, the system achieves multi-functionality, eliminating redundant storage and reducing the number of memory circuits that need to be activated.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Duration of action of stationary object

If operand vectors are stored in register file memory circuits, then data retention is improved, but power consumption increases

Engineering Contradiction:

Improvedata retentionVSAvoidpower consumption

Core Design Contradiction:

Duration of action of stationary objectVSUse of energy by stationary object

Solution Approach 1:

The system extracts only the essential operand vector data and stores it in a single location in the register file. By removing redundant duplicate storage of identical vectors, the system reduces the total amount of data that needs to be retained in memory circuits, thereby reducing the number of circuits that must remain active and lowering overall power consumption.

Inventive Principle:

Principle #2Taking out (Extraction)

4Use of energy by moving object

If scalar execution is implemented on a single execution unit, then power consumption is reduced, but computational throughput may decrease

Engineering Contradiction:

Improvepower consumptionVSAvoidcomputational throughput

Core Design Contradiction:

Use of energy by moving objectVSProductivity

Solution Approach 1:

The scalar execution unit merges the computational work of multiple threads into a single execution. By executing one instruction on one operand vector and then broadcasting the result to all execution units, the system combines multiple parallel operations into a single operation, reducing power consumption while maintaining the same computational output.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The single active execution unit serves multiple execution units by broadcasting its result to all threads. The execution unit performs its operation and then distributes the result to all other execution units, allowing the system to maintain full computational throughput while using only one execution unit actively.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS10592466B2Graphic processor unit providing reduced storage costs for similar operands

Publication Date: 2020.03.17 WISCONSIN ALUMNI RES FOUND

AI summary

A GPU architecture employs a crossbar switch to preferentially store operand vectors in a compressed form allowing reduction in the number of memory circuits that must be activated during an operand fetch and to allow existing execution units to be used for scalar execution. Scalar execution can be performed during branch divergence.