SIMD Wavefront Repacking for Thread Divergence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing systems face inefficiencies when using GPUs for general computation due to challenges such as separate memory systems, inefficient scheduling, programming models, and power consumption, particularly when conditional code causes thread divergence in wavefronts, leading to significant performance loss.
Innovation Solution
A method is introduced to allocate memory for local register states and repack wavefronts based on conditional statements, creating separate wavefronts for threads that pass or fail the condition, allowing for efficient execution and maximizing hardware utilization by re-arranging work items on SIMD hardware.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If threads in a wavefront execute conditional code that causes divergence, then each thread can follow its own execution path, but the processor operates at only 1/64 of peak performance when executing one thread at a time
Solution Approach 1:
The patent segments the wavefront into multiple sub-wavefronts based on conditional execution paths. When thread divergence is detected, the original wavefront is divided into separate groups (e.g., wavefront 0 for threads that pass the condition, wavefront 1 for threads that fail), allowing independent execution of each segment while maintaining overall adaptability to different execution paths.
Solution Approach 2:
The patent dynamically adjusts wavefront composition during execution by repacking threads based on conditional statement outcomes. The system transitions from a static wavefront structure to a dynamic one where threads are reorganized in real-time to maximize SIMD efficiency, switching between executing all threads together or splitting them based on divergence detection.
2Productivity
If wavefronts are repacked to separate threads that pass or fail conditional statements, then more threads can execute the same instruction concurrently, but additional memory allocation and repacking operations are required
Solution Approach 1:
The patent performs preliminary actions by pre-allocating memory space for register states and preparing repacking structures before execution. The system reserves memory regions and sets up data structures that will be used during wavefront repacking, reducing the complexity of on-the-fly memory management during actual execution.
Solution Approach 2:
The patent introduces intermediary data structures (such as repacking buffers and state tracking mechanisms) that mediate between the original wavefront and the repacked execution units. These intermediaries simplify the complex memory management by providing a structured layer that handles the transformation and organization of thread states during repacking.
3Power
If multiple processing devices with separate memory systems are used, then computational power is increased, but memory access efficiency and power consumption are degraded
Solution Approach 1:
The patent merges memory access patterns by consolidating threads with similar memory access requirements into the same repacked wavefront. This combining of threads that access the same memory regions reduces redundant memory transactions across multiple processing devices, thereby decreasing overall memory access energy consumption while maintaining high computational power.
Data Source
AI summary
Methods, media, and computing systems are provided. The method includes, the media are configured for, and the computing system includes a processor with control logic for allocating memory for storing a plurality of local register states for work items to be executed in single instruction multiple data hardware and for repacking wavefronts that include work items associated with a program instruction responsive to a conditional statement. The repacking is configured to create repacked wavefronts that include at least one of a wavefront containing work items that all pass the conditional statement and a wavefront containing work items that all fail the conditional statement.