GPU Burst Processing with Flip-Flop Registers for Lower SRAM Power
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
GPU register files consume significant power and area due to the large capacity required for parallel threads, with SRAMs not scaling efficiently, leading to increased power usage and bank clashes.
Innovation Solution
Implement a separate sub-processor within the GPU to execute atomically executable bursts of instructions, using flip-flop based registers and a dedicated instruction cache to reduce SRAM access, and incorporate forwarding paths to minimize register reads and writes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If SRAM-based register files are used to support large numbers of parallel threads, then thread parallelism is improved, but power consumption and area increase significantly
Solution Approach 1:
The processor is divided into multiple independent processing lanes that can execute instructions in parallel. Each lane has its own instruction pointer and can operate independently, allowing the system to support many parallel threads without requiring a single large register file. This segmentation enables scalable parallelism while controlling power and area usage.
Solution Approach 2:
The patent introduces a time dimension to instruction execution through speculative pre-fetching and out-of-order execution. Instructions are fetched and prepared in advance speculatively, then executed when ready rather than strictly in program order. This temporal dimension allows efficient utilization of resources without increasing register file size.
2Productivity
If SRAM-based register files are used to support large numbers of parallel threads, then thread parallelism is improved, but register file area increases
Solution Approach 1:
Each processing lane is an independent entity with its own instruction stream and execution units. This segmentation allows the system to support many threads through spatial parallelism at the lane level rather than requiring temporal parallelism managed by a large centralized register file, thereby reducing the area required for register storage.
Solution Approach 2:
The system uses speculative execution where instructions are fetched and prepared in advance. If speculation is correct, these pre-fetched instructions are executed immediately; if incorrect, the speculative work is discarded and corrected instructions are fetched. This approach reduces the need for large buffer and register files by allowing selective recovery of valid instruction states.
3Productivity
If threads are switched when unable to progress, then thread utilization is improved, but SRAM access frequency increases
Solution Approach 1:
The system performs preliminary speculative pre-fetching of instructions before they are actually needed for execution. Instructions are fetched in advance and staged in instruction buffers, so when execution is required, the instructions are already available locally. This preliminary action reduces the frequency of high-power SRAM accesses while maintaining high thread utilization.
4Productivity
If atomic bursts of instructions are extracted and executed separately, then execution efficiency is improved, but program complexity increases
Solution Approach 1:
The instruction stream is automatically segmented into atomic bursts that can be executed independently and efficiently. The processor identifies instruction boundaries and groups related instructions into bursts that maintain atomicity guarantees. This automatic segmentation improves execution efficiency by allowing parallel burst execution while the compiler or hardware handles the complexity of program structure transformation.
Solution Approach 2:
The system introduces an intermediary layer (such as a compiler pass or hardware instruction reordering unit) that transforms the original program into a form with extracted atomic bursts. This intermediary handles the complexity of analyzing program semantics, identifying atomic regions, and restructure instructions, while the runtime execution benefits from the improved burst-based parallel execution model.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A GPU comprising a shader core is described herein. The shader core comprises: a main processing portion and a sub-processor. The main processing portion comprises a scheduler, an instruction cache, a plurality of registers and a plurality of ALUs. The sub-processor operates independently of the main processing portion and comprises a burst scheduler, a plurality of registers and a plurality of ALUs. The sub-processor is arranged to execute bursts, wherein a burst comprises at least one group of instructions that can be executed atomically and which are extracted from a program. The main processing portion executes a modified version of the program, wherein the modified program is created from the program by replacing the instructions in a burst with an instruction that triggers the execution of the burst. The registers in the sub-processor are used to store one or more sources and/or results for bursts that are being executed by the sub-processor. To be accompanied, when published, by Figure 1 of the accompanying drawings.