Graphics Processor Latency Concealment via Interleaved Pixel Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The gap between memory access performance and processing performance in rendering engines for three-dimensional computer graphics leads to significant latency issues, reducing efficiency and making it difficult to implement complex shading algorithms and advanced arithmetic processing due to idle time and data consistency problems.
Innovation Solution
A graphics processor with a selector and arithmetic unit that alternately processes and schedules data units of rendering, shifting input timing based on latency to conceal processing latency and improve efficiency, allowing for flexible and complex graphic functions by executing instructions in parallel across multiple pipelines.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If the speed of CPUs is increased, then processing performance of arithmetic unit is improved, but the gap between memory access performance and processing performance grows, resulting in reduced rendering process performance
Solution Approach 1:
The patent applies preliminary action by pre-fetching pixel data from the frame buffer into the arithmetic unit before it is actually needed for processing. The arithmetic unit holds multiple pixels in its internal memory, so when a pixel is needed for rendering, it is already available, eliminating the latency penalty. This is achieved by the arithmetic unit proactively reading and storing pixel data in advance of the rendering pipeline's requirements.
2Reliability
If arithmetic units are suspended to wait for data dependency, then data consistency is guaranteed, but idle time (bubbles) is produced in the pipeline, lowering efficiency
Solution Approach 1:
The patent applies preliminary action by pre-computing arithmetic operations on multiple pixels before they are needed by the rendering pipeline. The arithmetic unit performs calculations in advance and stores results in its internal memory, so when the rendering pipeline needs the data, it is already computed and ready. This eliminates the need to suspend the arithmetic unit and eliminates bubbles in the pipeline while maintaining data consistency.
Solution Approach 2:
The patent applies dimensionality change by transitioning from a sequential single-pixel processing model to a parallel multi-pixel processing model. Instead of processing one pixel at a time through the entire pipeline, the arithmetic unit processes multiple pixels simultaneously in parallel, then feeds them to the rendering pipeline at the appropriate times. This dimensional shift from sequential to parallel processing eliminates pipeline bubbles while maintaining data consistency.
3Adaptability or versatility
If the shader is isolated from the RMW unit, then flexible execution of program is enabled, but memory latency is quite long, prohibiting the graphic process including the RMW function from having higher functionality
Solution Approach 1:
The patent applies merging by combining the shader and RMW unit into a single integrated architecture. The arithmetic unit (shader) and the read-modify-write unit are merged so that the shader can directly access the frame buffer without isolation. This integration allows the shader to execute complex shading algorithms with full RMW functionality while maintaining program execution flexibility, eliminating the need to choose between flexibility and functionality.
Solution Approach 2:
The patent applies preliminary action by having the arithmetic unit pre-fetch and cache pixel data from the frame buffer into its internal memory before RMW operations are needed. This preliminary caching action reduces the memory latency for subsequent RMW operations, allowing the integrated shader-RMW architecture to execute complex algorithms efficiently without the performance penalty of frequent frame buffer accesses.
Data Source
AI summary
Registers32a-32d hold data for pixels interleaved. An operator 34 reads the pixel data from the registers and processes the pixel data in accordance with a program code. The operator 34 writes the result of the process back to the registers via a cache 38 or writes it in a memory. Program counters PC0-PC3 provided in association with the number of pixels interleaved store the addresses of instructions in a program for the respective pixels. An instruction loader 76 alternately reads from the program counters. An incrementer 74 increments the count of the program counters. The instructions in the program for the pixels are alternately loaded and interleaved on a pixel by pixel basis, before being supplied to the operator 34 and the like.