Address Processing Unit for Graphics Memory Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Memory lookups in processing systems, such as GPUs, are costly in terms of processing time, limiting performance due to high memory latency, despite efforts to reduce this latency through caching and algorithms.
Innovation Solution
The introduction of an address processing unit (APU) that generates secondary memory lookup requests to optimize memory access, reducing latency by intercepting and processing returns from L2 cache, and storing necessary information in L1 cache for faster access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional memory lookup methods are used, then system simplicity is maintained, but memory latency is high and processing performance is limited
Solution Approach 1:
An address processing unit (APU) is introduced as an intermediary component between the GPU and memory system. The APU intercepts memory lookup requests, generates secondary lookup requests, and processes returns from L2 cache before forwarding to L1 cache. This intermediary structure reduces memory latency by minimizing the number of required memory lookups while maintaining a manageable system architecture through dedicated address processing functionality.
2Loss of time
If multiple secondary memory lookup requests are generated, then the desired address is accessed more efficiently, but the number of memory operations increases
Solution Approach 1:
The address processing unit performs preliminary actions by generating secondary memory lookup requests before the primary lookup is completed. By anticipating needed addresses and preparing secondary requests in advance, the system reduces overall memory lookup time. The APU executes instructions to generate secondary requests that will be needed, allowing overlapping of request generation and execution, thus reducing the critical path latency without significantly increasing total memory operations.
Data Source
AI summary
Methods and systems for processing memory lookup requests are provided. In an embodiment, an address processing unit includes an instructions module configured to store instructions to be executed to complete a primary memory lookup request and a logic unit coupled to the instructions module. The primary memory lookup request is associated with a desired address. Based on an instruction stored in the instructions module, the logic unit is configured to generate a secondary memory lookup request that requests the desired address.In another embodiment, a method of processing memory lookups requests includes receiving a primary memory lookup request that corresponds to a desired memory address and generating a plurality of secondary memory lookup requests.