Deferred Virtual Address Computation for Cache Tag Comparison
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems face performance issues in computing unique virtual addresses and cache tag comparisons for multiple threads, leading to increased dedicated circuitry requirements and reduced performance as the number of parallel threads increases.
Innovation Solution
The system defers complete virtual address computation until after a cache tag comparison, using a thread group address to determine cache hits/misses, and computes unique virtual addresses only when necessary, eliminating the need for dedicated circuitry for each thread and serializing cache accesses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If unique virtual addresses are computed in parallel for each thread, then address computation is completed, but dedicated circuitry increases and performance decreases
Solution Approach 1:
The patent merges the address computation function into a shared resource that serves multiple threads, rather than providing dedicated circuitry for each thread. The address computation unit is shared across thread groups, reducing overall hardware complexity while maintaining computational capability through time-multiplexed access.
Solution Approach 2:
The patent performs preliminary actions by computing addresses for threads that experience cache misses, while threads with cache hits can proceed without waiting for complete address computation. This staged approach allows productive threads to advance while address computation for non-productive threads is deferred.
2Reliability
If dedicated circuitry is provided for each thread, then unique virtual address computation is enabled, but the amount of dedicated circuitry increases
Solution Approach 1:
The address computation unit is designed as a universal resource that can serve multiple threads sequentially, rather than having dedicated units for each thread. The same computation logic is reused across different thread groups, achieving multi-functionality that reduces hardware complexity while maintaining accurate address computation for each thread.
Solution Approach 2:
Threads that experience cache misses self-service by waiting for address computation to complete, while threads with cache hits continue execution without being blocked. This self-service mechanism allows the system to maintain reliability for address computation while avoiding the need for complex coordination circuitry.
3Speed
If cache tag comparison is performed in parallel for each thread, then cache hits/misses are determined, but dedicated circuitry for comparison increases
Solution Approach 1:
The patent performs partial cache tag comparison using only the thread group address portion that is common to all threads in a group. This partial comparison is sufficient to determine whether any thread in the group might have a cache hit, avoiding the need for full parallel comparison of complete virtual addresses for each thread.
Solution Approach 2:
The virtual address is segmented into thread group address portions and thread-specific portions. The cache tag comparison uses only the thread group address segment, which is shared among threads in the group. This segmentation allows simplified comparison circuitry while maintaining cache lookup functionality.
4Productivity
If the number of parallel threads increases, then processing capacity increases, but the amount of dedicated circuitry increases and performance reduces
Solution Approach 1:
The system dynamically adjusts the number of threads that can be serviced by the address computation unit based on cache hit/miss patterns. When many threads experience cache hits, fewer address computation resources are needed. When cache misses occur, the system dynamically allocates computation cycles to those threads, allowing the system to handle variable thread counts efficiently without proportional increases in dedicated circuitry.
Data Source
AI summary
One embodiment of the present invention sets forth a technique for computing virtual addresses for accessing thread data. Components of the complete virtual address for a thread group are used to determine whether or not a cache line corresponding to the complete virtual address is not allocated in the cache. Actual computation of the complete virtual address is deferred until after determining that a cache line corresponding to the complete virtual address is not allocated in the cache.