Operand Cache Flush and Eviction Using Dirty Status Hints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Graphics Processing Units (GPUs) face challenges in managing operand caches due to features like parallel execution, out-of-order execution, and multi-threading, leading to high power consumption, particularly in mobile applications where battery power is used.
Innovation Solution
Implementing techniques to flush operands from the operand cache, evict entries, and perform cache clean operations in a controlled manner, including pre-emptive write backs for low-priority entries and maintaining higher-priority entries until necessary, while selecting entries for eviction based on priority and using compiler hints to optimize cache usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If operands are frequently accessed from the register file to support parallel and out-of-order execution, then execution flexibility and correctness are maintained, but power consumption increases significantly
Solution Approach 1:
The patent segments the operand access path into two parts: a small operand cache for frequently accessed operands and the full register file. This segmentation allows the system to serve common operands from the low-power cache while maintaining full register file access capability when needed, thus reducing overall power consumption without compromising execution correctness.
Solution Approach 2:
The patent implements preliminary action by pre-loading operands into the operand cache before they are actually needed by the execution units. The cache is populated in advance based on prediction or scheduling, so that when operands are needed for parallel or out-of-order execution, they are already available in the cache, avoiding high-power register file accesses.
2Use of energy by moving object
If the operand cache is made larger to extend operand availability, then power consumption is reduced, but device complexity and control difficulty increase
Solution Approach 1:
The patent applies local quality by implementing a small, specialized operand cache with specific properties optimized for its function, rather than a large general-purpose cache. The cache is sized and configured locally to match the specific access patterns of operands in parallel execution, providing sufficient capacity to reduce power consumption while keeping control logic simple and manageable.
3Device complexity
If dirty operand cache entries are pre-emptively written back to reduce cache complexity, then control is simplified, but execution time increases due to lost caching opportunities
Solution Approach 1:
The patent implements dynamic control of dirty cache entry write-back based on the current execution state. The system monitors which execution units are active and what operands they need, dynamically deciding when to write back dirty entries. This dynamic approach allows the cache to retain dirty entries as long as they might be needed, minimizing execution time penalties while keeping control logic manageable through event-driven write-back triggers.
4Reliability
If the operand cache is cleaned frequently to maintain data freshness, then data consistency is improved, but power consumption increases due to additional write-back operations
Solution Approach 1:
The patent implements periodic or event-driven cache clean operations rather than continuous cleaning. The operand cache is cleaned at specific intervals or triggered by specific events (such as thread switches or instruction boundaries), maintaining data consistency when necessary while avoiding unnecessary write-back operations that would increase power consumption during normal execution.
Data Source
AI summary
An apparatus includes an operand cache for storing operands from a register file for use by execution circuitry. In some embodiments, eviction priority for the operand cache is based on the status of entries (e.g., whether dirty or clean) and the retention priority of entries. In some embodiments, flushes are handled differently based on their retention priority (e.g., low-priority entries may be pre-emptively flushed). In some embodiments, timing for cache clean operations is specified on a per-instruction basis. Disclosed techniques may spread out write backs in time, facilitate cache clean operations, facilitate thread switching, extend the time operands are available in an operand cache, and/or improve the use of compiler hints, in some embodiments.