Cache Sub-block Partitioning for Bandwidth Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing demand for memory in multi-threaded and multi-core processor systems due to chip multithreading (CMT) and chip multiprocessing (CMP) leads to higher costs and power consumption, with existing methods failing to effectively reduce bandwidth needs and cache misses.
Innovation Solution
A cache management method that predicts future accesses by partitioning cache blocks into sub-blocks, where only likely-to-be-accessed sub-blocks are stored and transferred, using masks to indicate access patterns and reduce bandwidth requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If the size of on-chip cache is increased to reduce bandwidth bottleneck, then bandwidth needs are reduced, but the area available for processing cores is reduced
Solution Approach 1:
The cache block is segmented into multiple sub-blocks, and only the sub-blocks that are likely to be accessed are loaded into the cache. This segmentation allows the cache to store fewer bytes per block (e.g., 32 bytes instead of 64 bytes), reducing the area required for on-chip cache while still meeting bandwidth requirements by selectively loading only necessary sub-blocks.
Solution Approach 2:
Instead of loading entire cache blocks, only the necessary portion (partial action) of the block that is predicted to be accessed is loaded into the cache. This partial loading reduces the cache area requirement while maintaining adequate bandwidth for actual access patterns.
2Reliability
If traditional cache loading methods are used where entire blocks are loaded, then cache hit rate is maintained, but bandwidth requirements increase
Solution Approach 1:
Access patterns are monitored and sub-block validity is predicted in advance before cache loading occurs. This preliminary analysis allows the system to pre-determine which sub-blocks will be needed, enabling selective loading that reduces bandwidth requirements while ensuring that necessary data is available when needed, maintaining cache hit rate.
Solution Approach 2:
The system monitors access patterns to cache blocks and uses this feedback information to determine which sub-blocks should be loaded. This feedback mechanism ensures that cache loading decisions are based on actual access behavior, maintaining high cache hit rates while optimizing bandwidth usage by loading only necessary sub-blocks.
3Loss of energy
If compression of off-chip traffic is used to minimize bandwidth, then bandwidth requirements are reduced, but additional latency is introduced due to decompression overhead
Solution Approach 1:
Instead of compressing entire cache blocks, the cache is segmented into sub-blocks and only the necessary sub-blocks are loaded in uncompressed form. This avoids the decompression overhead entirely while achieving bandwidth reduction through selective loading, thus resolving the contradiction between bandwidth requirements and decompression latency.
Data Source
AI summary
A method and mechanism are managing caches. A cache is configured to store blocks of data based upon predictions of future accesses. Each block is partitioned into sub-blocks, and if it is predicted a given sub-block is unlikely to be accessed, the sub-block may not be stored in the cache. Associated with each block is a mask which indicates whether sub-blocks of the block are likely to be accessed. When a block is first loaded into the cache, the corresponding mask is cleared and an indication is set for the block to indicate a training mode for the block. Access patterns of the block are then monitored and stored in the mask. If a given sub-block is accessed a predetermined number of times, a bit in the mask is set to indicate that the sub-block is likely to be accessed. When a block is evicted from the cache, the mask is also transferred for storage and only the sub-blocks identified by the mask as being likely to be accessed may be transferred for storage. If previously evicted data is restored to the cache, a previously stored mask is accessed to determine which of the sub-blocks are predicted likely to be accessed. The lower level storage may then transfer only those sub-blocks predicted likely to be accessed to the cache.