Dynamic Cache Refill Size Adjustment for Hit Rate Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In a cache memory system, it is challenging to set an optimal refill size due to the difficulty in predicting the spatial locality of data access patterns among multiple operating programs, leading to inefficiencies and increased process delays.
Innovation Solution
A cache memory device that includes an address extraction unit, a hit/miss judgment unit, a management unit, a count unit, an identification unit, a determination unit, and a refill unit, which together manage and adjust the refill size based on the access patterns of sequential line sets to optimize data refilling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the refill size is increased to utilize spatial locality, then the hit rate can be increased, but the data capacity of the cache memory is limited and refilled data may be replaced, decreasing the hit rate
Solution Approach 1:
The refill size is made dynamic rather than fixed. The determination unit adjusts the refill size based on real-time access patterns detected by the count unit. When sequential accesses are detected, the refill size is increased to capture more spatially local data. When access patterns are non-sequential, the refill size is reduced to prevent wasting cache capacity. This dynamic adjustment resolves the contradiction by adapting the refill size to actual program behavior rather than using a static value.
Solution Approach 2:
The refill size parameter is changed based on detected access patterns. The system monitors the number of accesses to cache lines and uses this information to adjust the refill size parameter dynamically. This parameter change allows the system to optimize the hit rate by matching the refill size to the actual spatial locality exhibited by the program, preventing both under-utilization and overwriting of cache data.
2Reliability
If the refill size is set to be large due to misestimation of spatial locality, then more data is refilled, but most of refilled data is not used and process delay is caused
Solution Approach 1:
The system implements feedback through the count unit that monitors actual cache access patterns. This feedback information is used by the determination unit to adjust the refill size dynamically. When the count unit detects that accessed cache lines are not sequentially referenced, it provides feedback that triggers a reduction in refill size, preventing the refilling of data that will not be used and avoiding the associated process delays.
Solution Approach 2:
Instead of always performing a full or large refill, the system performs a partial refill only when the access pattern indicates sequential usage. The count unit detects whether accessed lines are referenced in sequence, and only when this condition is met does the system perform a larger refill. This partial action approach avoids the overhead of refilling data that won't be used, eliminating unnecessary process delays.
3Productivity
If the refill size is set to be small to avoid process delay, then the hit rate might be decreased due to insufficient utilization of spatial locality
Solution Approach 1:
The refill size is dynamically adjusted based on detected access patterns rather than being fixed at a small value. When the count unit detects sequential access patterns indicating strong spatial locality, the determination unit increases the refill size to capture more useful data, improving the hit rate without permanently committing to a large refill size that would cause delays in other scenarios.
4Device complexity
If a fixed refill size is used to simplify cache management, then the device complexity is reduced, but the ability to adapt to different program access patterns is limited
Solution Approach 1:
The cache management system performs self-service by automatically detecting access patterns and adjusting refill sizes without external intervention. The count unit autonomously monitors cache line access patterns, and the determination unit autonomously decides the appropriate refill size based on this monitoring. This self-service mechanism provides adaptability to different program behaviors while keeping the control logic integrated within the cache management unit, avoiding the need for complex external control systems.
Data Source
AI summary
To aim to provide a cache memory device that performs a line size determination process for determining a refill size, in advance of a refill process that is performed at cache miss time. According to the line size determination process, the number of reads/writes of a management target line that belongs to a set is acquired (S51), and in the case where the numbers of reads completely match one another and the numbers of writes completely match one another (S52: Yes), the refill size is determined to be large (S54). Otherwise (S52: No), the refill size is determined to be small (S55).