Cost Based Cache Eviction for Heterogeneous Computational Workloads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cache eviction strategies, such as LRU and LFU, are inefficient when dealing with heterogeneous computational costs across cache entries, as they do not account for the varying resources used to generate data, leading to increased system work during cache misses.
Innovation Solution
Implement a cost-based management system that calculates and utilizes a cost measure for each cache entry, prioritizing eviction based on the lowest cost to recompute, thereby reducing overall system work by preserving higher-cost entries in the cache.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional cache eviction strategies (LRU, LFU, FIFO) are used, then cache management is simple and fast, but system work increases during cache misses due to heterogeneous computational costs
Solution Approach 1:
The patent changes the eviction parameter from simple metrics (access time, access count) to a composite cost parameter that includes computational expense. Each cache entry is tagged with a cost value representing the computational resources required to regenerate it, and eviction decisions are based on selecting entries with the lowest cost values, thereby reducing system work during cache misses.
2Loss of energy
If cache entries with lowest computational cost are prioritized for eviction, then system work during cache misses is reduced, but cache management complexity increases due to cost calculation and tracking
Solution Approach 1:
The system performs preliminary action by calculating and storing the computational cost value for each cache entry at the time of insertion or regeneration. This pre-calculated cost information is then used during eviction decisions, avoiding the need to perform complex cost analysis at eviction time and reducing the overhead of cache management.
3Ease of operation
If homogeneous cache eviction is used (treating all entries equally), then cache management is straightforward, but it fails to account for varying resources used to generate different data types
Solution Approach 1:
The patent applies local quality by assigning different cost characteristics to different cache entries based on their specific computational requirements. Instead of treating all cache entries uniformly, the system evaluates and tags each entry with its own cost value, allowing the cache management to adapt locally to the specific needs of each data type while maintaining a unified eviction framework.
Data Source
AI summary
Technologies are described for cost based management of cache entries stored in a computer memory. In one example, a plurality of cache entries may be stored at a cache in a computer memory and the cache entries may have a cost measure associated with individual cache entries. A cost measure may represent a computing cost of an application to generate a cache entry. An incoming cache entry may be received at the cache, where the incoming cache entry has a cost measure associated with the incoming cache entry. In response to receiving the incoming cache entry, a cache entry that has a lower cost measure than the cost measure for other cache entries may be identified for eviction from the cache. The cache entry identified for eviction may be evicted from the cache, and the incoming cache entry may be written into the cache stored in the computer memory.