Cache Occupancy Estimation for Asymmetric Requester Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In data processing systems with shared caches, asymmetry in cache usage between multiple requesters can lead to inefficient data retrieval, as one requester may evict cache lines needed by another, and existing methods to monitor or manage this asymmetry are either storage-intensive or limit the number of requesters, causing inefficiencies and unfair resource allocation.
Innovation Solution
A cache memory system with cache storage and control circuitry that tracks the insertion of cache lines for each requester over a defined period, using this data to bias the retention policy and prioritize cache line eviction, ensuring fair or asymmetric usage based on usage metrics and performance requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the cache stores cache lines for multiple requesters without tracking usage, then the cache capacity is maximized, but asymmetry in cache usage occurs where one requester may evict cache lines needed by another, reducing system performance
Solution Approach 1:
The patent implements feedback by tracking the number of cache lines inserted for each requester and using this information to influence eviction decisions. The cache control circuitry monitors usage patterns and adjusts retention policies based on this feedback, preventing asymmetric cache usage while maintaining overall cache efficiency.
Solution Approach 2:
The patent applies dynamics by making the cache retention policy adaptive rather than static. The eviction policy dynamically adjusts based on real-time usage counts for each requester, allowing the cache to respond to changing access patterns and prevent any single requester from dominating cache resources.
2Measurement precision
If the cache monitors cache line ownership and usage for each requester, then asymmetry in cache usage can be detected and managed, but additional storage space is required
Solution Approach 1:
The patent extracts only the essential monitoring information needed for fair cache management - specifically, a count of cache line insertions per requester - rather than storing complete ownership metadata for every cache line. This minimal extraction achieves adequate monitoring accuracy while consuming minimal additional storage space.
Solution Approach 2:
The patent changes the monitoring parameter from detailed cache line ownership tracking to a simpler insertion count metric. This parameter change reduces the storage requirements while still providing sufficient information to detect and manage asymmetric cache usage patterns.
3Productivity
If the cache is partitioned to limit requester access, then cache usage asymmetry is controlled, but the number of requesters that can use the cache is limited and associativity effects are reduced
Solution Approach 1:
The patent implements a universal cache management approach that serves multiple requesters simultaneously without physical partitioning. The single cache structure with usage tracking and biased eviction policies provides fair access to any number of requesters, maintaining cache associativity while ensuring usage fairness through software-controlled policies.
Solution Approach 2:
The patent introduces an intermediary mechanism - the usage count tracking and biased eviction policy - that mediates between multiple requesters competing for cache space. This intermediary layer prevents any single requester from dominating the cache without requiring physical partitioning, thus maintaining cache versatility while ensuring fair access.
Data Source
AI summary
A cache memory and method of operating a cache memory are provided. The cache memory comprises cache storage that stores cache lines for a plurality of requesters and cache control circuitry that controls insertion of a cache line into the cache storage when a memory access request from one of the plurality of requesters misses in the cache memory. The cache memory further has cache occupancy estimation circuitry that holds a count of insertions of cache lines into the cache storage for each of the plurality of requesters over a defined period. The count of cache line insertions for each requester thus provides an estimation of the cache occupancy associated with each requester.