Cache Drop Feature for Temporal Data to Increase Memory Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cache misses in hierarchical memory subsystems lead to increased memory latency and power consumption due to unnecessary data storage and retrieval of temporal data that is accessed only briefly, causing inefficiencies in system performance.
Innovation Solution
Implementing a data set drop control mechanism that identifies and marks temporal data sets with a non-replaceable attribute, preventing eviction and allowing for explicit drop commands or hints to remove data from the cache without writing it back to lower-level memory, thereby reducing memory traffic and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If temporal data is stored in cache for extended periods, then cache hit rate improves, but memory bandwidth is wasted and power consumption increases
Solution Approach 1:
The patent implements dynamic cache line management by introducing a temporal data indicator that allows cache lines to be marked for retention or drop based on their access patterns. The cache controller dynamically decides whether to retain or drop cache lines using the formula: if (temporal_data_indicator == TRUE && access_count < retention_threshold) { drop_cache_line(cache_line_address); } else if (access_count > retention_threshold) {
Solution Approach 2:
The patent changes the state parameter of cache lines by introducing a temporal data indicator bit and access count metadata. These parameter changes enable the cache to distinguish between temporal and persistent data, allowing selective retention or drop operations that optimize both hit rate and power consumption based on data characteristics
2Speed
If cache size is increased to store more temporal data, then access latency decreases, but memory bandwidth is consumed for unnecessary data transfers
Solution Approach 1:
The patent implements a selective discarding mechanism for temporal cache lines based on access patterns. When a cache line is accessed, the access count is incremented and compared against a retention threshold. If the threshold is not met, the cache line is dropped using drop_cache_line(cache_line_address), preventing unnecessary memory bandwidth consumption while maintaining fast access for frequently used data
Solution Approach 2:
The patent applies partial retention by selectively keeping only those cache lines that meet the retention threshold criteria. This partial action approach avoids the excessive bandwidth consumption of retaining all temporal data while ensuring that frequently accessed data is maintained in cache for fast access
3Productivity
If cache replacement policy is changed to retain temporal data, then system performance improves, but cache efficiency decreases due to unnecessary data occupying cache slots
Solution Approach 1:
The patent segments the cache management logic by introducing a temporal data indicator that separates temporal data from persistent data. This segmentation allows the cache controller to apply different retention policies: temporal data with low access counts is dropped, while persistent data and frequently accessed temporal data are retained, improving overall cache efficiency and system performance
Data Source
AI summary
Systems, apparatuses, and methods for efficiently allocating data in a cache are described. In various embodiments, a processor decodes an indication in a software application identifying a temporal data set. The data set is flagged with a data set identifier (DSID) indicating temporal data to drop after consumption. When the data set is allocated in a cache, the data set is stored with a non-replaceable attribute to prevent a cache replacement policy from evicting the data set before it is dropped. A drop command with an indication of the DSID of the data set is later issued after the data set is read (consumed). A copy of the data set is not written back to the lower-level memory although the data set is removed from the cache. An interrupt is generated to notify firmware or other software of the completion of the drop command.