On-Chip Cache Priority for Speculative Code Lines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increased size of cache sizes in processors leads to performance loss due to speculative code lines occupying non-trivial cache capacity, especially with large code footprints from deep software stacks, interpreted or Just In Time code, and multiple code instances, which are rarely used and consume valuable cache space.
Innovation Solution
Implementing a Bloom filter and a Recent Multi-Level Cache (RMRQ) mechanism to identify and de-prioritize speculative code lines in on-chip caches, ensuring that non-speculative lines are prioritized, thereby optimizing cache usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If cache size is increased to allow for faster access to data, then cache access speed is improved, but cost increases due to additional footprint and power consumption
Solution Approach 1:
The patent segments the cache space by introducing a priority field that divides cache lines into different priority levels. This allows the cache to function as multiple logical segments (high priority and low priority regions) within the same physical structure, enabling selective management of speculative versus non-speculative code lines without increasing physical cache size.
Solution Approach 2:
The patent changes the parameter of cache line prioritization by adding a priority field to cache lines. This parameter change enables the cache controller to differentiate between speculative and non-speculative code lines, allowing non-speculative lines to be prioritized for retention while speculative lines can be evicted more readily, improving effective cache utilization without expanding cache capacity.
2Speed
If cache size is increased to allow for faster access to data, then cache access speed is improved, but cache footprint increases
Solution Approach 1:
The patent segments the cache space by introducing a priority field that divides cache lines into different priority levels. This allows the cache to function as multiple logical segments (high priority and low priority regions) within the same physical structure, enabling selective management of speculative versus non-speculative code lines without increasing physical cache size.
Solution Approach 2:
The patent changes the parameter of cache line prioritization by adding a priority field to cache lines. This parameter change enables the cache controller to differentiate between speculative and non-speculative code lines, allowing non-speculative lines to be prioritized for retention while speculative lines can be evicted more readily, improving effective cache utilization without expanding cache capacity.
3Quantity of substance
If speculative code lines are stored in cache, then cache capacity is occupied, but processor performance deteriorates due to displacement of useful code lines
Solution Approach 1:
The patent applies preliminary action by setting the priority field of speculative cache lines to a low priority state before they can displace useful code lines. This preemptive marking ensures that when cache eviction is needed, speculative lines are automatically selected for eviction first, preventing them from occupying cache space that would be needed for performance-critical non-speculative code lines.
Solution Approach 2:
The patent implements feedback through the priority field mechanism that continuously informs the cache controller about the nature of each cache line. The cache controller uses this feedback to make intelligent eviction decisions, preferentially selecting speculative lines (low priority) over non-speculative lines (high priority), thereby maintaining processor performance while utilizing full cache capacity.
Data Source
AI summary
Methods and apparatus relating to de-prioritizing speculative code lines in on-chip caches are described. In an embodiment, logic circuitry determines whether a storage structure includes a reference to a code miss request prior to transmission of the code miss request to a shared cache. The logic circuitry causes de-prioritization of a code line, corresponding to the code miss request, in the shared cache in response to an absence of the reference in the storage structure. Other embodiments are also disclosed and claimed.