Cache-Aware Memory Block Prefetching for Priority Request Isolation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The memory wall issue arises due to the disparity in speed between processors and memory, leading to prolonged idle periods as processors wait for data, which is addressed by existing prefetching techniques that often result in inaccurate data retrieval.
Innovation Solution
A prefetch engine that prioritizes high-priority memory access requests over low-priority multi-prefetch requests by verifying cache states and strategically timing prefetch bursts to avoid interference, using a Finite State Machine (FSM) for governance and queue management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If prefetching techniques are used to hide memory latency, then processor idle time is reduced, but prefetching accuracy deteriorates leading to incorrect data retrieval
Solution Approach 1:
The patent applies dynamics by making the prefetching behavior adaptive rather than static. The system dynamically adjusts prefetching based on cache state detection - when the cache is idle, prefetching is enabled to reduce processor waiting time; when the cache is busy, prefetching is suspended to avoid interfering with high-priority operations. This dynamic adaptation resolves the contradiction between maximizing prefetching benefits and maintaining accuracy.
Solution Approach 2:
The patent changes the operational parameters of the prefetching engine based on system conditions. By monitoring cache busy/idle states and adjusting the prefetching enable/disable parameter accordingly, the system optimizes both processor utilization and prefetching accuracy. The parameter change from enabled to disabled state resolves the trade-off between reducing idle time and maintaining precision.
2Productivity
If multi-prefetch requests are processed simultaneously with high-priority memory access requests, then memory throughput is increased, but interference between request types increases
Solution Approach 1:
The patent segments memory access requests into two distinct categories: high-priority requests (loads, stores, atomic operations) and low-priority prefetch requests. This segmentation allows the system to handle different request types through different processing paths, preventing interference while maintaining high throughput for both categories.
Solution Approach 2:
The patent introduces an intermediary mechanism - the cache state detection and control logic - that mediates between high-priority and low-priority requests. This intermediary monitors the cache state and selectively enables or disables prefetching based on current workload, ensuring that high-priority operations are never interfered with while maximizing prefetching opportunities when the cache is idle.
3Productivity
If prefetch engine operates continuously to maximize data availability, then processor performance is improved, but interference with high-priority operations increases
Solution Approach 1:
The patent applies preliminary anti-action by proactively preventing interference before it occurs. The system continuously monitors cache state and preemptively disables prefetching when high-priority operations are detected, rather than allowing interference to happen and then correcting it. This preliminary protective action ensures processor performance is maximized without the harmful side effect of interfering with critical operations.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods are provided of prefetching memory blocks in a processor system comprising a processor and a memory including a main memory, a cache and a prefetcher. Two types of memory requests are executable at the memory: low-priority type including multi-prefetch requests and high-priority type including requests different from multi-prefetch requests. These methods comprise: receiving, by the prefetcher, a multi-prefetch request of the low-priority type defining a 1D or 2D region of memory blocks in the main memory; verifying, by the prefetcher, whether the cache is in idle state which means that no high-priority memory access request is being executed or waiting to be executed; and, if the cache is in the idle state, triggering by the prefetcher a prefetch burst by instructing the cache to individually prefetch the memory blocks forming the 1D or 2D region. Computer programs and prefetchers are also provided that are suitable to perform such methods.