Joint Prefetch Scheduling for Pointer-Based Cache Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional cache management techniques exhibit limitations in responsiveness and adaptability to varying workloads and data access patterns, affecting data retrieval performance in modern computing systems.
Innovation Solution
A joint scheduler is employed to jointly schedule prefetch and demand accesses, associating instructions with cache entries and tracking invalidation/eviction, allowing for correlated dispatch of prefetch and demand operations to optimize cache usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional cache management techniques are used, then cache operations are simple to implement, but responsiveness and adaptability to varying workloads and data access patterns are limited
Solution Approach 1:
The patent merges the prefetch scheduler and demand scheduler into a single joint scheduler that handles both prefetch and demand accesses. This unified scheduler shares common data structures (valid indications and pointers) between prefetch and demand operations, enabling coordinated scheduling decisions that improve adaptability to workload patterns while avoiding the complexity of separate independent scheduling systems.
Solution Approach 2:
The joint scheduler implements multi-functionality by using the same scheduling infrastructure (valid indications and pointers) to serve both prefetch operations and demand operations. This universal approach allows the scheduler to adapt to different access patterns (prefetch vs demand) using a single flexible mechanism rather than requiring separate specialized schedulers for each operation type.
2Productivity
If prefetch accesses are performed for all instructions, then data retrieval performance is improved, but memory bandwidth is consumed unnecessarily for data already in cache
Solution Approach 1:
The system performs preliminary prefetch actions for instructions in the execution pipeline, checking cache validity in advance. By using valid indications to pre-mark cache entries as valid before demand accesses occur, the system prepares data in advance when possible, improving retrieval performance while avoiding redundant prefetch operations for already-cached data through the validity check mechanism.
Solution Approach 2:
The joint scheduler implements feedback through valid indications that track the validity status of cache entries. When a prefetch access hits valid data, the valid indication prevents subsequent redundant prefetch operations for the same data. This feedback mechanism allows the system to adapt prefetch behavior based on actual cache state, improving data retrieval performance while reducing unnecessary memory bandwidth consumption.
3Reliability
If demand accesses perform full cache lookups, then cache hit detection is accurate, but access latency increases due to redundant lookups
Solution Approach 1:
The system performs preliminary actions by setting valid indications and pointers during prefetch operations before demand accesses occur. When a prefetch access validates cache data, the valid indication is set in advance, so that subsequent demand accesses can rely on this pre-established validity information rather than performing complete cache lookups, thereby reducing latency while maintaining accurate hit detection.
Solution Approach 2:
The valid indication acts as an intermediary between the cache storage and the demand access logic. Instead of requiring demand accesses to perform full cache lookups to determine validity, the valid indication serves as a pre-computed intermediary signal that directly indicates cache hit status, reducing the time required for demand access validation while maintaining accurate cache hit detection.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A joint scheduler adapted for dispatching prefetch and demand accesses of data relating to a plurality of instructions loaded in an execution pipeline of processing circuit(s). Each prefetch access comprises checking whether a respective data is cached in a cache entry and each demand access comprises accessing a respective data. The joint scheduler is adapted to, responsive to each hit prefetch access dispatched for a respective data relating to a respective instruction, associate the respective instruction with a valid indication and a pointer to a respective cache entry storing the respective data such that the demand access relating to the respective instruction uses the associated pointer to access the respective data in the cache, and responsive to each missed prefetch access dispatched for a respective data relating to a respective instruction, initiate a read cycle for loading the respective data from next level memory and cache it in the cache.