Joint Scheduler for Low-Latency Prefetch and Demand Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional cache management techniques exhibit limitations in responsiveness and adaptability to varying workloads and data access patterns, affecting data retrieval performance in modern computing systems.
Innovation Solution
A joint scheduler is employed to jointly schedule prefetch and demand accesses, associating instructions with cache entries and tracking invalidation/eviction, allowing simultaneous dispatch of prefetch and demand accesses, and initiating read cycles as needed to maintain cache efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional cache management techniques are used, then cache structure is simple, but responsiveness and adaptability to varying workloads and data access patterns deteriorates
Solution Approach 1:
The patent merges the prefetch scheduler and demand scheduler into a single joint scheduler that handles both prefetch and demand accesses. This unified scheduler shares common data structures (valid indications and pointers) between prefetch and demand paths, reducing overall system complexity while improving adaptability through coordinated scheduling of both access types.
Solution Approach 2:
The joint scheduler serves multiple functions: it schedules both prefetch and demand accesses, manages cache entry validity, maintains pointers to cache entries, and coordinates read cycles. This multi-functional approach eliminates the need for separate management mechanisms, reducing complexity while enhancing versatility in handling different workload patterns.
2Loss of time
If separate prefetch and demand schedulers are used, then scheduling is simple, but memory access latency increases due to redundant cache lookups
Solution Approach 1:
By merging prefetch and demand scheduling into a single joint scheduler, the patent eliminates redundant cache lookups. The joint scheduler checks cache validity once during prefetch and reuses this information for demand accesses, significantly reducing memory access latency while maintaining manageable complexity through unified data structures.
Solution Approach 2:
The joint scheduler performs preliminary cache validity checks during the prefetch phase and stores the results in valid indications and pointers. When demand accesses occur, these pre-computed validity results are reused, eliminating the need for redundant cache lookups and reducing latency.
3Productivity
If prefetch accesses are dispatched without joint scheduling, then prefetch operation is simple, but bandwidth utilization deteriorates due to uncoordinated memory operations
Solution Approach 1:
The joint scheduler merges prefetch and demand access scheduling into a single coordinated system. This allows the scheduler to optimize bandwidth utilization by coordinating memory operations, ensuring that prefetch and demand accesses are efficiently managed without conflicts, while the shared data structures keep the implementation complexity manageable.
Data Source
AI summary
A joint scheduler adapted for dispatching prefetch and demand accesses of data relating to a plurality of instructions loaded in an execution pipeline of processing circuit(s). Each prefetch access comprises checking whether a respective data is cached in a cache entry and each demand access comprises accessing a respective data. The joint scheduler is adapted to, responsive to each hit prefetch access dispatched for a respective data relating to a respective instruction, associate the respective instruction with a valid indication and a pointer to a respective cache entry storing the respective data such that the demand access relating to the respective instruction uses the associated pointer to access the respective data in the cache, and responsive to each missed prefetch access dispatched for a respective data relating to a respective instruction, initiate a read cycle for loading the respective data from next level memory and cache it in the cache.