Virtual Request Trackers for Memory Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Systems on a chip (SoCs) face challenges in ensuring memory requirements for multiple clients while minimizing power consumption and silicon area, as existing solutions either increase power consumption, cost, or both, due to high memory latency and inefficient processor utilization.
Innovation Solution
A memory request optimizer is introduced, which includes a cache and a memory tracker with allocation and prefetch logic, using virtual request trackers and a least recently used algorithm to allocate and prefetch data based on historical request patterns, optimizing memory access and reducing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If predictive schemes or additional processors are added to reduce memory latency, then memory access speed is improved, but device complexity and system cost increase
Solution Approach 1:
The patent implements a predictive prefetching mechanism that proactively loads data into the cache before it is actually requested by the processor. The prefetcher analyzes memory access patterns and anticipates future data needs, performing preliminary data retrieval actions. This eliminates the need for complex additional processors or predictive schemes while reducing memory latency through advance data preparation.
2Speed
If cache sizes are increased to reduce memory latency, then memory access speed is improved, but silicon area and system cost increase
Solution Approach 1:
Instead of increasing cache size, the patent uses a predictive prefetching mechanism that proactively loads only the specific data elements likely to be needed soon into the existing cache. This selective preliminary action maximizes cache utilization efficiency without requiring additional silicon area for cache expansion.
Solution Approach 2:
The patent changes the operational parameters of the memory subsystem by implementing intelligent prefetching logic that dynamically adjusts data loading based on access patterns. This parameter change optimizes the existing cache's effectiveness without physical expansion, maintaining the same silicon area while improving memory access speed.
3Speed
If processor clock rate is increased to compensate for memory latency, then processing speed is improved, but power consumption increases
Solution Approach 1:
The patent reduces the need for high processor clock rates by implementing predictive prefetching that prepares data in advance. This allows the processor to operate at lower clock rates since it won't be stalling waiting for memory operations, thereby reducing power consumption while maintaining processing speed.
4Device complexity
If traditional memory access methods are used, then system simplicity is maintained, but memory latency and power consumption increase
Solution Approach 1:
The patent introduces a prefetching mechanism that performs preliminary data retrieval operations based on predicted access patterns. This adds minimal complexity to the traditional memory access method while significantly reducing memory latency by ensuring data is ready before the processor needs it.
Solution Approach 2:
The patent implements a feedback-based prefetching system that monitors actual memory access patterns and uses this information to improve future prefetching decisions. This feedback mechanism optimizes the balance between system simplicity and memory latency reduction by adapting to actual workload characteristics.
Data Source
AI summary
A memory request optimizer includes a memory tracker for receiving a read request from client devices and for determining whether the request address matches any of the virtual request trackers. If the request address does not match any virtual request tracker, an allocation logic allocates a next available virtual request tracker to track the request address. When the request address matches a virtual request tracker, a prefetch logic increments a current tracker match count for the virtual request tracker and determines whether a linear history of tracker match counts indicates a prefetch of a next request data is appropriate based on one or more predetermined criteria. If the linear history indicates the prefetch is appropriate, the prefetch logic obtains the next request data at an address equal to the request address plus a preconfigured request offset from the memory sub-system and stores the next request data in a cache.