Cache Memory Request Conversion for Low-Latency Address Translation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The latency associated with accessing data stored in main memory is high due to the need for address translation and cache misses, which can stall processor operations and increase power consumption.
Innovation Solution
A converter module is introduced between the cache and memory to handle memory requests, assigning transaction identifiers, tracking request status, and prioritizing MMU requests, allowing out-of-order issuance and reducing translation latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If address translation is performed sequentially before memory access, then translation accuracy is ensured, but memory access latency increases
Solution Approach 1:
The patent performs address translation in advance by creating translation lookaside buffer (TLB) entries before memory access is needed. When a virtual address needs to be translated, the system checks if a TLB entry already exists, and if so, uses it immediately without waiting for sequential translation. This preliminary preparation of translation mappings resolves the contradiction by ensuring both accurate translation (through proper TLB entry creation) and low latency (through cache-like retrieval of existing translations).
Solution Approach 2:
The patent introduces a translation lookaside buffer (TLB) as an intermediary structure between the virtual memory system and physical memory. The TLB acts as a cache for address translations, mediating between the need for accurate translation and the need for fast access. By inserting this intermediary layer, the system can provide rapid translation lookup while maintaining translation accuracy through proper TLB management and invalidation protocols.
2Stability of the object's composition
If memory requests are handled in strict order, then system stability is maintained, but processor productivity decreases due to stalling
Solution Approach 1:
The patent implements dynamic memory request handling where the order of processing memory requests can change based on availability of translation information. Instead of strict sequential processing, the system dynamically adjusts the processing order by allowing requests with available TLB entries to proceed while waiting for other requests. This dynamic approach maintains system stability through proper synchronization while improving processor productivity by preventing unnecessary stalling.
Solution Approach 2:
The patent enables continuous processor operation by allowing memory requests to be processed as soon as their translations are available, rather than waiting for all translations to complete in sequence. The processor can continue executing instructions and issuing memory requests without stalling, maintaining continuous useful action. The system ensures stability by properly managing the ordering and completion of memory operations through hardware mechanisms that track request status and ensure correct data flow.
3Productivity
If cache size is increased to reduce miss rate, then cache efficiency improves, but device complexity and power consumption increase
Solution Approach 1:
The patent segments the memory hierarchy into multiple cache levels (L1, L2, L3 caches) with different sizes and characteristics. Instead of using a single large cache, the system divides the caching function across multiple smaller cache structures, each optimized for specific access patterns. This segmentation reduces the complexity of any single cache structure while maintaining overall high cache efficiency through the hierarchical arrangement, where smaller faster caches handle frequent accesses and larger slower caches handle less frequent accesses.
Solution Approach 2:
The patent applies local quality optimization by creating different cache structures with different characteristics for different purposes. L1 cache is small and fast for frequently accessed data, L2 cache is larger for moderate-frequency access, and L3 cache is even larger for less frequent access. Each cache level has locally optimized properties (size, speed, associativity) suited to its specific function in the hierarchy, rather than using a uniform cache structure throughout. This resolves the contradiction by achieving high overall efficiency through localized optimizations rather than requiring one large complex cache.
Data Source
AI summary
A converter module handles memory requests issued by a cache (e.g. an on-chip cache), where these memory requests include memory addresses defined within a virtual memory space. The converter module receives these requests, issues each request with a transaction identifier and uses that identifier to track the status of the memory request. The converter module sends requests for address translation to a memory management unit and where there the translation is not available in the memory management unit receives further memory requests from the memory management unit. The memory requests are issued to a memory via a bus and the transaction identifier for a request is freed once the response has been received from the memory. When issuing memory requests onto the bus, memory requests received from the memory management unit may be prioritized over those received from the cache.