Fetch Request Arbiter for Microprocessor Cache Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In microprocessor systems with multi-level cache memory, cache miss latency is high due to the bottleneck of the address translation unit, which takes longer to resolve TLB misses and requires multiple computing cycles for restarting rejected fetch requests, leading to inefficiencies in data fetch operations across cache levels.
Innovation Solution
A Request Central Station (RCS) or fetch request arbiter prioritizes and manages rejected cache requests by determining the presence of absolute addresses in the TLB, passing requests with available translations to the next cache level, and retaining those without translations, allowing for efficient processing as system conditions change, thereby reducing cache miss latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the address translation unit processes TLB lookups sequentially, then translation accuracy is maintained, but cache miss latency increases due to bottleneck delays
Solution Approach 1:
The system performs preliminary address translation by loading translation tables into the TLB in advance before fetch requests are made. When a TLB miss occurs, the required translation tables are pre-loaded into the TLB during idle periods or in parallel with other operations, so that subsequent fetch requests can be completed quickly without waiting for translation table loading.
Solution Approach 2:
The patent introduces a multi-dimensional approach to address translation by implementing a hierarchical translation structure with multiple TLB levels and parallel translation paths. Instead of a single sequential translation pipeline, the system uses multiple translation units operating in parallel and a hierarchical TLB structure that can service requests from different levels simultaneously, effectively adding temporal and spatial dimensions to the translation process.
2Productivity
If rejected fetch requests are restarted immediately, then request throughput is maintained, but system efficiency decreases due to repeated translation failures
Solution Approach 1:
The system implements a feedback mechanism where the address translation unit monitors TLB hit/miss ratios and translation success rates. When translation failures are detected, the system adjusts its behavior by pre-loading the required translation tables into the TLB before restarting fetch requests. The feedback loop also triggers optimizations such as adjusting TLB replacement policies or pre-fetching translation tables for frequently accessed virtual addresses, thereby reducing repeated translation failures and improving overall system efficiency.
3Speed
If the TLB is kept small for fast access, then access speed is improved, but translation capacity is reduced leading to more misses
Solution Approach 1:
The patent divides the TLB into multiple segments or levels (TLB1, TLB2, etc.), where each level handles different portions of the virtual address space or different types of translations. This segmentation allows each TLB level to remain small and fast while collectively providing large translation capacity. The hierarchical structure enables the system to maintain fast access times for frequently used translations in smaller upper levels while storing less frequently accessed translations in larger lower levels.
Data Source
AI summary
Driving address translations in a microprocessor system by sending a rejected Lx+1 cache request from a first set of caches of a first level Lx to a central request unit, transferring an Lx+1 cache request having a translation of a virtual address into a physical address stored in a first buffer, from the central request unit to the at least one Lx+1 cache, and keeping an Lx+1 cache request lacking a translation of a virtual address into a physical address stored in the first buffer, pending in the central request unit.