Virtual Memory Pre-Fetch Circuit for MMU Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The inefficiency in virtual memory address translation due to bottlenecks in memory management units (MMUs) in System on Chip (SoC) devices, where multiple masters share a common physical memory, leading to latency and unpredictability in memory accesses.
Innovation Solution
Implementing a distributed memory pre-fetching system with multiple pre-fetch generating circuits for each port of the MMU, which monitor virtual memory access requests, detect trigger conditions, and generate pre-fetch requests to populate Translation Look-aside Buffers (TLBs) proactively, thereby reducing the need for main memory accesses and mitigating page table walk delays.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If virtual memory address translation is performed using a traditional MMU circuit, then address translation functionality is provided, but translation latency and bottleneck efficiency issues occur
Solution Approach 1:
The patent implements pre-fetch circuits that proactively load translation entries into the TLB before they are actually needed. The pre-fetch circuit monitors virtual address access patterns and initiates translation requests in advance, populating the TLB with predicted future translation needs. This preliminary action reduces the latency when actual translation is required, as the translation entries are already cached in the TLB rather than requiring a full page table walk at the moment of access.
2Reliability
If the TLB is populated with more translation entries, then TLB hit rate improves, but the complexity of managing the TLB and handling conflicts increases
Solution Approach 1:
The patent segments the TLB into multiple independent sets or ways, allowing parallel access and reducing conflicts. Instead of managing a single large TLB structure, the system divides it into smaller segments that can be independently managed and accessed. This segmentation reduces the complexity of managing a single large cache while maintaining high hit rates through distributed storage of translation entries across multiple segments.
Solution Approach 2:
The patent implements dynamic TLB management where the structure and size of TLB segments can be adjusted based on workload characteristics. The system dynamically allocates TLB resources and adjusts segmentation based on observed access patterns, allowing flexible adaptation to different translation demands without requiring a fixed complex structure. This dynamic approach simplifies management by adapting the TLB configuration to actual needs rather than maintaining a static complex structure.
3Adaptability or versatility
If multiple masters share a common physical memory through virtual memory, then resource sharing is enabled, but memory access predictability and efficiency deteriorate
Solution Approach 1:
The patent introduces an intermediary pre-fetch circuit that sits between the multiple masters and the MMU/TLB system. This pre-fetch intermediary monitors and analyzes access patterns from multiple masters, predicts their future translation needs, and proactively loads entries into the TLB. By acting as an intermediary that anticipates and prepares translation entries, the system maintains efficient memory access for multiple shared masters without requiring each master to wait for translation completion, thus preserving productivity while enabling versatile sharing.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Virtual memory pre-fetch requests are generated for a virtual memory and a multiple port memory management unit (MMU) circuit (114). Virtual memory access requests sent to a particular port (124, 126, 128) of the MMU circuit (114) are monitored. In response to the satisfaction of a trigger conditions, virtual memory pre-fetch requests are generated and transmitted to the MMU circuit (114) using the particular port (124, 126, 128). Physical access requests from the MMU circuit (114) are monitored for physical addresses corresponding to the virtual memory pre-fetch requests. The physical access requests corresponding to the virtual memory pre-fetch requests are filtered.