Shared Last-Level TLB for Multiprocessor Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing workload data-set sizes and number of hardware threads in modern multiprocessor systems lead to higher TLB miss rates, resulting in significant address-translation latency due to the need for page-table walks, which can take hundreds of clock cycles, impacting the performance of shared-memory server systems.
Innovation Solution
Implementing a shared last-level TLB that partitions TLB entries across multiple nodes, allowing nodes to share TLB entries and cancel speculative page-table walks if the needed entry is cached, thereby reducing the number of page-table walks and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple hardware threads share a TLB to improve resource utilization, then TLB pressure increases and miss rates increase, but adding more TLB capacity per node is limited by on-chip resources
Solution Approach 1:
The virtual address space is partitioned across multiple nodes, with each node's last-level TLB responsible for a specific subset of virtual addresses. This segmentation allows TLB entries to be distributed across nodes, effectively increasing total TLB capacity without requiring larger on-chip TLBs at each node.
Solution Approach 2:
Multiple nodes' last-level TLBs are merged to form a shared last-level TLB resource. When a node experiences a TLB miss, it can retrieve the translation from another node's TLB, combining the caching capabilities of multiple nodes to reduce overall miss rates.
2Reliability
If a TLB miss occurs and a page-table walk is initiated to retrieve the translation, then the translation can be obtained, but the latency increases by hundreds of clock cycles
Solution Approach 1:
The system performs preliminary actions by maintaining TLB entries in multiple nodes' last-level TLBs in advance. When a translation is needed, the system checks shared TLBs before initiating a full page-table walk, reducing the need for time-consuming walks by having translations pre-available in the TLB hierarchy.
Solution Approach 2:
The shared last-level TLB acts as an intermediary between the TLB and page tables. Instead of directly walking the page tables on every miss, the system first queries the shared TLB structure across nodes, which serves as a intermediate caching layer that reduces the frequency and impact of page-table walks.
3Reliability
If a distributed shared last-level TLB is implemented to reduce miss rates, then TLB capacity is effectively increased, but the system complexity increases
Solution Approach 1:
Each node's last-level TLB structure is designed to serve dual purposes: it functions as a local cache for that node's virtual addresses and simultaneously contributes to the shared last-level TLB for the entire multiprocessor system. This multi-functionality reduces the need for separate dedicated shared TLB hardware.
Solution Approach 2:
The distributed TLB system uses self-service mechanisms where each node independently manages its own last-level TLB entries while automatically participating in the shared TLB functionality. Nodes can autonomously determine whether to service TLB requests locally or forward them to other nodes, reducing the need for centralized control logic.
Data Source
AI summary
The disclosed embodiments provide techniques for reducing address-translation latency and the serialization latency of combined TLB and data cache misses in a coherent shared-memory system. For instance, the last-level TLB structures of two or more multiprocessor nodes can be configured to act together as either a distributed shared last-level TLB or a directory-based shared last-level TLB. Such TLB-sharing techniques increase the total amount of useful translations that are cached by the system, thereby reducing the number of page-table walks and improving performance. Furthermore, a coherent shared-memory system with a shared last-level TLB can be further configured to fuse TLB and cache misses such that some of the latency of data coherence operations is overlapped with address translation and data cache access latencies, thereby further improving the performance of memory operations.