Broadcast TLB Sharing via Optical Interconnect

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Shared-memory multiprocessor systems face significant latency due to address translation, particularly exacerbated by growing data-set sizes and increased hardware threads, leading to higher TLB miss rates and prolonged page-table walks.

Innovation Solution

Implementing a broadcast-based TLB sharing system across nodes connected by an optical interconnect, where a node with a TLB miss broadcasts a request parallel to initiating a speculative page-table walk, canceling it if a response is received from another node caching the translation, thus reducing the need for multiple DRAM accesses.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If multiple levels of TLB are used to cache page-table translations, then address translation speed is improved, but TLB pressure increases leading to higher miss rates

Engineering Contradiction:

Improveaddress translation speedVSAvoidTLB hit rate

Core Design Contradiction:

SpeedVSReliability

Solution Approach 1:

The patent merges TLB resources across multiple processor nodes by implementing a shared TLB system where any node can satisfy another node's TLB request. This combines the caching capacity of all nodes into a unified virtual TLB, increasing the effective hit rate while maintaining fast translation speeds.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent introduces an intermediary mechanism (the shared memory system and interconnect network) that mediates between nodes experiencing TLB misses and nodes that have the required translations cached. This intermediary allows indirect access to TLB entries across nodes, effectively expanding each node's TLB capacity.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If a page-table walk is initiated to resolve TLB misses, then the translation is eventually obtained, but hundreds of clock cycles are consumed

Engineering Contradiction:

Improvetranslation completionVSAvoidaddress translation latency

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent implements preliminary action by proactively broadcasting TLB requests to all nodes simultaneously when a miss occurs, rather than sequentially searching. This allows the system to prepare for potential hits across multiple nodes in parallel, significantly reducing the time to resolve misses compared to traditional sequential page-table walks.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent uses copying by replicating the TLB request to all other nodes in the system simultaneously through the interconnect network. This parallel copying of the request enables any node with the translation to respond, dramatically reducing the average time to resolve a TLB miss compared to sequential searching.

Inventive Principle:

Principle #26Copying

3Loss of information

If DRAM accesses are performed during page-table walks, then translations are retrieved, but the latency of hundreds of clock cycles is incurred

Engineering Contradiction:

Improvetranslation information retrievalVSAvoidDRAM access latency

Core Design Contradiction:

Loss of informationVSLoss of time

Solution Approach 1:

The patent extracts the translation lookup operation from the slow DRAM-based page-table walk path and relocates it to the faster interconnect network path. By checking other nodes' TLBs through the high-speed optical interconnect before falling back to DRAM page-table walks, the system removes the bottleneck of sequential DRAM accesses for common translation cases.

Inventive Principle:

Principle #2Taking out (Extraction)

4Speed

If TLB entries are cached locally at each node, then access speed is improved, but the system cannot utilize translations cached at other nodes

Engineering Contradiction:

ImproveTLB access speedVSAvoidshared translation utilization

Core Design Contradiction:

SpeedVSAdaptability or versatility

Solution Approach 1:

The patent implements universality by making the TLB system multi-functional: each node maintains its local TLB for fast access, but the system as a whole functions as a shared TLB resource pool. The interconnect network enables any node to access translations cached at any other node, making the entire system's caching capacity available to all nodes universally.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS9235529B2Using broadcast-based TLB sharing to reduce address-translation latency in a shared-memory system with optical interconnect

Publication Date: 2016.01.12 ORACLE INT CORP

AI summary

The disclosed embodiments provide a system that uses broadcast-based TLB sharing to reduce address-translation latency in a shared-memory multiprocessor system with two or more nodes that are connected by an optical interconnect. During operation, a first node receives a memory operation that includes a virtual address. Upon determining that one or more TLB levels of the first node will miss for the virtual address, the first node uses the optical interconnect to broadcast a TLB request to one or more additional nodes of the shared-memory multiprocessor in parallel with scheduling a speculative page-table walk for the virtual address. If the first node receives a TLB entry from another node of the shared-memory multiprocessor via the optical interconnect in response to the TLB request, the first node cancels the speculative page-table walk. Otherwise, if no response is received, the first node instead waits for the completion of the page-table walk.