Networked IOMMU Architecture for Memory Access Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional graphics processing systems face increased latency due to a single IOMMU handling memory access requests from multiple devices, including GPUs and peripheral devices, which cannot be positioned near all devices, leading to processing delays and increased round trip times, making it difficult to meet strict latency requirements for real-time-dependent devices.
Innovation Solution
A networked IOMMU system comprising a primary IOMMU and multiple secondary IOMMUs, where the primary IOMMU performs address translations for GPUs and bypasses translations for peripheral devices, forwarding requests to secondary IOMMUs disposed proximate to these devices, each with its own TLB and page tables, to reduce latency based on device-specific requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single IOMMU handles memory access requests from multiple devices, then device complexity is reduced, but latency increases due to processing delays and increased round trip times
Solution Approach 1:
The IOMMU system is segmented into a primary IOMMU and multiple secondary IOMMUs. The primary IOMMU handles address translations for devices that can be positioned near it (e.g., GPUs), while secondary IOMMUs are positioned near peripheral devices to handle their address translations. This segmentation allows the system to reduce latency for time-critical devices without significantly increasing overall system complexity.
Solution Approach 2:
Different IOMMU units are positioned locally near different device types based on their specific latency requirements. Secondary IOMMUs are disposed proximate to peripheral devices with strict latency requirements, while the primary IOMMU serves devices that can tolerate longer access times. This local quality approach optimizes performance for critical devices while maintaining simplicity for the overall system.
2Loss of time
If multiple secondary IOMMUs are deployed proximate to peripheral devices, then latency is reduced, but device complexity increases
Solution Approach 1:
The primary IOMMU acts as an intermediary between the simplified programming model and the distributed secondary IOMMUs. Software interacts with the primary IOMMU using a unified interface, while the primary IOMMU internally manages and coordinates the multiple secondary IOMMUs. This intermediary approach allows the system to achieve low latency through distributed architecture while maintaining programming simplicity through a unified interface.
3Ease of operation
If a single IOMMU is used, then programming model simplicity is maintained, but hard access deadlines cannot be met for real-time-dependent devices
Solution Approach 1:
The IOMMU functionality is segmented between primary and secondary units, where secondary IOMMUs are strategically positioned near real-time-dependent devices to ensure access deadline compliance. The segmentation allows critical devices to have dedicated local IOMMU support while non-critical devices share the primary IOMMU, thus meeting reliability requirements without unnecessary complexity.
Solution Approach 2:
Local quality is applied by providing different levels of IOMMU support to different device types. Real-time-dependent peripheral devices receive dedicated secondary IOMMU support with guaranteed access deadlines, while other devices use the shared primary IOMMU. This differentiated approach ensures reliability for critical devices while maintaining overall system simplicity.
4Device complexity
If address translation is performed at the primary IOMMU for all devices, then system complexity is reduced, but round trip time increases
Solution Approach 1:
The address translation functionality is extracted from the centralized primary IOMMU and placed in secondary IOMMUs that are positioned near peripheral devices. This extraction eliminates the need for memory access requests to travel to and from the primary IOMMU for devices with strict latency requirements, significantly reducing round trip time while maintaining manageable system complexity through a modular architecture.
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
A networked input/output memory management unit (IOMMU) includes a plurality of IOMMUs. The networked IOMMU receives a memory access request that includes a domain physical address generated by a first address translation layer. The networked IOMMU selectively translates the domain physical address into a physical address in a system memory using one of the plurality of IOMMUs that is selected based on a type of a device that generated the memory access request. In some cases, the networked IOMMU is connected to a graphics processing unit (GPU), at least one peripheral device, and the memory. The networked IOMMU includes a command queue to receive the memory access requests, a primary IOMMU to selectively translate the domain physical address in memory access requests from the GPU, and a secondary IOMMU to translate the domain physical address in memory requests from the peripheral device.