Dynamic Remapping of In-Process DMA Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Persistent memory (PM) enabled operating systems with direct access (DAX) mechanisms face challenges in address space management, leading to unbounded waits during device DMA redirection, especially with increasing I/O devices, which negatively impacts system performance.
Innovation Solution
Implementing a 'pause-remap-release' capability in computing platforms with PM, using mechanisms like PCIe bridges to track and throttle traffic, and coordinating IOMMU remap operations, allowing dynamic remapping of DMA operations without requiring new hardware, ensuring globally available platform attributes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the filesystem waits for device DMA to quiesce before remapping, then address space management can be performed safely, but unbounded and indefinite waits occur leading to performance degradation
Solution Approach 1:
The IOMMU pauses DMA operations before remapping is initiated, preventing new DMA transfers from starting while the address space is being modified. This preliminary action ensures that no new DMA operations will conflict with the upcoming remap, allowing safe remapping without waiting for all DMA to complete.
Solution Approach 2:
The IOMMU acts as an intermediary between the DMA device and the memory address space. It intercepts and pauses DMA transfers, coordinates the remapping operation, and then releases the pause to allow DMA to resume. This mediator capability allows the filesystem to perform remapping without directly waiting for DMA quiescence.
2Productivity
If DAX mappings are used to minimize overhead, then direct memory access is achieved, but the ability of the OS to perform address space management operations is compromised
Solution Approach 1:
The IOMMU serves as an intermediary that preserves DAX direct access semantics while enabling address space management. By pausing DMA at the IOMMU level rather than requiring OS buffer cache indirection, the system maintains direct memory access efficiency while the IOMMU coordinates remapping operations when needed.
Solution Approach 2:
The system dynamically pauses and resumes DMA operations through the IOMMU based on address space management needs. This dynamic control allows the system to switch between direct access mode (for performance) and remapping mode (for address space management) as needed, providing both efficiency and adaptability.
3Adaptability or versatility
If RDMA and other protocols keep DMA operations active indefinitely, then application control is maintained, but collisions with address space management operations increase
Solution Approach 1:
The IOMMU provides feedback control by monitoring DMA operations and automatically pausing them when address space management is needed. This feedback mechanism allows indefinite DMA operation control (when not pausing) while preventing collisions during remapping, as the IOMMU detects and pauses conflicting DMA transfers.
Solution Approach 2:
Before address space management operations occur, the IOMMU preliminarily pauses affected DMA operations. This prevents collisions between RDMA/indefinite DMA operations and remapping, allowing the system to maintain both application control over DMA and high system performance by avoiding conflicts.
Data Source
AI summary
Embodiments include a method to determine whether to dynamically remap an in-process update of a first page of memory to a second page of memory. When a dynamic remap is determined, the method causes a pause of the in-process update to the first page of memory by one or more bridges, draining of in-process direct memory access (DMA) operations, and redirecting the update to the second page of memory.