Page Migration Overhead Reduction via Batched TLB Shootdown
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional page migration techniques in non-uniform memory access systems incur substantial overhead due to page copy, cache invalidation, and translation lookaside buffer shootdowns, impacting performance, especially in workloads that frequently migrate memory.
Innovation Solution
A method and apparatus for reduced overhead page migration using a staging buffer to copy pages from higher latency memory to lower latency memory, consolidating multiple TLB shootdowns into a single event, and updating page tables and translation lookaside buffers to associate virtual addresses with physical addresses, allowing continuous access during migration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If conventional page migration techniques are used to move pages between memory levels, then memory access latency is reduced by storing frequently accessed data in lower latency memory, but substantial overhead is incurred due to page copy, cache invalidation, TLB shootdown, and page-table update operations
Solution Approach 1:
The patent merges multiple page migration operations into a single batch operation. Multiple pages are migrated together in one coordinated transaction, combining the page copy, cache invalidation, TLB shootdown, and page-table update operations into a unified process. This reduces the cumulative overhead of performing these operations separately for each page, as the expensive TLB shootdown and cache invalidation operations are performed once for the entire batch rather than repeatedly for individual pages.
Solution Approach 2:
The patent performs preliminary actions by selecting and grouping pages for migration in advance based on access patterns and memory hierarchy state. Pages are identified as candidates for migration before the actual migration operation begins, allowing the system to prepare the migration batch and minimize disruptions to ongoing processor operations. This preliminary selection process enables the migration to be performed during periods of lower activity, reducing the impact on system performance.
2Reliability
If multiple TLB shootdowns are performed for individual page migrations, then address translation consistency is maintained across processors, but processor operation is substantially interrupted
Solution Approach 1:
The patent merges multiple individual TLB shootdown operations into a single batched TLB shootdown operation. Instead of performing TLB shootdown for each page migration separately, the system accumulates multiple page migration requests and performs a single coordinated TLB shootdown that invalidates TLB entries for all pages in the batch simultaneously. This maintains TLB consistency across all processors while minimizing the total time that processor operations are interrupted, as the expensive inter-processor communication and TLB invalidation overhead is amortized over multiple page migrations.
3Productivity
If page migration is performed frequently to optimize memory access patterns, then memory access efficiency is improved, but system performance is substantially impacted by migration overhead
Solution Approach 1:
The patent implements periodic batched page migration operations instead of continuous individual page migrations. The system periodically collects page migration requests that accumulate during normal operation and processes them in batches at designated migration intervals. This periodic approach allows the system to maintain good memory access efficiency by frequently adjusting page placement based on access patterns, while reducing the overall migration overhead by consolidating multiple small migration operations into fewer, larger batch operations. The batching reduces the frequency of expensive TLB shootdown and cache invalidation operations.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A technique for use in a memory system includes swapping a first plurality of pages of a first memory of the memory system with a second plurality of pages of a second memory of the memory system. The first memory has a first latency and the second memory has a second latency. The first latency is less than the second latency. The technique includes updating a page table and triggering a translation lookaside buffer shootdown to associate a virtual address of each of the first plurality of pages with a corresponding physical address in the second memory and to associate a virtual address for each of the second plurality of pages with a corresponding physical address in the first memory.