Unified Dirty Page Tracking via Peripheral DMA Notification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computing systems face challenges in fully tracking memory pages accessed by both software and peripheral devices independently, leading to data inconsistency and suboptimal performance, especially during Virtual Machine migration and page-swapping operations.
Innovation Solution
Implementing a system where a peripheral device uses Direct Memory Access (DMA) to notify the processor of memory pages accessed, allowing the processor to maintain a data structure that tracks both software- and peripheral-device-accessed memory pages, ensuring comprehensive dirty page tracking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If the processor tracks only software-accessed memory pages, then the tracking complexity is reduced, but the data consistency deteriorates because peripheral device accesses are not accounted for
Solution Approach 1:
The patent merges the tracking of software-accessed memory pages and peripheral device-accessed memory pages into a single unified data structure (dirty page bitmap). The processor maintains this combined tracking system, where the same bitmap structure is used to record both types of accesses, eliminating the need for separate tracking mechanisms and ensuring data consistency across all memory access sources.
Solution Approach 2:
The patent introduces an intermediary notification mechanism where the peripheral device sends notifications to the processor about memory pages it has accessed. This intermediary communication allows the processor to update its dirty page tracking with peripheral device accesses without requiring the processor to directly monitor all peripheral operations, thus maintaining reliability while managing complexity.
2Reliability
If the processor tracks all memory pages accessed by both software and peripheral devices, then the data consistency is improved, but the device complexity increases
Solution Approach 1:
The patent combines multiple tracking functions into a single unified dirty page bitmap data structure. Instead of maintaining separate tracking systems for software and peripheral device accesses, the same bitmap is used to record all memory page accesses, simplifying the overall tracking architecture while ensuring complete data consistency.
Solution Approach 2:
The peripheral device performs self-service by autonomously identifying and notifying the processor about the memory pages it has accessed. This self-notification mechanism allows the peripheral device to contribute to the tracking system without requiring complex processor-controlled monitoring, reducing the burden on the processor while maintaining comprehensive tracking.
3Measurement precision
If the peripheral device notifies the processor of each individual memory page access, then the tracking precision is improved, but the loss of time increases due to frequent notifications
Solution Approach 1:
The peripheral device performs preliminary action by pre-identifying and buffering the addresses of memory pages it has accessed before notifying the processor. This preliminary preparation allows the notification to contain aggregated information about multiple accessed pages, reducing the frequency of processor notifications while maintaining precise tracking of all accessed memory pages.
Solution Approach 2:
The patent implements periodic notification where the peripheral device accumulates memory access information over a period and then notifies the processor in batches. This periodic approach reduces the notification frequency compared to notifying after every single access, thereby reducing time loss while still maintaining accurate tracking through the use of the dirty page bitmap to record all accesses.
4Loss of time
If the peripheral device buffers indications in a coalescing buffer, then the loss of time is reduced by batching notifications, but the device complexity increases
Solution Approach 1:
The peripheral device uses the coalescing buffer to perform preliminary action by accumulating and buffering addresses of accessed memory pages before notification. This preliminary buffering allows the device to batch multiple access indications into a single notification event, reducing notification overhead time while the buffer itself manages the complexity of tracking individual accesses.
Data Source
AI summary
A compute node includes a memory, a processor and a peripheral device. The memory is to store memory pages. The processor is to run software that accesses the memory, and to identify one or more first memory pages that are accessed by the software in the memory. The peripheral device is to directly access one or more second memory pages in the memory of the compute node using Direct Memory Access (DMA), and to notify the processor of the second memory pages that are accessed using DMA. The processor is further to maintain a data structure that tracks both (i) the first memory pages as identified by the processor and (ii) the second memory pages as notified by the peripheral device.