Burst Buffer Appliance Virtual Machine Checkpoint Prioritization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High-performance computer systems face challenges in ensuring that checkpoints are properly written to cluster file systems without interfering with other IO operations, often leading to dropped checkpoints due to conflicting priorities among different types of IO operations.
Innovation Solution
A burst buffer appliance is implemented between the high-performance computer system and the cluster file system, utilizing a high-speed memory and multiple virtual machines to prioritize and coordinate IO operations, providing different quality of service levels for various types of operations to prevent interference and ensure checkpoint integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If checkpoints are written to the cluster file system with high priority, then checkpoint reliability is improved, but other IO operations are disrupted and throughput decreases
Solution Approach 1:
The system segments IO operations into different priority levels using virtual machines. Checkpoint writes are assigned to high-priority virtual machines while other IO operations use lower-priority virtual machines, allowing simultaneous execution without mutual interference. This segmentation resolves the contradiction by enabling reliable checkpointing while maintaining overall IO throughput through parallel processing at different priority levels.
Solution Approach 2:
The patent introduces a burst buffer appliance as an intermediary layer between the computer system and the cluster file system. This intermediary absorbs and buffers IO operations, allowing checkpoints to be written reliably while smoothing out traffic to the file system. The burst buffer acts as a mediator that prevents direct conflicts between high-priority checkpoint operations and other IO operations, maintaining both reliability and throughput.
2Productivity
If multiple types of IO operations share the same file system access, then resource utilization is improved, but conflicting priorities cause checkpoints to be dropped
Solution Approach 1:
The system dynamically assigns different priority levels to various IO operations based on their importance. Virtual machines can adjust their priority levels dynamically, allowing checkpoint operations to receive higher priority when needed while other operations use lower priority. This dynamic priority assignment enables the system to maintain high resource utilization while ensuring checkpoint integrity by adapting priorities to current operational needs.
Solution Approach 2:
The patent applies local quality by providing different quality of service levels to different virtual machines based on their specific needs. Checkpoint-related virtual machines receive high-priority access with guaranteed bandwidth and low latency, while other virtual machines operate at lower priority levels. This localized quality differentiation allows the system to optimize resource allocation for critical operations while maintaining overall resource utilization.
Data Source
AI summary
A burst buffer appliance is adapted for coupling between a computer system and a file system. The burst buffer appliance comprises a flash memory or other high-speed memory having a substantially lower access time than the file system, and is configured to include a plurality of virtual machines for processing respective different types of input-output operations that involve utilization of the high-speed memory, with each of the virtual machines providing a different performance level for its associated type of input-output operations. The performance levels provided by the plurality of virtual machines may comprise respective different quality of service (QoS) levels for the respective different types of input-output operations, specified in terms of parameters such as latency and throughput rate. A highest QoS level may be provided by a particular virtual machine for operations involving writing checkpoints from the computer system to the high-speed memory.


