Burst Buffer Appliance Virtual Machine Checkpoint Prioritization

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

High-performance computer systems face challenges in ensuring that checkpoints are properly written to cluster file systems without interfering with other IO operations, often leading to dropped checkpoints due to conflicting priorities among different types of IO operations.

Innovation Solution

A burst buffer appliance is implemented between the high-performance computer system and the cluster file system, utilizing a high-speed memory and multiple virtual machines to prioritize and coordinate IO operations, providing different quality of service levels for various types of operations to prevent interference and ensure checkpoint integrity.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If checkpoints are written to the cluster file system with high priority, then checkpoint reliability is improved, but other IO operations are disrupted and throughput decreases

Engineering Contradiction:
Improvecheckpoint reliabilityVSAvoidIO throughput
Core Design Contradiction:
ReliabilityVSProductivity

Solution Approach 1:

The system segments IO operations into different priority levels using virtual machines. Checkpoint writes are assigned to high-priority virtual machines while other IO operations use lower-priority virtual machines, allowing simultaneous execution without mutual interference. This segmentation resolves the contradiction by enabling reliable checkpointing while maintaining overall IO throughput through parallel processing at different priority levels.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent introduces a burst buffer appliance as an intermediary layer between the computer system and the cluster file system. This intermediary absorbs and buffers IO operations, allowing checkpoints to be written reliably while smoothing out traffic to the file system. The burst buffer acts as a mediator that prevents direct conflicts between high-priority checkpoint operations and other IO operations, maintaining both reliability and throughput.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Productivity

If multiple types of IO operations share the same file system access, then resource utilization is improved, but conflicting priorities cause checkpoints to be dropped

Engineering Contradiction:
Improveresource utilizationVSAvoidcheckpoint integrity
Core Design Contradiction:
ProductivityVSReliability

Solution Approach 1:

The system dynamically assigns different priority levels to various IO operations based on their importance. Virtual machines can adjust their priority levels dynamically, allowing checkpoint operations to receive higher priority when needed while other operations use lower priority. This dynamic priority assignment enables the system to maintain high resource utilization while ensuring checkpoint integrity by adapting priorities to current operational needs.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The patent applies local quality by providing different quality of service levels to different virtual machines based on their specific needs. Checkpoint-related virtual machines receive high-priority access with guaranteed bandwidth and low latency, while other virtual machines operate at lower priority levels. This localized quality differentiation allows the system to optimize resource allocation for critical operations while maintaining overall resource utilization.

Inventive Principle:
Principle #3Local quality

Data Source

PatentUS9690728B1Burst buffer appliance comprising multiple virtual machines
Publication Date: 2017.06.27 EMC IP HLDG CO LLC
  • US9690728B1 patent drawing
  • US9690728B1 patent drawing
  • US9690728B1 patent drawing

AI summary

A burst buffer appliance is adapted for coupling between a computer system and a file system. The burst buffer appliance comprises a flash memory or other high-speed memory having a substantially lower access time than the file system, and is configured to include a plurality of virtual machines for processing respective different types of input-output operations that involve utilization of the high-speed memory, with each of the virtual machines providing a different performance level for its associated type of input-output operations. The performance levels provided by the plurality of virtual machines may comprise respective different quality of service (QoS) levels for the respective different types of input-output operations, specified in terms of parameters such as latency and throughput rate. A highest QoS level may be provided by a particular virtual machine for operations involving writing checkpoints from the computer system to the high-speed memory.