Distributed Checkpointing Using NVRAM Across Nodes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional checkpointing methods in multi-node systems face high latency and scalability issues due to reliance on disk-based I/O storage, which can be exacerbated by the introduction of additional nodes through burst buffers.
Innovation Solution
The implementation of distributed checkpointing that utilizes available non-volatile random access memories (NVRAMs) across multiple nodes to store and retrieve application state data, dynamically allocating memory space as needed to reduce I/O traffic and enhance system performance and scalability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If disk-based I/O storage units are used for checkpointing, then data can be stored reliably, but latency increases due to I/O traffic and bandwidth limitations
Solution Approach 1:
The patent extracts the checkpointing data from the traditional disk-based I/O storage path and places it directly into NVRAM resources within the computing nodes. This removes the data from the slow I/O storage subsystem while maintaining reliable storage through the non-volatile nature of NVRAM, thereby resolving the contradiction between reliability and latency.
Solution Approach 2:
The patent introduces NVRAM resources as an intermediary between the computing nodes and the disk-based storage system. This intermediary layer provides fast, reliable storage for checkpoint data while the underlying disk system continues to provide long-term persistence, thus reducing latency without sacrificing reliability.
2Speed
If burst buffers are introduced between nodes and disk-based storage, then caching speed improves, but system complexity increases and scalability is impacted
Solution Approach 1:
The patent makes the NVRAM resources within computing nodes serve dual purposes: they function as both the primary storage for checkpoint data and as the burst buffer caching layer. This eliminates the need for separate burst buffer hardware, reducing system complexity while maintaining high caching speed through the non-volatile memory's fast access characteristics.
Solution Approach 2:
The computing nodes themselves provide the burst buffer functionality by utilizing their own NVRAM resources for checkpointing operations. This self-service approach eliminates the need for additional dedicated burst buffer nodes, thereby reducing overall system complexity while maintaining fast caching performance.
3Ease of operation
If checkpoint memory space is allocated in a single node, then access is simple, but availability decreases if that node fails
Solution Approach 1:
The patent segments the checkpoint memory space allocation across multiple computing nodes rather than concentrating it in a single node. Each node maintains portions of the checkpoint data in its local NVRAM, and the system tracks the distribution. This segmentation provides fault tolerance through redundancy while maintaining operational simplicity through automated management of the distributed segments.
Solution Approach 2:
The patent implements local quality by allowing each computing node to maintain checkpoint data in its own local NVRAM resources. This local storage approach simplifies access for nodes that need their own checkpoint data while the distributed nature across multiple nodes with different local qualities (available memory space) ensures overall system availability and fault tolerance.
Data Source
AI summary
Methods and apparatus presented herein provide distributed checkpointing in a multi-node system, such as a network of servers in a data center. When checkpointing of application state data is needed in a node, the methods and apparatus determine whether checkpoint memory space is available in the node for checkpointing the application state data. If not enough checkpoint memory space is available in the node, the methods and apparatus request and find additional checkpoint memory space from other nodes in the system. In this manner, the methods and apparatus can checkpoint the application state data into available checkpoint memory spaces distributed among a plurality of nodes. This allows for high bandwidth and low latency checkpointing operations in the multi-node system.