Distributed Computing Checkpointing for Node Failure Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed computing systems face challenges in resuming computation after node failure without introducing expensive redundancy, as existing solutions require restarting from initial data or duplicating tasks, leading to increased resource usage and loss of internal variables.
Innovation Solution
A method that employs a multi-level acknowledgement system and checkpointing technique to store node states and acknowledge message delivery, allowing computation to resume from the last saved checkpoint without redundant computation, using a configurable acknowledgement level (N) to ensure robustness and flexibility.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional node failure recovery techniques are used (instantiation of new topology, restarting computation from beginning, or redundant computation), then node failure can be handled, but resource consumption (network I/O, CPU, memory) increases significantly
Solution Approach 1:
The patent implements preliminary action by maintaining N-level acknowledgments and checkpoint information in advance during normal operation. When a node fails, the system can immediately restore computation state from the most recent checkpoint without restarting from the beginning, because the necessary state information was preserved beforehand through the acknowledgment mechanism.
Solution Approach 2:
The patent changes the parameter of computation state management by introducing a configurable N-level acknowledgment system. Instead of traditional single-acknowledgment or full-restart approaches, the system tracks acknowledgment levels across N nodes, allowing flexible restoration to different checkpoint states based on failure patterns, thereby optimizing resource usage during recovery.
2Reliability
If redundant computation techniques are applied (active or passive standby nodes), then computation can resume after node failure, but the number of required nodes and resources increases
Solution Approach 1:
The patent applies universality by making each node multi-functional through the N-level acknowledgment system. Instead of dedicating specific nodes as standby replicas, any node in the network can potentially take over computation tasks by utilizing the distributed checkpoint information maintained through acknowledgments, thereby reducing the need for specialized redundant nodes.
Solution Approach 2:
The patent uses copying by maintaining copies of computation state information across multiple nodes through the acknowledgment mechanism. Rather than creating full redundant node instances, the system copies and distributes essential state information (checkpoints) through the N-level acknowledgment process, allowing any node to restore computation without requiring complete node duplication.
3Reliability
If upstream backup technique is used to maintain message relations, then node failure can be recovered, but internal variables are lost and computation must be recomputed
Solution Approach 1:
The patent implements feedback by establishing an N-level acknowledgment system where each node provides feedback about message receipt and processing state. This feedback mechanism allows the system to track which nodes have successfully received and processed messages, preserving internal variable state information through the acknowledgment chain rather than losing it during failure recovery.
Solution Approach 2:
The patent applies preliminary action by pre-establishing the N-level acknowledgment structure and maintaining checkpoint information before failures occur. This preliminary setup ensures that internal variable states are preserved in the acknowledgment chain, allowing computation to resume with existing state information rather than requiring complete recomputation after failure.
Data Source
AI summary
A method for operating a distributed computing system includes transmitting, by a first node in a first level to an intermediate node in a second level, a first message and storing, by the first node, the first message. The method further includes receiving, by the intermediate node, the first message, generating a second message, and transmitting, to a third node, the second message. The method additionally includes receiving, by the intermediate node from the third node, a first acknowledgement message indicating that the second message was successfully received by the third node, and transmitting, by the intermediate node to the first node after receiving the first acknowledgement message, a second acknowledgement message. The method includes deleting, by the first node in response to receiving the second acknowledgment message, the stored first message.