Distributed Graph Cancellation with Reversion Logic
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current graph processing technologies lack efficient mechanisms for interrupting and canceling long-running operations without reloading the graph, leading to high latency and separate handling of cancellation and error management, which complicates resource management and user experience.
Innovation Solution
Implementing a distributed handling of operation states with low-latency cancelable primitives and a unified lifecycle management system that allows for seamless cancellation and error handling, enabling fine-grained control over graph processing operations across multiple computers, including automatic rollback of transient states.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If coarse-grained orchestration of graph processing is used, then system simplicity is maintained, but interrupting processing before completion becomes impossible, resulting in high latency
Solution Approach 1:
The patent segments graph processing into fine-grained operations that can be individually interrupted and managed. Each graph operation is divided into discrete units that can be canceled independently, allowing precise control over processing interruption without requiring complete system redesign.
Solution Approach 2:
The patent implements dynamic cancelation capabilities that allow processing to be interrupted at any point during execution. The system transitions from static coarse-grained orchestration to dynamic fine-grained control, enabling real-time interruption and resumption of graph operations based on runtime conditions.
2Ease of operation
If separate handling of cancelation and error management is implemented, then each problem can be solved independently, but dual maintenance of source codes is required, increasing complexity
Solution Approach 1:
The patent merges cancelation and error management into a unified lifecycle framework. Both functions are handled through the same exception-based mechanism and state transition system, eliminating the need for separate code maintenance paths while providing consistent handling for both scenarios.
Solution Approach 2:
The patent creates a universal exception handling mechanism that serves multiple purposes: cancelation, error management, and state recovery. This multi-functional approach allows a single code path to handle diverse operational failures and interruptions, reducing maintenance burden while improving ease of operation.
3Reliability
If graph processing is canceled without reversion logic, then immediate cancellation is achieved, but graph consistency is compromised
Solution Approach 1:
The patent applies preliminary action by registering reversion logic before graph operations modify the graph state. This pre-registration ensures that when cancelation occurs, the recovery mechanism is already in place and can execute immediately, maintaining graph consistency without adding cancellation latency.
Solution Approach 2:
The patent implements beforehand cushioning by preparing reversion logic in advance that can compensate for any modifications made during graph processing. This preemptive preparation acts as a cushion against potential consistency violations, ensuring that even if operations are interrupted at any point, the graph can be restored to its original state without time loss.
Data Source
AI summary
In an embodiment, a computer of a cluster of computers receives graph logic that specifies a sequence of invocations, including a current invocation and a next invocation, of parallelism operations that can detect whether the graph logic should prematurely terminate. The computer initiates, on the computers of the cluster, execution of the graph logic to process a distributed graph. Before the current invocation, the graph logic registers reversion logic for a modification of the distributed graph that execution of the graph logic has caused. During the current invocation, it is detected that the graph logic should prematurely terminate. Execution of the graph logic on the cluster is terminated without performing the next invocation in the sequence of invocations. The reversion logic reverses the modification of the distributed graph to restore consistency. The distributed graph is retained in volatile memory of the cluster for reuse such as relaunch of the graph logic.