Resynchronization Mechanism for Distributed System Consistency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The Quorum algorithm in distributed systems faces challenges with split brain issues on the time axis when multiple computers fail simultaneously, leading to inconsistencies in process execution and potential system crashes, as it requires a quorum of operational computers to continue processing, whereas fail-over methods can continue with at least one operational computer.

Innovation Solution

Implementing a resynchronization mechanism where a rebooted processing element determines if it can communicate with at least F+1 synchronized elements, and then resynchronizes its state with the latest execution state held by one of these elements, ensuring consistency across the system without requiring a full quorum immediately.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If the quorum algorithm is used to prevent split brain, then system consistency is improved, but system availability deteriorates when multiple computers fail simultaneously

Engineering Contradiction:

Improvesystem consistencyVSAvoidsystem availability

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent applies preliminary action by having processing elements store their execution states in advance before failures occur. When a failure happens, rebooted processing elements can retrieve these pre-stored execution states and resynchronize without requiring other elements to be operational, thus maintaining system availability while ensuring consistency through the quorum algorithm.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent uses copying by having each processing element maintain a copy of its execution state locally. This allows rebooted elements to restore their state from local copies rather than requiring communication with other elements, resolving the contradiction between maintaining consistency (quorum requirement) and ensuring availability (operating with minimal elements).

Inventive Principle:

Principle #26Copying

2Productivity

If the fail-over method is used to maintain system operation, then system availability is improved, but split brain issues occur leading to system inconsistency

Engineering Contradiction:

Improvesystem availabilityVSAvoidprocess consistency

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent implements feedback by having processing elements continuously monitor and compare execution states with other elements in the quorum. When discrepancies are detected after resynchronization, the system uses feedback mechanisms to identify and resolve inconsistencies, ensuring process consistency while maintaining the high availability benefits of fail-over methods.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent replaces the mechanical heart beat communication mechanism with a state-based resynchronization mechanism. Instead of relying on continuous communication (mechanical system) that can fail during network partitions, the system uses stored execution states that can be restored independently, eliminating split brain issues while maintaining availability.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Difficulty of detecting and measuring

If heart beat communication is used to detect faults, then fault detection capability is improved, but false fault detection occurs under heavy load causing split brain

Engineering Contradiction:

Improvefault detection capabilityVSAvoidfault detection accuracy

Core Design Contradiction:

Difficulty of detecting and measuringVSReliability

Solution Approach 1:

The patent uses cheap short-living objects by implementing temporary resynchronization markers and version indicators that are created and discarded during the resynchronization process. These lightweight objects allow processing elements to verify their state consistency without requiring expensive or complex communication protocols, reducing false fault detection under heavy load while maintaining accurate fault detection capability.

Inventive Principle:

Principle #27Cheap short-living objects (Disposable)

Data Source

PatentUS7620845B2Distributed system and redundancy control method

Publication Date: 2009.11.17 TOSHIBA DIGITAL SOLUTIONS CORP

AI summary

A distributed system using a quorum redundancy method in which a redundancy process is executed by at least Q processing elements of N processing elements communicable with each other, each of N processing elements includes a resynchronization determining unit for determining that an execution state of the processing element itself can be resynchronized with a latest execution state in the distributed system in the case where the processing element can communicate with at least F+1 elements (F=N−Q) already synchronized of the N processing elements at the time of rebooting the processing element, and a resynchronizing unit for resynchronizing the execution state of the processing element itself to the latest one of the execution states of the at least F+1 processing elements in accordance with the result of determination by the resynchronizing unit.