Optimistic Locking for Distributed Search Storage Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed computing systems, coordinating write operations to shared storage is challenging due to race conditions that can lead to data loss, especially when a leader node goes down and later attempts to overwrite or corrupt data, and there is no mechanism to prevent misbehaving or stalled servers from locking resources.
Innovation Solution
Implementing optimistic locking and a creative file naming strategy within a coordination system to add transactional and consistency semantics to non-transactional shared storage systems, ensuring that healthy servers are isolated from misbehaving ones and preventing data corruption by using a coordinator to manage versioning and conditional updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a leader node is elected to control writes in a distributed system, then write coordination is improved, but the system becomes vulnerable to data corruption when the leader node fails and recovers
Solution Approach 1:
The system performs preliminary actions by having the coordination system record write operations and version information before actual writes occur. When a node fails and recovers, the preliminary recorded state allows the system to detect and prevent the recovered node from performing harmful write operations, thus preventing data corruption before it can occur.
Solution Approach 2:
A coordination system acts as an intermediary between distributed nodes and the shared storage. This intermediary maintains version information and coordinates write operations, preventing direct uncoordinated writes from recovered nodes. The intermediary validates and manages write operations to ensure consistency even when leader nodes fail and recover.
2Reliability
If optimistic locking is implemented in non-transactional shared storage, then transactional semantics are improved, but system complexity increases
Solution Approach 1:
The patent replaces complex mechanical locking mechanisms with a simpler software-based optimistic locking approach using version numbers and conditional updates. Instead of using heavy-weight locks and coordination protocols, the system uses lightweight version metadata that can be checked and updated atomically, reducing system complexity while maintaining transactional semantics.
Solution Approach 2:
The system changes the parameter approach by using version numbers as a key parameter for tracking data state. Each write operation includes a version number, and the coordination system uses these version parameters to validate and coordinate updates. This parameter-based approach is simpler than traditional locking mechanisms while providing equivalent transactional guarantees.
3Reliability
If versioning and conditional updates are used to prevent race conditions, then data consistency is improved, but write operation overhead increases
Solution Approach 1:
The system performs preliminary actions by recording version information and write operations in the coordination system before actual writes. This preliminary recording allows for efficient validation during write operations, as the version checks are based on pre-recorded state information rather than requiring complex real-time coordination, thus reducing write operation overhead.
Data Source
AI summary
System and methods are described for a server computing node including a processing device and a memory device. The processing device executes instructions stored in the memory device to receive a request to update a data object in a shared storage of a distributed computing system; get a current version identifier (ID) of the data object; get the data object identified by the current version ID from the shared storage; update the data object; write the updated data object to shared storage using a unique new version ID; and cause the conditional update of the current version ID of the data object to the new version ID.