Cluster Fencing via Cryptographic Access Keys
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In scalable data storage systems, particularly in cloud storage platforms, there is a lack of effective mechanisms to 'fence off' cluster components that have lost communication, leading to data inconsistency and resource ownership issues, as traditional SCSI reservations are not applicable to scaled-out systems.
Innovation Solution
A system generates a new secret access key based on a generation identifier from cluster membership messages, which is used to create signing keys for validating requests to shared storage, effectively relocating resources and restricting access from non-communicating nodes, ensuring data consistency and availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional SCSI reservations are used for fencing off cluster components, then resource ownership can be maintained in traditional storage systems, but this approach is not applicable to scaled-out cloud storage systems
Solution Approach 1:
The patent replaces the mechanical SCSI reservation protocol with a cryptographic signing key mechanism. Instead of using SCSI reservation codes that work for traditional storage systems, the system generates signing keys based on cluster membership generation identifiers. This substitution allows the fencing mechanism to work in cloud storage environments where SCSI reservations are not applicable, while maintaining data consistency through cryptographic validation of access requests.
Solution Approach 2:
The patent changes the fundamental parameter of the fencing mechanism from SCSI reservation codes to generation identifier-based signing keys. By deriving signing keys from generation identifiers that change with each cluster membership change, the system adapts the fencing approach to work with dynamic cloud storage environments. This parameter change enables the system to maintain reliable data consistency while being adaptable to scaled-out architectures.
2Reliability
If cluster nodes continue to access shared storage after losing communication, then storage availability is maintained, but data inconsistency occurs due to multiple owners
Solution Approach 1:
The patent implements preliminary action by generating and distributing new signing keys to all active cluster members before a node is completely fenced off. When cluster membership changes, the system proactively creates new generation identifiers and derives new signing keys from them. This preliminary key generation ensures that when a node loses communication, it automatically cannot access shared storage because its old signing keys are no longer valid, thus preventing data inconsistency while maintaining storage availability through seamless key transitions.
Solution Approach 2:
The patent uses feedback mechanisms where cluster members monitor each other's communication status. When a node fails to respond or loses communication, the cluster membership service detects this through feedback from other nodes and triggers the generation of new signing keys. This feedback-driven approach ensures that only actively communicating nodes can access shared storage, maintaining data consistency while preserving availability through automatic detection and response to communication failures.
3Measurement precision
If direct comparison of data segments is used for deduplication, then accurate duplicate detection is achieved, but system resources are wasted due to comparing large amounts of data
Solution Approach 1:
The patent applies segmentation by dividing data segments into smaller units for comparison and using hierarchical deduplication. Instead of comparing entire large data segments directly, the system breaks them into smaller chunks, generates fingerprints for each chunk, and performs comparisons at this finer granularity. This segmentation enables accurate duplicate detection while reducing the computational resources needed, as comparing small fingerprint values is much more efficient than comparing large data segments.
Solution Approach 2:
The patent uses copying by creating and storing fingerprint representations of data segments instead of storing and comparing the actual data segments. The system generates cryptographic fingerprints (hash values) that uniquely represent each data segment and stores these fingerprints in a catalog. When deduplication is needed, the system compares these compact fingerprint copies rather than the original large data segments, achieving accurate duplicate detection with minimal system resource consumption.
Data Source
AI summary
A fencing service receives a cluster message based on a first service's removal from membership in a first node in a cluster, and generates an access key based on a generation identifier in the cluster message. The fencing service sends the access key to a shared storage, thereby enabling the shared storage to create signing keys based on the access key, and which are used to validate signed requests to access resources stored by the shared storage. A second service in a second node in the cluster receives the cluster message, and generates the access key based on the generation identifier. The second service accesses a resource previously accessed by the first service, by generating a signing key based on the access key and a request to access the resource stored by the shared storage, and then sending the request signed by the signing key to the shared storage.