Distributed Node Mapping via Local State Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional distributed systems face downtime and reliability issues when nodes fail, as they require cluster-wide synchronization to maintain item-to-node mapping, leading to undesirable downtime and increased costs with larger clusters.
Innovation Solution
Each node maintains locally-stored system-state information to determine item-to-node mappings independently, allowing for node failure detection and propagation without cluster-wide synchronization, using techniques like bitmap representation and piggybacking of state information to ensure consistent mappings and avoid split-brain scenarios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If cluster-wide synchronization is used to maintain item-to-node mapping, then mapping consistency is improved, but system downtime increases
Solution Approach 1:
The patent segments the synchronization process by allowing each node to maintain its own locally-stored system-state information independently, rather than requiring all nodes to synchronize simultaneously. Each node autonomously determines item-to-node mappings based on its local state, eliminating the need for cluster-wide synchronization and thus avoiding downtime.
Solution Approach 2:
The patent implements preliminary action by having nodes continuously maintain and update their locally-stored system-state information in advance. When a node failure occurs, nodes already have the necessary state information cached locally, allowing them to immediately determine new mappings without waiting for synchronization messages from other nodes.
2Productivity
If larger clusters are deployed to increase capacity, then productivity is improved, but system complexity increases
Solution Approach 1:
The patent segments the mapping determination process so that each node independently manages its own mapping decisions based on locally-stored state information. This segmentation eliminates the need for complex centralized coordination or global synchronization protocols, allowing the system to scale to larger clusters without proportionally increasing operational complexity.
Solution Approach 2:
Each node performs self-service by autonomously determining item-to-node mappings based on its own locally-stored system-state information. Nodes independently detect failures and update their mappings without requiring coordination with other nodes, significantly reducing the complexity of managing large-scale clusters.
3Speed
If nodes operate independently without synchronization, then operational speed is improved, but mapping consistency deteriorates
Solution Approach 1:
The patent ensures mapping consistency through preliminary action by having nodes continuously maintain and update their locally-stored system-state information. Nodes proactively track the state of all nodes in the cluster and store this information locally, so that when mapping decisions are needed, consistent mappings can be determined immediately without requiring real-time synchronization with other nodes.
Solution Approach 2:
The patent implements feedback mechanisms where nodes continuously receive and process updates about system state changes. When a node detects a failure or state change, it uses this feedback information to update its locally-stored state and accordingly adjust mappings, ensuring consistency across the distributed system without requiring explicit synchronization messages.
Data Source
AI summary
A method and apparatus for maintaining an item-to-node mapping among nodes in a distributed cluster is provided. Each node maintains locally-stored system-state information indicating that node's understanding of which master nodes are alive and dead. Instead of employing a global item-to-node mapping, each node acts upon a locally determined mapping based on its locally-stored system-state information. For any two nodes with the same locally-stored system-state information, the locally determined mapping is the same. A node updates its locally-stored system-state information upon detecting a node failure or receiving a message from another node indicating different locally-stored system-state information. The new locally-stored system-state information is transmitted on a need-to-know basis, and consequently nodes with different item-to-node mappings may operate concurrently. Mechanisms to avoid nodes assuming conflicting ownership of items are employed, thus allowing node failures to propagate via asynchronous messaging instead of requiring a cluster-wide synchronization event.