Distributed State Partitioning for Storage Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed data-storage systems face challenges in managing concurrent and simultaneous processing of commands across multiple nodes, leading to high communications overhead, latency, and single-point failures due to the need for centralized state information management.
Innovation Solution
The system partitions state information into local, shared cached, and continuously consistent categories, allowing each node to manage local state independently while maintaining shared state consistency through distributed protocols, such as distributed storage registers and locks, to reduce overhead and ensure high availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If centralized state information management is used in distributed data-storage systems, then coordination of concurrent processing is simplified, but communications overhead and latency increase
Solution Approach 1:
The patent segments state information into local state information (managed independently by each node) and shared state information (managed collectively). This segmentation allows nodes to operate independently on local state without requiring centralized coordination, thereby reducing communications overhead and latency while maintaining ease of operation for local operations.
Solution Approach 2:
The patent introduces a new dimension of state management by creating a hierarchy with local state at the node level and shared state at the system level. This dimensional change allows concurrent processing to be coordinated through shared state updates rather than continuous centralized communication, reducing time loss while maintaining coordination capability.
2Ease of operation
If centralized state information management is used in distributed data-storage systems, then command coordination is simplified, but single-point failures occur
Solution Approach 1:
The patent segments state management authority across multiple nodes through local state information, eliminating the single-point failure vulnerability of centralized management. Each node independently manages its local state, so failure of one node does not cripple the entire system's command coordination capability.
Solution Approach 2:
The patent introduces shared state information as an intermediary mechanism that enables coordinated command execution across distributed nodes without requiring a central authority. This intermediary allows nodes to synchronize their actions through shared state updates while maintaining decentralized operation and avoiding single-point failures.
3Reliability
If state information is distributed across multiple nodes, then reliability and availability improve, but maintaining state consistency becomes complex
Solution Approach 1:
The patent segments state information into local and shared components, where local state can be managed independently without complex consistency protocols. This segmentation reduces the complexity of maintaining state consistency while preserving high availability through distributed local state management.
Solution Approach 2:
The patent applies local quality by allowing each node to manage its local state information with different consistency requirements than shared state. This differentiation reduces overall system complexity by applying appropriate consistency management only where necessary for shared resources, while local operations proceed with simpler management.
Data Source
AI summary
Various embodiments of the present invention are directed to distributed data-storage systems that provide non-distributed-data-storage-device-like interfaces to host computers that access the distributed data-storage systems. Distributed data-storage systems generally rely on state information used for execution of commands, received from host computers, by component data-storage systems. Command-executing component data-storage systems may access and modify state information shared among many or all of the component data-storage systems. Embodiment methods of the present invention provide methods for partitioning state information within a distributed data-storage system as well as communications protocols and methods for managing shared state information among the component data-storage systems within a distributed data-storage system. In certain embodiments of the present invention, state information is partitioned into one or more of: (1) local state information that is managed, accessed, and modified separately by each component data storage system; (2) shared state information that is locally cached on component data storage systems for immediate read access, that is periodically refreshed, but that is kept globally consistent among component data storage systems by distributing state-modifying operations; and (3) continuously consistent, shared state information.