Dynamic Partition Remapping for Storage Node Failures
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud-based database systems face disruptions due to storage node failures, which can lead to data unavailability and service interruptions, especially in multi-tenant environments where data is distributed across multiple failure domains.
Innovation Solution
A method and apparatus for dynamically and proactively remapping partitions of a distributed volume from a failed storage node to a complementary node within a different failure domain, ensuring seamless data access and minimal disruption by transmitting updated mappings to compute nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is distributed across multiple failure domains to improve reliability, then data availability is improved, but system complexity increases due to the need for dynamic remapping and domain management
Solution Approach 1:
The storage system is segmented into multiple failure domains, with each domain containing a subset of storage nodes. This segmentation allows the system to isolate failures to specific domains while maintaining accessibility to data in other domains, thereby improving reliability without requiring complete system redundancy
Solution Approach 2:
A mapping layer is introduced as an intermediary between compute nodes and storage nodes. This mapping layer dynamically tracks partition locations across failure domains and handles remapping operations when node failures occur, abstracting the complexity from compute nodes while maintaining high data availability
2Reliability
If partitions are remapped from failed nodes to ensure data access, then data recovery speed is improved, but service disruption increases during the remapping process
Solution Approach 1:
The system performs preliminary actions by pre-establishing the mapping layer that tracks all partition locations across failure domains before failures occur. When node failures are detected, the mapping layer can immediately initiate remapping operations without needing to discover or analyze the failure pattern, significantly reducing service disruption time
Solution Approach 2:
The partition mapping is made dynamic rather than static, allowing automatic remapping of partitions from failed nodes to healthy nodes within the same or different failure domains. This dynamic adaptation enables the system to respond to failures in real-time, improving data recovery speed while minimizing service disruption through automated operations
Data Source
AI summary
Software defined storage service (SDS) provides users with remote data volumes spread across multiple storage nodes across multiple failure domains. A distributed volume may be spread across replicas (e.g., failure domains), each replica having a number of partitions stored on storage nodes associated with a particular failure domain. In the event of a node failure, a partition stored on the failed node may be dynamically moved and remapped to another node in the same failure domain or within another failure domain that is different from a failure domain that includes a partition that is complementary to the partition stored on the failed node. The partition move and remapping may be transparent to a user. A partition move may occur while a distributed volume is in use by a client device or in an idle (e.g., offline) mode.