Distributed Storage Group Segmentation for Overlapping Failure Isolation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed storage systems, when the number of failing storage nodes exceeds the data protection level, data becomes non-recoverable, leading to a high probability of data loss due to overlapping failures.
Innovation Solution
The system organizes data blocks into groups, ensuring that multiple data blocks for the same data are stored within the same group, allowing for localized failure management and reducing the probability of overlapping failures by allocating storage nodes based on group management information and metadata tables.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data blocks are randomly distributed across all storage nodes in the storage pool, then storage capacity and scalability are improved, but the probability of overlapping failures exceeding the data protection level increases
Solution Approach 1:
The storage pool is segmented into multiple groups, each containing a specific number of storage nodes. Data is divided into data blocks and redundant blocks, which are then distributed to storage nodes within the same group. This segmentation isolates failures to specific groups, preventing system-wide data loss even when multiple nodes fail simultaneously.
Solution Approach 2:
Each group is designed with specific local characteristics - a defined number of storage nodes that work together as a unit. The data protection level is configured at the group level, creating localized redundancy zones. This allows the system to maintain high reliability within each group while preserving overall storage capacity across the entire pool.
2Productivity
If the number of storage nodes in a group is increased to handle more data blocks, then data distribution capability is improved, but the probability of simultaneous failures exceeding protection level increases
Solution Approach 1:
Rather than creating one large group with many storage nodes, the system segments the storage pool into multiple smaller groups. Each group contains a controlled number of nodes that matches the data protection requirements. This segmentation allows the system to scale by adding more groups rather than enlarging individual groups, maintaining reliability while increasing overall capacity.
Solution Approach 2:
The system transitions from a single-dimension approach (one large group) to a multi-dimensional approach (multiple smaller groups arranged in a group pool). This dimensional change allows independent management of each group's size and composition, enabling the system to accommodate more storage nodes overall while keeping each group's failure probability controlled.
Data Source
AI summary
A distributed storage system includes a plurality of storage nodes including: a storage device for storing data in such a way that the data can be written thereto and read therefrom; a memory in which a software program is recorded; and a CPU for executing the software program. The memory stores group management information in which a group configured with a plurality of storage nodes and the storage nodes that configure the group are associated with each other and recorded. The CPU converts data into a plurality of data blocks so that the data is redundant at a predetermined data protection level, and stores the data blocks into each of a plurality of storage nodes belonging to the same group based on the group management information.