Dynamic Storage Assignment for Scalable Fault Tolerance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face challenges in scaling fault tolerance incrementally, managing redundancy across subsystems, and efficiently handling failures in large collections of disks, particularly in small systems and non-disruptive migration to new hardware, with existing algorithms having limitations in scalability and efficiency.
Innovation Solution
A method for organizing a storage system that groups storage elements, constructs a storage assignment table to manage redundant data, and updates the table dynamically to accommodate changes in capacity and failure scenarios, using erasure resilient codes and hierarchical storage systems to ensure fault tolerance and efficient recovery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If RAID-based storage systems increase capacity by adding subsystems with internal RAID redundancy, then storage capacity increases, but system reliability decreases due to additional fallible subsystems
Solution Approach 1:
The system segments storage into independent failure domains (subsystems) where each subsystem's failure does not propagate to others. By organizing storage elements into separate subsystems with independent failure modes, the system can scale capacity while containing reliability risks within isolated segments.
Solution Approach 2:
The patent introduces an intermediate layer of redundancy management that coordinates across subsystems. This intermediary mechanism (the storage assignment table and redundancy management system) enables fault tolerance to span multiple subsystems without requiring them to be tightly coupled, thus maintaining reliability while scaling capacity.
2Reliability
If redundancy is provided by RAID running across subsystems, then fault tolerance across subsystems improves, but incremental scaling becomes difficult due to rigid correspondence of data components
Solution Approach 1:
The system employs dynamic storage assignment where the mapping between data components and storage elements can change as subsystems are added or removed. The storage assignment table is regenerated to reflect current system configuration, enabling flexible incremental scaling while maintaining fault tolerance through adaptive redundancy distribution.
Solution Approach 2:
The patent changes the parameter of storage assignment from fixed rigid correspondence to flexible dynamic mapping. By altering how data components are assigned to storage elements based on current system state, the system achieves both fault tolerance and scalability, allowing subsystems to be added or removed without rigid constraints.
3Adaptability or versatility
If randomized storage assignment algorithms like Chord are used, then scalability to large numbers of servers improves, but reliability for small numbers of servers deteriorates due to statistical guarantees failing
Solution Approach 1:
The system performs preliminary deterministic assignment of redundant copies to different storage elements before data is stored. By pre-establishing the storage assignment table with guaranteed separation of redundant copies, the system ensures reliability for small numbers of servers while maintaining scalability through the same deterministic algorithm that works for any system size.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method for organising a storage system that is scalable and fault tolerant, the method including grouping together a number D of storage elements to form the storage system, where D is more than one, constructing a storage assignment table that includes table entries, computing, for each of the storage elements, an available capacity that depends on constraints on the placement of redundant data within the storage system, summing the available capacities to form a total available capacity for the storage system; and assigning the table entries in the storage assignment table to each identify one of the storage elements, wherein the available capacity of each of the storage elements is its effective storage capacity when used as part of the storage system, wherein not all of the D storage elements that form the storage system have the same available capacity, and wherein the fraction of all table entries that identify a one of the storage elements depends upon its fraction, of the total available capacity. A method for encoding a block, of data to allow it to be stored or transmitted correctly in the face of accidental or deliberate modifications, the method including constructing a numbern greater than one of original components, each of which is derived from the block and each of which is smaller than the block, and combining original components to construct a number m greater than one of new components, wherein each of the new components is smaller than the sum of the sizes of the original components combined to produce it. wherein the block can be reconstructed from any set of n different components selected from the original components and new components, and wherein a. set ofn different components selected from the original components, and new components contains more redundant information about the block than the set of n original components.