Reliability-Based Erasure Coding Placement Across SSDs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud-based data storage systems face inefficiencies in data availability due to the arbitrary distribution of erasure coded data fragments and coding fragments across storage devices, which does not consider the varying failure probabilities of SSDs, leading to increased data unavailability and potential loss.
Innovation Solution
The solution involves using storage device failure probability information to strategically distribute data fragments to more reliable SSDs and coding fragments to less reliable SSDs, thereby minimizing data unavailability events and reducing the impact of data loss by prioritizing the loss of coding fragments over data fragments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If erasure coded data fragments and coding fragments are distributed arbitrarily across storage devices, then the system implementation is simple, but data availability deteriorates due to not considering varying failure probabilities of SSDs
Solution Approach 1:
The patent applies local quality by differentiating the treatment of data fragments and coding fragments based on their importance. Data fragments are stored on SSDs with higher reliability (lower failure probability), while coding fragments are stored on SSDs with lower reliability. This non-uniform distribution strategy prioritizes the protection of critical data fragments, thereby improving overall data availability without requiring complex systematic changes to the entire storage architecture.
Solution Approach 2:
The patent changes the distribution parameter from arbitrary to reliability-based by incorporating SSD failure probability information into the placement decision. The system evaluates reliability metrics of storage devices and uses these parameters to determine optimal placement locations for different types of fragments, transforming a simple uniform distribution into a reliability-aware stratified distribution that enhances data availability.
2Reliability
If data fragments are stored on less reliable SSDs to balance load, then storage utilization improves, but data loss probability increases
Solution Approach 1:
The patent applies local quality by assigning different storage locations based on fragment type and SSD reliability characteristics. Critical data fragments are placed on high-reliability SSDs to minimize data loss probability, while less critical coding fragments are placed on lower-reliability SSDs to maintain storage utilization efficiency. This differentiated approach ensures that the most important data is protected while still utilizing available storage capacity effectively.
3Reliability
If coding fragments are updated more often in RAID systems, then redundancy is maintained, but SSD failure probability increases due to more writes
Solution Approach 1:
The patent applies local quality by recognizing that not all fragments require equal protection levels. Data fragments, which are immutable and critical, are stored on SSDs optimized for reliability with minimized write operations. Coding fragments, which require more frequent updates for redundancy maintenance, are stored on separate SSDs that can tolerate higher write frequencies. This spatial separation of concerns reduces the failure probability of SSDs storing critical data while still maintaining redundancy through coding fragment updates.
Data Source
AI summary
The disclosed technology is generally directed towards selecting storage devices, based on predicted reliability, for storing erasure coded data fragments and coding fragments. In general, to increase data availability, data fragments, are stored to more reliable storage devices, while coding fragments are stored to less reliable storage devices. For example, solid state drives (SSDs) tend to fail based on the total number of writes they receive over time, whereby the total number of writes can be used to determine predicted reliability data for an SSD. Before writing the data and coding fragments to a number of storage devices, the storage devices can be sorted based on their predicted reliability such that the data fragments are written to (likely) more reliable devices and coding fragments to less likely storage devices.