Elastic Cloud Storage Recovery Without Restoring Deleted Fragments
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In cloud-based data storage systems, erasure coding methods often restore and store useless data fragments, wasting resources and storage capacity, especially in scenarios with high delete rates where missing or corrupt fragments contain only deleted user objects.
Innovation Solution
A method that analyzes unavailable data fragments to determine if they contain only useless data, using predetermined fake data in erasure coding processes to generate new coding fragments without recovering useless data, thereby protecting live data without wasting storage capacity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional erasure coding methods are used to restore missing data fragments, then data integrity is maintained, but useless data fragments are restored and stored wasting storage capacity and computational resources
Solution Approach 1:
The system performs preliminary analysis of unavailable data fragments before initiating full recovery. By checking metadata and determining in advance whether fragments contain only deleted objects, the system avoids wasting resources on recovering useless data while maintaining integrity protection for live data through selective recovery
2Reliability
If all missing data fragments are recovered using erasure coding, then complete data protection is achieved, but computational resources are wasted on generating and storing useless coding fragments
Solution Approach 1:
The system extracts and identifies useless data fragments through metadata analysis before applying erasure coding recovery. By separating useful live data from useless deleted data, the system applies computational resources only to recovering and protecting fragments that contain live data, eliminating waste on fragments containing only deleted objects
3Reliability
If conventional erasure coding is applied without analysis, then all data fragments are protected uniformly, but storage capacity is wasted on protecting useless data
Solution Approach 1:
The system applies different protection strategies to different data fragments based on their content analysis. Fragments containing live data receive full erasure coding protection, while fragments containing only deleted objects are identified and excluded from recovery operations, optimizing storage capacity utilization while maintaining protection where needed
Data Source
AI summary
A method for avoiding recovery and storage of useless data is disclosed. The operations comprise: determining that a subset of data fragments out of a plurality of data fragments have become unavailable, the plurality of data fragments forming a data chunk, wherein the plurality of data fragments are protected by additional redundancy data; determining whether all of data that were stored in the unavailable subset of data fragments when the unavailable subset of data fragments were available would be useless if recovered using remaining available data fragments of the data chunk and the redundancy data; and in response to determining that all of the data that were stored in the unavailable subset of data fragments would be useless if recovered, generating new redundancy data for protecting the remaining available data fragments of the data chunk without recovering the data that were stored in the unavailable subset of data fragments.