Tiered Cloud Storage Recovery with Self-Describing Data Shards
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional storage systems face inefficiencies in data management and reliability, particularly in flash storage systems, due to unnecessary write operations and lack of centralized control over storage processes, leading to reduced reliability and increased wear on storage devices.
Innovation Solution
Implementing a direct-mapped flash storage system where the operating system directly addresses data blocks without translation by storage controllers, offloading device management tasks, and utilizing non-volatile RAM for quick data buffering and proactive data recovery mechanisms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If storage controllers translate and manage data blocks, then data access control is provided, but device management tasks burden the operating system and reduce storage system reliability
Solution Approach 1:
The patent extracts the data translation function from the storage controller and implements it directly in the operating system. The OS directly addresses flash data blocks without controller translation, removing the controller from the critical data path and eliminating it as a potential failure point while maintaining access control through OS-level management
2Ease of operation
If storage controllers manage device operations, then data access control is provided, but write operations increase device wear and reduce lifespan
Solution Approach 1:
The patent removes the storage controller from the data translation process, allowing the OS to directly access flash blocks. This eliminates unnecessary write amplification caused by controller translation operations and reduces overall write workload on the flash memory, thereby extending device lifespan
Solution Approach 2:
The system performs proactive wear leveling and data redistribution before critical failures occur. The OS monitors flash block wear and proactively redistributes data to balance wear across all blocks, preventing any single block from failing due to excessive writes
3Reliability
If traditional storage systems use redundant writes for reliability, then data integrity is improved, but unnecessary write operations increase device wear
Solution Approach 1:
The patent implements self-healing capabilities where the system automatically detects and recovers from failures without requiring redundant writes. When a flash block fails, the OS automatically redistributes the data to healthy blocks, eliminating the need for pre-configured redundant copies and the associated write overhead
Data Source
AI summary
Data resiliency in a cloud-based storage system, including: receiving, for storage within a first tier of cloud storage of the cloud-based storage system, one or more segments of data; generating, for each of one or more shards of data of the one or more segments of data, self-describing information for recoverability of the one or more shards of data; and storing, within a second tier of cloud storage of the cloud-based storage system, both the one or more shards of data and the generated self-describing information for recoverability of the one or more shards of data.