Container-Scalable File Structure for Distributed Cluster Restoration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for protecting data on distributed storage clusters involve duplicating the distributed file system and creating snapshots, which are inefficient and limited in scalability and performance.
Innovation Solution
Implementing a container-scalable file structure that manages backed-up data through container directories with synthesized flat files and distributed metadata across multiple levels, eliminating the need for duplicating the file system and relying on snapshots.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a native copy of the distributed file system is stored on a backup device and point-in-time snapshots are created, then data protection is achieved, but scalability and performance are limited
Solution Approach 1:
The backup system segments data into container directories with flat file structures, dividing the backup process into manageable units that can be independently processed and scaled across multiple storage devices, eliminating the performance limitations of traditional snapshot approaches
Solution Approach 2:
The patent introduces a new dimensional approach by implementing a container directory structure with file-container maps that organize backup data across multiple levels (L1, L2, L3 metadata databases), transforming the traditional single-dimension snapshot model into a multi-dimensional scalable architecture
2Adaptability or versatility
If traditional snapshot methods are used for data protection, then data can be restored to previous states, but the system complexity and storage overhead increase
Solution Approach 1:
Instead of creating complex snapshot copies of the entire file system, the patent uses selective copying of data into container directories with associated file-container maps, maintaining restoration capability while significantly reducing system complexity and storage overhead
Solution Approach 2:
The system performs preliminary organization of backup data into container directories and metadata databases before restoration is needed, enabling efficient data retrieval without requiring complex snapshot management during the restoration process
Data Source
AI summary
A method for restoring files onto distributed storage clusters. The method includes: receiving a restoration request specifying a backup identifier (ID); identifying, within a container-scalable file structure, a backup directory associated with the backup ID; producing a file-container map based on information stored within the backup directory; and submitting, to a distributed workload framework and to service the restoration request, a copy-read request specifying the file-container map.


