Container-Scalable Backup Structure for Distributed Storage Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for protecting data in distributed storage clusters involve duplicating the distributed file system and relying on snapshots, which are inefficient and limited in scalability and performance.
Innovation Solution
Implementing a container-scalable file structure that manages backed-up data through container directories with synthesized flat files and distributed metadata across multiple levels, eliminating the need for duplicating the distributed file system and using snapshots.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional snapshot-based backup methods are used, then data protection is provided, but scalability and performance are limited
Solution Approach 1:
The backup system segments data into container directories organized by file types and access patterns. Each container directory can be independently managed, backed up, and restored, allowing parallel processing and improved backup efficiency while maintaining complete data protection across all segments.
Solution Approach 2:
The patent introduces a multi-level directory structure with container directories as an intermediate layer between traditional file systems and backup storage. This dimensional addition enables hierarchical data organization that improves both scalability (by allowing unlimited container creation) and performance (by enabling selective backup operations on specific container levels).
2Reliability
If distributed file system duplication is used, then data protection is achieved, but system complexity increases
Solution Approach 1:
Instead of duplicating the entire distributed file system infrastructure, the patent creates simplified container directory copies that reference the original data through metadata. This copying approach maintains data protection by preserving data access paths while significantly reducing system complexity by eliminating the need to replicate complex file system semantics.
3Reliability
If traditional backup structures are used, then data is protected, but access performance deteriorates
Solution Approach 1:
The system performs preliminary organization of data into container directories during the backup process, pre-categorizing files by type, access frequency, and other criteria. This preliminary action enables faster data retrieval during restore operations by allowing direct access to specific container directories without scanning entire backup structures, thus improving access performance while maintaining data protection.
Data Source
AI summary
A method for protecting files of distributed storage clusters. The method includes: receiving a full backup request specifying a source directory including at least one file sought to be protected; creating, within a container-scalable file structure, a backup directory to be associated with a full backup operation servicing the full backup request; producing a file-container map reflecting an assignment of the at least one file to at least one container directory under the backup directory; and submitting, to a distributed workload framework and to perform the full backup operation, a full copy-write request specifying the source directory and the file-container map.