Fractional Consistent Global Snapshots of Distributed Namespace
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Full global snapshots of distributed file systems are inefficient as they persist the entire namespace, leading to unnecessary resource usage and network traffic, and can fail if any storage nodes are unavailable, despite many applications only needing a subset of data.
Innovation Solution
Fractional consistent snapshots are created by identifying a subset of data through file set identifiers and only snapshotting available storage nodes, allowing for reduced resource usage and improved reliability by continuing operations on non-participating nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If full global snapshots are taken of the entire distributed namespace, then complete data consistency is achieved, but resource usage and network traffic increase unnecessarily
Solution Approach 1:
The patent divides the distributed namespace into separate file sets, allowing snapshots to be taken of only the required subsets rather than the entire namespace. This segmentation enables selective snapshotting that reduces resource consumption while maintaining consistency for the specific file sets needed.
Solution Approach 2:
The patent implements partial action by taking snapshots of only the necessary file sets rather than the complete namespace. This partial snapshot approach provides sufficient data consistency for applications that only need specific portions of the distributed file system, avoiding the excessive resource usage of full namespace snapshots.
2Reliability
If full global snapshots include all storage nodes, then complete namespace coverage is achieved, but the snapshot fails if any storage node is unavailable
Solution Approach 1:
By dividing the namespace into independent file sets, the patent allows snapshot operations to proceed on available file sets even when some storage nodes are unavailable. Each file set can be snapshotted independently, so failures in one area do not prevent snapshots of other areas.
Solution Approach 2:
The patent accepts partial snapshot completion by capturing only the file sets that are available and accessible. This partial action approach ensures that snapshots can be completed even when full namespace coverage is not achievable due to node unavailability, improving overall snapshot reliability.
3Reliability
If snapshots capture the entire namespace, then comprehensive data protection is provided, but network bandwidth is consumed unnecessarily
Solution Approach 1:
The patent segments the distributed namespace into discrete file sets, enabling snapshots to capture only the specific file sets that require protection. This selective approach reduces network bandwidth consumption by transmitting and storing only the necessary data portions rather than the entire namespace.
Solution Approach 2:
The patent implements partial data protection by capturing snapshots of only the required file sets rather than the complete namespace. This provides sufficient protection for applications that need specific data while minimizing network bandwidth consumption associated with transferring and storing snapshot data.
Data Source
AI summary
A method of capturing a fractional snapshot of a distributed file system includes receiving a snapshot request to generate a factional snapshot of a file system. The request includes an indication of a subset of data of the file system. The method further includes identifying, by a processing device, the subset of data of the file system indicated by the snapshot request and generating the fractional snapshot comprising the subset of data of the file system indicated by the snapshot request.