Snapshot Data Structure for Distributed File Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for taking snapshots of data in computer systems are inefficient, particularly in large-capacity storage devices, as they are time-consuming, expensive, and lack system flexibility, and do not effectively manage modifications or preserve previous versions of files and directories.
Innovation Solution
A method and system for taking snapshots of data in a computer system that allows for the creation of a snapshot of a portion of a file system by indicating a path, retrieving relevant data structures, and tracking modifications, enabling per-file or per-directory snapshot capabilities, and preserving data as of a selected time point using a hierarchical structure with snapshot modules.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire contents of the storage device are copied to a backup storage device, then previous versions of data can be preserved and accessed, but the process is time-consuming and delays write requests
Solution Approach 1:
Instead of copying the entire storage device, the system segments the file system into portions (directories, subdirectories, and files) and creates snapshots only of the modified portions. This is achieved by tracking modification times and comparing them against the snapshot creation time, thereby preserving only the necessary data changes rather than the entire file system.
Solution Approach 2:
The system extracts only the modified portions of the file system for inclusion in the backup. By identifying and isolating only the changed directories and files based on modification time tracking, the system creates a minimal representative snapshot that captures previous versions without the bulk of unchanged data.
2Reliability
If the entire contents of the storage device are copied to a backup storage device, then previous versions of data can be preserved, but additional storage space is required
Solution Approach 1:
The backup process segments the file system into only the modified portions that need preservation. By dividing the entire storage device into unchanged and changed sections, the system creates a compact backup that stores only the necessary modified data, significantly reducing the storage space required compared to full copies.
Solution Approach 2:
The system extracts and stores only the modified portions of the file system in the backup. By identifying which directories and files changed since the last snapshot and including only those in the backup, the system minimizes the quantity of stored data while maintaining the ability to restore previous versions.
3Reliability
If traditional backup methods are used, then data can be preserved, but system flexibility is reduced
Solution Approach 1:
The system dynamically determines which portions of the file system need to be backed up by continuously tracking modification times. This dynamic approach allows the backup to adapt to changing data patterns, enabling selective backup of only modified portions rather than rigid full-system copies, thereby enhancing system flexibility.
Solution Approach 2:
The backup system applies different qualities to different portions of the file system. By tracking and backing up only the modified portions (directories and files with modification times greater than the snapshot creation time) while leaving unchanged portions alone, the system creates a flexible, adaptive backup strategy that responds to local data characteristics.
4Quantity of substance
If modern large-capacity storage devices are used, then data storage capacity increases, but the problems of time consumption and expense are amplified
Solution Approach 1:
The system segments the large storage capacity into manageable portions by identifying and backing up only the modified files and directories. This segmentation approach ensures that even with large-capacity storage devices, the backup process remains efficient by processing only the necessary portions rather than the entire large data set.
Solution Approach 2:
The system extracts only the modified portions from the large storage capacity for inclusion in the backup. By using modification time tracking to identify and isolate only the changed data, the system creates efficient snapshots that do not scale linearly with storage capacity, thereby reducing the time and expense associated with backing up large datasets.
Data Source
AI summary
In one embodiment, a user or client device is connected to a distributed file system comprised of one or more physical nodes. The data on each of the physical nodes store metadata about files and directories within the file system. Some of the embodiments permit a user to take a snapshot of data stored on the file system. The snapshot may include a single file, a single directory, a plurality of files within a directory, a plurality of directories, a path on the file system that includes nested files and subdirectories, or more than one path on the file system that each includes nested files and directories. In an embodiment, a snapshot tracking data structure is maintained for efficient creation and deletion of the snapshot.