File System Slicing for NAS Data Protection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data backup systems for network attached storage (NAS) devices, such as array snapshots, NDMP protocol, and file system crawlers, face limitations in scalability, processing time, and vendor-specific workarounds, particularly when dealing with large-scale file systems, leading to inefficiencies in data recovery and backup processes.
Innovation Solution
The implementation of an intelligent file system slicer algorithm that breaks down file systems into manageable slices, utilizing real-time slicing techniques across multiple threads and backup agents to maximize throughput and reduce processing time, allowing for parallel processing and efficient data backup operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If file system crawlers are used to traverse millions and billions of files over network connection, then data backup can be achieved, but processing time becomes excessively long
Solution Approach 1:
The patent divides the file system into multiple slices or partitions based on metadata structures (directories, files, snapshots). Each slice can be processed independently and in parallel by multiple backup agents, significantly reducing the total processing time while maintaining complete data backup coverage.
Solution Approach 2:
The system performs preliminary actions by first traversing and analyzing the file system metadata to identify and create slices before the actual backup process. This preliminary slicing phase enables subsequent parallel backup operations to proceed much faster than sequential crawling would allow.
2Reliability
If array snapshots are used for data protection, then point-in-time recovery is achieved, but vendor-specific proprietary formats limit restoration flexibility
Solution Approach 1:
The patent creates a universal backup system that can restore data to multiple different device types and models, not just the original source device. The backup agents and slice-based architecture enable cross-vendor and cross-platform restoration capabilities, making the backup system versatile and adaptable to various restoration scenarios.
3Productivity
If NDMP protocol is used for data transport between NAS devices and backup devices, then backup speed is enhanced and backup server load is reduced, but parallel capabilities are limited and scalability to hundreds of terabytes is restricted
Solution Approach 1:
The patent segments the backup workload into multiple independent slices that can be processed in parallel by multiple backup agents simultaneously. This segmentation approach overcomes the single-stream limitation of NDMP and enables linear scaling with the number of available backup agents, supporting hundreds of terabytes and beyond.
Solution Approach 2:
The system transitions from single-dimensional sequential backup to multi-dimensional parallel processing by introducing multiple backup agents working simultaneously on different slices. This dimensional expansion enables the system to scale to large capacities while maintaining high backup speeds.
4Productivity
If file system crawlers run in parallel with multiple threads, then throughput is maximized, but a first pass is required to determine system layout adding to processing time
Solution Approach 1:
The patent performs the system layout analysis as a preliminary action during the slicing phase, using metadata traversal to identify directories, files, and snapshots. This one-time preliminary pass enables subsequent parallel backup operations to proceed efficiently without requiring repeated layout determination, reducing overall processing time while maintaining high throughput.
Data Source
AI summary
Embodiments are described for storing array snapshots of a file system in networked attached storage (NAS) device. A system and process determine an organization of the file system comprising directories and files in a defined hierarchy, and a size of each file and a number of files in each directory. One or more slicer processes slice the file system into a number of slices, wherein each slice contains at least one of: a defined number of files or files having a defined total size. Backup agents are deployed as proxies to back up the slices to the NAS device, and the slices are backed up as array snapshots in the NAS device.