File Categorization for Faster Incremental Snapshot Scanning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional file management systems waste time generating incremental data snapshots for large filesets with many unchanged files, as they rely on hierarchical relationships, making the scan phase inefficient.
Innovation Solution
Classify files into discontiguous filesets based on factors like frequency of change and purpose, and manage them differently according to their associations, rather than their directory hierarchy, to optimize snapshot processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional hierarchical file management is used, then files are organized by directory structure, but the scan phase time increases linearly with the number of files
Solution Approach 1:
The patent segments the large fileset into multiple smaller filesets based on file attributes and change frequency. Instead of scanning all files in a hierarchical structure, the system divides files into groups (e.g., frequently changed vs. rarely changed files) and processes them separately, reducing the time complexity from linear to more manageable segments.
Solution Approach 2:
The patent applies different scanning and processing strategies to different subsets of files based on their local characteristics. Frequently changed files receive more intensive monitoring while rarely changed files are scanned less frequently, optimizing the overall scan phase time by adapting the processing quality to each file's actual needs.
2Reliability
If incremental data snapshots are generated for all files, then data protection is comprehensive, but processing time is wasted on unchanged files
Solution Approach 1:
The patent implements partial action by generating incremental snapshots only for files that have actually changed, rather than scanning and processing all files in the fileset. The system identifies changed files through efficient metadata comparison and applies snapshot operations only to those specific files, eliminating wasted processing time on unchanged files while maintaining complete data protection.
Solution Approach 2:
The patent changes the parameter of file selection from 'all files in hierarchical structure' to 'files matching specific change criteria'. By using file attributes, modification timestamps, and change frequency as selection parameters, the system efficiently identifies which files require snapshot operations, improving productivity without compromising data protection reliability.
3Reliability
If filesets cover entire root file systems, then complete data protection is achieved, but the large number of files increases processing overhead
Solution Approach 1:
The patent segments the entire root file system into multiple logical filesets based on file attributes, directories, and change patterns. This segmentation reduces processing overhead by allowing the system to manage and process smaller, more manageable groups of files independently while still providing comprehensive data protection coverage across the entire file system.
Solution Approach 2:
The patent introduces dynamic fileset creation and management, where filesets are not static hierarchical structures but dynamically adjusted groups based on actual file change patterns and protection needs. This dynamic approach reduces processing overhead by adapting the fileset structure to match actual data protection requirements rather than forcing all files into a rigid hierarchical framework.
Data Source
AI summary
According to various embodiments, with respect to a target set of files being managed (e.g., protected by data snapshots), each file in the target set of files is classified into one of two or more filesets (discontiguous filesets), where each of these filesets comprises one or more files that are related to each other by one or more factors, such as frequency of file change or purpose of existence (e.g., used by a software application). Once classified, files within the target set of files can be uniquely processed by a data management operation (e.g., incremental data snapshot process) according to their association to a discontiguous fileset.