Incremental Scan Using Directory Extended Attributes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing methods for file migration in data storage systems are resource-intensive and time-consuming, especially as storage systems grow large, due to the need for scanning and comparing file attributes against user-defined policies, which can lead to excessive file operations and system resource drainage.
Innovation Solution
The implementation of an incremental scan method that utilizes extended attributes to determine which directories and files need to be scanned, reducing unnecessary file operations by checking directory extended attributes such as NextTentativeScanDate, ModifiedTime, and the Dirty flag to identify files qualifying for migration, thereby optimizing the scanning process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a full scan of entire primary storage is performed to retrieve all files and read attributes, then migration policy compliance is ensured, but scanning time and system resources increase significantly
Solution Approach 1:
The system performs preliminary actions by maintaining extended attributes (NextTentativeScanDate, ModifiedTime, Dirty flag) that are updated during previous scans. These pre-computed attributes eliminate the need to re-scan entire directories from scratch, allowing the system to skip to only the necessary file checking steps during incremental scans.
Solution Approach 2:
The scanning process is segmented into multiple levels: directory-level filtering using extended attributes (NextTentativeScanDate, ModifiedTime, Dirty flag) and file-level scanning only for directories that pass the preliminary filter. This segmentation allows the system to avoid scanning all files in the entire storage system, reducing overall scanning time while maintaining policy compliance.
2Reliability
If all files are scanned and attributes are read to check migration criteria, then policy accuracy is maintained, but system resources are excessively consumed
Solution Approach 1:
The system extracts and utilizes specific extended attributes (NextTentativeScanDate, ModifiedTime, Dirty flag) from directory metadata to determine scan eligibility. By taking out only the necessary filtering information rather than scanning all file attributes, the system maintains policy accuracy for migrated files while dramatically reducing the computational resources required for the scanning process.
Solution Approach 2:
The system applies partial action by scanning only the subset of files and directories that meet the criteria defined by extended attributes, rather than performing a complete scan of all files. This partial scanning approach maintains sufficient policy accuracy for migration decisions while consuming significantly fewer system resources.
3Productivity
If incremental scan using extended attributes is implemented, then scanning efficiency is improved, but system complexity increases due to extended attribute management
Solution Approach 1:
The extended attributes structure serves multiple functions simultaneously: it stores scan scheduling information (NextTentativeScanDate), modification tracking (ModifiedTime), and scan status indicators (Dirty flag). By making the attribute structure multi-functional, the system achieves high scanning efficiency through incremental scanning while avoiding the need for separate complex management systems for each attribute type.
Data Source
AI summary
This disclosure relates to a method, article of manufacture, and apparatus for scanning data. In some embodiments, this includes examining attributes of a first directory, wherein the attributes include at least one from a group comprising a next tentative scan date, a modified time, and a dirty indicator, determining if the dirty indicator has been set, if the dirty has been set, scanning files in the first directory, if the dirty indicator has not been set, determining if the first directory has been modified, if the first directory has not been modified, determining if the next tentative scan date equals to a system date, if the next tentative scan date equals to the system date, scanning the files, if the next tentative scan date does not equal to the system date, examining attributes of a second directory, and if the first directory has been modified, scanning the files.