Bulk Attribute Retrieval for File System Metadata Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current tiered storage solutions face inefficiencies in managing file system meta-data, leading to inappropriate storage usage and high resource utilization due to the time-consuming and resource-intensive process of populating and maintaining the Information Manager's database, which affects the accuracy and performance of file migration operations.
Innovation Solution
The implementation of a Bulk Attribute Retrieval (BAR) method that allows for the collection of file system object attributes through a single request, reducing network and CPU overhead by selectively retrieving only necessary attributes and using filters to narrow the results, thereby optimizing resource utilization and database maintenance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional NAS crawl operations are used to collect file system attributes, then complete attribute information can be retrieved, but network and CPU resource utilization becomes prohibitively high as the file system grows
Solution Approach 1:
The patent extracts and retrieves only the specific file system attributes that are necessary for the Information Manager's database, rather than collecting all possible attributes through traditional NAS crawl operations. This selective extraction approach reduces network and CPU resource utilization while maintaining the completeness of required attribute information.
Solution Approach 2:
The patent implements partial action by retrieving a subset of attributes that are sufficient for file migration decision-making, rather than performing exhaustive attribute collection. The Bulk Attribute Retrieval operation collects only the attributes needed for the Information Manager to function effectively, avoiding the excessive resource consumption of complete file system scans.
2Loss of information
If multiple NAS operations are performed to collect different attribute types, then comprehensive attribute data can be obtained, but the overhead of TCP and NAS protocol processing increases significantly
Solution Approach 1:
The patent merges multiple separate NAS attribute retrieval operations into a single Bulk Attribute Retrieval operation. This consolidation reduces the number of TCP connections and NAS protocol processing events required, thereby decreasing protocol overhead while still collecting comprehensive attribute data needed for file migration decisions.
3Reliability
If the Information Manager database is periodically synchronized with the file system, then data accuracy is maintained, but storage system performance is impacted during update operations
Solution Approach 1:
The patent implements periodic synchronization of the Information Manager database with the file system, where attribute information is collected and updated at scheduled intervals rather than continuously. This periodic approach maintains database accuracy while minimizing the performance impact on the storage system, as updates occur in controlled time windows rather than constantly interfering with normal storage operations.
Data Source
AI summary
Accordingly a method and interface allows an attribute data base used by an Information Manager to be quickly populated and accurately maintained. A single Bulk Attribute Retrieval Request triggers the primary storage device to collect object attribute information. The method allows for selective collection of objects and attributes by providing filters and attribute lists in the Requests. The Request may be used to provide an incremental scan with appropriate time stamp filtering. In addition, the size of the results can be controlled by the IM by eliminating attributes that are not of interest to the IM. The Request is advantageously issued over a FileMover interface, which is an HTTP connection, and encoded in XML, allowing the IM to easily customize the Request as desired.