Cloud File Tiering via Metadata Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Managing cloud-based storage tiers efficiently is challenging due to varying capabilities and prices, and existing methods like crawling the file system are costly and inefficient.
Innovation Solution
Utilizing file system meta-data to determine file characteristics and assign files to appropriate tiers, reducing the need for file transfers and network usage by only transferring meta-data, which minimizes costs and optimizes tier placement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If crawling the file system is used to determine file characteristics, then accurate file information can be obtained, but network usage and costs increase significantly
Solution Approach 1:
The patent extracts only the necessary metadata information from the file system without performing a complete crawl of all files. By taking out only the essential file characteristics stored in metadata (such as file size, modification time, access patterns), the system achieves accurate file information while avoiding the network overhead of transferring entire files or comprehensive file system structures.
Solution Approach 2:
The system performs preliminary actions by maintaining and utilizing metadata that is already available in the file system. Instead of crawling files when needed, the metadata is prepared and maintained in advance, allowing quick assessment of file characteristics for tiering decisions without triggering expensive network operations.
2Measurement precision
If files are transferred to local systems for analysis, then detailed file characteristics can be obtained, but transfer costs and network usage increase
Solution Approach 1:
The patent uses copies of file information in the form of metadata rather than transferring actual file contents. The metadata serves as a lightweight copy that contains all necessary characteristics for tiering decisions, eliminating the need to transfer large file data while maintaining the ability to make accurate placement decisions.
3Ease of operation
If manual tier management is performed, then control over file placement can be achieved, but time and operational complexity increase
Solution Approach 1:
The system implements self-service by automatically analyzing metadata and making tiering decisions without requiring manual intervention. The automated tiering engine evaluates file characteristics from metadata and assigns files to appropriate storage tiers autonomously, reducing both operational complexity and management time while maintaining effective control over file placement.
Solution Approach 2:
The system incorporates feedback mechanisms where tiering decisions are continuously evaluated and adjusted based on actual file access patterns and performance metrics. This feedback loop enables the system to automatically optimize tier placement over time, reducing the need for manual adjustments while improving storage efficiency.
4Device complexity
If cloud storage tiers are not optimized, then simpler management is possible, but storage costs increase
Solution Approach 1:
The system changes key parameters related to storage management by dynamically adjusting file tier placement based on metadata analysis. By monitoring and responding to changes in file characteristics (such as access frequency, file size, modification patterns), the system automatically optimizes storage tier selection to reduce costs without requiring complex manual management configurations.
Data Source
AI summary
In accordance with one or more embodiments of the invention, a method is provided for re-tiering files and folders located in a cloud-based file system without crawling all the files and folders in the cloud-based file system. The method utilizes the meta-data for the files and folders to determine, based upon provided rules, which tier a file or folder should be assigned to. By utilizing the meta-data, the use of expensive cloud resources may be minimized when there is a need for re-tiering the files and folders into appropriate cloud tiers.