Automated File Recall Clustering for Storage Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current Hierarchical Storage Management (HSM) systems face challenges in optimizing file recall for multiple users, leading to increased file reading times and device deterioration due to inefficient data retrieval policies, which require specialized knowledge to adjust and result in either high storage costs or suboptimal content recommendations.
Innovation Solution
An automated recall method that clusters users based on file usage history, selects files frequently used by a predetermined percentage of users within a cluster, and recalls these files from secondary to primary storage, optimizing file access times and reducing storage costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If files are migrated to secondary storage device to reduce storage costs, then storage costs are reduced, but file access time increases and device deterioration occurs
Solution Approach 1:
The system proactively recalls files to primary storage before they are actually needed by users. By analyzing user clusters and their file access patterns, the system predicts which files will be needed and retrieves them in advance from secondary storage, thus eliminating wait time when users actually access the files.
Solution Approach 2:
The system continuously monitors user file access patterns and feedback from cluster analysis to dynamically adjust recall policies. By tracking which files are frequently accessed by which user clusters, the system optimizes recall decisions to balance storage cost savings with user access speed requirements.
2Loss of time
If frequent recall operations are performed to improve file access speed, then file access time is reduced, but secondary storage device deteriorates faster
Solution Approach 1:
The system performs recall operations in advance based on predicted user needs rather than waiting for actual access requests. This spreads out recall operations over time and reduces the frequency of urgent recall operations that would accelerate device wear.
Solution Approach 2:
The system recalls only the specific files that are predicted to be needed by user clusters, rather than recalling all files or using fixed thresholds. This selective approach minimizes unnecessary recall operations that would contribute to device deterioration.
3Loss of energy
If manual policy adjustment is used to optimize HSM parameters, then storage cost optimization is achieved, but system complexity increases and requires specialized knowledge
Solution Approach 1:
The system automatically performs cluster analysis, file pattern recognition, and recall policy optimization without requiring manual intervention. The automated system analyzes user behavior patterns and adjusts recall parameters dynamically, eliminating the need for specialized knowledge to configure and tune HSM parameters.
Solution Approach 2:
The system continuously monitors storage usage patterns, access frequencies, and cost metrics to automatically adjust recall policies. This closed-loop feedback system replaces manual policy adjustment with automated optimization that adapts to changing user behavior and storage conditions.
4Loss of time
If all files are recalled to primary storage to ensure fast access, then file access speed is improved, but storage costs increase significantly
Solution Approach 1:
The system applies different storage strategies to different files based on their access patterns and user cluster characteristics. Frequently accessed files by high-priority clusters are kept in or recalled to primary storage, while less frequently accessed files remain in secondary storage, optimizing the balance between access speed and cost.
Solution Approach 2:
The system dynamically changes recall parameters such as the percentage of files to recall and time windows based on analyzed user patterns. By adjusting these parameters according to actual usage data, the system optimizes the trade-off between having files in fast primary storage versus saving on secondary storage costs.
Data Source
AI summary
In an approach to improve file recall between storage devices by optimizing file recall for multiple users. Embodiments of the present invention obtain one or more similarities among one or more users based on one or more usage points and divide the one or more users into a plurality of clusters based on a content of the one or more usage points. Further, embodiments select a cluster from the plurality of clusters based on cluster importance, and select the content used by more than a predetermined percentage of the weighted users pertaining to the selected cluster as an automated recall subject. Additionally, embodiments recal one or more selected content from a secondary storage device to a primary storage device.