Predictive File Storage Policy for Faster Distributed Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current distributed file systems face significant time consumption and performance issues during file storage and access due to the time-consuming process of accessing metadata and varying file value over different nodes, leading to inefficient storage and access speeds.
Innovation Solution
A file management method that utilizes an attribute prediction model to predict file size and life cycle, enabling efficient storage path and policy determination, reducing metadata access and improving storage and read speeds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If metadata information of each file is queried to measure file value in distributed storage, then file storage policy can be determined, but time consumption increases significantly
Solution Approach 1:
The system performs preliminary actions by predicting file attributes (size, life cycle) before actual file storage operations. The attribute prediction model pre-calculates these values based on storage path and file content characteristics, so that when files need to be stored or migrated, the system already has the necessary attribute information without needing to query metadata during the operation, thus reducing time consumption while maintaining measurement precision
Solution Approach 2:
The patent introduces an attribute prediction model as an intermediary between file storage and metadata querying. This model acts as a mediator that estimates file attributes based on storage path and content features, replacing the need for direct metadata queries. The intermediary provides predicted attribute values that enable file value measurement and storage policy determination without the time penalty of actual metadata access
2Reliability
If file attributes are determined through metadata querying, then accurate storage policy can be established, but storage and reading speed decreases
Solution Approach 1:
The system performs preliminary attribute prediction before file storage operations. By calculating file size and life cycle attributes in advance using the prediction model, the system establishes storage policies beforehand rather than determining them during file operations. This preliminary action ensures policy accuracy while avoiding the speed penalty of real-time metadata queries during storage and reading operations
Solution Approach 2:
The patent segments the file management process into two independent parts: attribute prediction (performed once before storage) and file operations (storage and reading). The attribute prediction model separately calculates file attributes based on storage path and content characteristics, decoupling this from the actual file I/O operations. This segmentation allows storage policy determination to not interfere with file operation speed, maintaining both reliability and performance
3Adaptability or versatility
If files are migrated between distributed systems, then system flexibility is improved, but time consumption increases due to attribute detection
Solution Approach 1:
The system performs preliminary attribute prediction before file migration operations. When files need to be migrated between distributed systems, the attribute prediction model has already calculated the necessary file attributes (size, life cycle) based on storage path and content characteristics. This preliminary action provides the migration system with ready-to-use attribute information, enabling flexible migration decisions without the time consumption of detecting attributes during the migration process itself
Data Source
AI summary
A file management method includes obtaining a storage path of a file; obtaining an attribute of the file through calculation based on the storage path and an attribute prediction model; and determining a storage policy for the file based on the attribute.