Partial Read Migration for Storage Cost and Access Delay Trade-off
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing file migration methods in data networks face challenges in balancing storage cost and access delay, particularly in policy-based hierarchical storage management, where file access transparency is often compromised, and there is a need for more flexible migration methods to manage data between primary and secondary storage efficiently.
Innovation Solution
The implementation of a file server system that selectively uses offline file read access methods, including partial read migration and pass-through methods, to manage file data migration between primary and secondary storage, allowing for transparent access and optimizing storage costs by enabling partial data migration and immediate access without full data recall.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If full file migration is performed from secondary storage to primary storage, then file access speed is improved, but storage cost increases and access delay increases
Solution Approach 1:
The patent segments the file migration process into two distinct phases: a background migration phase where files are moved from secondary to primary storage, and a foreground access phase where clients read files. This segmentation allows the system to optimize for storage cost during migration while maintaining fast access during read operations, resolving the contradiction between migration completeness and access speed.
Solution Approach 2:
The patent implements preliminary action by performing file migration from secondary storage to primary storage before client access is needed. The migration occurs in the background during system idle time, so when clients need to access files, they can immediately read from the fast primary storage without waiting for migration completion, thus reducing access delay while maintaining storage cost efficiency.
2Quantity of substance
If files are migrated to secondary storage to reduce storage cost, then storage cost is reduced, but file access transparency is compromised
Solution Approach 1:
The patent introduces an intermediary mechanism - a file access cache layer between the client and the storage system. This cache intercepts read requests for migrated files and serves them from primary storage, making the migration transparent to clients. The client simply reads from the cache as if the file were locally stored, maintaining ease of operation while achieving cost reduction through secondary storage.
Solution Approach 2:
The patent creates a copy of the file in primary storage during the migration process. This copy acts as a transparent proxy for the original file in secondary storage. When clients access the file, they read the copy from primary storage without knowing the original resides in secondary storage, thus maintaining access transparency while enabling cost-effective storage.
3Reliability
If complete file data is migrated, then data integrity is improved, but migration time increases and productivity decreases
Solution Approach 1:
The patent applies partial action by migrating only the necessary file data to primary storage during background operations, rather than requiring complete migration before client access. The system can maintain data integrity through checksums and versioning while performing partial migrations, thus improving productivity by reducing migration time and increasing the number of files that can be migrated concurrently.
Data Source
AI summary
A hierarchical storage system is constructed from file servers and a policy engine server by building upon a file migration service. Offline attributes are added to file system inodes in a primary file server, file system parameters are added in the primary server, an offline read access method field is added to a connection database, and the primary file server is programmed to use these attributes and parameters for selecting a particular method (such as a partial read migration or a pass through read method) for read access to an offline file. In this fashion, the primary file server is provided with flexibility for selecting a particular one of a number of migration methods depending on various conditions, in order to improve the trade-off between the cost of storage and delay in file access.