Dual Storage System for Data Migration and Cost Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Content addressable storage systems face high costs for storing large amounts of data, necessitating a solution to reduce storage expenses while maintaining data accessibility and integrity.
Innovation Solution
Implementing a dual storage system architecture with a primary, expensive storage device and a secondary, less expensive storage device, where content is replicated from the primary to the secondary system based on user-defined policies, allowing for data migration and deletion from the primary system to free up space, with a controller determining storage locations and managing access requests.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored on expensive high-performance storage devices, then data accessibility and integrity are maintained, but storage costs increase
Solution Approach 1:
The storage system is segmented into multiple storage devices with different performance characteristics (first storage device with first storage medium and second storage device with second storage medium). Data is divided and stored across these segments based on access patterns, allowing hot data to reside on faster storage while cold data moves to slower, cheaper storage.
Solution Approach 2:
The system changes the storage parameters of data by monitoring access patterns and automatically migrating data between storage devices based on usage frequency. Recently accessed data is maintained on the first storage device, while data not accessed for a threshold period is moved to the second storage device, optimizing both cost and accessibility.
2Quantity of substance
If data is migrated from primary to secondary storage system, then storage costs are reduced, but data accessibility may be impacted
Solution Approach 1:
The system performs preliminary actions by pre-fetching data from the second storage device to the first storage device based on predicted access patterns. The controller monitors access requests and proactively moves data that is likely to be needed soon, ensuring data accessibility is maintained without requiring expensive permanent storage.
Solution Approach 2:
The controller acts as an intermediary between the host and storage devices, managing data migration transparently. It intercepts access requests, determines whether data resides on the first or second storage device, and retrieves data from the appropriate location, masking the complexity from the host system.
3Adaptability or versatility
If a dual storage system architecture is implemented, then storage flexibility and cost optimization are improved, but system complexity increases
Solution Approach 1:
The storage system provides self-service through automated data migration and management. The controller automatically monitors access patterns, determines migration candidates, and executes data movement between storage devices without requiring manual intervention, reducing the operational complexity despite the dual-storage architecture.
4Reliability
If content addressable storage is used, then data integrity is maintained through content addressing, but storage efficiency decreases due to redundancy
Solution Approach 1:
The system uses copying to maintain data integrity by creating and storing content addresses (hashes) of data blocks. These content addresses serve as unique identifiers that enable verification of data integrity without requiring the original data to be stored in a single location, allowing efficient storage while maintaining reliability.
Data Source
AI summary
One embodiment is directed to a computer system that includes primary and secondary storage systems. When a request to store a content unit is received, it may be determined whether to store the content unit on the primary storage system or the secondary storage system. In another embodiment, a computer may store information relating to an action to be performed in one of the directories of a file system that corresponds to a period of time. When the period of time to which the directory corresponds arrives, the action may be performed. In another embodiment, a content unit stored on a primary storage system may be copied to a secondary storage system in a computer system. After the content unit has been copied, the secondary storage system may send a delete request to the primary storage system to delete the content unit from the primary storage system.