Storage System Ownership Logic for Data Access Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional Hierarchical Storage Management (HSM) systems suffer from high latency issues when accessing data stored on slower media like magnetic tape, making them unsuitable for high-performance applications such as remote storage digital video recorders (RS-DVRs) that require fast data retrieval.
Innovation Solution
Implementing a storage system with logic that determines ownership status, location, and access frequency of files across multiple storage tiers, using deduplicated replication techniques, unique identifiers, and 'data-less' file movement strategies to optimize data access and reduce latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If standard HSM systems are used to store data on slower media like magnetic tape, then storage cost is reduced, but data access latency increases significantly
Solution Approach 1:
The system pre-loads data from slower storage media (magnetic tape) into faster storage tiers (hard disk drives) before actual access is needed. This preliminary action ensures that when data is requested, it is already available in fast storage, eliminating the latency penalty of accessing tape directly while still maintaining the cost benefits of tape storage for archival data.
Solution Approach 2:
The patent introduces an intermediary storage tier (hard disk drives) between the slowest storage medium (magnetic tape) and the user access point. This intermediary layer acts as a buffer that can be pre-filled with data from tape, providing fast access to frequently requested data while maintaining the economic advantages of tape storage for less frequently accessed data.
2Speed
If data is moved quickly from tape to disk for high performance access, then data retrieval speed improves, but system complexity increases
Solution Approach 1:
The system performs preliminary data movement from tape to disk storage tiers in advance of actual user requests. This allows the complex operation of data migration to be scheduled and executed during off-peak times, rather than being triggered by each user request, thereby reducing the apparent complexity of the system while maintaining high data retrieval speeds.
Solution Approach 2:
The storage system automatically manages its own data migration between tiers based on predefined policies and access patterns, without requiring manual intervention or complex real-time decision-making. The system self-regulates by monitoring data access patterns and autonomously moving data between tape and disk tiers, simplifying the overall system architecture.
3Adaptability or versatility
If multiple copies of the same data are stored on different storage tiers, then data access flexibility improves, but storage efficiency decreases
Solution Approach 1:
The system creates copies of data only when necessary to satisfy specific access requirements, rather than maintaining multiple copies as a general strategy. Data is replicated from tape to disk tiers selectively based on access patterns and performance requirements, ensuring that copying occurs only when it provides tangible benefits while minimizing unnecessary storage consumption.
Solution Approach 2:
Different portions of the storage system are assigned different qualities based on local access requirements. Frequently accessed data is maintained in fast disk storage, while less frequently accessed data resides on slower tape storage. This local differentiation of storage quality allows the system to optimize for both access flexibility and storage efficiency by matching storage characteristics to actual usage patterns.
Data Source
AI summary
A storage system according to one embodiment includes logic integrated with and/or executable by a hardware processor, the logic being configured to: determine an ownership status for each of a plurality of instances of a file in a first storage tier and in a second storage tier of the storage system; determine locations of the instances of the file in the storage system; determine whether the instances of the file in the first storage tier are being accessed or not being accessed; and assign each of the instances of the file to one of a plurality of indices using the determined ownership status, location, and whether the instances are being accessed.