Autonomous Storage Tier Relocation for Big Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current multi-tiered storage systems lack intelligence to autonomously determine the optimal storage tier and protection level for data segments, especially in big data applications where large volumes of data require efficient storage and reliable access, such as in the Square Kilometer Array telescope and healthcare industries.
Innovation Solution
A storage system with at least two tiers, an access pattern evaluator, a classification unit, and a selector that assigns relevance classes to data segments based on their content, determines the storage tier and protection level based on access frequency and relevance, and a storage tier relocation manager to dynamically adjust storage locations and protection measures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data segments are stored in multiple storage tiers without intelligent classification, then storage capacity is utilized, but storage efficiency and access performance deteriorate due to lack of optimization
Solution Approach 1:
The storage system autonomously classifies data segments and determines optimal storage tiers without human intervention. The classification unit automatically evaluates data segments, assigns relevance classes, and the selector chooses appropriate storage tiers based on access frequency patterns, enabling the system to self-optimize storage efficiency
Solution Approach 2:
The access pattern evaluator continuously monitors access frequency of data segments and provides feedback to the selector. This feedback loop enables dynamic adjustment of storage tier assignments, improving storage efficiency by placing frequently accessed data in optimal tiers while adapting to changing access patterns over time
2Extent of automation
If manual intervention is used to determine storage tier placement, then storage decisions can be made, but time consumption and operational complexity increase
Solution Approach 1:
The storage system performs autonomous classification and tier selection without requiring manual human intervention. The classification unit automatically processes incoming data segments, and the selector independently determines optimal storage tiers based on evaluated access patterns, eliminating time-consuming manual operations
Solution Approach 2:
The access pattern evaluator pre-establishes access frequency patterns for data segments before final storage placement decisions are made. This preliminary evaluation of access patterns enables the selector to quickly determine optimal storage tiers without time-consuming manual analysis during the actual data placement process
3Reliability
If all data segments are stored with high protection levels, then reliability is improved, but storage capacity and cost increase
Solution Approach 1:
The system applies different protection levels to different data segments based on their individual relevance classes and access patterns. High-value, frequently accessed data segments receive higher protection levels with redundancy, while less critical data segments use lower protection levels, optimizing the balance between reliability and storage capacity utilization
4Speed
If storage tiers are not dynamically adjusted based on access patterns, then system simplicity is maintained, but access performance and retrieval efficiency deteriorate
Solution Approach 1:
The access pattern evaluator continuously monitors data access frequency and provides real-time feedback to the selector. This enables dynamic adjustment of storage tier assignments to optimize access speed - frequently accessed data is automatically placed in faster storage tiers while less accessed data moves to slower, more cost-effective tiers
Solution Approach 2:
The storage system dynamically adjusts data placement across tiers based on changing access patterns rather than using static assignments. As access frequency patterns evolve, the system automatically reclassifies and relocates data segments to appropriate storage tiers, maintaining optimal access performance without requiring manual reconfiguration
Data Source
AI summary
A storage system and a method for storing a data segment, a storage capacity manager and a method for managing a capacity of a storage unit, and a storage tier relocation manager and a method for relocating a data segment. The storage system includes at least two storage tiers, an access pattern evaluator, a classification unit, a selector, and logic. The storage capacitor manager includes a monitoring unit and capacity managing unit. The storage tier relocation manager includes a target storage tier, the data segment relocated to the target storage tier, and a protection measure.