Dynamic Erasure Code Redundancy for Tiered Storage Conditions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional tiered archive systems use pre-defined configuration settings to statically determine erasure code redundancy policies, which fail to adapt to dynamically changing conditions such as device availability and optimal redundancy needs.
Innovation Solution
The system dynamically identifies and configures optimal erasure code redundancy policies based on current system conditions, using user-defined or automated rules to determine the N/M redundancy policy for storing and distributing erasure codes across devices in a tiered archive system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If pre-defined configuration settings are used to statically determine erasure code redundancy policies, then system simplicity is maintained, but adaptability to changing conditions deteriorates
Solution Approach 1:
The patent implements dynamic redundancy policies that automatically adjust erasure code parameters based on real-time system conditions such as device availability, data access patterns, and storage capacity. The system transitions from static pre-defined configurations to dynamic adaptive policies that respond to changing environmental factors, thereby resolving the contradiction between adaptability and complexity.
Solution Approach 2:
The system incorporates feedback mechanisms that continuously monitor system state and use this information to adjust erasure code redundancy parameters. By implementing closed-loop control where system performance and conditions are fed back to modify redundancy policies, the system achieves adaptability without requiring complex manual configuration.
2Reliability
If optimal erasure code redundancy is achieved with large n values, then data protection reliability improves, but memory usage and CPU time increase
Solution Approach 1:
The patent dynamically changes erasure code parameters (k, n, r) based on system conditions and data characteristics. Instead of using fixed large n values for all scenarios, the system adjusts parameters to achieve adequate reliability while minimizing computational overhead. This allows optimal balance between data protection and resource consumption for different workloads and system states.
Solution Approach 2:
The system applies erasure code redundancy selectively based on data priority, access patterns, and risk assessment. Rather than applying maximum redundancy (large n) to all data uniformly, the system uses partial redundancy for less critical data and full redundancy only where necessary, thereby reducing overall CPU and memory usage while maintaining adequate protection.
3Reliability
If high levels of data redundancy are maintained, then data availability improves, but storage efficiency deteriorates
Solution Approach 1:
The system dynamically adjusts the number of redundancy blocks based on real-time assessment of data criticality, device reliability, and available storage capacity. When storage is abundant and data is critical, higher redundancy is applied. When storage is constrained or data is less critical, redundancy is reduced, thereby optimizing the trade-off between availability and storage efficiency.
Solution Approach 2:
Different redundancy levels are applied to different data objects or data segments based on their specific requirements. High-value or frequently accessed data receives higher redundancy protection, while less critical data uses lower redundancy. This localized approach to redundancy optimization improves overall storage efficiency while maintaining necessary availability for critical data.
Data Source
AI summary
Example apparatus and methods monitor conditions in a tiered storage system. The conditions monitored may include the availability of different numbers and types of devices including an erasure code based object storage system. The conditions monitored may also include the availability and type of devices available to the erasure code based object storage system. A redundancy policy for storing an item using the erasure code based object storage system may be determined based on the conditions. Erasure codes associated with the item may then be stored in the erasure code based object storage system as controlled, at least in part, by the redundancy policy. The redundancy policy for the erasure codes may be updated dynamically in response to changing conditions on the tiered storage system.