Erasure Coding Configuration for Dynamic Storage Topologies
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Administering erasure coding in dynamic and heterogeneous computing systems is burdensome for system administrators, often leading to suboptimal configurations and high costs due to the complexity of manually determining and implementing erasure coding configurations.
Innovation Solution
A multi-objective selection technique is implemented to dynamically select and manage erasure coding configurations in heterogeneous systems, using an erasure coding configurator that accesses fault tolerance policies, topology characteristics, and performance data to generate candidate configurations and compute configuration scores for efficient deployment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If administrative approaches are used to determine erasure coding configurations, then fault tolerance policies can be implemented, but system complexity and administrative burden increase significantly
Solution Approach 1:
The system automatically determines erasure coding configurations by analyzing workload characteristics, storage pool topology, and performance metrics. The configurator autonomously generates and evaluates candidate configurations without requiring manual administrative intervention, allowing the system to self-optimize its erasure coding strategy while maintaining fault tolerance policies.
Solution Approach 2:
The system dynamically adjusts erasure coding parameters (such as strip size, parity block count, and replication factor) based on changing workload conditions and system state. By continuously monitoring performance metrics and workload patterns, the configurator modifies configuration parameters to optimize storage efficiency and performance while adhering to fault tolerance requirements.
2Reliability
If manual erasure coding configuration is implemented, then fault tolerance can be achieved, but storage efficiency and performance suffer due to suboptimal configurations
Solution Approach 1:
The erasure coding configurator continuously monitors system performance metrics, workload characteristics, and storage pool status, using this feedback to dynamically adjust erasure coding configurations. The system evaluates the effectiveness of current configurations and automatically modifies parameters to improve storage efficiency and performance while maintaining required fault tolerance levels.
Solution Approach 2:
The system transitions from static, manually-configured erasure coding settings to dynamic, automatically-adjusted configurations. The configurator adapts erasure coding parameters in real-time based on changing workload patterns, system topology, and performance requirements, enabling the system to optimize storage efficiency as conditions evolve while preserving fault tolerance guarantees.
3Quantity of substance
If erasure coding is applied to reduce storage capacity demand, then storage efficiency improves, but computational overhead and processing requirements increase
Solution Approach 1:
The configurator applies erasure coding selectively to appropriate data workloads and storage pools rather than universally applying it to all data. By analyzing workload characteristics and access patterns, the system determines which data benefits most from erasure coding protection, applying the technique only where it provides net benefits in storage efficiency while minimizing unnecessary computational overhead.
Solution Approach 2:
The system dynamically adjusts erasure coding parameters such as strip size and parity block count based on data characteristics and workload requirements. By optimizing these parameters for different data types and access patterns, the system maximizes storage capacity efficiency while minimizing the computational overhead associated with encoding and decoding operations.
Data Source
AI summary
Dynamic erasure coding for computing and data storage systems. A method embodiment commences upon accessing a set of fault tolerance policy attributes associated with the computing and data storage system. The topology of the system is analyzed to form mappings between the computing nodes of the system and the availability domains of the system. Based on the fault tolerance policy attributes, the topology, and the generated mapping, a plurality of feasible erasure coding configurations are generated. The feasible erasure coding configurations are scored. One or more high-scoring feasible erasure coding configurations are selected and deployed to the computing and data storage system. The method is repeated when there is a change in the fault tolerance policy attributes or in the topology. Depending on the topology and/or the nature of a change in the topology, more than one erasure coding configurations can be deployed onto the computing and data storage system.