Mapped Cluster Stretching for Data Storage Workload Scaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data storage systems face inefficiencies in handling increased workloads, leading to either prolonged processing times or the risk of destabilizing the storage system when trying to accelerate data analytics, especially with large amounts of archived data.
Innovation Solution
The technology involves 'cluster stretching' by increasing the number of mapped nodes in a mapped Redundant Array of Independent Nodes (RAIN) system while maintaining the same number of storage devices, allowing for improved computing resources and flexible redistribution of storage devices among nodes, which can be further stretched or un-stretched as needed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If computing resources are heavily loaded to process large amounts of archived data faster, then processing speed is improved, but the storage system becomes unstable
Solution Approach 1:
The patent segments the storage system into multiple mapped clusters, each handling a portion of the data workload. By dividing the large-scale data processing task across multiple independent mapped clusters rather than overloading a single cluster, the system achieves faster aggregate processing speed while maintaining stability through distributed load management.
2Productivity
If the number of mapped nodes is increased to handle increased workload, then computing power is improved, but system complexity increases
Solution Approach 1:
The patent creates universal mapped cluster templates that can be replicated and scaled. Instead of manually configuring each mapped cluster individually, the system uses a standardized template approach where a single mapped cluster configuration can serve multiple purposes and be copied to create additional clusters, thereby increasing productivity while minimizing the complexity increase through reuse of proven configurations.
3Reliability
If data is processed at moderate speed to maintain system stability, then system reliability is preserved, but processing time becomes unacceptably long
Solution Approach 1:
The patent transitions from a single-dimension approach (one mapped cluster processing data sequentially) to a multi-dimensional approach by creating multiple mapped clusters that process different portions of data simultaneously in parallel. This dimensional expansion from sequential to parallel processing dramatically reduces total processing time while each individual cluster operates at stable, moderate speeds, thus preserving reliability.
Data Source
AI summary
The described technology is generally directed towards stretching a mapped storage clusters by adding nodes to a mapped cluster of mapped nodes and storage devices mapped to a real cluster of nodes and storage devices. Stretching the mapped cluster can provide additional computing resources to a set of storage devices. In one implementation, one or more newly mapped nodes are added to increase the node count of an existing mapped cluster to form a stretched cluster, with the storage devices distributed among the increased number of nodes; a mapping table is updated to relate the stretched cluster nodes and storage devices to the real cluster nodes and storage devices. Also described is un-stretching a stretched cluster, or further stretching a stretched cluster.