Cycled Clustering for Redundancy Coded Data Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern network computing and data storage systems face challenges in handling demand spikes, particularly in unpredictable access patterns, as existing database services optimized for certain types of increased demand may not be capable of handling different access rates effectively.
Innovation Solution
Implementing a hybridized data storage system that uses redundancy coding to distribute data across multiple entities, including data transfer devices and data storage systems, allowing for scalable and durable storage by provisioning data transfer devices externally and leveraging redundancy coding techniques such as erasure coding to ensure data availability and durability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If database services are optimized for certain types of increased demand (e.g., payload size), then performance for those specific demand types is improved, but the system becomes incapable of handling different access rates effectively
Solution Approach 1:
The patent implements dynamic provisioning of data transfer devices that can be added or removed from the cluster based on real-time demand characteristics. The system transitions from static database service configurations to dynamic clusters where device allocation adapts to different access patterns, payload sizes, and transaction rates, resolving the contradiction between optimization for specific demand types and adaptability to varying access rates
2Reliability
If data is distributed across multiple data transfer devices in a cluster, then scalability and durability are improved, but system complexity increases
Solution Approach 1:
The patent implements self-healing capabilities where the cluster automatically detects failed data transfer devices and redistributes their data to remaining devices without manual intervention. The system performs automated device provisioning, data redistribution, and failure recovery, reducing operational complexity while maintaining high reliability through redundancy across multiple devices
Solution Approach 2:
The patent divides data into shards distributed across multiple data transfer devices in a cluster. Each device handles a segment of the total data, enabling independent failure isolation and targeted recovery. This segmentation allows the system to achieve high reliability through distribution while managing complexity through modular, independent device units
3Reliability
If redundancy coding is used to ensure data availability during demand spikes, then data accessibility is improved, but storage capacity efficiency decreases
Solution Approach 1:
The patent dynamically adjusts redundancy parameters based on observed demand patterns and access rates. The system modifies encoding rates, replication factors, and cluster size parameters in response to changing workload characteristics, allowing it to maintain data accessibility during demand spikes while optimizing storage efficiency under normal conditions through parameter adaptation
Data Source
AI summary
A cluster of data transfer devices is used to augment the capabilities of a data storage system. For example, the cluster of data transfer devices may be configured to store a portion of a bundle of redundancy coded shards in a similar fashion as a data storage system. As another example, the cluster may be configured to provide other capabilities incident to the devices used, such as computational capabilities. Data stored on the cluster may be read from and written directly to the cluster without transfer of data to the data storage system. In some embodiments, a connecting entity (such as a customer entity) may interchangeably interface with the data storage system and the cluster, and the requested capabilities may be directed to either in a fashion that is transparent to the requestor.