Reshapable Caching for Cloud-Native File Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems face limitations in expanding cache capacity without requiring all disks to be equally expanded, necessitating offline operations and complex formatting processes, which restrict flexibility and performance.
Innovation Solution
Implementing a 'reshapable' caching scheme that allows conversion from an n-way stripe set to an n+1-way stripe set by redistributing data blocks, enabling flexible expansion and online operation without the need for initial striping or disaster recovery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional striping is used for cache expansion, then cache capacity can be increased, but all disks must be equally expanded and the filer must be offline during the operation
Solution Approach 1:
The patent implements dynamic cache expansion by allowing the cache to transition between different stripe configurations (e.g., from 2-way to 3-way striping) without requiring the filer to be offline. The system can add disks to the cache array at any time, automatically recalculating and redistributing data blocks to utilize the new configuration, thereby providing flexible, online cache capacity increases.
Solution Approach 2:
The system changes the striping parameter (stripe width) dynamically based on the number of available disks. When a new disk is added, the system transitions from an n-way stripe set to an (n+1)-way stripe set, automatically adjusting the data distribution strategy to maximize cache capacity utilization without requiring manual reconfiguration or offline operations.
2Quantity of substance
If traditional striping is used for cache expansion, then cache capacity can be increased, but the filer must be offline during the stripe operation
Solution Approach 1:
The system performs preliminary calculations of the new stripe configuration and data block destinations before actually beginning the data redistribution process. This allows the system to prepare the expansion operation in advance, minimizing the actual downtime required during the transition from n-way to (n+1)-way striping, and enabling the filer to remain online throughout the process.
Solution Approach 2:
The patent ensures continuous cache operation during expansion by maintaining data accessibility throughout the stripe set transition. The system redistributes data blocks in a manner that preserves cache functionality, allowing the filer to continue serving requests while the cache configuration evolves from n-way to (n+1)-way striping without interruption.
3Productivity
If traditional striping is used, then cache performance can be improved through concurrent disk operations, but new devices must be added in multiples of the stripe width
Solution Approach 1:
The system dynamically adjusts the stripe width to match the current number of disks in the cache array. Instead of requiring disks to be added in fixed multiples, the system automatically transitions from n-way to (n+1)-way striping, allowing flexible device addition while maintaining optimal concurrent disk operations and cache performance.
Solution Approach 2:
The cache system is designed to accept any number of disks without requiring adherence to fixed stripe width multiples. The universal stripe configuration approach allows the same cache infrastructure to adapt to varying numbers of disks, maintaining performance benefits of concurrent operations while providing versatile device addition options.
Data Source
AI summary
A cloud-native global file system in which a local filer creates objects and forward them to a cloud-based object store is augmented to include a reshapable caching scheme for the local filer. Like striped caches, the approach uses a stripe, but the striping is implemented via a true RAID 0 (disk striping) rather than as a striped LV (logical volume) device. This approach allows for a “reshape” operation to convert from a n-way stripe set to a n+1-way stripe set. Preferably, a reshape involves redistributing each block on disk to its new calculated home. For example, going from a single disk to a two disk set would move every other block from disk 1 to disk 2, and rearrange the blocks on disk 1 to fill in the “holes”. Performance after the reshape matches that of a striped cache. In one embodiment, the cache is structured as a “degraded” RAID 4.