Distributed File System Storage Allocation via Stripe Chunking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed file systems in cloud computing environments face challenges in maintaining data protection and parity due to large capacity and dynamic flexibility, leading to difficulties in managing storage volume failures and ensuring data availability.
Innovation Solution
The system employs a method to manage storage capacity by determining the number of stripes that fit in the file system, assigning chunks to storage volumes and nodes based on protection factor information, and optimizing storage allocation to ensure data redundancy and availability across multiple storage volumes and nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If distributed file systems employ various parity or protection schemes to mitigate storage volume failure, then data reliability is improved, but device complexity increases
Solution Approach 1:
The patent segments data into stripes and chunks that can be independently managed and distributed across storage volumes. Each stripe contains multiple chunks with redundancy information, allowing the system to handle failures at the chunk level rather than requiring complex global protection schemes across entire storage systems.
Solution Approach 2:
The system dynamically adjusts protection parameters such as stripe width, number of chunks per stripe, and redundancy factors based on storage capacity and failure scenarios. This allows the same distributed file system to adapt to different storage configurations and failure modes without requiring fundamentally different protection mechanisms.
2Quantity of substance
If the file system includes many storage volumes to accommodate large data sets, then storage capacity is improved, but the likelihood of storage volume failure increases
Solution Approach 1:
The system pre-calculates and provisions recovery capacity by creating redundant chunks and parity information before failures occur. When storage volumes are initially configured, the stripe and chunk structure is established with built-in redundancy, so that when failures happen, recovery can proceed immediately using pre-positioned backup data rather than requiring complex real-time reconstruction.
3Reliability
If recovery capacity is determined by evaluating all possible combinations of storage volume failures, then data protection is improved, but computation time increases
Solution Approach 1:
Instead of evaluating all possible failure combinations, the system performs partial evaluation by focusing on the most critical failure scenarios based on the stripe and chunk structure. The redundancy factors and protection levels are calculated using simplified models that consider only the essential failure modes, providing sufficient protection without the computational overhead of exhaustive analysis.
Data Source
AI summary
Embodiments are directed to managing data in a file system. The file system that includes storage nodes that may be associated with storage volumes that may have a different capacity for storing data. A storage capacity of the file system may be determined based on a number of stripes of data that fit in the file system such that each stripe may be comprised of chunks that have a same chunk storage capacity. Slots in the file system that each match the chunk storage capacity may be determined based on the storage volumes. The chunks may be assigned to the slots in the file system based on the capacity of the storage nodes such that a number of chunks allocated to a same storage volume or a same storage node may be based on protection factor information.