Site Cache Error Handling in Distributed File Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed file systems face performance degradation due to geographical diversity of data storage, leading to increased latency and potential data loss from hardware failures in site caches.
Innovation Solution
Implementing a site cache at client locations with high bandwidth, low latency connections to process IO requests and employing redundancy and eviction policies to manage storage capacity and prevent data loss from hardware failures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored geographically diverse in distributed file systems, then storage capacity and availability are improved, but latency increases and data loss risk increases due to hardware failures
Solution Approach 1:
The patent segments the distributed file system into two distinct layers: a globally distributed storage layer for capacity and a local site cache layer for speed. By dividing the system architecture, data can be stored remotely for scalability while frequently accessed data is cached locally to minimize latency, thus resolving the contradiction between storage capacity and access speed.
Solution Approach 2:
The site cache acts as an intermediary between clients and the geographically distributed storage system. It buffers and caches data locally, mediating between the need for remote storage capacity and the requirement for fast local access, thereby reducing the impact of geographical distance on latency.
2Quantity of substance
If data is stored geographically diverse in distributed file systems, then storage capacity is improved, but reliability deteriorates due to potential data loss from hardware failures
Solution Approach 1:
The patent implements beforehand cushioning through redundancy mechanisms including erasure coding and replication across multiple storage nodes. Before data can be lost due to hardware failures, protective measures are already in place to reconstruct lost data from remaining replicas or encoded fragments, thus maintaining reliability while enabling geographic distribution for capacity.
Solution Approach 2:
The system creates multiple copies of data across different geographical locations and storage devices. By replicating data, the system ensures that if one copy is lost due to hardware failure, other copies remain available, thus maintaining reliability while allowing data to be distributed across multiple locations for increased storage capacity.
3Loss of time
If site cache is used to improve performance, then latency is reduced, but storage capacity is limited and data loss from cache failures remains a risk
Solution Approach 1:
The patent applies local quality by implementing site caches at specific client locations with high-speed local storage, while the bulk storage capacity resides in the distributed system. Each site cache is optimized for local performance with fast storage media, while the overall system maintains large capacity through distributed storage, thus achieving both low latency and high capacity.
4Reliability
If redundancy is implemented to prevent data loss, then reliability is improved, but storage capacity efficiency deteriorates
Solution Approach 1:
The patent employs erasure coding which changes the redundancy parameter from traditional replication (requiring full copies) to fractional redundancy (requiring only k out of n fragments). This parameter change in the redundancy mechanism allows data to be protected while using storage more efficiently, as less space is consumed for redundancy compared to full replication.
Data Source
AI summary
A failing computer readable storage medium comprising a portion of a transient object store is detected at a site cache. The site cache associated with the transient object store is set to an error read only state. The failing computer readable storage medium is replaced with an operational computer readable storage medium.