Cloud Controller Archival File Restoration in Distributed Filesystems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud-based storage systems face challenges in maintaining data consistency and accessibility due to increased access latency and network failures, which can lead to data consistency issues and prolonged downtime for clients.
Innovation Solution
A distributed filesystem architecture where multiple cloud controllers collectively manage and cache data, ensuring data consistency by using transactional filesystems and incremental snapshots, and archive infrequently accessed files in an archival cloud storage system, allowing for efficient restoration and balancing performance and cost through billing models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored in cloud-based storage systems, then storage capacity and scalability are improved, but access latency increases and data consistency problems occur
Solution Approach 1:
The patent segments the storage system into multiple cloud controllers that collectively manage the distributed filesystem. Each cloud controller maintains local caches and metadata, allowing data to be divided across multiple locations. This segmentation enables parallel access operations and reduces the latency impact by distributing the access load across multiple nodes rather than a single centralized storage system.
Solution Approach 2:
The patent implements preliminary actions by pre-fetching data blocks before they are actually needed and maintaining local caches in cloud controllers. The system anticipates data access patterns and loads data into memory buffers in advance, reducing the perceived access latency for clients. Incremental snapshots are also created in advance to enable fast restoration operations.
2Ease of operation
If data is stored remotely in cloud-based storage, then storage management overhead is reduced for clients, but network failures and outages increase downtime
Solution Approach 1:
The patent implements beforehand cushioning by creating incremental snapshots of the distributed filesystem at regular intervals and maintaining them in the cloud storage system. These snapshots serve as a safety cushion that can be rapidly restored if network failures or outages occur. The snapshots are stored redundantly and can be accessed even when primary network connections fail, ensuring business continuity.
Solution Approach 2:
The patent introduces cloud controllers as intermediary components between clients and the remote cloud storage system. These controllers maintain local caches and metadata, acting as a buffer that can serve client requests even when the remote storage system is temporarily inaccessible due to network failures. The intermediaries handle data consistency and coordination, reducing the impact of network issues on overall system availability.
3Quantity of substance
If infrequently accessed files are archived in archival cloud storage, then storage cost is reduced, but restore time increases
Solution Approach 1:
The patent applies preliminary action by creating incremental snapshots in advance that capture the state of archived files. When a restore operation is needed, the system can rapidly apply these pre-computed snapshots to reconstruct archived files without having to retrieve and process all underlying data blocks from the archival storage system. This significantly reduces restore time while maintaining cost-effective archival storage.
Solution Approach 2:
The patent changes the parameter of data representation by storing incremental snapshots in a compressed and optimized format suitable for rapid restoration. The snapshots capture only the changes from the previous snapshot, allowing for efficient storage and quick application during restore operations. This parameter change enables the system to balance between low-cost archival storage and fast restore capabilities.
Data Source
AI summary
The disclosed embodiments disclose techniques for restoring an archived file in a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem. Furthermore, cloud controllers may archive infrequently-accessed files in an archival cloud storage system. During operation, a cloud controller receives a request from a client system to access an archived file, and restores this archived file from the archival cloud storage system.