Tiered Cache Retrieval Across Local and Cloud Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current caching approaches in distributed storage systems are limited in their ability to reduce latency for clients accessing data, particularly when data is stored across multiple tiers including local and cloud storage.
Innovation Solution
A cache volume that retrieves data from both an origin volume and a cloud storage endpoint to satisfy a single data request, allowing for improved performance by distributing load, reducing latency, and enhancing availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is stored only on local storage devices, then data access speed is fast, but storage capacity and cost are limited
Solution Approach 1:
The storage system is segmented into multiple tiers: high-performance local storage devices (SSDs, HDDs) for frequently accessed data and lower-performance cloud storage for less frequently accessed data. This segmentation allows the system to optimize for both speed and capacity by placing different types of data on different storage media.
Solution Approach 2:
The patent introduces a new dimension to the storage architecture by integrating cloud storage with local storage devices. This creates a multi-dimensional storage hierarchy that extends beyond traditional single-location storage, enabling the system to access data from multiple sources (local and remote) simultaneously or sequentially based on access patterns.
2Quantity of substance
If data is stored on cloud storage, then storage capacity increases, but data access latency increases
Solution Approach 1:
The cache volume pre-loads or caches frequently accessed data from cloud storage to local storage devices before actual client requests arrive. This preliminary action ensures that when data is needed, it is already available locally, reducing access latency while maintaining the ability to store large amounts of data in the cloud.
Solution Approach 2:
The cache volume acts as an intermediary between cloud storage and clients. It sits between the cloud storage endpoint and the client, intercepting requests and serving data from local cache when possible, thereby reducing the time clients wait for data while still utilizing cloud storage capacity.
3Reliability
If all data requests are handled by the origin volume, then data consistency is maintained, but load on the origin volume increases
Solution Approach 1:
The cache volume serves as an intermediary that handles read requests before they reach the origin volume. It maintains data consistency by validating cached data against the origin volume when necessary, thereby reducing the load on the origin volume while preserving data integrity.
Solution Approach 2:
The system creates copies of data in the cache volume from the origin volume. These copies allow the cache to serve read requests independently, reducing origin volume load. Data consistency is maintained through validation mechanisms that ensure cached copies match the origin data when needed.
4Loss of time
If a cache volume serves read requests from local storage, then access latency is reduced, but storage complexity increases
Solution Approach 1:
The cache volume is designed to perform multiple functions: caching frequently accessed data, serving read requests, and managing data between cloud storage and clients. This multi-functionality reduces the need for separate specialized components, thereby managing complexity while providing comprehensive caching capabilities.
Data Source
AI summary
A system, method, and machine-readable storage medium for retrieving data are provided. In some embodiments, a cache may receive a request for data from a client. The cache may determine that a first subset of the data is stored on a storage device and that a second subset of the data is stored at a cloud address located at a cloud storage endpoint. The cache may also receive from the storage device the first subset of data. The cache further receives from the cloud storage endpoint the second subset of data in response to transmitting a request for the second subset of data stored at the cloud address to the cloud storage endpoint. The cache then transmits to the client the first and second subsets of data from the various sources in response to the data request.