VM Snapshot Access via Frequent Storage to Cut I/O Amplification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
I/O amplification occurs when accessing virtual machine snapshots in cloud storage due to the granularity of storage operations, leading to inefficient data retrieval and amplification of I/O operations, especially when accessing small pieces of data scattered across different chunks.
Innovation Solution
A backup system maintains a frequent access storage that stores duplicative copies of frequently accessed data in a smaller chunk size, bypassing retrieval from cloud storage to reduce I/O amplification by providing duplicative copies from a duplicative data store.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored in large chunks in cloud storage, then storage efficiency is improved, but data retrieval efficiency deteriorates due to I/O amplification when accessing small pieces of data
Solution Approach 1:
The patent segments the storage system into two distinct layers: a cloud storage layer that maintains large chunk sizes for efficient bulk storage, and a frequent access storage layer that uses smaller granular chunks for efficient retrieval of individual data pieces. This segmentation allows each layer to optimize for its specific function without compromising the other.
Solution Approach 2:
The frequent access storage acts as an intermediary layer between the cloud storage and the virtual machine. It intercepts small data access requests, serves them from its own smaller chunks, and only accesses the cloud storage when necessary for larger operations, thereby eliminating I/O amplification caused by direct cloud storage access.
2Adaptability or versatility
If small pieces of data are accessed from cloud storage, then data access flexibility is improved, but I/O operations are amplified due to chunk granularity constraints
Solution Approach 1:
The patent applies local quality by making the chunk size granularity local to the storage layer being accessed. The frequent access storage layer uses small chunk sizes locally optimized for individual data piece access, while the cloud storage layer maintains large chunk sizes locally optimized for bulk operations. This resolves the contradiction by allowing small data access without forcing the entire cloud storage system to use small chunks.
Solution Approach 2:
The frequent access storage pre-loads and maintains duplicative copies of frequently accessed data in smaller chunks than would be efficient for general cloud storage. This partial action (pre-loading only frequent data) eliminates I/O amplification for those specific data pieces without requiring the entire system to operate with fine-grained chunks.
3Speed
If duplicative copies of frequently accessed data are maintained in frequent access storage, then data retrieval speed is improved, but storage space consumption increases
Solution Approach 1:
Instead of maintaining duplicative copies of all data, the system applies partial action by only creating duplicative copies in frequent access storage for data that is identified as frequently accessed. This selective duplication achieves speed improvements for the critical subset of data while limiting storage space consumption to only what is necessary for high-frequency access patterns.
Solution Approach 2:
The system dynamically changes the parameter of chunk size based on access frequency. For frequently accessed data, it uses small chunk sizes in frequent access storage to enable fast retrieval. For less frequently accessed data, it relies on the original large chunk storage in the cloud. This parameter change allows the system to optimize retrieval speed for frequent operations without permanently allocating excessive storage space.
Data Source
AI summary
A system includes a duplicative data store different from a cloud storage. The data store stores duplicative copies of data that are used by one or more virtual machines. The system receives a request to retrieve from the cloud storage data associated with a VM. The cloud storage is configured to store data in a first chunk granularity larger than a second chunk granularity of the duplicative data store. The system determines whether a duplicative copy of the requested data is stored in the data store, and responsive to determining the duplicative copy of the requested data is stored in the data store, the system retrieves the duplicative copy of requested data from the data store. The system bypasses a retrieval of the chunk from the cloud storage, and provides the duplicative copy as a response to the request.


