Snapshot Cache Layering for Faster Incremental VM Reads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face inefficiencies in optimizing jobs that retrieve data from snapshot information, particularly when dealing with a combination of full and incremental snapshots of virtual machines, characterized by semi-repeatable random input/output operations at a slow rate.
Innovation Solution
Implementing a cache organized in levels corresponding to snapshots, where reads from one level cause writes to a cached snapshot in the same level, allowing data retrieval from solid state disk memory instead of hard disk drives, optimizing performance by reusing the initialized cache during job executions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is retrieved from hard disk drives for snapshot processing, then storage capacity is maintained, but data access speed is slow
Solution Approach 1:
The storage system is segmented into two distinct parts: a cache layer using solid state disk memory for frequently accessed snapshot data, and a storage layer using hard disk drives for full snapshot retention. This segmentation allows fast access to hot data while maintaining economical long-term storage capacity.
Solution Approach 2:
A cache module acts as an intermediary between the job module and the snapshot information stored on hard disk drives. The cache module intercepts read requests, serves data from solid state disk memory when available, and only accesses hard disk drives when data is not in cache, thereby mediating between fast access needs and storage capacity constraints.
2Productivity
If solid state disk memory is used for caching snapshot data, then data retrieval efficiency is improved, but system cost increases
Solution Approach 1:
The system applies different quality characteristics to different parts of the storage hierarchy: solid state disk memory provides high-speed access for actively accessed snapshot data in the cache, while hard disk drives provide high-capacity storage for complete snapshot information. This local differentiation optimizes both performance and cost.
Solution Approach 2:
Instead of caching all snapshot data, the system implements partial caching by storing only the portions of snapshot information that are actually read during job execution. The cache module loads data from hard disk drives on-demand and retains it in solid state disk memory for subsequent accesses, avoiding the need to pre-cache entire snapshots.
3Speed
If cache is initialized with snapshot information before job execution, then data access speed improves, but initialization time is required
Solution Approach 1:
The cache module is pre-configured with the structure and organization of snapshot information during system setup. Snapshot information is organized in a hierarchical structure with full snapshots stored on hard disk drives and incremental snapshots indexed for quick reference. This preliminary organization enables rapid data retrieval during job execution without requiring runtime initialization.
Solution Approach 2:
The cache module automatically manages its own content by monitoring job execution patterns and pre-loading snapshot data that is likely to be accessed. The system self-optimizes by identifying frequently accessed data and maintaining it in solid state disk memory without requiring manual intervention or extensive pre-initialization.
Data Source
AI summary
Systems, methods, and machine-storage mediums for optimizing snapshot image processing are described. The system receives a first read request to read data from optimized snapshot information including snapshot information and cached snapshot information. The first read request includes a first offset identifying a first storage location and a first length. The snapshot information includes a full snapshot and at least one incremental snapshot. The system identifies a first portion of the data is stored in the snapshot information responsive to identifying the first portion of the data is not stored in the cache snapshot information. The system identifies a second portion of data is stored in the optimized snapshot information, reads the first portion of data and the second portion of data from the optimized snapshot information, and communicates the data, including the first and second portions of the data, to the job.