Virtual Disk Sparseness Optimization via Dual Metadata Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Maintaining sparseness in virtual machine storage volumes is resource-intensive and inefficient, reducing performance in virtual machine environments.
Innovation Solution
Utilizing both metadata that describes the storage of a volume in a virtual storage volume and metadata that describes the storage within an underlying storage volume to increase the sparseness of the underlying storage volume by creating and processing block bitmaps and block mapping tables, resulting in a new virtual disk file that reflects allocated and unallocated blocks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If metadata is written to describe empty blocks in the primary storage volume, then sparseness of the underlying storage volume is increased, but resource consumption and processing time increase significantly
Solution Approach 1:
The system performs sparseness optimization as a preliminary action during off-peak hours or maintenance windows. The hypervisor analyzes metadata and reclaims space in advance before storage resources become constrained, preventing performance degradation before it occurs rather than reacting to it in real-time.
Solution Approach 2:
The system creates a copy of the virtual disk file and processes the copy to generate sparse regions. By working on a copy rather than the live virtual disk, the system can perform intensive metadata analysis and space reclamation operations without impacting the performance of running virtual machines, thus reducing the time penalty for sparseness optimization.
2Quantity of substance
If metadata analysis is performed to increase sparseness, then storage efficiency is improved, but virtual machine performance decreases due to resource intensity
Solution Approach 1:
The hypervisor acts as an intermediary layer between the virtual machine and the physical storage system. It intercepts I/O operations and performs metadata analysis at this intermediate level, allowing sparseness optimization to occur without requiring direct intervention in the virtual machine's operational processes, thus minimizing performance impact.
Solution Approach 2:
The system identifies and discards redundant data blocks that are no longer needed by virtual machines, then recovers the space by creating sparse regions in the underlying storage volume. This selective discarding and recovering process improves storage efficiency while maintaining virtual machine performance by only affecting truly redundant data.
3Quantity of substance
If the underlying storage volume is made sparse by writing metadata, then space is saved, but the process is very resource intensive
Solution Approach 1:
Instead of analyzing and optimizing the entire virtual disk file at once, the system performs partial analysis on subsets of the data or focuses on specific regions with high sparseness potential. This partial action approach achieves meaningful space savings while reducing the computational resources required for the optimization process.
Solution Approach 2:
The system changes parameters such as the granularity of metadata analysis, the threshold for declaring blocks as sparse, and the frequency of optimization operations. By adjusting these parameters, the system can balance space utilization improvements against resource consumption, selecting optimal settings based on current system conditions and requirements.
Data Source
AI summary
Disclosed is a method of operating a data storage system. The method comprises generating first metadata describing storage of a volume of data in a first storage volume, storing the volume of data within a second storage volume, generating second metadata describing storage of the volume of data in the second storage volume, and processing the first metadata and the second metadata to increase sparseness of the volume of data stored in the second storage volume.