Asynchronous Unmap for Thinly Provisioned Virtual Machine Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing virtualization systems face storage inefficiencies due to the allocation of physical storage space for virtual disks and LUNs, which can lead to unused 'stale' data blocks remaining allocated even after files are deleted or virtual disk migrations, as the current methods for reclaiming these blocks are inefficient.
Innovation Solution
Implementing asynchronous unmap commands to release storage blocks allocated to virtual machines, where deleted files are moved to a delete directory and unmap commands are issued by a background thread based on criteria such as time, file size, or type, allowing for efficient reclamation of unused storage space.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If unmap commands are issued synchronously upon file deletion to reclaim storage blocks, then storage efficiency is improved, but system overhead and processing time increase
Solution Approach 1:
The system moves deleted files to a delete directory as a preliminary action, preparing them for future unmap operations without immediately executing the unmap command. This allows the system to batch process unmap operations later, improving storage efficiency while avoiding immediate processing overhead.
Solution Approach 2:
Instead of issuing unmap commands immediately upon file deletion, the system implements periodic unmap operations that cycle through the delete directory at scheduled intervals. This periodic approach reclaims storage blocks over time while distributing the processing load, reducing peak overhead and improving overall system performance.
2Reliability
If all physical storage space is allocated upfront for virtual disks, then storage availability is ensured, but storage waste increases due to unused blocks
Solution Approach 1:
The system implements self-service storage management where storage blocks are automatically reclaimed through asynchronous unmap operations when files are deleted. The delete directory serves as a self-managed queue that automatically triggers unmap operations, eliminating the need for manual storage management while preventing storage waste from stale data blocks.
Solution Approach 2:
The system changes the storage allocation parameter from static upfront allocation to dynamic allocation based on actual usage. By implementing thin provisioning with asynchronous unmap, storage space is allocated only when needed and reclaimed when no longer used, optimizing the balance between storage availability and storage efficiency.
3Loss of substance
If synchronous unmap operations are performed upon file deletion, then storage reclamation is immediate, but system performance degrades due to overhead
Solution Approach 1:
The system performs preliminary file deletion and moves files to the delete directory without immediately executing unmap operations. This separates the file deletion action from the storage reclamation action, allowing the system to maintain high productivity during file operations while still achieving storage reclamation through subsequent asynchronous unmap operations.
Solution Approach 2:
The delete directory serves as an intermediary structure between file deletion and storage reclamation. Instead of directly mapping file deletion to immediate unmap operations, the system uses the delete directory as a buffer that queues files for future unmap processing, reducing the direct overhead impact on system performance while maintaining storage reclamation functionality.
Data Source
AI summary
In a computer system having virtual machines running therein, a hypervisor that supports execution of the virtual machines allocates blocks of storage to the virtual machines from a thinly provisioned logical block device. When the hypervisor deletes a file or receives commands to delete a file, the hypervisor moves the file into a delete directory. An unmap thread running in the background issues unmap commands to the storage device to release one or more blocks of the logical block device that are allocated to the files in the delete directory, so that the unmap operation can be executed asynchronously with respect to the file delete event.