Virtual Disk Data Rearrangement for Storage Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deduplication methods in virtual storage disks become less effective over time as virtual machines diverge, leading to significant storage space inefficiencies due to duplicate data not being properly deduplicated, especially when data blocks are not arranged in the same sequence within deduplication chunks.
Innovation Solution
A method that identifies and rearranges data objects in multiple virtual storage disks to create identical chunks, allowing for efficient deduplication by physically relocating identical data objects without altering their logical state, and updating file system pointers and free block lists to maintain data integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If data is stored in virtual storage disks for virtual machines derived from a master image, then initial deduplication effectiveness is high, but deduplication effectiveness deteriorates over time as virtual machines diverge and data blocks become misaligned within deduplication chunks
Solution Approach 1:
The system performs preliminary data block rearrangement within deduplication chunks before deduplication occurs. By proactively repositioning data blocks to align identical blocks from different virtual machines into the same chunk boundaries, the system ensures optimal deduplication effectiveness is maintained throughout the storage lifecycle, preventing the deterioration that normally occurs as virtual machines diverge
2Productivity
If large data chunks are used for deduplication, then deduplication processing efficiency is improved, but the amount of undeduplicated duplicate data increases due to data block misalignment
Solution Approach 1:
The system applies localized rearrangement operations specifically within deduplication chunk boundaries. By repositioning data blocks only in the necessary local regions to achieve alignment, the system enables efficient processing of large chunks while ensuring that identical data blocks from different virtual machines are positioned to be captured by deduplication, thereby reducing the quantity of undeduplicated duplicate data
3Loss of energy
If data blocks are physically relocated to create identical chunks, then deduplication efficiency is improved, but system complexity increases due to the need to maintain data integrity and update file system pointers
Solution Approach 1:
The system introduces a data block management layer that acts as an intermediary between the file system and physical storage. This intermediary layer tracks data block locations and manages the rearrangement process, automatically updating file system metadata and pointers when blocks are moved. This abstraction handles the complexity of maintaining data integrity during physical relocation, allowing the system to improve storage efficiency without proportionally increasing overall system complexity
Data Source
AI summary
A method for managing virtual disk data storage may include: 1) identifying first and second virtual storage disks on a physical storage system; 2) identifying relocatable data in the first and second virtual storage disks; 3) comparing the relocatable data from the first and second virtual storage disks to identify one or more data objects in the first virtual storage disk that are identical to one or more data objects in the second virtual storage disk; 4) physically relocating the data objects from the first virtual storage disk to create a first chunk of data without changing a logical state of data in the first virtual storage disk; and 5) physically relocating the data objects from the second virtual storage disk to create a second chunk of data that is identical to the first chunk of data without changing a logical state of data in the second virtual storage disk.