Virtual Disk Meta-Operations During Active I/O
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Virtual disk I/O and meta-operations can conflict, leading to data corruption, and existing solutions that prevent meta-operations during I/O usage are not desirable as they limit the scenarios for using meta-operations.
Innovation Solution
Implementing a mechanism that allows meta-operations on virtual disks while they are in use for I/O by coordinating virtual disk I/Os with meta-operation performance, throttling or prioritizing reads and writes to prevent interference, and using de-duplication data structures for operations like copying, merging, or defragmenting without actual data copying.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If meta-operations are disallowed while virtual disk is in use for I/O, then data corruption is prevented, but the scenarios for using meta-operations are limited
Solution Approach 1:
The virtual disk offset range is divided into multiple segments or zones. The system identifies and segments the specific offset ranges involved in the meta-operation, then allows other I/O operations to proceed concurrently in non-overlapping segments. This segmentation enables parallel processing of I/O and meta-operations on different portions of the virtual disk, resolving the contradiction between data integrity and operational versatility.
Solution Approach 2:
The system dynamically adjusts the coordination between I/O operations and meta-operations based on real-time conditions. When a meta-operation is detected on a virtual disk, the system dynamically coordinates with the I/O servicing to identify overlapping offset ranges and applies throttling or prioritization only to those specific ranges and time periods, while allowing other operations to proceed. This dynamic coordination enables meta-operations to be performed on virtual disks in use without causing data corruption.
2Adaptability or versatility
If meta-operations are performed on virtual disk while I/O is servicing, then operational flexibility is improved, but data corruption risk increases
Solution Approach 1:
The system introduces an intermediary coordination mechanism that mediates between I/O operations and meta-operations. This intermediary monitors both types of operations, identifies potential conflicts in offset ranges, and coordinates their execution to prevent data corruption. The intermediary enables meta-operations to proceed on virtual disks in use by orchestrating their interaction, thus maintaining data consistency while preserving operational flexibility.
Solution Approach 2:
The system performs preliminary identification of offset ranges that will be affected by the meta-operation before the operation begins. By预先 identifying these ranges, the system can proactively coordinate with ongoing I/O operations to prevent overlaps and potential data corruption. This preliminary action allows meta-operations to be safely performed on virtual disks in use, maintaining both operational capability and data reliability.
3Reliability
If virtual disk reads and writes are throttled to allow meta-operation completion, then meta-operation success rate is improved, but I/O performance deteriorates
Solution Approach 1:
The system applies throttling or prioritization locally only to the specific offset ranges involved in the meta-operation, rather than globally to the entire virtual disk. This localized coordination allows I/O operations to continue at full performance on non-overlapping portions of the virtual disk, while ensuring meta-operation completion on the affected ranges. This resolves the contradiction between meta-operation success rate and I/O throughput by concentrating the performance impact only where necessary.
Data Source
AI summary
Described is a technology by which a virtual hard disk is able to continue servicing virtual disk I/O (reads and writes) while a meta-operation (e.g., copying, moving, deleting, merging, compressing, defragmenting, cryptographic signing, lifting, dropping, converting, or compacting virtual disk data) is performed on the virtual disk. The servicing of virtual disk I/Os may be coordinated with meta-operation performance, such as by throttling and/or prioritizing the virtual disk I/Os. Also described is performing a meta-operation by manipulating one or more de-duplication data structures.