Hypervisor Write IO Optimization for Virtual Disk Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Write input/output operations in virtualized computing systems, particularly those using Virtual Machine File System (VMFS), face performance degradation due to the involvement of multiple subsystems and transactional overhead, leading to inefficiencies in metadata updates and resource management.
Innovation Solution
The solution involves sorting write IO operations based on file offsets and separating transaction-dependent and transaction-independent operations, allowing for batch allocation of resources and parallel data writing, thereby reducing the need for serial metadata updates and improving CPU efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If write IO operations are processed through multiple subsystems with transactional overhead for metadata updates, then data integrity and consistency are maintained, but system performance and write speed deteriorate
Solution Approach 1:
The patent segments write IO operations into two distinct categories: transaction-dependent operations (requiring metadata updates for data integrity) and transaction-independent operations (where data can be written without immediate metadata updates). This segmentation allows the system to apply different processing paths, maintaining reliability for critical operations while improving performance for non-critical ones by bypassing unnecessary subsystem overhead.
Solution Approach 2:
The patent applies partial action by selectively applying transactional metadata updates only when necessary (for transaction-dependent operations) rather than uniformly for all write operations. For transaction-independent operations, the system performs only the essential data writing without the full transactional overhead, thus improving performance while maintaining sufficient data integrity through selective application of the complete transactional process.
2Stability of the object's composition
If metadata updates are performed serially through multiple subsystems, then consistency is ensured, but processing time and latency increase
Solution Approach 1:
The patent segments the metadata update process by identifying which operations truly require serial transactional processing versus those that can be handled more efficiently. By categorizing operations as transaction-dependent or transaction-independent, the system maintains consistency for critical metadata updates while avoiding unnecessary serial processing for operations where consistency can be achieved through alternative means, thereby reducing overall processing latency.
Solution Approach 2:
The patent applies preliminary action by pre-classifying write IO operations into transaction-dependent and transaction-independent categories before processing. This allows the system to prepare appropriate handling paths in advance, ensuring that operations requiring serial metadata updates are properly queued and processed for consistency, while operations that don't require such updates can be processed more quickly without waiting for serial metadata operations, thus reducing overall latency.
3Manufacturing precision
If resource manager invocations are performed for each write operation, then resource allocation accuracy is maintained, but CPU efficiency and throughput decrease
Solution Approach 1:
The patent segments resource manager invocations based on operation type, invoking the resource manager only for transaction-dependent operations that require precise resource allocation tracking. For transaction-independent operations, the system bypasses resource manager invocations or uses simplified allocation mechanisms, thereby maintaining accurate resource allocation where necessary while significantly improving CPU efficiency and throughput by avoiding redundant invocations for operations that don't require such precision.
Solution Approach 2:
The patent applies partial action by performing full resource manager processing only when necessary for transaction-dependent operations, rather than invoking the resource manager for every write operation. This selective approach maintains adequate resource allocation accuracy for critical operations while reducing CPU overhead and improving overall throughput by using simplified or deferred resource management for non-critical operations.
Data Source
AI summary
An example method of handling, at a hypervisor on a host in a virtualized computing system, a write input/output (IO) operation to a file on a storage device having a virtual machine file system (VMFS) is described. The method includes: sorting, at the hypervisor, a scatter-gather array for the write IO operation into sets of scatter-gather elements, each of the sets including at least one scatter-gather element targeting a common file block address; resolving offsets of the sets of scatter-gather elements to identify a first scatter-gather array of transaction-dependent scatter-gather elements; generating logical transactions for the first scatter-gather array having updates to metadata of the VMFS for the file; batching the logical transactions into a physical transaction; and executing the physical transaction to commit the updates to the metadata of the VMFS on the storage device for the file.