Virtual Disk Format Conversion for Read Performance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Virtualized storage systems face performance bottlenecks due to latency-sensitive read operations, which are impacted by write optimization techniques, leading to Quality of Service (QoS) violations and hindered operations, especially during data recovery and migration.
Innovation Solution
A method to convert a write-optimized virtual disk format into a read-optimized format by identifying conditions for data conversion, moving data to a thin-provisioned format, and compacting blocks to reclaim physical memory space, resulting in a read-optimized virtual disk format that improves read performance without affecting write performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is written in log-structured format to optimize write performance, then write speed is improved, but read performance deteriorates due to multiple random I/O requests and index traversals
Solution Approach 1:
The system dynamically converts data between log-structured format (for write optimization) and contiguous format (for read optimization) based on operational needs. The conversion process transforms scattered log-structured blocks into contiguous sequences, enabling the system to adapt its data organization format to current performance requirements rather than being static in one format.
Solution Approach 2:
The invention changes the physical organization parameter of data from log-structured (scattered) format to contiguous format through a conversion process. This parameter change involves reorganizing data blocks from their original log-structured positions into continuous sequences, fundamentally altering how data is physically arranged on storage media to improve read performance.
2Adaptability or versatility
If multiple index traversals are performed to read one version of data, then data versioning capability is maintained, but read performance suffers due to sequential reads across multiple HDDs
Solution Approach 1:
The conversion process merges scattered data blocks from multiple log-structured segments into a single contiguous sequence. By combining previously separated data blocks into continuous storage locations, the system eliminates the need for multiple sequential reads across different HDDs, thereby improving read throughput while preserving versioning information.
Solution Approach 2:
The system performs preliminary conversion of log-structured data to contiguous format in advance of read operations. This preliminary action reorganizes data beforehand so that when reads are required, the data is already in an optimized contiguous format, eliminating the need for multiple index traversals and sequential reads during actual read operations.
3Productivity
If data is stored in write-optimized format, then write efficiency is improved, but QoS violations occur during data recovery and migration due to poor read performance
Solution Approach 1:
The system dynamically adjusts data format based on operational context, maintaining log-structured format for write operations while converting to contiguous format when read operations or data recovery are anticipated. This dynamic adaptation ensures QoS compliance during critical operations without sacrificing write efficiency during normal write-intensive workloads.
Data Source
AI summary
A method of converting a data format in a storage system is provided. The method includes identifying a condition for when to make a data conversion from a write-optimized virtual disk format to a read-optimized virtual disk format; and locating, in response to the identified condition, a version of data. The method may include moving the version of data to a thin-provisioned format in-place virtual disk and detecting a set of overwrites from a remaining set of versions of the data, including blocks of data having been previously overwritten. The method may further include compacting each block within the set of overwrites to create a contiguous block such that an associated physical memory space is reclaimed, wherein the read-optimized virtual disk format of the data includes the thin-provisioned format in-place virtual disk and the contiguous block.