Grain-Range Table Compression for Virtual Disk Storage Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Managing grain tables for virtual storage disks is inefficient, requiring multiple disk reads and consuming significant memory, which can conflict with production demands in hypervisors managing large numbers of virtual disks.
Innovation Solution
Implementing in-memory grain-range tables, which store entries for sequences of consecutive grains with linearly varying offsets, reducing the need for storage accesses and enabling efficient conversion from grain tables, thereby improving performance and reducing bandwidth requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If grain tables are kept in memory to improve access performance, then storage access time is reduced, but memory consumption increases significantly
Solution Approach 1:
The grain table is segmented into multiple zones based on access frequency and grain offset ranges. Hot zones (frequently accessed grains) are kept in memory while cold zones are stored on disk. This segmentation allows the system to maintain performance for critical operations while reducing overall memory consumption.
Solution Approach 2:
The patent implements a nested structure where grain tables are organized in hierarchical levels. The most recently accessed or frequently accessed grain table segments are nested within the memory-resident portion, while less frequently accessed segments are nested in the disk-based storage. This nested organization enables efficient access patterns with reduced memory requirements.
2Reliability
If complete grain tables are stored on physical storage for each virtual disk, then data integrity is maintained, but the number of disk reads increases
Solution Approach 1:
The system performs preliminary actions by pre-loading grain table segments that are likely to be accessed into memory before they are actually needed. Access patterns are analyzed and predicted, allowing the system to proactively load relevant grain table portions, thereby reducing the number of disk reads required during actual data operations.
Solution Approach 2:
The patent introduces an intermediary layer (cache memory) between the physical storage and the processing units. This intermediary holds frequently accessed grain table segments, acting as a buffer that reduces direct disk reads. The intermediary maintains data integrity while significantly reducing access time for common operations.
3Quantity of substance
If grain tables are compressed to reduce memory usage, then memory consumption is reduced, but access complexity increases
Solution Approach 1:
The patent applies parameter changes by using different compression techniques for different zones of the grain table based on their access characteristics. Hot zones use lighter compression or no compression to enable fast access, while cold zones use more aggressive compression to minimize memory usage. This zone-based parameter differentiation reduces overall memory consumption while maintaining acceptable access complexity.
Data Source
AI summary
A “grain” is a unit of space allocation in a virtual disk. Grains are represented in physical storage only when used, that is, when they contain data. Grains may be located in storage using an in-memory grain-range table. The grain-range table is derived from a grain table in storage. A grain-range table includes entries for the starting grains of incrementing and or decrementing ranges; grain-table entries that do not start a range are omitted in the grain-range table. Accordingly, a grain-range table can serve as a compressed form of a grain table. This compression makes it feasible to store large numbers of grain-range tables in memory in cases where it would not be feasible to store the corresponding grain tables in memory. As a result, one rather than two storage accesses are require per storage access request, resulting in a substantial performance improvement.