Logical Map Table Compressed Size Field for Datastore Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed datastores like vSAN, there is a need for an efficient mechanism to store the size of each compressed data block to facilitate decompression during read operations, as existing methods lack a reliable method to track the size of compressed data blocks across different storage layers.
Innovation Solution
Incorporating a compressed size field in the logical map table with additional metadata, such as a counter, to indicate the size of compressed data blocks, allowing for efficient storage and retrieval of decompressed data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If data is compressed before writing to the performance tier, then CPU cost and bandwidth costs are reduced, but the need to store and manage compressed data block sizes increases system complexity
Solution Approach 1:
The patent introduces a compressed size field as an intermediary data structure within the logical map table. This field acts as a mediator that stores the size information of compressed data blocks, enabling the system to track and manage compressed data without requiring complex external tracking mechanisms. The compressed size field serves as a bridge between the compression process and the data storage management system.
2Quantity of substance
If compressed data blocks are stored across multiple physical blocks, then storage efficiency is improved, but the difficulty of tracking and managing compressed data block sizes increases
Solution Approach 1:
The patent segments the management of compressed data by introducing a dedicated compressed size field within each logical map table entry. This segmentation allows the system to track the size of compressed data blocks independently from the physical storage location, enabling efficient management even when compressed data is distributed across multiple physical blocks. The logical map table serves as a segmented index that decouples logical data organization from physical storage details.
3Ease of operation
If a compressed size field is added to the logical map table, then decompression capability is enabled, but the size of metadata structures increases
Solution Approach 1:
The patent changes the parameter structure of the logical map table by adding a compressed size field. This parameter change enables the system to store and retrieve compressed data block sizes efficiently. The compressed size field is designed to store only the necessary size information, minimizing the increase in metadata structure size while providing the essential decompression capability. The field uses appropriate data types and storage formats to optimize space utilization.
Data Source
AI summary
A method for compressing is provided. The method including receiving a block of data to store on at least one physical disk; determining whether to store the data in a data log as uncompressed or compressed data based on a determined size of resulting compressed data. When the method determines to store the data as compressed, compressing the data and storing the compressed data in at least one sector in the data log. Otherwise, the method stores the data, uncompressed, in a plurality of sectors in the data log. The method generates a one or more state bits indicating (i) whether the data is stored as uncompressed or compressed, and (ii) if the data is stored as compressed, a size of the compressed data. The method then stores the one or more state bits in an entry of a logical map table associated with an LBA that corresponds to the data block.