Direct IO Path for Compressed Data Replication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Asynchronous replication schemes in data storage systems face inefficiencies due to significant processing overhead from accessing and decompressing snapshot data, particularly when cache misses occur, leading to suboptimal data transfer efficiency between source and destination storage systems.
Innovation Solution
A direct IO path is provided for compressed data on storage media by issuing a specialized IO read request that bypasses the IO transaction cache, allowing the mapping (logical) layer to obtain and transmit compressed data directly from storage media to the destination system, thereby avoiding decompression and cache-related overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If compressed snapshot data are read from storage media, decompressed, and placed in IO transaction cache, then data can be compared with source object, but significant processing overhead is incurred particularly with cache management and IO request handling
Solution Approach 1:
The patent extracts the decompression operation from the traditional IO path by introducing a specialized IO read compressed request that bypasses the IO transaction cache. The decompression is performed directly on compressed snapshot data during the comparison operation, eliminating the need to decompress and cache the data first. This extraction of the decompression step from the traditional read-decompress-cache-compare sequence reduces processing overhead while maintaining data transfer efficiency.
2Reliability
If IO read request is made for snapshot data absent from IO transaction cache, then cache miss occurs, but significant processing overhead is involved
Solution Approach 1:
The patent inverts the traditional approach by not reading compressed snapshot data into the IO transaction cache at all. Instead of following the conventional path of cache access (which results in cache misses and overhead), the system uses a specialized IO read compressed request that reads directly from storage media to the comparison operation, effectively inverting the data flow path to avoid the cache bottleneck entirely.
3Productivity
If compressed snapshot data are decompressed and placed in IO transaction cache, then data comparison can proceed, but processing overhead increases without likely cache hits
Solution Approach 1:
The patent implements self-service by performing the decompression operation in-situ during the data comparison process rather than pre-decompressing and caching the data. The specialized IO read compressed request enables the system to decompress and compare data in a single operation flow, allowing the data processing itself to serve the decompression need without separate cache management overhead, thereby reducing energy consumption while maintaining transfer speed.
Data Source
AI summary
Techniques for providing a direct IO path to compressed data on storage media of a storage system. The techniques include triggering a transaction cache to perform a flush operation for updating mapping metadata for a storage object containing the compressed data. Having updated the mapping metadata for the storage object, the techniques further include issuing, by a copier module, an IO read request for the compressed data of the storage object to a namespace layer, which issues the IO read request to a mapping layer. The techniques further include forwarding the IO read request to a logical layer of the mapping layer, bypassing the transaction cache. The techniques further include reading, by the logical layer, the compressed data of the storage object from the storage media, and providing, via the mapping layer and the namespace layer, the compressed data to the copier module for transfer to a destination storage system.