Sparse File Access Header Assembly for Storage Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face inefficiencies in transferring non-materialized blocks, as they traditionally send full-sized data blocks filled with a reserved bit pattern, leading to unnecessary data transfer and reduced throughput, especially over limited networks.
Innovation Solution
The system optimizes sparse file access by sending a header that identifies non-materialized blocks instead of full-sized data blocks, reducing payload size and improving throughput by assembling content that combines materialized blocks with metadata indicating non-materialized blocks, allowing for efficient read and write operations without transferring actual data for non-materialized blocks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If full-sized data blocks filled with reserved bit pattern are sent for non-materialized blocks, then block transfer is maintained with existing protocols, but data transfer volume increases unnecessarily and throughput decreases
Solution Approach 1:
The patent extracts only the essential information (block address and non-materialized status) from the full data block, eliminating the unnecessary reserved bit pattern data. The external interface sends a reduced representation containing only the block address and a flag indicating non-materialization, rather than transferring the complete 64KB block filled with reserved patterns.
Solution Approach 2:
The patent creates a simplified copy or representation of the block metadata instead of the actual block data. The external interface transmits a condensed structure that references the non-materialized block through its address and status flag, rather than copying the full block content which would be filled with reserved patterns.
2Reliability
If full-sized data blocks are transferred for non-materialized blocks, then compatibility with existing storage protocols is maintained, but network capacity is eroded due to limited network bandwidth
Solution Approach 1:
The patent segments the block transfer into two parts: the essential metadata (block address and non-materialized flag) that is actually transferred over the network, and the actual block data which is never transferred for non-materialized blocks. This segmentation allows protocol compatibility while dramatically reducing network traffic.
Solution Approach 2:
The patent introduces an intermediary representation layer between the storage system and the external interface. This intermediary structure (containing block address and status flag) serves as a mediator that conveys the necessary information about non-materialized blocks without requiring transfer of the full block data, thus maintaining compatibility while improving throughput.
3Ease of operation
If non-materialized blocks are transferred using traditional methods, then storage operations can be performed, but latency increases due to unnecessary data transfer
Solution Approach 1:
The patent extracts only the essential identification information (block address and non-materialized flag) from the full block structure. By taking out only what is necessary to identify and handle non-materialized blocks, the system eliminates the time-consuming transfer of unnecessary reserved bit pattern data, thereby reducing access latency.
Solution Approach 2:
The patent applies partial action by transferring only the necessary metadata portion of the block (address and status flag) rather than the complete block data. This partial transfer approach maintains operational functionality while minimizing the time required for data transfer, thus reducing latency.
Data Source
AI summary
Techniques herein are for accessing non-materialized blocks of a sparse file. A method involves a storage system receiving a storage command to access a sparse file. A combined content of a set of materialized blocks and a header that identifies one or more non-materialized blocks is assembled. The combined content does not comprise a content of the one or more non-materialized blocks. Responsive to the assembling, the combined content is transferred between the storage system and a computer system.