Clustered Storage Bandwidth Optimization for Small Writes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Content addressable storage systems experience performance degradation when processing write requests for data blocks smaller than the native page size, leading to excessive network bandwidth consumption in clustered implementations.
Innovation Solution
A distributed content addressable storage system with a clustered implementation, utilizing control and data modules to manage content-based signatures, reduces network bandwidth by computing and updating signatures efficiently, allowing small writes to be processed without degrading system performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If content addressable storage systems process write requests for data blocks smaller than native page size using traditional methods, then storage efficiency through deduplication and compression is maintained, but network bandwidth consumption increases excessively
Solution Approach 1:
The patent segments the page update operation into multiple phases: (1) control module computes content-based signature of the small data block, (2) control module sends only the signature to the data module, (3) data module updates page in storage, (4) data module computes updated page signature and returns it. This segmentation allows small writes to be processed without transferring entire pages over the network, significantly reducing bandwidth consumption while maintaining system performance.
Solution Approach 2:
The patent extracts the content-based signature computation and transfer from the traditional page-level operation. Instead of transferring entire pages or large portions of data, the system extracts and transfers only the essential signature information (hash values) which is minimal in size. This extraction principle enables efficient processing of small writes by eliminating unnecessary data transmission.
2Reliability
If clustered storage systems transfer entire pages for small write operations, then data consistency is maintained, but network bandwidth is excessively consumed
Solution Approach 1:
The patent uses content-based signatures (hash copies) as representatives of the actual data pages. When a small write occurs, the control module computes a signature copy of the data block and sends this signature to the data module. The data module uses this signature copy to update the actual page data in storage, ensuring data consistency without transferring the entire page over the network.
Solution Approach 2:
The patent changes the parameter being transmitted from page-level data (large size) to signature-level data (minimal size). By transforming the data representation from actual page content to content-based signatures, the system maintains the essential identifying and verification properties needed for data consistency while dramatically reducing the amount of data that must be transferred across the network.
Data Source
AI summary
A control module of a clustered storage system in one embodiment is configured to receive a write request for a data block of a page, to retrieve an existing content-based signature for the page, to identify from the existing content-based signature a first one of the data modules currently storing the page, and to compute a content-based signature of the data block. The control module is further configured to send the content-based signature of the data block to the first data module, to receive from the first data module an updated content-based signature for the page computed utilizing the content-based signature of the data block, to identify from the updated content-based signature a second one of the data modules that will store the page as updated to include the data block, and to provide information regarding the data block and the updated content-based signature to the second data module for utilization in storage of the updated page.