Fibonacci Array for Disk Storage of Random Data Changes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Disks are inefficient for small random accesses, which can slow performance when trying to replicate data from byte-addressable persistent memory, as they operate best in sequential modes rather than random access modes.
Innovation Solution
Implementing an on-disk Fibonacci array structure that accumulates small, random data changes into large blocks for efficient storage, using a user-level process to manage and replicate data at byte-addressable granularity, and employing a merge-sort technique to maintain internal ordering and coalesce entries, thereby optimizing disk layout for streaming bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If small random data changes are directly written to disk, then data replication is achieved, but disk performance deteriorates due to random seek operations
Solution Approach 1:
The patent applies preliminary action by accumulating small random data changes in an in-core array structure before writing to disk. The system pre-processes multiple small random writes by buffering them in memory, then performs a single sequential write operation to disk when the buffer is full, thereby avoiding random seek operations and improving disk performance while maintaining data replication reliability
Solution Approach 2:
The patent introduces an intermediary in-core array structure that acts as a buffer between persistent memory and disk storage. This intermediary accumulates small random data changes and transforms them into large sequential blocks for disk writing, mediating between the byte-addressable nature of persistent memory and the block-based sequential optimization of disk storage
2Productivity
If byte-addressable persistent memory is used for high-frequency small random accesses, then data modification capability is improved, but disaster recovery capability deteriorates due to single copy vulnerability
Solution Approach 1:
The patent merges the advantages of byte-addressable persistent memory with block-based disk storage by combining them in a hybrid architecture. Small random data changes are initially written to persistent memory for high-speed modification capability, then merged with sequential disk writes through the in-core array buffer, achieving both fast data modification and disaster recovery capability through replicated storage
3Productivity
If disks are optimized for sequential bandwidth, then streaming performance is improved, but random access performance deteriorates
Solution Approach 1:
The patent applies inversion by reversing the typical write pattern: instead of writing small random blocks directly to disk (which causes seeking), it inverts the approach by accumulating writes in an in-core array and then writing large sequential blocks to disk. This inversion transforms random access operations into sequential streaming operations, maximizing disk bandwidth while maintaining the ability to handle random data changes from persistent memory
Data Source
AI summary
The embodiments described herein provide a system and method for efficiently storing small, random modifications or changes to data on one or more storage devices, such as disks, of storage servers coupled to a host computer in a network environment. Illustratively, the data is stored in a region of a byte-addressable, persistent memory of the host computer and is replicated (i.e., copied) as changed data of the region on the disks at the granularity at which it was modified, e.g., at the byte-addressable granularity. To that end, each storage server employs a data structure (e.g., a Fibonacci array) that is configured to efficiently accumulate the small, random data changes into one or more large blocks of changed data for storage on the disks in a manner that realizes the streaming bandwidth of the disk.