Asynchronous Write-Back Caching with Tracking Metafile
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for write-back caching in clustered networks face performance limitations, such as small data thresholds for write operations and write-back messages, leading to increased write latency and decreased throughput, especially for large datasets, and challenges in maintaining data consistency across clusters during scenarios like shutdowns or reboots.
Innovation Solution
The implementation of techniques that enable write-back caching across clusters by managing cache thresholds, payload sizes, and asynchronous write-back operations, using a tracking metafile and asynchronous write-back tracker to optimize write performance and ensure data consistency, allowing for concurrent write-backs and efficient data transfer without interrupting cache writes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is written to the cache and completion is confirmed only after writing to the underlying data store (write-through policy), then data consistency is ensured, but write latency increases
Solution Approach 1:
The patent applies preliminary action by writing data to the cache first and confirming completion before the underlying data store write occurs. The cache write is performed in advance, and the system confirms completion at this earlier stage rather than waiting for the slower underlying storage operation to finish, thereby reducing write latency while maintaining data consistency through subsequent write-back operations.
2Loss of time
If data is written to the cache with immediate completion confirmation (write-back policy), then write latency is reduced, but data consistency across clusters during shutdowns or reboots becomes challenging
Solution Approach 1:
The patent implements feedback mechanisms through tracking metafiles and asynchronous write-back trackers that monitor the state of cached data and coordinate write-back operations across clusters. These feedback systems track which data has been written to the cache, manage the asynchronous write-back process, and ensure that data consistency is maintained across distributed clusters even during shutdowns or reboots by providing visibility into the write state.
Solution Approach 2:
The patent introduces intermediaries in the form of tracking metafiles and asynchronous write-back trackers that mediate between the cache and underlying data stores across clusters. These intermediaries coordinate the asynchronous write-back operations, manage data state information, and ensure consistency is maintained across distributed systems without requiring synchronous communication, thereby solving the data consistency challenge in write-back caching across clusters.
3Device complexity
If small data thresholds are used for write operations and write-back messages in clustered networks, then device complexity is reduced, but throughput decreases for large datasets
Solution Approach 1:
The patent applies segmentation by dividing large datasets into manageable segments that can be processed through the caching system. The tracking metafile and asynchronous write-back mechanism enable efficient handling of segmented data by tracking individual segments or blocks independently, allowing the system to maintain simplicity in operation while achieving high throughput for large datasets through parallel processing of multiple segments.
Data Source
AI summary
A method, computing device, and non-transitory machine-readable medium for performing asynchronous write-backs. Data is written to a cache file in a cache. The cache corresponds to a volume. A tracking metafile is updated based on the data written to the cache file. A record in the tracking metafile is determined to be full. The record corresponds to a group of blocks in the cache file. A write-back of data stored in the group of blocks in the cache file that corresponds to the record to the volume is initiated. The write-back is determined to have been completed. The tracking metafile us updated to indicate that the write-back has been completed.