Distributed Cache Erasure Coding Write Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed storage systems employing erasure coding, efficient caching of write data is hindered by the need for a full stripe of data to be received, as reading missing data blocks from disk is time-consuming, and data blocks often arrive asynchronously, making it desirable to cache write data without reading from disk.
Innovation Solution
A method is introduced where m data blocks and p parity blocks are stored across m plus p independent computing devices, allowing for efficient caching by reading from a distributed cache with redundancy, ensuring data consistency through timestamp management and confirmation mechanisms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data blocks are cached in memory of independent computing devices, then write data caching efficiency is improved, but the ability to use memory of other independent computing devices for caching is limited
Solution Approach 1:
The patent implements a distributed caching system where memory resources across multiple independent computing devices can be shared and used collectively for caching write data. Any computing device in the array can contribute its memory to the distributed cache, making the caching capability universal across the entire system rather than limited to individual devices.
Solution Approach 2:
The caching system is segmented across multiple independent computing devices, with each device contributing a portion of the distributed cache. The data is divided into blocks that can be distributed across different devices' memory, allowing parallel caching operations and enabling the system to scale by adding more devices to the segment.
2Reliability
If a full stripe of data is received before caching, then erasure coded data caching reliability is improved, but caching speed deteriorates due to needing to read missing data blocks from disk
Solution Approach 1:
The system performs preliminary actions by receiving and caching data blocks as they arrive, rather than waiting for the complete stripe. The distributed cache preliminarily stores incoming blocks, and the erasure coding process is completed in the background, allowing caching to proceed without waiting for all blocks to arrive before starting the caching operation.
Solution Approach 2:
The distributed cache acts as an intermediary between the incoming data blocks and the final erasure coded storage. Instead of requiring complete stripes before caching, the intermediary cache system temporarily holds individual blocks and facilitates the erasure coding process, mediating between the asynchronous data arrival and the reliability requirements.
3Adaptability or versatility
If data blocks arrive asynchronously, then data reception flexibility is improved, but the ability to efficiently cache without reading from disk deteriorates
Solution Approach 1:
The caching system is designed to be dynamic, adapting to the asynchronous arrival of data blocks. Rather than requiring a fixed complete stripe before caching, the system dynamically processes blocks as they arrive, adjusting the caching operations based on the current state of received blocks and maintaining efficiency despite variable arrival patterns.
Data Source
AI summary
Data blocks are read from a distributed cache. The distributed cache comprises m replicated caches, each replicated cache including a plurality of independent computing devices. Each independent computing device of the replicated caches holds a replica of a particular one of the m data blocks in memory. The m data blocks and p parity blocks are stored across m plus p independent computing devices. Each of the m plus p independent computing devices stores a single block selected from the m data blocks and the p parity blocks.