Intent Log Overlap Logging for Storage Write Endurance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The use of high-performance storage devices for intent logs increases the cost of storage systems due to their higher manufacturing costs, while lower-cost devices compromise performance, and existing systems write data to intent logs for every request, reducing the lifetime of these devices.
Innovation Solution
A storage system selectively writes data to an intent log only when concurrent read requests overlap with write requests, using high-performance devices for intent logs and lower-cost devices for persistent storage, thereby reducing the amount of data written to the intent log and extending the lifetime of the high-performance devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is written to intent log for every write request, then system performance and data resilience are improved, but the lifetime of high-performance storage devices is reduced due to excessive write operations
Solution Approach 1:
The patent extracts only the essential data that needs to be logged for failure recovery - specifically, data that overlaps between concurrent read and write requests. This selective extraction reduces the volume of data written to the intent log while maintaining the ability to recover from failures, thus extending device lifetime without sacrificing reliability.
Solution Approach 2:
Instead of writing all write request data to the intent log (excessive action), the patent implements partial action by writing only the subset of data that overlaps with concurrent read requests. This partial logging approach is sufficient for failure recovery while significantly reducing write amplification and extending storage device lifetime.
2Productivity
If high-performance storage devices are used for intent logs, then write performance and system responsiveness are improved, but manufacturing costs and overall system cost increase
Solution Approach 1:
The patent effectively treats the intent log as a short-living component by accepting that it will wear out faster due to write operations. By reducing the data volume written to it through selective logging, the effective lifetime is extended, allowing the use of high-performance devices without proportionally increasing overall system cost, as the expensive device is used more efficiently.
Solution Approach 2:
The patent changes the parameter of data volume written to the intent log by implementing selective logging based on overlap detection. This parameter change reduces the effective write workload on high-performance storage devices, allowing the system to achieve better performance-to-cost ratio by optimizing how the expensive storage resource is utilized.
3Reliability
If all write data is stored in intent log, then complete data recovery is possible after failure, but the amount of data written to fast storage increases, reducing device lifetime
Solution Approach 1:
The patent extracts only the necessary portion of write data that needs to be preserved for failure recovery - specifically, the overlapping data between concurrent read and write requests. This extraction approach maintains recovery completeness for affected data while minimizing the total data volume written to the intent log, thereby extending device lifetime.
Solution Approach 2:
The patent implements partial logging action by writing only the minimal necessary data (overlapping regions) to the intent log rather than all write data. This partial action is sufficient to enable recovery of concurrent read operations after failure, while avoiding the excessive writing of non-overlapping data that would unnecessarily consume the limited write endurance of fast storage devices.
Data Source
AI summary
A storage system stores data on a persistent storage device and intent log on a high performance persistent storage device. The storage system receives requests to write data and in response, writes the data to the persistent storage device. The storage system receives a request to write data and a request to read data before the write request is processed. The storage system determines whether the data being read by the read request overlaps the data being written by the write request. If the data being read overlaps the data being written, the storage system writes the data being written in the intent log stored on the high performance persistent storage device. Subsequent to storing the data in the intent log on the high performance persistent storage device, the storage system sends the data being read to the client that sent the read request.