Storage Processor Flush Ownership Assignment for Memory Overhead Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In active-active cluster/storage systems, flushing data across multiple storage processors requires significant memory overhead to maintain flushing work sets, as conventional approaches copy and reclaim data only when both processors are ready, leading to inefficient memory utilization.
Innovation Solution
Assigning flush ownership for containers of a common frozen flushing work set to specific storage processors, allowing for independent flushing and reclaiming of storage space without waiting for both processors, and handling XCOPY commands and storage processor failures to optimize data processing and memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is mirrored across duplicated flushing work sets on multiple storage processors, then data reliability is improved, but memory overhead increases significantly
Solution Approach 1:
The flushing work set is segmented into multiple containers, with each container independently manageable. This allows the system to flush and reclaim containers individually rather than requiring all containers across all storage processors to be flushed simultaneously, thereby reducing the peak memory overhead while maintaining data reliability through mirroring.
Solution Approach 2:
The system performs partial flushing actions by allowing one storage processor to flush its assigned containers independently without waiting for the other storage processor to complete its flushing. This partial action approach reduces the total memory overhead required while still achieving the necessary data reliability through the mirrored structure.
2Stability of the object's composition
If flushing waits for both storage processors to be ready, then data consistency is improved, but flushing speed decreases
Solution Approach 1:
The system performs preliminary actions by assigning specific containers to specific storage processors in advance and allowing independent flushing of assigned containers. This preliminary assignment enables one storage processor to flush its containers without waiting for the other, improving flushing speed while maintaining data consistency through the pre-established container assignment and mirroring structure.
3Reliability
If storage space is reclaimed only when both processors complete flushing, then data integrity is improved, but storage utilization efficiency decreases
Solution Approach 1:
The system performs partial reclaiming actions by allowing storage space to be reclaimed from containers that have been flushed by at least one storage processor, rather than requiring both processors to complete flushing. This partial reclaiming improves storage utilization efficiency while maintaining data integrity through the mirrored container structure and selective flushing approach.
Data Source
AI summary
A method, computer program product, and computing system for assigning flush ownership for a plurality of containers of a common frozen flushing work set to a storage processor of a pair of storage processors of a storage system. For each container of the plurality of containers of the common frozen flushing work set not assigned to a particular storage processor, the storage space of the container not assigned to the particular storage processor is reclaimed. For each container of the plurality of containers of the common frozen flushing work set assigned to a particular storage processor, data stored in the container assigned to the particular storage processor is flushed to persistent memory of the storage system.