Startup Metadata Buffering for Intermediate Persistent Storage Failure
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Storage devices that store metadata face failures during startup, leading to potential data disasters due to the loss of metadata when intermediate persistent storage devices fail.
Innovation Solution
Metadata is stored in volatile memory during the startup stage and transferred to an intermediate persistent storage device, with system operations suspended if the intermediate device fails, ensuring metadata is saved in persistent storage upon recovery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If metadata is stored in volatile memory during startup stage, then system operations can be performed, but metadata may be lost when intermediate persistent storage device fails
Solution Approach 1:
The system performs preliminary actions by suspending operations before metadata loss can occur. When intermediate persistent storage failure is detected, the system immediately suspends system operations to prevent any further metadata changes, ensuring the metadata in volatile memory remains stable and can be safely transferred to persistent storage without risk of corruption or loss.
Solution Approach 2:
The patent introduces an intermediary mechanism involving the suspension of system operations as a mediator between the volatile memory and persistent storage. This intermediary state allows safe data transfer by isolating the system from ongoing operations that could interfere with the metadata transfer process, ensuring reliable persistence without direct conflicts.
2Reliability
If system operations are suspended during startup stage, then metadata loss is prevented, but system productivity decreases
Solution Approach 1:
The system implements self-service by automatically detecting intermediate persistent storage failures and autonomously suspending operations to protect metadata. This self-protecting mechanism ensures metadata safety without requiring external intervention, while the suspension is automatically lifted when storage reliability is restored, minimizing productivity impact.
Solution Approach 2:
The system employs feedback mechanisms to monitor the status of intermediate persistent storage and dynamically adjust operations accordingly. When failure is detected, feedback triggers suspension; when health is restored, feedback signals resumption. This closed-loop control ensures metadata protection only when necessary, maintaining productivity during normal operation.
3Reliability
If metadata is transferred to persistent storage device, then metadata persistence is improved, but system complexity increases
Solution Approach 1:
The patent applies universality by making the suspension mechanism serve multiple functions: it protects metadata during failure, enables safe transfer to persistent storage, and can be triggered by different failure conditions. This multi-functional approach reduces the need for separate complex protection mechanisms, simplifying the overall system while improving reliability.
Data Source
AI summary
Techniques for storing metadata involve: storing first metadata associated with a system operation in a volatile memory of a storage system while the storage system is in a startup stage. The storage system does not perform read/write requests of a client during the startup stage. Such techniques further involve: suspending the system operation of the storage system during the startup stage in response to determining that the intermediate persistent storage device has failed. Such techniques further involve: storing the first metadata in the volatile memory into the persistent storage device. Such techniques further involve: storing the first metadata into the intermediate persistent storage device in response to determining that the intermediate persistent storage device is available again. Accordingly, the risk of losing metadata when the storage device is powered down can be reduced.