Metadata Garbage Collection via Backup Check Markers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In storage systems that maintain geographic consistency, the inefficiency of garbage collection is due to the need to wait for full backups of user data and metadata in remote devices, leading to prolonged retention of useless metadata as garbage, which occupies resources and reduces resource utilization.
Innovation Solution
The solution involves recording the location of log information associated with each operation in metadata, allowing for early determination of backed-up user data and metadata through a check marker, enabling immediate marking as removable metadata without waiting for metadata integration tasks to complete.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the system waits for full backup completion before garbage collection, then geographic consistency is maintained, but garbage collection efficiency deteriorates
Solution Approach 1:
The system performs preliminary actions by recording log information locations in metadata and using check markers to track backup progress. This allows the system to determine in advance which metadata can be safely removed without waiting for complete backup verification, thus improving garbage collection efficiency while maintaining geographic consistency through the preliminary tracking mechanism.
2Reliability
If metadata is retained until backup completion, then data safety is ensured, but resource utilization deteriorates
Solution Approach 1:
The system implements feedback through check markers that continuously track the backup status of user data and metadata. This feedback mechanism allows the system to dynamically determine when metadata has been successfully backed up and can be removed, enabling timely resource release while ensuring data safety through continuous status monitoring and verification.
3Stability of the object's composition
If garbage collection is delayed until backup completion, then consistency is maintained, but time consumption increases
Solution Approach 1:
The system segments the garbage collection process by dividing metadata into groups that can be independently evaluated for removal. Using log information locations and check markers, the system can identify and remove specific metadata segments that have been successfully backed up, rather than waiting for complete backup of all data. This segmentation approach maintains consistency for backed-up portions while reducing overall time consumption.
Data Source
AI summary
Embodiments of the present disclosure provide a method, an electronic device and a computer-readable medium for managing metadata. The method disclosed herein comprises: obtaining a check marker from a remote device; determining whether the check marker indicates that user data and metadata have been backed up in the remote device, the user data and the metadata being generated by an operation executed by a user on an object; and in response to the check marker indicating that the user data and the metadata have been backed up in the remote device, marking the metadata as removable metadata. In this way, the embodiments of the present disclosure can improve garbage collection efficiency of a storage system and execution efficiency of a metadata integration task.