Segment Cleaning Algorithm Using Identifier Copying
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing segment cleaning mechanisms in log-structured file systems for object storage are inefficient due to the need for extensive metadata updates and additional compute resource usage, especially when dealing with large datasets like virtual machine disks, as they require updating physical block addresses in metadata tables during the cleaning process.
Innovation Solution
The proposed solution involves storing an identifier associated with the segment instead of physical block addresses, allowing for efficient segment cleaning by creating a new segment with the same identifier, thus eliminating the need for additional metadata computations and reducing the requirement for metadata updates during the cleaning process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional segment cleaning mechanisms are used to clean segments in log-structured file systems, then data cleaning functionality is provided, but computational overhead increases and metadata updates are required
Solution Approach 1:
The patent creates a copy of the segment with the same identifier instead of modifying the original segment in place. This copying approach allows the cleaning process to generate a new segment with live blocks while preserving the original segment structure, thereby avoiding complex metadata updates and reducing computational overhead during the cleaning operation.
Solution Approach 2:
The patent divides the segment cleaning process into distinct phases: identifying live blocks, creating a new segment with the same identifier, copying only the live blocks to the new segment, and then deleting the original segment. This segmentation of the cleaning process enables efficient execution by processing only necessary blocks and avoiding full segment reprocessing.
2Reliability
If physical block addresses are stored in metadata tables during segment cleaning, then data location tracking is maintained, but additional metadata computations and updates are required
Solution Approach 1:
The patent uses a copy approach where a new segment is created with the same identifier as the original segment. This allows the metadata tables to continue referencing the same identifier without requiring updates to physical block addresses. The live blocks are copied to the new segment, and the original segment is deleted, maintaining data location tracking accuracy while eliminating the need for metadata computations and updates during the cleaning process.
3Reliability
If extensive metadata updates are performed during segment cleaning, then data integrity is maintained, but computational resources are consumed
Solution Approach 1:
The patent employs a copying mechanism where a new segment is created with the same identifier instead of performing extensive metadata updates. This approach maintains data integrity by preserving the mapping between identifiers and data locations while significantly reducing compute resource usage. The cleaning process only needs to copy live blocks to the new segment rather than updating all metadata entries, thereby reducing energy consumption while maintaining reliability.
Solution Approach 2:
The patent discards the original segment after copying the necessary live blocks to the new segment. This discarding approach simplifies the cleaning process by eliminating the need to update metadata for the entire original segment. The system recovers only the essential live blocks and discards the rest of the original segment, reducing computational resources required for metadata maintenance while preserving data integrity.
Data Source
AI summary
A method for cleaning an object storage having a plurality of segments is provided. Each segment includes an identifier through which the segment is accessed. The method identifies a first segment in the plurality of segments. The first segment includes a first identifier and a first size. The method determines that a utilization ratio for the first segment is below a threshold. As a result, the method generates a second segment from the first segment, such that the second segment includes a second identifier that is the same as the first identifier and a second size that is smaller than the first size. The method then writes the second segment to the object storage.