Space Accounting Counter Rebuilding in Storage Mapping Layers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Global data deduplication in storage appliances can adversely affect the performance of filesystem checking (FSCK) processing by increasing memory requirements and decreasing processing speed, particularly in large data deduplication domains.
Innovation Solution
The techniques involve uniquely associating top levels of the mapping layer with storage objects, determining logical and physical storage space consumption through mappings of logical block addresses to virtual blocks, and using multi-threading to dynamically adjust ranges, thereby achieving high processing parallelism while maintaining sufficient memory space for space accounting bookkeeping data structures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If global data deduplication is performed with a large data deduplication domain spanning multiple volume families, then data reduction is increased, but memory space required for space accounting bookkeeping data structures increases and processing speed decreases
Solution Approach 1:
The patent divides the large data deduplication domain into multiple smaller segments or partitions, allowing space accounting to be performed on each segment independently. This segmentation reduces the memory footprint of bookkeeping data structures while maintaining the overall data reduction benefits of global deduplication across multiple volume families.
Solution Approach 2:
The patent introduces a hierarchical or multi-dimensional approach to space accounting, organizing bookkeeping data in a structured hierarchy that allows efficient aggregation and summarization. This dimensional organization enables reduced memory requirements by storing only essential aggregation levels rather than all detailed information.
2Loss of substance
If global data deduplication is performed with a large data deduplication domain spanning multiple volume families, then data reduction is increased, but processing speed decreases
Solution Approach 1:
The patent segments the large data deduplication domain into smaller processing units that can be handled independently and in parallel. This segmentation enables faster processing by distributing the computational workload across multiple processing threads or cores, thereby improving processing speed while maintaining global data reduction benefits.
Solution Approach 2:
The patent performs preliminary actions such as pre-computing and caching frequently accessed space accounting information, or pre-organizing data structures before deduplication operations. This preliminary preparation reduces the computational burden during actual FSCK processing, thereby improving processing speed.
3Productivity
If multi-threading is used to determine amounts of logical storage space consumed by dynamically adjustable ranges, then processing parallelism is increased, but complexity of space accounting increases
Solution Approach 1:
The patent divides the storage space accounting task into multiple independent segments or ranges that can be processed by different threads simultaneously. Each thread handles a specific range with its own localized data structures, reducing inter-thread synchronization complexity while achieving high processing parallelism.
Solution Approach 2:
The patent implements dynamically adjustable ranges for multi-threaded processing, allowing the granularity and distribution of work segments to be adapted based on system conditions, data characteristics, and performance requirements. This dynamic adjustment optimizes parallelism while managing complexity through flexible, context-aware task distribution.
Data Source
AI summary
Techniques for rebuilding space accounting counters in mapping layers of storage appliances. The techniques include uniquely associating top levels of a mapping layer of a storage appliance with respective storage objects. The techniques further include determining amounts of logical storage space consumed by the respective storage objects from mappings of LBAs of the respective storage objects to virtual blocks of a virtual layer of the storage appliance. The techniques further include determining amounts of physical storage space consumed by the respective storage objects from logged information pertaining to each leaf pointer of a leaf level of the mapping layer that points to a virtual block in the virtual layer, each virtual block being mapped to a physical block in a physical layer of the storage appliance. The techniques further include using multi-threading to determine amounts of logical storage space consumed by dynamically adjustable ranges of the respective storage objects.