Storage Element Cloning with Dynamic In-Memory Mapping Tablets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional storage mapping techniques face issues with metadata write amplification and contention, leading to reduced throughput, increased latency, and premature wear on storage media, limiting the performance and reliability of virtual volume systems.
Innovation Solution
The implementation of a mapping apparatus that includes a pre-mapper with a dynamic in-memory ingress tablet and persisted frozen tablets, utilizing a Cuckoo Filter and Broad Operation Database to reduce metadata write amplification and contention, allowing for efficient data storage and retrieval while minimizing metadata writes and maintaining data integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional storage mapping techniques are used, then mapping functionality is provided, but metadata write amplification increases and contention occurs
Solution Approach 1:
The mapping system is segmented into multiple mapping tablets (ingress tablet, frozen tablets, clone tablets) that distribute mapping entries across separate data structures. This segmentation allows parallel access and reduces contention on any single tablet, thereby reducing metadata write amplification while maintaining mapping functionality and improving throughput.
Solution Approach 2:
The patent introduces a temporal dimension to mapping tablets through versioning and freezing mechanisms. Mapping tablets transition from active (ingress) to frozen states, allowing concurrent reads on frozen tablets while writes occur on ingress tablets. This dimensional transformation eliminates write-contention and reduces metadata write amplification.
2Productivity
If conventional storage mapping techniques are used, then mapping functionality is provided, but latency increases
Solution Approach 1:
Mapping entries are pre-computed and stored in frozen tablets before being needed for read operations. The ingress tablet pre-processes write operations and prepares mapping entries in advance, allowing read operations to directly access pre-computed mappings in frozen tablets without computation delays, thereby reducing latency while maintaining high throughput.
Solution Approach 2:
The patent creates copy structures (frozen tablets as copies of ingress tablet states) that can be accessed concurrently without blocking writes. This copying mechanism allows read operations to access static copies while write operations modify the current ingress tablet, reducing operational latency and improving overall throughput.
3Reliability
If conventional storage mapping techniques are used, then mapping operations are performed, but wear on physical storage media increases
Solution Approach 1:
The patent extracts frequently accessed mapping entries from persistent storage into in-memory mapping tablets. By keeping active mapping data in volatile memory and only persisting frozen tablets to storage, the system reduces the number of write operations to physical storage media, extending media lifetime while maintaining high throughput for active mappings.
Solution Approach 2:
The patent implements a lifecycle management system where mapping tablets are frozen, persisted to storage, and then the original ingress tablet is discarded or reused. This discard-and-recover pattern allows the system to reclaim in-memory resources and reduce persistent storage writes, extending storage media lifetime without compromising throughput for active operations.
Data Source
AI summary
A mapping apparatus comprises a mapper that translates from an input key to an output key in one or more storage devices. A pre-mapper for processing update operations comprises a plurality of mapping tablets and an in-memory look-up filter to identify a given mapping table storing a given input key. The mapping tablets comprise at least one dynamic in-memory ingress tablet and a plurality of persisted frozen tablets. For a given update operation, a first entry is added to one dynamic in-memory ingress tablet comprising the input key for the given update operation and a corresponding output key where data for the given update operation is stored; and a second entry is added to the look-up filter comprising the input key of the first entry and an identifier of the dynamic in-memory ingress tablet storing the first entry for the given update operation. The dynamic in-memory ingress tablet is persisted as a persisted frozen tablet. A given storage element is cloned by making a copy of the given storage element and assigning a new internal identifier to the given storage element and the copy.