RDP Metadata Space Allocation for IO Thread Contention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The frequent creation of snapshots in data storage systems leads to significant resource burdens, particularly in memory allocation for replication data pointers (RDPs), causing contention among memory consumers due to the need for frequent allocation of memory space.
Innovation Solution
A system comprising shared memory and multi-core processors with uniquely associated IO threads, where each IO thread is allocated no more than one RDP page at a time, with RDP pages configured to accommodate metadata for targetless snapshots, and a method for allocating and recycling these pages to reduce latency and contention by using ranked lists of unallocated pages.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If snapshots are created frequently with targetless snapshots using RDPs, then data protection capability is improved, but memory allocation contention increases
Solution Approach 1:
The shared memory space for RDP metadata is segmented into multiple RDP pages, each allocated to a specific IO thread. This segmentation isolates memory access patterns of different threads, preventing contention while maintaining frequent snapshot creation capability across multiple storage objects.
Solution Approach 2:
The patent introduces a new dimension of thread-local memory allocation by assigning dedicated RDP pages to each IO thread. This transforms the traditional single-shared memory model into a multi-dimensional allocation scheme where each thread operates in its own memory subspace, eliminating contention while preserving data protection functionality.
2Productivity
If memory space is frequently allocated for RDPs in targetless snapshots, then snapshot creation speed is improved, but latency due to allocation contention increases
Solution Approach 1:
RDP pages are pre-allocated and maintained in a pool structure before being needed. When an IO thread needs to create a snapshot, it can immediately obtain a pre-prepared page from the pool without undergoing allocation contention, thereby maintaining high snapshot creation speed while eliminating allocation latency.
Solution Approach 2:
Each IO thread maintains its own dedicated RDP page and manages its own metadata allocations within that page. This self-service approach eliminates the need for threads to compete for shared memory resources, allowing each thread to operate independently and maintain high productivity without allocation delays.
3Quantity of substance
If multiple IO threads share the same memory space for RDP metadata, then memory utilization is improved, but access contention increases
Solution Approach 1:
The shared memory space is divided into separate RDP pages, with each page assigned to a specific IO thread. This segmentation allows multiple threads to simultaneously use memory without interfering with each other, maintaining high memory utilization across the system while eliminating access contention through spatial isolation.
Data Source
AI summary
Memory management processes allocate and recycle pages of replication data pointer (RDP) metadata space in shared memory. When the RDP page currently allocated to an IO thread becomes full, that RDP page is released, and a new RDP page is allocated to the IO thread. The released page eventually becomes fragmented and is added to a list of RDP pages that are ranked based on partial fullness. An IO thread that needs a new RDP page is allocated a mostly empty RDP page from the ranked list, if such a page is available. Otherwise, a new completely empty RDP page is allocated to the IO thread. Use of the ranked lists reduces latency associated with allocation of new RDP pages. Contention between IO threads for RDP metadata space is reduced because each IO thread has exclusive rights to the free RDP metadata space in its currently allocated page.