Data Storage Scale-Out via Local Address Remapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing scale-out data storage solutions face heavy overhead when adding new storage nodes, limiting the efficient growth of storage capacity and performance in network disaggregated data storage systems.
Innovation Solution
The implementation of direct block placement algorithms and logical address compaction methods allows for efficient scaling of storage capacity and performance by mapping and remapping data segments across multiple storage devices, minimizing data copying and movement during node additions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional scale-out solutions are used to add storage nodes, then storage capacity can be increased, but heavy overhead is introduced in terms of data copying and system complexity
Solution Approach 1:
The patent segments the global logical address space into multiple segments, with each segment further divided into sub-segments. This segmentation allows data to be distributed across storage nodes in a structured manner, enabling scale-out without requiring complex global data movement. Each storage device maintains a local logical address space that maps to its portion of the global address space, reducing the complexity of address management during scale-out operations.
Solution Approach 2:
The patent introduces a hierarchical address mapping dimension by creating multiple levels of mapping: global logical address space → segments → sub-segments → storage devices. This dimensional approach allows the system to scale out by adding storage nodes while maintaining manageable address translation at each level, avoiding the need for complex global remapping operations.
2Productivity
If data is redistributed across added storage devices, then capacity balancing is optimized, but significant data movement is required
Solution Approach 1:
The patent performs preliminary segmentation of the global logical address space into segments and sub-segments before scale-out operations. This preliminary structuring allows new storage devices to be integrated by simply assigning them specific segments, rather than requiring redistribution of existing data. The segmentation is established in advance, enabling efficient capacity balancing without significant data movement during scale-out.
Solution Approach 2:
The patent extracts specific sub-segments from the global address space and assigns them to new storage devices. Rather than moving existing data, the system identifies and extracts unused or underutilized segments, then maps these to new storage nodes. This extraction approach enables capacity balancing while minimizing data movement, as only metadata mappings need to be updated rather than physical data transfer.
3Productivity
If existing scale-out methods are used, then storage performance can grow with added nodes, but heavy overhead limits efficient growth
Solution Approach 1:
The patent implements local quality by allowing each storage device to maintain its own local logical address space and segment mapping independently. Each storage node can manage its assigned segments autonomously, with local address translation tables that map to its specific portion of the global address space. This local autonomy reduces the overhead of centralized address management and enables efficient scale-out, as each node operates independently within its assigned segment range.
Solution Approach 2:
The patent creates a universal segment structure that can be applied regardless of the number of storage nodes. The same segmentation methodology and address mapping approach works for both scale-up and scale-out scenarios, making the system versatile. The hierarchical segment structure serves multiple functions: address translation, data distribution, and capacity management, reducing the need for separate mechanisms and thereby reducing overall system overhead.
Data Source
AI summary
A system and method improve the performance of non-volatile memory storage by automatically, when one or more data storage devices are added to a first set of storage devices in a storage system, resulting in a second set of storage devices, remapping data stored in the first set of storage devices so as to redistribute data across the second set of storage devices while minimizing the amount of data moved to the newly added storage devices. In addition, the remapping and redistribution of data results in empty logical address regions in data storage devices from which data is copied, and a logical address compaction operation is used to remap one or more logical address ranges so as to eliminate the empty logical address regions, without moving data corresponding to the remapped logical address ranges.