Zero-Copy Volume Moves Through Global PVBN Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing distributed storage systems suffer from sub-optimal infrastructure architectures that do not effectively utilize compute, network, and storage resources, leading to storage silos and inefficient data movement between nodes, which can cause bottlenecks and increased load on nodes during data redistribution.
Innovation Solution
Implement a distributed storage system with a storage pod that allows zero-copy volume moves by maintaining volume data in place and copying only metadata, enabling constant-time operations and eliminating storage silos through the use of dynamically extensible file systems that have visibility into a global physical volume block number space, allowing independent file system operations and reduced internode communications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If volume data is copied during node migration in traditional distributed storage systems, then data integrity is maintained, but network bandwidth is consumed and migration time increases proportionally with volume size
Solution Approach 1:
The patent applies the copying principle by creating a metadata copy (container file copy) instead of copying the actual volume data. The container file metadata is copied from source to destination node, while the volume data remains in place on the storage pool, achieving constant-time migration independent of volume size while maintaining data integrity through proper metadata updates and ownership transfer protocols
2Reliability
If volume data is copied during migration, then destination nodes receive complete data, but network infrastructure becomes a bottleneck and load increases on source nodes
Solution Approach 1:
The patent extracts only the essential metadata (container file) from the volume structure and transfers it to the destination node, leaving the bulk volume data in place on the storage pool. This extraction approach eliminates network bandwidth consumption for data transfer while ensuring data completeness through proper metadata that references the unchanged data blocks
Solution Approach 2:
The storage pool acts as an intermediary that maintains the volume data in place while metadata updates orchestrate the migration. The storage pool's global PVBN space and allocation area mechanisms enable the destination node to access the same data blocks through updated metadata pointers, eliminating the need for data copying while ensuring data completeness
3Speed
If each node has dedicated storage space (node-level aggregates), then local access performance is improved, but storage utilization efficiency decreases due to storage silos
Solution Approach 1:
The patent implements universality by allowing the storage pool to serve multiple nodes simultaneously with a single global PVBN space. The storage pool can be accessed by any node in the cluster through proper metadata updates and ownership transfer, enabling the same storage resources to be dynamically allocated to different nodes based on workload requirements while maintaining fast local access through cached metadata
Solution Approach 2:
The patent introduces dynamics by enabling flexible volume migration between nodes without data copying. Volumes can be moved from source to destination nodes by updating container file metadata and allocation area ownership, allowing storage resources to be dynamically reallocated based on system needs while maintaining high access performance through the storage pool's architecture
4Ease of operation
If volume metadata is copied during migration, then migration operation is simplified, but metadata consistency across nodes may be compromised
Solution Approach 1:
The patent implements feedback mechanisms through ownership transfer protocols and metadata update verification. When a volume is migrated, the system tracks the transfer of container file ownership and allocation area ownership between nodes, ensuring that metadata consistency is maintained across the distributed system. The storage pool monitors and updates metadata state to prevent inconsistencies while keeping the migration process simple
Data Source
AI summary
Systems and methods for performing a zero-copy volume move between nodes of a distributed storage system are provided. In one example, an approach for performing a zero-copy volume move is proposed in which volume data may be maintained in place within a storage pod and need not be copied to move a given volume between the source node and the destination node. In one embodiment, file system metadata (e.g., a top-most physical volume block number (PVBN) of a node tree representing the volume at issue) of a write-anywhere file system is copied from the source node to the destination node. Since the storage pod is associated with a global PVBN space that is visible to all nodes of the distributed storage system, as a result of copying the top-mode PVBN of the volume to the destination node, anything below the top-most PVBN will automatically be visible to the destination node.