Data Relocation in Scale-Out Storage Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed storage systems face inefficiencies due to data units and their metadata entries being stored on separate nodes, leading to increased storage capacity and performance overhead, especially when data units are duplicated.
Innovation Solution
A system that relocates data units to the same node as their metadata entry when the number of versions meets or exceeds a threshold, allowing operations to interact with a single node, thereby reducing storage requirements and improving efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data units are distributed across multiple nodes in a distributed storage system, then scalability and redundancy are improved, but storage overhead and performance efficiency deteriorate due to separate metadata management
Solution Approach 1:
The patent combines data units and their metadata entries on the same node by relocating data units from remote nodes to the node hosting their metadata. This merging eliminates the need for separate metadata management and cross-node operations, directly reducing storage overhead while preserving distributed system scalability.
2Reliability
If data units and metadata entries are stored on separate nodes, then data distribution and fault tolerance are improved, but operation efficiency and performance deteriorate due to increased node interactions
Solution Approach 1:
The patent merges data units with their metadata on the same node, eliminating cross-node operations for metadata access. This reduces the number of node interactions required for storage operations, directly improving operation efficiency while fault tolerance is maintained through the distributed nature of the metadata itself across the network.
3Reliability
If multiple copies of data units are stored across nodes, then availability and redundancy are improved, but storage capacity requirements increase
Solution Approach 1:
The patent uses metadata entries that contain references or pointers to data unit locations instead of storing complete data copies. This allows the system to maintain availability through distributed references while significantly reducing actual storage capacity requirements, as only metadata (not full data copies) needs to be distributed across nodes.
Data Source
AI summary
Data units can be relocated in scale-out storage systems. For example, a computing device can receive, at a first node of a scale-out storage system, a request for a data unit. The first node can include a metadata entry associated with the data unit. The computing device can determine, based on the metadata entry, that a second node of the scale-out storage system includes the data unit. The computing device can determine, from the metadata entry, that a number of versions of the data unit in the scale-out storage system meets or exceeds a threshold. The computing device can output a command to cause the data unit to be relocated to the first node with the metadata entry.