Object Placement Control in Consistent Hashing Object Stores
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Consistent hashing-based object stores limit user and process control over object placement, leading to performance issues during rename operations and latency-sensitive processes due to static mapping between objects and nodes.
Innovation Solution
Implementing a two-component object namespace with a numerical namespace (NNS) and a lexicographical namespace (LNS) allows users or processes to specify node placement by using the LNS value for primary replicas and NNS value for additional replicas, enabling controlled object placement without disrupting existing operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If consistent hashing is used to map objects to nodes, then uniform distribution and fault tolerance are improved, but user control over object placement is lost
Solution Approach 1:
The patent segments the object namespace into two independent components: a numerical namespace (NNS) for consistent hashing-based distribution and a lexicographical namespace (LNS) for user-specified placement control. This segmentation allows each component to serve its specific function without interfering with the other, resolving the contradiction between automatic distribution and user control.
Solution Approach 2:
The patent adds a new dimension to the object namespace by introducing the LNS component alongside the traditional NNS. This dimensional expansion enables users to specify placement preferences in the LNS dimension while the system automatically handles distribution in the NNS dimension, thus maintaining both user control and system-managed fault tolerance.
2Reliability
If objects are copied during rename operations, then data integrity is maintained, but performance degrades due to potential remapping to different nodes
Solution Approach 1:
The patent performs preliminary actions by creating the LNS component during object creation or initial storage. This preliminary establishment of placement information allows subsequent rename operations to preserve the LNS value, avoiding the need for remapping and copying to different nodes, thus improving rename performance while maintaining data integrity.
3Speed
If a static mapping between objects and nodes is used, then lookup speed is improved, but adaptability to user placement requirements is reduced
Solution Approach 1:
The patent segments the namespace into NNS for fast consistent hashing-based lookups and LNS for flexible user-specified placement. This segmentation allows the system to maintain fast lookup performance through the static NNS mapping while simultaneously providing adaptability through the user-controllable LNS component.
Solution Approach 2:
The patent introduces dynamics into the previously static mapping by allowing the LNS component to be dynamically set by users. While the NNS provides static fast lookup, the LNS enables dynamic adaptation to user placement requirements, resolving the contradiction between speed and flexibility.
Data Source
AI summary
Techniques are disclosed herein for controlling object placement in object storage. A placement component of a storage application receives a request to store a first object in an object store having multiple nodes. The object store determines a placement of the first object to one of the nodes based on an object namespace including a numerical namespace and a lexicographical namespace. Each node is assigned a corresponding subspace of the object namespace for storing objects. The first object includes a numerical namespace value and a lexicographical namespace value. A second object (a replica of the first object) is generated. The first object is stored to a first node based on the lexicographical namespace value. The second object is stored to a second node based on the numerical namespace value.