Object Storage Key Interval Subdivision for Parallel Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Object storage systems face inefficiencies in retrieving keys associated with objects, particularly due to the unstructured nature of data storage and the need for fast and efficient data manipulation, which is not adequately addressed by existing methods.
Innovation Solution
A computer-implemented method that subdivides the key interval into subintervals using arithmetic computations based on character encoding, allowing for parallel scanning and optimization of key retrieval by adjusting intermediate numeric sequences to ensure validity and adherence to storage limits, while utilizing statistical information to refine the subdivision process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If sequential scanning of keys is performed in object storage system, then all keys are retrieved, but retrieval time is excessive and efficiency is low
Solution Approach 1:
The key interval is divided into multiple subintervals based on intermediate keys derived from numeric sequences. Each subinterval is scanned independently and in parallel, transforming a single sequential scanning task into multiple concurrent tasks, thereby significantly reducing total retrieval time while maintaining completeness of key collection
2Speed
If key interval is subdivided into subintervals for parallel scanning, then retrieval speed increases, but complexity of subdivision process increases
Solution Approach 1:
The patent replaces complex string-based key comparison and subdivision operations with numeric sequence arithmetic. By converting keys to numeric sequences and performing arithmetic computations (e.g., averaging) to generate intermediate keys, the system achieves simple, deterministic, and computationally efficient subdivision without complex logic
Solution Approach 2:
The patent transforms the key representation from character strings to numeric sequences, enabling arithmetic operations. This parameter change allows intermediate keys to be calculated through simple mathematical operations on numeric values, greatly simplifying the subdivision process while ensuring valid key generation within the specified interval
3Productivity
If parallel scanning of key subintervals is performed, then retrieval efficiency improves, but overlap between subintervals increases
Solution Approach 1:
The system employs a feedback mechanism where the last key retrieved from each parallel scanning task is communicated to other tasks. This feedback allows overlapping subintervals to dynamically adjust their scanning ranges to exclude already-retrieved keys, eliminating duplicates while maintaining the benefits of parallel execution
Data Source
Figure 1~2
Figure 3A
Figure 3B
AI summary
The present disclosure relates to a method for retrieving a listing of keys associated with respective objects stored in an object storage system supporting a remote object access protocol, the keys comprising respective strings of characters encoded by means of a character encoding, the method comprising: obtaining a start key and an end key for defining a key interval for the listing; subdividing the key interval into key subintervals based on one or more intermediate keys; scanning the key subintervals, thereby retrieving a listing of the keys in the key subintervals; and wherein the subdividing comprises: representing the start key and end key as a start and an end numeric sequence according to the character encoding; and calculating the one or more intermediate keys as one or more intermediate numeric sequences from the start and end numeric sequences by means of arithmetic computation.