Object-Based Storage Queue Parallelization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current document storage systems, particularly file-based systems, face challenges such as poor scalability, limited metadata capacity, sequential processing of operations, high access costs, and cache memory overload, which hinder efficient document management and access.
Innovation Solution
The implementation of a computer-implemented method and system for object-based data storage, which involves maintaining a queue of operations, identifying independent operations to enable parallel processing, and processing these operations in parallel to improve efficiency and scalability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If operations are processed sequentially in object-based storage systems, then processing simplicity is maintained, but processing speed and efficiency deteriorate
Solution Approach 1:
The patent segments the queue of operations into multiple independent parallel processing chains. Each chain can process operations simultaneously, dividing the overall processing workload into concurrent segments that improve throughput without requiring complete system redesign
Solution Approach 2:
The system performs preliminary analysis of the operation queue to identify independent operations that can be processed in parallel. By pre-processing the queue to detect parallelization opportunities, the system prepares the groundwork for efficient concurrent execution without adding complexity during runtime
2Speed
If cache memory is used to store documents going to or coming from object-based storage, then access speed is improved, but cache memory becomes overburdened resulting in slowdowns
Solution Approach 1:
The patent extracts frequently accessed documents from the main storage queue and places them in a dedicated cache memory structure. This separation removes the burden of managing frequently accessed items from the general storage system, allowing the cache to optimize for speed while the main storage maintains overall system integrity
Solution Approach 2:
The cache memory acts as an intermediary layer between the processing system and the object-based storage. It buffers and manages document access requests, absorbing peaks in demand and preventing direct overload on the main storage system, thereby maintaining reliable performance
3Adaptability or versatility
If file-based storage systems are used, then implementation simplicity is maintained, but scalability deteriorates when workload increases
Solution Approach 1:
The patent implements a universal processing framework that can handle both traditional file-based operations and modern object-based storage operations. This multi-functional system allows seamless transition and integration, enabling scalability without requiring complete system replacement or complex dual-system management
Data Source
AI summary
Systems and methods for object-based data storage are provided. A queue may be maintained of operations relating to a plurality of documents operable to be maintained at an object-based data storage. An independent operation may be identified in the queue that must be processed prior to processing at least one dependent operation to thereby enable parallelization of processing of operations in the queue. The identified independent operation may then be processed. Subsequently, the dependent operations may be processed.