Asynchronous Storage Mapping Service for High TPS Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Remote storage systems face challenges in processing and updating mapping information efficiently, especially during high transaction-per-second bursts, leading to potential denial of storage requests or data handling errors due to insufficient computing resources for processing mapping information.
Innovation Solution
An asynchronous processing service is implemented to queue and process unique storage keys separately from the storage operations, allowing for asynchronous updating of mapping information, which enables remote storage systems to handle high TPS without immediate scaling of computing resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If computing resources are scaled to handle high TPS bursts for processing mapping information, then storage request handling capability is improved, but system complexity and resource cost increase
Solution Approach 1:
The patent segments the storage system into two independent processing paths: a fast path for storage operations that does not require mapping information processing, and a separate asynchronous path for mapping information processing. This segmentation allows storage requests to be handled quickly while mapping updates occur independently in the background, resolving the contradiction between handling high TPS and avoiding resource scaling complexity.
Solution Approach 2:
The patent implements preliminary action by pre-computing and caching mapping information in the fast path, and using asynchronous batch processing to update mapping information after storage operations complete. This allows the system to handle storage requests immediately without waiting for mapping updates, while still maintaining accurate mapping information through preliminary caching and subsequent async updates.
2Reliability
If mapping information is processed synchronously with storage operations, then data consistency is improved, but processing speed decreases
Solution Approach 1:
The patent uses preliminary action by pre-computing mapping information and caching it in the fast path before storage operations complete. This allows storage operations to proceed quickly using pre-prepared mapping data, while the actual mapping updates occur asynchronously after the fact, maintaining consistency without blocking fast path operations.
Solution Approach 2:
The patent introduces an intermediary mechanism - an asynchronous processing queue - that decouples the storage operation from the mapping update operation. The queue acts as a buffer that receives mapping update requests from the fast path and processes them asynchronously, allowing the fast path to continue processing storage requests without waiting for mapping updates, thus maintaining both speed and consistency.
3Loss of energy
If computing resources for mapping information processing are not scaled, then resource cost is reduced, but storage requests may be denied or throttled during high TPS
Solution Approach 1:
The patent segments processing into a resource-light fast path that handles storage operations without intensive mapping computation, and a separate async path for mapping updates. This segmentation allows the system to handle high TPS with minimal computing resources in the fast path, while mapping information processing occurs independently without competing for the same resources, thus maintaining request success rate without increasing resource consumption.
Solution Approach 2:
The patent implements preliminary action by pre-caching mapping information and using asynchronous batch processing to update mappings after storage operations. This allows the system to handle storage requests immediately using cached data without performing computationally intensive mapping updates in real-time, reducing resource consumption while maintaining reliable request handling through the async update mechanism.
Data Source
AI summary
Functionality is disclosed herein for providing an asynchronous processing service for processing storage mapping information. The asynchronous processing service is configured to receive a storage request including identification of a storage object and a description of a storage operation, perform the storage operation for the storage object in response to receiving the storage request, and asynchronously update mapping information for the performed storage operation.