Distributed Storage Passthrough Operations for Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed data storage systems face performance degradation due to increased data access requests from multiple hosts and the limitations of memory with a limited service life, particularly when upstream controllers manage and process data access requests, leading to bottlenecks and inefficiencies.
Innovation Solution
The system bypasses upstream network and enclosure controllers to allow data storage devices to independently process, schedule, and execute data access requests and policy operations, utilizing their high processing and bandwidth capabilities, and employing in-drive mutex to synchronize and optimize request servicing across multiple hosts and logical volumes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If upstream controllers process data access requests, then centralized management is achieved, but performance degradation and bottlenecks occur
Solution Approach 1:
The patent segments the request processing function from the upstream network/enclosure controllers and assigns it to individual data storage devices. Each device independently processes received requests using its own controller, eliminating the bottleneck at the upstream controller while maintaining system-wide coordination through the passthrough mechanism.
Solution Approach 2:
The patent extracts the request processing logic from the upstream controllers and relocates it to the data storage devices themselves. This extraction removes the performance bottleneck at the upstream controller level while distributing the processing load across multiple storage devices, thereby improving overall system productivity.
2Adaptability or versatility
If multiple hosts access data storage devices simultaneously, then system versatility is improved, but request synchronization and performance degradation occur
Solution Approach 1:
Each data storage device independently manages its own request queue and processing without requiring coordination from upstream controllers. The devices self-service by autonomously handling request synchronization, priority management, and execution, which improves efficiency while supporting multiple hosts simultaneously.
Solution Approach 2:
The system implements dynamic request handling where each storage device adapts its processing based on current load conditions and request priorities. The passthrough mechanism allows flexible routing of requests from multiple hosts to appropriate devices, with each device dynamically adjusting its processing schedule to optimize throughput.
3Reliability
If upstream controllers manage all data access requests, then centralized control is maintained, but latency increases
Solution Approach 1:
The patent extracts request processing functionality from upstream controllers and embeds it directly in data storage devices. This extraction eliminates the intermediate processing step at the upstream controller, thereby reducing latency while maintaining centralized control through the passthrough architecture that routes requests through the upstream controller for scheduling purposes.
Solution Approach 2:
The upstream controller acts as an intermediary that receives requests from hosts and passes them through to the appropriate data storage devices without processing them. This mediator role maintains centralized control and request routing while allowing the actual processing to occur at the storage devices, minimizing latency.
Data Source
AI summary
A distributed data storage system can have a plurality of data storage devices connected to a host via a network controller. Receipt of a data access request from the host with the network controller can result in passing the data access request to a predetermined data storage device of the plurality of data storage devices with the network controller without processing the data access request. The data access request may then be satisfied with a device controller of the predetermined data storage device.