Storage Node Direct Data Transfer Bypassing Controller Bottleneck
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In computer storage systems, data transmission between a host and a storage device is often slow due to the need for a controller to forward data, which can limit the data read/write rate and increase transmission time, especially when dealing with large amounts of data.
Innovation Solution
A data processing system and method where a controller connects to multiple storage nodes and directs them to transmit data directly with the host using connections such as RDMA, TCP/IP, or iSCSI, bypassing the need for the controller to forward data, thereby shortening the transmission path and reducing time consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is transmitted through the controller between host and storage device, then data transmission can be managed and controlled, but data transmission time increases and data read/write rate decreases
Solution Approach 1:
The patent introduces a bypass mechanism that allows data to transmit directly between host and storage device through dedicated data paths, while the controller maintains management authority through separate control channels. This intermediary approach preserves control functionality while eliminating the bottleneck of controller-mediated data forwarding.
Solution Approach 2:
The system segments communication into control channels (through controller) and data channels (direct path). Control requests and responses traverse the controller for management, while actual data transmission uses direct paths between host and storage devices, separating management functions from data transfer functions to improve performance.
2Extent of automation
If controller forwards all data, then centralized control is maintained, but controller bandwidth becomes insufficient for large data amounts
Solution Approach 1:
The architecture segments traffic into control plane (through controller) and data plane (direct paths). Control requests, authentication, and management commands flow through the controller maintaining centralized control, while bulk data transmission uses direct storage-to-host paths bypassing the controller's bandwidth limitations.
Solution Approach 2:
The patent adds a direct data path dimension parallel to the controller-mediated path. Instead of single-path controller-mediated communication, the system creates multiple dimensional paths: control dimension (through controller) and data dimension (direct paths), allowing simultaneous operation without interference.
3Extent of automation
If controller processes all data requests, then centralized management is achieved, but controller computing capability cannot meet processing requirements
Solution Approach 1:
The controller acts as an intermediary for control plane operations (authentication, request routing, monitoring) while delegating data plane processing to the storage devices themselves. Storage devices independently handle data read/write operations directly with hosts, reducing controller computational burden while maintaining centralized management oversight.
Solution Approach 2:
Storage devices are empowered to autonomously process data requests directly with hosts without requiring controller intervention for each operation. The devices self-manage data transfer, error handling, and protocol compliance, while the controller provides high-level management and coordination.
Data Source
AI summary
A data processing system and method, and a corresponding apparatus, where the data processing system includes a controller and at least two storage nodes. The controller is configured to receive, using a first coupling between the controller and a host, an operation request received from the host, where the operation request includes an identity of target data and an operation type, determine at least one target storage node from the at least two storage nodes according to the identity of the target data, and send an instruction message to the at least one target storage node using a second coupling to the at least one target storage node, where the at least one target storage node is configured to send the target data to the host or obtain the target data from the host according to the instruction message.