Storage Controller RDMA Write Batching for Small Data Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face challenges in maintaining high throughput for small control data transfers over long distances using RDMA over Ethernet, particularly when numerous small control data transfers occur, leading to performance degradation.
Innovation Solution
A storage system architecture that accumulates control data in memory and collectively transfers multiple control data packets using a write request, optimizing the use of RDMA protocols to improve throughput by reducing the number of individual data transfers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Length of stationary object
If RDMA over Ethernet is used for long-distance transmission between storage controllers, then transmission distance is improved, but throughput deteriorates when numerous small control data transfers occur
Solution Approach 1:
The patent combines multiple small control data packets into a single collective transfer operation. The storage controller accumulates multiple control data items and transmits them together through one RDMA write request, rather than sending each packet individually. This merging approach reduces the overhead of multiple separate transmissions and improves throughput for small data transfers over Ethernet while maintaining long-distance transmission capability.
2Ease of operation
If individual control data packets are transferred separately, then data transfer simplicity is maintained, but performance deteriorates due to repeated small data transfers
Solution Approach 1:
The storage controller performs preliminary accumulation of control data packets before transmission. Multiple control data items are collected and buffered in memory, then transferred collectively in a single RDMA operation. This preliminary gathering action simplifies the transfer process by reducing the number of individual transmission operations while maintaining ease of data transfer through the accumulated batch operation.
Data Source
AI summary
A storage system includes a plurality of storage controllers. The storage controller includes a processor, a memory, and a transfer device that processes control data for controlling an internal operation of the storage system, the control data being transmitted and received between the plurality of storage controllers. The processor accumulates the control data in the memory when a transfer request for the control data is generated, generates a write request for transmitting a plurality of the control data stored in the memory, and transmits the write request to the other storage controller. The transfer device writes a plurality of the control data included in the write request to the memory upon receiving the write request.