System and method for remotely reading and writing secondary storage in network

Abstract

The invention provides a system and a method for remotely reading and writing secondary storage in a network. The system comprises a control module, a memory module, a cache module, a secondary storage device, a system bus module and a network interface device, wherein the network interface device is configured to send a received read / write request command for the secondary storage device to the memory module; the caching module is configured to store received data related to a write request and data to be transmitted related to a read request. The secondary storage device is configured to execute secondary storage device commands in the memory module, data interaction with the cache module is implemented via a system bus module to forward data related to a write request from the cache module to a secondary storage device via the system bus module or to forward data related to a read request from the secondary storage device to the cache module. The system and the method of the invention can improve the efficiency of remote data reading and writing and reduce the burden of the processor.

Description

technical field [0001] The invention relates to the technical field of network storage, in particular to a system and method for remotely reading and writing secondary storage through a network. Background technique [0002] Internet services are rich and diverse. Applications such as big data, machine learning, and graph computing all need to read and write data in secondary storage devices on different nodes in the system through the network. The read and write performance of remote data (such as bandwidth and latency) has a significant impact on the overall performance of the application. [0003] In the prior art, there are problems of low data processing efficiency and resource utilization in the network remote reading and writing system. Typically, the network reading and writing system in the prior art includes a network interface device for receiving network remote reading and writing , the control module, memory space, and secondary storage device implemented by th...

AI Technical Summary

Problems solved by technology

[0003] In the prior art, there are problems of low data processing efficiency and resource utilization in the network remote reading and writing system. Typically, the network reading and writing system in the prior art includes a network interface device for receiving network remote reading and writing , the control module, memory space, and secondary storage device implemented by the CPU, no matter for the read command or the write command, the data needs to be temporarily stored in the "read and write data cache space" of the memory, which increases the network interface device and the secondary storage device. The number of copies of the storage device during the data exchange process increases the data read and write delay, and also wastes valuable CPU processing time and memory space

In addition, in the existing remote network reading and writing system, the control information path overlaps with the data path, both of which are processed by the control module (i.e., the CPU), resulting in a large number of tasks for the control module and complicated processing procedures. , puts a lot of pressure on the processor and increases processing time

[0004] At present, there are two main methods to improve the performance of remote data read and write: one is to reduce the processing time of remote read and write commands by increasing the computing power of the general-purpose processor in the node. Small size, good forward compatibility, but high cost and power consumption; the other type is to embed special components on the data path to improve the data handling efficiency between the network interface device and the secondary storage device, for example, the existing Mellanox InfiniBand The card is equipped with an nvme over fabrics offloading engine, which can realize remote reading and writing of high-bandwidth, low-latency secondary storage devices, but its data path and control path are completely implemented by Mellanox InfiniBand cards, the hardware design is complicated, and the implementation cost is high

Images