Data transmission method and device for computing cluster and storage medium

Abstract

The invention relates to a data transmission method and device for a computing cluster and a storage medium, and the method comprises the steps: carrying out the unified addressing of the storage space of the storage medium of a plurality of nodes, and obtaining the addressing information; constructing a cache space in the storage space of each node of the plurality of nodes; responding to a data access request initiated by an application program in any target node in the plurality of nodes, and determining demand data; determining a first node where the demand data is located and a target physical address according to the demand data; when the first node is not the target node, querying whether the demand data is hit in a local cache space of the target node according to the target physical address; and under the condition that the local cache space does not hit the demand data, transmitting the demand data in the first node to the local cache space of the target node, so that the application program obtains the demand data from the local cache space of the target node.

Description

technical field [0001] The present disclosure relates to the field of information transmission, and in particular, to a data transmission method, device and storage medium for computing clusters. Background technique [0002] At present, with the rapid development of computing hardware and high-speed networks, the construction of multi-node heterogeneous computing clusters has received extensive research and attention. However, despite the rapid development of network hardware and communication protocols, there is still a gap in bandwidth and delay between network transmission and local PCI bus transmission; how to improve cross-node data transmission capabilities and improve the overall computing performance of the cluster is the current cluster An important direction for performance optimization. [0003] In cross-node clusters, current network transmissions use a variety of methods to achieve and improve data transmission performance. However, in related technologies, t...

AI Technical Summary

Problems solved by technology

However, in related technologies, the direct memory data copy method requires the CPU to intervene in the processing of the network software stack, which brings additional performance overhead and poor performance; RDMA technology can effectively realize high-speed cross-node data communication, but cannot solve heterogeneous Communication problems with hardware (such as GPU), etc., there is a problem of poor flexibility in cross-node data access

Images