Transmission method, electronic equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, and provides a transmission method, electronic equipment and a storage medium, and the method comprises the steps: obtaining to-be-transmitted data composed of a plurality of data fragments, the plurality of data fragments being obtained through parallel calculation of a plurality of local computing devices; based on the multiple transmission links, the multiple data fragments are transmitted to the opposite-end host in parallel, so that the opposite-end host distributes target data to the multiple opposite-end computing devices, and the target data is composed of the multiple received data fragments. According to the method, the electronic device and the storage medium provided by the invention, the plurality of data fragments obtained by parallel calculation of the plurality of home terminal computing devices are transmitted in parallel to the opposite terminal host through the plurality of transmission links, and then the data fragments are distributed to the plurality of opposite terminal computing devices by the opposite terminal host, so that multi-link data transmission between the home terminal computing devices and the opposite terminal computing devices is realized; therefore, the transmission delay between the home terminal computing device and the opposite terminal computing device is greatly reduced, and the transmission efficiency between the home terminal computing device and the opposite terminal computing device is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a transmission method, electronic equipment and storage medium. Background Art

[0002] With the rapid development of artificial intelligence technology, especially the rise of large language models (LLM), the demand for computing resources has exploded. The scale of these models, the scale of training data, and the number of GPU (Graphics Processing Unit) resources required are all growing at an exponential rate.

[0003] In some cases, thousands or even tens of thousands of GPUs may be needed to meet training needs. However, in the current GPU cloud service and resource usage environment, these tens of thousands of GPUs may come from different manufacturers or belong to different product models of the same manufacturer, with different hardware architectures, that is, they are heterogeneous.

[0004] In this context, the mixed use of GPUs of different models and manufacturers to support large-scale model training is called heterogeneous mixed training (also known as "heterogeneous training"). Heterogeneous mixed training can effectively solve the "computing power island" problem of GPUs in data centers.

[0005] The communication efficiency between GPUs of different manufacturers or models directly affects the overall efficiency of heterogeneous mixed training. How to improve the communication efficiency between heterogeneous GPUs to achieve efficient interconnection between heterogeneous GPUs has become a hot research topic in the industry. Summary of the invention

[0006] The present invention provides a transmission method, an electronic device and a storage medium, which are used to solve the defect of low communication efficiency between heterogeneous GPUs in the related art.

[0007] The present invention provides a transmission method, which is applied to a local host. The transmission method comprises: Acquire data to be transmitted consisting of a plurality of data slices, wherein the plurality of data slices are obtained by parallel calculation by a plurality of local computing devices; Based on multiple transmission links, the multiple data slices are transmitted in parallel to the opposite host, so that the opposite host distributes the target data to multiple opposite computing devices, and the target data is composed of the received multiple data slices.

[0008] According to a transmission method provided by the present invention, the multiple transmission links correspond one-to-one to the multiple local computing devices, and each of the transmission links is used to transmit data slices calculated by the corresponding local computing device.

[0009] According to a transmission method provided by the present invention, the multiple data fragments are transmitted in parallel to the opposite host based on multiple transmission links, and the method also includes: Based on the local network card corresponding to each of the local computing devices, a transmission link corresponding to each of the local computing devices is established.

[0010] According to a transmission method provided by the present invention, the transmission link corresponding to each local computing device is established based on the local network card corresponding to each local computing device, and the method also includes: Based on the connection relationship and communication bandwidth between each of the local computing devices and each of the local network cards, a local network card corresponding to each of the local computing devices is determined.

[0011] According to a transmission method provided by the present invention, the transmission link corresponding to each local computing device is established based on the local network card corresponding to each local computing device, including: Based on the local network card corresponding to each of the local computing devices and the opposite network card of the opposite computing device corresponding to each of the local computing devices, a transmission link corresponding to each of the local computing devices is established.

[0012] According to a transmission method provided by the present invention, the multiple data fragments are transmitted in parallel to the opposite host based on multiple transmission links, including: Based on the multiple transmission links, the multiple data slices are transmitted in parallel to the opposite host through remote direct memory access technology.

[0013] According to a transmission method provided by the present invention, the multiple data fragments are transmitted in parallel to the opposite host based on multiple transmission links, including: Based on multiple transmission links, transmit the data blocks of the multiple data slices to the opposite host in parallel; The size of the data block of each of the data slices is the target size of the transmission link for transmitting each of the data slices.

[0014] According to a transmission method provided by the present invention, the multiple data fragments are transmitted in parallel to the opposite host based on multiple transmission links, and the method also includes: Divide the test data into multiple data blocks of candidate sizes; For each of the transmission links, based on the transmission link, multiple data blocks of the candidate size are transmitted to the opposite host, and the data transmission time of the candidate size is obtained; Based on the data transmission time consumption under the multiple candidate sizes, a target size under the transmission link is determined.

[0015] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned transmission methods when executing the computer program.

[0016] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the transmission method described above is implemented.

[0017] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the transmission methods described above.

[0018] The transmission method, electronic device and storage medium provided by the present invention transmit multiple data slices obtained by parallel calculation of multiple local computing devices to the opposite host through multiple transmission links, and then distribute them to multiple opposite computing devices by the opposite host to realize multi-link data transmission between the local computing device and the opposite computing device, thereby greatly reducing the transmission delay between the local computing device and the opposite computing device, and improving the transmission efficiency between the local computing device and the opposite computing device. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a structural diagram of a computing system based on a heterogeneous GPU cluster in the related technology.

[0021] Figure 2 It is a schematic diagram of the communication process between GPUs in the related technology.

[0022] Figure 3 This is one of the flow charts of the transmission method provided by the present invention.

[0023] Figure 4 This is the second flow chart of the transmission method provided by the present invention.

[0024] Figure 5 This is the third flow chart of the transmission method provided by the present invention.

[0025] Figure 6 It is a schematic diagram of the structure of the transmission device provided by the present invention.

[0026] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] As the parameter scale and training data volume of large language models increase, the demand for storage resources and hardware computing power has exceeded the upper limit that a single computing device can provide. In order to break through this limitation and improve the training throughput at the same time, a 3D parallel training strategy is often adopted, that is, distributed collaborative training of the model is implemented through data parallelism, tensor parallelism, and pipeline parallelism. Here, distributed collaborative training refers to the use of collaborative cooperation between multiple computing devices to jointly participate in the training process of the model to achieve model training acceleration and performance improvement. It should be noted that the computing device here is an artificial intelligence chip, such as a GPU, TPU (Tensor Processing Unit), NPU (Neural network Processing Unit), DPU (Deep learning Processing Unit), APU (Accelerated Processing Unit), and GPGPU (General-Purpose computing on Graphics Processing Unit), etc., and the embodiment of the present invention does not specifically limit this.

[0029] In distributed collaborative training, thousands or even tens of thousands of GPUs may be needed to meet training needs. Traditional distributed training requires completely identical GPU devices (same model from the same manufacturer), and insufficient resources become a bottleneck for large-scale model training, especially large language model training.

[0030] In the current GPU cloud service and resource usage environment, these tens of thousands of GPUs may come from different manufacturers or belong to different product models of the same manufacturer, with different hardware architectures, that is, they are heterogeneous. In the embodiment of the present invention, a cluster composed of GPUs with different hardware architectures is called a heterogeneous GPU cluster.

[0031] For example, Figure 1It is a schematic diagram of the structure of a computing system based on a heterogeneous GPU cluster in the related technology. Figure 1 As shown, the system may include a host 110 and a device end 120, wherein the device end 120 may include multiple first-type devices, multiple second-type devices, and other types of devices, where the devices can all be regarded as GPUs or other types of artificial intelligence chips. The host 110 is connected to multiple devices on the device end 120 to control these devices to perform various computing tasks and collaboratively complete the training of the model. On the device end 120, multiple first-type devices are all devices of the same manufacturer and the same model, and these devices constitute a homogeneous device cluster, while the first-type devices and the second-type devices may be devices of different manufacturers or different models, and these devices constitute a heterogeneous device cluster. Heterogeneous mixed training for the model is to use these different types of devices (including homogeneous and heterogeneous devices) to jointly participate in the training task of the model during the model training process. These devices may have different computing capabilities, memory sizes, data formats, and other characteristics, but they can work together to complete the training of the model through the control and coordination of the host.

[0032] In a heterogeneous GPU cluster, which includes GPUs from different manufacturers and models, the differences in hardware characteristics between GPUs often cause communication efficiency issues between GPUs.

[0033] In the related art, communication between GPUs requires first moving data from one GPU memory to the system memory, then transferring it from the system memory to the system memory of another computer, and then moving the data from the system memory of another computer to the memory of another GPU.

[0034] For example, Figure 2 It is a schematic diagram of the communication process between GPUs in the related technology. Figure 2 In the example, CPU (Central Processing Unit) 1 and GPU1 cluster belong to one computer, and CPU2 and GPU2 cluster belong to another computer. GPU1 cluster includes GPU1-1 to GPU1-m, with a total of m GPUs, and GPU2 cluster includes GPU2-1 to GPU2-n, with a total of n GPUs. Here, the values ​​of m and n can be the same or different.

[0035] The GPUs in the GPU1 cluster and the GPU2 cluster can be GPUs produced by different manufacturers, or GPUs of different product models produced by the same manufacturer. To achieve communication from the GPU1 cluster to the GPU2 cluster, steps ① to ④ can be completed.

[0036] Among them, step ① is to copy the data to be transmitted from the HBM (High Bandwidth Memory) of the GPU1 cluster to the memory of CPU1. Step ② is to transmit the data from the memory of CPU1 to the memory of CPU2 through NIC (Network Interface Controller) 1 and NIC2 via TCP (Transmission Control Protocol). Step ③ is to copy the data from the memory of CPU2 to the HBM of the GPU2 cluster. Step ④ is to convert the data layout in the GPU2 cluster so that the data layout obtained by transmission can conform to the layout of the GPU2 cluster.

[0037] Among them, the data copy between the HBM of the GPU and the CPU memory in step ① and step ③ can be achieved through PCI (Peripheral Component Interconnect) SW (Switch). NIC1 and NIC2 used in step ② are the network cards of CPU1 and CPU2 on the business side respectively. NIC1 and NIC2 are both unique and fixed, so there is only one transmission link from CPU1 to CPU2. In the case where there are multiple GPUs in the GPU1 cluster whose data needs to be transmitted to the GPU2 cluster, the data of each GPU needs to be transmitted to the GPU2 cluster in sequence through the transmission link formed by CPU1, NIC1, NIC2 to CPU2, which makes the concurrent data transmission inefficient in actual execution and the real-time requirements of distributed collaborative training are difficult to meet.

[0038] In view of the above problems, an embodiment of the present invention provides a transmission method. Figure 3 It is one of the flow charts of the transmission method provided by the present invention, such as Figure 3 As shown, the transmission method is applied to the local host, and the transmission method includes: Step 310: Acquire data to be transmitted consisting of a plurality of data slices, wherein the plurality of data slices are obtained by parallel calculation by a plurality of local computing devices.

[0039] Specifically, the communication method is applied to data transmission between two computing systems. Each computing system here may include a host and multiple computing devices connected to the host. The computing devices here may specifically be artificial intelligence chips, such as GPUs, TPUs, NPUs, APUs, and GPGPUs, etc., which are not specifically limited in the embodiments of the present invention.

[0040] For ease of distinction, in the two computing systems, one computing system that serves as the executor of the transmission method is recorded as the local computing system, the host in the local computing system is recorded as the local host, and the multiple computing devices in the local computing system are recorded as the local computing devices; in addition, the other computing system is recorded as the opposite computing system, the host in the opposite computing system is recorded as the opposite host, and the multiple computing devices in the opposite computing system are recorded as the opposite computing devices. It can be understood that the local computing device and the opposite computing device here can come from different manufacturers or belong to different product models of the same manufacturer, that is, the local computing device and the opposite computing device can have different hardware architectures, and the local computing device and the opposite computing device can be heterogeneous; in addition, multiple local computing devices can also come from different manufacturers or belong to different product models of the same manufacturer, that is, multiple local computing devices can have different hardware architectures, and multiple local computing devices can be heterogeneous; and multiple opposite computing devices can also come from different manufacturers or belong to different product models of the same manufacturer, that is, multiple opposite computing devices can have different hardware architectures, and multiple opposite computing devices can be heterogeneous. The embodiments of the present invention do not specifically limit this.

[0041] In the local computing system and the peer computing system, distributed model training or model reasoning can be performed based on multiple local computing devices and multiple peer computing devices. For example, at the local computing system, tensor parallelism (TP) can be performed at multiple local computing devices. Tensor parallelism means that different parts of the model are distributed to different computing devices, each computing device processes a part of the model, and exchanges necessary intermediate results with other computing devices to complete forward and backward propagation. Therefore, by performing tensor parallelism at multiple local computing devices, parallel computing can be achieved at multiple local computing devices to obtain multiple data slices of a set of data.

[0042] Here, multiple data slices obtained by parallel calculation of multiple local computing devices are taken as a group of data to be transmitted, that is, the data to be transmitted in the embodiment of the present invention, that is, multiple data slices constitute complete data to be transmitted. For example, there are 4 local GPUs in the local computing system, and each GPU is responsible for calculating 1 / 4 of a group of data, thereby obtaining 4 data slices.

[0043] The process of obtaining multiple data shards can be a process of copying the data shards obtained by parallel calculation from the memories of multiple local computing devices to the memory of the local host. As a result, there are multiple data shards to be transmitted in the memory of the local host that can form a group of data.

[0044] Step 320, based on multiple transmission links, transmit the multiple data slices to the opposite host in parallel, so that the opposite host distributes the target data to multiple opposite computing devices, and the target data is composed of the received multiple data slices.

[0045] Specifically, in the case where there are multiple data slices to be transmitted, multiple transmission links can be established between the local host and the opposite host, so that at the local host, multiple data slices can be transmitted in parallel through multiple transmission links. Specifically, one data slice can be transmitted on each transmission link through the same number of transmission links as the data slices, thereby realizing the parallel transmission of multiple data slices on multiple transmission links. For example, in the case where there are four data slices, namely, data slices a, b, c, and d, the parallel transmission of the four data slices can be realized through four transmission links, namely, data slice a is transmitted through transmission link a, data slice b is transmitted through transmission link b, data slice c is transmitted through transmission link c, and data slice d is transmitted through transmission link d.

[0046] Here, the multiple transmission links established between the local host and the opposite host can be realized by multiple network cards connected to the local host and multiple network cards connected to the opposite host. That is, the construction of the transmission link for parallel transmission of data fragments is no longer limited to a single fixed network card on the service side of the local host and the opposite host, but can be realized by using the network cards on the non-service side of the local host and the opposite host.

[0047] Therefore, in the presence of multiple transmission links, a group of multiple data slices of data to be transmitted can be transmitted in parallel under multiple transmission links, thereby shortening the time consumption of transmitting multiple data slices from the sum of the time consumption of transmitting each data slice in the relevant method to the maximum value of the time consumption of transmitting multiple data slices separately, thereby greatly improving the data transmission efficiency.

[0048] For the opposite host, the opposite host can receive data slices through multiple transmission links respectively. Therefore, after receiving multiple data slices, the opposite host can combine the received multiple data slices to form complete target data, and then distribute the target data to multiple opposite computing devices to achieve data transmission from the local computing device to the opposite computing device. It can be understood that, under the premise of successful transmission of multiple data slices, the target data combined by the opposite host is consistent with the data to be transmitted that can be composed of multiple data slices at the local host.

[0049] In the method provided in an embodiment of the present invention, multiple data slices obtained by parallel calculations of multiple local computing devices are transmitted in parallel to a peer host through multiple transmission links, and then distributed by the peer host to multiple peer computing devices to achieve multi-link data transmission between the local computing device and the peer computing device, thereby greatly reducing the transmission delay between the local computing device and the peer computing device and improving the transmission efficiency between the local computing device and the peer computing device.

[0050] Based on the above embodiment, the multiple transmission links correspond one-to-one to the multiple local computing devices, and each of the transmission links is used to transmit the data slices calculated by the corresponding local computing device.

[0051] Specifically, in the transmission method provided by the embodiment of the present invention, the multiple transmission links used for the parallel transmission of the multiple data slices correspond one-to-one to the multiple local computing devices that calculate the multiple data slices.

[0052] That is, in the local computing system, each local computing device corresponds to a transmission link for data transmission between the local host and the opposite host, and different local computing devices correspond to different transmission links.

[0053] In this way, in the scenario where multiple local computing devices are computing in parallel, when there is a demand for data transmission at each local computing device, that is, when each local computing device has generated data slices that need to be transmitted to the peer computing system, the transmission of the data slices at each local computing device can be implemented based on the transmission link corresponding to each local computing device, without being affected by the transmission of data slices at other local computing devices, thereby ensuring the independence of the transmission of each data slice, and further ensuring the reliability and transmission efficiency of the parallel transmission of the data to be transmitted consisting of multiple data slices on multiple transmission links.

[0054] Based on any of the above embodiments, in the method, the multiple data slices are transmitted in parallel to the peer host based on multiple transmission links, and the method also includes: Based on the local network card corresponding to each of the local computing devices, a transmission link corresponding to each of the local computing devices is established.

[0055] Specifically, in the local computing system, there may be multiple network cards, which are referred to as local network cards here. Each local network card may be connected to the local host, that is, each local network card may serve as an interface of a transmission link for data transmission between the local host and the remote host.

[0056] A corresponding local network card can be set in advance for each local computing device under the local computing system. That is, under the local computing system, each local computing device has a local network card corresponding to it, and the local network card corresponding to the local computing device can be responsible for the data transmission associated with the local computing device, which can specifically include the sending of data calculated by the local computing device and the receiving of data required for calculation by the local computing device.

[0057] Therefore, when each local computing device has a corresponding local network card, a transmission link between the local host and the opposite host can be established based on the local network card corresponding to each local computing device, and the transmission link thus established, that is, the transmission link corresponding to each local computing device, is dedicated to data transmission related to the local computing device.

[0058] For example, Figure 4 FIG. 2 is a flow chart of the transmission method provided by the present invention. Figure 4 As shown, there are a network card NIC1 cluster and a network card NIC2 cluster. The network cards NIC1-1 to NIC1-m in the network card NIC1 cluster are respectively connected to CPU1, and the network cards NIC2-1 to NIC2-m in the network card NIC2 cluster are respectively connected to CPU2.

[0059] Based on the network card NIC1 cluster and the network card NIC2 cluster, m transmission links can be constructed between CUP1 and CPU2, namely, link 1 (GPU1 to NIC1-1 to NIC2-1 to CPU2), link 2 (GPU1 to NIC1-2 to NIC2-2 to CPU2), ..., link m (GPU1 to NIC1-m to NIC2-m to CPU2).

[0060] For the GPU1 cluster, GPU1-1 to GPU1-m in the GPU1 cluster correspond to NIC1-1 to NIC1-m in the network card NIC1 cluster respectively. Therefore, among the m transmission links between CUP1 and CPU2, each transmission link can correspond to a GPU in the GPU1 cluster, specifically, GPU1-1 corresponds to link 1, GPU1-2 corresponds to link 2, ..., GPU1-m corresponds to link m.

[0061] Based on this, when the m GPUs in the GPU1 cluster calculate in parallel to obtain m data slices, CPU1 can copy the m data slices from the HBM of the m GPUs to the CPU1 memory, and then trigger NIC1-1 to NIC1-m to send the m data slices in the CPU1 memory to the CPU2 memory in parallel. CPU2 then combines the received m data slices and copies them to the HBM of the n GPUs in the GPU2 cluster.

[0062] Based on any of the above embodiments, in the method, the transmission link corresponding to each of the local computing devices is established based on the local network card corresponding to each of the local computing devices, and the method also includes: Based on the connection relationship and communication bandwidth between each of the local computing devices and each of the local network cards, a local network card corresponding to each of the local computing devices is determined.

[0063] Specifically, in the local computing system, various connection relationships may exist between different local computing devices and different local network cards. The connection relationship referred to here may include whether a physical connection is connected, and the specific connection path under the premise of determining that a physical connection exists. The characteristics of the connection relationship directly affect the complexity, delay and potential communication bottleneck of data transmission.

[0064] In addition, under the premise that there is a physical connection between the local computing device and the local network card, the communication bandwidth between the local computing device and the local network card can also be counted. The communication bandwidth here refers to the amount of data that can be transmitted per unit time in the connection channel between the local computing device and the local network card, usually in Mbps (million bits per second) or Gbps (billion bits per second). Communication bandwidth is a key indicator for evaluating connection performance. High-bandwidth connections can support faster data synchronization and distribution, while low-bandwidth connections may limit data transmission efficiency and become a communication bottleneck.

[0065] By combining the connection relationship and communication bandwidth between each local computing device and each local network card, the local network card with the best communication performance with the local computing device can be selected from the local network cards, and the local network card can be used as the local network card corresponding to the local computing device, thereby ensuring that each local network card can give full play to its performance advantages in data transmission and further improve the parallel transmission efficiency.

[0066] Based on any of the above embodiments, in the method, establishing a transmission link corresponding to each of the local computing devices based on the local network card corresponding to each of the local computing devices includes: Based on the local network card corresponding to each of the local computing devices and the opposite network card of the opposite computing device corresponding to each of the local computing devices, a transmission link corresponding to each of the local computing devices is established.

[0067] Specifically, in the peer computing system, there may also be multiple network cards, which are referred to as peer network cards here, and each peer network card may be connected to the peer host, that is, each peer network card may be used as an interface of a transmission link for data transmission between the peer host and the local host. Thus, the transmission link established between the local host and the peer host may be transmitted from the local host to the local network card, from the local network card to the peer network card, and then from the peer network card to the peer host.

[0068] In actual applications, the number of local network cards in the local computing system and the number of peer network cards in the peer computing system can be the same or different. In the case where the number of network cards in the computing systems at both ends is different, it is necessary to implement non-dual networking for the computing systems at both ends.

[0069] That is, not only in the local computing system, a corresponding local network card can be configured for each local computing device, but also in the peer computing system, a corresponding peer network card can be configured for each peer computing device. It can be understood that the method of configuring a corresponding peer network card for a peer computing device is consistent with the method of configuring a corresponding local network card for a local computing device in the above embodiment, and will not be repeated here.

[0070] Before transmission, it is possible to determine the peer computing device to which each data slice calculated by the local computing device needs to be transmitted, that is, the recipient of the data slice calculated by the local computing device, which is recorded here as the peer computing device corresponding to the local computing device.

[0071] Therefore, after determining the opposite computing device corresponding to the local computing device, the opposite network card of the opposite computing device corresponding to the local computing device can be determined, so that the local network card corresponding to the local computing device and the opposite network card of the opposite computing device corresponding to the local computing device are regarded as a pair of network cards, and a transmission link based on the pair of network cards is established.

[0072] It can be understood that since the computing devices of the computing systems at both ends are configured with corresponding network cards, the binding of the computing devices and the network cards is realized. Therefore, when establishing a transmission link, it can be directly implemented based on the network cards configured on each end. Therefore, there is no need to consider whether the number of network cards under the computing systems at both ends is the same, and the non-dual networking of the computing systems at both ends can be quickly realized.

[0073] For example, Figure 5 FIG. 3 is a flow chart of the transmission method provided by the present invention. Figure 5 As shown, there are two end computing systems, namely, a local computing system and a remote computing system, and the number of network cards under the local computing system is different from the number of network cards under the remote computing system.

[0074] The local computing system includes CPU1, CPU2, PCI SW1, PCI SW2, network card NIC1 cluster and GPU1 cluster. The network card NIC1 cluster includes 4 local network cards, namely NIC1-1 to NIC1-4, and the GPU1 cluster includes GPU1-1 to GPU1-8, wherein GPU1-1 and GPU1-2 correspond to NIC1-1, GPU1-3 and GPU1-4 correspond to NIC1-2, GPU1-5 and GPU1-6 correspond to NIC1-3, and GPU1-7 and GPU1-8 correspond to NIC1-4.

[0075] The peer computing system includes CPU3, CPU4, PCI SW3, PCI SW4, network card NIC2 cluster and GPU2 cluster. The network card NIC2 cluster includes two peer network cards, namely NIC2-1 and NIC2-2, and the GPU2 cluster includes GPU2-1 to GPU2-8, wherein GPU2-1 to GPU2-4 correspond to NIC2-1, and GPU2-5 to GPU2-8 correspond to NIC2-2.

[0076] Assuming that the local computing device GPU1-1 corresponds to the peer computing device GPU2-1, that is, the data slices calculated by GPU1-1 need to be transmitted to GPU2-1, a transmission link 1 (GPU1-1 to CPU1 to NIC1-1 to NIC2-1 to CPU3 to GPU2-1) is established based on NIC1-1 corresponding to GPU1-1 and NIC2-1 corresponding to GPU2-1.

[0077] It can be understood that since the network card corresponding to the computing device is the network card with the best communication performance between the computing device and the computing device, the transmission link established based on the network cards corresponding to the computing devices at both ends can improve the parallel transmission efficiency.

[0078] Assume that the corresponding relationship between GPU and network card is not considered when establishing the transmission link between GPU1-1 and GPU2-1, that is, establish transmission link 2 (GPU1-1 to CPU1 to NIC1-1 to NIC2-2 to CPU4 to CPU3 to GPU2-1). Compared with the above transmission link 1, it can be seen that in transmission link 2, the data fragments received by NIC2-2 need to pass through CPU4 and CPU3 in turn before being transmitted to GPU2-1. This obviously passes through more nodes than the transmission from NIC2-1 to GPU2-1 through CPU3, which directly affects the efficiency of data transmission.

[0079] In the method provided by the embodiment of the present invention, corresponding network cards are respectively configured for the computing devices in the computing systems at both ends, thereby establishing a transmission link, which can further improve the transmission efficiency.

[0080] In the related art, data transmission between hosts is implemented based on the TCP protocol. Figure 2 In the example, data transmission between CPU1 and CPU2 is implemented based on the TCP protocol. In the data transmission process based on the TCP protocol, the sending and receiving of data are implemented by the network transmission system running in the kernel state, which requires that the sent and received data itself be in the kernel state. Therefore, after CPU1 obtains the data copied from the GPU1 cluster, it needs to convert the data from the user state to the kernel state, and then transmit the kernel state data to CPU2 through NIC1 and NIC2 based on the TCP protocol.

[0081] That is, in Figure 2 In the example, the implementation of step ② involves data copying from user state to kernel state by CPU1, as well as data transmission based on the TCP protocol. It can be understood that data copying from user state to kernel state is a necessary operation for data transmission based on the TCP protocol, and data copying from user state to kernel state itself requires CPU1's computing power and data transmission time. In addition, data transmission based on the TCP protocol uses a sliding window mechanism to control the sending and receiving of data. The size of the sliding window limits the amount of data that can be transmitted by a TCP connection at any given time, which enables data transmission based on the TCP protocol to achieve maximum bandwidth utilization.

[0082] In view of the above problem, based on any of the above embodiments, in step 320, the transmitting the multiple data slices to the opposite host in parallel based on multiple transmission links includes: Based on the multiple transmission links, the multiple data slices are transmitted in parallel to the opposite host through remote direct memory access technology.

[0083] Specifically, Remote Direct Memory Access (RDMA) technology is a high-performance network communication technology that allows a computer to directly access the memory of a remote computer. Applying RDMA technology in the embodiments of the present invention can realize data transmission from a local host to a remote host.

[0084] Since RDMA technology supports kernel bypass technology, that is, data transmission can be performed directly in user mode without context switching between kernel mode and user mode, it can effectively reduce data transmission delay compared to the operation mode that needs to switch from user mode to kernel mode in related technologies. In addition, since RDMA technology can directly perform data transmission under the network transport layer, it has higher bandwidth utilization and lower latency compared to the transmission method based on TCP protocol.

[0085] In the embodiment of the present invention, replacing the TCP protocol transmission mode with data transmission based on RDMA technology can greatly improve the bandwidth utilization of data transmission, thereby further improving the data transmission efficiency.

[0086] Based on any of the above embodiments, in step 320, the transmitting the multiple data slices to the opposite host in parallel based on multiple transmission links includes: Based on multiple transmission links, transmit the data blocks of the multiple data slices to the opposite host in parallel; The size of the data block of each of the data slices is the target size of the transmission link for transmitting each of the data slices.

[0087] Specifically, in the transmission process for each data fragment, the transmission of the data fragment can be carried out in the form of data blocks, that is, in a transmission link, the data to be transmitted can be segmented and divided into multiple data blocks, and the multiple data blocks are transmitted in sequence to form a data stream.

[0088] The data block here is a data unit with a fixed structure and format. The size of the data block can be adjusted according to the actual transmission scenario. The size of the data block is recorded as the target size here, and the target size can be one of multiple candidate sizes. The candidate size selected as the target size here can be the candidate size with the highest transmission efficiency among multiple candidate sizes, or the default candidate size of the transmission link, or the candidate size used in the previous transmission with the opposite host. The embodiment of the present invention does not make specific limitations on this.

[0089] Furthermore, since there are multiple transmission links between the local host and the remote host, and the transmission networks of different transmission links may be different, the target size can be set for different transmission links respectively, that is, the values ​​of the target sizes of different transmission links can be the same or different.

[0090] Based on any of the above embodiments, before transmitting the multiple data fragments to the opposite host in parallel based on multiple transmission links, the method further includes: Divide the test data into multiple data blocks of candidate sizes; For each of the transmission links, based on the transmission link, multiple data blocks of the candidate size are transmitted to the opposite host, and the data transmission time of the candidate size is obtained; Based on the data transmission time consumption under the multiple candidate sizes, a target size under the transmission link is determined.

[0091] Specifically, for each transmission link, before data transmission is performed based on the transmission link, the transmission efficiency of each candidate size on the transmission link can be tested first, thereby selecting the candidate size with the highest transmission efficiency as the target size under the transmission link.

[0092] In this process, test data may be determined first. The test data here is data used to test the transmission efficiency of each candidate size. The test data may be pre-constructed data or any other data that can be used for transmission between the local host and the remote host.

[0093] After the test data is obtained, the test data can be divided into multiple data blocks under the candidate sizes. It can be understood that in the case where there are multiple candidate sizes, the test data needs to be divided for each candidate size, so that under each candidate size, there are multiple data blocks with the same size as the candidate size.

[0094] For each candidate size, multiple data blocks of the candidate size can be transmitted to the opposite host through the transmission link, and the time used to transmit the multiple data blocks of the candidate size can be recorded, that is, the data transmission time of the candidate size can be obtained. It can be understood that since data blocks are transmitted for each candidate size, each candidate size can obtain the corresponding data transmission time, and the data transmission time here can reflect the communication efficiency of different candidate sizes under the transmission link under the same amount of transmission data. It can be understood that the longer the data transmission time, the lower the communication efficiency, and the shorter the data transmission time, the higher the communication efficiency.

[0095] Therefore, after obtaining the data transmission time under each candidate size, one can be selected from the candidate sizes as the target size under the transmission link based on the data transmission time under each candidate size. For example, the candidate size with the shortest data transmission time can be selected from multiple candidate sizes as the target size under the transmission link.

[0096] In the method provided in the embodiment of the present invention, the target size is determined based on the data transmission time under multiple candidate sizes, which helps to improve the communication efficiency between computing devices.

[0097] The transmission device provided by the present invention is described below. The transmission device described below and the transmission method described above can be referenced to each other.

[0098] Figure 6 Schematic diagram of the structure of the transmission device provided by the present invention. Figure 6 As shown, the device is applied to a local host, and the device includes: A fragment acquisition unit 610 is used to acquire data to be transmitted consisting of multiple data fragments, where the multiple data fragments are obtained by parallel calculations of multiple local computing devices; The parallel transmission unit 620 is used to transmit the multiple data slices in parallel to the peer host based on multiple transmission links, so that the peer host distributes the target data to multiple peer computing devices, and the target data is composed of the received multiple data slices.

[0099] In the device provided in the embodiment of the present invention, multiple data slices obtained by parallel calculations of multiple local computing devices are transmitted in parallel to the opposite host through multiple transmission links, and then distributed by the opposite host to multiple opposite computing devices, so as to realize multi-link data transmission between the local computing device and the opposite computing device, thereby greatly reducing the transmission delay between the local computing device and the opposite computing device, and improving the transmission efficiency between the local computing device and the opposite computing device.

[0100] Based on any of the above embodiments, the multiple transmission links correspond one-to-one to the multiple local computing devices, and each of the transmission links is used to transmit the data slices calculated by the corresponding local computing device.

[0101] Based on any of the above embodiments, the device further includes a link establishing unit, configured to: Based on the local network card corresponding to each of the local computing devices, a transmission link corresponding to each of the local computing devices is established.

[0102] Based on any of the foregoing embodiments, the link establishing unit is further configured to: Based on the connection relationship and communication bandwidth between each of the local computing devices and each of the local network cards, a local network card corresponding to each of the local computing devices is determined.

[0103] Based on any of the above embodiments, the link establishment unit is specifically used for: Based on the local network card corresponding to each of the local computing devices and the opposite network card of the opposite computing device corresponding to each of the local computing devices, a transmission link corresponding to each of the local computing devices is established.

[0104] Based on any of the above embodiments, the parallel transmission unit is specifically used for: Based on the multiple transmission links, the multiple data slices are transmitted in parallel to the opposite host through remote direct memory access technology.

[0105] Based on any of the above embodiments, the parallel transmission unit is specifically used for: Based on multiple transmission links, transmit the data blocks of the multiple data slices to the opposite host in parallel; The size of the data block of each of the data slices is the target size of the transmission link for transmitting each of the data slices.

[0106] Based on any of the above embodiments, the parallel transmission unit is further used for: Divide the test data into multiple data blocks of candidate sizes; For each of the transmission links, based on the transmission link, multiple data blocks of the candidate size are transmitted to the opposite host, and the data transmission time of the candidate size is obtained; Based on the data transmission time consumption under the multiple candidate sizes, a target size under the transmission link is determined.

[0107] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7 As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the transmission method, which includes: Acquire data to be transmitted consisting of a plurality of data slices, wherein the plurality of data slices are obtained by parallel calculation by a plurality of local computing devices; Based on multiple transmission links, the multiple data slices are transmitted in parallel to the opposite host, so that the opposite host distributes the target data to multiple opposite computing devices, and the target data is composed of the received multiple data slices.

[0108] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0109] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the transmission method provided by the above methods, the method comprising: Acquire data to be transmitted consisting of a plurality of data slices, wherein the plurality of data slices are obtained by parallel calculation by a plurality of local computing devices; Based on multiple transmission links, the multiple data slices are transmitted in parallel to the opposite host, so that the opposite host distributes the target data to multiple opposite computing devices, and the target data is composed of the received multiple data slices.

[0110] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the transmission method provided by the above methods is implemented, and the method includes: Acquire data to be transmitted consisting of a plurality of data slices, wherein the plurality of data slices are obtained by parallel calculation by a plurality of local computing devices; Based on multiple transmission links, the multiple data slices are transmitted in parallel to the opposite host, so that the opposite host distributes the target data to multiple opposite computing devices, and the target data is composed of the received multiple data slices.

[0111] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0112] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A transmission method, characterized in that: The transmission method is applied to the local host, and the transmission method includes: Acquire data to be transmitted consisting of a plurality of data slices, wherein the plurality of data slices are obtained by parallel calculation by a plurality of local computing devices; Based on multiple transmission links, the multiple data slices are transmitted in parallel to the opposite host, so that the opposite host distributes the target data to multiple opposite computing devices, and the target data is composed of the received multiple data slices.

2. The transmission method according to claim 1, characterized in that: The multiple transmission links correspond one-to-one to the multiple local computing devices, and each of the transmission links is used to transmit the data slices calculated by the corresponding local computing device.

3. The transmission method according to claim 2, characterized in that: The method of transmitting the plurality of data slices to the opposite host in parallel based on the plurality of transmission links also includes: Based on the local network card corresponding to each of the local computing devices, a transmission link corresponding to each of the local computing devices is established.

4. The transmission method according to claim 3, characterized in that: The step of establishing a transmission link corresponding to each of the local computing devices based on the local network card corresponding to each of the local computing devices also includes: Based on the connection relationship and communication bandwidth between each of the local computing devices and each of the local network cards, a local network card corresponding to each of the local computing devices is determined.

5. The transmission method according to claim 3, characterized in that: The establishing of a transmission link corresponding to each of the local computing devices based on the local network card corresponding to each of the local computing devices includes: Based on the local network card corresponding to each of the local computing devices and the opposite network card of the opposite computing device corresponding to each of the local computing devices, a transmission link corresponding to each of the local computing devices is established.

6. The transmission method according to any one of claims 1 to 5, characterized in that: The method of transmitting the plurality of data slices to the opposite host in parallel based on the plurality of transmission links comprises: Based on the multiple transmission links, the multiple data slices are transmitted in parallel to the opposite host through remote direct memory access technology.

7. The transmission method according to any one of claims 1 to 5, characterized in that: The method of transmitting the plurality of data slices to the opposite host in parallel based on the plurality of transmission links comprises: Based on multiple transmission links, transmit the data blocks of the multiple data slices to the opposite host in parallel; The size of the data block of each of the data slices is the target size of the transmission link for transmitting each of the data slices.

8. The transmission method according to claim 7, characterized in that: The method of transmitting the plurality of data slices to the opposite host in parallel based on the plurality of transmission links also includes: Divide the test data into multiple data blocks of candidate sizes; For each of the transmission links, based on the transmission link, multiple data blocks of the candidate size are transmitted to the opposite host, and the data transmission time of the candidate size is obtained; Based on the data transmission time consumption under the multiple candidate sizes, a target size under the transmission link is determined.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the transmission method according to any one of claims 1 to 8 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the transmission method according to any one of claims 1 to 8 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the transmission method according to any one of claims 1 to 8 is implemented.

Images