Communication method, device, storage medium and program product

Abstract

The invention relates to the technical field of artificial intelligence, and provides a communication method and device, a storage medium and a program product, and the method comprises the steps: calling a data transmission interface matched with a home terminal artificial intelligence chip based on a universal interface installed at the home terminal artificial intelligence chip, the universal interface is integrated with data transmission interfaces respectively matched with various artificial intelligence chips; based on the data transmission interface matched with the local-end artificial intelligence chip, communication with the opposite-end artificial intelligence chip is carried out through a direct connection transmission path, a universal interface is installed at the opposite-end artificial intelligence chip, and the data transmission interface matched with the opposite-end artificial intelligence chip is called through the universal interface. According to the method, the device, the storage medium and the program product provided by the invention, direct connection communication with various artificial intelligence chips is realized, so that communication delay between heterogeneous artificial intelligence chips is reduced, and the communication efficiency between the heterogeneous artificial intelligence chips is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a communication method, device, storage medium and program product. Background Art

[0002] With the rapid development of artificial intelligence technology, especially the rise of large language models (LLM), the demand for computing resources has exploded. The scale of these models, the scale of training data, and the number of GPU (Graphics Processing Unit) resources required are all growing at an exponential rate.

[0003] In some cases, thousands or even tens of thousands of GPUs may be needed to meet training needs. However, in the current GPU cloud service and resource usage environment, these tens of thousands of GPUs may come from different manufacturers or belong to different product models of the same manufacturer, with different hardware architectures, that is, they are heterogeneous.

[0004] In this context, the mixed use of GPUs of different models and manufacturers to support large-scale model training is called heterogeneous mixed training (also known as "heterogeneous training"). Heterogeneous mixed training can effectively solve the "heterogeneous island" problem of GPUs in data centers.

[0005] The communication efficiency between GPUs of different manufacturers or models directly affects the overall efficiency of heterogeneous mixed training. How to improve the communication efficiency between heterogeneous GPUs to achieve efficient interconnection between heterogeneous GPUs has become a hot research topic in the industry. Summary of the invention

[0006] The present invention provides a communication method, a device, a storage medium and a program product, which are used to solve the defect of low communication efficiency between GPUs in the related art.

[0007] The present invention provides a communication method, comprising: Based on the universal interface installed at the local artificial intelligence chip, a data transmission interface adapted to the local artificial intelligence chip is called, wherein the universal interface is integrated with data transmission interfaces adapted to various artificial intelligence chips respectively; Based on the data transmission interface compatible with the local artificial intelligence chip, communicating with the opposite artificial intelligence chip through a direct transmission path, the opposite artificial intelligence chip is installed with the universal interface, and the data transmission interface compatible with the opposite artificial intelligence chip is called through the universal interface; The direct transmission path is composed of the local artificial intelligence chip, the local network card connected to the local artificial intelligence chip, the opposite network card connected to the opposite artificial intelligence chip, and the opposite artificial intelligence chip.

[0008] According to a communication method provided by the present invention, the data transmission interface includes a memory copy interface; The method of communicating with the other-end artificial intelligence chip through a direct transmission path based on a data transmission interface compatible with the local artificial intelligence chip includes: Based on a memory copy interface compatible with the local artificial intelligence chip, memory copy is performed in the target memory area of ​​the local artificial intelligence chip, and after the copy is completed, the data in the target memory area is transmitted to the other-end artificial intelligence chip through the direct transmission path.

[0009] According to a communication method provided by the present invention, the data transmission interface further includes a memory registration interface; The memory copying is performed in the target memory area of ​​the local artificial intelligence chip based on the memory copy interface compatible with the local artificial intelligence chip, and the method further includes: Based on a memory registration interface compatible with the local artificial intelligence chip, memory registration is performed on the local artificial intelligence chip to obtain a target memory area in the memory of the local artificial intelligence chip.

[0010] According to a communication method provided by the present invention, based on the universal interface installed at the local artificial intelligence chip, calling the data transmission interface adapted to the local artificial intelligence chip includes: Based on the universal interface installed at the local artificial intelligence chip, a data transmission interface compatible with at least one of the manufacturer and product model of the local artificial intelligence chip is called.

[0011] According to a communication method provided by the present invention, the communication with the other end artificial intelligence chip through a direct transmission path based on a data transmission interface adapted to the local end artificial intelligence chip includes: Based on a data transmission interface compatible with the local artificial intelligence chip, a data block is transmitted to the other artificial intelligence chip through a direct transmission path, and the size of the data block is a target size; The target size is one of multiple candidate sizes.

[0012] According to a communication method provided by the present invention, the communication with the other end artificial intelligence chip through a direct transmission path based on a data transmission interface adapted to the local artificial intelligence chip also includes: Dividing the test data into a plurality of data blocks of the candidate size; Based on a data transmission interface compatible with the local artificial intelligence chip, multiple data blocks of the candidate size are sent to the opposite artificial intelligence chip through a direct transmission path to obtain a data transmission time consumption of the candidate size; The target size is determined based on the data transmission time consumption under the multiple candidate sizes.

[0013] According to a communication method provided by the present invention, the method further includes: calling a data transmission interface compatible with the local artificial intelligence chip based on a universal interface installed at the local artificial intelligence chip; When it is determined that the local artificial intelligence chip supports direct memory access between chips and a universal interface is installed at the local artificial intelligence chip, universal interface registration is performed.

[0014] The present invention also provides a communication device, comprising: A calling unit, configured to call a data transmission interface compatible with the local artificial intelligence chip based on a universal interface installed at the local artificial intelligence chip, wherein the universal interface is integrated with data transmission interfaces compatible with various types of artificial intelligence chips; A communication unit, configured to communicate with the peer artificial intelligence chip through a direct transmission path based on a data transmission interface compatible with the local artificial intelligence chip, the peer artificial intelligence chip being provided with the universal interface, and calling the data transmission interface compatible with the peer artificial intelligence chip through the universal interface; The direct transmission path is composed of the local artificial intelligence chip, the local network card connected to the local artificial intelligence chip, the opposite network card connected to the opposite artificial intelligence chip, and the opposite artificial intelligence chip.

[0015] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned communication methods when executing the program.

[0016] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the communication method described above is implemented.

[0017] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above communication methods.

[0018] The communication method, device, storage medium and program product provided by the present invention realize unified management of data transmission operations for various types of artificial intelligence chips by integrating a universal interface with data transmission interfaces that are respectively compatible with various types of artificial intelligence chips. Based on this, the data transmission interface that is compatible with various types of artificial intelligence chips can be called to realize direct communication with various types of artificial intelligence chips, thereby reducing the communication delay between heterogeneous artificial intelligence chips and improving the communication efficiency between heterogeneous artificial intelligence chips. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a structural diagram of a computing system based on a heterogeneous GPU cluster in the related technology.

[0021] Figure 2 It is a schematic diagram of the communication process between GPUs in the related technology.

[0022] Figure 3 This is one of the flow charts of the communication method provided by the present invention.

[0023] Figure 4 This is the second flow chart of the communication method provided by the present invention.

[0024] Figure 5 This is the third flow chart of the communication method provided by the present invention.

[0025] Figure 6 This is the fourth flow chart of the communication method provided by the present invention.

[0026] Figure 7 It is a structural schematic diagram of the communication device provided by the present invention.

[0027] Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] As the parameter scale and training data volume of large language models increase, the demand for storage resources and hardware computing power has exceeded the upper limit that a single computing device can provide. In order to break through this limitation and improve the training throughput at the same time, a 3D parallel training strategy is often adopted, that is, distributed collaborative training of the model is implemented through data parallelism, tensor parallelism, and pipeline parallelism. Here, distributed collaborative training refers to the use of collaborative cooperation between multiple computing devices to jointly participate in the training process of the model to achieve model training acceleration and performance improvement. It should be noted that the computing device here is an artificial intelligence chip, such as a GPU, TPU (Tensor Processing Unit), NPU (Neural network Processing Unit), DPU (Deep learning Processing Unit), APU (Accelerated Processing Unit), and GPGPU (General-Purpose computing on Graphics Processing Unit), etc., and the embodiment of the present invention does not specifically limit this.

[0030] In distributed collaborative training, thousands or even tens of thousands of GPUs may be needed to meet training needs. Traditional distributed training requires completely identical GPU devices (same model from the same manufacturer), and insufficient resources become a bottleneck for large-scale model training, especially large language model training.

[0031] In the current GPU cloud service and resource usage environment, these tens of thousands of GPUs may come from different manufacturers or belong to different product models of the same manufacturer, with different hardware architectures, that is, they are heterogeneous. In the embodiment of the present invention, a cluster composed of GPUs with different hardware architectures is called a heterogeneous GPU cluster.

[0032] For example, Figure 1 It is a schematic diagram of the structure of a computing system based on a heterogeneous GPU cluster in the related technology. Figure 1As shown, the system may include a host 110 and a device end 120, wherein the device end 120 may include multiple first-type devices, multiple second-type devices, and other types of devices, where the devices can all be regarded as GPUs or other types of artificial intelligence chips. The host 110 is connected to multiple devices on the device end 120 to control these devices to perform various computing tasks and collaboratively complete the training of the model. On the device end 120, multiple first-type devices are all devices of the same manufacturer and the same model, and these devices constitute a homogeneous device cluster, while the first-type devices and the second-type devices may be devices of different manufacturers or different models, and these devices constitute a heterogeneous device cluster. Heterogeneous mixed training for the model is to use these different types of devices (including homogeneous and heterogeneous devices) to jointly participate in the training task of the model during the model training process. These devices may have different computing capabilities, memory sizes, data formats, and other characteristics, but they can work together to complete the training of the model through the control and coordination of the host.

[0033] In a heterogeneous GPU cluster, which includes GPUs from different manufacturers and models, the differences in hardware characteristics between GPUs often cause communication efficiency issues between GPUs.

[0034] In related technologies, communication between GPUs requires first moving data from one GPU memory to the system memory, then transmitting it to the system memory of another computer through RDMA (Remote Direct Memory Access) technology, and then moving the data from the system memory of another computer to the memory of another GPU.

[0035] For example, Figure 2 It is a schematic diagram of the communication process between GPUs in the related technology. Figure 2 In the example, GPU1 and CPU (Central Processing Unit) 1 belong to one computer, and GPU2 and CPU2 belong to another computer. GPU1 and GPU2 can be GPUs produced by different manufacturers, or GPUs of different product models produced by the same manufacturer. To achieve communication from GPU1 to GPU2, steps ① to ④ can be completed.

[0036] Among them, step ① is to copy the data to be transmitted from the HBM (High Bandwidth Memory) of GPU1 to the memory of CPU1. Step ② is to transmit the data from the memory of CPU1 to the memory of CPU2 through the NIC (Network Interface Controller) via TCP (Transmission Control Protocol). Step ③ is to copy the data from the memory of CPU2 to the HBM of GPU2. Step ④ is to convert the layout of the data in GPU2 so that the data layout obtained by transmission can conform to the layout of GPU2.

[0037] Among them, the data copying between the HBM of the GPU and the CPU memory in step ① and step ③ can be achieved through PCI (Peripheral Component Interconnect, peripheral component interconnect standard) SW (Switch).

[0038] It is understandable that this process involves multiple data copies from GPU to CPU and from CPU to GPU, and multiple data copies will inevitably lead to increased communication delays and affect the communication efficiency between GPUs.

[0039] In order to improve the communication efficiency between GPUs, GPU Direct RDMA technology came into being. GPU Direct RDMA is a network communication method based on DMA (Direct Memory Access) technology. Among them, DMA allows the hardware controller to exchange data directly with the memory without the intervention of the CPU, which directly reduces the burden on the CPU and improves the communication efficiency between the hardware controller and the memory. RDMA uses zero-copy network technology and kernel memory bypass technology to achieve high-performance remote direct memory access.

[0040] Among them, zero-copy network technology allows NIC to directly transfer data with application memory without first copying the data to the kernel and then copying it from the kernel to the application memory. This eliminates the step of copying data between application memory and kernel, thereby reducing transmission latency. Kernel memory bypass technology allows applications to send commands directly through NIC without calling through the kernel. This reduces the number of environment switches between the kernel's memory space and the application's space, further improving communication efficiency.

[0041] The application of GPU Direct RDMA technology allows the GPU of one computer to directly access the GPU memory of another computer, thereby reducing the number of data copies and effectively reducing communication latency.

[0042] However, the implementation of RDMA technology relies on specific software programs, which need to work closely with the software and hardware design of the GPU to achieve direct access to remote memory. Because GPUs from different manufacturers or different models of the same manufacturer have differences in software and hardware design, and the software programs that RDMA relies on are not compatible with GPUs from different manufacturers or different models of each manufacturer, RDMA technology can usually only support GPUs produced by specific manufacturers.

[0043] With the widespread application of GPUs produced by various manufacturers and GPUs of various product models in high-performance computing and data centers, how to achieve GPU Direct RDMA communication across the network between GPUs of different manufacturers or different models, that is, how to achieve GPU Direct RDMA communication between heterogeneous GPUs, is still a problem to be solved.

[0044] Figure 3 It is one of the flow charts of the communication method provided by the present invention, such as Figure 3 As shown, a communication method includes: Step 310, based on the universal interface installed at the local artificial intelligence chip, call the data transmission interface compatible with the local artificial intelligence chip, and the universal interface integrates data transmission interfaces that are compatible with various types of artificial intelligence chips.

[0045] Specifically, this communication method is applied to the communication between two artificial intelligence chips, where the artificial intelligence chip may be, for example, a GPU, a TPU, an NPU, an APU, and a GPGPU, etc., and the embodiment of the present invention does not specifically limit this.

[0046] For the sake of distinction, among the two artificial intelligence chips, one artificial intelligence chip that is the executor of the communication method is recorded as the local artificial intelligence chip, and the other artificial intelligence chip is recorded as the opposite artificial intelligence chip. Both artificial intelligence chips can regard themselves as the local artificial intelligence chip, and both artificial intelligence chips are the opposite artificial intelligence chips of each other. It can be understood that the two artificial intelligence chips here can come from different manufacturers or belong to different product models of the same manufacturer, that is, the two artificial intelligence chips here can have different hardware architectures, and the two artificial intelligence chips can be heterogeneous.

[0047] For the local artificial intelligence chip, a universal interface can be pre-installed at the local artificial intelligence chip. The universal interface here is an interface obtained by abstracting the data transmission operations adapted to artificial intelligence chips of various manufacturers and various product models produced by various manufacturers, that is, the universal interface integrates data transmission interfaces adapted to artificial intelligence chips of various manufacturers and various product models produced by various manufacturers, so it can support the communication between the local artificial intelligence chip and any type of artificial intelligence chip.

[0048] Here, the data transmission interface is an interface for calling data transmission operations, which may include memory registration, memory copy, etc. It is understandable that the memory registration, memory copy and other operations adapted by different types of artificial intelligence chips may be different, but the functions of the memory registration, memory copy and other operations adapted by different types of artificial intelligence chips are consistent. Therefore, the functions of specific operations can be used as an abstract basis to integrate various data transmission operations adapted by different types of artificial intelligence chips, while hiding the underlying hardware implementation details of different data transmission operations, thereby obtaining a general interface that can call the data transmission interfaces adapted by various types of artificial intelligence chips.

[0049] In the case where a universal interface is pre-installed at the local artificial intelligence chip, the data transmission interface compatible with the local artificial intelligence chip can be determined and called from the data transmission interfaces compatible with various artificial intelligence chips integrated in the universal interface based on the universal interface. It can be understood that the data transmission interface compatible with the local artificial intelligence chip here can specifically be a data transmission interface corresponding to the manufacturer of the local artificial intelligence chip, or a data transmission interface corresponding to the product model of the local artificial intelligence chip, etc., and the embodiment of the present invention does not specifically limit this.

[0050] Step 320, based on the data transmission interface compatible with the local artificial intelligence chip, communicate with the opposite artificial intelligence chip through a direct transmission path, the opposite artificial intelligence chip is installed with the universal interface, and the data transmission interface compatible with the opposite artificial intelligence chip is called through the universal interface; the direct transmission path is composed of the local artificial intelligence chip, the local network card connected to the local artificial intelligence chip, the opposite network card connected to the opposite artificial intelligence chip, and the opposite artificial intelligence chip.

[0051] Specifically, similar to the artificial intelligence chip on this end, the artificial intelligence chip on the other end is also installed with the above-mentioned universal interface, and can call the data transmission interface compatible with the artificial intelligence chip on the other end through the universal interface. Therefore, the data transmission interface called by the artificial intelligence chip on the other end can also support communication between the artificial intelligence chip on the other end and any type of artificial intelligence chip.

[0052] Therefore, communication with the peer artificial intelligence chip can be achieved based on a data transmission interface that is compatible with the peer artificial intelligence chip and is called by a universal interface.

[0053] Here, the communication between the local artificial intelligence chip and the other artificial intelligence chip is achieved through a direct transmission path.

[0054] Figure 4 is a second flow chart of the communication method provided by the present invention. The direct transmission path may be Figure 4 In the form shown in FIG. 1 , data at GPU1 at the local end can be transmitted to the network card NIC at the other end through the network card NIC at the local end and then to GPU2 at the other end. In this direct transmission path, data does not need to pass through the CPUs of the local end and the other end, thus realizing direct transmission of data from the local GPU to the other end GPU. Specifically, Figure 4 In the example, the communication from GPU1 to GPU2 can be completed through steps ① and ②. Step ① is to copy the data to be transmitted from the HBM of GPU1 directly to the HBM of GPU2 through the network cards NIC1 and NIC2. Step ② is to perform layout conversion on the data in GPU2 so that the layout of the transmitted data can conform to the layout of GPU2.

[0055] It is understandable that compared to Figure 2 In the related technology shown, it is necessary to implement the communication between the two GPUs through steps ① to ④. In the embodiment of the present invention, the communication between the two GPUs is implemented through a direct transmission path, which can avoid multiple data copies from GPU to CPU and from CPU to GPU, thereby greatly reducing communication delay and improving communication efficiency.

[0056] The premise of constructing a direct transmission path between the heterogeneous local artificial intelligence chip and the opposite artificial intelligence chip to realize direct communication between the local artificial intelligence chip and the opposite artificial intelligence chip is that the data transmission interface adapted to the local artificial intelligence chip and the opposite artificial intelligence chip are respectively called. It can be understood that compared with the RDMA technology in the related technology that only supports specific manufacturers or specific models, resulting in a large number of artificial intelligence chips being unable to adapt to the software programs of RDMA technology, since the universal interface in this application can integrate the data transmission interfaces adapted to various artificial intelligence chips, both the local and opposite artificial intelligence chips can call the data transmission interface adapted to themselves through the universal interface, thereby ensuring the compatibility of various artificial intelligence chips in the communication method provided in the embodiment of the present invention. Based on the data transmission interface that is compatible with themselves, both the local artificial intelligence chip and the other artificial intelligence chip can perform data transmission operations that are compatible with themselves, including memory registration, memory copy, etc., thereby breaking the communication barrier caused by the heterogeneity between the local artificial intelligence chip and the other artificial intelligence chip, so as to realize direct communication between the local artificial intelligence chip and the other artificial intelligence chip. Therefore, zero-copy transmission (GPU Direct RDMA, GDR) between heterogeneous artificial intelligence chips can be realized without bypassing the CPU.

[0057] In the method provided in the embodiment of the present invention, a universal interface is integrated with data transmission interfaces respectively compatible with various artificial intelligence chips, thereby realizing unified management of data transmission operations for various artificial intelligence chips. Based on this, the data transmission interface compatible with various artificial intelligence chips can be called to realize direct communication with various artificial intelligence chips, thereby reducing the communication delay between heterogeneous artificial intelligence chips and improving the communication efficiency between heterogeneous artificial intelligence chips.

[0058] Moreover, since the universal interface integrates data transmission interfaces that are compatible with various types of artificial intelligence chips, it can support various current and future artificial intelligence chips. The communication method based on the universal interface has excellent compatibility and scalability, and can be easily integrated into the RDMA communication framework, providing support for future technological development.

[0059] In addition, compared with the inter-chip communication realized through the CPU, the direct communication between various artificial intelligence chips in the embodiment of the present invention can improve the utilization rate of PCIE (Peripheral Component Interconnect Express) bandwidth, thereby optimizing the overall performance of the computing system.

[0060] Based on the above embodiment, the data transmission interface includes a memory copy interface; In step 320, the communication with the peer artificial intelligence chip through a direct transmission path based on a data transmission interface compatible with the local artificial intelligence chip includes: Based on a memory copy interface compatible with the local artificial intelligence chip, memory copy is performed in the target memory area of ​​the local artificial intelligence chip, and after the copy is completed, the data in the target memory area is transmitted to the other-end artificial intelligence chip through the direct transmission path.

[0061] Specifically, in the universal interface, a memory copy interface compatible with various artificial intelligence chips is integrated. Here, the memory copy interface is a type of interface in the data transmission interface used to implement memory copy operations. It can be understood that different artificial intelligence chips can correspond to different memory copy interfaces, and the memory copy interfaces corresponding to different artificial intelligence chips have the same functionality, that is, they are all used to implement memory copy operations.

[0062] It can be understood that memory copy refers to the process of copying data from one memory area to another memory area. In the embodiment of the present invention, it can specifically refer to the process of copying data from a memory area of ​​the local artificial intelligence chip to the target memory area of ​​the local artificial intelligence chip. Here, the target memory area is a memory area for storing data used to communicate with the other end artificial intelligence chip. The target memory area can be any memory area in the local artificial intelligence chip, or a memory area pre-divided by the local artificial chip for communication with the other end artificial intelligence chip. The embodiment of the present invention does not specifically limit this.

[0063] Accordingly, during the communication process, memory copy can be performed on the data that needs to be transmitted in the local artificial intelligence chip based on the memory copy interface that is compatible with the local artificial intelligence chip, so that the data that needs to be transmitted can be stored in the target memory area of ​​the local artificial intelligence chip. After the memory copy is completed, the local artificial intelligence chip can transmit the data in the target memory area to the other end artificial intelligence chip through a direct transmission path, thereby realizing communication between the local artificial intelligence chip and the other end artificial intelligence chip.

[0064] Based on any of the above embodiments, the data transmission interface further includes a memory registration interface; In step 320, the memory copy is performed in the target memory area of ​​the local artificial intelligence chip based on the memory copy interface compatible with the local artificial intelligence chip, and the process also includes: Based on a memory registration interface compatible with the local artificial intelligence chip, memory registration is performed on the local artificial intelligence chip to obtain a target memory area in the memory of the local artificial intelligence chip.

[0065] Specifically, in the universal interface, a memory registration interface compatible with various artificial intelligence chips is also integrated. Here, the memory registration interface is a type of interface used to implement memory registration operations in the data transmission interface. It can be understood that different artificial intelligence chips can correspond to different memory registration interfaces, and the memory registration interfaces corresponding to different artificial intelligence chips have the same functionality, that is, they are all used to implement memory registration operations.

[0066] It is understandable that memory registration refers to the operation of marking a memory area as a specific purpose or state. In the embodiment of the present invention, it can specifically refer to the operation of marking a memory area in the local artificial intelligence chip as a memory area for communicating with the peer artificial intelligence chip.

[0067] Accordingly, during the communication process, the registration operation for the memory area of ​​the local artificial intelligence chip can be implemented through a direct transmission path based on a memory registration interface compatible with the local artificial intelligence chip, thereby registering a memory area in the memory area of ​​the local artificial intelligence chip for communication with the opposite artificial intelligence chip, that is, obtaining the target memory area in the local artificial intelligence chip. After completing the memory registration, subsequent memory copy operations based on the memory copy interface can be to copy the data to be transmitted to the target memory area of ​​the local artificial intelligence chip. After the memory copy is completed, the local artificial intelligence chip can transmit the data in the target memory area to the opposite artificial intelligence chip through a direct transmission path.

[0068] In the method provided in the embodiment of the present invention, the two operations of memory registration and memory copy are abstracted respectively to realize data transmission. Therefore, when heterogeneous artificial intelligence chips communicate with each other, memory registration and memory copy for the local artificial intelligence chip can be realized by selecting a memory registration interface and a memory copy interface adapted to the local artificial intelligence chip, thereby ensuring efficient communication between heterogeneous artificial intelligence chips.

[0069] Moreover, the abstraction of the memory registration interface and the memory copy interface can realize unified management of memory registration and copying for different types of artificial intelligence chips, thereby improving the communication efficiency and reliability between chips.

[0070] For example, Figure 5 FIG. 3 is a flow chart of the communication method provided by the present invention. Figure 5 As shown, a universal interface can be installed at each of the two artificial intelligence chips.

[0071] At each artificial intelligence chip, a memory registration interface that is compatible with itself can be called through a general interface to allocate device memory, thereby realizing memory registration. It can be understood that Figure 5The result of the memory registration in is to obtain the target memory area at each intelligent chip, and register the target memory area to the network card connected to itself, so that the network card can send the data in the target memory area to the network card connected to the peer artificial intelligence chip during the communication process, and write the data received through the network card connected to the peer artificial intelligence chip into the target memory area.

[0072] In addition, at each AI chip, the memory copy interface that is compatible with itself can be adjusted through the universal interface to copy the data to be transmitted in the user buffer to the target memory area to achieve device memory copy. After the memory copy is completed, the data sending operation of the network card connected to itself can be triggered, thereby achieving communication between AI chips.

[0073] In addition, before any two artificial intelligence chips communicate, the two artificial intelligence chips need to perform the following steps to initialize the communication between the chips: First, initialize the RDMA device, where the RDMA device can be an artificial intelligence chip.

[0074] Subsequently, a PD (Protection Domain), CQ (Completion Queue), and QP (Queue Pair) are created. Subsequently, a QP connection can be established and the QP information of the communicating parties can be exchanged.

[0075] After that, at each AI chip, the memory registration interface adapted to itself can be called to register the target memory area to achieve memory creation. In addition, during the data transmission and reception process, at each AI chip, the memory copy interface adapted to itself can be called to copy the data to be transmitted to the target memory area to achieve memory copy.

[0076] Based on any of the above embodiments, in step 310, calling a data transmission interface compatible with the local artificial intelligence chip based on a universal interface installed at the local artificial intelligence chip includes: Based on the universal interface installed at the local artificial intelligence chip, a data transmission interface compatible with at least one of the manufacturer and product model of the local artificial intelligence chip is called.

[0077] Specifically, in order to realize communication between artificial intelligence chips, for the artificial intelligence chip on this side, based on the universal interface, the data transmission interface compatible with the artificial intelligence chip on this side can be determined and called from the data transmission interfaces integrated in the universal interface that are compatible with various artificial intelligence chips.

[0078] Since artificial intelligence chips from different manufacturers may achieve data transmission through different data transmission operations, or artificial intelligence chips of different product models from the same manufacturer may also achieve data transmission through different data transmission operations, when determining the data transmission interface compatible with the artificial intelligence chip on this side, at least one of the manufacturer and product model of the artificial intelligence chip on this side can be used as the identifier of the data transmission interface compatible with the artificial intelligence chip on this side.

[0079] Therefore, the data transmission interface compatible with the artificial intelligence chip on this end can specifically be a data transmission interface compatible with the manufacturer of the artificial intelligence chip on this end, or a data transmission interface compatible with the product model of the artificial intelligence chip on this end, or a data transmission interface compatible with both the manufacturer and product model of the artificial intelligence chip on this end.

[0080] Based on any of the above embodiments, in step 320, the communicating with the other end artificial intelligence chip through a direct transmission path based on a data transmission interface compatible with the local end artificial intelligence chip includes: Based on a data transmission interface compatible with the local artificial intelligence chip, a data block is transmitted to the other artificial intelligence chip through a direct transmission path, and the size of the data block is a target size; The target size is one of multiple candidate sizes.

[0081] Specifically, in the communication process between the artificial intelligence chip on the local end and the artificial intelligence chip on the other end, the data transmitted based on the data transmission interface compatible with the artificial intelligence chip on the local end can be transmitted in the form of data blocks, that is, the data to be transmitted can be divided into multiple data blocks, and the multiple data blocks are transmitted in sequence to form a data stream.

[0082] The data block here is a data unit with a fixed structure and format. The size of the data block can be adjusted according to the actual transmission scenario. The size of the data block is recorded as a target size here, and the target size can be one of multiple candidate sizes.

[0083] That is, multiple candidate sizes can be preset. Each candidate size here may be selected as the target size of the data block. In an actual transmission scenario, one of the multiple candidate sizes can be selected as the target size, and data transmission can be performed accordingly. It can be understood that the candidate size selected as the target size here can be the candidate size with the highest transmission efficiency among the multiple candidate sizes, or the default candidate size of the local artificial intelligence chip, or the candidate size used in the previous communication with the peer artificial intelligence chip, and the embodiment of the present invention does not specifically limit this.

[0084] Based on any of the above embodiments, before step 320, the method further includes: Dividing the test data into a plurality of data blocks of the candidate size; Based on a data transmission interface compatible with the local artificial intelligence chip, multiple data blocks of the candidate size are sent to the opposite artificial intelligence chip through a direct transmission path to obtain a data transmission time consumption of the candidate size; The target size is determined based on the data transmission time consumption under the multiple candidate sizes.

[0085] Specifically, before realizing communication between the local artificial intelligence chip and the opposite artificial intelligence chip based on the data transmission interface, the transmission efficiency of each candidate size between the local artificial intelligence chip and the opposite artificial intelligence chip can be tested first, thereby selecting the candidate size with the highest transmission efficiency as the target size.

[0086] In this process, the test data can be determined first. The test data here is the data used to test the transmission efficiency of each candidate size. The test data can be pre-constructed data or any other data that can be used for communication between artificial intelligence chips.

[0087] After obtaining the test data, the test data can be divided into multiple data blocks under the candidate sizes. It can be understood that in the case where there are multiple candidate sizes, the test data needs to be divided for each candidate size, so that under each candidate size, there are multiple data blocks with the same size as the candidate size.

[0088] For each candidate size, multiple data blocks of the candidate size can be transmitted to the other end artificial intelligence chip through a direct transmission path based on a data transmission interface compatible with the local artificial intelligence chip, and the time used to transmit the multiple data blocks of the candidate size can be recorded, that is, the data transmission time of the candidate size can be obtained. It can be understood that since data blocks are transmitted for each candidate size, each candidate size can obtain the corresponding data transmission time, and the data transmission time here can reflect the communication efficiency between the local artificial intelligence chip and the other end artificial intelligence chip of different candidate sizes under the same amount of transmission data. It can be understood that the longer the data transmission time, the lower the communication efficiency, and the shorter the data transmission time, the higher the communication efficiency.

[0089] Therefore, after obtaining the data transmission time under each candidate size, one can be selected from the candidate sizes as the target size based on the data transmission time under each candidate size. For example, the candidate size with the shortest data transmission time can be selected from multiple candidate sizes as the target size.

[0090] In the method provided in an embodiment of the present invention, the target size is determined based on the data transmission time under multiple candidate sizes, which helps to improve the communication efficiency between artificial intelligence chips.

[0091] Based on any of the above embodiments, before step 310, the method further includes: When it is determined that the local artificial intelligence chip supports direct memory access between chips and a universal interface is installed at the local artificial intelligence chip, universal interface registration is performed.

[0092] Specifically, before calling the data transmission interface based on the universal interface at the local artificial intelligence chip, function judgment, installation detection and interface registration can be performed on the local artificial intelligence chip.

[0093] Among them, the function judgment refers to judging whether the local artificial intelligence chip itself supports direct memory access between chips, that is, whether the artificial intelligence chip itself has the GDR (GPU Direct RDMA) function. It can be understood that the communication method provided in the embodiment of the present invention is only executed under the condition that the local artificial intelligence chip supports direct memory access between chips. If the local artificial intelligence chip does not support direct memory access between chips, the local artificial intelligence chip needs to communicate with the other artificial intelligence chip through a host such as a CPU.

[0094] Installation detection is used to detect whether a universal interface is installed at the local artificial intelligence chip. It can be understood that the communication method provided in the embodiment of the present invention is only executed under the condition that the local artificial intelligence chip is installed with a universal interface. If the local artificial intelligence chip is not installed with a universal interface, the local artificial intelligence chip cannot call the data transmission interface adapted to itself through the universal interface, and needs to communicate with the other artificial intelligence chip through a host such as a CPU.

[0095] Interface registration means registering the universal interface as a plug-in at the local artificial intelligence chip, so that the local artificial intelligence chip can manage and use the universal interface, so as to call the data transmission interface compatible with itself through the universal interface for communication. It can be understood that the communication method provided in the embodiment of the present invention requires interface registration first, and then calling the data transmission interface based on the universal interface. Once the interface registration fails, the data transmission interface cannot be called through the universal interface. At this time, the local artificial intelligence chip needs to communicate with the other artificial intelligence chip through a host such as a CPU.

[0096] For example, Figure 6 FIG. 4 is a flow chart of the communication method provided by the present invention. Figure 6As shown, for the local artificial intelligence chip, the GDR function can be detected first, that is, whether the local artificial intelligence chip supports the GDR function, that is, whether the local artificial intelligence chip itself supports direct memory access between chips. If it is detected that the local artificial intelligence chip supports the GDR function, proceed to the next step, otherwise, the communication between chips is carried out through RDMA communication between CPUs.

[0097] When it is detected that the local artificial intelligence chip supports the GDR function, a GDR plug-in detection can be performed. The GDR plug-in here is the universal interface referred to in the embodiment of the present invention. The GDR plug-in detection is to detect whether the local artificial intelligence chip is installed with a universal interface. If it is detected that the local artificial intelligence chip is installed with a universal interface, the next step is entered. Otherwise, the communication between chips is carried out through RDMA communication between CPUs.

[0098] It is detected that the local artificial intelligence chip is installed with a universal interface, and GDR plug-in registration can be performed. The GDR plug-in registration here is the interface registration of the universal interface referred to in the embodiment of the present invention. If the interface registration is successful, proceed to the next step, otherwise the inter-chip communication is performed through RDMA communication between CPUs.

[0099] When the interface registration is successful, GDR can be performed. Specifically, based on the universal interface installed at the local artificial intelligence chip, the data transmission interface compatible with the local artificial intelligence chip can be called to achieve communication with the other end artificial intelligence chip. The communication referred to here may include multiple rounds of data sending and / or data receiving operations.

[0100] The communication device provided by the present invention is described below. The communication device described below and the communication method described above can be referenced to each other.

[0101] Figure 7 is a schematic diagram of the structure of the communication device provided by the present invention, such as Figure 7 As shown, the device comprises: A calling unit 710 is used to call a data transmission interface compatible with the local artificial intelligence chip based on a universal interface installed at the local artificial intelligence chip, wherein the universal interface is integrated with data transmission interfaces compatible with various types of artificial intelligence chips; The communication unit 720 is used to communicate with the other-end artificial intelligence chip through a direct transmission path based on the data transmission interface adapted to the local artificial intelligence chip, the universal interface being installed at the other-end artificial intelligence chip, and the data transmission interface adapted to the other-end artificial intelligence chip is called through the universal interface; The direct transmission path is composed of the local artificial intelligence chip, the local network card connected to the local artificial intelligence chip, the opposite network card connected to the opposite artificial intelligence chip, and the opposite artificial intelligence chip.

[0102] In the device provided by the embodiment of the present invention, a universal interface is integrated with data transmission interfaces respectively compatible with various artificial intelligence chips, thereby realizing unified management of data transmission operations for various artificial intelligence chips. Based on this, the data transmission interface compatible with various artificial intelligence chips can be called to realize direct communication with various artificial intelligence chips, thereby reducing the communication delay between heterogeneous artificial intelligence chips and improving the communication efficiency between heterogeneous artificial intelligence chips.

[0103] Based on the above embodiment, the data transmission interface includes a memory copy interface; The communication unit is specifically used for: Based on a memory copy interface compatible with the local artificial intelligence chip, memory copy is performed in the target memory area of ​​the local artificial intelligence chip, and after the copy is completed, the data in the target memory area is transmitted to the other-end artificial intelligence chip through the direct transmission path.

[0104] Based on any of the above embodiments, the data transmission interface further includes a memory registration interface; The communication unit is also used for: Based on a memory registration interface compatible with the local artificial intelligence chip, memory registration is performed on the local artificial intelligence chip to obtain a target memory area in the memory of the local artificial intelligence chip.

[0105] Based on any of the above embodiments, the calling unit is specifically used for: Based on the universal interface installed at the local artificial intelligence chip, a data transmission interface compatible with at least one of the manufacturer and product model of the local artificial intelligence chip is called.

[0106] Based on any of the above embodiments, the communication unit is specifically used for: Based on a data transmission interface compatible with the local artificial intelligence chip, a data block is transmitted to the other artificial intelligence chip through a direct transmission path, and the size of the data block is a target size; The target size is one of multiple candidate sizes.

[0107] Based on any of the above embodiments, the communication unit is further used for: Dividing the test data into a plurality of data blocks of the candidate size; Based on a data transmission interface compatible with the local artificial intelligence chip, multiple data blocks of the candidate size are sent to the opposite artificial intelligence chip through a direct transmission path to obtain a data transmission time consumption of the candidate size; The target size is determined based on the data transmission time consumption under the multiple candidate sizes.

[0108] Based on any of the above embodiments, the calling unit is further used for: When it is determined that the local artificial intelligence chip supports direct memory access between chips and a universal interface is installed at the local artificial intelligence chip, universal interface registration is performed.

[0109] Figure 8 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 8 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the communication method, which includes: Based on the universal interface installed at the local artificial intelligence chip, a data transmission interface adapted to the local artificial intelligence chip is called, wherein the universal interface is integrated with data transmission interfaces adapted to various artificial intelligence chips respectively; Based on the data transmission interface compatible with the local artificial intelligence chip, communicating with the opposite artificial intelligence chip through a direct transmission path, the opposite artificial intelligence chip is installed with the universal interface, and the data transmission interface compatible with the opposite artificial intelligence chip is called through the universal interface; The direct transmission path is composed of the local artificial intelligence chip, the local network card connected to the local artificial intelligence chip, the opposite network card connected to the opposite artificial intelligence chip, and the opposite artificial intelligence chip.

[0110] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0111] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the communication method provided by the above methods, the method comprising: Based on the universal interface installed at the local artificial intelligence chip, a data transmission interface adapted to the local artificial intelligence chip is called, wherein the universal interface is integrated with data transmission interfaces adapted to various artificial intelligence chips respectively; Based on the data transmission interface compatible with the local artificial intelligence chip, communicating with the opposite artificial intelligence chip through a direct transmission path, the opposite artificial intelligence chip is installed with the universal interface, and the data transmission interface compatible with the opposite artificial intelligence chip is called through the universal interface; The direct transmission path is composed of the local artificial intelligence chip, the local network card connected to the local artificial intelligence chip, the opposite network card connected to the opposite artificial intelligence chip, and the opposite artificial intelligence chip.

[0112] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the communication method provided by the above methods is implemented, and the method includes: Based on the universal interface installed at the local artificial intelligence chip, a data transmission interface adapted to the local artificial intelligence chip is called, wherein the universal interface is integrated with data transmission interfaces adapted to various artificial intelligence chips respectively; Based on the data transmission interface compatible with the local artificial intelligence chip, communicating with the opposite artificial intelligence chip through a direct transmission path, the opposite artificial intelligence chip is installed with the universal interface, and the data transmission interface compatible with the opposite artificial intelligence chip is called through the universal interface; The direct transmission path is composed of the local artificial intelligence chip, the local network card connected to the local artificial intelligence chip, the opposite network card connected to the opposite artificial intelligence chip, and the opposite artificial intelligence chip.

[0113] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0114] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A communication method, characterized in that: include: Based on the universal interface installed at the local artificial intelligence chip, a data transmission interface adapted to the local artificial intelligence chip is called, wherein the universal interface is integrated with data transmission interfaces adapted to various artificial intelligence chips respectively; Based on the data transmission interface compatible with the local artificial intelligence chip, communicating with the opposite artificial intelligence chip through a direct transmission path, the opposite artificial intelligence chip is installed with the universal interface, and the data transmission interface compatible with the opposite artificial intelligence chip is called through the universal interface; The direct transmission path is composed of the local artificial intelligence chip, the local network card connected to the local artificial intelligence chip, the opposite network card connected to the opposite artificial intelligence chip, and the opposite artificial intelligence chip.

2. The communication method according to claim 1, characterized in that: The data transmission interface includes a memory copy interface; The method of communicating with the other-end artificial intelligence chip through a direct transmission path based on a data transmission interface compatible with the local artificial intelligence chip includes: Based on a memory copy interface compatible with the local artificial intelligence chip, memory copy is performed in the target memory area of ​​the local artificial intelligence chip, and after the copy is completed, the data in the target memory area is transmitted to the other-end artificial intelligence chip through the direct transmission path.

3. The communication method according to claim 2, characterized in that: The data transmission interface also includes a memory registration interface; The memory copying is performed in the target memory area of ​​the local artificial intelligence chip based on the memory copy interface compatible with the local artificial intelligence chip, and the method further includes: Based on a memory registration interface compatible with the local artificial intelligence chip, memory registration is performed on the local artificial intelligence chip to obtain a target memory area in the memory of the local artificial intelligence chip.

4. The communication method according to claim 1, characterized in that: The method of calling a data transmission interface compatible with the local artificial intelligence chip based on a universal interface installed at the local artificial intelligence chip includes: Based on the universal interface installed at the local artificial intelligence chip, a data transmission interface compatible with at least one of the manufacturer and product model of the local artificial intelligence chip is called.

5. The communication method according to claim 1, characterized in that: The method of communicating with the other-end artificial intelligence chip through a direct transmission path based on a data transmission interface compatible with the local artificial intelligence chip includes: Based on a data transmission interface compatible with the local artificial intelligence chip, a data block is transmitted to the other artificial intelligence chip through a direct transmission path, and the size of the data block is a target size; The target size is one of multiple candidate sizes.

6. The communication method according to claim 5, characterized in that: The method further comprises: communicating with the other end artificial intelligence chip through a direct transmission path based on a data transmission interface compatible with the local artificial intelligence chip; Dividing the test data into a plurality of data blocks of the candidate size; Based on a data transmission interface compatible with the local artificial intelligence chip, multiple data blocks of the candidate size are sent to the opposite artificial intelligence chip through a direct transmission path to obtain a data transmission time consumption of the candidate size; The target size is determined based on the data transmission time consumption under the multiple candidate sizes.

7. The communication method according to any one of claims 1 to 6, characterized in that: The method further includes: calling a data transmission interface compatible with the local artificial intelligence chip based on a universal interface installed at the local artificial intelligence chip; When it is determined that the local artificial intelligence chip supports direct memory access between chips and a universal interface is installed at the local artificial intelligence chip, universal interface registration is performed.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the communication method according to any one of claims 1 to 7 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the communication method according to any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the communication method according to any one of claims 1 to 7 is implemented.

Images