Communication control method, communication system, electronic device, and storage medium

Abstract

Embodiments of the present disclosure provide a communication control method, a communication system, an electronic device and a storage medium. The method is used for a first node in a communication group, and includes: obtaining a first pointer and a second pointer of to-be-transmitted data; determining a second-level buffer for the to-be-transmitted data in the first node according to the first pointer and a data transmission capability of the first node; in response to the first-level buffer and the second-level buffer in the first node both belonging to an address space of a second-type processor, and the first-level buffer and the second-level buffer in the second node both belonging to an address space of a first-type processor, determining that a target transmission channel between the first node and the second node is a direct transmission channel between the address space of the second-type processor of the first node and the address space of the first-type processor of the second node. The method can automatically select a high-efficiency communication path, avoid introducing an additional data copy operation, and reduce the complexity of cross-memory-type communication.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of computer, and particularly to a communication control method, a communication system, an electronic device and a storage medium. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, artificial intelligence has been widely applied to various complex tasks. However, the traditional single machine environment faces the bottleneck of too long time consumption in large-scale model training. Distributed system becomes the mainstream solution to improve training efficiency through multi-node parallel computing. Heterogeneous computing platform can be applied to distributed architecture. By integrating multiple types of processors with different architectures to work together, the advantages of various types of processors in performance and function can be fully utilized to meet the increasingly complex computing needs. However, its heterogeneity also leads to increasing communication overhead between nodes, which becomes a problem to be solved. SUMMARY

[0003] To solve the problems of low communication performance and large overhead between nodes in the heterogeneous computing platform, at least one embodiment of the present disclosure provides a communication control method, a communication system, an electronic device and a storage medium.

[0004] The communication control method provided by at least one embodiment of the present disclosure is used for a first node in a communication group, and includes: obtaining a first pointer and a second pointer of to-be-transmitted data, wherein the first pointer points to a first-level buffer for the to-be-transmitted data in the first node, the communication group further includes a second node, the second pointer points to a second-level buffer for the to-be-transmitted data in the second node, and each node in the communication group includes a first-type processor and a second-type processor; determining a second-level buffer for the to-be-transmitted data in the first node according to the first pointer and a data transmission capability of the first node; in response to the first-level buffer and the second-level buffer in the first node both belonging to an address space of the second-type processor, and the first-level buffer and the second-level buffer in the second node both belonging to an address space of the first-type processor, determining that a target transmission channel for transmitting the to-be-transmitted data between the first node and the second node is a direct transmission channel between the address space of the second-type processor of the first node and the address space of the first-type processor of the second node.

[0005] For example, in the communication control method provided by at least one embodiment of the present disclosure, the data transmission capability indicates whether to allow inter-node data transmission based on the address space of which type of processor in the corresponding node, or not to allow inter-node data transmission based on the address space of which type of processor in the corresponding node.

[0006] For example, in the communication control method provided in at least one of embodiments of the present disclosure, after the first pointer and the second pointer of the data to be transmitted are acquired, the communication control method further includes: identifying a type of an address space in which the first level buffer pointed to by the first pointer is located; and determining whether the type of the address space in which the first level buffer pointed to by the first pointer is located matches the data transmission capability of the first node.

[0007] For example, in the communication control method provided in at least one of embodiments of the present disclosure, the determination of whether the address space in which the first level buffer pointed to by the first pointer is located matches the data transmission capability of the first node includes: in response to the type of the address space of the processor allowed by the data transmission capability of the first node including the type of the address space in which the first level buffer pointed to by the first pointer is located, determining that the address space in which the first level buffer pointed to by the first pointer is located matches the data transmission capability of the first node.

[0008] For example, in the communication control method provided in at least one of embodiments of the present disclosure, the determination of the second level buffer for the data to be transmitted in the first node according to the first pointer and the data transmission capability of the first node includes: in response to the first level buffer pointed to by the first pointer belonging to the address space of the second type processor and the address space of the processor allowed by the data transmission capability of the first node including the address space of the second type processor, determining that the second level buffer for the data to be transmitted in the first node is the second level buffer in the address space of the second type processor; or in response to the first level buffer pointed to by the first pointer belonging to the address space of the second type processor and the address space of the processor allowed by the data transmission capability of the first node not including the address space of the second type processor, determining that the second level buffer for the data to be transmitted in the first node is the second level buffer in the address space of the first type processor.

[0009] For example, the communication control method provided in at least one of embodiments of the present disclosure further includes: determining a target transmission channel between the first node and the second node according to the first pointer, the second pointer, and the data transmission capability of the first node.

[0010] For example, in the communication control method provided in at least one of embodiments of the present disclosure, the determination of the target transmission channel between the first node and the second node according to the first pointer, the second pointer, and the data transmission capability of the first node includes: determining a plurality of transmission channels according to the first pointer and the second pointer; and determining the target transmission channel from the plurality of transmission channels according to the data transmission capability of the first node.

[0011] For example, in the communication control method provided by at least one of the embodiments of the present disclosure, the target transmission channel is determined from the plurality of transmission channels according to the data transmission capability of the first node, including: determining candidate transmission channels that meet the data transmission capability of the first node from the plurality of transmission channels; and selecting a channel that at least one of has the highest affinity or requires the least number of data copies as the target transmission channel from the candidate transmission channels.

[0012] For example, in the communication control method provided by at least one of the embodiments of the present disclosure, the target transmission channel between the first node and the second node is determined according to the first pointer, the second pointer, and the data transmission capability of the first node, including: in response to the address space where the first-level buffer pointed to by the first pointer does not match the data transmission capability of the first node, determining the target transmission channel as: sending the to-be-transmitted data from the first-level buffer pointed to by the first pointer to the second-level buffer pointed to by the second pointer via the second-level buffer corresponding to the first processor in the first node, wherein the first processor is different from the processor corresponding to the address space where the first-level buffer pointed to by the first pointer is located.

[0013] For example, the communication control method provided by at least one of the embodiments of the present disclosure further includes: determining the data transmission capability of the first node according to the communication routing table, wherein the step of creating the communication routing table includes: detecting a local address space in the first node that allows inter-node data transmission, and registering address space information of the second-level buffer in the local address space for the first node; obtaining address space information of the second-level buffer in a remote address space registered for the second node; and creating the communication routing table according to the address space information registered for the first node and the second node.

[0014] For example, in the communication control method provided by at least one of the embodiments of the present disclosure, the address space information in the communication routing table at least includes the address space type of the second-level buffer, and the address space type includes a first type and a second type, the second-level buffer of the first type belongs to the address space of the first type processor in the corresponding node, and the second-level buffer of the second type belongs to the address space of the second type processor in the corresponding node.

[0015] For example, in the communication control method provided by at least one of the embodiments of the present disclosure, the data transmission capability of the first node is determined according to the communication routing table, including: in response to the address space type of the second-level buffer in the local address space including the first type and / or the second type, determining that the data transmission capability of the first node includes allowing inter-node data transmission based on the address space of the processor of the corresponding type.

[0016] For example, in the communication control method provided by at least one embodiment of the present disclosure, the address space information in the communication routing table further includes a node identifier, a start address and a buffer capacity corresponding to each registered second-level buffer.

[0017] At least one embodiment of the present disclosure further provides a communication system, including a plurality of nodes, wherein each node includes a communication interface, a communication controller and at least two types of processors, the at least two types of processors including a first type of processor and a second type of processor, and the plurality of nodes including a first node and a second node; the communication interface of the first node is configured to: in response to the first node triggering a data transmission operation between the first node and the second node, obtain a first pointer and a second pointer of to-be-transmitted data, and determine a second-level buffer in the first node for the to-be-transmitted data according to the first pointer and a data transmission capability of the first node, wherein the first pointer points to a first-level buffer in the first node for the to-be-transmitted data, and the second pointer points to a second-level buffer in the second node for the to-be-transmitted data; and the communication controller of the first node is configured to: in response to the first-level buffer and the second-level buffer in the first node both belonging to an address space of the second type of processor, and the first-level buffer and the second-level buffer in the second node both belonging to an address space of the first type of processor, determine that a target transmission channel for transmitting the to-be-transmitted data between the first node and the second node is a direct transmission channel between the address space of the second type of processor of the first node and the address space of the first type of processor of the second node.

[0018] At least one embodiment of the present disclosure further provides an electronic device, including: at least one processor; and at least one memory, wherein the at least one memory has program code stored therein, and the program code, when executed by the at least one processor, causes the at least one processor to perform the communication control method provided by any one of the embodiments of the present disclosure.

[0019] At least one embodiment of the present disclosure provides a non-transitory computer-readable storage medium having computer-readable instructions stored thereon, and the computer-readable instructions, when executed by at least one processor, cause the processor to perform the communication control method provided by any one of the embodiments of the present disclosure.

[0020] In the communication control method provided by at least one embodiment of the present disclosure, the target transmission channel of the to-be-transmitted data is determined through the address space where the first level buffer pointed by the first pointer of the to-be-transmitted data is located and the data transmission capability of the first node, the data direct transmission operation between the address spaces of different types of processors of the heterogeneous nodes (for example, the data direct transmission operation between the address space of the second type processor of the first node and the address space of the first type processor of the second node) can be realized, the efficient communication path (data transmission channel) can be automatically scheduled, the additional data copy operation is avoided, and the path code for data transmission between different nodes does not need to be written, thereby improving the communication efficiency and simplifying the complexity of user programming; the member nodes using different types of processor address spaces can be supported in the same communication group, the complexity of cross-memory type communication is reduced, and the complexity of user programming is simplified. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, not limit the present disclosure.

[0022] Figure 1 A communication mode of an exemplary heterogeneous computing platform is shown.

[0023] Figure 2 A flowchart of the communication control method provided by at least one embodiment of the present disclosure is shown.

[0024] Figure 3 A flowchart of creating a communication routing table provided by at least one embodiment of the present disclosure is shown.

[0025] Figure 4 A schematic diagram of a data transmission process provided by at least one embodiment of the present disclosure is shown.

[0026] Figure 5 A flowchart of another communication control method provided by at least one embodiment of the present disclosure is shown.

[0027] Figure 6A A schematic diagram of another data transmission process provided by at least one embodiment of the present disclosure is shown.

[0028] Figure 6B A schematic diagram of another data transmission process provided by at least one embodiment of the present disclosure is shown.

[0029] Figure 7A A schematic diagram of a communication system provided by at least one embodiment of the present disclosure is shown.

[0030] Figure 7BA communication flow diagram between nodes provided by at least one embodiment of the present disclosure is shown.

[0031] Figure 8 A structural diagram of an electronic device provided by at least one embodiment of the present disclosure is shown.

[0032] Figure 9 A structural diagram of a non-transitory computer-readable storage medium provided by at least one embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions of the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are some but not all of the embodiments of the present disclosure. Based on the described embodiments of the present disclosure, all other embodiments obtained by a person of ordinary skill in the art without creative effort belong within the scope of the present disclosure.

[0034] Unless otherwise defined, technical terms or scientific terms used in the present disclosure should be understood as having the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs. The terms “first”, “second”, and similar terms used in the present disclosure do not denote any order, quantity, or importance, but are used to distinguish different components. Similarly, the terms “one”, “a”, or “the” or similar terms do not denote a quantity restriction, but mean that there is at least one. The terms “include”, “contain”, or “comprise” or similar terms mean that the elements or objects before the terms encompass the elements or objects listed after the terms and their equivalents, and do not exclude other elements or objects. The terms “connect” or “connected” or similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “up”, “down”, “left”, “right”, and the like only represent relative positional relationships, and when the absolute positions of the described objects change, the relative positional relationships can also change accordingly.

[0035] A distributed system refers to a system composed of multiple dispersed nodes connected by a network. The nodes work together in the distributed system to complete one or more target tasks, such as training or reasoning of a large-scale language model.

[0036] A communication group refers to a set of nodes logically divided for a specific communication purpose (such as data reading and writing, data exchange, task coordination, state synchronization, etc.) in a distributed system or parallel computing environment. Nodes within the same communication group can directly communicate with each other, and nodes outside the group usually do not participate in the internal interaction of the communication group (unless through a cross-group mechanism).

[0037] The "node" in the present disclosure can be a physical device (hardware device), such as a computer, a server, an edge computing device, etc., or a virtual device, such as a virtual machine, etc., and the present disclosure does not limit the same.

[0038] A heterogeneous computing platform can be applied to a distributed architecture, and by integrating multiple types of processors with different architectures to work together, the advantages of various types of processors in performance and function can be fully utilized to meet the increasingly complex computing needs. For example, the heterogeneous computing platform can be a computing platform adopting a central processing unit-graphics processing unit (CPU-GPU) architecture, or a computing platform adopting a central processing unit-general purpose graphics processing unit (CPU-GPGPU) architecture. In the heterogeneous computing platform, communication efficiency is crucial to overall performance.

[0039] Current mainstream communication libraries generally adopt a "single type memory communication group" mode. This mode requires that the communication operations in the communication group explicitly distinguish the memory type (also referred to as the "address type of data") where the data is located, for example, explicitly distinguishing a central processing unit pointer (hereinafter also referred to as a "CPU pointer", i.e., a pointer to the address space of a CPU) and a graphics processing unit pointer (hereinafter also referred to as a "GPU pointer", i.e., a pointer to the address space of a GPU), which brings many problems in actual application.

[0040] For example, the identification of the memory type relies on manual judgment, i.e., the user needs to manually specify the memory type (CPU or GPU) where the data is located. For example, the communication path (or data transmission channel) cannot be automatically scheduled, and the developer needs to select a communication method according to the address type of the data, or explicitly copy the data to a transit area before communication. This process introduces additional operation delay, reducing communication efficiency. For example, the communication group semantics lacks uniformity, and existing communication libraries do not support simultaneously containing member nodes using CPU address space and GPU address space in the same communication group, which limits the possibility of direct communication between heterogeneous nodes and increases the complexity of cross-memory type communication. For example, when integrating these communication libraries, deep learning frameworks or cluster communication middleware have difficulty in efficiently and automatically fusing the communication process between CPUs and GPUs, resulting in low software development efficiency and failing to fully utilize the communication performance potential of the heterogeneous computing platform. These problems seriously reduce the communication performance of the heterogeneous computing platform and the development efficiency of related software.

[0041] For example, Figure 1 An exemplary communication mode of a heterogeneous computing platform is shown.

[0042] As Figure 1As shown, the communication group includes at least node Rank0 and node Rank1, and each node includes a CPU and a GPU. DRAM_0 represents the address space (also known as "memory") corresponding to the CPU in node Rank0, and VRAM_0 represents the address space (also known as "video memory") corresponding to the GPU in node Rank0. The meanings of DRAM_1 and VRAM_1 are similar to those of DRAM_0 and VRAM_0, and will not be repeated here.

[0043] When node Rank0 transmits data to node Rank1, it is necessary to manually determine the type of the address space where the data is located and the type of the address space of the receiving data, and select the data transmission path based on the type of the address space where the data is located and the type of the address space of the receiving data.

[0044] exist Figure 1 In the example shown, data is transferred from VRAM_0 of node Rank0 to DRAM_1 of node Rank1, with the target transfer channel as follows: Figure 1 As shown by the dashed arrow, node Rank0 copies data Data directly from buffer V0 of VRAM_0 to buffer D1 of DRAM_1. However, due to the lack of uniformity in communication group semantics, only data transfer between address spaces of the same type is supported (e.g., data transfer between address spaces corresponding to the CPU, data transfer between address spaces corresponding to the GPU). The actual transmission channel is as follows: Figure 1 As shown by the solid arrow, node Rank0 typically copies data from buffer V0 of VRAM_0 to buffer V1 of VRAM_1 of node Rank1, and then node Rank1 copies data from buffer V1 of VRAM_1 to buffer D1 of DRAM_1. This data transfer introduces an extra data copy operation, thus reducing communication efficiency. The lack of semantic uniformity in communication groups limits heterogeneous nodes (e.g., Figure 1 The possibility of direct communication between nodes Rank0 and Rank1 in the system.

[0045] This disclosure provides at least one embodiment of a communication control method and a communication system.

[0046] The communication control method provided in at least one embodiment of the present disclosure is used for a first node in a communication group, and the method comprises the following steps: obtaining a first pointer and a second pointer of to-be-transmitted data, wherein the first pointer points to a first-level buffer for the to-be-transmitted data in the first node, the communication group further comprises a second node, the second pointer points to a second-level buffer for the to-be-transmitted data in the second node, and each node in the communication group comprises a first-type processor and a second-type processor; determining a second-level buffer for the to-be-transmitted data in the first node according to the first pointer and a data transmission capability of the first node; and in response to the fact that the first-level buffer and the second-level buffer in the first node both belong to an address space of the second-type processor, and the fact that the first-level buffer and the second-level buffer in the second node both belong to an address space of the first-type processor, determining that a target transmission channel for transmitting the to-be-transmitted data between the first node and the second node is a direct transmission channel between the address space of the second-type processor of the first node and the address space of the first-type processor of the second node.

[0047] In the communication control method provided in at least one embodiment of the present disclosure, the target transmission channel of the to-be-transmitted data is determined according to the address space in which the first-level buffer pointed to by the first pointer of the to-be-transmitted data is located and the data transmission capability of the first node, so that the data direct transmission operation between the address spaces of different types of processors of the heterogeneous nodes (for example, the data direct transmission operation between the address space of the second-type processor of the first node and the address space of the first-type processor of the second node) can be implemented, an efficient communication path (data transmission channel) can be automatically scheduled, an additional data copying operation can be avoided, and path code does not need to be written for data transmission between different nodes, thereby improving the communication efficiency and simplifying the complexity of user programming; the member nodes using different types of processor address spaces can be simultaneously included in the same communication group, the complexity of cross-memory type communication is reduced, the complexity of user programming is simplified, and system compatibility is improved.

[0048] The communication control method provided in at least one embodiment of the present disclosure can be applied to a Deep Learning training system and an Inference system. For example, the method can be applied to a checkpoint saving operation in a training process, and can also be applied to a Key-Value Cache (KVCache) transmission operation in an Inference system under a Prefill-Decode (PD) separation architecture.

[0049] For example, in a deep learning training scenario, checkpoint saving is a key operation to ensure the continuity of training, which requires reliable storage of a large amount of model parameters and training state data. The communication control method provided by at least one embodiment of the present disclosure can optimize the data transmission strategy according to the characteristics of this process, and ensure the complete saving and rapid recovery of model parameters without significantly affecting the training efficiency.

[0050] For example, in an inference system, when a PD separation architecture is adopted, the key-value cache is important intermediate data generated in the inference process, and its transmission efficiency directly affects the inference delay and system response speed. The communication control method provided by at least one embodiment of the present disclosure can improve the transmission efficiency of the key-value cache, and provide strong support for the high-performance operation of the inference system.

[0051] The present disclosure will be described below through several specific embodiments. In order to keep the following description of the embodiments of the present disclosure clear and concise, the detailed description of known functions and known components can be omitted. When any component of the embodiments of the present disclosure appears in more than one figure, the component is denoted by the same or similar reference numeral in each figure.

[0052] The communication control method provided by at least one embodiment of the present disclosure can be used in a first node in a communication group to implement data transmission with a second node in the communication group. It should be noted that the "first node" in the embodiments of the present disclosure refers to the node that initiates the data transmission operation in the communication group, that is, the initiator of the data transmission operation, and does not refer to a certain fixed node.

[0053] For example, when the communication mode is a write mode, the data transmission operation is actively triggered by the node of the data sending end, for example, a write operation is actively initiated to write data to the node of the data receiving end. In this case, the first node is the node of the data sending end, and the second node is the node of the data receiving end.

[0054] For example, when the communication mode is a read mode, the data transmission operation is actively triggered by the node of the data receiving end, for example, a read operation is actively initiated to read data from the node of the data sending end. At this time, although the data flow direction is from the sending end to the receiving end, the initiator of the operation is the node of the data receiving end. Therefore, in this case, the first node is the node of the data receiving end, and the second node is the node of the data sending end.

[0055] For example, when the communication mode is a send / recv mode, both the node of the data sending end and the node of the data receiving end trigger the data transmission operation, the node of the data sending end performs the send operation, and the node of the data receiving end performs the recv operation. Since the two operations are independently initiated by both parties, in their respective operation contexts, they are both the initiator of the data transmission operation. Therefore, when the communication control method provided by the embodiments of the present disclosure is applied from the perspective of the data sending end, the first node is the node of the data sending end, and the second node is the node of the data receiving end; when the communication control method provided by the embodiments of the present disclosure is applied from the perspective of the data receiving end, the first node is the node of the data receiving end, and the second node is the node of the data sending end.

[0056] In other words, in the communication control method provided by at least one embodiment of the present disclosure, the execution subject is the node that initiates the data transmission operation in the current communication context.

[0057] Figure 2 A flowchart of the communication control method provided by at least one embodiment of the present disclosure is shown.

[0058] For example, as shown in Figure 2 The communication control method provided by at least one embodiment of the present disclosure can include the following steps S210-S230.

[0059] Step S210: Obtain a first pointer and a second pointer of to-be-transmitted data, wherein the first pointer points to a first-level buffer for the to-be-transmitted data in the first node, the communication group further includes a second node, the second pointer points to a second-level buffer for the to-be-transmitted data in the second node, and each node in the communication group includes a first-type processor and a second-type processor.

[0060] Step S220: Determine a second-level buffer for the to-be-transmitted data in the first node according to the first pointer and a data transmission capability of the first node.

[0061] Step S230: In response to the first-level buffer and the second-level buffer in the first node both belonging to the address space of the second-type processor, and the first-level buffer and the second-level buffer in the second node both belonging to the address space of the first-type processor, determine that a target transmission channel for transmitting the to-be-transmitted data between the first node and the second node is a direct transmission channel between the address space of the second-type processor of the first node and the address space of the first-type processor of the second node.

[0062] It should be noted that the data transmission process (or communication process) involves two types of buffers, a user buffer (i.e., a "first level buffer") and a staging buffer (i.e., a "second level buffer"). The user buffer represents a buffer in which a user actually sends data or actually stores data, and is visible to the user. The staging buffer represents a buffer registered to a corresponding node on a network card, and is invisible to the user. The nodes in the same communication group perform inter-node data transmission through the staging buffers of the nodes, for example, a sending node copies data in a user buffer to a staging buffer of the sending node, then transmits the data through the staging buffer of the sending node to a staging buffer of a receiving node, and finally, the receiving node copies the data received by the staging buffer to a user buffer of the receiving node. In addition, the user buffer of a node can also serve as the staging buffer of the node, so that additional copying of data between the user buffer and the staging buffer can be avoided.

[0063] In addition, in order to realize the data copying between the user buffer and the staging buffer of the same node, the staging buffer of the node can be determined according to the user buffer of the node and the data transmission capability of the node.

[0064] For step S210, for example, in the write mode, the first node initiates a write operation to write data (i.e., "to-be-transmitted data") to the second node, the first node can obtain a first pointer (e.g., send_user_ptr, representing a storage address of the to-be-transmitted data in the first node) of the to-be-transmitted data and a size (e.g., size_t size) of the to-be-transmitted data according to a communication interface (e.g., send(void* send_user_ptr, size_t size)) of the first node. Then, the first node obtains a second pointer (i.e., an actual receiving address of the to-be-transmitted data received by the second node from the first node) of the to-be-transmitted data by negotiating with the second node (e.g., establishing a temporary Transmission Control Protocol (TCP) link). The first pointer points to an address of a first-level buffer (i.e., "user buffer") in the first node for storing the to-be-transmitted data, and the second pointer points to an address of a second-level buffer (i.e., "temporary storage buffer") in the second node for receiving the to-be-transmitted data. The second-level buffer pointed to by the second pointer is determined by the second node according to an actual write address (i.e., an address of a first-level buffer (i.e., "user buffer") in the second node for the to-be-transmitted data) of the to-be-transmitted data and a data transmission capability of the second node, and the address of the temporary storage buffer is sent to the first node through, for example, the TCP link. The specific manner in which the second node determines the second pointer can refer to the manner in which the first node determines the second-level buffer for the to-be-transmitted data, which will not be described here.

[0065] For example, the first-level buffer (i.e., "user buffer") in the second node for the to-be-transmitted data can be determined by the second node. For example, in response to the write operation initiated by the first node, the second node triggers a receiving operation and obtains a write address (receive_user_ptr represents an actual write address of the to-be-transmitted data in the second node, i.e., an address of a first-level buffer for the to-be-transmitted data) of the to-be-transmitted data through a communication interface (e.g., receive(void* receive_user_ptr, size_t size)) of the second node, and the first-level buffer for the to-be-transmitted data in the second node can be determined through the address. In addition, send_user_ptr and receive_user_ptr can be specified by a user.

[0066] For example, each node in the communication group includes at least two types of processors, a first type of processor and a second type of processor. In some examples, the first type of processor can be a central processing unit (CPU) and the second type of processor can be a graphic processing unit (GPU). For example, the first type of processor and the second type of processor can also be other types of processors, such as a general-purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), a neural network processing unit (NPU), a deep learning processing unit (DPU), an accelerated processing unit (APU), a field programmable gate array (FPGA), and a multi-core processor with artificial intelligence computing capability, and the present disclosure does not limit this.

[0067] For example, each type of processor has its own dedicated address space, which corresponds to a storage medium adapted to the characteristics of the processor. For example, the address space of the CPU is usually mapped to dynamic random access memory (DRAM), and the address space of the GPU is usually mapped to video random access memory (VRAM), and the present disclosure does not limit this.

[0068] In at least one embodiment of the present disclosure, the data transmission capability indication allows inter-node data transmission based on the address space of which type of processor in the corresponding node, or does not allow inter-node data transmission based on the address space of which type of processor in the corresponding node.

[0069] For example, in the case that the first node comprises a CPU and a GPU, and the data transmission capability of the first node indicates that inter-node data transmission based on the address space of the CPU is not allowed, and inter-node data transmission based on the address space of the GPU is allowed, if the first pointer of the data to be transmitted points to the first-level buffer belonging to the address space of the GPU, the data to be transmitted can be directly transmitted from the address space of the GPU of the first node to the second node, without being transmitted from the address space of the GPU of the first node to the second node via the address space of the CPU of the first node; if the first pointer of the data to be transmitted points to the first-level buffer belonging to the address space of the CPU, the data to be transmitted needs to be transmitted from the address space of the CPU of the first node to the second node via the address space of the GPU of the first node.

[0070] In at least one embodiment of the present disclosure, the data transmission capability of the first node can be determined according to the communication routing table. In addition, the data transmission capability of the second node can also be determined by the first node through the communication routing table.

[0071] Figure 3 A flowchart for creating a communication routing table provided by at least one embodiment of the present disclosure is shown.

[0072] As shown in Figure 3 , the steps of creating a communication routing table include steps S310-S330.

[0073] Step S310: detecting the second-level buffer of the local address space in the first node that allows inter-node data transmission, and registering the address space information of the second-level buffer in the local address space for the first node.

[0074] Step S320: obtaining the address space information of the second-level buffer in the remote address space that allows inter-node data transmission registered for the second node.

[0075] Step S330: creating a communication routing table according to the address space information registered for the first node and the second node.

[0076] For step S310, after creating a communication group and performing a communication group establishment connection operation, for the first node in the communication group, the local address space in the first node that allows inter-node data transmission (i.e., the second-level buffer (temporary buffer) in the first node that allows inter-node data transmission) is detected, and the address space information of the second-level buffer in the local address space is registered for the first node.

[0077] For example, the address space information at least includes an address space type of the second level buffer, the address space type includes a first type and a second type, the second level buffer of the first type belongs to an address space of a processor of the first type in the corresponding node, and the second level buffer of the second type belongs to an address space of a processor of the second type in the corresponding node.

[0078] For example, the address space information can further include a node identifier corresponding to each registered second level buffer, a start address of each registered second level buffer, and a buffer capacity, wherein the node identifier indicates that the corresponding second level buffer belongs to a "local address space" (i.e., a second level buffer of the current node) or a "remote address space" (i.e., a second level buffer of another node other than the current node). The address space information can further include other information (e.g., a device number of the corresponding node, a remote key (rkey) for access permission verification), which is not limited by the present disclosure.

[0079] It should be noted that the "address space type of the second level buffer" in the address space information indicates to which type of processor the second level buffer belongs.

[0080] For step S320, a temporary communication link (e.g., a TCP link) can be established for the first node and other nodes (e.g., a second node) in the same communication group, and the first node exchanges information with the second node through the temporary communication link to obtain address space information of the second level buffer in the remote address space registered by the second node and allowed to perform inter-node data transmission. While the information exchange operation is being performed, the second node can also obtain address space information of the second level buffer in the local address space registered by the first node and allowed to perform inter-node data transmission.

[0081] It should be noted that the "local address space" and the "remote address space" are relative. For example, for the first node, the "local address space" indicates the address space of the first node, and the "remote address space" indicates the address space of other nodes (e.g., the second node) in the same communication group. For example, for the second node, the "local address space" indicates the address space of the second node, and the "remote address space" indicates the address space of other nodes (e.g., the first node) in the same communication group.

[0082] For step S330, a corresponding communication routing table is created for the first node according to the address space information registered for the first node and the second node.

[0083] For example, the step of determining the data transfer capability of the first node according to the communication routing table can comprise: in response to the address space type of the second level buffer in the local address space comprising the first type and / or the second type, determining that the data transfer capability of the first node comprises allowing the inter-node data transfer based on the address space of the corresponding type of processor.

[0084] For example, Table 1 shows an example of the communication routing table created for the first node.

[0085] Table 1

[0086]

[0087] As shown in Table 1, the node identification “local” indicates that the table entry 1 and the table entry 2 are used to store the address space information of the second level buffer in the local address space of the first node which allows the inter-node data transfer, and the node identification “remote” indicates that the table entry 3 and the table entry 4 are used to store the address space information of the second level buffer in the remote address space of the second node which allows the inter-node data transfer. For example, the address space type “DRAM” indicates that the corresponding second level buffer belongs to the address space of the CPU in the corresponding node, and the address space type “VRAM” indicates that the corresponding second level buffer belongs to the address space of the GPU in the corresponding node.

[0088] For example, according to the table entry 1 and the table entry 2 in the communication routing table, the address space type of the second level buffer in the local address space comprises DRAM and VRAM, it can be determined that the data transfer capability of the first node comprises allowing the inter-node data transfer based on the second level buffer in the address space of the CPU in the first node (i.e., the second level buffer in DRAM) and the second level buffer in the address space corresponding to the GPU (i.e., the second level buffer in VRAM), i.e., the first node supports Remote Direct Memory Access (RDMA) and GPU Direct RDMA (GDR). In the first node, the starting address of the second level buffer in DRAM is 0x10000, and the buffer capacity of the second level buffer is 64MB. In the first node, the starting address of the second level buffer in VRAM is 0x20000, and the buffer capacity of the second level buffer is 128MB.

[0089] For example, if the communication routing table shown in Table 1 does not include the above-described table item 2, it is determined that the data transmission capability of the first node includes allowing inter-node data transmission based on the second level buffer in the address space of the CPU (i.e., the second level buffer in the DRAM) in the first node, but does not allow inter-node data transmission based on the second level buffer in the address space of the GPU (i.e., the second level buffer in the VRAM), i.e., the first node supports RDMA but does not support GDR.

[0090] For example, according to the table item 3 and the table item 4 in the communication routing table, the address space type of the second level buffer in the remote address space includes DRAM and VRAM, it can be determined that the data transmission capability of the second node includes allowing inter-node data transmission based on the second level buffer in the address space of the CPU (i.e., the second level buffer in the DRAM) and the second level buffer in the address space of the GPU (i.e., the second level buffer in the VRAM) in the second node, i.e., the second node supports Remote Direct Memory Access (RDMA) and GPU Direct RDMA (GDR). In the second node, the starting address of the second level buffer in the DRAM is 0x30000, and the buffer capacity of the second level buffer is 64MB. The starting address of the second level buffer in the VRAM in the second node is 0x40000, and the buffer capacity of the second level buffer is 128MB.

[0091] For example, the communication routing table shown in Table 1 can also include a table item 5, and the address space information in the table item 5 includes: node identification “remote”, address space type “VRAM”, starting address “0x50000”, and capacity “128MB”. In this example, according to the table item 4 and the table item 5 in the communication routing table, it is determined that the address space (VRAM) of the GPU in the second node includes two second level buffers, and the second node can perform inter-node data transmission with other nodes (e.g., the first node) through any one of the two second level buffers. The present disclosure does not limit the number of second level buffers in the address space of each type of processor.

[0092] It should be noted that the second node can also create a communication routing table in a manner similar to steps S310-S330. For example, the second node can perform the following operations: detecting the second level buffer of the local address space in the second node that allows inter-node data transmission, and registering the address space information of the second level buffer in the local address space for the second node; obtaining the address space information of the second level buffer in the remote address space that allows inter-node data transmission registered for the first node; and creating a communication routing table according to the address space information registered for the first node and the second node.

[0093] For the step S220, after obtaining the first pointer of the to-be-transmitted data, a second level buffer (i.e., a "temporary buffer") for the to-be-transmitted data in the first node can be determined according to the first buffer pointed by the first pointer and the data transmission capability of the first node, so as to copy the to-be-transmitted data from the first buffer in the first node to the determined second level buffer, and then the first node performs the inter-node data transmission operation with other nodes (e.g., the second node) in the same communication group through the second level buffer.

[0094] For example, in response to the first level buffer pointed by the first pointer belonging to the address space (e.g., VRAM) of the second type processor (e.g., GPU) and the address space of the processor allowed by the data transmission capability of the first node including the address space of the second type processor, the second level buffer for the to-be-transmitted data in the first node is determined as the second level buffer in the address space of the second type processor.

[0095] For example, in response to the first level buffer pointed by the first pointer belonging to the address space (e.g., VRAM) of the second type processor (e.g., GPU) and the address space of the processor allowed by the data transmission capability of the first node not including the address space of the second type processor, the second level buffer for the to-be-transmitted data in the first node is determined as the second level buffer in the address space (e.g., DRAM) of the first type processor (e.g., CPU).

[0096] For example, in response to the first level buffer pointed by the first pointer belonging to the address space (e.g., DRAM) of the first type processor (e.g., CPU) and the address space of the processor allowed by the data transmission capability of the first node including the address space of the first type processor, the second level buffer for the to-be-transmitted data in the first node is determined as the second level buffer in the address space of the first type processor.

[0097] For example, in response to the first level buffer pointed by the first pointer belonging to the address space (e.g., DRAM) of the first type processor (e.g., CPU) and the address space of the processor allowed by the data transmission capability of the first node not including the address space of the first type processor, the second level buffer for the to-be-transmitted data in the first node is determined as the second level buffer in the address space (e.g., VRAM) of the second type processor (e.g., GPU).

[0098] Figure 4 A schematic diagram of a data transmission process provided by at least one embodiment of the present disclosure is shown.

[0099] As Figure 4As shown, the first node Rank0 and the second node Rank1 each include a CPU and a GPU, and DRAM_0 and VRAM_0 represent the address space of the CPU and the address space of the GPU of the first node Rank0 respectively, and DRAM_1 and VRAM_1 represent the address space of the CPU and the address space of the GPU of the second node Rank1 respectively.

[0100] The following describes the step S230 in combination with Figure 4 The step S230 is described as follows.

[0101] For the step S230, as Figure 4 shown, in the write mode, the first level buffer (i.e., the user buffer user_buffer_v0 where the to-be-transmitted data Data is located) pointed by the first pointer belongs to the address space (VRAM_0) of the second type processor (GPU), and since the data transmission capability of the first node Rank0 includes allowing the inter-node data transmission based on the address space of the second type processor, it is determined that the second level buffer for the to-be-transmitted data Data in the first node Rank0 is the staging buffer stg_buffer_v0 in the address space VRAM_0. Since the first level buffer (i.e., the user buffer user_buffer_d1) for writing the to-be-transmitted data Data in the second node Rank1 belongs to the address space (DRAM_1) of the first type processor (CPU), and the data transmission capability of the second node Rank1 includes allowing the inter-node data transmission based on the address space of the first type processor, it is determined that the second level buffer for the to-be-transmitted data Data in the second node Rank1 is the staging buffer stg_buffer_d1 in the address space DRAM_1.

[0102] In Figure 4In the example shown, in response to the first-level buffer (i.e., the user buffer user_buffer_v0) and the second-level buffer (i.e., the staging buffer stg_buffer_v0) in the first node Rank0 both belonging to the address space (VRAM_0) of the second-type processor (GPU), and the first-level buffer (i.e., the user buffer user_buffer_d1) and the second-level buffer (i.e., the staging buffer stg_buffer_d1) in the second node Rank1 both belonging to the address space (DRAM_1) of the first-type processor (CPU), it is determined that the target transmission channel between the first node Rank0 and the second node Rank1 for transmitting the to-be-transmitted data Data is a direct transmission channel between the address space (VRAM_0) of the second-type processor (GPU) of the first node Rank0 and the address space (DRAM_1) of the first-type processor (CPU) of the second node Rank1, that is, the to-be-transmitted data Data is transmitted from the user buffer user_buffer_v0 of the first node Rank0 to the user buffer user_buffer_d1 of the second node Rank1 via the staging buffer stg_buffer_v0 of the first node Rank0, the staging buffer stg_buffer_d1 of the second node Rank1.

[0103] Figure 5 A flowchart of another communication control method provided by at least one embodiment of the present disclosure is shown.

[0104] For example, as Figure 5 shown, the communication control method provided by at least one embodiment of the present disclosure can include steps S510-S540. Step S510 can refer to step S210 described above, and will not be described here.

[0105] In the communication control method provided by at least one embodiment of the present disclosure, after obtaining the first pointer and the second pointer of the to-be-transmitted data, steps S520, S530 and S540 can be sequentially executed.

[0106] Step S520: Identify the type of the address space in which the first-level buffer pointed to by the first pointer is located.

[0107] Step S530: Determine whether the type of the address space in which the first-level buffer pointed to by the first pointer is located matches the data transmission capability of the first node.

[0108] Step S540: According to the first pointer, the second pointer and the data transmission capability of the first node, determine the target transmission channel between the first node and the second node.

[0109] For example, the implementation of the step S520 can include: identifying whether the address space where the first-level buffer pointed by the first pointer is located belongs to the address space of the first type processor or the address space of the second type processor.

[0110] In addition, after identifying whether the first-level buffer (i.e., the user buffer) pointed by the first pointer belongs to the address space of the first type processor or the address space of the second type processor, the second-level buffer (i.e., the staging buffer) for the data to be transmitted in the first node can be further determined according to the data transmission capability of the first node and the first-level buffer pointed by the first pointer.

[0111] For the step S530, the data transmission capability of the first node can be determined according to the communication routing table, and the specific steps can refer to the corresponding embodiment of Table 1 described above, which will not be repeated here.

[0112] For example, the step S530 can include: in response to the type of the address space of the processor allowed by the data transmission capability of the first node including the type of the address space where the first-level buffer pointed by the first pointer is located, determining that the address space where the first-level buffer pointed by the first pointer is located matches the data transmission capability of the first node.

[0113] For example, the step S540 can include the step S540A and the step S540B.

[0114] The step S540A: determining a plurality of transmission channels according to the first pointer and the second pointer.

[0115] The step S540B: determining a target transmission channel from the plurality of transmission channels according to the data transmission capability of the first node.

[0116] For example, the step S540B can include: determining a candidate transmission channel from the plurality of transmission channels that meets the data transmission capability of the first node; and selecting a channel that at least one of satisfies the highest affinity or requires the least number of data copies as the target transmission channel from the candidate transmission channel.

[0117] For example, the channel that satisfies the highest affinity can be selected as the target transmission channel from the candidate transmission channel, or the channel that requires the least number of data copies can be selected as the target transmission channel, or the channel that satisfies the highest affinity and requires the least number of data copies can be selected as the target transmission channel, which can be set according to actual needs.

[0118] For example, the affinity of a transfer channel is used to measure the transfer efficiency, performance and resource overhead of the transfer channel. The affinity can be determined based on one or more of the following factors: Distance / Hops, Bandwidth, Latency, Hardware Feature Support, Resource Dedication & Contention, etc. For Distance / Hops, the fewer physical nodes the data needs to pass through, the higher the affinity. For Bandwidth, the higher the theoretical or actual transfer bandwidth of the transfer channel, the higher the affinity. For Latency, the lower the transfer latency of the data through the channel, the higher the affinity. For Resource Dedication & Contention, the affinity of a transfer channel that is dedicated and not shared with other high-load tasks is higher than that of a shared and congested transfer channel. For Hardware Feature Support, when a transfer channel supports advanced features such as Zero-Copy, Remote Direct Memory Access (RDMA), GPU Direct RDMA (GDR), etc., the affinity of the transfer channel is higher. For example, a network card supporting GDR can directly access the GPU's video memory, and the affinity of this transfer channel is much higher than that of a transfer channel that first copies data from the GPU's video memory to the CPU's memory and then reads it by the network card.

[0119] It should be noted that the "number of data copies" of the present disclosure refers to the copy operation between the address spaces of different types of processors in the same node (for example, the copy operation between the address space of the CPU of the first node and the address space of the GPU of the first node) is counted in the "number of data copies", and the copy operation between different buffers in the address space of the same processor (for example, the user buffer and the staging buffer in the address space of the CPU DRAM) is not counted in the "number of data copies".

[0120] The following will be described in combination with Figure 4 The steps S540A and S540B will be specifically introduced.

[0121] In Figure 4In the shown example, the first pointer points to the first level buffer (user buffer user_buffer_v0) belonging to the address space VRAM_0 of the GPU in the first node Rank0, and the second pointer points to the second level buffer (staging buffer stg_buffer_d1) belonging to the address space DRAM_1 of the CPU in the second node Rank1, then the plurality of transmission channels for the to-be-transmitted data Data includes: channel 1 (from VRAM_0 to DRAM_1), channel 2 (from VRAM_0 to DRAM_1 via DRAM_0), and channel 3 (from VRAM_0 to DRAM_1 via VRAM_1).

[0122] If the first node Rank0 and the second node Rank1 both support RDMA and GDR (i.e., the DRAM_0 and VRAM_0 in the first node Rank0 and the DRAM_1 and VRAM_1 in the second node Rank1 can all perform inter-node data transmission operations), then the channel 1, the channel 2 and the channel 3 all conform to the data transmission capabilities of the first node Rank0 and the second node Rank1, and can all be candidate transmission channels. A channel that at least meets one of the highest affinity or the least required number of data copying is selected as the target transmission channel from the candidate transmission channels (channel 1, channel 2 and channel 3). The to-be-transmitted data Data can be directly transmitted from VRAM_0 to DRAM_1 through the channel 1, since the media types of the user buffer user_buffer_v0 and the staging buffer stg_buffer_v0 for the to-be-transmitted data Data in the first node Rank0 are consistent, and the media types of the user buffer user_buffer_d1 and the staging buffer stg_buffer_d1 for the to-be-transmitted data Data in the second node Rank1 are consistent, and the time consumption of the data copying operations between the user buffer user_buffer_v0 and the staging buffer stg_buffer_v0 and between the user buffer user_buffer_d1 and the staging buffer stg_buffer_d1 is extremely short and can be ignored, therefore, the channel 1 realizes zero-copy and has a relatively short total time consumption of data transmission operations. The to-be-transmitted data Data needs to be transmitted through the channel 2 or the channel 3, which both need 1 extra data copying operation, and the time consumption of the data copying operation between the buffers of different media types is relatively long, therefore, the channel 1 has the highest affinity and the least required number of data copying, and the channel 1 is scheduled as the target transmission channel for the to-be-transmitted data Data to improve the data transmission efficiency.

[0123] For example, the step S540 can further include: in response to the address space where the first-level buffer pointed by the first pointer is not matched with the data transfer capability of the first node, determining the target transfer channel as: sending the data to be transferred from the first-level buffer pointed by the first pointer to the second-level buffer pointed by the second pointer via the second-level buffer corresponding to the first processor in the first node, where the first processor is different from the processor corresponding to the address space where the first-level buffer pointed by the first pointer is located.

[0124] For example, in response to the address space where the first-level buffer pointed by the first pointer is the address space of the first type of processor, the "first processor" in the first node is a processor (for example, the second type of processor) different from the first type of processor.

[0125] Figure 6A and Figure 6B respectively show a schematic diagram of another data transfer process provided by at least one embodiment of the present disclosure.

[0126] As shown in Figure 6A , if the data transfer capability of the first node Rank0 includes allowing the inter-node data transfer based on the address space of the first type of processor (i.e., the DRAM_0 of the CPU) and not allowing the inter-node data transfer based on the address space of the second type of processor (i.e., the VRAM_0 of the GPU), it is determined that the first-level buffer (i.e., the user buffer user_buffer_v0) pointed by the first pointer is not matched with the data transfer capability of the first node Rank0. In this case, the first node Rank0 automatically triggers an implicit copy operation, that is, the data to be transferred Data is copied from the first-level buffer (the user buffer user_buffer_d0) pointed by the first pointer to the second-level buffer (i.e., the staging buffer stg_buffer_d0) corresponding to the processor (i.e., the first type of processor CPU) different from the first processor (i.e., the second type of processor GPU) in the first node Rank0, and then the data to be transferred Data is sent to the second-level buffer (i.e., the staging buffer stg_buffer_d1) pointed by the second pointer via the staging buffer stg_buffer_d0, and finally the data to be transferred is transmitted to the first-level buffer (i.e., the user buffer user_buffer_d1) of the second node Rank1 through the staging buffer stg_buffer_d1.

[0127] As shown in Figure 6BAs shown, if the data transfer capability of the first node Rank0 includes allowing inter-node data transfer based on the address space of the second type of processor (i.e., the VRAM_0 of the GPU), it is determined that the first-level buffer (i.e., the user buffer user_buffer_v0) pointed to by the first pointer matches the data transfer capability of the first node Rank0. If the second-level buffer pointed to by the second pointer belongs to the address space of the second type of processor in the second node Rank1 (i.e., the second-level buffer pointed to by the second pointer is the staging buffer stg_buffer_v1), the first node Rank0 copies the to-be-transferred data Data from the first-level buffer (i.e., the user buffer user_buffer_v0) pointed to by the first pointer to the staging buffer stg_buffer_v0, the first node Rank0 transmits the to-be-transferred data Data from the staging buffer stg_buffer_v0 to the second-level buffer (i.e., the staging buffer stg_buffer_v1) pointed to by the second pointer, and finally, the second node Rank1 transmits the received to-be-transferred data Data from the staging buffer stg_buffer_v1 to the first-level buffer (i.e., the user buffer user_buffer_d1) for the to-be-transferred data Data in the second node.

[0128] For example, after completing the transmission operation on the to-be-transferred data, the first node returns a sending handle (Handle) to monitor and manage the status of the sent data, and the second node returns a receiving handle to report the receiving status of the data to the first node (for example, confirming that the data has been successfully received, confirming that the data has been partially received, or confirming that an error occurs in data receiving). The "handle" here is an abstract resource identifier, which allows both parties (the first node and the second node) to further operate or query the data transmission or associated resources.

[0129] The communication control method provided by at least one embodiment of the present disclosure can be integrated into an existing training inference framework, and a unified hybrid communication package is provided for a deep learning framework to effectively support direct communication between CPUs and GPUs.

[0130] It should be further noted that in various embodiments of the present disclosure, the execution order of each step of the communication control method is not limited, and although the execution process of each step is described in a specific order above, this does not constitute a limitation on the embodiments of the present disclosure. Each step in the communication control method can be executed in series or in parallel, which can be determined according to actual needs.

[0131] For example, compared with the above description, the communication control method provided by at least one embodiment of the present disclosure can further include more or fewer steps, and the embodiments of the present disclosure do not limit this.

[0132] The communication system corresponding to the communication control method is also provided in at least one embodiment of the present disclosure. Since the principle of solving problems of the communication system in the embodiments of the present disclosure is similar to the communication control method described above, in the following description, the implementation of the communication system can also be referred to the implementation of the method, and the repeated parts will not be described herein.

[0133] Figure 7A A schematic diagram of the communication system provided by at least one embodiment of the present disclosure is shown.

[0134] As Figure 7A shown, the communication system 700 can include multiple nodes (for example, node Rank0, node Rank1), wherein each node includes a communication interface, a communication controller, and at least two types of processors including a first type of processor and a second type of processor, for example, the node Rank0 (i.e., the first node) includes a communication interface 701, a communication controller 702, a first type of processor 703 (for example, CPU) and a second type of processor 704 (for example, GPU). For example, the node Rank1 (i.e., the second node) includes a communication interface 711, a communication controller 712, a first type of processor 713 (for example, CPU) and a second type of processor 714 (for example, GPU). It should be noted that, Figure 7A For example only, the communication system 700 can also include more nodes in addition to the node Rank0, the node Rank1, for example, the node Rank2, and the embodiments of the present disclosure do not limit this. The structure of other nodes is similar to that of the node Rank0, which will not be described herein.

[0135] For example, in response to the node Rank0 triggering a data transmission operation between the node Rank0 and the node Rank1 (i.e., the second node), the communication interface 701 of the node Rank0 is configured to obtain a first pointer and a second pointer of the to-be-transmitted data Data, and determine a second level buffer for the to-be-transmitted data Data in the node Rank0 according to the first pointer and the data transmission capability of the node Rank0, wherein the first pointer points to a first level buffer for the to-be-transmitted data Data in the node Rank0, and the second pointer points to a second level buffer for the to-be-transmitted data Data in the node Rank1. The second pointer is encapsulated in the communication interface 701 of the node Rank0 by the node Rank1 through the established temporary communication link (for example, TCP link) and sent to the identification and negotiation module (not shown in Figure 7A ).

[0136] For example, the communication controller 702 of the node Rank0 is configured to determine that the target transmission channel between the node Rank0 and the node Rank1 for transmitting the to-be-transmitted data Data is a direct transmission channel between the address space of the second-type processor 704 of the node Rank0 and the address space of the first-type processor 713 of the node Rank1, in response to the first-level buffer and the second-level buffer in the node Rank0 belonging to the address space of the second-type processor 704 (for example, the VRAM of the GPU), and the first-level buffer and the second-level buffer in the node Rank1 belonging to the address space of the first-type processor 713 (for example, the DRAM of the CPU).

[0137] For example, the communication interface of the node Rank0 as the sending end can be represented as send(void* send_user_ptr, size_t size), where send_user_ptr represents the storage address of the to-be-transmitted data in the node Rank0; and the communication interface of the node Rank1 as the receiving end can be represented as receive(void* receive_user_ptr, size_t size), where receive_user_ptr represents the storage (or write-in) address of the to-be-transmitted data in the node Rank1 after the node Rank1 receives the to-be-transmitted data.

[0138] In the embodiments of the present disclosure, by providing a unified communication interface for each node, the difference in the underlying memory type can be shielded, that is, the application program only needs to send or receive data through a simple function call, without needing to care about whether the data is located in the CPU memory (for example, DRAM), the GPU memory (for example, VRAM), or other types of memory. The communication system automatically identifies the memory type at the bottom layer, and selects the optimal transmission path (such as direct RDMA, GDR, or via a temporary buffer), to ensure efficient transmission of data.

[0139] In addition, the node Rank0 can also simultaneously communicate with multiple nodes such as the node Rank1 and the node Rank2, and the specific operation is similar to that of the communication (or data transmission) between the node Rank0 and the node Rank1, which will not be described herein again.

[0140] For example, the data transmission capability indication allows or does not allow inter-node data transmission based on the address space of which type of processor in the corresponding node.

[0141] For example, after the communication interface 701 obtains the first pointer and the second pointer of the data Data to be transmitted, the identification and negotiation module encapsulated in the communication interface 701 is configured to identify the type of the address space where the first level buffer pointed to by the first pointer is located; and determine whether the type of the address space where the first level buffer pointed to by the first pointer is located matches the data transmission capability of the node Rank0. For example, the identification and negotiation module is further configured to determine that the address space where the first level buffer pointed to by the first pointer is located matches the data transmission capability of the first node, in response to the type of the address space of the processor allowed by the data transmission capability of the node Rank0 including the type of the address space where the first level buffer pointed to by the first pointer is located.

[0142] For example, the identification and negotiation module is further configured to determine that the second level buffer for the data Data to be transmitted in the node Rank0 is the second level buffer in the address space of the second type processor 704, in response to the first level buffer pointed to by the first pointer belonging to the address space of the second type processor 704 and the address space of the processor allowed by the data transmission capability of the node Rank0 including the address space of the second type processor 704; or determine that the second level buffer for the data Data to be transmitted in the node Rank0 is the second level buffer in the address space of the first type processor 703, in response to the first level buffer pointed to by the first pointer belonging to the address space of the second type processor 704 and the address space of the processor allowed by the data transmission capability of the node Rank0 not including the address space of the second type processor 704.

[0143] For example, the communication controller 702 of the node Rank0 is further configured to determine the target transmission channel between the node Rank0 and the node Rank1 according to the first pointer, the second pointer and the data transmission capability of the node Rank0.

[0144] For example, in the process of determining the target transmission channel between the node Rank0 and the node Rank1 according to the first pointer, the second pointer and the data transmission capability of the node Rank0, the communication controller 702 is specifically configured to determine a plurality of transmission channels according to the first pointer and the second pointer; and determine the target transmission channel from the plurality of transmission channels according to the data transmission capability of the node Rank0.

[0145] For example, in the process of determining the target transmission channel from the plurality of transmission channels according to the data transmission capability of the node Rank0, the communication controller 702 is specifically configured to determine a candidate transmission channel from the plurality of transmission channels that meets the data transmission capability of the node Rank0; and select a channel that at least one of has the highest affinity or requires the least number of data copying times as the target transmission channel from the candidate transmission channel.

[0146] For example, in determining the target transmission channel between node Rank0 and node Rank1 according to the first pointer, the second pointer and the data transmission capability of node Rank0, the communication controller 702 is specifically configured to determine the target transmission channel as transmitting the to-be-transmitted data Data from the first-level buffer pointed by the first pointer to the second-level buffer pointed by the pointer Rank1 via the second-level buffer corresponding to a first processor in node Rank0, in response to the address space where the first-level buffer pointed by the first pointer is located not matching the data transmission capability of node Rank0, wherein the first processor is different from the processor corresponding to the address space where the first-level buffer pointed by the pointer Rank0 is located.

[0147] For example, the data transmission capability of node Rank0 is determined according to the communication routing table. Each node in the communication system 700 further comprises a routing table creation module. The routing table creation module of node Rank0 is configured to detect the local address space in node Rank0 that allows inter-node data transmission, and register the address space information of the second-level buffer in the local address space for node Rank0; obtain the address space information of the second-level buffer in the remote address space that allows inter-node data transmission registered for node Rank1; and create the communication routing table according to the address space information registered for node Rank0 and node Rank1.

[0148] For example, the address space information in the communication routing table at least comprises the address space type of the second-level buffer, and the address space type comprises a first type and a second type, the second-level buffer of the first type belongs to the address space of the first type processor in the corresponding node, and the second-level buffer of the second type belongs to the address space of the second type processor in the corresponding node.

[0149] For example, in determining the data transmission capability of node Rank0 according to the communication routing table, the identification and negotiation module encapsulated in the communication interface 701 is configured to determine that the data transmission capability of node Rank0 comprises allowing inter-node data transmission based on the address space of the corresponding type processor, in response to the address space type of the second-level buffer in the local address space comprising the first type and / or the second type.

[0150] For example, the address space information in the communication routing table further comprises the node identifier, the start address and the buffer capacity corresponding to each registered second-level buffer.

[0151] Figure 7B A communication flowchart between nodes provided by at least one embodiment of the present disclosure is shown.

[0152] As Figure 7BAs shown, the node Rank0 and the node Rank1 perform the communication group creation operation, the communication group setup operation and the address space information registration operation, respectively. For example, when performing the address space information registration operation, the node Rank0 detects the second level buffer of the local address space in the node Rank0 which is allowed to perform the inter-node data transmission, and registers the address space information of the detected second level buffer on the network card (for example, refer to the above step S310). When performing the address space information registration operation, the node Rank1 detects the second level buffer of the local address space in the node Rank1 which is allowed to perform the inter-node data transmission, and registers the address space information of the detected second level buffer on the network card (for example, refer to the above step S310). After the address space information registration operation is completed, the node Rank0 and the node Rank1 perform the address space information synchronization operation. For example, the node Rank0 and the node Rank1 can exchange information by establishing a temporary communication link, acquire the address space information of the second level buffer registered by the other party, and then create their own communication routing table (for example, refer to the above step S330).

[0153] As Figure 7BAs shown, when the communication mode is a send / recv mode, both the node Rank0 (as a sender) and the node Rank1 (as a receiver) trigger the data transmission operation, i.e., the node Rank0 initiates the data sending operation, and the node Rank1 initiates the data receiving operation. In the address space identification and negotiation phase, the node Rank0 determines a "first pointer" of the data to be transmitted (i.e., an address send_user_ptr of a user buffer in the node Rank0 for the data to be transmitted), and determines an address of a "second-level buffer" (i.e., an address send_stg_ptr of a staging buffer) in the node Rank0 for the data to be transmitted according to the "first pointer" and address space information of the second-level buffer registered locally by the node Rank0; the node Rank1 determines an address of a "first-level buffer" of the data to be transmitted (i.e., an address receive_user_ptr of a user buffer in the node Rank1 for the data to be transmitted), and determines a "second pointer" in the node Rank1 for the data to be transmitted (i.e., an address receive_stg_ptr of a "second-level buffer" (staging buffer) in the node Rank0 for the data to be transmitted) according to the user buffer receive_user_ptr and address space information of the second-level buffer registered locally by the node Rank1. Then, the node Rank0 and the node Rank1 exchange the address information of the "second-level buffer" determined by both sides for the inter-node data transmission through the established temporary communication link, for example, the node Rank0 sends the send_stg_ptr to the node Rank1, and the node Rank1 sends the receive_stg_ptr to the node Rank0.

[0154] After the address space identification and negotiation phase is completed, the node Rank0 determines the address space type registered locally by the node Rank0 and the address space type registered remotely by the node Rank1 from the communication routing table, and selects a target transmission channel for the data transmission operation according to the send_stg_ptr and the receive_stg_ptr. The specific selection manner can refer to the above steps S540, S540A, S540B, Figure 6A and Figure 6B Corresponding examples are not described herein again. After the data transmission operation is completed, the node Rank0 returns a sending handle, and the node Rank1 returns a receiving handle.

[0155] Figure 8 A structural schematic diagram of an electronic device provided by at least one embodiment of the present disclosure is shown. Figure 8 The electronic device shown is merely an example, and should not bring any limitation to the functions and use range of the embodiments of the present disclosure.

[0156] The electronic devices in at least one embodiment of this disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (PADs), portable multimedia players (PMPs), in-vehicle terminals (e.g., in-vehicle navigation terminals), wearable electronic devices, and fixed terminals such as digital TVs and desktop computers. Figure 8 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0157] The electronic device includes at least one processor and a memory. The processor may be referred to as processing device 801 as described below, and the memory may include at least one of read-only memory (ROM), random access memory (RAM), and storage device 808 as described below. The memory is used to store programs for performing the methods described in the various method embodiments above; the processor is configured to execute the programs stored in the memory. The processor may include a central processing unit (CPU) or other forms of processing unit having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0158] For example, such as Figure 8 As shown, in some examples, electronic device 800 includes a processing unit 801 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) or loaded from storage device 808 into random access memory (RAM). Various programs and data required for the operation of the computer system are also stored in RAM 803. Processing unit 801, ROM 802, and RAM 803 are connected via bus 804. Input / output (I / O) interfaces are also connected to bus 804.

[0159] For example, the following components can be connected to I / O interface 805: input devices 806 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 807 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 808 including, for example, magnetic tapes, hard disks, etc.; and communication devices 809 including, for example, network interface cards such as LAN cards, modems, etc. Communication device 809 allows electronic device 800 to communicate wirelessly or wiredly with other devices to exchange data and perform communication processing via networks such as the Internet. Although Figure 8The electronic device 800 is shown comprising various means, but it is to be understood that not all of the means illustrated are required to practice or implement the various embodiments. Greater or fewer means can alternatively be employed.

[0160] For example, the electronic device 800 can further include a peripheral interface (not shown in the figure) and the like. The peripheral interface can be various types of interfaces, such as a USB interface, a lighting interface, and the like. The communication device 809 can communicate with a network and other devices through wireless communication.

[0161] For example, according to at least one embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, at least one embodiment of the present disclosure includes a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through the communication device 809, or installed from the storage device 808, or installed from the ROM 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the methods of at least one embodiment of the present disclosure are performed.

[0162] It should be noted that the computer readable medium in the above disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In at least one embodiment of the disclosure, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In at least one embodiment of the disclosure, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or apparatus. The program code contained in the computer readable medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, a radio frequency (RF), etc., or any suitable combination thereof.

[0163] The computer readable medium described above can be contained in the electronic device 800 described above; or can exist separately and not be assembled into the electronic device 800.

[0164] Embodiments of the disclosure also provide a non-transitory computer readable storage medium having computer readable instructions stored thereon, which, when executed by a processor, cause the processor to perform the steps of the communication control method according to the disclosure.

[0165] Figure 9 A structural schematic diagram of a non-transitory computer readable storage medium provided by at least one embodiment of the disclosure is shown.

[0166] For example, as shown in Figure 9 The non-transitory computer readable storage medium 900 stores computer readable instructions 901, which, when executed by at least one processor, perform one or more steps of the communication control method described above.

[0167] For example, the storage medium can include a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM), a portable compact disc read only memory (CD-ROM), a flash memory, or any combination of the above storage mediums, or other applicable storage medium.

[0168] Embodiments of the present disclosure provide a communication control method, a communication system, an electronic device, and a storage medium. With the communication control method, the communication system, the electronic device, and the storage medium provided by the present disclosure, by identifying the pointer types of the first pointer and the second pointer of the to-be-transmitted data and the data transmission capabilities of the first node and the second node, determining the target transmission channel of the to-be-transmitted data, the data direct transmission operation between different types of processors between heterogeneous nodes (for example, the data direct transmission operation between the first type processor of the first node and the second type processor of the second node) can be realized, an efficient communication path (data transmission channel) can be automatically scheduled, and additional data copy operations can be avoided, thereby improving communication efficiency; it can support member nodes using CPU address space and GPU address space in the same communication group, reduce the complexity of cross-memory type communication, and simplify the complexity of user programming.

[0169] Although the present disclosure has been described in detail with general description and specific embodiments above, some modifications or improvements can be made to the present disclosure on the basis of the embodiments of the present disclosure, which is obvious to those skilled in the art. Therefore, these modifications or improvements made on the basis of not deviating from the spirit of the present disclosure, all belong to the scope of protection of the present disclosure.

[0170] In addition to the above exemplary description, the following points need to be explained:

[0171] (1) The drawings of the embodiments of the present disclosure only involve the structures involved in the embodiments of the present disclosure, and other structures can be referred to the general design.

[0172] (2) In the case of no conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0173] The above description is only exemplary embodiments of the present disclosure, and is not intended to limit the protection scope of the present disclosure, and the protection scope of the present disclosure is determined by the appended claims.

Claims

1. A communication control method characterized by comprising: The communication control method is used for a first node in a communication group, and comprises: obtaining a first pointer and a second pointer of to-be-transmitted data, wherein the first pointer points to a first-level buffer for the to-be-transmitted data in the first node, the communication group further comprises a second node, and the second pointer points to a second-level buffer for the to-be-transmitted data in the second node, each node in the communication group comprises a first-type processor and a second-type processor; determining a second-level buffer for the to-be-transmitted data in the first node according to the first pointer and a data transmission capability of the first node; in response to the first-level buffer and the second-level buffer in the first node both belonging to an address space of the second-type processor, and the first-level buffer and the second-level buffer in the second node both belonging to an address space of the first-type processor, determining that a target transmission channel for transmitting the to-be-transmitted data between the first node and the second node is a direct transmission channel between the address space of the second-type processor of the first node and the address space of the first-type processor of the second node.

2. The communication control method according to claim 1, characterized by, The data transmission capability indicates whether data transmission between nodes is allowed based on an address space of which type of processor in a corresponding node or is not allowed based on the address space of which type of processor in the corresponding node.

3. The communication control method according to claim 1 or 2, characterized by, After obtaining the first pointer and the second pointer of the to-be-transmitted data, the communication control method further comprises: identifying a type of an address space in which the first-level buffer pointed to by the first pointer is located; determining whether the type of the address space in which the first-level buffer pointed to by the first pointer is located matches a data transmission capability of the first node.

4. The communication control method according to claim 3, characterized by, The determination of whether the address space in which the first-level buffer pointed to by the first pointer is located matches the data transmission capability of the first node comprises: in response to a type of an address space of a processor allowed by the data transmission capability of the first node including the type of the address space in which the first-level buffer pointed to by the first pointer is located, determining that the address space in which the first-level buffer pointed to by the first pointer is located matches the data transmission capability of the first node.

5. The communication control method according to claim 1 or 2, characterized by, The determination of the second-level buffer for the to-be-transmitted data in the first node according to the first pointer and the data transmission capability of the first node comprises: in response to the first-level buffer pointed to by the first pointer belonging to the address space of the second-type processor and the address space of the processor allowed by the data transmission capability of the first node including the address space of the second-type processor, determining that the second-level buffer for the to-be-transmitted data in the first node is a second-level buffer in the address space of the second-type processor; or in response to the first-level buffer pointed to by the first pointer belonging to the address space of the first-type processor and the address space of the processor allowed by the data transmission capability of the first node including the address space of the first-type processor, determining that the second-level buffer for the to-be-transmitted data in the first node is a second-level buffer in the address space of the first-type processor. In response to the first-level buffer pointed by the first pointer belonging to the address space of the second type processor and the data transmission capability of the first node allowing the address space of the processor not including the address space of the second type processor, the second-level buffer for the to-be-transmitted data in the first node is determined as the second-level buffer in the address space of the first type processor.

6. The communication control method according to claim 1 or 2, characterized by, The communication control method further comprises: determining a target transmission channel between the first node and the second node according to the first pointer, the second pointer and the data transmission capability of the first node.

7. The communication control method according to claim 6, characterized by, The determining a target transmission channel between the first node and the second node according to the first pointer, the second pointer and the data transmission capability of the first node comprises: determining a plurality of transmission channels according to the first pointer and the second pointer; and determining the target transmission channel from the plurality of transmission channels according to the data transmission capability of the first node.

8. The communication control method according to claim 7, characterized by, The determining the target transmission channel from the plurality of transmission channels according to the data transmission capability of the first node comprises: determining a candidate transmission channel from the plurality of transmission channels that meets the data transmission capability of the first node; and selecting a channel that at least one of has the highest affinity or requires the least number of data copying from the candidate transmission channels as the target transmission channel.

9. The communication control method according to claim 6, characterized by, The determining a target transmission channel between the first node and the second node according to the first pointer, the second pointer and the data transmission capability of the first node comprises: in response to the address space where the first-level buffer pointed by the first pointer is located not matching the data transmission capability of the first node, determining the target transmission channel as transmitting the to-be-transmitted data from the first-level buffer pointed by the first pointer to the second-level buffer pointed by the second pointer via a second-level buffer corresponding to a first processor in the first node, wherein the first processor is different from the processor corresponding to the address space where the first-level buffer pointed by the first pointer is located.

10. The communication control method according to claim 1 or 2, characterized by, The communication control method further comprises: determining the data transmission capability of the first node according to a communication routing table, wherein the step of creating the communication routing table comprises: detecting a local address space in the first node that allows inter-node data transmission, and registering address space information of the second-level buffer in the local address space for the first node; obtaining address space information of the second-level buffer in a remote address space that allows inter-node data transmission registered for the second node; and creating the communication routing table according to the address space information registered for the first node and the second node.

11. The communication control method according to claim 10, characterized by, The address space information in the communication routing table at least comprises an address space type of the second-level buffer, and the address space type comprises a first type and a second type, the second-level buffer of the first type belongs to the address space of a first type processor in a corresponding node, and the second-level buffer of the second type belongs to the address space of a second type processor in the corresponding node.

12. The communication control method according to claim 11, wherein The determining the data transmission capability of the first node according to the communication routing table comprises: In response to the address space type of the second-level buffer in the local address space comprising the first type and / or the second type, the determining the data transmission capability of the first node comprises allowing the inter-node data transmission based on the address space of the processor of the corresponding type.

13. The communication control method according to claim 11, wherein The address space information in the communication routing table further comprises the node identifier, the start address and the buffer capacity corresponding to each registered second-level buffer.

14. A communication system, characterized by The communication system comprises a plurality of nodes, each node comprising a communication interface, a communication controller and at least two types of processors, the at least two types of processors comprising a first type of processor and a second type of processor, the plurality of nodes comprising a first node and a second node, The communication interface of the first node is configured to, in response to the first node triggering a data transmission operation between the first node and the second node, acquire a first pointer and a second pointer of to-be-transmitted data, and determine a second-level buffer in the first node for the to-be-transmitted data according to the first pointer and the data transmission capability of the first node, wherein the first pointer points to a first-level buffer in the first node for the to-be-transmitted data, and the second pointer points to a second-level buffer in the second node for the to-be-transmitted data; and The communication controller of the first node is configured to, in response to the first-level buffer and the second-level buffer in the first node both belonging to the address space of the second type of processor, and the first-level buffer and the second-level buffer in the second node both belonging to the address space of the first type of processor, determine that a target transmission channel for transmitting the to-be-transmitted data between the first node and the second node is a direct transmission channel between the address space of the second type of processor of the first node and the address space of the first type of processor of the second node.

15. An electronic device, comprising: The electronic device comprises at least one processor and at least one memory, wherein the at least one memory has program code stored therein, and the program code, when executed by the at least one processor, causes the at least one processor to perform the communication control method according to any one of claims 1-13.

16. A non-transitory computer-readable storage medium, comprising: The non-transitory computer-readable storage medium has computer-readable instructions stored thereon, and the computer-readable instructions, when executed by at least one processor, cause the at least one processor to perform the communication control method according to any one of claims 1-13.

Images