Communication control method, communication system, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a communication control method, a communication system, electronic equipment and a storage medium. The method is used for a first node in a communication group, and comprises the following steps: acquiring a first pointer and a second pointer of data to be transmitted; according to the first pointer and the data transmission capability of the first node, determining a second-level buffer area for the to-be-transmitted data in the first node; in response to the fact that both a first-level buffer area and a second-level buffer area in the first node belong to an address space of a second-type processor, and both a first-level buffer area and a second-level buffer area in the second node belong to an address space of a first-type processor; and determining that the target transmission channel between the first node and the second node is a direct transmission channel between the address space of the second type processor of the first node and the address space of the first type processor of the second node. According to the method, a high-efficiency communication path can be automatically selected, the introduction of additional data copying operation is avoided, and the complexity of cross-memory type communication is reduced.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of computer technology, and more specifically to a communication control method, a communication system, an electronic device, and a storage medium. Background Technology

[0002] With the rapid development of artificial intelligence (AI) technology, AI has been widely applied to various complex tasks. However, traditional single-machine environments face bottlenecks such as excessively long training times in large-scale model training. Distributed systems, through multi-node parallel computing, have become the mainstream solution to improve training efficiency. Heterogeneous computing platforms can be applied to distributed architectures. By integrating processors of various types and architectures to work collaboratively, the performance and functional advantages of each processor can be fully utilized to meet increasingly complex computing demands. However, their heterogeneity also leads to increasingly prominent communication overhead between nodes, becoming a problem that urgently needs to be solved. Summary of the Invention

[0003] To address the issues of low communication performance and high overhead between nodes in heterogeneous computing platforms, at least one embodiment of this disclosure provides a communication control method, a communication system, an electronic device, and a storage medium.

[0004] The communication control method provided in at least one embodiment of this disclosure is used for a first node in a communication group, comprising: acquiring a first pointer and a second pointer for data to be transmitted, wherein the first pointer points to a first-level buffer in the first node for data to be transmitted, the communication group further comprising a second node, the second pointer pointing to a second-level buffer in the second node for data to be transmitted, each node in the communication group comprising a first-type processor and a second-type processor; determining the second-level buffer in the first node for data to be transmitted based on the first pointer and the data transmission capability of the first node; in response to the fact that both the first-level buffer and the second-level buffer in the first node belong to the address space of the second-type processor, and both the first-level buffer and the second-level buffer in the second node belong to the address space of the first-type processor, determining the target transmission channel between the first node and the second node for transmitting the data to be transmitted as a direct transmission channel between the address space of the second-type processor of the first node and the address space of the first-type processor of the second node.

[0005] For example, in the communication control method provided in at least one embodiment of this disclosure, the data transmission capability indicator allows inter-node data transmission based on the address space of which type of processor in the corresponding node, or disallows inter-node data transmission based on the address space of which type of processor in the corresponding node.

[0006] For example, in the communication control method provided in at least one embodiment of this disclosure, after obtaining the first pointer and the second pointer of the data to be transmitted, the communication control method further includes: identifying the type of the address space where the first level buffer pointed to by the first pointer is located; and determining whether the type of the address space where the first level buffer pointed to by the first pointer is located matches the data transmission capability of the first node.

[0007] For example, in the communication control method provided in at least one embodiment of this disclosure, determining whether the address space where the first level buffer pointed to by the first pointer is located matches the data transmission capability of the first node includes: determining that the address space where the first level buffer pointed to by the first pointer is located matches the data transmission capability of the first node in response to the type of the processor's address space allowed by the data transmission capability of the first node including the type of the address space where the first level buffer pointed to by the first pointer is located matches the data transmission capability of the first node.

[0008] For example, in the communication control method provided in at least one embodiment of this disclosure, determining a second-level buffer in the first node for data to be transmitted based on a first pointer and the data transmission capability of the first node includes: in response to the first-level buffer pointed to by the first pointer belonging to the address space of a second type of processor, and the address space of the processor allowed by the data transmission capability of the first node including the address space of the second type of processor, determining the second-level buffer in the first node for data to be transmitted as a second-level buffer in the address space of the second type of processor; or, in response to the first-level buffer pointed to by the first pointer belonging to the address space of a second type of processor, and the address space of the processor allowed by the data transmission capability of the first node not including the address space of the second type of processor, determining the second-level buffer in the first node for data to be transmitted as a second-level buffer in the address space of the first type of processor.

[0009] For example, at least one embodiment of the communication control method provided in this disclosure further includes: determining a target transmission channel between the first node and the second node based on the data transmission capabilities of the first pointer, the second pointer, and the first node.

[0010] For example, in the communication control method provided in at least one embodiment of this disclosure, determining a target transmission channel between a first node and a second node based on a first pointer, a second pointer, and the data transmission capability of a first node includes: determining multiple transmission channels based on the first pointer and the second pointer; and determining a target transmission channel from the multiple transmission channels based on the data transmission capability of the first node.

[0011] For example, in the communication control method provided in at least one embodiment of this disclosure, determining a target transmission channel from multiple transmission channels based on the data transmission capability of the first node includes: determining candidate transmission channels that meet the data transmission capability of the first node from multiple transmission channels; and selecting from the candidate transmission channels at least one of the following: the channel that satisfies the highest affinity or the minimum number of data copies required, as the target transmission channel.

[0012] For example, in the communication control method provided in at least one embodiment of this disclosure, determining the target transmission channel between the first node and the second node based on the data transmission capabilities of the first pointer, the second pointer, and the first node includes: in response to a mismatch between the address space where the first-level buffer pointed to by the first pointer is located and the data transmission capability of the first node, determining the target transmission channel as: sending the data to be transmitted from the first-level buffer pointed to by the first pointer to the second-level buffer via the second-level buffer corresponding to the first processor in the first node to the second-level buffer pointed to by the second pointer, wherein the first processor is different from the processor corresponding to the address space where the first-level buffer pointed to by the first pointer is located.

[0013] For example, at least one embodiment of the communication control method provided in this disclosure further includes: determining the data transmission capability of a first node according to a communication routing table, wherein the step of creating a communication routing table includes: detecting the local address space in the first node that allows data transmission between nodes, and registering the address space information of a second-level buffer in the local address space for the first node; obtaining the address space information of a second-level buffer in the remote address space registered for the second node; and creating a communication routing table according to the address space information registered for the first node and the second node.

[0014] For example, in the communication control method provided in at least one embodiment of this disclosure, the address space information in the communication routing table includes at least the address space type of the second-level buffer. The address space type includes a first type and a second type. The second-level buffer of the first type belongs to the address space of the first type processor in the corresponding node, and the second-level buffer of the second type belongs to the address space of the second type processor in the corresponding node.

[0015] For example, in the communication control method provided in at least one embodiment of this disclosure, determining the data transmission capability of a first node according to a communication routing table includes: in response to the address space type of a second-level buffer in the local address space including a first type and / or a second type, determining the data transmission capability of the first node includes allowing data transmission between nodes based on the address space of the corresponding type of processor.

[0016] For example, in the communication control method provided in at least one embodiment of this disclosure, the address space information in the communication routing table also includes the node identifier, starting address, and buffer capacity corresponding to each registered second-level buffer.

[0017] At least one embodiment of this disclosure also provides a communication system comprising multiple nodes, wherein each node includes a communication interface, a communication controller, and at least two types of processors, including a first type processor and a second type processor, and the multiple nodes include a first node and a second node; the communication interface of the first node is configured to: in response to the first node triggering a data transmission operation with the second node, acquire a first pointer and a second pointer to the data to be transmitted, and determine a second-level buffer in the first node for the data to be transmitted based on the first pointer and the data transmission capability of the first node, wherein the first pointer points to the first-level buffer in the first node for the data to be transmitted, and the second pointer points to the second-level buffer in the second node for the data to be transmitted; and the communication controller of the first node is configured to: in response to the first-level buffer and the second-level buffer in the first node both belonging to the address space of the second type processor, and the first-level buffer and the second-level buffer in the second node both belonging to the address space of the first type processor, determine the target transmission channel between the first node and the second node for transmitting the data to be transmitted as: a direct transmission channel between the address space of the second type processor of the first node and the address space of the first type processor of the second node.

[0018] At least one embodiment of this disclosure also provides an electronic device, including: at least one processor; and at least one memory, wherein the at least one memory stores program code, which, when executed by the at least one processor, causes the at least one processor to perform the communication control method provided in any embodiment of this disclosure.

[0019] At least one embodiment of this disclosure provides a non-transitory computer-readable storage medium storing computer-readable instructions thereon, which, when executed by at least one processor, cause the processor to perform the communication control method provided in any embodiment of this disclosure.

[0020] In the communication control method provided in at least one embodiment of this disclosure, the target transmission channel of the data to be transmitted is determined by the address space of the first-level buffer pointed to by the first pointer of the data to be transmitted and the data transmission capability of the first node. This enables direct data transmission between address spaces of different types of processors between heterogeneous nodes (e.g., direct data transmission between the address space of the second type of processor of the first node and the address space of the first type of processor of the second node), automatically schedules efficient communication paths (data transmission channels), avoids introducing additional data copy operations and eliminates the need to write path code for data transmission between different nodes, thereby improving communication efficiency and simplifying the complexity of user programming. It can also support member nodes using address spaces of different types of processors in the same communication group, reducing the complexity of cross-memory type communication and simplifying the complexity of user programming. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0022] Figure 1 An exemplary communication mode for a heterogeneous computing platform is shown.

[0023] Figure 2 A flowchart illustrating a communication control method provided in at least one embodiment of this disclosure is shown.

[0024] Figure 3 A schematic diagram of the process for creating a communication routing table is shown, according to at least one embodiment of the present disclosure.

[0025] Figure 4 A schematic diagram of a data transmission process provided by at least one embodiment of the present disclosure is shown.

[0026] Figure 5 A flowchart illustrating another communication control method provided in at least one embodiment of this disclosure is shown.

[0027] Figure 6A A schematic diagram of another data transmission process provided by at least one embodiment of the present disclosure is shown.

[0028] Figure 6B A schematic diagram of yet another data transmission process provided by at least one embodiment of the present disclosure is shown.

[0029] Figure 7A A schematic diagram of a communication system provided in at least one embodiment of the present disclosure is shown.

[0030] Figure 7BA flowchart illustrating the communication process between nodes provided in at least one embodiment of this disclosure is shown.

[0031] Figure 8 A schematic diagram of the structure of an electronic device provided in at least one embodiment of the present disclosure is shown.

[0032] Figure 9 A schematic diagram of the structure of a non-transitory computer-readable storage medium provided in at least one embodiment of the present disclosure is shown. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0034] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an,” “a,” or “the,” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “including,” “comprising,” or “containing,” and similar terms mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. The terms “connected,” “linked,” or similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “upper,” “lower,” “left,” and “right,” etc., are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described objects changes.

[0035] A distributed system is a system composed of multiple distributed nodes connected by a network. The nodes in a distributed system work collaboratively to complete one or more target tasks, such as training or inference of a large-scale language model.

[0036] A communication group is a logically divided set of nodes in a distributed system or parallel computing environment to achieve specific communication goals (such as data reading and writing, data exchange, task coordination, state synchronization, etc.). Nodes within the same communication group can communicate directly with each other, while nodes outside the group typically do not participate in the internal interactions of the communication group (unless through a cross-group mechanism).

[0037] In this disclosure, a “node” can be a physical device (hardware device), such as a computer, server, edge computing device, etc.; or a virtual device, such as a virtual machine, etc. This disclosure does not limit the scope of the term.

[0038] Heterogeneous computing platforms can be applied to distributed architectures. By integrating processors of various types and architectures to work collaboratively, they can fully leverage the performance and functional advantages of each processor to meet increasingly complex computing demands. For example, a heterogeneous computing platform can be a CPU-GPU architecture or a CPU-GPGPU architecture. In heterogeneous computing platforms, communication efficiency is crucial to overall performance.

[0039] Most mainstream communication libraries currently use a "single-type memory communication group" model. This model requires communication operations within the communication group to explicitly distinguish the memory type where the data resides (also known as the "data address type"). For example, it requires explicitly distinguishing between the CPU pointer (hereinafter referred to as the "CPU pointer," i.e., a pointer to the CPU's address space) and the GPU pointer (hereinafter referred to as the "GPU pointer," i.e., a pointer to the GPU's address space). This has brought many problems to practical applications.

[0040] For example, memory type identification relies on manual judgment, requiring users to manually specify the memory type (CPU or GPU) where the data resides. For example, communication paths (or data transfer channels) cannot be automatically scheduled; developers must select the communication method based on the data's address type or explicitly copy the data to a relay area before communication. This process introduces additional operational latency, reducing communication efficiency. For example, communication group semantics lack uniformity; existing communication libraries do not support including member nodes using both CPU and GPU address spaces within the same communication group, limiting the possibility of direct communication between heterogeneous nodes and increasing the complexity of cross-memory type communication. For example, when integrating these communication libraries, deep learning frameworks or cluster communication middleware struggle to achieve efficient and automatic fusion of communication processes between CPU and GPU, leading to low software development efficiency and failing to fully leverage the communication performance potential of heterogeneous computing platforms. These problems severely reduce the communication performance of heterogeneous computing platforms and the development efficiency of related software.

[0041] For example, Figure 1 An exemplary communication mode for a heterogeneous computing platform is shown.

[0042] like Figure 1As shown, the communication group includes at least node Rank0 and node Rank1, and each node includes a CPU and a GPU. DRAM_0 represents the address space (also known as "memory") corresponding to the CPU in node Rank0, and VRAM_0 represents the address space (also known as "video memory") corresponding to the GPU in node Rank0. The meanings of DRAM_1 and VRAM_1 are similar to those of DRAM_0 and VRAM_0, and will not be repeated here.

[0043] When node Rank0 transmits data to node Rank1, it is necessary to manually determine the type of the address space where the data is located and the type of the address space of the receiving data, and select the data transmission path based on the type of the address space where the data is located and the type of the address space of the receiving data.

[0044] exist Figure 1 In the example shown, data is transferred from VRAM_0 of node Rank0 to DRAM_1 of node Rank1, with the target transfer channel as follows: Figure 1 As shown by the dashed arrow, node Rank0 copies data Data directly from buffer V0 of VRAM_0 to buffer D1 of DRAM_1. However, due to the lack of uniformity in communication group semantics, only data transfer between address spaces of the same type is supported (e.g., data transfer between address spaces corresponding to the CPU, data transfer between address spaces corresponding to the GPU). The actual transmission channel is as follows: Figure 1 As shown by the solid arrow, node Rank0 typically copies data from buffer V0 of VRAM_0 to buffer V1 of VRAM_1 of node Rank1, and then node Rank1 copies data from buffer V1 of VRAM_1 to buffer D1 of DRAM_1. This data transfer introduces an extra data copy operation, thus reducing communication efficiency. The lack of semantic uniformity in communication groups limits heterogeneous nodes (e.g., Figure 1 The possibility of direct communication between nodes Rank0 and Rank1 in the system.

[0045] This disclosure provides at least one embodiment of a communication control method and a communication system.

[0046] The communication control method provided in at least one embodiment of this disclosure is used for a first node in a communication group. The method includes: acquiring a first pointer and a second pointer for data to be transmitted, wherein the first pointer points to a first-level buffer in the first node for data to be transmitted, the communication group further includes a second node, and the second pointer points to a second-level buffer in the second node for data to be transmitted, each node in the communication group includes a first-type processor and a second-type processor; determining the second-level buffer in the first node for data to be transmitted based on the first pointer and the data transmission capability of the first node; and in response to the fact that both the first-level buffer and the second-level buffer in the first node belong to the address space of the second-type processor, and both the first-level buffer and the second-level buffer in the second node belong to the address space of the first-type processor, determining the target transmission channel between the first node and the second node for transmitting the data to be transmitted as a direct transmission channel between the address space of the second-type processor of the first node and the address space of the first-type processor of the second node.

[0047] In the communication control method provided in at least one embodiment of this disclosure, the target transmission channel of the data to be transmitted is determined by the address space of the first-level buffer pointed to by the first pointer of the data to be transmitted and the data transmission capability of the first node. This enables direct data transmission between address spaces of different types of processors between heterogeneous nodes (e.g., direct data transmission between the address space of the second type of processor of the first node and the address space of the first type of processor of the second node), automatically scheduling efficient communication paths (data transmission channels), avoiding the introduction of additional data copy operations and eliminating the need to write path code for data transmission between different nodes, thereby improving communication efficiency and simplifying the complexity of user programming. It can also support the simultaneous inclusion of member nodes using address spaces of different types of processors in the same communication group, reducing the complexity of cross-memory type communication and simplifying the complexity of user programming, thus improving system compatibility.

[0048] The communication control method provided in at least one embodiment of this disclosure can be applied to deep learning training systems and inference systems. For example, it can be applied to checkpoint saving operations during training, and also to key-value cache (KVCache) transmission operations in inference systems under a prefill-decode (PD) separation architecture.

[0049] For example, in deep learning training scenarios, checkpoint saving is a crucial operation to ensure training continuity, requiring the reliable storage of a large amount of model parameters and training state data. The communication control method provided in at least one embodiment of this disclosure can optimize data transmission strategies to address the characteristics of this process, ensuring the complete saving and rapid recovery of model parameters without significantly impacting training efficiency.

[0050] For example, in an inference system, when a PD (Programmable Detection) separation architecture is adopted, the key-value cache, as important intermediate data generated during inference, directly affects inference latency and system response speed due to its transmission efficiency. The communication control method provided in at least one embodiment of this disclosure can improve the transmission efficiency of the key-value cache, providing strong support for the high-performance operation of the inference system.

[0051] The present disclosure will now be described through several specific embodiments. To keep the following description of the embodiments of the present disclosure clear and concise, detailed descriptions of known functions and components may be omitted. When any component of the embodiments of the present disclosure appears in more than one drawing, the component is represented by the same or similar reference numerals in each drawing.

[0052] The communication control method provided in at least one embodiment of this disclosure can be used in a first node of a communication group to realize data transmission with a second node in the communication group. It should be noted that the "first node" in the embodiments of this disclosure refers to the node in the communication group that initiates the data transmission operation, that is, the initiator of the data transmission operation, and does not refer to a fixed node.

[0053] For example, when the communication mode is write mode, the data transmission operation is actively triggered by the data sending node, such as actively initiating a write operation to write data to the data receiving node. In this case, the first node is the data sending node, and the second node is the data receiving node.

[0054] For example, in read mode, the data receiving node actively triggers the data transmission operation, such as initiating a read operation to read data from the sending node. In this case, although the data flow is from the sending end to the receiving end, the operation is initiated by the receiving node. Therefore, in this scenario, the first node is the receiving node, and the second node is the sending node.

[0055] For example, when the communication mode is send / recv, both the data sending node and the data receiving node trigger data transmission operations. The data sending node performs the send operation, and the data receiving node performs the recv operation. Since these two operations are initiated independently by both parties, each is the initiator of the data transmission operation within its own operational context. Therefore, when applying the communication control method provided in this embodiment from the perspective of the data sending end, the first node is the data sending end node, and the second node is the data receiving end node; when applying the communication control method provided in this embodiment from the perspective of the data receiving end, the first node is the data receiving end node, and the second node is the data sending end node.

[0056] In other words, in the communication control method provided in at least one embodiment of this disclosure, the executing entity is a node that actively initiates data transmission operations in the current communication context.

[0057] Figure 2 A flowchart illustrating a communication control method provided in at least one embodiment of this disclosure is shown.

[0058] For example, such as Figure 2 As shown, the communication control method provided in at least one embodiment of this disclosure may include the following steps S210-S230.

[0059] Step S210: Obtain a first pointer and a second pointer for the data to be transmitted, wherein the first pointer points to a first-level buffer in the first node for the data to be transmitted, the communication group also includes a second node, and the second pointer points to a second-level buffer in the second node for the data to be transmitted, and each node in the communication group includes a first-type processor and a second-type processor.

[0060] Step S220: Based on the data transmission capability of the first pointer and the first node, determine the second-level buffer in the first node for the data to be transmitted.

[0061] Step S230: In response to the fact that both the first-level buffer and the second-level buffer in the first node belong to the address space of the second type of processor, and both the first-level buffer and the second-level buffer in the second node belong to the address space of the first type of processor, the target transmission channel for transmitting the data to be transmitted between the first node and the second node is determined to be: a direct transmission channel between the address space of the second type of processor of the first node and the address space of the first type of processor of the second node.

[0062] It's important to note that data transmission (or communication) involves two types of buffers: user buffers (the "first-level buffer") and staging buffers (the "second-level buffer"). User buffers represent the buffers where the user of the corresponding node actually sends or stores data, and are visible to the user. Staging buffers represent the buffers registered on the network interface card (NIC) of the corresponding node, and are invisible to the user. Nodes within the same communication group transmit data between themselves through their respective staging buffers. For example, a sending node copies data from its user buffer to its staging buffer, then transmits the data to the receiving node's staging buffer via its staging buffer. Finally, the receiving node copies the data received in its staging buffer back to its user buffer. Furthermore, a node's user buffer can also serve as its staging buffer, thus avoiding additional copying of data between the user buffer and the staging buffer.

[0063] In addition, in order to enable data copying between the user buffer and the temporary buffer of the same node, the temporary buffer of the node can be determined based on the user buffer of the node and the data transmission capability of the node.

[0064] For step S210, for example, in write mode, the first node actively initiates a write operation to write data (i.e., "data to be transmitted") to the second node. The first node can obtain a first pointer to the data to be transmitted (e.g., send_user_ptr, representing the storage address of the data to be transmitted in the first node) and the size of the data to be transmitted (e.g., size_t size) based on its communication interface (e.g., send(void*send_user_ptr, size_t size)). Then, the first node obtains a second pointer to the data to be transmitted (i.e., the actual receiving address from which the second node receives the data to be transmitted) by negotiating with the second node (e.g., establishing a temporary Transmission Control Protocol (TCP) link). The address pointed to by the first pointer is the address corresponding to the first-level buffer (i.e., the "user buffer") in the first node used to store the data to be transmitted, and the address pointed to by the second pointer is the address corresponding to the second-level buffer (i.e., the "temporary buffer") in the second node used to receive the data to be transmitted. The second pointer points to the second-level buffer, which is determined by the second node based on the actual write address of the data to be transmitted (i.e., the address corresponding to the first-level buffer ("user buffer") in the second node used for that data) and the data transmission capability of the second node. The address of the temporary buffer is then sent to the first node via, for example, a TCP link. The specific method by which the second node determines the second pointer can be found in the section below on how the first node determines the second-level buffer used for the data to be transmitted; it will not be elaborated upon here.

[0065] For example, the first-level buffer ("user buffer") used for the data to be transmitted in the second node can be determined by the second node. For instance, in response to a write operation initiated by the first node, the second node triggers a receive operation and obtains the write address of the data to be transmitted (receive_user_ptr represents the actual write address of the data to be transmitted in the second node, i.e., the address of the first-level buffer used for the data to be transmitted) through the second node's communication interface (e.g., receive(void* receive_user_ptr, size_t size)). This address allows the determination of the first-level buffer in the second node used for the data to be transmitted. Furthermore, send_user_ptr and receive_user_ptr can be specified by the user.

[0066] For example, each node in the communication group includes at least two different types of processors: a first type processor and a second type processor. In some examples, the first type processor can be a Central Processing Unit (CPU), and the second type processor can be a Graphics Processing Unit (GPU). Alternatively, the first and second types of processors can also be other types of processors, such as a General-Purpose Graphics Processing Unit (GPGPU), a Tensor Processing Unit (TPU), a Neural Network Processing Unit (NPU), a Deep Learning Processing Unit (DPU), an Accelerated Processing Unit (APU), a Field Programmable Gate Array (FPGA), and a multi-core processor with artificial intelligence computing capabilities, etc., without limitation in this disclosure.

[0067] For example, each type of processor has its own dedicated address space, which corresponds to a storage medium adapted to the characteristics of that processor. For example, the address space of a CPU is typically mapped to Dynamic Random Access Memory (DRAM), while the address space of a GPU is typically mapped to Video Random Access Memory (VRAM), and this disclosure is not limiting in this regard.

[0068] In at least one embodiment of this disclosure, the data transfer capability indicates which type of processor address space in the corresponding node is allowed to perform inter-node data transfer, or which type of processor address space in the corresponding node is not allowed to perform inter-node data transfer.

[0069] For example, if the first node includes both a CPU and a GPU, and the data transfer capability of the first node indicates that inter-node data transfer is not allowed based on the CPU's address space but is allowed based on the GPU's address space, then if the first pointer of the data to be transferred points to the first-level buffer, which belongs to the GPU's address space, the data to be transferred can be directly transferred from the GPU's address space of the first node to the second node, without having to transfer it from the GPU's address space of the first node through the CPU's address space to the second node. If the first pointer of the data to be transferred points to the first-level buffer, which belongs to the CPU's address space, then the data to be transferred needs to be transferred from the CPU's address space of the first node through the GPU's address space to the second node.

[0070] In at least one embodiment of this disclosure, the data transmission capability of the first node can be determined based on the communication routing table. Furthermore, the first node can also determine the data transmission capability of the second node using the same communication routing table.

[0071] Figure 3 A schematic diagram of the process for creating a communication routing table is shown, according to at least one embodiment of the present disclosure.

[0072] like Figure 3 As shown, the steps for creating a communication routing table include steps S310 to S330.

[0073] Step S310: Detect the second-level buffer of the local address space in the first node that allows inter-node data transmission, and register the address space information of the second-level buffer in the local address space for the first node.

[0074] Step S320: Obtain the address space information of the second-level buffer in the remote address space that is registered for the second node and allows inter-node data transmission.

[0075] Step S330: Create a communication routing table based on the address space information registered for the first and second nodes.

[0076] For step S310, after creating a communication group and performing a communication group connection establishment operation, for the first node in the communication group, the local address space in the first node that allows data transmission between nodes is detected (i.e., the second-level buffer (temporary buffer) in the first node that allows data transmission between nodes is detected), and the address space information of the second-level buffer in the local address space is registered for the first node.

[0077] For example, the address space information includes at least the address space type of the second-level buffer. The address space type includes a first type and a second type. The second-level buffer of the first type belongs to the address space of the first type processor in the corresponding node, and the second-level buffer of the second type belongs to the address space of the second type processor in the corresponding node.

[0078] For example, the address space information may also include the node identifier corresponding to each registered second-level buffer, the starting address of each registered second-level buffer, and the buffer capacity. The node identifier indicates whether the corresponding second-level buffer belongs to the "local address space" (i.e., the second-level buffer of the current node) or the "remote address space" (i.e., the second-level buffer of another node besides the current node). The address space information may also include other information (e.g., the device number of the corresponding node, the remote key (rkey) used for access permission verification), which is not limited in this disclosure.

[0079] It is important to note that the "address space type of the second-level buffer" in the address space information indicates which type of processor's address space the second-level buffer belongs to.

[0080] For step S320, a temporary communication link (e.g., a TCP link) can be established between the first node and other nodes in the same communication group (e.g., the second node). The first node exchanges information with the second node through this temporary communication link to obtain the address space information of the second-level buffer in the remote address space registered for the second node to allow inter-node data transmission. Simultaneously with the information exchange, the second node can also obtain the address space information of the second-level buffer in the local address space registered for the first node to allow inter-node data transmission.

[0081] It is important to note that "local address space" and "remote address space" are relative terms. For example, for the first node, "local address space" refers to the address space of the first node, while "remote address space" refers to the address space of other nodes in the same communication group (e.g., the second node). For the second node, "local address space" refers to the address space of the second node, while "remote address space" refers to the address space of other nodes in the same communication group (e.g., the first node).

[0082] For step S330, a corresponding communication routing table is created for the first node based on the address space information registered for the first node and the second node.

[0083] For example, the step of determining the data transmission capability of the first node based on the communication routing table may include: in response to the address space type of the second-level buffer in the local address space including a first type and / or a second type, determining the data transmission capability of the first node includes allowing inter-node data transmission based on the address space of the corresponding type of processor.

[0084] For example, Table 1 shows an example of a communication routing table created for the first node.

[0085] Table 1

[0086]

[0087] As shown in Table 1, the node identifier "local" indicates that entries 1 and 2 are used to store the address space information of the second-level buffer in the local address space of the first node, which allows inter-node data transfer. The node identifier "remote" indicates that entries 3 and 4 are used to store the address space information of the second-level buffer in the remote address space of, for example, the second node, which allows inter-node data transfer. For example, the address space type "DRAM" indicates that the corresponding second-level buffer belongs to the address space of the CPU in the corresponding node, and the address space type "VRAM" indicates that the corresponding second-level buffer belongs to the address space of the GPU in the corresponding node.

[0088] For example, based on entries 1 and 2 in the communication routing table, the address space types of the second-level buffers in the local address space include DRAM and VRAM. Therefore, it can be determined that the data transfer capability of the first node includes allowing inter-node data transfer based on the second-level buffers in the CPU's address space (i.e., the second-level buffers in DRAM) and the second-level buffers in the GPU's address space (i.e., the second-level buffers in VRAM). In other words, the first node supports Remote Direct Memory Access (RDMA) and GPU Direct RDMA (GDR). In the first node, the starting address of the second-level buffer in DRAM is 0x10000, and its buffer capacity is 64MB. In the first node, the starting address of the second-level buffer in VRAM is 0x20000, and its buffer capacity is 128MB.

[0089] For example, if the communication routing table shown in Table 1 does not include the above-mentioned Table 2, then the data transmission capability of the first node is determined to include allowing inter-node data transmission based on the second-level buffer in the address space of the CPU in the first node (i.e., the second-level buffer in DRAM), but not allowing inter-node data transmission based on the second-level buffer in the address space of the GPU (i.e., the second-level buffer in VRAM), that is, the first node supports RDMA but does not support GDR.

[0090] For example, based on entries 3 and 4 in the communication routing table, the address space types of the second-level buffers in the remote address space include DRAM and VRAM. Therefore, it can be determined that the data transfer capability of the second node includes allowing inter-node data transfer based on the second-level buffers in the CPU address space (i.e., the second-level buffers in DRAM) and the second-level buffers in the GPU address space (i.e., the second-level buffers in VRAM). In other words, the second node supports Remote Direct Memory Access (RDMA) and GPU Direct RDMA (GDR). In the second node, the starting address of the second-level buffer in DRAM is 0x30000, and its buffer capacity is 64MB. The starting address of the second-level buffer in VRAM in the second node is 0x40000, and its buffer capacity is 128MB.

[0091] For example, the communication routing table shown in Table 1 may also include entry 5, where the address space information includes: node identifier "remote", address space type "VRAM", starting address "0x50000", and capacity "128MB". In this example, based on entries 4 and 5 in the communication routing table, it is determined that the GPU's address space (VRAM) in the second node includes two second-level buffers, and the second node can perform inter-node data transfer with other nodes (e.g., the first node) through either of these two second-level buffers. This disclosure does not limit the number of second-level buffers in the address space of various types of processors.

[0092] It should be noted that the second node can also create a communication routing table in a similar manner to steps S310 to S330. For example, the second node can perform the following operations: detect the second-level buffers in the local address space that allow inter-node data transmission in the second node, and register the address space information of the second-level buffers in the local address space for the second node; obtain the address space information of the second-level buffers in the remote address space that allow inter-node data transmission registered for the first node; and create a communication routing table based on the address space information registered for the first and second nodes.

[0093] For step S220 above, after obtaining the first pointer of the data to be transmitted, a second-level buffer (i.e., "temporary buffer") in the first node can be determined based on the first buffer pointed to by the first pointer and the data transmission capability of the first node, so that the data to be transmitted can be copied from the first buffer in the first node to the determined second-level buffer. Then, the first node performs inter-node data transmission operations with other nodes in the same communication group (e.g., the second node) through the second-level buffer.

[0094] For example, in response to the fact that the first-level buffer pointed to by the first pointer belongs to the address space (e.g., VRAM) of the second-type processor (e.g., GPU), and the address space of the processor allowed by the data transfer capability of the first node includes the address space of the second-type processor, the second-level buffer in the first node used for the data to be transferred is determined to be the second-level buffer in the address space of the second-type processor.

[0095] For example, in response to the first-level buffer pointed to by the first pointer belonging to the address space (e.g., VRAM) of a second-type processor (e.g., GPU), and the address space of the processor allowed by the data transfer capability of the first node does not include the address space of the second-type processor, the second-level buffer in the first node used for the data to be transferred is determined to be a second-level buffer in the address space (e.g., DRAM) of the first-type processor (e.g., CPU).

[0096] For example, the first level buffer pointed to by the first pointer belongs to the address space (e.g., DRAM) of a first type of processor (e.g., CPU), and the address space of the processor allowed by the data transfer capability of the first node includes the address space of the first type of processor. The second level buffer in the first node used for the data to be transferred is determined to be the second level buffer in the address space of the first type of processor.

[0097] For example, in response to the first level buffer pointed to by the first pointer belonging to the address space (e.g., DRAM) of a first type of processor (e.g., CPU), and the address space of the processor allowed by the data transfer capability of the first node does not include the address space of the first type of processor, the second level buffer in the first node used for the data to be transferred is determined to be a second level buffer in the address space (e.g., VRAM) of a second type of processor (e.g., GPU).

[0098] Figure 4 A schematic diagram of a data transmission process provided by at least one embodiment of the present disclosure is shown.

[0099] like Figure 4As shown, both the first node Rank0 and the second node Rank1 include CPU and GPU. DRAM_0 and VRAM_0 represent the address space of the CPU and GPU of the first node Rank0, respectively. DRAM_1 and VRAM_1 represent the address space of the CPU and GPU of the second node Rank1, respectively.

[0100] The following is combined with Figure 4 The above steps S230 are described.

[0101] For step S230, as Figure 4 As shown, in write mode, the first-level buffer pointed to by the first pointer (i.e., the user buffer user_buffer_v0 where the data to be transferred, Data, is located) belongs to the address space (VRAM_0) of the second-type processor (GPU). Since the data transfer capability of the first node Rank0 includes allowing inter-node data transfer based on the address space of the second-type processor, the second-level buffer in the first node Rank0 used for the data to be transferred, Data, is determined to be the temporary buffer stg_buffer_v0 in the address space VRAM_0. Since the first-level buffer in the second node Rank1 used for writing the data to be transferred, Data (i.e., the user buffer user_buffer_d1), belongs to the address space (DRAM_1) of the first-type processor (CPU), and the data transfer capability of the second node Rank1 includes allowing inter-node data transfer based on the address space of the first-type processor, the second-level buffer in the second node Rank1 used for the data to be transferred, Data, is determined to be the temporary buffer stg_buffer_d1 in the address space DRAM_1.

[0102] exist Figure 4In the example shown, in response to the fact that the first-level buffer (i.e., user buffer user_buffer_v0) and the second-level buffer (i.e., temporary buffer stg_buffer_v0) in the first node Rank0 both belong to the address space (VRAM_0) of the second type of processor (GPU), and the first-level buffer (i.e., user buffer user_buffer_d1) and the second-level buffer (i.e., temporary buffer stg_buffer_d1) in the second node Rank1 both belong to the address space (DRAM_1) of the first type of processor (CPU), it is determined that there is a connection between the first node Rank0 and the second node Rank1. The target transmission channel for transmitting the data to be transmitted is a direct transmission channel between the address space (VRAM_0) of the second type processor (GPU) of the first node Rank0 and the address space (DRAM_1) of the first type processor (CPU) of the second node Rank1. That is, the data to be transmitted is transmitted from the user buffer user_buffer_v0 of the first node Rank0 through the temporary buffer stg_buffer_v0 of the first node Rank0 and the temporary buffer stg_buffer_d1 of the second node Rank1 to the user buffer user_buffer_d1 of the second node Rank1.

[0103] Figure 5 This is a flowchart illustrating another communication control method provided in at least one embodiment of the present disclosure.

[0104] For example, such as Figure 5 As shown, the communication control method provided in at least one embodiment of this disclosure may include steps S510-S540. Step S510 can be referred to step S210 above, and will not be repeated here.

[0105] In the communication control method provided in at least one embodiment of this disclosure, after obtaining the first pointer and the second pointer of the data to be transmitted, steps S520, S530 and S540 can be executed sequentially.

[0106] Step S520: Identify the type of the address space where the first level buffer pointed to by the first pointer is located.

[0107] Step S530: Determine whether the type of the address space where the first-level buffer pointed to by the first pointer is located matches the data transmission capability of the first node.

[0108] Step S540: Determine the target transmission channel between the first node and the second node based on the data transmission capabilities of the first pointer, the second pointer, and the first node.

[0109] For example, an implementation of step S520 above may include: identifying whether the address space where the first level buffer pointed to by the first pointer is located belongs to the address space of a first type of processor or the address space of a second type of processor.

[0110] Furthermore, after identifying whether the first-level buffer (i.e., the user buffer) pointed to by the first pointer belongs to the address space of the first type of processor or the address space of the second type of processor, the second-level buffer (i.e., the temporary buffer) in the first node used for the data to be transmitted can be further determined based on the first-level buffer pointed to by the first pointer and the data transmission capability of the first node.

[0111] For step S530, the data transmission capability of the first node can be determined according to the communication routing table. The specific steps can be referred to the embodiment corresponding to Table 1 above, and will not be repeated here.

[0112] For example, step S530 above may include: determining that the address space of the processor, which is allowed by the data transfer capability of the first node, includes the type of the address space where the first level buffer pointed to by the first pointer is located, and that the address space where the first level buffer pointed to by the first pointer is located matches the data transfer capability of the first node.

[0113] For example, step S540 above may include steps S540A and S540B.

[0114] Step S540A: Determine multiple transmission channels based on the first pointer and the second pointer.

[0115] Step S540B: ​​Determine the target transmission channel from multiple transmission channels based on the data transmission capability of the first node.

[0116] For example, step S540B may include: determining candidate transmission channels from a plurality of transmission channels that meet the data transmission capabilities of the first node; and selecting, from the candidate transmission channels, the channel that satisfies at least one of the following: highest affinity or fewest required data copy times as the target transmission channel.

[0117] For example, you can select the channel with the highest affinity from the candidate transmission channels as the target transmission channel, or you can select the channel with the fewest required data copies as the target transmission channel, or you can select the channel with both the highest affinity and the fewest required data copies as the target transmission channel, which can be set according to actual needs.

[0118] For example, the affinity of a transport channel is used to measure its transmission efficiency, performance, and resource overhead. Affinity can be determined based on one or more of the following factors: distance / hops, bandwidth, latency, hardware feature support, and dedication & contention. For distance / hops, the fewer physical nodes the data needs to traverse, the higher the affinity. For bandwidth, the higher the theoretical or actual transmission bandwidth of the transport channel, the higher the affinity. For latency, the lower the transmission latency of data through the channel, the higher the affinity. For dedication & contention, a dedicated transport channel that is not shared with other high-load tasks has higher affinity than a shared, congested transport channel. For hardware feature support, a transport channel has higher affinity when it supports advanced features such as zero-copy, remote direct memory access (RDMA), and GPU direct RDMA (GDR). For example, network cards that support GDR can directly access the GPU's video memory. This transmission channel is far more compatible than the transmission channel that first copies data from the GPU's video memory to the CPU's memory and then the network card reads it.

[0119] It should be noted that the “data copy count” in this disclosure refers to copy operations between the address spaces of different types of processors in the same node (e.g., copy operations between the address space of the CPU of the first node and the address space of the GPU of the first node). Copy operations between different buffers in the address space of the same processor (e.g., user buffers and temporary buffers in the DRAM of the CPU's address space) are not included in the “data copy count”.

[0120] The following is combined with Figure 4 The steps S540A and S540B are described in detail.

[0121] exist Figure 4In the example shown, the first level buffer (user buffer user_buffer_v0) pointed to by the first pointer belongs to the address space VRAM_0 of the GPU in the first node Rank0, and the second level buffer (temporary buffer stg_buffer_d1) pointed to by the second pointer belongs to the address space DRAM_1 of the CPU in the second node Rank1. Then, the multiple transmission channels used for the data to be transmitted include: channel 1 (from VRAM_0 to DRAM_1), channel 2 (from VRAM_0 via DRAM_0 to DRAM_1), and channel 3 (from VRAM_0 via VRAM_1 to DRAM_1).

[0122] If both Rank0 and Rank1 support RDMA and GDR (i.e., DRAM_0 and VRAM_0 in Rank0 and DRAM_1 and VRAM_1 in Rank1 can perform inter-node data transfer operations), then Channel 1, Channel 2, and Channel 3 all meet the data transfer capabilities of Rank0 and Rank1, and can all be candidate transmission channels. From the candidate transmission channels (Channel 1, Channel 2, and Channel 3), the channel that satisfies at least one of the following: highest affinity or fewest required data copy times, is selected as the target transmission channel. The data to be transmitted, Data, can be directly transferred from VRAM_0 to DRAM_1 via channel 1. Since the user buffer (user_buffer_v0) and temporary buffer (stg_buffer_v0) used for the data to be transmitted in the first node Rank0 have the same media type, and the user buffer (user_buffer_d1) and temporary buffer (stg_buffer_d1) used for the data to be transmitted in the second node Rank1 have the same media type, and the data copy operations between user buffer (user_buffer_v0) and temporary buffer (stg_buffer_v0), as well as between user buffer (user_buffer_d1) and temporary buffer (stg_buffer_d1), are extremely short and negligible, channel 1 achieves zero copying and has a short total data transmission time. Data transmission via channel 2 or channel 3 requires one additional data copy operation, and data copy operations between buffers of different media types are time-consuming. Therefore, channel 1 has the highest affinity and requires the fewest data copies. Channel 1 is scheduled as the target transmission channel for the data to be transmitted, thereby improving data transmission efficiency.

[0123] For example, step S540 above may further include: in response to the mismatch between the address space where the first level buffer pointed to by the first pointer is located and the data transmission capability of the first node, determining the target transmission channel as: sending the data to be transmitted from the first level buffer pointed to by the first pointer to the second level buffer via the second level buffer corresponding to the first processor in the first node to the second level buffer pointed to by the second pointer, wherein the first processor is different from the processor corresponding to the address space where the first level buffer pointed to by the first pointer is located.

[0124] For example, in response to the address space of the first-level buffer pointed to by the first pointer being the address space of a first-type processor, the "first processor" in the first node is a processor different from the first-type processor (e.g., a second-type processor).

[0125] Figure 6A and Figure 6B Schematic diagrams of another data transmission process provided by at least one embodiment of the present disclosure are shown.

[0126] like Figure 6A As shown, if the data transfer capability of the first node Rank0 includes allowing inter-node data transfer based on the address space of the first type of processor (i.e., the CPU's DRAM_0) but not allowing inter-node data transfer based on the address space of the second type of processor (i.e., the GPU's VRAM_0), then it is determined that the first-level buffer (i.e., the user buffer user_buffer_v0) pointed to by the first pointer does not match the data transfer capability of the first node Rank0. In this scenario, the first node Rank0 automatically triggers an implicit copy operation. That is, the data to be transferred, Data, is copied from the first-level buffer (user buffer user_buffer_d0) pointed to by the first pointer to the corresponding second-level buffer (i.e., temporary buffer stg_buffer_d0) of the processor in the first node Rank0 that is different from the first processor (i.e., the second type of processor GPU) (i.e., the first type of processor CPU). Then, the data to be transferred, Data, is sent to the second-level buffer (i.e., temporary buffer stg_buffer_d1) pointed to by the second pointer via the temporary buffer stg_buffer_d0. Finally, the data to be transferred is transferred to the first-level buffer (i.e., user buffer user_buffer_d1) of the second node Rank1 via the temporary buffer stg_buffer_d1.

[0127] like Figure 6BAs shown, if the data transfer capability of the first node Rank0 includes allowing inter-node data transfer based on the address space of the second type of processor (i.e., the VRAM_0 of the GPU), then it is determined that the first-level buffer pointed to by the first pointer (i.e., the user buffer user_buffer_v0) matches the data transfer capability of the first node Rank0. If the second level buffer pointed to by the second pointer belongs to the address space of the second type of processor in the second node Rank1 (i.e., the second level buffer pointed to by the second pointer is the temporary buffer stg_buffer_v1), the first node Rank0 copies the data to be transmitted from the first level buffer pointed to by the first pointer (i.e., the user buffer user_buffer_v0) to the temporary buffer stg_buffer_v0. The first node Rank0 then transmits the data to be transmitted from the temporary buffer stg_buffer_v0 to the second level buffer pointed to by the second pointer (i.e., the temporary buffer stg_buffer_v1). Finally, the second node Rank1 transmits the received data to be transmitted from the temporary buffer stg_buffer_v1 to the first level buffer in the second node used for the data to be transmitted (i.e., the user buffer user_buffer_d1).

[0128] For example, after completing the transmission of data, the first node returns send handles to monitor and manage the status of the transmitted data, while the second node returns receive handles to report the data reception status to the first node (e.g., acknowledging successful data reception, acknowledging partial data reception, or indicating an error during data reception). Here, a "handle" is an abstract resource identifier that allows both parties (the first and second nodes) to perform further operations or queries on the data transmission or associated resources.

[0129] The communication control method provided in at least one embodiment of this disclosure can be integrated into existing training and inference frameworks, and provides a unified hybrid communication encapsulation for deep learning frameworks, effectively supporting direct communication between CPU and GPU.

[0130] It should also be noted that the execution order of the various steps of the communication control method in the various embodiments of this disclosure is not limited. Although the execution process of each step has been described in a specific order above, this does not constitute a limitation on the embodiments of this disclosure. The various steps in the communication control method can be executed serially or in parallel, which can be determined according to actual needs.

[0131] For example, compared to the above description, the communication control method provided in at least one embodiment of this disclosure may include more or fewer steps, and the embodiments of this disclosure do not limit this.

[0132] At least one embodiment of this disclosure also provides a communication system corresponding to the communication control method. Since the principle of the communication system in this disclosure is similar to the communication control method described above, the implementation of the communication system can also be referred to the implementation of the method in the following description, and repeated details will not be repeated.

[0133] Figure 7A A schematic diagram of a communication system provided in at least one embodiment of the present disclosure is shown.

[0134] like Figure 7A As shown, the communication system 700 may include multiple nodes (e.g., node Rank0, node Rank1), wherein each node includes a communication interface, a communication controller, and at least two types of processors, including a first type of processor and a second type of processor. For example, node Rank0 (i.e., the first node) includes a communication interface 701, a communication controller 702, a first type of processor 703 (e.g., CPU), and a second type of processor 704 (e.g., GPU). For example, node Rank1 (i.e., the second node) includes a communication interface 711, a communication controller 712, a first type of processor 713 (e.g., CPU), and a second type of processor 714 (e.g., GPU). It should be noted that... Figure 7A This is merely one example; the communication system 700 may also include more nodes besides node Rank0 and node Rank1, such as node Rank2. This disclosure does not limit the scope of the embodiments. The structures of other nodes are similar to those of node Rank0, and will not be described further here.

[0135] For example, in response to node Rank0 triggering a data transmission operation with node Rank1 (i.e., the second node), node Rank0's communication interface 701 is configured to acquire a first pointer and a second pointer to the data to be transmitted (Data), and based on the first pointer and node Rank0's data transmission capability, determine the second-level buffer in node Rank0 for the data to be transmitted (Data). The first pointer points to the first-level buffer in node Rank0 for the data to be transmitted (Data), and the second pointer points to the second-level buffer in node Rank1 for the data to be transmitted (Data). The second pointer is sent by node Rank1 to the identification and negotiation module encapsulated in node Rank0's communication interface 701 via an established temporary communication link (e.g., a TCP link). Figure 7A (Not shown in the image).

[0136] For example, the communication controller 702 of node Rank0 is configured to: in response to the fact that both the first-level buffer and the second-level buffer in node Rank0 belong to the address space of the second-type processor 704 (e.g., the VRAM of the GPU), and both the first-level buffer and the second-level buffer in node Rank1 belong to the address space of the first-type processor 713 (e.g., the DRAM of the CPU), determine that the target transmission channel for transmitting the data to be transmitted between node Rank0 and node Rank1 is: a direct transmission channel between the address space of the second-type processor 704 of node Rank0 and the address space of the first-type processor 713 of node Rank1.

[0137] For example, the communication interface of node Rank0, which is the sending end, can be represented as send(void* send_user_ptr, size_t size), where send_user_ptr represents the storage address of the data to be transmitted in node Rank0; the communication interface of node Rank1, which is the receiving end, can be represented as receive(void* receive_user_ptr, size_tsize), where receive_user_ptr represents the storage (or writing) address of the data to be transmitted in node Rank1 after node Rank1 receives the data to be transmitted.

[0138] In the embodiments of this disclosure, by providing a unified communication interface for each node, the differences in underlying memory types can be masked. That is, the application only needs to send or receive data through simple function calls, without needing to know whether the data is located in CPU memory (e.g., DRAM), GPU video memory (e.g., VRAM), or other types of memory. The communication system automatically identifies the memory type at the underlying level and selects the optimal transmission path (such as direct RDMA, GDR, or via a temporary buffer) to ensure efficient data transmission.

[0139] In addition, node Rank0 can communicate with multiple nodes such as node Rank1 and node Rank2 at the same time. The specific operation is similar to the communication (or data transmission) between node Rank0 and node Rank1, which will not be described in detail here.

[0140] For example, data transfer capability indicates which type of processor address space in the corresponding node is allowed for inter-node data transfer, or which type of processor address space in the corresponding node is not allowed for inter-node data transfer.

[0141] For example, after the communication interface 701 acquires the first pointer and the second pointer of the data to be transmitted (Data), the identification and negotiation module encapsulated in the communication interface 701 is configured to: identify the type of the address space where the first-level buffer pointed to by the first pointer is located; and determine whether the type of the address space where the first-level buffer pointed to by the first pointer is located matches the data transmission capability of node Rank0. For example, the identification and negotiation module is also configured to: in response to the type of the processor's address space allowed by the data transmission capability of node Rank0, including the type of the address space where the first-level buffer pointed to by the first pointer is located, determine that the address space where the first-level buffer pointed to by the first pointer is located matches the data transmission capability of the first node.

[0142] For example, the identification and negotiation module is further configured to: in response to the first-level buffer pointed to by the first pointer belonging to the address space of the second-type processor 704, and the address space of the processor allowed by the data transmission capability of node Rank0 including the address space of the second-type processor 704, determine that the second-level buffer in node Rank0 used for the data to be transmitted Data is a second-level buffer in the address space of the second-type processor 704; or, in response to the first-level buffer pointed to by the first pointer belonging to the address space of the second-type processor 704, and the address space of the processor allowed by the data transmission capability of node Rank0 not including the address space of the second-type processor 704, determine that the second-level buffer in node Rank0 used for the data to be transmitted Data is a second-level buffer in the address space of the first-type processor 703.

[0143] For example, the communication controller 702 of node Rank0 is also configured to determine the target transmission channel between node Rank0 and node Rank1 based on the first pointer, the second pointer, and the data transmission capability of node Rank0.

[0144] For example, when determining the target transmission channel between node Rank0 and node Rank1 based on the first pointer, the second pointer, and the data transmission capability of node Rank0, the communication controller 702 is specifically configured to: determine multiple transmission channels based on the first pointer and the second pointer; and determine the target transmission channel from the multiple transmission channels based on the data transmission capability of node Rank0.

[0145] For example, when determining a target transmission channel from multiple transmission channels based on the data transmission capability of node Rank0, the communication controller 702 is specifically configured to: determine candidate transmission channels that meet the data transmission capability of node Rank0 from multiple transmission channels; and select from the candidate transmission channels at least one of the following: the channel that satisfies the highest affinity or the minimum number of data copies required, as the target transmission channel.

[0146] For example, when determining the target transmission channel between node Rank0 and node Rank1 based on the first pointer, the second pointer, and the data transmission capability of node Rank0, the communication controller 702 is specifically configured to: in response to a mismatch between the address space of the first-level buffer pointed to by the first pointer and the data transmission capability of node Rank0, determine the target transmission channel as follows: send the data to be transmitted from the first-level buffer pointed to by the first pointer through the second-level buffer corresponding to the first processor in node Rank0 to the second-level buffer pointed to by the pointer Rank1, wherein the first processor is different from the processor corresponding to the address space of the first-level buffer pointed to by the pointer Rank0.

[0147] For example, the data transmission capability of node Rank0 is determined based on the communication routing table. Each node in the communication system 700 also includes a routing table creation module. The routing table creation module of node Rank0 is configured to: detect the local address space in node Rank0 that allows inter-node data transmission, and register the address space information of the second-level buffer in the local address space for node Rank0; obtain the address space information of the second-level buffer in the remote address space that allows inter-node data transmission registered for node Rank1; and create a communication routing table based on the address space information registered for nodes Rank0 and Rank1.

[0148] For example, the address space information in the communication routing table includes at least the address space type of the second-level buffer. The address space type includes a first type and a second type. The second-level buffer of the first type belongs to the address space of the first type processor in the corresponding node, and the second-level buffer of the second type belongs to the address space of the second type processor in the corresponding node.

[0149] For example, when determining the data transmission capability of node Rank0 based on the communication routing table, the identification and negotiation module encapsulated in the communication interface 701 is configured to: in response to the address space type of the second-level buffer in the local address space including a first type and / or a second type, determine the data transmission capability of node Rank0 to include allowing inter-node data transmission based on the address space of the corresponding type of processor.

[0150] For example, the address space information in the communication routing table also includes the node identifier, starting address, and buffer capacity corresponding to each registered second-level buffer.

[0151] Figure 7B A flowchart illustrating the communication process between nodes provided in at least one embodiment of this disclosure is shown.

[0152] like Figure 7BAs shown, nodes Rank0 and Rank1 perform communication group creation, communication group connection, and address space information registration operations, respectively. For example, when node Rank0 performs address space information registration, it detects the second-level buffer of its local address space that allows inter-node data transmission and registers the address space information of the detected second-level buffer on its network interface card (NIC) (e.g., refer to step S310 above). When node Rank1 performs address space information registration, it detects the second-level buffer of its local address space that allows inter-node data transmission and registers the address space information of the detected second-level buffer on its NIC (e.g., refer to step S310 above). After completing the address space information registration, nodes Rank0 and Rank1 perform address space information synchronization. For example, nodes Rank0 and Rank1 can exchange information by establishing a temporary communication link, obtain the address space information of the second-level buffer registered by the other party, and then create their respective communication routing tables (e.g., refer to step S330 above).

[0153] like Figure 7BAs shown, when the communication mode is send / recv mode, both node Rank0 (as the sender) and node Rank1 (as the receiver) trigger data transmission operations. That is, node Rank0 initiates a data sending operation, and node Rank1 initiates a data receiving operation. During the address space identification and negotiation phase, node Rank0 determines the "first pointer" of the data to be transmitted (i.e., the address of the user buffer used for the data to be transmitted in node Rank0, send_user_ptr), and determines the address of the "second-level buffer" (i.e., the address of the temporary buffer, send_stg_ptr) used for the data to be transmitted in node Rank0 based on the "first pointer" and the address space information of the second-level buffer registered locally in node Rank0. Node Rank1 determines the address of the "first-level buffer" of the data to be transmitted (i.e., the address of the user buffer used for the data to be transmitted in node Rank1, receive_user_ptr), and determines the "second pointer" (i.e., the address of the "second-level buffer" (temporary buffer) used for the data to be transmitted in node Rank1) based on the user buffer receive_user_ptr and the address space information of the second-level buffer registered locally in node Rank1. Then, nodes Rank0 and Rank1 exchange the address information of the "second-level buffer" determined by both parties for data transmission between nodes through the established temporary communication link. For example, node Rank0 sends send_stg_ptr to node Rank1, and node Rank1 sends receive_stg_ptr to node Rank0.

[0154] After completing the address space identification and negotiation phase, node Rank0 determines the address space type registered locally by node Rank0 and the address space type registered remotely by node Rank1 from the communication routing table. It then selects the target transmission channel for data transmission based on send_stg_ptr and receive_stg_ptr. The specific selection method can be found in steps S540, S540A, and S540B above. Figure 6A and Figure 6B The corresponding examples will not be repeated here. After the data transmission operation is completed, node Rank0 returns the send handle, and node Rank1 returns the receive handle.

[0155] Figure 8 A schematic diagram of the structure of an electronic device provided in at least one embodiment of the present disclosure is shown. Figure 8 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0156] The electronic devices in at least one embodiment of this disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (PADs), portable multimedia players (PMPs), in-vehicle terminals (e.g., in-vehicle navigation terminals), wearable electronic devices, and fixed terminals such as digital TVs and desktop computers. Figure 8 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0157] The electronic device includes at least one processor and a memory. The processor may be referred to as processing device 801 as described below, and the memory may include at least one of read-only memory (ROM), random access memory (RAM), and storage device 808 as described below. The memory is used to store programs for performing the methods described in the various method embodiments above; the processor is configured to execute the programs stored in the memory. The processor may include a central processing unit (CPU) or other forms of processing unit having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0158] For example, such as Figure 8 As shown, in some examples, electronic device 800 includes a processing unit 801 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) or loaded from storage device 808 into random access memory (RAM). Various programs and data required for the operation of the computer system are also stored in RAM 803. Processing unit 801, ROM 802, and RAM 803 are connected via bus 804. Input / output (I / O) interfaces are also connected to bus 804.

[0159] For example, the following components can be connected to I / O interface 805: input devices 806 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 807 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 808 including, for example, magnetic tapes, hard disks, etc.; and communication devices 809 including, for example, network interface cards such as LAN cards, modems, etc. Communication device 809 allows electronic device 800 to communicate wirelessly or wiredly with other devices to exchange data and perform communication processing via networks such as the Internet. Although Figure 8An electronic device 800 including various devices is shown; however, it should be understood that implementation or inclusion of all shown devices is not required. More or fewer devices may be implemented or included alternatively.

[0160] For example, the electronic device 800 may further include a peripheral interface (not shown in the figure). This peripheral interface can be various types of interfaces, such as a USB interface, a Lightning interface, etc. The communication device 809 can communicate with networks and other devices wirelessly.

[0161] For example, according to at least one embodiment of this disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, at least one embodiment of this disclosure includes a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device 809, or installed from a storage device 808, or installed from a ROM 802. When the computer program is executed by the processing device 801, it performs the functions defined in the methods of at least one embodiment of this disclosure.

[0162] It should be noted that the computer-readable medium described above in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In at least one embodiment of this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In at least one embodiment of this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, radio frequency (RF), etc., or any suitable combination thereof.

[0163] The aforementioned computer-readable medium may be included in the aforementioned electronic device 800; or it may exist independently and not assembled into the electronic device 800.

[0164] Embodiments of this disclosure also provide a non-transitory computer-readable storage medium storing computer-readable instructions thereon, which, when executed by a processor, cause the processor to perform steps of the communication control method according to this disclosure.

[0165] Figure 9 A schematic diagram of the structure of a non-transitory computer-readable storage medium provided in at least one embodiment of the present disclosure is shown.

[0166] For example, such as Figure 9 As shown, a non-transitory computer-readable storage medium 900 stores computer-readable instructions 901, which, when executed by at least one processor, perform one or more steps of the communication control method described above.

[0167] For example, the storage medium may include a memory card for a smartphone, a storage component for a tablet computer, a hard disk for a personal computer, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), flash memory, or any combination of the above storage media, or other suitable storage media.

[0168] Embodiments of this disclosure provide a communication control method, a communication system, an electronic device, and a storage medium. Utilizing the communication control method, communication system, electronic device, and storage medium provided by this disclosure, by identifying the pointer types of the first and second pointers of the data to be transmitted, as well as the data transmission capabilities of the first and second nodes, the target transmission channel for the data to be transmitted can be determined. This enables direct data transmission between heterogeneous nodes using different types of processors (e.g., direct data transmission between a first-type processor of the first node and a second-type processor of the second node), automatically scheduling efficient communication paths (data transmission channels), and avoiding the introduction of additional data copy operations, thereby improving communication efficiency. It can also support member nodes using both CPU and GPU address spaces simultaneously within the same communication group, reducing the complexity of cross-memory type communication and simplifying user programming complexity.

[0169] Although the present disclosure has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to the embodiments of the present disclosure, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present disclosure are within the scope of protection claimed by the present disclosure.

[0170] In addition to the illustrative examples described above, the following points also need to be noted:

[0171] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0172] (2) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0173] The above description is merely an exemplary embodiment of this disclosure and is not intended to limit the scope of protection of this disclosure, which is determined by the appended claims.

Claims

1. A communication control method, characterized in that, The communication control method is used for the first node in the communication group and includes: Obtain a first pointer and a second pointer for the data to be transmitted, wherein the first pointer points to a first-level buffer in the first node for the data to be transmitted, the communication group further includes a second node, and the second pointer points to a second-level buffer in the second node for the data to be transmitted, and each node in the communication group includes a first type processor and a second type processor; Based on the first pointer and the data transmission capability of the first node, a second-level buffer for the data to be transmitted is determined in the first node; In response to the fact that both the first-level buffer and the second-level buffer in the first node belong to the address space of the second type of processor, and both the first-level buffer and the second-level buffer in the second node belong to the address space of the first type of processor, the target transmission channel between the first node and the second node for transmitting the data to be transmitted is determined to be: a direct transmission channel between the address space of the second type of processor of the first node and the address space of the first type of processor of the second node.

2. The communication control method according to claim 1, characterized in that, The data transmission capability indicates which type of processor's address space is allowed for inter-node data transmission, or which type of processor's address space is not allowed for inter-node data transmission.

3. The communication control method according to claim 1 or 2, characterized in that, After acquiring the first and second pointers of the data to be transmitted, the communication control method further includes: Identify the type of the address space where the first-level buffer pointed to by the first pointer resides; Determine whether the type of the address space where the first level buffer pointed to by the first pointer is located matches the data transmission capability of the first node.

4. The communication control method according to claim 3, characterized in that, Determining whether the address space where the first-level buffer pointed to by the first pointer is located matches the data transmission capability of the first node includes: The type of processor address space allowed by the data transmission capability of the first node includes the type of address space where the first level buffer pointed to by the first pointer is located, and it is determined that the address space where the first level buffer pointed to by the first pointer is located matches the data transmission capability of the first node.

5. The communication control method according to claim 1 or 2, characterized in that, The step of determining the second-level buffer in the first node for the data to be transmitted based on the first pointer and the data transmission capability of the first node includes: In response to the fact that the first-level buffer pointed to by the first pointer belongs to the address space of the second type of processor, and the address space of the processor allowed by the data transmission capability of the first node includes the address space of the second type of processor, the second-level buffer in the first node used for the data to be transmitted is determined to be the second-level buffer in the address space of the second type of processor; or In response to the fact that the first-level buffer pointed to by the first pointer belongs to the address space of the second type of processor, and the address space of the processor allowed by the data transmission capability of the first node does not include the address space of the second type of processor, the second-level buffer in the first node used for the data to be transmitted is determined to be the second-level buffer in the address space of the first type of processor.

6. The communication control method according to claim 1 or 2, characterized in that, The communication control method further includes: Based on the first pointer, the second pointer, and the data transmission capability of the first node, the target transmission channel between the first node and the second node is determined.

7. The communication control method according to claim 6, characterized in that, Determining the target transmission channel between the first node and the second node based on the first pointer, the second pointer, and the data transmission capability of the first node includes: Based on the first pointer and the second pointer, multiple transmission channels are determined; and Based on the data transmission capability of the first node, the target transmission channel is determined from the plurality of transmission channels.

8. The communication control method according to claim 7, characterized in that, The step of determining the target transmission channel from the plurality of transmission channels based on the data transmission capability of the first node includes: From the plurality of transmission channels, candidate transmission channels that meet the data transmission capabilities of the first node are determined; and From the candidate transmission channels, select the channel that satisfies at least one of the following: highest affinity or fewest required data copy times, as the target transmission channel.

9. The communication control method according to claim 6, characterized in that, Based on the first pointer, the second pointer, and the data transmission capability of the first node, the target transmission channel between the first node and the second node is determined, including: In response to a mismatch between the address space of the first-level buffer pointed to by the first pointer and the data transmission capability of the first node, the target transmission channel is determined to be: sending the data to be transmitted from the first-level buffer pointed to by the first pointer through the second-level buffer corresponding to the first processor in the first node to the second-level buffer pointed to by the second pointer. The first processor is different from the processor in the address space where the first level buffer pointed to by the first pointer is located.

10. The communication control method according to claim 1 or 2, characterized in that, The communication control method further includes: The data transmission capability of the first node is determined based on the communication routing table. The steps for creating the communication routing table include: Detect the local address space in the first node that allows inter-node data transmission, and register the address space information of the second-level buffer in the local address space for the first node; Obtain the address space information of the second-level buffer in the remote address space registered for the second node, which allows inter-node data transfer; and The communication routing table is created based on the address space information registered for the first node and the second node.

11. The communication control method according to claim 10, characterized in that, The address space information in the communication routing table includes at least the address space type of the second-level buffer. The address space type includes a first type and a second type. The second-level buffer of the first type belongs to the address space of the first type of processor in the corresponding node, and the second-level buffer of the second type belongs to the address space of the second type of processor in the corresponding node.

12. The communication control method according to claim 11, characterized in that, Determining the data transmission capability of the first node based on the communication routing table includes: In response to the address space type of the second-level buffer in the local address space including the first type and / or the second type, determining the data transmission capability of the first node includes allowing inter-node data transmission based on the address space of the corresponding type of processor.

13. The communication control method according to claim 11, characterized in that, The address space information in the communication routing table also includes the node identifier, starting address, and buffer capacity corresponding to each registered second-level buffer.

14. A communication system, characterized in that, The communication system includes multiple nodes, each node including a communication interface, a communication controller, and at least two types of processors, the at least two types of processors including a first type of processor and a second type of processor, and the multiple nodes including a first node and a second node. The communication interface of the first node is configured to: in response to the first node triggering a data transmission operation with the second node, acquire a first pointer and a second pointer to the data to be transmitted, and determine a second-level buffer in the first node for the data to be transmitted based on the first pointer and the data transmission capability of the first node, wherein the first pointer points to the first-level buffer in the first node for the data to be transmitted, and the second pointer points to the second-level buffer in the second node for the data to be transmitted; and The communication controller of the first node is configured to: in response to the fact that both the first-level buffer and the second-level buffer in the first node belong to the address space of the second type of processor, and both the first-level buffer and the second-level buffer in the second node belong to the address space of the first type of processor, determine the target transmission channel between the first node and the second node for transmitting the data to be transmitted as: a direct transmission channel between the address space of the second type of processor of the first node and the address space of the first type of processor of the second node.

15. An electronic device, characterized in that, The electronic device includes at least one processor and at least one memory, wherein the at least one memory stores program code that, when executed by the at least one processor, causes the at least one processor to perform the communication control method according to any one of claims 1-13.

16. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer-readable instructions, which, when executed by at least one processor, cause the at least one processor to perform the communication control method according to any one of claims 1-13.

Images