Group communication method and apparatus, electronic device, storage medium, and program product

Abstract

The present disclosure provides a collection communication method, device, electronic equipment, storage medium and program product, relating to the technical field of artificial intelligence and computer, the collection communication method comprises: determining the collection communication operation that a communication group should execute according to a collection communication request;Wherein, the communication group comprises a plurality of graphic processing chips (GPU);According to the type of the collection communication operation and the memory residence of the corresponding input / output tensor, a matching target graph template is selected from a plurality of alternative graph templates;The target graph template is instantiated to obtain a graph execution instance;Running the graph execution instance obtains the collection communication result corresponding to the collection communication operation.The method can reduce the repeated task organization overhead of each round of collection communication operation.

Description

Technical Field

[0001] The solutions provided in this disclosure relate to the fields of artificial intelligence and computer technology, specifically to a combination of communication methods, devices, electronic devices, storage media, and program products. Background Technology

[0002] In the fields of distributed training and high-performance computing, multiple GPUs (Graphics Processing Units) often need to perform collective communication operations. Collective communication operations are used for data synchronization, collection, distribution, and reduction operations among GPUs in a communication group. They are crucial for achieving efficient distributed training or inference and have a significant impact on training throughput and single-step latency. Summary of the Invention

[0003] This disclosure presents a collection of communication methods, devices, electronic devices, computer-readable storage media, and computer program products.

[0004] In a first aspect, embodiments of this disclosure provide a set communication method, comprising: determining a set communication operation to be performed by a communication group based on a set communication request; wherein the communication group includes multiple graphics processing chips (GPUs); selecting a matching target graph template from multiple candidate graph templates based on the type of the set communication operation and the memory residency status of the corresponding input / output tensors; instantiating the target graph template to obtain a graph execution instance; and running the graph execution instance to obtain a set communication result corresponding to the set communication operation.

[0005] In some embodiments, the candidate graph templates include pre-created graph templates corresponding to the type of collective communication operation and different execution modes. Based on the type of collective communication operation and the memory residency status of the corresponding input / output tensors, a matching target graph template is selected from multiple candidate graph templates. This includes: determining the execution mode of the collective communication operation based on the memory residency status of the input / output tensors; and selecting a target graph template from the candidate graph templates that matches the type and execution mode of the collective communication operation. The execution modes include push mode and pull mode. Push mode refers to the sender actively sending data to the receiver, while pull mode refers to the receiver actively requesting data from the sender. Thus, a unified graph execution framework is achieved by supporting graph templates for different types of collective communication operations. Furthermore, the candidate graph templates can support selecting either push mode or pull mode to execute collective communication operations based on conditions such as the memory residency status of the input / output tensors, target layout, intermediate buffer requirements, and whether in-place writes are allowed, thereby enhancing the adaptability and scalability of the unified graph execution framework.

[0006] In some embodiments, input / output tensors can be stored in each GPU's dedicated video memory region and / or shared symmetric memory, with each GPU having access to the shared symmetric memory. Based on the memory residency of the input / output tensors, the execution mode of the set communication operation is determined, including: when the input tensor is located in shared symmetric memory and the output tensor is not, the execution mode is determined to be pull mode; when the output tensor is located in shared symmetric memory and the input tensor is not, the execution mode is determined to be push mode. This allows for flexible adaptation to tensor allocation scenarios combining shared symmetric memory and cache allocators, improving the usability of the set communication method in real-world framework environments.

[0007] In some embodiments, the target graph template includes a root node and node paths extending from the root node. Each node path includes multiple sequentially linked child nodes and dependencies between these child nodes. Instantiating the target graph template to obtain a graph execution instance involves: determining the root node for initiating a set communication operation; for each GPU, determining the child nodes for executing the set communication operation, and determining the dependencies between child nodes based on their execution order, constructing multiple node paths between the GPU and each GPU in the communication group; and instantiating the root node and each node path to generate the graph execution instance. By pre-constructing and instantiating the graph template, only the graph node parameters are updated at runtime, and the graph execution instance is reused, avoiding the overhead of repeatedly constructing the copy operation set and synchronizing batches in each round. This is particularly suitable for high-frequency set communication scenarios.

[0008] In some embodiments, the multiple child nodes in the node path sequentially include: a waiting node, used to wait for the collective communication operation to enter the ready state; an execution node, used to execute the collective communication operation; a completion write node, used to write the completion signal of the collective communication operation; and a completion waiting node, used to wait for the collective communication operation to enter the completion state. Compared to existing synchronization control, which typically revolves around the entire round of collective communication or batch replication operation, by explicitly modeling the execution relationship of collective communication as a graph node dependency of each node's ready state, execution state, and completion state, fine-grained control over the progression order of different nodes and different stages can be achieved.

[0009] In some embodiments, the set communication operation includes multiple rounds; running a graph execution instance to obtain the set communication result corresponding to the set communication operation includes: updating the node parameters in the graph execution instance according to the round identifier of the current round when executing the set communication operation in each round; wherein, the round identifier is used to indicate the execution round of the set communication operation; starting the graph execution instance, executing the set communication operation in the current round, and obtaining the set communication result corresponding to the set communication operation in the current round. By generating the round identifier of the current round, during the execution of the graph instance in each round, each waiting node and each completed node are synchronized around the round identifier, thereby realizing the synchronization of graph instance reuse across multiple rounds.

[0010] In some embodiments, updating node parameters in the graph execution instance based on the round identifier of the current round includes: updating data copy parameters of the execution node based on the round identifier of the current round; wherein, the data copy parameters are used to execute the data copy task included in the set communication operation of the current round, and the data copy parameters include the source address, destination address, and data length of the target data; the target data includes the data to be sent and / or the data to be received in the data copy task; the source address includes the address of the data to be sent, the destination address includes the address of the data to be received, and the data length includes the byte length and / or the number of elements of the data to be sent or received. By pre-building and instantiating the graph template, only the node parameters in the graph execution instance need to be updated at runtime, reducing the repetitive control plane overhead of organizing data copy tasks in each round of set communication operation and reducing the continuous occupation of GPU computing resources.

[0011] In some embodiments, updating the node parameters in the graph execution instance according to the round identifier of the current round further includes: updating the waiting signal parameters of waiting nodes and the completion signal parameters of completed waiting nodes to the target values ​​corresponding to the round identifier, respectively, according to the round identifier of the current round; wherein, the target value of the waiting signal parameter is used to characterize the preparation state of the collective communication operation in the current round; and the target value of the completion signal parameter is used to characterize the completion state of the collective communication operation in the current round. Thus, by using the round identifier as the target value for each waiting node and each completed node, signal synchronization can be achieved when graph instances are reused across multiple rounds, without needing to reset the entire signal area to its initial value in each round.

[0012] In some embodiments, a synchronization signal region corresponding to the collective communication operation is provided in the shared symmetric memory. The synchronization signal region includes a wait signal region with multiple wait signal slots and a completion signal region with multiple completion signal slots. Each GPU has independent wait signal slots and completion signal slots in the synchronization signal region, wherein the wait signal slots are used to store wait signal parameters, and the completion signal slots are used to store completion signal parameters. Compared with the existing shared synchronization region approach, by setting up independent synchronization signal regions, the synchronization states of different collective communication operations are isolated from each other, reducing the risk of control plane reset operations and synchronization pollution.

[0013] In some embodiments, when the type of the collection communication operation is global collection collection communication, a graph execution instance is started, the collection communication operation of the current round is executed, and the collection communication result corresponding to the collection communication operation of the current round is obtained. This includes: for each node path, the following steps are executed: a graph execution instance is started, the root node is activated, and the process jumps to the waiting node; when the preparation state is complete, the target value corresponding to the round identifier is written to the waiting signal parameter; in response to the waiting signal parameter reaching the target value corresponding to the round identifier, the process jumps to the execution node and executes the data transmission operation corresponding to the global collection collection communication operation; in response to the completion of the data transmission operation, the process jumps to the completion write node and writes the target value corresponding to the round identifier to the completion signal parameter; in response to the completion signal parameter reaching the target value corresponding to the round identifier, the node path is determined to be completed, and the collection communication result corresponding to the global collection collection communication operation is obtained. In this way, by unifying the execution control, runtime parameter update, signal synchronization, and multi-round multiplexing mechanism of collection communication, a unified graph execution framework for high-frequency collection communication is formed, enabling efficient execution of global collection high-frequency communication operators in a single-machine multi-accelerator environment that supports symmetric memory.

[0014] In some embodiments, when the type of set communication operation is reduction-based distributed set communication, a graph execution instance is started, the set communication operation of the current round is executed, and the set communication result corresponding to the set communication operation of the current round is obtained. This includes: for each node path, the following steps are performed: a graph execution instance is started, the root node is activated, and the process jumps to a waiting node; when the preparation state is complete, the target value corresponding to the round identifier is written to the waiting signal parameter; in response to the waiting signal parameter reaching the target value corresponding to the round identifier, the process jumps to an execution node and executes the data operation corresponding to the reduction-based distributed set communication operation; in response to the completion of the data operation, the process jumps to a completion writing node and writes the target value corresponding to the round identifier to the completion signal parameter; in response to the completion signal parameter reaching the target value corresponding to the round identifier, the node path is determined to be completed, and the set communication result corresponding to the reduction-based distributed set communication operation is obtained. In this way, by incorporating reduction execution, waiting, and completion notification into a unified graph execution framework, reduction can be completed in a unified dependency chain through reduction execution nodes, thereby achieving efficient execution of reduction-based distributed high-frequency communication operators.

[0015] In some embodiments, when the input tensor is located in shared symmetric memory, the source address is determined by the base address, address offset information, and fragment offset information of the data to be sent in the shared symmetric memory; when the output tensor is located in shared symmetric memory, the destination address is determined by the base address, address offset information, and fragment offset information of the data to be received in the shared symmetric memory. Thus, for scenarios supporting shared symmetric memory, when updating node parameters in a graph execution instance, the actual access address is determined by simultaneously considering the shared symmetric memory base address, the tensor's offset relative to the base address, and the data fragment offset corresponding to the currently participating node, providing a foundation for low-overhead direct access and data transfer between devices.

[0016] In some embodiments, when the input tensor is located in the video memory region of each GPU, the source address is determined by the data pointer of the data to be sent in the video memory region and the fragment offset information; when the output tensor is located in the video memory region of each GPU, the destination address is determined by the data pointer of the data to be received in the video memory region and the fragment offset information. This adapts to tensor allocation scenarios combining symmetrical memory and cache allocators, allowing the collection communication method of this disclosure to be directly used in upper-level tensor allocation systems without forcing tensors to exclusively occupy the entire symmetrical memory block, thereby improving the usability of the collection communication method in real-world framework environments.

[0017] Secondly, embodiments of this disclosure provide a collective communication device, comprising: a communication control module configured to determine a collective communication operation to be performed by a communication group based on a collective communication request; wherein the communication group includes multiple graphics processing units (GPUs); a graph template management module configured to select a matching target graph template from multiple candidate graph templates based on the type of the collective communication operation and the memory residency status of the corresponding input / output tensors; instantiate the target graph template to obtain a graph execution instance; and an execution node processing module configured to run the graph execution instance to obtain a collective communication result corresponding to the collective communication operation.

[0018] Thirdly, embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more computer programs stored thereon, wherein when the one or more computer programs are executed by the one or more processors, the one or more processors implement a collective communication method as mentioned in any of the first aspects.

[0019] Fourthly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements a collection communication method as mentioned in any of the first aspects.

[0020] Fifthly, embodiments of this disclosure provide a computer program product including a computer program that, when executed by a processor, implements a collection communication method as mentioned in any of the first aspects.

[0021] The collection communication method, apparatus, and electronic device provided in the embodiments of this disclosure realize a unified graph execution framework by supporting graph templates for different types of collection communication operations. By generating graph execution instances that can be repeatedly started, the overhead of organizing repetitive tasks in each round of collection communication can be reduced. Furthermore, the alternative graph templates can support different data flow organization methods under the unified graph execution framework, enhancing the adaptability and scalability of the framework.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0023] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart illustrating an embodiment of the collective communication method according to the present disclosure; Figure 2 This is a schematic diagram of a collection communication method according to an exemplary embodiment of the present disclosure; Figure 3 This is a schematic diagram of the data flow and signal synchronization process of a globally collected type of collection communication in a push mode according to an exemplary embodiment of this disclosure; Figure 4 This is a schematic diagram of the data flow and signal synchronization process of a global collection type collection communication in a pull mode according to an exemplary embodiment of this disclosure; Figure 5 This is a schematic diagram of the data flow and signal synchronization process of a distributed type of aggregate communication in a push mode according to an exemplary embodiment of this disclosure; Figure 6 This is a schematic diagram of the data flow and signal synchronization process of a collection communication of a protocol-distributed type under the pull mode according to an exemplary embodiment of this disclosure; Figure 7 This is a schematic diagram of the structure of the synchronization signal region according to an exemplary embodiment of the present disclosure; Figure 8 This is a schematic diagram of an address mapping method for shared symmetric memory according to an exemplary embodiment of this disclosure; Figure 9a and Figure 9b These are flowcharts illustrating the initialization and runtime phases of an embodiment of the collective communication method disclosed herein. Figure 10 This is a schematic diagram of a structure of a collective communication device according to an embodiment of the present disclosure; Figure 11 This is a schematic diagram of another embodiment of the collective communication device according to the present disclosure; Figure 12 This is a flowchart illustrating a collection communication method in a specific application scenario based on this disclosure; Figure 13 This is a schematic diagram of the structure of an electronic device suitable for implementing embodiments of the present disclosure. Detailed Implementation

[0024] The present disclosure will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0025] As the scale and computational demands of AI (Artificial Intelligence) models continue to grow, a distributed deployment architecture is typically adopted to distribute model training and inference tasks across multiple accelerators for parallel execution. Accelerators refer to various hardware devices used to process data and perform computational tasks, such as GPUs or TPUs (Tensor Processing Units). In distributed training or parallel computing, ensemble communication operations are widely used to synchronize model parameters, data shards, and intermediate computation results across multiple GPUs, and are a core component for achieving efficient distributed training and parallel computing.

[0026] Collective communication refers to the specific collective communication operations performed by participating peers (nodes) in a communication group, which involve sending, receiving, copying, and synchronizing. The communication group comprises multiple participating nodes, each of which is a hardware device capable of independently executing computational tasks, such as a GPU or other computing device.

[0027] Common assembly communication operations can include All-Gather, Reduce-Scatter, All-Reduce, and All-to-All. For example, All-Gather is used to collect local data from each participating node and concatenate it into a complete result, while Reduce-Scatter is used to perform reduction on the corresponding data fragments from each participating node and then distribute them back to each participating node.

[0028] Currently, aggregated communication operations mainly include the following schemes: Scheme 1, submitting aggregated communication tasks based on the CPU (Central Processing Unit) or host-side runtime, and completing data exchange between devices through device drivers or communication libraries; Scheme 2, using device-side kernel functions, DMA (Direct Memory Access) transfer, or point-to-point copying to complete data transfer; Scheme 3, in reduction-type aggregated communication such as Reduce-Scatter, data arrival is usually completed first, then an additional reduction kernel function is started, and finally the result is written back. To achieve timing consistency among multiple participating nodes, existing schemes generally use global barriers, coarse-grained events, or stream synchronization to complete synchronization control.

[0029] However, some of the above solutions have at least one of the following problems: (1) The control plane of the aggregate communication operation is still mainly based on the temporary organization of copy operation sets at runtime. In each round of aggregate communication, it is necessary to repeatedly organize copy (data copy) tasks, synchronization tasks, and execute control logic, which involves a lot of repetitive work and has a large control plane overhead. For example, each round requires the temporary organization of a set of data copy tasks, including updating parameters such as source address, destination address, and data copy length, to perform local or inter-device data copying; and, through synchronization mechanisms such as waiting / notification, to ensure that all participating nodes achieve phase alignment.

[0030] (2) The synchronization between participating nodes in the communication group is more reflected at the batch level, with a coarse synchronization granularity, and cannot achieve fine-grained node-level synchronization or strict timing synchronization between participating nodes.

[0031] (3) Existing solutions are mainly geared towards data-transfer type collection communication operations, and lack implementations of collection communication operations with reduction semantics, such as Reduce-Scatter.

[0032] The collection communication scheme provided according to the embodiments of this disclosure at least partially solves the above-mentioned problems existing in the related art.

[0033] The present disclosure will now be described in detail with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the present disclosure can be combined with each other. Furthermore, unless explicitly limited or contradicted by the context, the specific steps included in the methods described in this disclosure are not limited to the order in which they are described, but can be performed in any order or in parallel.

[0034] Figure 1 A flow diagram of one embodiment of the collection communication method according to this disclosure is shown. See also Figure 1 The processing flow 100 of the collection communication method in this embodiment may include the following steps: Step 101: Determine the collective communication operation that the communication group should perform based on the collective communication request.

[0035] This step aims to have the executing entity of the collective communication method determine, based on the received collective communication request, the corresponding communication group and the collective communication operation to be performed by the communication group. The executing entity could be, for example, a server or a host.

[0036] In this embodiment, the types of group communication operations may include All-Gather, Reduce-Scatter, All-Reduce, or All-to-All.

[0037] A communication group is a logical unit composed of multiple computing devices with independent processing capabilities, such as GPUs, which collaborate to complete collective communication operations. Each computing device in the communication group is called a participating node.

[0038] In distributed training or parallel computing frameworks, upper-layer applications or frameworks initiate set communication operations by issuing set communication requests, and can encapsulate information such as the type of set communication operation, input / output tensors, and communication group identifier in the set communication request.

[0039] The ensemble communication method according to embodiments of this disclosure is applicable to distributed computing scenarios such as parameter-sharded parallel training, tensor-parallel training, and other scenarios requiring repeated ensemble communication operations. For example, it is applicable to single-machine multi-accelerator scenarios, such as inserting 4 or 8 GPU chips or graphics cards into a single server. Multiple GPUs can communicate with each other via direct links or channels, jointly completing training tasks or running a large AI model through parallel computing technology.

[0040] According to some embodiments of this disclosure, the aforementioned execution entity receives a collective communication request initiated by the upper-layer framework, and determines the communication group corresponding to the collective communication request and the collective communication operation to be performed by the communication group based on the type information of the collective communication operation, the communication group identifier, the input / output tensors, and other information passed in by the upper-layer framework. For example, the corresponding communication group can be determined based on the communication group identifier, and the type of collective communication operation can be determined based on the type information of the collective communication operation.

[0041] Step 102: Select the matching target graph template from multiple candidate graph templates based on the type of set communication operation and the memory residency status of the corresponding input / output tensors.

[0042] This step aims to have the aforementioned execution entity select a target graph template that matches the set communication operation from among several pre-created candidate graph templates, based on the type of set communication operation and the memory residency of the corresponding input / output tensors.

[0043] In group communication operations, it is usually necessary to send data from a specified location of the current participating node to a specified location of the destination node, or to receive data from a specified location of the source node to a specified location of the current participating node.

[0044] In this context, the input tensor for a set communication operation refers to the data provided by each participating node in the communication group, while the output tensor refers to the result data obtained by each participating node after the operation is completed.

[0045] For ease of understanding, the memory residency of input / output tensors corresponding to set communication operations will be explained below. According to some embodiments of this disclosure, for distributed computing scenarios with multiple accelerators on a single machine, each participating node in the communication group, such as each GPU, supports a symmetric memory mechanism. By sharing a unified virtual address space among multiple participating nodes, multiple participating nodes can access each other's corresponding buffers through a unified mapping relationship without CPU intervention, thus providing a foundation for low-overhead direct memory access across devices or processes.

[0046] In some implementations, some heterogeneous accelerators support an asynchronous copy engine, which can complete data transfer without starting the kernel, making it possible to perform data transfer between device memory or shared symmetric memory without continuously occupying the main computing core.

[0047] In practical applications, depending on the type of set communication operation, the input tensor corresponding to the set communication operation, such as the parameters to be calculated or the fragmented data, can be stored in the dedicated video memory area of ​​each GPU within the communication group, or it can be stored in the shared symmetric memory corresponding to the communication group. Similarly, the output tensor corresponding to the set communication operation, such as the calculated result, can also reside in the local video memory of each GPU or in the shared symmetric memory.

[0048] For example, in All-Gather type aggregate communication, the input tensor can be stored in the local video memory of each GPU; that is, each GPU holds a different copy of the tensor data in its own video memory. Alternatively, the input tensor can be stored in shared symmetric memory, allowing all GPUs to directly access the data in the shared symmetric memory and perform read and write operations. After the aggregate communication is complete, each GPU holds a copy of the global aggregation result in its own video memory. It should be understood that in practical applications, the memory residency of input / output tensors can vary depending on different application scenarios or different input / output requirements.

[0049] According to some embodiments of this disclosure, the alternative graph templates may include corresponding graph templates pre-created for each type of collection communication operation. Taking the All-Gather type as an example, a corresponding global collection graph template can be constructed; taking the Reduce-Scatter type as an example, a corresponding reduction-distributed execution graph template can be constructed.

[0050] In some embodiments, the graph template corresponding to each type of pre-created collection communication operation can support different data flow organization methods, such as whether the source side writes data to the target side, or the target side pulls data from the source side to the local machine.

[0051] In some implementations, when a collection communication request initiated by the upper-level framework is received, a graph template that matches the type of collection communication operation is first selected from the candidate graph templates. Then, the data flow organization method of the collection communication operation is determined based on at least one of the following conditions: the memory region where the input / output tensors are located, the target layout, the data size, the element type, the intermediate buffer requirements, and whether in-place writing is allowed. In this way, a graph template that matches the different data flow organization methods is selected.

[0052] Step 103: Instantiate the target graph template to obtain a graph execution instance.

[0053] This step aims to instantiate the constructed target graph template using the aforementioned execution entity, thereby obtaining an executable graph execution instance.

[0054] For applications in distributed computing scenarios, a graph template can contain multiple graph nodes and data flow relationships, which are used to describe how each accelerator synchronizes data at different stages of performing aggregate communication operations.

[0055] Specifically, the target graph template is an uninstantiated computation graph. Graph instantiation is a process of binding the graph template to specific resources to generate repeatable execution instances. By maintaining corresponding communicator objects for the communication group, and storing graph execution information of each participating node in the communication group based on the communicator objects, such as domain size information and shared symmetric memory information, graph instantiation is performed after the target graph template is constructed, resulting in repeatable graph execution instances.

[0056] Step 104: Run the graph execution instance to obtain the set communication result corresponding to the set communication operation.

[0057] This step aims to have the aforementioned execution entity perform set communication operations through the startup graph execution instance, and obtain the corresponding set communication results.

[0058] Specifically, the execution process of a graph execution instance involves the actual execution of data exchange and collaborative computation across multiple accelerators in a distributed cluster. This process involves instantiating a collection of communication operations—that is, binding them to specific devices, memory, and communicators—and performing them. Further implementation details regarding graph execution instances can be found in the detailed explanation below.

[0059] The collection communication method provided in the above embodiments of this disclosure realizes a unified graph execution framework by supporting graph templates that support different types of collection communication operations. The graph templates are instantiated to generate graph execution instances that can be repeatedly started, which can reduce the overhead of organizing repetitive tasks in each round of collection communication. Furthermore, the alternative graph templates can support different data flow organization methods under the unified graph execution framework. Compared with the data flow organization method in the prior art, which mainly uses the source side to write to the target side's receive buffer, the adaptability and scalability of the framework are enhanced.

[0060] To facilitate understanding and to highlight the effects of the collection communication method provided in this disclosure in practical application scenarios as much as possible, the collection communication method provided in this disclosure will be described in detail below.

[0061] Based on the above embodiments, some exemplary implementations of this disclosure also provide graph templates for collection communication. As described above, alternative graph templates may include pre-created graph templates corresponding to the type of collection communication operation.

[0062] In some embodiments, to match different execution strategies for collection communication operations, graph templates corresponding to different execution modes are created for each type of collection communication operation. Different execution modes for collection communication operations can include Push mode and Pull mode. For example, Push mode graph templates and Pull mode graph templates can be created separately. The main difference between Push mode and Pull mode lies in who initiates the data transmission—whether it is the data sender or the receiver. Push mode refers to the data sender (source node) actively sending data to other receivers. Pull mode refers to the data receiver (target node) actively requesting or pulling data from the sender.

[0063] In some implementations, the candidate graph templates may include pre-created graph templates corresponding to the type of collection communication operation and different execution modes. Upon receiving a collection communication request initiated by the upper-level framework, the execution mode of the collection communication operation can be determined based on the memory residency of the input / output tensors, and then a matching target graph template can be selected based on the type of collection communication operation and the execution mode. For example, if the type of collection communication operation is All-Gather and the execution mode is Push mode, then the corresponding All-Gather graph template for Push mode can be selected.

[0064] See Figure 2 The graph template according to embodiments of this disclosure includes multiple graph nodes and dependencies between them. Each graph node may include a root node and multiple child nodes.

[0065] In some implementations, the graph template may include a root node and multiple peer paths (node ​​paths) extending from the root node. Each peer path may include multiple sequentially linked child nodes and dependencies between these child nodes. The root node is used to set the synchronization signal for the current ensemble communication operation, indicating that the relevant data for the current ensemble communication operation is ready. Each child node in each peer path is used to perform data operations or synchronization operations related to that peer.

[0066] Figure 2 This is a schematic diagram of a collection communication method according to an exemplary embodiment of this disclosure. (In conjunction with...) Figure 2 As shown, this graph template may include a root node and multiple node paths, node path 1, node path 2, ..., node path M, where M is a positive integer. For example, each node path includes the following four child nodes in sequence: Waiting nodes are used to wait for the collection communication operation to enter the Ready state; Execution nodes are used to perform collection communication operations, such as data transmission or data reduction operations; Complete the write node, which is used for the Complete signal of the write collection communication operation; A completion waiting node is used to wait for a collection communication operation to enter the completion state. That is, it waits for the Complete signal associated with the path of that node. Each waiting node and completion node can perform synchronization control based on synchronization signals such as the Ready signal and the Complete signal.

[0067] The execution nodes mentioned above may have different meanings depending on the type of aggregate communication operation. For example, for All-Gather, the execution node is usually a data transmission node; for Reduce-Scatter, the execution node is a reduction execution node, etc. Waiting nodes and completion waiting nodes, among other child nodes, are used for synchronization control between peers. Once the graph template is constructed, the dependency topology between its nodes remains unchanged; in subsequent runs, only the node parameters in the graph structure are updated, without altering the graph structure itself.

[0068] It should be understood that the above graph template structure is only an illustrative example. In actual applications, the number of child nodes included in each Peer path may be more or less, and the dependencies between nodes are not limited to the waiting node, execution node, write completion node and wait completion node mentioned above. This disclosure does not impose any specific restrictions on this.

[0069] Unlike related technologies that express a round of set communication as only a set of copy operations to be executed in this round and two synchronizations before and after the copy operations, this disclosure models the preparation, waiting, data transmission or protocol execution, and completion notification stages of set communication as node dependencies in a graph template, and solidifies its control structure during the initialization stage. During runtime, only the parameters and round values ​​are replaced, thereby reducing the overhead of repeated control plane organization in high-frequency repetitive call scenarios and reducing the continuous occupation of GPU computing resources.

[0070] Next, refer to Figure 3 and Figure 4 , Figure 3 and Figure 4 The data flow and signal synchronization process of All-Gather type collection communication are shown in Push Mode and Pull Mode, respectively.

[0071] For distributed computing scenarios with multiple accelerators on a single machine, the physical devices such as GPUs participating in the group communication are abstracted into unified virtual nodes, distinguished by Peer identifiers or Rank (process) identifiers. Each Rank runs on an independent GPU hardware device to execute the parallel computing tasks assigned to its corresponding GPU. In other words, each Rank corresponds to one GPU, and all communication and computation operations are executed within the corresponding GPU. Taking a communication group consisting of 4 GPUs as an example, they are assigned independent identifiers Rank0, Rank1, Rank2, and Rank3.

[0072] When building a graph template for a communication group, multiple node paths are constructed for each GPU between that GPU and all GPUs in the communication group. Taking Rank 0 as an example, a set of Peer paths are constructed for Peer 0: Rank 0→Rank 0, Rank 0→Rank 1, Rank 0→Rank 2, Rank 0→Rank 3.

[0073] In practical application, operations such as sending, receiving, and reducing data blocks in aggregate communication can be synchronized between different Ranks using a Signal / Wait synchronization mechanism. For example, a synchronization signal is initiated by the source Rank, and the target Rank waits for this signal until it arrives before continuing execution, thus synchronizing the execution flow between Ranks. The entire aggregate communication process controls the execution sequence at different stages through a series of Wait (waiting for data or computation to be ready) and Signal (notifying the sending or computation completion of signals) operations.

[0074] refer to Figure 3For All-Gather in push mode, each Rank needs to set a global signal to initiate the corresponding node path. For each Rank's set of node paths, the first child node, the waiting node, needs to wait for Ready signals: Signal R00, Signal R11, Signal R22, and Signal R33, respectively. This ensures that all Ranks reach a consistent state before batch data copying. After the corresponding Ready signal arrives, it jumps to the second child node, the execution node, to perform data transmission operations. In Push mode, each Rank pushes its local buffer's input data fragments (Buf0, Buf1, Buf2, and Buf3) to the target buffers of each Peer. After pushing is complete, it jumps to the next child node, the write node, to write a Complete signal to each Peer. Then, it jumps to the next child node, the wait node, to wait for its local Complete signal to be written, confirming the completion of all data transmissions through this synchronization operation. In Push mode, the data source actively writes local data fragments to the corresponding locations in the shared symmetric memory of the remote target side.

[0075] Continue to refer to Figure 4 For All-Gather in Pull mode, compared to Figure 3 The main difference in Push mode lies in the fact that, in the second child node of each node path, the execution node, pulls the corresponding data fragments Buf0, Buf1, Buf2, and Buf3 from each Peer in the communication group (Peer0, Peer1, Peer2, and Peer3) to the local target buffer. After pulling is complete, it jumps to the completion write node, writes a Complete signal to each Peer, and then jumps to the completion wait node, waiting for the local completion signal to be written. In other words, in Pull mode, the target side actively reads the data fragments from the corresponding buffer in the remote shared symmetric memory and writes them to the local output location.

[0076] In some alternative implementations, input / output tensors can be stored in each GPU's dedicated video memory area and / or shared symmetric memory, which can be accessed by each GPU.

[0077] In some embodiments, the execution mode of the set communication operation is determined based on the memory residency of the input / output tensors, including: When the input tensor is located in shared symmetric memory and the output tensor is not located in shared symmetric memory, the execution mode of the set communication operation is determined to be Pull mode. When the output tensor is located in shared symmetric memory and the input tensor is not located in shared symmetric memory, the execution mode of the set communication operation is determined to be Push mode.

[0078] In practical terms, the first step is to determine whether the input and output tensors can be executed via a communication group to share symmetric memory. In some cases, if the input tensor is located in shared symmetric memory but the output tensor is not, the Pull mode can be used; in other cases, if the output tensor is located in shared symmetric memory but the input tensor is not, the Push mode can be used. For scenarios where both the input and output tensors are located in shared symmetric memory, a more suitable execution mode can be flexibly selected based on factors such as the target layout, memory usage, whether an intermediate buffer is needed, and whether in-place writes are allowed.

[0079] Let's refer to it again. Figure 5 and Figure 6 , Figure 5 and Figure 6 The data flow and signal synchronization process of the Reduce-Scatter type collection communication are shown in Push mode and Pull mode, respectively.

[0080] like Figure 5 As shown, for the Reduce-Scatter in Push mode, each Rank also needs to set a global Signal to start the corresponding node path. In the second child node of each node path, the execution node, the local input data chunks are pushed to the target buffers of each Peer, and reduction is performed on the target side. After the reduction is completed, a completion signal is written to each Peer, and the node waits for its local completion signal to be written. This execution mode is suitable for directly distributing local input data chunks according to the target Rank to different target sides, and performing reduction on the target side.

[0081] like Figure 6 As shown, for Reduce-Scatter in Pull mode, and Figure 5 The main difference in the Push mode is that the execution node pulls the corresponding data fragments belonging to its own Rank from each Peer and performs the reduction operation on the local output buffer. This execution mode is suitable for situations where the local output is the collection target, and the current Rank actively pulls the reduction input fragments belonging to its own Rank.

[0082] In some implementations, both the Push and Pull modes of Reduce-Scatter can be incorporated into a unified graph template management framework, thereby allowing for the selection of a more suitable execution mode under different memory layouts and runtime constraints.

[0083] According to the collection communication method of the above embodiments of this disclosure, multiple stages of collection communication, such as waiting, execution, completion notification, and completion waiting, are pre-organized into a unified graph template. By constructing and instantiating the graph template of collection communication, reusable graph execution instances are obtained, thereby avoiding the repeated organization of data copy sets and reducing control plane overhead. Compared with existing solutions where synchronization is usually only reflected in Ready consistency before batch processing and Complete consistency after batch processing, the above embodiments of this disclosure explicitly model the execution relationship of collection communication into the graph node dependencies of waiting, execution, completion write, and completion waiting on each Peer path. This allows synchronization control to advance from the batch level to the node level, Peer level, and stage level, rather than using the entire round of communication or the entire batch copy as the only synchronization unit. This enables fine-grained control over the advancement order of different Peers and different stages.

[0084] Furthermore, the above implementation also supports push mode and pull mode as optional execution modes for collection communication. Based on the memory residency status of input and output tensors, target layout, intermediate buffer requirements, and whether in-place writing is allowed, push mode or pull mode is selected to execute collection communication, thereby enhancing the adaptability and scalability under the unified framework.

[0085] Based on the above embodiments, after determining the graph template, a graph execution instance that can be repeatedly started is obtained by instantiating the graph template.

[0086] In some optional implementations, step 103 above, which instantiates the target graph template to obtain a graph execution instance, includes: first, determining the root node used to initiate the ensemble communication operation; then, based on the determined graph template, determining the child node used to execute the ensemble communication operation for each GPU in the communication group, and determining the dependencies between the child nodes according to the execution order of each child node, and constructing multiple node paths between the GPU and each GPU in the communication group; and then, instantiating the root node and each node path to generate a graph execution instance.

[0087] In some optional implementations, the aforementioned execution entity maintains a communicator object for each communication group by identifying the communication context. Then, a unified graph initialization process is invoked to create push / pull mode graph templates for all-gather, all-reduce, and reduce-scatter type collection communication, and initializes shared symmetric memory and the graph template. After initialization, when the communication group subsequently performs collection communication operations again, it is not necessary to create graph objects or reorganize node relationships.

[0088] In some alternative implementations, communicator objects can be stored in a mapping table with the communication group identifier as the key. Different communication groups maintain their own graph execution instances, thereby ensuring that the control planes of different communication groups do not interfere with each other, which facilitates the expansion of multiple concurrent scenarios.

[0089] Based on the above embodiments, according to some optional implementations of this disclosure, when performing the collective communication in each round, the node parameters in the graph execution instance are updated according to the round identifier of the current round, and the graph execution instance is run to perform the collective communication operation of the current round.

[0090] In some embodiments, when running a graph execution instance, the node parameters in the graph execution instance are first updated. After the node parameters are updated, the graph execution instance is started to perform a set communication operation and obtain the corresponding set communication result. The node parameters may include data copy parameters for the data copy task used to perform the set communication operation, and synchronization signal parameters for indicating the synchronization status of different nodes in the set communication operation.

[0091] In some optional implementations, for aggregate communication operations comprising multiple rounds, a round identifier (SeqNum) is used as the logical distinguishing identifier for each round of aggregate communication. During round updates of aggregate communication, the round identifier SeqNum, used to indicate the current round, is updated. Whenever a new round of aggregate communication is scheduled, SeqNum in the graph information is incremented. Subsequently, the target values ​​of the synchronization signal parameters of each waiting node and completion waiting node in the graph execution instance are updated to the SeqNum of the current round. During the execution of the graph execution instance, both waiting nodes and completion waiting nodes synchronize around the SeqNum of the current round.

[0092] In some implementations, updating node parameters in the graph execution instance based on the round identifier of the current round includes: updating the data copy parameters of the execution node based on the round identifier of the current round. The data copy parameters are used to execute the data copy task included in the collective communication operation of the current round, and may include the source address, destination address, and data length of the target data. The target data may include data to be sent and / or data to be received in the data copy task. The source address may be the address of the data to be sent, the destination address may be the address of the data to be received, and the data length may be the byte length or number of elements of the data to be sent and / or received.

[0093] In some optional implementations, synchronization signal regions corresponding to different types of collective communication operations can be set to carry the synchronization state in the collective communication operation. In some optional implementations, synchronization signal regions corresponding to different types of collective communication operations can be set in shared symmetric memory. Logical isolation between each independent synchronization signal region can be achieved through different base offsets.

[0094] refer to Figure 7 As shown, a signal panel is allocated in shared symmetric memory, and the signal panel is divided into multiple independent signal regions according to the type of aggregate communication. For example, there are All-Gather signal regions and Reduce-Scatter signal regions, and different base offsets are used to achieve logical isolation between these signal regions. For instance, the All-Gather signal region can correspond to the all-gather signal base offset, and the Reduce-Scatter signal region can correspond to the reduce-scattersignal base offset. By assigning different base offsets to different types of aggregate communication operations, logical control signal conflicts between different aggregate communication operations can be avoided.

[0095] Each independent signal region can be further divided into a Ready (waiting signal) region and a Complete (completion signal) region. The Ready region has multiple waiting signal slots, and the Complete region has multiple completion signal slots. The waiting signal slots in the Ready region store the corresponding Ready values, indicating the preparation status of each participating Rank in the current round; the completion signal slots in the Complete region store the corresponding Complete values, indicating the completion status of each participating Rank in the current round.

[0096] For each Rank, the synchronization signal area can correspond to at least one Ready slot in the Ready area and a set of Complete slots in the Complete area. The Ready slots are used to store the Ready value corresponding to the current Rank; the set of Complete slots are used to store the Complete value corresponding to the current Rank, where each Complete slot reflects the completion status of the current Rank relative to other participating Ranks. Figure 7 As shown, for Rank0, there is one corresponding Ready slot in the Ready area, namely the Rank0 slot, used to store the Ready value R00 corresponding to Rank0; and a group of Complete slots in the Complete area, namely the Rank0 slot group, used to store the Complete value corresponding to Rank0, such as... Figures 3 to 6The values ​​of C00, C01, C02, and C03 shown in any of them.

[0097] In some implementations, updating the node parameters in the graph execution instance according to the round identifier of the current round further includes: updating the waiting signal parameters of waiting nodes and the completion signal parameters of completed waiting nodes to the target values ​​corresponding to the round identifier of the current round, respectively, according to the round identifier of the current round. The target value of the waiting signal parameter is used to characterize the preparation state of the aggregate communication operation in the current round; the target value of the completion signal parameter is used to characterize the completion state of the aggregate communication operation in the current round.

[0098] Taking the current Rank as an example, before each round of aggregate communication starts, the Ready value for this round is first written into the corresponding Ready slot, and the local system waits for the Ready values ​​of other participating Ranks to reach the target value for this round, so that all Ranks reach a consistent state before the batch data copy. Then, the data copy operation for the current round is organized according to the aggregate communication type, a set of data copy parameters including source address, destination address, and data length are updated, and then the data transmission or data operation for the current round is executed. After the data transmission or data operation is completed, the Complete value is written to the remote or local synchronization area, and the system waits for other participating Ranks to complete their writing.

[0099] For example, each Rank maintains a pair of Ready / Complete signals for all participating Ranks within the communication group. For instance, Ready_i indicates that Peer_i is ready to send, and Complete_i indicates that Peer_i has completed receiving, thus enabling low-latency pipelined synchronization. Here, i = 0, 1, 2, ..., N-1.

[0100] By setting up a separate synchronization signal area for the set communication operation and setting the target value (waiting value) of the synchronization signal parameter for the current round according to the round identifier, both the Ready node and the Complete node complete synchronization around the current round identifier during graph execution.

[0101] It should be understood that the specific size, number of slots, arrangement order, and alignment of the Ready / Complete area can be adjusted according to actual application requirements; and the signal areas of different types of collection communication operations can be isolated using different base offset partitioning methods.

[0102] In some exemplary implementations, for an All-Gather type collective communication operation, the process of initiating a graph execution instance and executing the collective communication operation for the current round may include: First, the initiating graph execution instance activates the root node and jumps to the waiting nodes of each node path. The root node can also be considered as the initiating node for each node path corresponding to each Rank. For each node path of each Rank, when the Ready state is completed, the target value of the waiting signal parameter for the current round is written to the Ready slot; when the waiting signal parameter reaches the target value corresponding to the current round identifier, the process jumps to the execution node to execute the data transmission operation corresponding to All-Gather; after the data transmission operation is completed, the process jumps to the completion write node and writes the target value corresponding to the current round identifier to the Complete slot; when the completion signal parameter reaches the target value corresponding to the current round identifier, the current node path is determined to be completed, thus obtaining the collective communication result of the data transmission operation for the current round.

[0103] In some exemplary implementations, for a Reduce-Scatter type collection communication operation, the process of launching the graph execution instance and executing the collection communication operation of the current round may include: First, the graph execution instance activates the root node and jumps to the waiting nodes of each node path. The root node can also be considered as the starting node for the node path corresponding to each Rank. For each node path of each Rank, when the Ready state is complete, the target value of the waiting signal parameter for the current round is written to the Ready slot; when the waiting signal parameter reaches the target value corresponding to the current round identifier, the execution node is jumped to execute the data operation corresponding to Reduce-Scatter; after the data operation is completed, the execution node is jumped to complete write, and the target value corresponding to the current round identifier is written to the Complete slot; when the completion signal parameter reaches the target value corresponding to the current round identifier, the current node path is determined to be completed, and the collection communication result of the data reduction and distribution operation of the current round is obtained.

[0104] According to the above-described embodiments of the present disclosure, the set communication method divides the shared symmetric memory into independent Ready / Complete dual signal areas for different set communications, and updates the node parameters based on the round identifier in each round of set communication, thereby achieving synchronous isolation during multi-round graph execution reuse and avoiding the old round state from mistakenly triggering the current round.

[0105] According to related technologies, the batch memory copy-based collection communication scheme includes a set of data copy operations to be executed in this round and two synchronizations before and after the data copy. The collection communication method disclosed in this invention models the preparation, waiting, data transmission or reduction execution, and completion notification stages of collection communication as node dependencies in a graph template, and solidifies its control structure during the initialization phase. During runtime, only node parameters and round values ​​need to be replaced. Therefore, this invention can reduce the overhead of repeated control plane organization in high-frequency repetitive call scenarios and reduce the continuous occupation of GPU computing resources.

[0106] Furthermore, compared with the related technologies that switch between shared synchronization areas and reset when necessary, the above-mentioned method of dividing independent signal areas and using incremental round identifiers as target values ​​for synchronization signal parameters enables continuous reuse of graph execution instances in multi-round aggregate communication. It eliminates the need to reset the entire synchronization signal area to its initial value in each round, prevents old round write states from mistakenly satisfying the waiting conditions of the current round, and isolates the synchronization states of different aggregate communications from each other, thus reducing control plane reset operations and the risk of synchronization contamination.

[0107] According to some embodiments of this disclosure, a memory management model based on shared symmetric memory enables each Rank in a communication group to access the remote memory of other Ranks or Peers. Specifically, during the initialization phase, a symmetrically reachable memory region is allocated to the communication group corresponding to the aggregated communication, and a virtual address space of the same size is allocated to each device participating in the aggregated communication. This allows local devices to map the memory of peer devices to their own virtual memory address space, enabling each device to directly access the data buffers of other devices through a unified mapping relationship. Here, "symmetric" means that each Rank participates in memory registration with a consistent shape, data type, and layout, thereby establishing a stable cross-Rank address mapping during operation.

[0108] To access remote memory of different ranks, the base address and offset of that remote rank in shared symmetric memory are required. The corresponding actual physical memory address is then determined based on the virtual memory address of the input tensor.

[0109] Combination Figure 8 As shown, the shared symmetric memory region includes a symmetric memory base address, a front area / other symmetric memory allocations, and the tensor actual data region. The symmetric memory base address corresponds to the base address of the virtual address space of each symmetric memory object, and the tensor actual data region is used to store the actual data of the upper-level tensor. The tensor actual data region may include multiple chunks (slices), such as chunk0 to chunk3.

[0110] In practical applications, symmetric memory objects typically provide a symmetric memory base address, while the actual data address of the upper-level tensor may be obtained by the cache allocator from a large allocation area of ​​shared symmetric memory, and is therefore usually located at an offset after the symmetric memory base address. Therefore, using only the symmetric memory base address is usually insufficient to correctly locate tensor data. During addressing, for each Rank, the starting position of the data for that Rank can be determined based on the symmetric memory base address and the allocator offset; that is, the local tensor start point = base address + allocator offset.

[0111] Furthermore, when group communication partitions data into chunks based on Rank / Peer, the chunk offset corresponding to the current Rank / Peer should also be considered within the actual data region of the tensor. In scenarios combining shared symmetric memory and cache allocators, the source or destination address in a graph node is not simply equal to the symmetric memory base address, but needs to be determined by combining the allocator offset and the chunk offset to determine the actual access address. When determining the starting position of each data chunk, the corresponding chunk offset needs to be added to the actual starting address of the tensor. For example, when accessing chunk0, the address calculation relationship is: chunk0 = base address + allocator offset + chunk0 offset; when accessing chunkN, the address calculation relationship is: chunkN = base address + allocator offset + chunkN offset.

[0112] In some optional implementations, the offset of each tensor relative to the symmetric memory base address is recorded. When updating graph node parameters, the symmetric memory base address, the tensor's offset relative to the base address, and the data fragment offset corresponding to the current Rank or Peer are all considered. The actual access address can be expressed as: Actual address = Symmetric memory base address + Allocator offset + Fragment offset.

[0113] During the execution of the graph execution instance, when updating the data copy parameters, the source and destination addresses can include the base address (i.e., the symmetric memory base address) of the data to be sent and received in shared symmetric memory, and can also include address offset information and fragment offset information. In some implementations, when the input tensor is located in shared symmetric memory, the source address can be determined by the base address, address offset information, and fragment offset information of the data to be sent in shared symmetric memory. When the output tensor is located in shared symmetric memory, the destination address can be determined by the base address, address offset information, and fragment offset information of the data to be received in shared symmetric memory. Furthermore, during data transmission or reduction operations, the actual addresses corresponding to the source and destination addresses of the target data, i.e., the physical addresses, can be determined based on the updated data copy parameters and the unified mapping relationship of shared symmetric memory. The data copy task is then completed based on this physical address to perform data transmission or reduction operations on the target data.

[0114] In practical terms, if the source tensor (input tensor) or the destination tensor (output tensor) is located in shared symmetric memory, the actual access address can be calculated based on its corresponding symmetric memory base address, address offset information (such as allocator offset), and corresponding chunk offset (such as chunk offset). The calculation method is as follows: If the source tensor is located in shared symmetric memory, then the source address = symmetric memory base address + address offset information + Rank / Peer corresponding slice offset; If the destination tensor is located in shared symmetric memory, then the destination address = symmetric memory base address + address offset information + Rank / Peer corresponding slice offset.

[0115] In some implementations, when the input tensor is located in the GPU's local memory, the source address can be determined by the data pointer and fragment offset information of the data to be sent in the local memory. When the output tensor is located in the GPU's local memory, the destination address can be determined by the data pointer and fragment offset information of the data to be received in the local memory. For example, if the input or output tensor is located in the GPU's local memory, its actual physical address can be determined by using its data pointer in the local memory as the base address and adding the corresponding fragment offset when necessary. The data copying task is then completed based on this actual physical address, and data transfer operations or reduction operations are performed on the target data.

[0116] The set communication method according to the above embodiments of this disclosure supports combined addressing of symmetric memory base address, cache allocator offset, and fragment offset. During the runtime phase of the graph execution instance, by updating data copy parameters, the symmetric memory base address, cache allocator offset, and fragment offset corresponding to Rank / Peer are explicitly processed, which can adapt to real tensor allocation scenarios in deep learning frameworks. It also supports local memory data pointer mapping addressing, which can be directly used in upper-layer tensor allocation systems without forcing tensors to exclusively occupy the entire symmetric memory block. Furthermore, it does not only treat symmetric memory as an reachable address space or synchronization area, but also combines it with graph templates, signal panels, runtime parameter updates, and set communication operator scheduling to form a complete execution framework, thereby improving the availability of set communication in real framework environments.

[0117] In some implementations, the main data transfer and synchronization control processes in the collection communication are completed based on the asynchronous copy engine, shared symmetric memory mechanism, and graph execution mechanism. This is particularly suitable for high-frequency communication scenarios such as parameter sharding training. For example, in scenarios such as full-shard data parallel training, parameter sharding training, and tensor parallel training, All-Gather and Reduce-Scatter are called frequently. The above mechanism can reduce the continuous occupation of GPU computing resources by the communication execution path, thereby reducing resource contention between the communication execution process and forward computation, backward computation, or other computation tasks, and improving the overall execution efficiency in the main training path.

[0118] Please see below. Figure 9a and Figure 9b This illustrates the processing flow of a collection communication method in a specific application scenario. The executing entity of this collection communication method can be, for example, a host or a server.

[0119] Combination Figure 9a As shown, during the initialization phase, the processing flow 900a of the collection communication method may include the following steps: Step 901: Create a communicator object.

[0120] In this embodiment, the execution entity of the collective communication method determines information such as the communication group identifier, input tensor, output tensor, and reduction type related to the collective communication operation based on the collective communication request initiated by the upper-layer framework, such as All-Gather, All-Reduce, Reduce-Scatter, and All-To-All requests.

[0121] When a communication group triggers a collection communication operation for the first time, a communicator object is created based on the communication group identifier, communication group World Size, Rank, etc.

[0122] In some implementations, the communicator object stores at least the following information: Rank identifier; communication group WorldSize; signal symmetry memory object; and graph information object. For example, the graph information object may include push-mode All-Gather graph information, pull-mode All-Gather graph information, push-mode Reduce-Scatter graph information, or pull-mode Reduce-Scatter graph information, etc.

[0123] In some alternative implementations, each graph information object may include: a graph object; a graph execution instance; a two-dimensional array of nodes organized by peer; the current device context; an array of transport node parameters; an array of protocol node parameters; an array of waiting node parameters; a round number SeqNum, etc.

[0124] Step 902: Assign and set up the signal panel.

[0125] In some implementations, separate signal regions are defined for different types of aggregate communication operations. For example, there are All-Gather signal regions and Reduce-Scatter signal regions. These independent signal regions can be logically isolated from each other by different base offsets to avoid control signal conflicts between different aggregate communication operations.

[0126] In some embodiments, each logical region can be further divided into a Ready (waiting signal) region and a Complete (completion signal) region. Combined with... Figure 7 As shown, in a single-machine multi-accelerator direct connection scenario, if the domain size is N, the total number of signal elements for aggregate communication can be set to N×2, with half used for the Ready area and the other half for the Complete area.

[0127] For each Rank, the Ready area can correspond to one Ready slot, and the Complete area can correspond to a group of Complete slots, which are used to indicate the ready synchronization relationship and the completed synchronization relationship between the current Rank and different Peers in the communication group, respectively.

[0128] Step 903: Obtain alternative image templates.

[0129] A general graph template structure can be pre-created, or a corresponding graph template structure can be created according to the type of set communication operation as a candidate graph template.

[0130] Combination Figure 2As shown, the candidate graph template may include a root node and multiple sets of node chains expanded along peer paths. For set communication operations involving multiple rounds, the root node is used to trigger the execution of the set communication operation in the current round. Each peer path corresponds to a set of node dependencies related to that peer. For example, taking 4 child nodes as an example, each peer path may include: Child node 1 (waiting node): Used to wait for the Ready slot of the current round of communication operations to reach the current round identifier; Child node 2 (execution node): Used to perform data transmission, protocol execution, or other processing operations related to collective communication; Child node 3 (Complete write node): Used to write the completion status of the current round to the corresponding Complete slot after the execution node has finished processing; Child node 4 (Completion waiting node): Used to wait for the Complete slot associated with the current Peer path to reach the current round identifier.

[0131] For details regarding the specific structure of the template, please refer to the relevant descriptions in the aforementioned embodiments; they will not be repeated here.

[0132] Step 904: Select a matching graph template based on the type and execution mode of the set communication operation.

[0133] The system can select a matching graph template from multiple candidate templates based on the type and execution mode of the collection communication operation. For example, for an All-Gather type collection communication operation, the All-Gather graph template with either push or pull modes can be selected based on its execution mode; for a Reduce-Scatter type collection communication operation, the Reduce-Scatter graph template with either push or pull modes can be selected based on its execution mode, and so on.

[0134] Step 905: Instantiate the graph template to generate a graph execution object.

[0135] Based on the selected graph template, the graph template is instantiated to obtain a graph execution instance that can be repeatedly started.

[0136] Step 906: The object to be executed enters the running phase.

[0137] Once instantiated, the graph execution object can enter the runtime phase. During the runtime phase, multiple rounds of collective communication operations can be achieved through the reuse of graph execution instances.

[0138] For each round of aggregate communication, the logical identifier of the current round is generated by incrementing the round identifier SeqNum. Each waiting node uses the current round identifier SeqNum as the target value of the synchronization signal parameter and updates the Ready / Complete slot through the device side write value operation, thereby realizing synchronization when multiple rounds of graph instance reuse.

[0139] Combination Figure 9b As shown, the data flow and signal synchronization process of the collective communication in each round of the operation phase are explained in detail below. In practical terms, for a new round of collective communication, the SeqNum maintained by the communicator object is incremented to a new sequence number, for example, incrementing from round k to round k+1, from round k+1 to round k+2, and so on. The value of k can be a positive integer. The data flow and signal synchronization process of the collective communication in each round are explained in detail below. For example... Figure 9b As shown, during the runtime phase, the processing flow 900b of the collection communication method may include the following steps: Step S10, k-th round of set communication.

[0140] In some embodiments, step S10 may include the following sub-steps S11 to S13.

[0141] S11 increments SeqNum to the current round identifier.

[0142] Increment the SeqNum of the graph information to a new sequence number, for example, increment it from k-1 to k, to obtain the current round identifier of the set communication.

[0143] S12, Update the parameters of the waiting node.

[0144] Set the target values ​​of the Ready signal parameter of the waiting node and the Complete signal parameter of the completed write node for each Peer in the graph execution instance to the current round identifier SeqNum.

[0145] S13, Update the execution node parameters.

[0146] Based on the current input / output tensor's memory address, symmetric memory base address, cache offset, fragment offset, byte length, number of elements, and data type, update the data transfer node or data reduction execution node in the graph instance.

[0147] Once both the waiting node parameters and the execution node parameters have been updated, a graph execution instance is started on the current device stream. Each graph node automatically advances according to the predetermined dependencies: waiting for the Ready signal, executing data transmission or reduction, writing to the Complete signal, and waiting for the Complete signal.

[0148] For example, in Push mode All-Gather, after the current Rank writes a Ready signal locally, it waits for the Ready signals of each Peer to reach the target value for the current round, and then pushes the local input fragments to the corresponding slots in the target buffers of each Peer. After completion, it writes a Complete signal to each Peer and waits for the local Complete slots to reach the target value for the current round. In other words, in Push mode, the data source actively writes the local fragments to the corresponding locations in the symmetric memory of the remote target side.

[0149] For All-Gather in Pull mode, after the current Rank writes a Ready signal locally, it waits for the Ready signals of each Peer to reach the target value for the current round, and then pulls the corresponding data fragments from each Peer to the local target buffer. After completion, it writes a completion signal to each Peer and waits for the local Complete slot to reach the target value for the current round. In other words, in Pull mode, the target side actively reads data from the corresponding buffer in remote symmetric memory and writes it to the local output location.

[0150] For the Reduce-Scatter in Push mode, after the current Rank is written to the local Ready signal, it waits for the Ready signals of each Peer to reach the target value for the current round, and then pushes the different data fragments in the local input to the target buffers of the corresponding Peers, performing reduction on the target side; after completion, it writes a completion signal to each Peer and waits for the local Complete slot to reach the target value. This method is suitable for directly distributing the local input to different target sides according to the fragments corresponding to the target Rank and performing reduction.

[0151] For the Reduce-Scatter in Pull mode, after the current Rank writes a Ready signal locally, it waits for the Ready signals of each Peer to reach the target value for the current round, and then pulls the corresponding fragments belonging to this Rank from each Peer, performs reduction on the local output buffer; after completion, it writes a completion signal to each Peer and waits for the local Complete slot to reach the target value. This method is suitable for using local output as the collection target, where the current Rank actively pulls the reduction input fragments belonging to this Rank.

[0152] After the set communication operation of the current round (round k) is completed, the corresponding set communication result is obtained. Then, the set communication result is output or post-processing is performed. For example, for the average reduction mode, a scaling process can be run after the execution is completed.

[0153] Step S20, the (k+1)th round of set communication.

[0154] In some embodiments, step S20 may include the following sub-steps S21 to S23.

[0155] S21, SeqNum continues to increase.

[0156] Increment the current round identifier SeqNum of the set communication from k to k+1.

[0157] S22, Update the waiting node parameters for the current round.

[0158] Update the target value of each waiting node in the graph execution instance to k+1; and update the data access parameters such as the source address, destination address, and copy length of the data to be operated on in the current round.

[0159] S23, Update the execution node parameters for the current round.

[0160] For each Rank, after it is ready, the value corresponding to k+1 is written to the Ready slot via device side-writing; the execution node only starts running after the value corresponding to k+1 in the Ready slot is reached; after the execution node completes the corresponding operation, the completion write node writes the value corresponding to k+1 to the Complete slot; the completion wait node only continues to advance after the value corresponding to k+1 in the Complete slot is reached.

[0161] In some alternative implementations, the Ready and Complete slots are updated by device-side write operations, and their values ​​correspond to the current SeqNum, thus avoiding the introduction of additional host-side involvement.

[0162] Through the above mechanism, even if the Ready or Complete slot of the previous round of collection communication has reached k, it will not trigger the current round of waiting by mistake, because the waiting condition of the current round is k+1.

[0163] Step S30, the (k+2)th round of set communication.

[0164] The specific implementation of step S30 can be referred to the description in step S20 above. For example, based on step S20, the current round identifier SeqNum of the set communication is incremented from k+1 to k+2, the target value of each waiting node in the graph execution instance is updated to k+2, and the data access parameters such as the source address, destination address and replication length of the data to be operated on in the current round are updated; and the execution node parameters of the current round are updated, etc., which will not be elaborated on here.

[0165] Thus, by updating the round identifier and the target values ​​of the Ready and Complete signals for each round, multi-round reuse of graph execution instances is supported. In the aggregated communication operations of each round, only the round identifier value and node parameters change, while the graph topology remains unchanged. By employing dual-synchronization signal areas for Ready / Complete and peer-organized waiting relationships, stage progression can be controlled at the node, peer, and stage levels, avoiding the additional waiting caused by existing pre-batch / post-batch unified synchronization.

[0166] Unlike related technologies that rely on shared synchronization areas, unified synchronization routines, state switching, or reset when necessary, this disclosure achieves the following effects by combining an independent signal area corresponding to the aggregated communication, an incrementing round identifier, and a target value for the synchronization signal waiting for the current round: There is no need to reset the entire signal region to its initial value in each round; The state written in the old round will not mistakenly satisfy the waiting conditions of the current round; Graph execution instances can be reused continuously across multiple rounds of collection communication operations; The synchronization states of different collection communication types are isolated from each other; Reduce control surface reset operations and the risk of simultaneous contamination.

[0167] According to the above embodiments of the present disclosure, the set communication method pre-constructs a fixed set communication graph template structure during the initialization phase, obtains reusable graph execution instances through instantiation, and reuses graph execution instances by only updating graph node parameters during the runtime phase. This avoids the overhead of repeatedly constructing copy operation sets and synchronizing batches in each round, making it particularly suitable for high-frequency set communication scenarios. At the same time, a fine-grained synchronization mechanism that can be reused in multiple rounds is implemented through the Ready / Complete dual signal area and round identifier. This enables efficient execution of high-frequency communication operators such as All-Gather and Reduce-Scatter in a single-machine multi-accelerator environment that supports symmetric memory, and supports compatibility with data-transfer set communication and set communication operations containing reduction semantics under a unified execution framework.

[0168] Next reference Figure 10 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a collective communication device, which corresponds to the aforementioned collective communication method embodiment. This device can be specifically applied to various servers or electronic devices.

[0169] like Figure 10 As shown, the aggregate communication device 1000 of this embodiment includes: a communication control module 1001, a graph template management module 1002, and an execution node processing module 1003.

[0170] In this embodiment, the communication control module 1001 is configured to determine the collective communication operation to be performed by the communication group based on the collective communication request. The communication group includes multiple graphics processing units (GPUs).

[0171] The graph template management module 1002 is configured to select a matching target graph template from multiple candidate graph templates based on the type of set communication operation and the memory residency of the corresponding input / output tensors; and to instantiate the target graph template to obtain a graph execution instance. The input / output tensors can be stored in the dedicated video memory area of ​​each GPU and / or shared symmetric memory, and each GPU can access the shared symmetric memory. In some embodiments, graph templates can be constructed and instantiated for different types of set communication and different execution modes, graph node parameters can be updated at runtime, and graph execution instances can be started, etc.

[0172] The execution node processing module 1003 is configured to run a graph execution instance to obtain the set communication result corresponding to the set communication operation. In some embodiments, data transmission or reduction execution is completed during the execution of the graph execution instance, and it cooperates with waiting nodes and completion notification nodes to advance each execution stage of the set communication.

[0173] In some optional implementations, the graph template management module 1002 is further configured to pre-create graph templates corresponding to the type of set communication operation and different execution modes as candidate graph templates. Based on the type of set communication operation and the memory residency of the corresponding input / output tensors, a matching target graph template is selected from multiple candidate graph templates in the following manner: The execution mode of the set communication operation is determined based on the memory residency of the input / output tensors; a target graph template matching the type and execution mode of the set communication operation is selected from the candidate graph templates based on the type and execution mode of the set communication operation; wherein, the execution mode includes push mode and pull mode, where push mode refers to the data sender actively sending data to the receiver, and pull mode refers to the data receiver actively requesting data from the sender.

[0174] In some optional implementations, the graph template management module 1002 is further configured to determine the execution mode of the set communication operation based on the memory residency of the input / output tensors in the following ways: when the input tensor is located in shared symmetric memory and the output tensor is not located in shared symmetric memory, the execution mode of the set communication operation is determined to be pull mode; when the output tensor is located in shared symmetric memory and the input tensor is not located in shared symmetric memory, the execution mode of the set communication operation is determined to be push mode.

[0175] In some optional implementations, the target graph template includes a root node and node paths extending from the root node. Each node path includes multiple sequentially linked child nodes and dependencies between the child nodes. The graph template management module 1002 is further configured to instantiate the target graph template to obtain a graph execution instance by: determining the root node used to initiate the set communication operation; for each GPU, determining the child nodes used to execute the set communication operation, and determining the dependencies between the child nodes according to their execution order, constructing multiple node paths between the GPU and each GPU in the communication group; and instantiating the root node and each node path to generate a graph execution instance.

[0176] In some optional implementations, the multiple child nodes in the node path sequentially include: a waiting node for waiting for the collective communication operation to enter the preparation state; an execution node for executing the collective communication operation; a completion write node for writing the completion signal of the collective communication operation; and a completion waiting node for waiting for the collective communication operation to enter the completion state.

[0177] In some optional implementations, the collective communication operation includes multiple rounds; the execution node processing module 1003 is further configured to run a graph execution instance to obtain the collective communication result corresponding to the collective communication operation by: updating the node parameters in the graph execution instance according to the round identifier of the current round when executing the collective communication operation in each round; wherein, the round identifier is used to indicate the execution round of the collective communication operation; starting the graph execution instance, executing the collective communication operation of the current round, and obtaining the collective communication result corresponding to the collective communication operation of the current round.

[0178] In some optional implementations, the execution node processing module 1003 is further configured to update the node parameters in the graph execution instance according to the round identifier of the current round in the following manner: update the data copy parameters of the execution node according to the round identifier of the current round; wherein, the data copy parameters are used to execute the data copy task included in the collection communication operation of the current round, and the data copy parameters include the source address, destination address and data length of the target data; the target data includes the data to be sent and / or the data to be received in the data copy task; the source address includes the address of the data to be sent, the destination address includes the address of the data to be received, and the data length includes the byte length and / or the number of elements of the data to be sent or received.

[0179] In some optional implementations, the execution node processing module 1003 is further configured to update the node parameters in the graph execution instance according to the round identifier of the current round in the following manner: according to the round identifier of the current round, update the waiting signal parameter of the waiting node and the completion signal parameter of the completed waiting node to the target value corresponding to the round identifier respectively; wherein, the target value of the waiting signal parameter is used to characterize the preparation state of the collection communication operation in the current round; the target value of the completion signal parameter is used to characterize the completion state of the collection communication operation in the current round.

[0180] Based on the aforementioned integrated communication device 1000, and referring to Figure 11 As shown, in some optional implementations, the aggregate communication device 1000 may further include: The symmetric memory and signal management module 1004 is configured to manage the data buffer and signal panel in the shared symmetric memory, and to complete address mapping and signal region division. This includes dividing independent Ready / Complete signal areas according to the type of set communication operation, and determining the actual access address based on the symmetric memory base address, offset, and fragmentation information.

[0181] For the local device side of the aggregated communication device, the data buffer and synchronization signal area of ​​the shared symmetric memory area of ​​the peer device, such as the Ready / Complete signal area, can be accessed through the symmetric memory and signal management module 1004.

[0182] More specifically, the communication control module 1001 can receive collective communication operation requests initiated by upper-layer applications or distributed training frameworks through the collective communication request interface, maintain information such as communication group, Rank, domain size and execution round, determine the type and execution mode of the collective communication operation, and control the synchronous advancement of Ready / Complete signals, etc.

[0183] The graph template management module 1002 can create a unified graph template structure, and model the preparation, waiting, data transmission or protocol execution, and completion notification stages of the collection communication operation as graph node dependencies in the graph template.

[0184] During the initialization phase, the execution node processing module 1003 can determine the execution mode based on the memory region where the input and output tensors are located, the data size, the element type, and the communication group information, selecting either push or pull mode. It locates the corresponding communicator object and graph template, solidifies their control structure, and then performs a hot update on the node parameters in the graph template. The updated content includes, but is not limited to, wait values, source address, destination address, offset, transmission length, number of elements, and reduction execution configuration. After completing the hot update of the node parameters, the graph execution instance is directly started on the current device stream to execute this round of aggregate communication. Subsequent rounds of aggregate communication only require updating node parameters, such as transmission node parameters, reduction node parameters, wait node parameters, and completion wait node parameters, as well as the target value corresponding to the round number.

[0185] For more details on the communication control module 1001, the graph template management module 1002, the execution node processing module 1003, and the symmetric memory and signal management module 1004 of the aggregate communication device 1000 according to the embodiments of this disclosure, and their resulting effects, please refer to [link / reference]. Figure 1 The relevant descriptions in the aforementioned method embodiments are not repeated here.

[0186] It should be understood that the asynchronous copy engine interface and various APIs (Application Programming Interfaces) in the above embodiments are merely illustrative examples. The collection communication method disclosed herein is not limited to a specific hardware manufacturer or a specific driver interface, and can be applied as long as it supports asynchronous copying on the device side and the operation of graph execution instances.

[0187] The following will provide an exemplary description of the collection communication method provided in this disclosure, in conjunction with specific application scenarios.

[0188] See Figure 12 This illustrates an exemplary flow of a collection communication method in a specific application scenario. In this application scenario, the processing flow 1200 of the collection communication method may include the following steps: Step 1201: Receive the collection communication request.

[0189] The upper-level framework initiates a collection communication request and can pass in information such as input tensor, output tensor, communication group identifier, collection communication type, and protocol type in the request message.

[0190] Step 1202: Identify the communication context.

[0191] The communication control module can locate communicator objects based on the communication group identifier. If the communicator corresponding to the communication group has not yet been created, the communicator object is created based on the Rank, world size, and communication group identifier, and the synchronization signal area and graph template are initialized.

[0192] Step 1203: Select the diagram template and execution mode.

[0193] The appropriate graph template can be selected based on the type of aggregate communication, whether the input and output tensors are located in symmetric memory, and the current implementation strategy. For All-Gather and Reduce-Scatter, when the input tensor is located in symmetric memory but the output tensor is not, the pull-mode graph template is preferred; when the output tensor is located in symmetric memory but not in symmetric memory, the push-mode graph template is preferred; when both input and output tensors are located in symmetric memory, a more suitable execution mode can be selected based on factors such as target layout, memory usage, and whether in-place writes are allowed.

[0194] Step 1204: Update the round identifier for the current round.

[0195] Based on the selected graph template, generate the logical identifier for the current round, for example, by incrementing seqNum to obtain the round identifier for the current set communication.

[0196] Step 1205: Update the parameters of the waiting node.

[0197] Set the target value of the synchronization signal parameter of the waiting node and the completed waiting node corresponding to each Peer in the graph template to the current seqNum.

[0198] Step 1206: Update the execution node parameters.

[0199] An execution node can correspond to a data transfer node or a reduction execution node. The execution node parameters are updated based on the data address, symmetric memory base address, cache offset, fragment offset, byte length, number of elements, and data type of the current input / output tensor.

[0200] Step 1207: Start the execution of the diagram.

[0201] The graph template is instantiated to obtain a graph execution instance, which is then started on the current device stream. Each node in the graph template automatically progresses according to predetermined dependencies, such as: waiting for the Ready signal, performing data transmission or reduction, writing to the Complete signal, and waiting for the Complete signal.

[0202] Step 1208: Output the set communication results.

[0203] Once the graph execution is complete, the current round of ensemble communication operation is finished, and the result of this round of ensemble communication is obtained. After the graph execution is complete, some post-processing can be performed, such as running a scaling operation for the average reduction mode.

[0204] It should be noted that the collection, gathering, updating, analysis, processing, use, transmission, and storage of user personal information involved in this disclosed technical solution all comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken to prevent unauthorized access to user personal information data and to safeguard user personal information security and network security.

[0205] According to embodiments of this disclosure, this disclosure also provides an electronic device and a readable storage medium.

[0206] An electronic device includes: one or more processors; and a storage device storing one or more computer programs thereon, which, when executed by the one or more processors, cause the one or more processors to implement the collective communication method described in the foregoing embodiments.

[0207] A computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the collection communication method described in the foregoing embodiments.

[0208] Figure 13 A schematic block diagram of an example electronic device 1300 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0209] like Figure 13 As shown, the electronic device 1300 includes a computing unit 1301, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 1302 or a computer program loaded from storage unit 1308 into random access memory (RAM) 1303. The RAM 1303 may also store various programs and data required for the operation of the electronic device 1300. The computing unit 1301, ROM 1302, and RAM 1303 are interconnected via a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0210] Multiple components in electronic device 1300 are connected to I / O interface 1305, including: input unit 1306, such as keyboard, mouse, etc.; output unit 1307, such as various types of monitors, speakers, etc.; storage unit 1308, such as disk, optical disk, etc.; and communication unit 1309, such as network card, modem, wireless transceiver, etc. Communication unit 1309 allows electronic device 1300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0211] The computing unit 1301 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc.

[0212] The computing unit 1301 performs the various methods and processes described above, such as the collective communication method. For example, in some embodiments, the collective communication method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1308. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 1300 via ROM 1302 and / or communication unit 1309. When the computer program is loaded into RAM 1303 and executed by the computing unit 1301, one or more steps of the collective communication method described above may be performed. Alternatively, in other embodiments, the computing unit 1301 may be configured to perform the collective communication method by any other suitable means (e.g., by means of firmware).

[0213] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0214] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0215] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0216] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. This disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Embodiments of this disclosure can be implemented, in whole or in part, by software, hardware, firmware, or any other combination. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. This application can take the form of a computer program product embodied on one or more computer-usable storage media containing computer-usable program code. A computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, all or part of the flow or function according to embodiments of this disclosure is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0217] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A collective communication method, characterized in that, include: The collective communication operation to be performed by the communication group is determined based on the collective communication request; wherein, the communication group includes multiple graphics processing chips (GPUs); Based on the type of the set communication operation and the memory residency of the corresponding input / output tensors, a matching target graph template is selected from multiple candidate graph templates; The target graph template is instantiated to obtain a graph execution instance; Run the graph execution instance to obtain the set communication result corresponding to the set communication operation.

2. The method according to claim 1, characterized in that, The alternative graph templates include pre-created graph templates corresponding to the types of set communication operations and different execution modes; The step of selecting a matching target graph template from multiple candidate graph templates based on the type of the set communication operation and the memory residency status of the corresponding input / output tensors includes: The execution mode of the collection communication operation is determined based on the memory residency status of the input / output tensors. Based on the type of the set communication operation and the execution mode, select a target graph template that matches the type of the set communication operation and the execution mode from the candidate graph templates; The execution modes include push mode and pull mode. In push mode, the data sender actively sends the data to the receiver, and in pull mode, the data receiver actively requests the data from the sender.

3. The method according to claim 2, characterized in that, The input / output tensors can be stored in each GPU's dedicated video memory area and / or shared symmetric memory, and each GPU can access the shared symmetric memory; Determining the execution mode of the set communication operation based on the memory residency status of the input / output tensors includes: When the input tensor is located in the shared symmetric memory and the output tensor is not located in the shared symmetric memory, the execution mode of the set communication operation is determined to be the pull mode; When the output tensor is located in the shared symmetric memory and the input tensor is not located in the shared symmetric memory, the execution mode of the set communication operation is determined to be the push mode.

4. The method according to claim 1, characterized in that, The target graph template includes a root node and node paths expanded from the root node. The node paths include multiple child nodes linked in sequence and the dependencies between the child nodes. The instantiation of the target graph template to obtain a graph execution instance includes: Determine the root node used to initiate the collection communication operation; For each GPU, a child node is determined to perform the set communication operation, and the dependency relationship between the child nodes is determined according to the execution order of the child nodes, and multiple node paths are constructed between the GPU and each GPU in the communication group; The root node and each of the node paths are instantiated to generate a graph execution instance.

5. The method according to claim 4, characterized in that, The multiple child nodes in the node path include, in sequence: A waiting node is used to wait for the collection communication operation to enter the ready state. An execution node is used to perform the collective communication operation. Complete the write node, which is used to write the completion signal of the set communication operation; A completion waiting node is used to wait for the collection communication operation to enter the completion state.

6. The method according to claim 5, characterized in that, The collective communication operation includes multiple rounds; Running the graph execution instance yields a set communication result corresponding to the set communication operation, including: When performing the set communication operation in each round, the node parameters in the graph execution instance are updated according to the round identifier of the current round; wherein, the round identifier is used to indicate the execution round of the set communication operation; The graph execution instance is started, the set communication operation of the current round is executed, and the set communication result corresponding to the set communication operation of the current round is obtained.

7. The method according to claim 6, characterized in that, The step of updating the node parameters in the graph execution instance according to the round identifier of the current round includes: Update the data copy parameters of the execution node according to the round identifier of the current round; The data copy parameters are used to execute the data copy task included in the collection communication operation of the current round, and the data copy parameters include the source address, destination address and data length of the target data; The target data includes the data to be sent and / or the data to be received in the data copy task; the source address includes the address of the data to be sent, the destination address includes the address of the data to be received, and the data length includes the byte length or number of elements of the data to be sent and / or the data to be received.

8. The method according to claim 7, characterized in that, The step of updating the node parameters in the graph execution instance according to the round identifier of the current round also includes: Based on the round identifier of the current round, update the waiting signal parameter of the waiting node and the completion signal parameter of the completed waiting node to the target value corresponding to the round identifier, respectively. Wherein, the target value of the waiting signal parameter is used to characterize the preparation status of the collective communication operation in the current round; the target value of the completion signal parameter is used to characterize the completion status of the collective communication operation in the current round.

9. The method according to claim 8, characterized in that, It also includes setting a synchronization signal area corresponding to the collection communication operation, wherein the synchronization signal area includes a waiting signal area with multiple waiting signal slots and a completion signal area with multiple completion signal slots; Each GPU has an independent wait signal slot and a completion signal slot in the synchronization signal area, wherein the wait signal slot is used to store the wait signal parameters and the completion signal slot is used to store the completion signal parameters.

10. The method according to claim 8 or 9, characterized in that, When the type of the set communication operation is global set communication, starting the graph execution instance, executing the set communication operation of the current round, and obtaining the set communication result corresponding to the set communication operation of the current round includes: for each node path, performing the following steps: Start the graph execution instance, activate the root node, and jump to the waiting node; Once the preparation state is completed, the target value corresponding to the round identifier is written into the waiting signal parameter; In response to the waiting signal parameter reaching the target value corresponding to the round identifier, the process jumps to the execution node to execute the data transmission operation corresponding to the global collection set communication operation; In response to the completion of the data transmission operation, the process jumps to the completion write node and writes the target value corresponding to the round identifier into the completion signal parameter; In response to the completion signal parameter reaching the target value corresponding to the round identifier, it is determined that the node path has been completed, and the collection communication result corresponding to the global collection collection communication operation is obtained.

11. The method according to claim 8 or 9, characterized in that, When the type of the set communication operation is reduction-distributed set communication, starting the graph execution instance, executing the set communication operation of the current round, and obtaining the set communication result corresponding to the set communication operation of the current round includes: for each node path, performing the following steps: Start the graph execution instance, activate the root node, and jump to the waiting node; Once the preparation state is completed, the target value corresponding to the round identifier is written into the waiting signal parameter; In response to the waiting signal parameter reaching the target value corresponding to the round identifier, the process jumps to the execution node to perform the data operation corresponding to the protocol distributed set communication operation; In response to the completion of the data operation, the process jumps to the completion write node and writes the target value corresponding to the round identifier into the completion signal parameter. In response to the completion signal parameter reaching the target value corresponding to the round identifier, it is determined that the node path has been executed and the set communication result corresponding to the protocol distributed set communication operation is obtained.

12. The method according to any one of claims 7 to 9, characterized in that, When the input tensor is located in shared symmetric memory, the source address is determined by the base address, address offset information, and fragment offset information of the data to be sent in the shared symmetric memory; When the output tensor is located in shared symmetric memory, the destination address is determined by the base address, address offset information, and fragment offset information of the data to be received in the shared symmetric memory.

13. The method according to any one of claims 7 to 9, characterized in that, When the input tensor is located in the video memory region of each GPU, the source address is determined by the data pointer and fragment offset information of the data to be sent in the video memory region; When the output tensor is located in the video memory region of each GPU, the destination address is determined by the data pointer of the data to be received in the video memory region and the fragment offset information.

14. A collective communication device, characterized in that, include: A communication control module is configured to determine the collective communication operation to be performed by the communication group based on a collective communication request; wherein the communication group includes multiple graphics processing chips (GPUs). The graph template management module is configured to select a matching target graph template from multiple candidate graph templates based on the type of the set communication operation and the memory residency status of the corresponding input / output tensors; and to instantiate the target graph template to obtain a graph execution instance. The execution node processing module is configured to run the graph execution instance and obtain the set communication result corresponding to the set communication operation.

15. An electronic device, characterized in that, include: One or more processors; Storage device, on which one or more computer programs are stored, When one or more computer programs are executed by one or more processors, the one or more processors implement the collection communication method as described in any one of claims 1-13.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When a computer program is executed by a processor, it implements the collection communication method as described in any one of claims 1-13.

17. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the collection communication method as described in any one of claims 1-13.

Images