Collective communication method, system, computer device, medium and program product

Abstract

The present disclosure provides a collection communication method, system, computer device, medium and program product, relates to the technical field of computers, in particular to the fields of multi-card cooperative communication, collection communication and the like, and the implementation scheme is as follows: determining the communication relationship between each computing unit based on hardware topology; loading a static graph model and traversing the operator nodes in the static graph model, generating a calculation instruction for a calculation operator and writing the calculation instruction into a first instruction buffer, allocating a second instruction buffer for a communication operator and creating a placeholder in a first instruction sequence, pointing to the second instruction buffer; after the traversal is completed, generating a second instruction sequence for each computing unit for controlling data communication between the computing unit and adjacent computing units and writing the second instruction sequence into the second instruction buffer; and a runtime hardware control unit sequentially executes the first instruction sequence, jumps to the second instruction buffer to execute a communication instruction when the placeholder is executed, and realizes data communication between the computing units.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, particularly to multi-card collaborative communication, aggregated communication, and especially to an aggregated communication method, system, computer device, computer-readable storage medium, and computer program product. Background Technology

[0002] In the deployment of large-scale deep learning models for inference, as the number of model parameters continues to grow, the computing power and storage of a single accelerator card are insufficient to meet the inference requirements. Therefore, the industry generally adopts a model parallelism strategy, dividing large models and deploying them across multiple accelerator cards for collaborative execution. In this scenario, after each accelerator card completes its local computation, it needs to exchange and synchronize data between cards through cascaded communication operations to ensure that subsequent computation layers can obtain complete input data. Therefore, the efficiency of cascaded communication directly affects the overall performance of the multi-card inference system.

[0003] However, current mainstream collective communication libraries are typically designed for dynamic execution environments, relying on the real-time participation of the host processor during runtime. This on-demand communication model presents a challenge in adapting to accelerators that use instruction preloading. Such accelerators require all execution instructions to be preloaded onto the device before runtime and do not accept dynamic instruction insertion from the host during operation. Under this execution mode, the runtime dynamic negotiation mechanism of traditional collective communication libraries is difficult to apply directly, impacting communication capabilities in multi-GPU collaborative inference scenarios. Therefore, a collective communication method adapted to the instruction preloading execution mode is urgently needed to address this adaptation requirement. Summary of the Invention

[0004] This disclosure provides a collection of communication methods, systems, computer devices, computer-readable storage media, and computer program products.

[0005] According to one aspect of this disclosure, a collective communication method is provided, applied to a system including multiple neural network processors, each neural network processor including at least one computing unit, each computing unit including a hardware control unit, comprising: determining a predecessor computing unit and a successor computing unit corresponding to each computing unit based on the hardware topology of the multiple computing units; loading a pre-generated static graph model for describing operator execution relationships, traversing multiple operator nodes in the static graph model, the multiple operator nodes including at least one communication operator and at least one computing operator; for each operator node: in response to determining that the operator node is a computing operator, generating computing instructions based on the computing operator, and writing the computing instructions into a first instruction buffer corresponding to each computing unit, the first instruction buffer being used to store the hardware... The control unit executes a first instruction sequence; in response to determining that an operator node is a communication operator, it allocates a second instruction buffer for each computing unit and creates a placeholder pointing to the entry address of the second instruction buffer in the first instruction sequence; after traversing multiple operator nodes, for each communication operator, it generates multiple sets of second instruction sequences for each computing unit, wherein the multiple sets of second instruction sequences are written into the second instruction buffer of the computing unit to control data communication between the computing unit and its predecessor and successor computing units; and controls the hardware control unit of each computing unit to execute the corresponding first instruction sequence, so that when the placeholder is executed, it jumps to the entry address of the second instruction buffer to execute multiple sets of second instruction sequences to execute data communication between the computing units.

[0006] In some embodiments, before generating multiple sets of second instruction sequences for each computing unit for each communication operator after traversal is completed, the aggregate communication method further includes: for each computing unit, determining the target address for the computing unit to send data to the successor computing unit; and determining the synchronization address for the computing unit to receive a ready flag, wherein the synchronization address is a pre-set address, and the predecessor computing unit of the computing unit writes a ready flag to the synchronization address after completing data transmission.

[0007] In some embodiments, for each computing unit, each set of second instruction sequences includes a first sub-instruction, a second sub-instruction, and a third sub-instruction. The first sub-instruction is used to determine whether the computing unit has detected a ready flag from the synchronization address. The second sub-instruction is used to transfer the data of the computing unit to the target address of the computing unit's successor computing unit. The third sub-instruction is used to write the ready flag to the synchronization address corresponding to the computing unit's successor computing unit after the second sub-instruction is completed.

[0008] In some embodiments, the second sub-instruction includes an intra-card transfer sub-instruction and an inter-card transfer sub-instruction. The intra-card transfer sub-instruction is used for data communication between computing units within the same neural network processor, and the inter-card transfer sub-instruction is used for data communication between computing units of different neural network processors.

[0009] In some embodiments, each group of second instruction sequences further includes an execution flag bit, which is used to indicate whether each sub-instruction in the group of second instruction sequences is executed by the hardware control unit.

[0010] In some embodiments, the second instruction buffer further includes header information, which is used to record the starting address of multiple sets of second instruction sequences and the total number of sets of second instruction sequences, and to control the hardware control unit of each computing unit to execute the corresponding first instruction sequence, so that when the placeholder is executed, it jumps to the entry address of the second instruction buffer to execute multiple sets of second instruction sequences, so as to perform data communication between computing units, including: reading the header information to obtain the total number of sets and the starting address; and executing multiple sets of second instruction sequences sequentially according to the starting address until all the second instruction sequences corresponding to the total number of sets are executed.

[0011] According to one aspect of this disclosure, a collective communication system is provided, comprising: multiple neural network processors, each neural network processor including at least one computing unit, each computing unit including a hardware control unit; a structure determination module configured to determine, based on the hardware topology of the multiple computing units, a predecessor computing unit and a successor computing unit corresponding to each computing unit; a model loading module configured to load a pre-generated static graph model describing operator execution relationships, traversing multiple operator nodes in the static graph model, the multiple operator nodes including at least one communication operator and at least one computing operator; for each operator node: a first sequence generation module configured to, in response to determining that an operator node is a computing operator, generate computing instructions based on the computing operator, and write the computing instructions into a first instruction buffer corresponding to each computing unit, the first instruction buffer being used to store instructions executed by the hardware control unit. The system comprises: a first instruction sequence; a buffer allocation module configured to allocate a second instruction buffer for each computing unit in response to determining that an operator node is a communication operator, and to create a placeholder pointing to the entry address of the second instruction buffer in the first instruction sequence; a second sequence generation module configured to generate multiple sets of second instruction sequences for each computing unit for each communication operator after traversing multiple operator nodes, wherein the multiple sets of second instruction sequences are written into the second instruction buffer of the computing unit for controlling data communication between the computing unit and its predecessor and successor computing units; and an instruction execution module configured to control the hardware control unit of each computing unit to execute the corresponding first instruction sequence, so that when the placeholder is executed, the system jumps to the entry address of the second instruction buffer to execute the multiple sets of second instruction sequences to execute data communication between the computing units.

[0012] According to another aspect of this disclosure, a computer device is provided, comprising: at least one processor; and a memory having a computer program stored thereon, wherein the computer program, when executed by the at least one processor, causes the at least one processor to perform the methods provided above in this disclosure.

[0013] According to another aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, causes the processor to perform the methods provided above in this disclosure.

[0014] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, causes the processor to perform the methods provided above in this disclosure.

[0015] According to one or more embodiments of this disclosure, a complete execution sequence of all collection communication operations can be pre-generated and embedded in the main instruction stream during the loading phase, so that the hardware control unit executes the pre-generated sequence in the running phase, thereby achieving efficient multi-card data exchange in inference scenarios.

[0016] These and other aspects of this disclosure will be apparent from the embodiments described below, and will be elucidated with reference to the embodiments described below. Attached Figure Description

[0017] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of this disclosure. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0018] Figure 1 This is a flowchart illustrating a collection communication method according to an exemplary embodiment.

[0019] Figure 2 This is a schematic diagram illustrating the data transmission relationship between computing units according to an exemplary embodiment.

[0020] Figure 3 This is a schematic diagram illustrating the execution of an instruction sequence jump according to an exemplary embodiment.

[0021] Figure 4 This is a schematic diagram illustrating the execution of each group of second instruction sequences according to an exemplary embodiment.

[0022] Figure 5 This is a schematic diagram illustrating the data communication process of a full collection operation according to an exemplary embodiment.

[0023] Figure 6 This is a schematic block diagram illustrating a collection communication system according to an exemplary embodiment.

[0024] Figure 7 This is a block diagram illustrating an exemplary computer device that can be applied to an exemplary embodiment. Detailed Implementation

[0025] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0026] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0027] The terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. As used herein, the term "multiple" means two or more, and the term "based on" should be interpreted as "at least partially based on". Furthermore, the terms "and / or" and "at least one of..." cover any one of the listed items and all possible combinations thereof.

[0028] In related technologies, existing aggregated communication libraries adopt an on-demand execution mode. Each communication operation requires the host to dynamically participate in address negotiation, transmission configuration and synchronization management at runtime. This mode is difficult to adapt to the execution architecture of instruction pre-loaded accelerators, making it difficult to achieve efficient multi-card aggregated communication on such accelerators.

[0029] Existing communication libraries initiate communication operations one by one at runtime. Each communication requires interrupting the instruction execution flow on the device side, returning to the host side to complete buffer address exchange and transmission path configuration before issuing the instruction for execution. However, instruction pre-loading accelerators require all instructions to be pre-written to the device side before execution, and executed by the hardware control unit in a fixed sequence. This difference in execution mode makes existing communication libraries unsuitable for direct application to such accelerators.

[0030] To address this, embodiments of this disclosure propose a more efficient collective communication method. By creating placeholders for communication operators during the model loading phase and uniformly generating a communication instruction sequence containing data communication and synchronization control after traversal, and embedding it into the main instruction stream, the hardware control unit can autonomously complete data communication between multiple cards during the runtime phase. This enables the deployment of computation and communication in the same instruction sequence, thereby improving the communication efficiency of instruction preloading accelerators in multi-card collaborative inference scenarios.

[0031] Exemplary embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0032] In the embodiments of this disclosure, the aggregated communication method is applied to a system containing multiple neural network processors, which can be a type of accelerator. In practical deployment, the system can be configured with multiple neural network processors deployed within a single server, or multiple servers can each deploy multiple neural network processors interconnected via a network. Each neural network processor has at least one computing unit, for example, each neural network processor may include a master computing unit and a slave computing unit. Each computing unit has an independent hardware control unit and independent storage space. The hardware control unit is responsible for sequentially reading and executing instruction sequences pre-loaded into the storage space of the computing unit. The hardware control units of each computing unit can work in parallel, each independently executing its own instruction sequence.

[0033] Figure 1 This is a flowchart illustrating a collection communication method according to an exemplary embodiment.

[0034] like Figure 1 As shown, the collection communication method includes steps S101 to S106.

[0035] In step S101, based on the hardware topology of multiple computing units, the predecessor computing unit and the successor computing unit corresponding to each computing unit are determined.

[0036] In the example, based on the physical connections between multiple computing units, the upstream and downstream objects for data communication with each computing unit can be determined. The hardware topology refers to the physical interconnection of each computing unit, including the location of the neural network processor where each unit resides, the type of bus or link connecting the computing units, and the physical node to which each unit belongs. Further, a topology map reflecting the connections between all computing units can be constructed by parsing a topology description file that records the aforementioned physical connection information. A predecessor computing unit refers to the computing unit that sends data to it, and a successor computing unit refers to the computing unit that receives the data sent by it. The process of determining predecessor and successor computing units can be completed during the initialization phase. For example, processes can exchange device information of the computing units they manage through a bootstrap network, including the computing unit number, the neural network processor number, and the physical node identifier. Combining this information with the topology map, an appropriate communication topology algorithm can be selected to determine the predecessor and successor computing units for each computing unit, as well as the master-slave relationship between computing units within the same neural network processor. This step provides a clear data flow basis for the subsequent generation of instruction sequences, enabling the generation of instruction sequences for each computing unit to know which computing unit to send data to and from which computing unit to receive data, thereby ensuring that the computing units can cooperate and coordinate with each other.

[0037] Figure 2 This is a schematic diagram illustrating the data transmission relationship between computing units according to an exemplary embodiment.

[0038] like Figure 2 As shown, the system comprises four neural network processors (represented by dashed boxes), each with two computing units. Computing units 0, 1, 2, and 3 are the master computing units on their respective neural network processors, while computing units 4, 5, 6, and 7 are the slave computing units. A circular data transmission relationship can be established between the master computing units based on the hardware topology. Specifically, the successor computing unit of computing unit 0 is computing unit 1, the successor of computing unit 1 is computing unit 2, the successor of computing unit 2 is computing unit 3, and the successor of computing unit 3 is computing unit 0, thus forming a closed-loop transmission path of computing units 0, 1, 2, 3, and 0. Within this circular path, each master computing unit sends data to its successor computing unit and receives data from its predecessor computing unit. For example, for computing unit 1, its predecessor computing unit is computing unit 0, and its successor computing unit is computing unit 2. Computing unit 1 receives data from computing unit 0 and sends data to computing unit 2.

[0039] like Figure 2 As shown, within the same neural network processor, there is a parent-child relationship between the master computing unit and the slave computing units. Figure 2 As shown, computation unit 0 and computation unit 4 are located on the same neural network processor, with computation unit 0 as the parent node and computation unit 4 as the child node. Similarly, computation unit 1 and computation unit 5, computation unit 2 and computation unit 6, and computation unit 3 and computation unit 7 form parent-child relationships, respectively. During data communication, the master computation unit and slave computation unit within the same neural network processor can act as each other's predecessor and successor computation units, respectively, and bidirectional data transmission occurs between them through the processor's internal transmission channel.

[0040] Understandable, Figure 2 The illustration uses only four neural network processors as an example; however, the embodiments of this disclosure are not limited to this, and the number of neural network processors may not be limited to four.

[0041] Refer again Figure 1 In step S102, a pre-generated static graph model for describing operator execution relationships is loaded, and multiple operator nodes in the static graph model are traversed. The multiple operator nodes include at least one communication operator and at least one computation operator.

[0042] In the example, the static graph model can be generated by the compiler after pre-parallel partitioning and orchestration of the neural network model. It describes the execution relationships between steps during model execution in the form of operator nodes, including the type, execution order, and data dependencies of each operator node. In the embodiments of this disclosure, the pre-generated static graph model can be loaded, and each operator node within it can be traversed sequentially. Since the division of labor among the computing units during parallel partitioning of the model determines which locations require computation and which require inter-card data exchange, the operator nodes in the static graph model include both computational operators for performing matrix operations and activation functions, and communication operators for implementing inter-card data exchange, such as the full collection operation. During the traversal, the type of each operator node can be identified one by one, and different processing logic can be executed according to its type. By processing all operator nodes in the static graph model in a complete loading process, both computational and communication operators can complete their necessary preparations during the loading phase, providing a foundation for the deployment of subsequent instructions and the execution of the hardware control unit.

[0043] In step S103, for each operator node: in response to determining that the operator node is a computation operator, a computation instruction is generated based on the computation operator, and the computation instruction is written into the first instruction buffer corresponding to each computation unit. The first instruction buffer is used to store the first instruction sequence executed by the hardware control unit.

[0044] In the example, when the current operator node is identified as a computation operator during the traversal, it can be converted into computation instructions that the hardware control unit can recognize and execute. A computation instruction is a hardware-level representation of the operation described by the computation operator. For example, a matrix multiplication operator can be converted into register configuration instructions that the hardware control unit can directly execute. The generated computation instructions can be written into the first instruction buffer corresponding to each computation unit. The first instruction buffer is a pre-defined storage area in the storage space of each computation unit, used to store the first instruction sequence to be executed by the hardware control unit of that computation unit. As the traversal progresses, the computation instructions are written into the first instruction buffer sequentially according to the order of the operator nodes in the static graph model, gradually constructing the first instruction sequence. This first instruction sequence will constitute the execution basis of the hardware control unit during the runtime phase.

[0045] In step S104, for each operator node: in response to determining that the operator node is a communication operator, a second instruction buffer is allocated to each computing unit, and a placeholder pointing to the entry address of the second instruction buffer is created in the first instruction sequence.

[0046] In the example, when the current operator node is identified as a communication operator during the traversal, the communication operation is not executed immediately. Instead, preparation is made for the subsequent unified generation of a communication instruction sequence. Specifically, an independent second instruction buffer can be allocated to each computing unit. This second instruction buffer is also a storage area defined in the storage space of each computing unit, used to store the communication instruction sequence corresponding to the communication operator in subsequent stages. At the same time, a placeholder can be created at the position of the communication operator in the first instruction sequence being constructed. This placeholder records the entry address of the allocated second instruction buffer. The placeholder itself does not contain the specific communication execution logic; its function is to mark the execution position of the communication instruction sequence in the first instruction sequence and associate the first instruction sequence with the second instruction buffer through the entry address. Only a placeholder is created at this stage because the generation of the communication instruction sequence depends on information such as the buffer addresses between computing units. This information can only be obtained through information exchange between computing units after all operator nodes have been traversed. Therefore, execution positions and storage space can be reserved for the communication instruction sequence.

[0047] In step S105, after traversing multiple operator nodes, for each communication operator, multiple sets of second instruction sequences are generated for each computing unit. The multiple sets of second instruction sequences are written into the second instruction buffer of the computing unit to control the data communication between the computing unit and its predecessor and successor computing units.

[0048] In the example, after all operator nodes have been traversed, the communication instruction sequence generation stage can begin. Generating the communication instruction sequence requires determining information such as the buffer addresses of adjacent computing units. This information can be obtained by each computing unit through a bootstrapping network after the metadata of all communication operators has been collected. After obtaining the necessary address information, a communication instruction sequence corresponding to each communication operator recorded during the traversal can be generated for each computing unit, i.e., multiple sets of second instruction sequences. These multiple sets of second instruction sequences correspond to multiple execution steps in the data communication process. Each set of second instruction sequences controls one data transfer and synchronization action with adjacent computing units (previous and subsequent computing units). After all multiple sets of second instruction sequences have been executed sequentially, the computing unit has completed complete data communication with its predecessor and successor computing units. After the multiple sets of second instruction sequences are generated, they are written as a whole into the second instruction buffer pre-allocated by the computing unit in the previous steps.

[0049] In step S106, the hardware control unit controlling each computing unit executes the corresponding first instruction sequence, so that when the placeholder is executed, it jumps to the entry address of the second instruction buffer to execute multiple sets of second instruction sequences to perform data communication between the computing units.

[0050] In the example, after all instruction sequences are ready, a run instruction can be issued to each computing unit, controlling the hardware control unit of each computing unit to begin executing its corresponding first instruction sequence. The hardware control units of each computing unit work in parallel, independently reading and executing the first instruction sequence sequentially from their respective first instruction buffers. When the hardware control unit executes a calculation instruction, it directly drives the computing unit to complete the corresponding calculation operation. When the hardware control unit executes a placeholder, it reads the entry address of the second instruction buffer from that placeholder, then jumps to that address and executes multiple pre-stored sets of second instruction sequences in the second instruction buffer sequentially, completing data communication between the computing unit and its predecessor and successor computing units. After all sets of second instruction sequences have been executed, the hardware control unit returns to the position after the placeholder in the first instruction sequence and continues executing subsequent calculation instructions. Throughout the entire operation, the hardware control units of each computing unit coordinate with each other through a pre-set synchronization mechanism. This execution method allows calculation operations and communication operations to be completed in the same execution flow, avoiding the delay caused by repeated communication configuration during runtime.

[0051] Figure 3 This is a schematic diagram illustrating the execution of an instruction sequence jump according to an exemplary embodiment.

[0052] like Figure 3As shown, each computing unit has a first instruction buffer and a second instruction buffer. The first instruction buffer stores a first instruction sequence, which consists of multiple computation instructions and at least one placeholder arranged in execution order. The computation instructions are generated based on computation operators and written sequentially during the traversal of the static graph model. The placeholders are created and inserted into the corresponding positions in the first instruction sequence when a communication operator is encountered during the traversal. The second instruction buffer is a storage area allocated separately for each computing unit when a communication operator is encountered. It stores multiple sets of second instruction sequences generated for that communication operator after traversal. That is, the second instruction buffer can contain a first set of second instruction sequences, a second set of second instruction sequences, a third set of second instruction sequences, etc., each set of second instruction sequences corresponding to an execution step in the data communication process. The placeholders record the entry address of the corresponding second instruction buffer, and the two are associated through this entry address. During the runtime phase, the hardware control unit of the computing unit starts from the beginning of the first instruction buffer and executes each instruction in the first instruction sequence sequentially. When the execution reaches a placeholder, the hardware control unit reads the entry address recorded therein, jumps to the second instruction buffer, and executes multiple sets of second instruction sequences sequentially to complete data communication between the computing unit and its predecessor and successor computing units. After all instruction sets in the second instruction buffer have been executed, the hardware control unit returns to the position after the placeholder in the first instruction sequence and continues to execute subsequent calculation instructions.

[0053] Therefore, through the above scheme, a second instruction buffer is allocated and placeholders are created for the communication operator during the traversal of the static graph model. After the traversal is completed, multiple sets of second instruction sequences corresponding to the communication operator are uniformly generated and written into the second instruction buffers of each computing unit. This allows the communication operator and the computing operator to be uniformly deployed in the same first instruction sequence and executed sequentially by the hardware control unit, solving the problem that traditional real-time execution aggregate communication methods cannot work on instruction preloading accelerators. Simultaneously, since multiple sets of second instruction sequences are fully generated and fixed in the second instruction buffer during the loading phase, the hardware control unit only needs to read and execute the pre-generated instructions sequentially during the runtime phase, without needing to perform address negotiation, dynamic memory allocation, or other decisions, ensuring the determinism and efficiency of instruction execution. Furthermore, the computing instructions and placeholders together constitute the first instruction sequence. When the hardware control unit executes a placeholder, it jumps to the second instruction buffer to complete data communication. After execution, it returns to the first instruction sequence to continue executing subsequent computing instructions, enabling computing operations and communication operations to be completed continuously in the same execution flow, achieving integrated static deployment of computing and communication.

[0054] In some embodiments, before generating multiple sets of second instruction sequences for each computing unit for each communication operator after traversal is completed, the method further includes: for each computing unit, determining the target address for the computing unit to send data to the successor computing unit; and determining the synchronization address for the computing unit to receive a ready flag, wherein the synchronization address is a pre-set address, and the predecessor computing unit of the computing unit writes a ready flag to the synchronization address after completing data transmission.

[0055] In the example, since each computing unit has its own independent storage space and its own unique buffer address, it cannot know in advance where the buffer address of its successor computing unit is located. Therefore, before generating multiple sets of second instruction sequences, a target address needs to be determined for each computing unit. The target address can refer to the storage location that the computing unit points to when sending data to its successor computing unit, i.e., which buffer address of the successor computing unit the data will be written to. Each computing unit can send its own buffer address to its neighboring computing units with which it has a communication relationship through a bootstrapping network, enabling each computing unit to obtain the buffer address of its successor computing unit and thus determine the target address.

[0056] In the example, a synchronization address can be used to solve the execution coordination problem between adjacent computing units. During data communication, the computing unit needs to know whether its predecessor computing unit has finished sending data; only after confirming the completion of data transmission can subsequent operations be performed. To this end, a fixed synchronization address can be pre-assigned to each computing unit, pointing to a storage location accessible to adjacent computing units. After the predecessor computing unit completes data transmission, it writes a ready flag to this synchronization address. The computing unit determines whether the predecessor computing unit's data transmission is complete by continuously checking whether the ready flag appears in the synchronization address. For example, the synchronization address can point to a fixed location in on-chip static random access memory (CSRAM), a high-speed storage area integrated inside the chip, which can be read and written by the hardware control unit of each computing unit. During the loading phase, a fixed address in the CSRAM can be pre-assigned to each computing unit as its synchronization address. For each computing unit, after the current computing unit completes data transmission, its hardware control unit writes a ready identifier to the corresponding CSRAM address. This ready identifier contains the identifier number of the sending computing unit. The hardware control unit of this computing unit then continuously reads its own corresponding CSRAM address to check if the identifier number of the sending computing unit appears. Once a matching identifier is read, it confirms that the data is ready. The aforementioned target address and synchronization address are determined during the loading phase. When multiple sets of second instruction sequences are subsequently generated, these addresses are directly written into the corresponding instructions, allowing the hardware control unit to directly execute data communication and synchronization detection according to the preset addresses during the runtime phase.

[0057] Therefore, by pre-determining the target address and synchronization address of each computing unit before generating multiple sets of second instruction sequences, the address information required for the running phase can be determined once during the loading phase and directly written into the subsequently generated second instruction sequence. This enables the hardware control unit to perform data communication according to the preset target address and detect the ready flag according to the preset synchronization address during the running phase.

[0058] In some embodiments, for each computing unit, each set of second instruction sequences includes a first sub-instruction, a second sub-instruction, and a third sub-instruction. The first sub-instruction is used to determine whether the computing unit has detected a ready flag from the synchronization address. The second sub-instruction is used to transfer the data of the computing unit to the target address of the computing unit's successor computing unit. The third sub-instruction is used to write the ready flag to the synchronization address corresponding to the computing unit's successor computing unit after the second sub-instruction is completed.

[0059] In the example, when multiple computing units execute their respective sets of second instruction sequences in parallel, a clear coordination mechanism is needed to ensure that the preceding computing unit has prepared the data before a computing unit sends it; otherwise, incomplete data may be read. Therefore, each set of second instruction sequences can include three sub-instructions, corresponding to the three stages of a complete execution step in the data communication process: polling detection, data transmission, and completion notification, i.e., the first sub-instruction, the second sub-instruction, and the third sub-instruction. The sequential execution of the three sub-instructions ensures that in each execution step, data readiness is confirmed first, then transmission is performed, and finally the downstream is notified, forming a strict execution order.

[0060] In the example, when the hardware control unit executes each set of second instruction sequences, it can first execute the first sub-instruction. This first sub-instruction causes the hardware control unit to continuously poll and check the synchronization address corresponding to the computing unit to determine whether a ready flag has been written there. If no ready flag has been detected, the hardware control unit will wait until a ready flag appears in the synchronization address, indicating that the preceding computing unit has sent data to the buffer address specified by that computing unit. Subsequently, the hardware control unit executes the second sub-instruction to transfer the data to be transmitted from the computing unit's buffer address to the target address of the subsequent computing unit, completing one data transfer operation. After the data transmission is complete, the hardware control unit executes the third sub-instruction to write a ready flag to the synchronization address corresponding to the subsequent computing unit, informing the subsequent computing unit that the data transmission has been completed. The hardware control unit of the subsequent computing unit can then detect the ready flag when executing its own first sub-instruction and continue with subsequent operations.

[0061] Therefore, in this way, a synchronous chain of transmission is formed between the hardware control units of each computing unit. After each computing unit executes multiple sets of second instruction sequences in sequence, data communication between all computing units can be completed.

[0062] In some embodiments, the second sub-instruction includes an intra-card transfer sub-instruction and an inter-card transfer sub-instruction. The intra-card transfer sub-instruction is used for data communication between computing units within the same neural network processor, and the inter-card transfer sub-instruction is used for data communication between computing units of different neural network processors.

[0063] In the example, in a real-world multi-GPU deployment scenario, the data communication transmission path varies depending on the physical location of the communicating parties. When the successor computing unit of a given computing unit is located on the same neural network processor (i.e., in a master-slave relationship), data transmission between them can be completed through the processor's internal transmission channel. In this case, the intra-card transfer sub-instruction in the second sub-instruction is executed. For example, the hardware control unit can configure the intra-card DMA engine to move data from the buffer of this computing unit to the buffer address of another computing unit on the same neural network processor. When the successor computing unit of a given computing unit is not located on the same neural network processor, i.e., when data communication is required between computing units on different neural network processors, data can be transferred across cards via external buses such as PCIe. In this case, the cross-card transfer sub-instruction in the second sub-instruction is executed. For example, the hardware control unit can configure the cross-card DMA engine to send data to the buffer address of the corresponding computing unit on the target neural network processor. When generating multiple sets of second instruction sequences, the decision of which transfer sub-instruction to configure in each set of second instruction sequences can be determined based on whether the computing unit and its successor computing unit are located in the same neural network processor. This decision is completed and solidified in the instructions during the loading stage.

[0064] Therefore, by subdividing the second sub-instruction into intra-card transmission sub-instructions and inter-card transmission sub-instructions, the corresponding transmission channels and addressing modes can be configured according to the physical locations of the computing units of both communicating parties, so that the same set of instruction sequences can simultaneously adapt to both intra-processor transmission and inter-processor transmission scenarios.

[0065] Figure 4 This is a schematic diagram illustrating the execution of each group of second instruction sequences according to an exemplary embodiment.

[0066] like Figure 4As shown, when executing each set of second instruction sequences, the hardware control unit first executes the first sub-instruction, namely, polling to detect the ready flag. The hardware control unit polls and reads the synchronization address corresponding to the computing unit, checking whether the ready flag has been written by the predecessor computing unit. After detecting the ready flag, the hardware control unit continues to execute subsequent operations to ensure that the data of the predecessor computing unit has been transmitted and is ready. Subsequently, the hardware control unit executes the second sub-instruction to transfer the data of the computing unit to the target address of the successor computing unit. The second sub-instruction is divided into two types according to the physical location relationship between the computing unit and the successor computing unit: intra-card transfer sub-instruction and cross-card transfer sub-instruction. When the computing unit and the successor computing unit are located on the same neural network processor, the intra-card transfer sub-instruction is executed, and the data transfer is completed through the internal transmission channel of the processor. When they are located on different neural network processors, the cross-card transfer sub-instruction is executed. If the two neural network processors are located in the same physical node, the data transfer can be completed through the PCIe bus, and the target address uses the PCIe BAR mapped address; if the two neural network processors are located in different physical nodes, the data transfer can be completed through the RDMA network, and the target address uses the device physical address, while the synchronization address uses the chip-level global address. Finally, the hardware control unit executes the third sub-instruction, writing a ready flag to the synchronization address corresponding to the subsequent computing unit, notifying the subsequent computing unit that data transmission has been completed. The hardware control unit of the subsequent computing unit can detect this ready flag when executing its own first sub-instruction, thus continuing to execute subsequent operations. Through this hierarchical synchronization mechanism, the hardware control units of each computing unit can complete data transmission between each other.

[0067] In some embodiments, each group of second instruction sequences further includes an execution flag bit, which is used to indicate whether each sub-instruction in the group of second instruction sequences is executed by the hardware control unit.

[0068] In the example, when generating multiple sets of second instruction sequences, a general template can be used to uniformly generate a complete structure containing all sub-instructions for each set of second instruction sequences. However, not every computing unit needs to execute all sub-instructions in every execution step. For example, when an execution step only involves data transfer within the same neural network processor, the cross-card transfer sub-instructions in that set of second instruction sequences do not actually need to be executed. To flexibly control the execution of each sub-instruction without changing the instruction structure, an execution flag can be set in each set of second instruction sequences to determine whether each sub-instruction needs to be executed. This execution flag is determined during the loading phase based on the actual communication needs of each computing unit in the current execution step. Then, steps that do not need to be executed can be removed based on this execution flag to compress the instruction sequence.

[0069] Therefore, when the hardware control unit executes each set of second instruction sequences during the operation phase, it can execute only the sub-instructions marked as needing to be executed, thereby reducing redundant data communication operations and improving execution efficiency.

[0070] In some embodiments, the second instruction buffer further includes header information, which is used to record the starting address of multiple sets of second instruction sequences and the total number of sets of second instruction sequences, and to control the hardware control unit of each computing unit to execute the corresponding first instruction sequence, so that when the placeholder is executed, it jumps to the entry address of the second instruction buffer to execute multiple sets of second instruction sequences, so as to perform data communication between computing units, including: reading the header information to obtain the total number of sets and the starting address; and executing multiple sets of second instruction sequences sequentially according to the starting address until all the second instruction sequences corresponding to the total number of sets are executed.

[0071] In the example, the second instruction buffer, in addition to storing multiple sets of second instruction sequences, also includes a header at its starting position. This header provides the hardware control unit with the global parameters required to execute the second instruction sequences, allowing the hardware control unit to know exactly where to begin execution and how many sets of instructions need to be executed after jumping to the second instruction buffer. The total number of sets recorded in the header reflects the number of valid second instruction sequences, while the starting address indicates the location of the first set of second instruction sequences in the second instruction buffer. When the hardware control unit encounters a placeholder during the execution of the first instruction sequence, it jumps to the second instruction buffer based on the entry address recorded in the placeholder. First, it reads the header to obtain the total number of sets and the starting address. Then, starting from the position pointed to by the starting address, the hardware control unit reads and executes the second instruction sequences one by one, moving to the next set after each set is completed, while simultaneously counting the number of executed sets until the number of executed sets reaches the total number recorded in the header, indicating that all communication operations corresponding to this communication operator have been completed. Afterward, the hardware control unit returns to the position after the placeholder in the first instruction sequence and continues executing subsequent instructions.

[0072] Therefore, by setting header information containing the starting address and the total number of groups in the second instruction buffer, the hardware control unit can independently determine the start and end points of execution by reading the header information after jumping to the second instruction buffer, and thus independently complete the execution of the entire second instruction sequence.

[0073] Figure 5 This is a schematic diagram illustrating the data communication process of a full collection operation according to an exemplary embodiment.

[0074] like Figure 5As shown, computing units 0, 1, 2, and 3 form a predecessor-successor relationship according to a ring transmission path. Each computing unit's buffer includes four data slots, corresponding to the respective data blocks of the four computing units. In the initial state, each computing unit only holds its own local data; that is, computing unit 0 holds data block A, computing unit 1 holds data block B, computing unit 2 holds data block C, and computing unit 3 holds data block D, with the remaining slots empty.

[0075] In the first round of transmission, each computing unit executes the first set of the second instruction sequence, sending its local data to the subsequent computing unit. Computing unit 0 sends data block A to computing unit 1, computing unit 1 sends data block B to computing unit 2, computing unit 2 sends data block C to computing unit 3, and computing unit 3 sends data block D to computing unit 0. After the first round, each computing unit holds two data blocks: computing unit 0 holds data blocks A and D, computing unit 1 holds data blocks A and B, computing unit 2 holds data blocks B and C, and computing unit 3 holds data blocks C and D.

[0076] In the second round of transmission, each computing unit executes the second set of second instruction sequences. First, after polling and detecting the ready flag, it forwards the data blocks received from the preceding computing unit in the previous round to the subsequent computing unit. Computing unit 0 forwards data block D received in the previous round to computing unit 1; computing unit 1 forwards data block A received in the previous round to computing unit 2; computing unit 2 forwards data block B received in the previous round to computing unit 3; and computing unit 3 forwards data block C received in the previous round to computing unit 0. After the second round, each computing unit holds three data blocks: computing unit 0 holds data blocks A, C, and D; computing unit 1 holds data blocks A, B, and D; computing unit 2 holds data blocks A, B, and C; and computing unit 3 holds data blocks B, C, and D.

[0077] In the third round of transmission, each computing unit executes the third set of second instruction sequences. After polling and detecting the ready flag, it forwards the data block received from the preceding computing unit in the previous round to the succeeding computing unit. Computing unit 0 forwards data block C to computing unit 1, computing unit 1 forwards data block D to computing unit 2, computing unit 2 forwards data block A to computing unit 3, and computing unit 3 forwards data block B to computing unit 0. After the third round, all four computing units hold complete data blocks A, B, C, and D. That is, the four computing units have completed the collection of all data through three rounds of transmission, where each round of transmission corresponds to the execution of one set of instructions from multiple sets of second instruction sequences. In each round, each computing unit simultaneously sends one data block to the succeeding computing unit and receives one data block from the preceding computing unit. The hardware control unit of each computing unit achieves step-by-step synchronization through polling and writing of the ready flag, ensuring that data is only used by subsequent steps after transmission is completed.

[0078] Understandable, Figure 5 The illustration uses only four computing units as an example; however, the embodiments of this disclosure are not limited to this, and the number of computing units may not be limited to four.

[0079] Embodiments of this disclosure also provide a collective communication system.

[0080] Figure 6 This is a schematic block diagram illustrating a collection communication system 600 according to an exemplary embodiment.

[0081] In some embodiments, the aggregate communication system 600 may include a structure determination module 601, a model loading module 602, a first sequence generation module 603, a buffer allocation module 604, a second sequence generation module 605, and an instruction execution module 606.

[0082] The structure determination module 601 is configured to be a hardware topology based on multiple computing units, and determines the predecessor computing unit and successor computing unit corresponding to each computing unit.

[0083] The model loading module 602 is configured to load a pre-generated static graph model that describes the operator execution relationship, and traverse multiple operator nodes in the static graph model, wherein the multiple operator nodes include at least one communication operator and at least one computation operator.

[0084] The first sequence generation module 603 is configured to, for each operator node, generate computation instructions based on the computation operator in response to determining that the operator node is a computation operator, and write the computation instructions into the corresponding first instruction buffer of each computation unit. The first instruction buffer is used to store the first instruction sequence executed by the hardware control unit.

[0085] The buffer allocation module 604 is configured to allocate a second instruction buffer for each computational unit for each operator node in response to determining that the operator node is a communication operator, and to create a placeholder in the first instruction sequence pointing to the entry address of the second instruction buffer.

[0086] The second sequence generation module 605 is configured to generate multiple sets of second instruction sequences for each computing unit for each communication operator after traversing multiple operator nodes. The multiple sets of second instruction sequences are written into the second instruction buffer of the computing unit to control the data communication between the computing unit and its predecessor and successor computing units.

[0087] The instruction execution module 606 is configured to control the hardware control unit of each computing unit to execute the corresponding first instruction sequence, so that when the placeholder is executed, it jumps to the entry address of the second instruction buffer to execute multiple sets of second instruction sequences to perform data communication between the computing units.

[0088] It should be understood that Figure 6 The structure determination module 601, model loading module 602, first sequence generation module 603, buffer allocation module 604, second sequence generation module 605, and instruction execution module 606 shown can respectively correspond to, as follows: Figure 1 The operations of steps S101, S102, S103, S104, S105, and S106 are shown. Therefore, the details of each aspect will not be elaborated here.

[0089] In some embodiments, before generating multiple sets of second instruction sequences for each computing unit for each communication operator after traversal is completed, the aggregate communication system 600 may further include a target address determination module 607 and a synchronization address determination module 608. The target address determination module 607 may be configured to determine, for each computing unit, the target address for the computing unit to send data to its successor computing unit. The synchronization address determination module 608 may be configured to determine the synchronization address for the computing unit to receive a ready flag, wherein the synchronization address is a pre-set address, and the predecessor computing unit of the computing unit writes a ready flag to the synchronization address after completing data transmission.

[0090] In some embodiments, the second instruction buffer further includes header information, which records the starting addresses of multiple sets of second instruction sequences and the total number of sets of second instruction sequences. The instruction execution module 606 may include a header information reading module 6061 and a second sequence execution module 6062. The header information reading module 6061 may be configured to read the header information to obtain the total number of sets and the starting address. The second sequence execution module 6062 may be configured to execute multiple sets of second instruction sequences sequentially according to the starting address until all second instruction sequences corresponding to the total number of sets have been executed.

[0091] While specific functions have been discussed above with reference to specific modules, it should be noted that the functions of the modules discussed herein can be divided into multiple modules, and / or at least some functions of multiple modules can be combined into a single module. The specific actions performed by the modules discussed herein include the specific module itself performing the action, or alternatively, the specific module calling or otherwise accessing another component or module that performs the action (or performs the action in conjunction with the specific module). Therefore, a specific module performing an action can include the specific module performing the action itself and / or another module that performs the action, called or otherwise accessed by the specific module.

[0092] It should also be understood that the above regarding Figure 6 The described modules can be implemented in hardware or in hardware in combination with software and / or firmware. For example, these modules can be implemented as computer program code / instructions configured to execute in one or more processors and stored in a computer-readable storage medium. Alternatively, these modules can be implemented as hardware logic / circuit. For example, these modules can be implemented together in a System on Chip (SoC). An SoC may include an integrated circuit chip (which includes one or more components in a processor (e.g., a Central Processing Unit (CPU), microcontroller, microprocessor, digital signal processor (DSP), etc.), memory, one or more communication interfaces, and / or other circuitry) and may optionally execute received program code and / or include embedded firmware to perform functions.

[0093] According to one aspect of this disclosure, a computer device is also provided, including a memory, a processor, and a computer program stored in the memory. The processor is configured to execute the computer program to implement the steps of any of the method embodiments described above.

[0094] According to one aspect of this disclosure, a non-transitory computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of any of the method embodiments described above.

[0095] According to one aspect of this disclosure, a computer program product is also provided, which includes a computer program that, when executed by a processor, implements the steps of any of the method embodiments described above.

[0096] In the following text, combined with Figure 7 Illustrative examples describing such computer devices, non-transitory computer-readable storage media, and computer program products.

[0097] Figure 7 An example configuration of a computer device 700 that can be used to implement the methods described herein is shown.

[0098] Computer device 700 can be a variety of different types of devices. Examples of computer device 700 include, but are not limited to: desktop computers, server computers, laptop or netbook computers, mobile devices (e.g., tablet computers, cellular or other wireless phones (e.g., smartphones), notebook computers, mobile stations), wearable devices (e.g., glasses, watches), entertainment devices (e.g., entertainment appliances, set-top boxes communicatively coupled to a display device, game consoles), televisions or other display devices, automotive computers, and so on.

[0099] Computer device 700 may include at least one processor 702, memory 704, communication interfaces(s) 706, display device 708, other I / O devices 710, and one or more mass storage devices 712 capable of communicating with each other, such as via system bus 714 or other suitable connections.

[0100] Processor 702 may be a single processing unit or multiple processing units, and all processing units may include a single or multiple computing cores or multiple cores. Processor 702 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and / or any device that manipulates signals based on operating instructions. Among other capabilities, processor 702 may be configured to acquire and execute computer-readable instructions stored in memory 704, mass storage device 712, or other computer-readable media, such as program code of operating system 716, program code of application program 718, program code of other program 720, etc.

[0101] Memory 704 and mass storage device 712 are examples of computer-readable storage media for storing instructions that are executed by processor 702 to perform the various functions described above. For example, memory 704 can generally include both volatile and non-volatile memory (e.g., RAM, ROM, etc.). Furthermore, mass storage device 712 can generally include hard disk drives, solid-state drives, removable media, including external and removable drives, memory cards, flash memory, floppy disks, optical disks (e.g., CDs, DVDs), storage arrays, network-attached storage, storage area networks, etc. Both memory 704 and mass storage device 712 can be collectively referred to herein as memory or computer-readable storage media, and can be non-transitory media capable of storing computer-readable, processor-executable program instructions as computer program code, which can be executed by processor 702 as a specific machine configured to perform the operations and functions described in the examples herein.

[0102] Multiple programs may be stored on mass storage device 712. These programs include operating system 716, one or more application programs 718, other programs 720, and program data 722, and they may be loaded into memory 704 for execution. Examples of such application programs or program modules may include computer program logic (e.g., computer program code or instructions) of the methods described herein and / or other embodiments described herein.

[0103] Although Figure 7 The data is illustrated as being stored in memory 704 of computer device 700, but the operating system 716, application program 718, other programs 720, and program data 722, or portions thereof, may be implemented using any form of computer-readable medium accessible by computer device 700. As used herein, "computer-readable medium" includes at least two types of computer-readable media: computer-readable storage media and communication media.

[0104] Computer-readable storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, DVD, or other optical storage devices, magnetic cassettes, magnetic tapes, disk storage devices or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by computer devices. In contrast, communication media can embody computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms. Computer-readable storage media as defined herein do not include communication media.

[0105] One or more communication interfaces 706 are used for exchanging data with other devices, such as via a network, direct connection, etc. Such communication interfaces can be one or more of the following: any type of network interface (e.g., a network interface card (NIC)), wired or wireless (such as IEEE 802.11 Wireless LAN (WLAN)) wireless interface, Wi-MAX interface, Ethernet interface, Universal Serial Bus (USB) interface, cellular network interface, Bluetooth. TM Interfaces include near field communication (NFC) interfaces. Communication interface 706 facilitates communication across various network and protocol types, including wired networks (e.g., LAN, cable, etc.) and wireless networks (e.g., WLAN, cellular, satellite, etc.), the Internet, etc. Communication interface 706 can also provide communication with external storage devices (not shown) such as storage arrays, network-attached storage, storage area networks, etc.

[0106] In some examples, a display device 708, such as a monitor, may be included for displaying information and images to the user. Other I / O devices 710 may be devices that receive various inputs from the user and provide various outputs to the user, and may include touch input devices, gesture input devices, cameras, keyboards, remote controls, mice, printers, audio input / output devices, and so on.

[0107] The technologies described herein can be supported by these various configurations of computer device 700, and are not limited to specific examples of the technologies described herein. For example, the functionality can also be implemented wholly or partially on a “cloud” using a distributed system. A cloud includes and / or represents a platform for resources. The platform abstracts the underlying functionality of the cloud’s hardware (e.g., servers) and software resources. Resources may include applications and / or data that can be used when performing computational processing on a server remote from computer device 700. Resources may also include services provided via the Internet and / or via subscriber networks such as cellular or Wi-Fi networks. The platform can abstract resources and functionality to connect computer device 700 to other computer devices. Therefore, the implementation of the functionality described herein can be distributed throughout the cloud. For example, the functionality can be implemented partly on computer device 700 and partly through a platform that abstracts the functionality of the cloud.

Claims

1. A collective communication method applied to a system comprising multiple neural network processors, each neural network processor including at least one computing unit, each computing unit including a hardware control unit, characterized in that, The method includes: Based on the hardware topology of the multiple computing units, the predecessor computing unit and the successor computing unit corresponding to each computing unit are determined. Load a pre-generated static graph model to describe the operator execution relationship, and traverse multiple operator nodes in the static graph model, wherein the multiple operator nodes include at least one communication operator and at least one computation operator; For each operator node: In response to determining that the operator node is a computation operator, a computation instruction is generated based on the computation operator, and the computation instruction is written into the first instruction buffer corresponding to each computation unit. The first instruction buffer is used to store the first instruction sequence executed by the hardware control unit. In response to determining that the operator node is a communication operator, a second instruction buffer is allocated to each computing unit, and a placeholder pointing to the entry address of the second instruction buffer is created in the first instruction sequence; After traversing the plurality of operator nodes, for each communication operator, multiple sets of second instruction sequences are generated for each computing unit. These multiple sets of second instruction sequences are written into the second instruction buffer of the computing unit to control data communication between the computing unit and its predecessor and successor computing units. The hardware control unit that controls each computing unit executes the corresponding first instruction sequence, so that when the placeholder is executed, it jumps to the entry address of the second instruction buffer to execute the multiple sets of second instruction sequences to perform data communication between the computing units.

2. The method according to claim 1, characterized in that, Before generating multiple sets of second instruction sequences for each computational unit for each communication operator after the traversal is completed, the method further includes: For each computing unit, determine the target address from which the computing unit sends data to its successor computing unit; and The synchronization address for receiving the ready flag of the computing unit is determined, wherein the synchronization address is a pre-set address, and the predecessor computing unit of the computing unit writes the ready flag to the synchronization address after completing the data transmission.

3. The method according to claim 2, characterized in that, For each computing unit, each set of second instruction sequences includes a first sub-instruction, a second sub-instruction, and a third sub-instruction. The first sub-instruction is used to determine whether the computing unit has detected the ready identifier from the synchronization address. The second sub-instruction is used to transfer the data of the computing unit to the target address of the successor computing unit of the computing unit. The third sub-instruction is used to write the ready identifier to the synchronization address corresponding to the successor computing unit of the computing unit after the second sub-instruction is completed.

4. The method according to claim 3, characterized in that, The second sub-instruction includes an intra-card transfer sub-instruction and an inter-card transfer sub-instruction. The intra-card transfer sub-instruction is used for data communication between computing units within the same neural network processor, and the inter-card transfer sub-instruction is used for data communication between computing units of different neural network processors.

5. The method according to claim 4, characterized in that, Each group of second instruction sequences also includes an execution flag, which indicates whether each sub-instruction in the group of second instruction sequences is executed by the hardware control unit.

6. The method according to any one of claims 1 to 5, characterized in that, The second instruction buffer also includes header information, which records the starting address of the plurality of second instruction sequences and the total number of the plurality of second instruction sequences. The hardware control unit controlling each computing unit executes the corresponding first instruction sequence, so that when the placeholder is executed, it jumps to the entry address of the second instruction buffer to execute the plurality of second instruction sequences, thereby performing data communication between the computing units, including: Read the header information to obtain the total number of groups and the starting address; and The multiple sets of second instruction sequences are executed sequentially according to the starting address until all the second instruction sequences corresponding to the total number of sets have been executed.

7. A collective communication system, the system comprising multiple neural network processors, each neural network processor including at least one computing unit, each computing unit including a hardware control unit, characterized in that, The system includes: The structure determination module is configured to determine the predecessor computing unit and the successor computing unit corresponding to each computing unit based on the hardware topology of the plurality of computing units. The model loading module is configured to load a pre-generated static graph model that describes the operator execution relationship, and traverse multiple operator nodes in the static graph model, wherein the multiple operator nodes include at least one communication operator and at least one computation operator; For each operator node: The first sequence generation module is configured to, in response to determining that the operator node is a computation operator, generate computation instructions based on the computation operator and write the computation instructions into a first instruction buffer corresponding to each computation unit, wherein the first instruction buffer is used to store a first instruction sequence executed by the hardware control unit; The buffer allocation module is configured to allocate a second instruction buffer to each computing unit in response to determining that the operator node is a communication operator, and to create a placeholder in the first instruction sequence pointing to the entry address of the second instruction buffer; The second sequence generation module is configured to, after traversing the plurality of operator nodes, generate multiple sets of second instruction sequences for each computing unit for each communication operator, wherein the multiple sets of second instruction sequences are written into the second instruction buffer of the computing unit for controlling data communication between the computing unit and its predecessor and successor computing units; and The instruction execution module is configured to control the hardware control unit of each computing unit to execute the corresponding first instruction sequence, so that when the placeholder is executed, it jumps to the entry address of the second instruction buffer to execute the multiple sets of second instruction sequences to perform data communication between the computing units.

8. A computer device, characterized in that, The computer device includes: At least one processor; A memory having a computer program stored thereon, wherein, when executed by the at least one processor, the computer program causes the at least one processor to perform the method of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the method of any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, causes the processor to perform the method of any one of claims 1-6.

Images