Computing devices, synchronization methods, electronic devices, and storage media

Abstract

The present disclosure provides a computing device, a synchronization method for a computing device, an electronic device, and a computer-readable storage medium. The computing device includes a plurality of compute clusters partitioned into a plurality of broadcast groups, and one or more intra-cluster interfaces respectively located at each compute cluster of the plurality of compute clusters, wherein a compute cluster in each broadcast group, when acting as a consumer compute cluster, is configured to send a broadcast synchronization barrier message to the intra-cluster interface when a data space for all producer compute clusters in the broadcast group is ready, the broadcast synchronization barrier message including a broadcast indicator and an identifier of the broadcast group, and the intra-cluster interface is configured to unfold the broadcast synchronization barrier message into a synchronization barrier message for each producer compute cluster in the broadcast group, and send each synchronization barrier message to the respective producer compute cluster.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to the field of processors, and more particularly, to a computing device, a synchronization method for the computing device, an electronic device, and a computer-readable storage medium. BACKGROUND

[0002] Currently, the structure of various computing devices is becoming more and more complex, and the hardware hierarchy inside the computing devices is becoming more and more numerous, and the synchronization problem of data interaction between various hierarchies needs to be solved. For example, for a computing device containing multiple computing clusters, if a computing cluster that generates data needs to write the generated data to the same computing cluster for the computing cluster to use the data to perform subsequent processing, the computing cluster that uses the data needs to inform the computing cluster that generates the data whether it is ready to have space to receive the generated data.

[0003] In the prior art, a synchronization barrier mechanism based on thread wrap is used, and the computing cluster that uses the data sends a synchronization barrier message to each computing cluster that generates data one by one to inform whether it is ready to have space to receive the data generated by the computing cluster. However, this synchronization barrier mechanism will cause the computing cluster that uses the data to consume a large number of instruction cycles, especially in the case of broadcasting. SUMMARY

[0004] To solve the above problem, the present disclosure provides a scheme for space synchronization by a computing cluster that uses data sending a broadcast synchronization barrier message, and being unfolded by an intra-cluster interface of the computing cluster, which greatly reduces the instruction cycles required by the computing cluster to send the synchronization barrier.

[0005] According to one aspect of the present disclosure, a computing device is provided. The computing device includes a plurality of computing clusters divided into a plurality of broadcast groups, and one or more intra-cluster interfaces respectively located in each of the plurality of computing clusters, wherein a computing cluster in each broadcast group, when acting as a consumer computing cluster, is configured to send a broadcast synchronization barrier message to the intra-cluster interface when data spaces for all producer computing clusters in the broadcast group are ready, the broadcast synchronization barrier message including a broadcast indicator and an identifier of the broadcast group, and the intra-cluster interface is configured to unfold the broadcast synchronization barrier message into a synchronization barrier message for each producer computing cluster in the broadcast group, and send each synchronization barrier message to the corresponding producer computing cluster.

[0006] In some implementations, the intra-cluster interface is configured to determine the corresponding producer computing cluster based on the identifier of the broadcast group in the broadcast synchronization barrier message.

[0007] In some implementations, the intra-cluster interface includes a plurality of mask registers, each mask register corresponding to one of the plurality of broadcast groups, the mask registers being used to indicate computing clusters of the plurality of computing clusters that belong to the broadcast group, and wherein the intra-cluster interface is configured to determine a barrier register corresponding to the broadcast group based on the identifier of the broadcast group in the broadcast synchronization barrier message, and determine a respective producer computing cluster based on the barrier register corresponding to the broadcast group.

[0008] In some implementations, the consumer computing cluster is further configured to: determine whether data spaces for all producer computing clusters in the broadcast group are ready; in response to data spaces for all producer computing clusters in the broadcast group being ready, generate the broadcast synchronization barrier message; and in response to data spaces for some producer computing clusters in the broadcast group not being ready, continue to wait for the spaces for the some producer computing clusters to be ready.

[0009] In some implementations, each producer computing cluster in the broadcast group is configured to: determine whether a synchronization barrier message from all other computing clusters in the broadcast group is received; in response to receiving a synchronization barrier message from all other computing clusters in the broadcast group, send generated data to the other computing clusters; and in response to not receiving a synchronization barrier message from all other computing clusters in the broadcast group, continue to wait.

[0010] In some implementations, the intra-cluster interface is configured to send each synchronization barrier message to a respective producer computing cluster in sequence through an inter-cluster interface.

[0011] According to another aspect of the disclosure, there is provided a synchronization method for a computing device including a plurality of computing clusters and one or more intra-cluster interfaces that are located at each computing cluster of the plurality of computing clusters, wherein the plurality of computing clusters are divided into a plurality of broadcast groups. The synchronization method includes: by one computing cluster in each broadcast group as a consumer computing cluster, sending a broadcast synchronization barrier message to the intra-cluster interface when data spaces for all producer computing clusters in the broadcast group are ready, the broadcast synchronization barrier message including a broadcast indicator and an identifier of the broadcast group; and by the intra-cluster interface, unfolding the broadcast synchronization barrier message into a synchronization barrier message for each producer computing cluster in the broadcast group, and sending each synchronization barrier message to a respective producer computing cluster.

[0012] In some implementations, the unwinding, by the intra-cluster interface, of the broadcast synchronization barrier message into synchronization barrier messages for each producer cluster in the broadcast group includes determining, by the intra-cluster interface, respective producer clusters based on identifiers of the broadcast group in the broadcast synchronization barrier message.

[0013] In some implementations, the intra-cluster interface includes a plurality of mask registers, each mask register corresponding to one of the plurality of broadcast groups, the mask registers being used to indicate the computing clusters in the plurality of computing clusters that belong to the broadcast group, and wherein the determining, by the intra-cluster interface, of respective producer clusters based on identifiers of the broadcast group in the broadcast synchronization barrier message includes determining, by the intra-cluster interface, a barrier register corresponding to the broadcast group based on the identifiers of the broadcast group in the broadcast synchronization barrier message, and determining the respective producer clusters based on the barrier register corresponding to the broadcast group.

[0014] In some implementations, the synchronization method further includes determining, by the consumer cluster, whether data space for all producer clusters in the broadcast group has been prepared; in response to the data space for all producer clusters in the broadcast group having been prepared, generating the broadcast synchronization barrier message; and in response to the data space for some producer clusters in the broadcast group not having been prepared, continuing to wait for the space for the some producer clusters to be prepared.

[0015] In some implementations, the synchronization method further includes determining, by each producer cluster in the broadcast group, whether a synchronization barrier message has been received from all other clusters in the broadcast group; in response to the synchronization barrier message having been received from all other clusters in the broadcast group, sending the generated data to the other clusters; and in response to the synchronization barrier message not having been received from all other clusters in the broadcast group, continuing to wait.

[0016] According to yet another aspect of the present disclosure, there is provided an electronic device, comprising: a memory, which non-transitorily stores computer-executable instructions; and a processor, which is configured to run the computer-executable instructions; wherein the computer-executable instructions, when run by the processor, implement the synchronization method as described above.

[0017] According to still another aspect of the present disclosure, there is provided a computer-readable storage medium, which stores computer program codes, the computer program codes, when run, performing the synchronization method as described above. BRIEF DESCRIPTION OF DRAWINGS

[0018] The present disclosure will be better understood with reference to the detailed description of specific embodiments of the disclosure given by way of example only, in conjunction with the following drawings.

[0019] Figure 1A An exemplary structural diagram of a computing device is shown.

[0020] Figure 1A A schematic diagram of a computing device connecting multiple compute clusters through a multi-level inter-cluster interface is shown.

[0021] Figure 2 A detailed structural diagram of a compute cluster is shown.

[0022] Figure 3 An exemplary diagram of a barrier register in an intra-cluster interface according to an embodiment of the present invention is shown.

[0023] Figure 4 A flowchart of a synchronization method for a computing device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0024] Preferred embodiments of the present disclosure will be described herein below with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0025] The term "comprising" and variations thereof as used herein are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements is not necessarily limited to those elements, but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Unless otherwise specified, the term "or" as used herein is intended to mean "and / or". The term "based on" means "based, at least in part, on". The terms "one embodiment" and "some embodiments" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", and the like, do not imply importance or significance to one or the other, but merely distinguish different features.

[0026] Figure 1A An exemplary structural diagram of a computing device 100 is shown. As shown in Figure 1A The computing device 100 can include multiple compute clusters 110 Figure 1A Eight compute clusters 110-1, 110-2,..., 110-8 are schematically shown in

[0027] The compute clusters 110 in the computing device 100 can be further divided into multiple broadcast groups 120, as Figure 1AThe eight compute clusters 110 are divided into two broadcast groups 120-1 and 120-2, where compute clusters 110-1, 110-2, 110-5, and 110-6 belong to broadcast group 120-1, and compute clusters 110-3, 110-4, 110-7, and 110-8 belong to broadcast group 120-2. In a broadcast group, a compute cluster 110 that produces data needs to send the produced data to all other compute clusters in the same broadcast group for consumption by the other compute clusters. Here, the division of compute clusters into broadcast groups can be based on, for example, physical distribution of the compute clusters or application requirements.

[0028] Each compute cluster 110 can include one or more intra-cluster interfaces 112 (only one intra-cluster interface 112 is shown schematically for each compute cluster 110 in FIG. 1). The various compute clusters 110, and more specifically, the intra-cluster interfaces 112 of the various compute clusters 110, are also connected through an inter-cluster interface 130. Each compute cluster 110 can exchange data and instructions with other compute clusters 110 through its own intra-cluster interface 112 and inter-cluster interface 130. The intra-cluster interface 112 can be a sub-module of the compute cluster 110 or a separate hardware module attached to the compute cluster 110.

[0029] The inter-cluster interface 130 can include one or more levels of hardware interfaces. Figure 1B A schematic diagram of a computing device 100 that connects multiple compute clusters 110 through a multi-level inter-cluster interface 130 is shown. As Figure 1B shown, the entire computing device 100 can be composed of multiple device parts 140, each of which can include multiple dies 150, each of which can include multiple compute clusters 110. Figure 1B A three-level inter-cluster interface 130 is shown schematically, i.e., a first level inter-cluster interface 130-1 that connects the intra-cluster interfaces 112 of the various compute clusters 110, sometimes also referred to as a network-on-chip (NOC); a second level inter-cluster interface 130-2 that connects the various dies 150, sometimes also referred to as a D2D interface, and a third level inter-cluster interface 130-3 that connects the various device parts 140, sometimes also referred to as a P2P interface.

[0030] Note that, Figure 1B The computing device 100 is shown schematically to include two device parts 140, each of which is shown to include two dies 150, and each of which is shown to include two compute clusters 110. However, those skilled in the art will appreciate that, Figure 1BThe number shown in the figure is merely exemplary, each computing device 100 can include more or less device components 140, each device component 140 can include more or less wafers 150, and / or each wafer 150 can include more or less computing clusters 110.

[0031] In addition, the division of the broadcast groups 120 does not necessarily have a certain correspondence relationship with the computing clusters 110, wafers 150 and device components 140. For example, the number of computing clusters 110 in one broadcast group can be greater than the number of computing clusters 110 in one wafer 150, which can reach the order of magnitude of nodes containing multiple device components 140 (such as 8 devices). In addition, one computing cluster 110 can be located in multiple broadcast groups 120, that is, different broadcast groups 120 can contain the same computing cluster 110.

[0032] In this article, the computing cluster 110 that generates data is also called the producer computing cluster, and the computing cluster 110 that consumes data is also called the consumer computing cluster. In addition, if a computing cluster 110 itself both generates data and uses the data it generates itself, then the computing cluster 110 is both a producer computing cluster and a consumer computing cluster.

[0033] Figure 2 A detailed structure diagram of the computing cluster 110 is shown. As shown in Figure 2 As shown in the figure, in addition to the intra-cluster interface 112, the computing cluster 110 can also include one or more computing units (CUs) 114, and each computing unit 114 can include one or more execution units (EUs) 116. One or more thread bundles can run on each execution unit 116, but one thread bundle only runs on one execution unit 116. Thread bundles are a basic scheduling unit frequently used in GPUs for parallel processing, and each thread bundle can contain a certain number of threads, such as 32 or 16, etc., to run various different instructions.

[0034] In the computing device 100 shown in FIG. 1, in a broadcast group 120 (such as the broadcast group 120-1), assume that the computing cluster 110-6 is a consumer computing cluster, and three computing clusters 110-1, 110-2 and 110-5 are producer computing clusters, all of which write the generated data to the consumer computing cluster 110-6. Before writing, the producer computing clusters need to know whether the consumer computing cluster 110-6 has prepared the required data space for them. In a conventional thread bundle-based synchronization barrier mechanism, the consumer computing cluster 110-6 can send a synchronization barrier message to each of the producer computing clusters 110-1, 110-2 and 110-5 respectively to inform the consumer computing cluster 110-6 that it has prepared the required data space for the corresponding producer computing cluster 110-1, 110-2 and 110-5, i.e., the data space is ready.

[0035] However, in this way, the consumer computing cluster 110-6 needs to send a synchronization barrier message to each of the other computing clusters in the broadcast group 120-1, such as bar_to_spc_1, bar_to_spc_2, bar_to_spc_5, etc., thereby consuming 3 instruction cycles of the computing cluster 110-6. For a broadcast group containing N computing clusters, N-1 instruction cycles are required for space synchronization. If the number N of computing clusters contained in the broadcast group is large, the consumption of instruction cycles will become very large. In the computing device 100, the computing clusters 110 undertake complex computation tasks, and the excessive consumption of instruction cycles for space synchronization will inevitably reduce the available instruction cycles for other computation tasks, resulting in a decline in the overall efficiency of the computing device.

[0036] To solve the above problems, the present disclosure provides a method in which a consumer computing cluster sends a single broadcast synchronization barrier message, and then expands the broadcast synchronization barrier message into individual synchronization barrier messages through an intra-cluster interface, thereby saving the instruction cycles required for space synchronization between computing clusters.

[0037] Specifically, the consumer computing cluster 110-6 can be configured to determine whether the data space for all of the producer computing clusters 110-1, 110-2 and 110-5 in the broadcast group 120-1 is ready. If it is determined that the data space for part of the producer computing clusters (such as the computing cluster 110-1) in the broadcast group 120-1 is not ready, the consumer computing cluster 110-6 continues to wait for the space of the part of the producer computing clusters to be ready. If it is determined that the data space for all of the producer computing clusters 110-1, 110-2 and 110-5 in the broadcast group 120-1 is ready, the consumer computing cluster 110-6 can generate a broadcast synchronization barrier message, and send the broadcast synchronization barrier message to the intra-cluster interface 112 of the consumer computing cluster 110-6.

[0038] That is, instead of sending a synchronization barrier to each producer cluster as soon as the data space of that producer cluster is ready, the consumer cluster 110-6 waits until all producer clusters' spaces are ready, and then generates and sends the broadcast synchronization barrier message.

[0039] Since the intra-cluster interface 112 can be used to communicate with all clusters 110 in the computing device 100, in order for the intra-cluster interface 112 to accurately process the broadcast synchronization barrier message, the broadcast synchronization barrier message should at least contain a broadcast indicator and an identifier of the broadcast group 120-1. For example, the broadcast synchronization barrier message can be denoted as bar broadcast groupID, where "bar" indicates that this is a synchronization barrier, "broadcast" as a broadcast indicator, indicating to the intra-cluster interface 112 that this is a broadcast message, and "groupID" to indicate the identifier of the broadcast group (e.g., the broadcast group 120-1).

[0040] The intra-cluster interface 112 is configured to expand the broadcast synchronization barrier message into a synchronization barrier message for each producer cluster in the broadcast group 120-1, and send each synchronization barrier message to the corresponding producer cluster.

[0041] Specifically, the intra-cluster interface 112 can determine the corresponding producer clusters, such as the producer clusters 110-1, 110-2 and 110-5, based on the identifier of the broadcast group (groupID) in the broadcast synchronization barrier message.

[0042] In some embodiments, the intra-cluster interface 112 can utilize mask registers to determine the producer clusters in the broadcast group. Figure 3 An exemplary schematic diagram of the barrier registers in the intra-cluster interface 112 according to an embodiment of the present application is shown. As shown in Figure 3 As shown in the above diagram, the intra-cluster interface 112 can include a plurality of mask registers 1122( Figure 3 two barrier registers 1122-1 and 1122-2 are exemplarily shown in the above diagram), each mask register 1122 corresponding to one of the plurality of broadcast groups 120, for indicating the clusters 110 in the computing device 100 that belong to the broadcast group. Therefore, the number of bits contained in each mask register 140 should be at least equal to the number of clusters 110 contained in the computing device 100. More specifically, in the mask register 1122 for one broadcast group 120, the clusters 110 that belong to the broadcast group 120 and the clusters 110 that do not belong to the broadcast group 120 should be indicated differently, for example, using 1 and 0, respectively.

[0043] As shown in Figure 3 Assuming that, as shown in FIG. 1, the computing clusters 110-1, 110-2, 110-5 and 110-6 belong to the broadcast group 120-1, the computing clusters 110-3, 110-4, 110-7 and 110-8 belong to the broadcast group 120-2, and the mask register 1122-1 corresponds to the broadcast group 120-1 (with an identifier of group1) and the mask register 1122-2 corresponds to the broadcast group 120-2 (with an identifier of group2), the bit pattern of the mask register 1122-1 is 11001100 and the bit pattern of the mask register 1122-2 is 00110011. Note that the above grouping is only exemplary, and in fact, the computing clusters included in each broadcast group can overlap. For example, a computing cluster 110 can belong to both the broadcast group 120-1 and the broadcast group 120-2, i.e., different broadcast groups 120 can include the same computing cluster 110. Of course, a computing cluster 110 in the computing device 100 can also not belong to any broadcast group.

[0044] The intra-cluster interface 112 is configured to determine the barrier register corresponding to the broadcast group in the broadcast synchronization barrier message based on the identifier of the broadcast group in the broadcast synchronization barrier message, and determine the corresponding producer computing cluster based on the barrier register corresponding to the broadcast group.

[0045] For example, if the groupID in the broadcast synchronization barrier message bar broadcast groupID received by the intra-cluster interface 112 is group1, which indicates that the broadcast synchronization barrier message is for the broadcast group 120-1 with an identifier of group1, the intra-cluster interface 112 can determine that the barrier register corresponding to the broadcast group 120-1 is the barrier register 1122-1. The bit pattern of the mask register 1122-1 is 11001100, which indicates that the broadcast group 120-1 includes four computing clusters 110-1, 110-2, 110-5 and 110-6, and thus the producer computing clusters corresponding to the consumer computing cluster 110-6 that sent the broadcast synchronization barrier message are the computing clusters 110-1, 110-2 and 110-5.

[0046] Then, the intra-cluster interface 112 can expand the broadcast synchronization barrier message bar broadcast groupID into synchronization barrier messages for the 3 producer computing clusters 110-1, 110-2, 110-5, such as bar_to_spc_1, bar_to_spc_2, bar_to_spc_5, and send them to the corresponding producer computing clusters 110-1, 110-2 and 110-5, respectively.

[0047] Here, the intra-cluster interface 112 can be configured to send the unrolled synchronization barrier messages to the respective producer computing cluster through the inter-cluster interface 130 one by one. For example, as shown in FIG. 1 IB, the intra-cluster interface 112 of the computing cluster 110-1 can be configured to send the unrolled synchronization barrier messages to the computing cluster 110-2 through the inter-cluster interface 130-1, and the computing cluster 110-2 can be configured to send the unrolled synchronization barrier messages to the computing cluster 110-5 through the inter-cluster interface 130-1. Figure 1B Here, the inter-cluster interface 130 can include one or more levels of inter-cluster interfaces 130-1, 130-2, and 130-3, each of which can be configured to send the unrolled synchronization barrier messages to the respective producer computing cluster through the inter-cluster interface 130 one by one. For example, as shown in FIG. 1 IB, the intra-cluster interface 112 of the computing cluster 110-1 can be configured to send the unrolled synchronization barrier messages to the computing cluster 110-2 through the inter-cluster interface 130-1, and the computing cluster 110-2 can be configured to send the unrolled synchronization barrier messages to the computing cluster 110-5 through the inter-cluster interface 130-1.

[0048] In this way, the instruction cycles used by the computing clusters to perform spatial synchronization in a broadcast case can be reduced, and the hardware burden caused by the intra-cluster interface as a hardware module to unroll the broadcast synchronization barrier message into synchronization barrier messages for each producer computing cluster is also small.

[0049] The above describes the synchronization barrier broadcast operation of the consumer computing cluster from the perspective of the data space readiness. In a many-to-many broadcast case, each computing cluster in a broadcast group needs to send a synchronization barrier message to all other computing clusters in the broadcast group, and only when the thread bundle count for one synchronization barrier is equal to the number of computing clusters in the broadcast group, each producer computing cluster sends the generated data to all consumer computing clusters. Therefore, from the perspective of the producer computing cluster, it also needs to determine whether it has received data space readiness messages from all consumer computing clusters for the same synchronization barrier before sending the generated data to all consumer computing clusters.

[0050] Specifically, still taking Figure 1A as an example, assume that the computing cluster 110-1 in the broadcast group 120-1 is configured as a producer computing cluster, which needs to determine whether it has received synchronization barrier messages from all other computing clusters 110-2, 110-5, and 110-6 in the broadcast group 120-1. Here, the synchronization barrier messages of the computing clusters 110-2 and 110-5 can also be sent in the manner described above in conjunction with the computing cluster 110-6.

[0051] The compute cluster 110-1 can count the synchronization barrier messages for the same synchronization barrier from all other compute clusters in the same broadcast group. For example, the synchronization barrier message can include, in addition to the broadcast indicator and the identifier of the broadcast group, an identifier of the synchronization barrier (e.g., barID) for which the message is directed and an arrival indication.

[0052] If it is determined that the synchronization barrier messages from all other compute clusters in the broadcast group have been received, the compute cluster 110-1 can send the produced data to the other compute clusters.

[0053] If it is determined that the synchronization barrier messages from all other compute clusters in the broadcast group have not been received, the compute cluster 110-1 can continue to wait for the other synchronization barrier messages.

[0054] In some embodiments, the compute cluster 110-1 can execute a synchronization barrier wait instruction, such as "bar_groupID_wait barID, N", where "groupID" indicates an identifier of the broadcast group for which the instruction is directed (e.g., the identifier groupl of the broadcast group 120-1), "wait" indicates a wait instruction, "barID" indicates the synchronization barrier for which the instruction is directed, and "N" indicates a synchronization barrier count value (i.e., the number of compute clusters included in the broadcast group). Here, the synchronization barrier count value of N includes N-1 thread bundles from N-1 consumer compute clusters and one thread bundle from the producer compute cluster itself (i.e., the thread bundle running the synchronization barrier wait instruction). In the example shown in the figure, N = 3.

[0055] Figure 4 A flowchart of a synchronization method 400 for the computing device 100 according to an embodiment of the application is shown.

[0056] As shown in Figure 4 At block 410, a compute cluster (e.g., compute cluster 110-6) in each broadcast group (e.g., broadcast group 120-1) as a consumer compute cluster sends a broadcast synchronization barrier message to the intra-cluster interface 112 of the consumer compute cluster when a data space for all producer compute clusters (e.g., compute clusters 110-1, 110-2, and 110-5) in the broadcast group 120-1 is ready. The broadcast synchronization barrier message includes a broadcast indicator and an identifier of the broadcast group 120-1.

[0057] At block 420, the intra-cluster interface 112 unpacks the broadcast synchronization barrier message into a synchronization barrier message for each producer compute cluster (e.g., compute clusters 110-1, 110-2, and 110-5) in the broadcast group 120-1 and sends each synchronization barrier message to the corresponding producer compute cluster.

[0058] Those skilled in the art will understand that the computing device shown in the above figure is only illustrative. In some embodiments, the computing device can include more or fewer components than shown.

[0059] The synchronization operation of the computing device and the computing cluster included therein according to the present disclosure is described above in conjunction with the accompanying drawings. However, those skilled in the art will understand that the execution of the computing device and its synchronization operation is not limited to the order shown in the figures and described above, but can be executed in any other reasonable order. In addition, the computing device does not necessarily include all the components shown in the figures, but can only include some or more components necessary to perform the functions described in the present disclosure, and the connection manner of the components is not limited to the form shown in the figures.

[0060] The present disclosure can be implemented as a method, a computing device, a system, and / or a computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions stored therein, which are used to perform various aspects of the present disclosure. The computing device can include at least one processor and at least one memory coupled to the at least one processor, and the memory can store instructions for execution by the at least one processor. When the instructions are executed by the at least one processor, the computing device can perform the asymmetric synchronization method described above.

[0061] In one or more example designs, the functions described in the present disclosure can be implemented in hardware, software, firmware, or any combination thereof. For example, if implemented in software, the functions can be stored as one or more instructions or code on a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium.

[0062] The various units of the apparatus disclosed herein can be implemented using discrete hardware components, or integrated on a hardware component, such as a processor. For example, the various exemplary logical blocks, modules, and circuits described in conjunction with the present disclosure can be implemented or executed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of the above for performing the functions described herein.

[0063] Those of ordinary skill in the art will also understand that the various exemplary logical blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments of the present disclosure can be implemented as electronic hardware, computer software, or a combination of the two.

[0064] The above description of the disclosure is to enable any person skilled in the art to implement or use the disclosure. Various modifications of the disclosure are obvious to those skilled in the art, and the general principles defined herein can also be applied to other variations without departing from the spirit and protection scope of the disclosure. Therefore, the disclosure is not limited to the examples and designs described herein, but is consistent with the broadest scope of the principles and novel characteristics disclosed herein.

Claims

1. A computing device comprising: a plurality of compute clusters, the plurality of compute clusters being partitioned into a plurality of broadcast groups; and one or more intra-cluster interfaces respectively located in each of the plurality of compute clusters, wherein a compute cluster in each broadcast group, when acting as a consumer compute cluster, is configured to send a broadcast synchronization barrier message to the intra-cluster interface when data spaces for all producer compute clusters in the broadcast group are ready, the broadcast synchronization barrier message including a broadcast indicator and an identifier of the broadcast group, and the intra-cluster interface is configured to fan out the broadcast synchronization barrier message into a synchronization barrier message for each producer compute cluster in the broadcast group and send each synchronization barrier message to the respective producer compute cluster.

2. The computing device of claim 1, wherein the intra-cluster interface is configured to determine the respective producer compute cluster based on the identifier of the broadcast group in the broadcast synchronization barrier message.

3. The computing device of claim 2, wherein the intra-cluster interface includes a plurality of mask registers, each mask register corresponding to a broadcast group of the plurality of broadcast groups, the mask registers being used to indicate compute clusters of the plurality of compute clusters that belong to the broadcast group, and wherein the intra-cluster interface is configured to determine the respective producer compute cluster based on the barrier register corresponding to the broadcast group based on the identifier of the broadcast group in the broadcast synchronization barrier message.

4. The computing device of claim 1, wherein the consumer compute cluster is further configured to: determine whether data spaces for all producer compute clusters in the broadcast group are ready; in response to data spaces for all producer compute clusters in the broadcast group being ready, generate the broadcast synchronization barrier message; and in response to data spaces for a portion of the producer compute clusters in the broadcast group not being ready, continue to wait for the spaces for the portion of the producer compute clusters to be ready.

5. The computing device of claim 1, wherein each producer compute cluster in the broadcast group is configured to: determine whether a synchronization barrier message from all other compute clusters in the broadcast group is received; in response to receiving a synchronization barrier message from all other compute clusters in the broadcast group, send generated data to the other compute clusters; and in response to not receiving a synchronization barrier message from all other compute clusters in the broadcast group, continue to wait.

6. The computing device of claim 1, the intra-cluster interface is configured to send each synchronization barrier message to the respective producer compute cluster in sequence through an inter-cluster interface.

7. A synchronization method for a computing device comprising a plurality of compute clusters and a plurality of intra-cluster interfaces respectively located in each of the plurality of compute clusters, wherein the plurality of compute clusters are partitioned into a plurality of broadcast groups, the synchronization method comprising: ​ sending, by one compute cluster of a plurality of compute clusters that is a consumer compute cluster in each of a plurality of broadcast groups, a broadcast synchronization barrier message to an intra-cluster interface when data spaces for all producer compute clusters in the broadcast group are ready, the broadcast synchronization barrier message including a broadcast indicator and an identifier of the broadcast group, and expanding, by the intra-cluster interface, the broadcast synchronization barrier message into a synchronization barrier message for each producer compute cluster in the broadcast group, and sending each synchronization barrier message to a respective producer compute cluster.

8. The synchronization method of claim 7, wherein expanding, by the intra-cluster interface, the broadcast synchronization barrier message into a synchronization barrier message for each producer compute cluster in the broadcast group comprises: determining, by the intra-cluster interface, a respective producer compute cluster based on the identifier of the broadcast group in the broadcast synchronization barrier message.

9. The synchronization method of claim 8, wherein the intra-cluster interface includes a plurality of mask registers, each mask register corresponding to one of the plurality of broadcast groups, the mask registers to indicate compute clusters of the plurality of compute clusters that belong to the broadcast group, and wherein determining, by the intra-cluster interface, a respective producer compute cluster based on the identifier of the broadcast group in the broadcast synchronization barrier message comprises: determining, by the intra-cluster interface, a barrier register corresponding to the broadcast group based on the identifier of the broadcast group in the broadcast synchronization barrier message, and determining a respective producer compute cluster based on the barrier register corresponding to the broadcast group.

10. The synchronization method of claim 7, further comprising: determining, by the consumer compute cluster, whether data spaces for all producer compute clusters in the broadcast group are ready; in response to data spaces for all producer compute clusters in the broadcast group being ready, generating the broadcast synchronization barrier message; and in response to data spaces for a portion of the producer compute clusters in the broadcast group not being ready, continuing to wait for the spaces for the portion of the producer compute clusters to be ready.

11. The synchronization method of claim 7, further comprising: determining, by each producer compute cluster in the broadcast group, whether a synchronization barrier message was received from all other compute clusters in the broadcast group; in response to receiving a synchronization barrier message from all other compute clusters in the broadcast group, sending generated data to the other compute clusters; and in response to not receiving a synchronization barrier message from all other compute clusters in the broadcast group, continuing to wait.

12. An electronic device, comprising: a memory that non-transitorily stores computer-executable instructions; a processor configured to execute the computer-executable instructions; wherein the computer-executable instructions, when executed by the processor, implement the synchronization method of any one of claims 7-11.

13. A computer-readable storage medium having computer program code stored thereon, the computer program code, when executed, performing the synchronization method of any one of claims 7-11. ​ ​

Images