Thread group inter-data synchronization method, device, computer equipment and program product

Abstract

The application relates to a thread group data synchronization method and device, computer equipment and a program product. The method comprises the following steps: comparing the number of a current thread group with the number of a current thread bundle; using a data writing mode corresponding to the comparison result to write the calculation result of the current thread bundle into the specified memory of a target thread group, the number of the target thread group being consistent with the number of the current thread bundle; sending a first instruction to a data ready synchronization unit of the target thread group, the first instruction being used to instruct the data ready synchronization unit to update a first count value, the first count value being used to judge whether the data ready synchronization unit is in a ready state; for any target thread group, if the data ready synchronization unit in the target thread group is in the ready state, each thread bundle participating in parallel accumulation operation in the target thread group loads data from the specified memory of the target thread group and executes accumulation operation based on the loaded data. The method can improve the parallel computing execution efficiency.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, computer device, and program product for data synchronization between thread groups. Background Technology

[0002] In parallel computing architectures, multiple thread groups (TGs) are typically combined into a TG cluster as the computation execution unit, with each TG mapping to a computation unit (CU). Different TGs within a TG cluster need to communicate with each other to achieve collaborative computation. A typical scenario is that multiple TGs compute the same result in parallel and then accumulate the data. To avoid redundant computation, each TG only receives and accumulates the computation data of a specific warp, writes its own computation result to the group shared memory (GSM) of the target TG, and simultaneously reads the target data from other TGs from the GSM.

[0003] In related technologies, this collaborative computing process needs to rely on memory barriers (mbar) to ensure that GSM data is ready and buffer is ready. That is, two types of mbar resources are set up: data ready synchronization unit (which can be represented as s_data_ready_mbar) and buffer ready synchronization unit (which can be represented as s_buffer_ready_mbar).

[0004] However, in related technologies, the process of achieving collaborative computing by using data ready and buffer ready involves double-nested traversal and multiple branch judgment operations. Since conditional jump instructions need to be executed in each round of traversal, there is a long communication delay, which in turn reduces the execution efficiency of parallel computing. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, apparatus, computer device, and program product for data synchronization between thread groups that can reduce communication latency and improve the execution efficiency of parallel computing, in order to address the above-mentioned technical problems.

[0006] In a first aspect, this application provides a method for data synchronization between thread groups, the method comprising:

[0007] The current thread group number is compared with the current thread bundle number to obtain the comparison result;

[0008] Using the data writing method corresponding to the comparison result, the calculation result of the current thread bundle is written to the specified memory of the target thread group, wherein the number of the target thread group is consistent with the number of the current thread bundle;

[0009] Send a first instruction to the data ready synchronization unit of the target thread group. The first instruction is used to instruct the data ready synchronization unit to update a first count value. The first count value is used to determine whether the data ready synchronization unit is in a ready state.

[0010] For any of the target thread groups, if the data-ready synchronization unit in the target thread group is in a ready state, then each thread bundle participating in the parallel accumulation operation in the target thread group loads data from the designated memory of the target thread group and performs the accumulation operation based on the loaded data.

[0011] In one embodiment, writing the calculation result of the current thread bundle into the designated memory of the target thread group using the data writing method corresponding to the comparison result includes:

[0012] If the comparison result indicates that the number of the current thread group is consistent with the number of the current thread bundle, the calculation result of the current thread bundle is written to the designated memory of the current thread group using a local data write instruction.

[0013] In one embodiment, writing the calculation result of the current thread bundle into the designated memory of the target thread group using the data writing method corresponding to the comparison result includes:

[0014] If the comparison result indicates that the number of the current thread group is inconsistent with the number of the current thread bundle, the target thread group is determined according to the number of the current thread bundle, and the calculation result of the current thread bundle is written to the designated memory of the target thread group using a remote data write instruction.

[0015] In one embodiment, the method further includes:

[0016] After completing the data writing using the remote data writing instruction, a second instruction is sent to the data ready synchronization unit of the target thread group. The second instruction is used to instruct the data ready synchronization unit to update the byte count value.

[0017] When the first count value is updated to 0 and the byte count value is also updated to 0, it is determined that the data ready synchronization unit in the target thread group is in a ready state.

[0018] In one embodiment, the method further includes:

[0019] After any of the thread bundles completes the accumulation operation, a third instruction is sent to the buffer ready synchronization unit of each thread group. The third instruction instructs the buffer ready synchronization unit to update the second count value, and when the second count value is updated to 0, it is determined that the buffer ready synchronization unit is in a ready state.

[0020] In one embodiment, the initial value of the first count of the data ready synchronization unit is N, where N is the number of thread groups in the cluster; the initial value of the byte count of the data ready synchronization unit is D*(N-1), where D is the number of bytes written in a single operation; and the initial value of the second count of the buffer ready synchronization unit is N*M, where M is the number of thread bundles participating in the accumulation operation within the thread group, where N, D, and M are all positive integers.

[0021] Secondly, this application also provides a data synchronization device between thread groups, comprising:

[0022] The comparison module is used to compare the number of the current thread group with the number of the current thread bundle to obtain the comparison result;

[0023] The data writing module is used to write the calculation result of the current thread bundle into the specified memory of the target thread group using the data writing method corresponding to the comparison result, wherein the number of the target thread group is consistent with the number of the current thread bundle;

[0024] The first sending module is used to send a first instruction to the data ready synchronization unit of the target thread group. The first instruction is used to instruct the data ready synchronization unit to update a first count value. The first count value is used to determine whether the data ready synchronization unit is in a ready state.

[0025] The computation module is configured to, for any of the target thread groups, if the data-ready synchronization unit in the target thread group is in a ready state, load data from the designated memory of the target thread group for each thread bundle participating in the parallel accumulation operation, and perform the accumulation operation based on the loaded data.

[0026] In one embodiment, writing the calculation result of the current thread bundle into the designated memory of the target thread group using the data writing method corresponding to the comparison result includes:

[0027] If the comparison result indicates that the number of the current thread group is consistent with the number of the current thread bundle, the calculation result of the current thread bundle is written to the designated memory of the current thread group using a local data write instruction.

[0028] In one embodiment, writing the calculation result of the current thread bundle into the designated memory of the target thread group using the data writing method corresponding to the comparison result includes:

[0029] If the comparison result indicates that the number of the current thread group is inconsistent with the number of the current thread bundle, the target thread group is determined according to the number of the current thread bundle, and the calculation result of the current thread bundle is written to the designated memory of the target thread group using a remote data write instruction.

[0030] In one embodiment, the device further includes:

[0031] The second sending module is used to send a second instruction to the data ready synchronization unit of the target thread group after the data writing is completed using the off-site data writing instruction. The second instruction is used to instruct the data ready synchronization unit to update the byte count value.

[0032] The determination module is used to determine that the data ready synchronization unit in the target thread group is in a ready state when the first count value is updated to 0 and the byte count value is also updated to 0.

[0033] In one embodiment, the device further includes:

[0034] The third sending module is used to send a third instruction to the buffer ready synchronization unit of each thread group after any of the thread bundles has completed the accumulation operation. The third instruction instructs the buffer ready synchronization unit to update the second count value, and when the second count value is updated to 0, it is determined that the buffer ready synchronization unit is in a ready state.

[0035] In one embodiment, the initial value of the first count of the data ready synchronization unit is N, where N is the number of thread groups in the cluster; the initial value of the byte count of the data ready synchronization unit is D*(N-1), where D is the number of bytes written in a single operation; and the initial value of the second count of the buffer ready synchronization unit is N*M, where M is the number of thread bundles participating in the accumulation operation within the thread group, where N, D, and M are all positive integers.

[0036] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the thread group data synchronization method described above.

[0037] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the thread group data synchronization method described above.

[0038] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the inter-thread group data synchronization method described above.

[0039] The aforementioned inter-thread group data synchronization method, apparatus, computer device, readable storage medium, and program product compare the current thread group number with the current thread bundle number to obtain a comparison result. Using the data writing method corresponding to the comparison result, the calculation result of the current thread bundle is written to a designated memory location of the target thread group, where the target thread group number is consistent with the current thread bundle number. A first instruction is sent to the data-ready synchronization unit of the target thread group, instructing the data-ready synchronization unit to update a first count value, which is used to determine whether the data-ready synchronization unit is in a ready state. For any target thread group, if the data-ready synchronization unit in the target thread group is in a ready state, each thread bundle participating in the parallel accumulation operation in the target thread group loads data from the designated memory location of the target thread group and performs the accumulation operation based on the loaded data. The thread group data synchronization method, apparatus, computer device, readable storage medium, and program product provided in this application simplify the data interaction and synchronization process of parallel accumulation operation to the execution logic of comparing the number of the current thread group with the number of the current thread bundle, and realizing targeted writing and targeted synchronization based on the number of the current thread bundle. This eliminates the redundant operation of double-nested traversal of all thread groups and the condition jump delay caused by multiple branch judgments in related technologies, significantly reducing the communication delay of data interaction and synchronization waiting between thread groups, improving the execution efficiency of parallel accumulation operation, and thus improving the overall computing performance of artificial intelligence chips. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a schematic diagram illustrating data synchronization between TGs within a TG cluster in one embodiment;

[0042] Figure 2 This is a flowchart illustrating a data synchronization method between thread groups in one embodiment;

[0043] Figure 3 This is a schematic diagram of the structure of an artificial intelligence chip in one embodiment;

[0044] Figure 4This is a flowchart illustrating a data synchronization method between thread groups in another embodiment;

[0045] Figure 5 This is a structural block diagram of a data synchronization device between thread groups in one embodiment;

[0046] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0048] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0049] In related technologies, refer to Figure 1 As shown, assuming a TG cluster has 4 TGs, each TG needs to store the current TG's computation result data in its own GSM (Gateway Controller) and simultaneously send this data to the GSMs of the other TGs. At the same time, the TG's GSM needs to receive data sent from other TGs. When a TG stores data to the GSMs of other TGs, it uses the `st.async.csm` command (a remote data write command), specifying the destination TG's address. When a TG stores data to the GSM of its own TG, it uses the `st.gsm` command (a local data write command), which does not require specifying the destination TG address. Additionally, `mbar` is used to ensure the GSM data is data ready and the buffer is buffer ready.

[0050] In an exemplary scenario, different TGs can compute a certain result in parallel (multiple warps in parallel), and then the computation results of each TG are accumulated. For example, a thread group cluster includes different thread groups (TGs) and multiple internal thread bundles (warps) that share the tensor multi-channel dimension computation task in parallel. Taking four TGs as an example, the overall computation task is equally divided into four parts, with each of the four TGs undertaking one-quarter of the global computation, and the computation tasks are denoted as A, B, C, and D respectively; each TG contains four thread bundles, which can further subdivide the computation undertaken by the TG. Taking TG0, which is responsible for the computation of part A, as an example, its internal warps 0 to 3 independently complete the subtasks A1, A2, A3, and A4 respectively. Thus, each thread bundle in the cluster ultimately undertakes only a sub-computation of 1 / 16 of the global granularity and generates the corresponding sub-computation result.

[0051] Due to the inherent arrangement characteristics of tensor multichannel computation, the sub-computation fragments corresponding to the same channel will be distributed across thread bundles with the same number in each TG. Therefore, the sub-computation results of the same thread bundle in all TGs can be uniformly grouped into a fixed TG to complete the full accumulation of a single channel. For example, after each of the four TGs completes its subdivision computation, Warp0 in TG0 produces result A1, Warp0 in TG1 produces result B1, Warp0 in TG2 produces result C1, and Warp0 in TG3 produces result D1. TG1, TG2, and TG3 all write the B1, C1, and D1 sub-computation results corresponding to their own Warp0 to the GSM of TG0 through cross-thread group communication. Ultimately, the GSM of TG0 aggregates the sub-calculation results of A1, B1, C1, and D1 of all Warp0 channels in the entire cluster. Only TG0 independently reads and completes the full accumulation and summarization of this channel dimension. Similarly, all Warp1 data of TG are uniformly assigned to TG1 for accumulation and summarization, and Warp2 and Warp3 are assigned to TG2 and TG3 respectively to complete the accumulation and summarization.

[0052] For example, the data communication relationship between TGs is shown in Table 1 below, where TgIdx represents the TG number, WarpIdx represents the thread bundle number, and DstTgIdx represents the target TG number.

[0053] Table 1

[0054]

[0055] Each TG can request two mbar resources to control data production (s_data_ready_mbar) and consumption (s_buffer ready_mbar). Assuming the amount of data produced by a certain warp in each TG is D, then s_data_ready_mbar init count (first count value) can be set to 3 (from the perspective of TG0, it needs to receive data from warp0 in the other 3 TGs); s_data_ready_mbar init tx-count (byte count value) can be set to D*3 (after a st.async.csm instruction is completed, the tx-count will be reduced by the amount of data operated by this instruction); s_buffer_ready_mbar init count (second count value) can be set to 12 (from the perspective of TG0, after data consumption, it needs to notify TG1, TG2 and TG3 that all warp data has been consumed and can be rewritten), where curTgIdx is the TgIdx of the current TG and curWarpIdx is the WarpIdx of the current warp.

[0056] The cross-thread group storage process for executing calculation results is as follows: Traverse all four thread groups in the thread group cluster and read the target thread group number DstTgIdx in sequence; during the traversal, first determine whether the target thread group number matches the current thread bundle number curWarpIdx. If they do not match, skip the current thread group and continue traversing. If they match, further nested checks are made to determine whether the current thread bundle number curWarpIdx is consistent with the current thread group number curTgIdx.

[0057] When the two numbers match, it is determined to be a local write scenario: the local write instruction is called to write the calculation result to the GSM of the current thread group, and the L1 cache confirmation operation and L1 cache synchronization wait operation are executed in sequence to complete the local data storage; when the two numbers do not match, it is determined to be a cross-thread group write scenario: the asynchronous cross-thread group write instruction is called to write the data to the GSM of the corresponding target thread group, and a synchronization instruction is sent to the target thread group to decrement the first count value of s_data_ready_mbar by 1 and the byte count value by D.

[0058] The readiness status of s_data_ready_mbar can be determined by its first count value and byte count value. Before performing a data read operation, it is necessary to block and wait for the memory barrier to complete the readiness verification. After confirming that the data is ready, the corresponding data is read from the GSM and loaded into the local register TLR.

[0059] After completing the data reading and loading operations, all four thread groups in the cluster are traversed again. During the traversal, it is determined whether the target thread group is the current local thread group. Synchronization control instructions are sent only to the other thread groups that are not local, decrementing the second count value of s_buffer_ready_mbar by 1 each time. s_buffer_ready_mbar also relies on the second count value to determine the ready state, thereby achieving complete data interaction and synchronization control among all thread groups in the cluster.

[0060] As can be seen, this processing flow involves double-nested traversal and multiple branch judgment operations (since curTgIdx and curWarpIdx are dynamic parameters stored in the WSR (warp scalar register), they are represented by jump instructions in assembly instructions, which cause a long branch jump delay). Since the jump instruction needs to be executed in each round of traversal, there is a long communication delay, which in turn reduces the execution efficiency of parallel computing.

[0061] This application provides a method for data synchronization between thread groups. By simplifying the data interaction and synchronization process of parallel accumulation operations to comparing the number of the current thread group with the number of the current thread bundle, and implementing the execution logic of targeted writing and targeted synchronization based on the number of the current thread bundle, this method eliminates the redundant operations of double-nested traversal of all thread groups and the condition jump delay caused by multiple branch judgments in related technologies. This significantly reduces the communication latency of data interaction and synchronization waiting between thread groups, improves the execution efficiency of parallel accumulation operations, and thus improves the overall computing performance of artificial intelligence chips.

[0062] like Figure 2 As shown, a method for data synchronization between thread groups is provided, which is applied to artificial intelligence chips. In this embodiment, the artificial intelligence chip is any one of GPU (Graphics Processing Unit), TPU (Tensor Processing Unit), NPU (Neural Network Processing Unit), DPU (Deep Learning Processing Unit), APU (Accelerated Processing Unit), and GPGPU (General-Purpose Graphics Processing Unit). This embodiment does not specifically limit the type of chip, and the following description uses GPGPU as an example.

[0063] Reference Figure 3The diagram shows a schematic of a GPGPU. A GPGPU is actually an array of Streaming Processor Clusters (SPCs), including, for example,... Figure 3 The diagram shows streaming processor clusters 1, ..., M, where M is a positive integer greater than 1. In a graphics processing unit (GPU), one streaming processor cluster processes one computational task, or multiple streaming processor clusters process one computational task. Multiple streaming processor clusters share data through a global cache or global memory.

[0064] like Figure 3 As shown, taking streaming processor cluster 1 as an example, one streaming processor cluster includes multiple computing units, such as... Figure 3 The system is structured as Computation Unit 1, Computation Unit 2, ..., Computation Unit N, where N is a positive integer. Each Computation Unit (CU) performs arithmetic and logical operations other than matrix calculations such as matrix multiplication and convolution, including operations like accumulation, reduction, and standard addition, subtraction, multiplication, and division. A Computation Unit contains multiple cores (also called computational kernels), each including an Arithmetic Logic Unit (ALU), a floating-point unit, etc., which are used to execute specific computational tasks. Furthermore, the Computation Unit also includes registers (e.g., ...). Figure 3 The register file and shared cache in a computing unit are used to store source and destination data related to computing tasks in a hierarchical manner. The shared cache in a computing unit is used to share data between the cores of that computing unit.

[0065] In parallel computing, computational tasks are typically executed by multiple threads. These threads are divided into multiple thread blocks before execution in a general-purpose graphics processor (or parallel computing processor), and then dispatched via a thread block distribution module. Figure 3 (Not shown in the image) Multiple thread blocks are distributed to various computation units. All threads in a thread block must be assigned to the same computation unit for execution. Simultaneously, thread blocks are broken down into minimum execution thread bundles (or simply warps), each containing a fixed number (or less than this fixed number) of threads, for example, 32 threads. Multiple thread blocks can execute in the same computation unit or in different computation units.

[0066] In each computing unit, the thread beam scheduling / distribution module ( Figure 3(Not shown in the diagram) Thread bundles are scheduled and allocated so that multiple computing cores within the computing unit can run thread bundles. Depending on the number of computing cores in the computing unit, multiple thread bundles within a thread block can be executed concurrently or in a time-sharing manner. Multiple threads within each thread bundle execute the same instructions. Memory execution instructions are issued to the shared cache within the computing unit or further issued to intermediate-level caches, global caches, or global memory for read and write operations, etc.

[0067] like Figure 3 As shown, the streaming processor cluster 1 also includes a tensor operation unit, which is used to perform tensor calculations, such as matrix multiplication, convolution operations, etc.

[0068] Reference Figure 2 As shown in the figure, an embodiment of this application provides a method for data synchronization between thread groups, including the following steps 202, 204, 206, and 208. Wherein:

[0069] Step 202: Compare the number of the current thread group with the number of the current thread bundle to obtain the comparison result.

[0070] In this embodiment, the current thread group number is a unique identifier for each thread group within the thread group cluster, and the current thread bundle number is a unique identifier for each thread bundle within a single thread group. The current thread group number and the current thread bundle number can be stored in the thread bundle scalar register of each thread bundle, that is, the current thread group number and the current thread bundle number can be directly read from the register of each thread bundle for comparison.

[0071] Step 204: Using the data writing method corresponding to the comparison result, write the calculation result of the current thread bundle into the specified memory of the target thread group. The number of the target thread group is consistent with the number of the current thread bundle.

[0072] In this embodiment, the data writing method includes a local writing method and a cross-thread group asynchronous writing method. For example, if the comparison result shows that the current thread group number matches the current thread bundle number, then the current thread group is determined to be the target thread group of the current thread bundle. The local writing method can be used to directly write the calculation result of the current thread bundle to the specified memory of the current thread group, without specifying the target address corresponding to the target thread group. If the comparison result shows that the current thread group number does not match the current thread bundle number, then the current thread group is not the target thread group of the current thread bundle. Cross-thread group data writing is required, so a cross-thread group asynchronous writing method can be used to write the calculation result of the current thread bundle to the specified memory of the target thread group. The target address corresponding to the target thread group with the same number as the current thread bundle must be specified in the instruction. The specified memory may include the group shared memory of the thread group.

[0073] In this embodiment of the application, by setting the number of the target thread group to be consistent with the number of the current thread bundle, the calculation result of each thread bundle can be accurately written into the designated memory of the corresponding thread group responsible for accumulation, avoiding the accumulation duplication or omission caused by data writing misalignment.

[0074] In an exemplary embodiment, writing the calculation result of the current thread bundle into a specified memory of the target thread group using the data writing method corresponding to the comparison result may include the following steps:

[0075] If the comparison result indicates that the number of the current thread group is consistent with the number of the current thread bundle, the calculation result of the current thread bundle is written to the specified memory of the current thread group using a local data write instruction.

[0076] In this embodiment, when the number of the current thread group is the same as the number of the current thread bundle, it indicates that the specified memory to be written by the current thread bundle belongs to the current thread group itself, and no cross-thread group data transfer is required. For example, assuming the current thread group is TG0 (corresponding to number 0) and the current thread bundle is Warp0 (corresponding to number 0), that is, both numbers are 0, it can be determined that the number of the current thread group is the same as the number of the current thread bundle, and the target thread group corresponding to Warp0 is TG0. At this time, using the local data write instruction, the local write operation of the calculation result of the current thread bundle can be directly completed without specifying the target address of the target thread group, and without starting the hardware logic related to cross-thread group communication. This can effectively reduce data interaction latency, reduce redundant instructions and communication overhead, and at the same time ensure the reliability and timing consistency of data writing.

[0077] In another exemplary embodiment, writing the calculation result of the current thread bundle into a designated memory of the target thread group using the data writing method corresponding to the comparison result may include the following steps:

[0078] If the comparison results indicate that the number of the current thread group is inconsistent with the number of the current thread bundle, the target thread group is determined based on the number of the current thread bundle, and the calculation result of the current thread bundle is written to the specified memory of the target thread group using a remote data write instruction.

[0079] In this embodiment, when the number of the current thread group is inconsistent with the number of the current thread bundle, it indicates that the specified memory to be written by the current thread bundle is not a local resource of the current thread group, but belongs to another thread group. For example, assuming the current thread group is TG1 (corresponding to number 1) and the current thread bundle is Warp0 (corresponding to number 0), it can be determined that the number of the current thread group is inconsistent with the number of the current thread bundle. In this case, the number of the current thread bundle can be directly used as the number of the target thread group to determine the target thread group (the target thread group corresponding to Warp0 is TG0), and the cross-thread group data writing is completed using the off-site data writing instruction, writing the calculation result of the current thread bundle into the specified memory of the target thread group.

[0080] In this way, during the data synchronization process, there is no need to traverse the address or make additional branch judgments, and the target address can be directly hit, ensuring that the data writing path is unique and clear. This enables efficient and reliable data interaction between thread groups, providing a data foundation for each thread group to independently execute parallel reduction accumulation.

[0081] Step 206: Send a first instruction to the data ready synchronization unit of the target thread group. The first instruction is used to instruct the data ready synchronization unit to update the first count value. The first count value is used to determine whether the data ready synchronization unit is in a ready state.

[0082] In this embodiment, the data-ready synchronization unit, also known as the data-ready memory barrier (or s_data_ready_mbar), is a hardware unit used to ensure data production synchronization between thread groups. A first count value is used to characterize the amount of data from the thread group that the data-ready synchronization unit needs to receive. After the current thread group completes data writing (whether local or remote), a first instruction can be sent to the data-ready synchronization unit of the target thread group. This first instruction triggers the data-ready synchronization unit to update the first count value, which is the basis for determining whether all data is ready to be written (i.e., whether the data-ready synchronization unit is ready).

[0083] The readiness state of the data-ready synchronization unit is the trigger condition for the thread group to start the accumulation operation. Only when the data-ready synchronization unit is in the readiness state does it indicate that the specified memory of the thread group has collected the calculation results of all thread bundles. For example, the data-ready synchronization unit decrements its first count value by one each time it receives the first instruction. When the first count value reaches 0, it can be determined that the data-ready synchronization unit is in the readiness state, indicating that the specified memory of the target thread group has received all the calculation results to be accumulated.

[0084] Step 208: For any target thread group, if the data-ready synchronization unit in the target thread group is in the ready state, then each thread bundle participating in the parallel accumulation operation in the target thread group loads data from the specified memory of the target thread group and performs the accumulation operation based on the loaded data.

[0085] In this embodiment, each thread bundle participating in the parallel accumulation operation within a thread group is the execution entity responsible for completing the accumulation operation. A fixed mapping relationship between thread group numbers and thread bundle numbers is pre-established in this embodiment. Each thread group is only responsible for summarizing and accumulating the calculation results of thread bundles with the same number in all thread groups within the cluster. During the data writing phase, all thread bundles with the same number in all thread groups have uniformly written their calculation results into the designated memory of the corresponding thread group responsible for accumulation. Therefore, during the data loading phase, data can be loaded from the designated memory of the thread group to the thread local register (TLR) of each thread bundle using data loading instructions (e.g., data read request instructions). This ensures that each thread group only loads the calculation results of the thread bundle matching its own number for summarization and accumulation (e.g., thread group 0 only loads the calculation results of thread bundle 0 in all thread groups for final summarization and accumulation), avoiding redundant calculations and improving the efficiency of the accumulation operation.

[0086] For example, for TTi, the thread bundle within TTi can send a data read request command to its own GSM to request the reading of data written by all TG warpi (TG0's warpi, TG1's warpi, TG2's warpi, and TG3's warpi). After reading the data, it can perform the corresponding accumulation and summarization calculation. After completing the accumulation and summarization calculation, it can send an accumulation completion command to its own GSM to store the accumulation result in the GSM.

[0087] The above-described inter-thread group data synchronization method compares the current thread group number with the current thread bundle number to obtain the comparison result. Using the data writing method corresponding to the comparison result, it writes the calculation result of the current thread bundle into the designated memory of the target thread group. The target thread group number is consistent with the current thread bundle number. A first instruction is sent to the data-ready synchronization unit of the target thread group. This first instruction instructs the data-ready synchronization unit to update a first count value, which is used to determine whether the data-ready synchronization unit is in a ready state. For any thread group, if the data-ready synchronization unit in the thread group is in a ready state, each thread bundle participating in the parallel accumulation operation in the thread group loads data from the designated memory of the thread group and performs the accumulation operation based on the loaded data. The thread group data synchronization method provided in this application simplifies the data interaction and synchronization process of parallel accumulation operations to comparing the number of the current thread group with the number of the current thread bundle, and implementing the execution logic of targeted writing and targeted synchronization based on the number of the current thread bundle. This eliminates the redundant operation of double-nested traversal of all thread groups and the condition jump delay caused by multiple branch judgments in related technologies, significantly reducing the communication latency of data interaction and synchronization waiting between thread groups, improving the execution efficiency of parallel accumulation operations, and thus improving the overall computing performance of artificial intelligence chips.

[0088] In one exemplary embodiment, reference is made to Figure 4 As shown, the method may further include steps 402 and 404, wherein:

[0089] Step 402: After completing the data writing using the off-site data writing instruction, a second instruction is sent to the data ready synchronization unit of the target thread group. The second instruction is used to instruct the data ready synchronization unit to update the byte count value.

[0090] Step 404: If the first count value is updated to 0 and the byte count value is also updated to 0, determine that the data ready synchronization unit in the target thread group is in a ready state.

[0091] In this embodiment, the byte count value is a built-in transmission byte count parameter of the data-ready synchronization unit, used to count the total amount of effective data transmitted across thread groups. The second instruction is a count update instruction of the data-ready synchronization unit, which is a hardware-level instruction. After completing the cross-thread group data writing using the off-site data write instruction, a data write completion instruction can be received from the target thread group. In response to this instruction, a second instruction can be sent to the data-ready synchronization unit of the target thread group. This second instruction triggers the data-ready synchronization unit to deduct and update the byte count value. That is, each time a cross-thread group data write is completed, the byte count value is reduced by the corresponding amount of data written, so as to synchronize the data transmission progress of the cross-thread group in real time, avoid the problem of performing subsequent operations before the data is completely transmitted, and ensure the integrity of data transmission.

[0092] For example, assuming the amount of data written across thread groups in a single instance is a fixed value D, and the initial configuration of the byte count value is 3×D, when a thread group completes a data transmission using a remote data write instruction, it sends a second instruction to the data ready synchronization unit of the target thread group. The data ready synchronization unit then deducts D from the byte count value, thereby realizing real-time updates of the amount of data transmitted and accurately recording the total amount of data to be transmitted.

[0093] In this embodiment, the first count value is used to count the number of thread bundles that have completed data writing. The first count value and the byte count value work together to ensure the accuracy of data synchronization status determination. Specifically, when the first count value is updated to 0, it means that all target thread bundles have completed the data writing operation and there are no write events to be executed; when the byte count value is updated to 0, it means that all data transmitted across thread groups has been completely delivered to the target group's shared memory and there are no remaining bytes to be transmitted. When both the first count value and the byte count value return to 0, it can be determined that the data writing and transmission process has been fully completed, and at this time, the data ready synchronization unit switches to the ready state.

[0094] In one exemplary embodiment, the above method may further include:

[0095] After any thread bundle completes the accumulation operation, a third instruction is sent to the buffer ready synchronization unit of each thread group. The third instruction instructs the buffer ready synchronization unit to update the second count value, and when the second count value is updated to 0, it is determined that the buffer ready synchronization unit is in a ready state.

[0096] In this embodiment, the buffer-ready synchronization unit refers to the buffer-ready memory barrier (which can also be represented as s_buffer_ready_mbar), which is a hardware unit that manages the synchronization of reuse and data consumption of a specified memory buffer. The second count value is the consumption completion count parameter built into the buffer-ready synchronization unit, which is used to count the total number of thread bundles that have completed the data accumulation operation. The third instruction is the count update instruction of the buffer-ready synchronization unit, which is also a hardware-level count update instruction.

[0097] The completion of the accumulation operation signifies that the corresponding data in the specified memory has been consumed, and the original data can be overwritten and reused. At this point, sending a third instruction triggers the buffer-ready synchronization unit to decrement and update the second count value. Each time a thread bundle completes accumulation and releases buffer resources, the second count value is decremented by a fixed step, synchronizing the data consumption progress of each thread group in real time. When the second count value is updated to 0, it means that all thread bundles participating in the accumulation operation have completed data consumption, and the specified memory buffer has been completely released. At this point, it can be determined that the buffer-ready synchronization unit is in a ready state, marking the completion of the current round of data writing, loading, and accumulation processes. Each thread group can then receive new calculation results and start the next round of parallel computing and data synchronization processes, avoiding buffer occupancy conflicts and ensuring the continuous and efficient execution of multiple rounds of parallel computing.

[0098] In an exemplary embodiment, the initial value of the first count of the data ready synchronization unit is N, where N is the number of thread groups in the cluster, the initial value of the byte count of the data ready synchronization unit is D*(N-1), where D is the number of bytes written in a single operation, and the initial value of the second count of the buffer ready synchronization unit is N*M, where M is the number of thread bundles participating in the accumulation operation within the thread group, where N, D, and M are all positive integers.

[0099] In this embodiment, the initial value of the first count of the data-ready synchronization unit is set to the total number of thread groups N in the thread group cluster. This corresponds to each thread group completing a write event trigger, ensuring that the target thread bundles of all thread groups participate in the data writing process, covering all parallel computing units. The initial value of the byte count is set to D*(N-1), which can adapt to cross-thread group data transmission scenarios, eliminating scenarios with no transmission overhead where the current thread group writes locally, and only counting the total number of bytes transmitted across units, thus achieving precise control over the amount of data transmission. The initial value of the second count of the buffer-ready synchronization unit is set to N*M, corresponding to the total number of thread bundles participating in the accumulation operation in each thread group within the cluster. After each thread bundle completes the accumulation operation, a count value deduction is triggered, ensuring the accuracy of buffer reuse synchronization.

[0100] For example, assuming the number of thread groups in the thread group cluster is N=4, the number of bytes written in a single instance is D, and the number of thread bundles participating in the accumulation within a single thread group is M=4. The initial value of the first count for the data-ready synchronization unit is 4, and the initial value of the byte count is D*(4-1)=3*D; the initial value of the second count for the buffer-ready synchronization unit is 4*4=16. In this configuration, the first count covers the write events of the four thread groups, the byte count accurately counts the total number of bytes written across thread groups, and the second count matches all 16 thread bundles participating in the accumulation. This ensures the accuracy of data synchronization and buffer reuse while significantly reducing redundant instruction overhead.

[0101] To enable those skilled in the art to better understand the embodiments of this application, the embodiments of this application are described below through specific examples. In this example, a thread group cluster includes 4 TGs, and each TG includes 4 Warp instances:

[0102] The AI ​​chip first sends an initialization instruction to the s_data_ready_mbar of each TG, presetting its first count value (which can be represented as count) to 4 and its byte count value (which can be represented as tx-count) to 3×D; at the same time, it sends an initialization instruction to the s_buffer_ready_mbar of each TG, presetting its second count value (which can be represented as count) to M (M=4 when the accumulation method is single warp accumulation; M=4*4 when the accumulation method is parallel accumulation). After receiving the instructions, s_data_ready_mbar and s_buffer_ready_mbar complete the parameter configuration and feed back the initialization ready instruction to the AI ​​chip.

[0103] The AI ​​chip sends parameter configuration instructions to each TG (TG0~TG3). The parameter configuration instructions may include curTgIdx and curWarpIdx. The data communication relationship between TGs can be written into the registers of each TG through the parameter configuration instructions. After receiving the instructions, each TG completes the parameter loading and feeds back the parameter ready instruction to the AI ​​chip.

[0104] After each Warp completes its calculation independently, it temporarily stores the result in its own register and simultaneously sends a calculation completion instruction to its corresponding TG. Each TG then aggregates the calculation completion instructions from its four Warps and sends its own calculation ready instruction to the AI ​​chip. Upon receiving the calculation ready instructions from all four TGs, the AI ​​chip sends executable data write instructions to all Warps, ending the parallel computation phase and entering the data writing phase.

[0105] Each TG reads curTgIdx (current TG number) and curWarpIdx (current Warp number) from the register of all Warp, and performs a single conditional judgment "curTgIdx == curWarpIdx" to determine whether the current data writing method is local writing or cross-TG writing.

[0106] If "curTgIdx == curWarpIdx" is true, the write mode is determined to be local write mode. Taking Warp0, which is currently TG0, as an example, Warp0 sends a local data write command to its parent TG0, which carries its own calculation result. After receiving the command, TG0 writes the calculation result carried in the command into TG0's GSM, and after completing the data write, sends a write completion command back to Warp0 of TG0. Then, Warp0 of TG0 sends a first command (such as mbar.arv count=1) to s_data_ready_mbar of TG0, notifying it to decrement the first count value. After receiving the first command, s_data_ready_mbar decrements its own first count value by 1 (updating from 4 to 3), and sends a count update completion command back to Warp0 of TG0. Warp0 of TG0 completes this stage of the operation.

[0107] If "curTgIdx == curWarpIdx" is not true, the write method is determined to be a cross-TG write method. Taking Warp0 of TG1 as an example, the target TG corresponding to Warp0 is TG0. Warp0 sends a remote data write command to TG0, carrying its own calculation result; after receiving the command, TG0 writes the calculation result carried in the command into TG0's GSM, and after completing the data write, sends a write completion command to warp0 of TG1; then, warp0 of TG1 sends a second command to s_data_ready_mbar of TG0, notifying it to deduct the byte count value; and sends a first command to s_data_ready_mbar of TG0, notifying it to deduct the first count value; after receiving the command, s_data_ready_mbar deducts its own first count value by 1 (updating from 3 to 2), and sends a count update completion command to warp0 of TG1. Warp1 of TG1 completes this stage of the operation.

[0108] When the first count value of s_data_ready_mbar in TG0 decreases to 0 and the byte count value also decreases to 0, s_data_ready_mbar sends a data write ready instruction to its parent TG0, informing TG0 that all warps have completed data writing and data transmission, the data write phase ends, and the data accumulation phase can begin. The warps in other TGs perform the same data write phase, which will not be described further.

[0109] After receiving the data write ready instruction sent by s_data_ready_mbar, TG0 starts its own data accumulation logic. Accumulation methods include single-warp accumulation or parallel accumulation. Single-warp accumulation refers to a single thread bundle within a thread group serially reading the written data from a specified memory location and completing the entire accumulation operation sequentially, while the other thread bundles remain idle and wait. In single-warp accumulation, the warp participating in the accumulation operation is that single thread bundle. Parallel accumulation refers to multiple thread bundles within a scheduled thread group reading data slices written from a specified memory location in parallel and simultaneously performing accumulation calculations. In parallel accumulation, the warp participating in the accumulation operation is the scheduled multiple thread bundles. Warp units within a TG that participate in the accumulation calculation can send a data read request command to their own GSM to request the reading of data written by all TG warp0 units (TG0's Warp0, TG1's Warp0, TG2's Warp0, and TG3's Warp0). After reading the data, they perform the accumulation calculation, and upon completion, send an accumulation completion command to the GSM unit to store the accumulation result.

[0110] After completing the accumulation calculation, the Warp component sends a third instruction to the s_buffer_ready_mbar of each TG, notifying it to deduct the second count value. Upon receiving the third instruction, each TG's s_buffer_ready_mbar deducts its second count value by 1, until the second count value is reduced to 0, at which point it is updated to the ready state.

[0111] The thread group data synchronization method provided in this application addresses the problems of instruction redundancy, excessive branch jumps, and low synchronization waiting efficiency in related parallel computing. By combining hardware built-in features and memory barrier parameter optimization, it simplifies the process and improves execution efficiency.

[0112] First, branch jump instructions are simplified based on hardware-built-in variables. This application establishes a direct mapping between thread groups and thread bundles using hardware-built-in variables. Relying on the current thread group number and current thread bundle number stored in the thread bundle scalar register, standardized branch jump conditions are set, replacing the multi-layered nested judgment logic in related technologies. Related technical solutions require separate judgment of whether the target thread group is the current thread group, setting two independent execution paths for local writes and cross-thread group writes, resulting in numerous branch jump instructions and long hardware execution time. The optimized solution achieves a one-to-one mapping of numbers through built-in variables, eliminating the need for additional condition filtering and directly unifying the instruction execution path, thus reducing branch judgments and jump instructions at the source. By reasonably adjusting the initialization count value of the data ready synchronization unit (s_data_ready_mbar), the logic of all-path instructions is unified, completely eliminating branch jumps for target thread group ownership judgment. The initial value of the first count of the data ready synchronization unit is adjusted from 12 to 16, covering the local thread group corresponding to the current computing unit. This allows both local writes and cross-thread group writes to send count deduction instructions in a unified manner, without distinguishing whether the target thread group is the current thread group. The original branch judgment logic is completely omitted, and all thread bundles execute the completely consistent instruction sending process, achieving path normalization and further reducing instruction jump overhead.

[0113] Secondly, the local data write synchronization waiting mechanism is optimized to achieve asynchronous synchronization. Hardware-level optimizations are made to the synchronization waiting logic of local data write instructions, abandoning the original ackl1 and waitl1 serial waiting execution method and reusing the memory barrier synchronization mechanism to achieve asynchronous waiting and notification. In related technologies, after local writing is completed, a dedicated instruction is needed to serially wait for cache synchronization, blocking the execution of subsequent instructions, resulting in low overall parallel efficiency. After optimization, after local data writing is completed, the data ready synchronization unit count is directly updated by decrementing the instruction (first instruction), relying on the memory barrier to achieve asynchronous notification. There is no need for serial blocking waiting; the data ready state is uniformly managed by the memory barrier. The synchronization logic for local writing and cross-thread group writing is completely unified, balancing execution efficiency and synchronization accuracy.

[0114] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0115] Based on the same inventive concept, this application also provides an inter-thread group data synchronization device for implementing the above-described inter-thread group data synchronization method. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the inter-thread group data synchronization device provided below can be found in the limitations of the inter-thread group data synchronization method described above, and will not be repeated here.

[0116] In one exemplary embodiment, such as Figure 5 As shown, a data synchronization device 500 between thread groups is provided, including: a comparison module 502, a data writing module 504, a first sending module 506, and a calculation module 508, wherein:

[0117] The comparison module 502 is used to compare the number of the current thread group with the number of the current thread bundle to obtain the comparison result;

[0118] The data writing module 504 is used to write the calculation result of the current thread bundle into a specified memory of the target thread group using the data writing method corresponding to the comparison result, wherein the number of the target thread group is consistent with the number of the current thread bundle;

[0119] The first sending module 506 is used to send a first instruction to the data ready synchronization unit of the target thread group. The first instruction is used to instruct the data ready synchronization unit to update a first count value. The first count value is used to determine whether the data ready synchronization unit is in a ready state.

[0120] The operation module 508 is configured to, for any of the target thread groups, if the data ready synchronization unit in the target thread group is in a ready state, load data from the designated memory of the target thread group for each thread bundle participating in the parallel accumulation operation, and perform the accumulation operation based on the loaded data.

[0121] The thread group data synchronization device provided in this application simplifies the data interaction and synchronization process of parallel accumulation operation to comparing the number of the current thread group with the number of the current thread bundle, and implementing the execution logic of targeted writing and targeted synchronization based on the number of the current thread bundle. This eliminates the redundant operation of double-nested traversal of all thread groups and the condition jump delay caused by multiple branch judgments in related technologies, significantly reducing the communication delay of data interaction and synchronization waiting between thread groups, improving the execution efficiency of parallel accumulation operation, and thus improving the overall computing performance of artificial intelligence chip.

[0122] In one embodiment, writing the calculation result of the current thread bundle into the designated memory of the target thread group using the data writing method corresponding to the comparison result includes:

[0123] If the comparison result indicates that the number of the current thread group is consistent with the number of the current thread bundle, the calculation result of the current thread bundle is written to the designated memory of the current thread group using a local data write instruction.

[0124] In one embodiment, writing the calculation result of the current thread bundle into the designated memory of the target thread group using the data writing method corresponding to the comparison result includes:

[0125] If the comparison result indicates that the number of the current thread group is inconsistent with the number of the current thread bundle, the target thread group is determined according to the number of the current thread bundle, and the calculation result of the current thread bundle is written to the designated memory of the target thread group using a remote data write instruction.

[0126] In one embodiment, the device further includes:

[0127] The second sending module is used to send a second instruction to the data ready synchronization unit of the target thread group after the data writing is completed using the off-site data writing instruction. The second instruction is used to instruct the data ready synchronization unit to update the byte count value.

[0128] The determination module is used to determine that the data ready synchronization unit in the target thread group is in a ready state when the first count value is updated to 0 and the byte count value is also updated to 0.

[0129] In one embodiment, the device further includes:

[0130] The third sending module is used to send a third instruction to the buffer ready synchronization unit of each thread group after any of the thread bundles has completed the accumulation operation. The third instruction instructs the buffer ready synchronization unit to update the second count value, and when the second count value is updated to 0, it is determined that the buffer ready synchronization unit is in a ready state.

[0131] In one embodiment, the initial value of the first count of the data ready synchronization unit is N, where N is the number of thread groups in the cluster; the initial value of the byte count of the data ready synchronization unit is D*(N-1), where D is the number of bytes written in a single operation; and the initial value of the second count of the buffer ready synchronization unit is N*M, where M is the number of thread bundles participating in the accumulation operation within the thread group, where N, D, and M are all positive integers.

[0132] Each module in the aforementioned inter-thread data synchronization device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0133] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 6As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a method for data synchronization between thread groups. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0134] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0135] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0136] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0137] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0138] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0139] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0140] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0141] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for data synchronization between thread groups, characterized in that, The method includes: The current thread group number is compared with the current thread bundle number to obtain the comparison result; Using the data writing method corresponding to the comparison result, the calculation result of the current thread bundle is written to the specified memory of the target thread group, wherein the number of the target thread group is consistent with the number of the current thread bundle; Send a first instruction to the data ready synchronization unit of the target thread group. The first instruction is used to instruct the data ready synchronization unit to update a first count value. The first count value is used to determine whether the data ready synchronization unit is in a ready state. For any of the target thread groups, if the data-ready synchronization unit in the target thread group is in a ready state, then each thread bundle participating in the parallel accumulation operation in the target thread group loads data from the designated memory of the target thread group and performs the accumulation operation based on the loaded data.

2. The method according to claim 1, characterized in that, The step of writing the calculation result of the current thread bundle into the designated memory of the target thread group using the data writing method corresponding to the comparison result includes: If the comparison result indicates that the number of the current thread group is consistent with the number of the current thread bundle, the calculation result of the current thread bundle is written to the designated memory of the current thread group using a local data write instruction.

3. The method according to claim 1, characterized in that, The step of writing the calculation result of the current thread bundle into the designated memory of the target thread group using the data writing method corresponding to the comparison result includes: If the comparison result indicates that the number of the current thread group is inconsistent with the number of the current thread bundle, the target thread group is determined according to the number of the current thread bundle, and the calculation result of the current thread bundle is written to the designated memory of the target thread group using a remote data write instruction.

4. The method according to claim 3, characterized in that, The method further includes: After completing the data writing using the remote data writing instruction, a second instruction is sent to the data ready synchronization unit of the target thread group. The second instruction is used to instruct the data ready synchronization unit to update the byte count value. When the first count value is updated to 0 and the byte count value is also updated to 0, it is determined that the data ready synchronization unit in the target thread group is in a ready state.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: After any of the thread bundles completes the accumulation operation, a third instruction is sent to the buffer ready synchronization unit of each thread group. The third instruction instructs the buffer ready synchronization unit to update the second count value, and when the second count value is updated to 0, it is determined that the buffer ready synchronization unit is in a ready state.

6. The method according to claim 5, characterized in that, The initial value of the first count of the data ready synchronization unit is N, where N is the number of thread groups in the cluster. The initial value of the byte count of the data ready synchronization unit is D*(N-1), where D is the number of bytes written in a single operation. The initial value of the second count of the buffer ready synchronization unit is N*M, where M is the number of thread bundles participating in the accumulation operation within the thread group. N, D, and M are all positive integers.

7. A data synchronization device between thread groups, characterized in that, The device includes: The comparison module is used to compare the number of the current thread group with the number of the current thread bundle to obtain the comparison result; The data writing module is used to write the calculation result of the current thread bundle into a specified memory of the target thread group using the data writing method corresponding to the comparison result, wherein the number of the target thread group is consistent with the number of the current thread bundle; The first sending module is used to send a first instruction to the data ready synchronization unit of the target thread group. The first instruction is used to instruct the data ready synchronization unit to update a first count value. The first count value is used to determine whether the data ready synchronization unit is in a ready state. The computation module is configured to, for any of the target thread groups, if the data-ready synchronization unit in the target thread group is in a ready state, load data from the designated memory of the target thread group for each thread bundle participating in the parallel accumulation operation, and perform the accumulation operation based on the loaded data.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images