Data multicast method, device and system in multi-core computing cluster

Abstract

The invention relates to the technical field of multi-core computing, and provides a data multicast method, device and system in a multi-core computing cluster. The method comprises the following steps: receiving a data loading instruction from each thread; the data loading instruction carries a data address, a thread ID and a thread group mask; the data loading instructions corresponding to the plurality of first threads have the same thread group mask, the thread group mask also represents the number of the first threads, and the first threads refer to threads multiplexing first data; determining whether data loading instructions of all first threads are received or not according to the received thread group masks and the number of the same thread group masks; after it is determined that the data loading instructions of all the first threads are received, the first data are written into all the first threads. According to the method and the device, the thread group mask is added in the data loading instruction, so that the thread group mask is used for monitoring and managing the storage period of the corresponding data, and normal operation of data multicast multiplexing is ensured.

Description

Technical Field

[0001] The present invention relates to the field of multi-core computing technology, and in particular to a method, device and system for multicasting data in a multi-core computing cluster. Background Art

[0002] In existing high-performance computing processors, there are usually multiple computing cores, which form a computing cluster to jointly complete a computing task that can achieve data parallelism. These data parallel computing tasks have a certain degree of data reuse in terms of data.

[0003] In the prior art, there are two implementation methods. The first implementation method is: each computing core reads data separately. This method directly causes a piece of data to be read repeatedly, resulting in low data reading efficiency. In addition, due to the increased data transmission volume, it has higher requirements for the external bandwidth of the computing core cluster. When the external bandwidth of the computing core cluster is not sufficient, it will directly affect the speed of data parallel processing. The second implementation method is: the computing cluster will be configured with a corresponding cache area. The space of the cache area is usually small and is only used to store a limited number of data. For the same data request, only one request will be sent to the external memory bus through the cache area. Subsequent requests can reuse the data stored in the cache area. This method has a strong dependence on the scheduling behavior of hardware and software. If the time difference between the access of the same data request is very long, the cached data will be overwritten by other instructions, causing the old data to be kicked out of the cache area, resulting in the inability to realize data reuse.

[0004] In view of this, overcoming the defects of the prior art is an urgent problem to be solved in the field of this technology. Summary of the invention

[0005] The technical problem to be solved by the present invention is that in the existing data reading method, if the time points of access to the same data request are very different, the cached data will be overwritten by other instructions, causing the old data to be kicked out of the cache area, making it impossible to realize data reuse.

[0006] The present invention adopts the following technical solution: In a first aspect, the present invention provides a method for multicasting data in a multi-core computing cluster, comprising: Receive data loading instructions from each thread; wherein the data loading instructions carry the address of the data, the thread ID and the thread group mask; wherein the data loading instructions corresponding to the plurality of first threads have the same thread group mask, and the thread group mask also represents the number of first threads, and the first threads refer to threads that reuse the first data; Determining whether data loading instructions of all first threads are received according to the received thread group mask and the number of thread group masks with the same thread group mask; After determining that the data loading instructions of all the first threads are received and the first data are written to each first thread, the instructions related to the first data are released.

[0007] Preferably, the thread group mask includes N bits, and in the first thread group mask, the bits corresponding to all first threads are 1, and the other bits are 0; The determining, according to the received thread group mask and the number of the same thread group mask, whether the data loading instructions of all the first threads are received specifically includes: Each time a first data loading instruction is received, a corresponding bit in the first arrival mask is updated to 1 according to the thread ID in the first data loading instruction; wherein, in an initial state, all bits of the first arrival mask are 0; and the first data loading instruction is a data loading instruction carrying a first thread group mask; It is determined whether the first arrival mask is consistent with the first thread group mask. If it is determined that the first arrival mask is consistent with the first thread group mask, it is determined that data loading instructions for all first threads are received.

[0008] Preferably, the first arrival mask is established when the first data loading instruction is received for the first time, and specifically includes: When receiving the corresponding first data loading instruction, determining whether the first thread group mask carried in the first data loading instruction exists in the multicast request table; If the first thread group mask does not exist in the multicast request table, the first thread group mask is added to the multicast request table, and a first arrival mask is generated accordingly, where all bits of the first arrival mask are 0; And according to the thread ID in the first data loading instruction, the corresponding bit in the first arrival mask is updated to 1; If the first thread group mask exists in the multicast request table, the first arrival mask is directly updated.

[0009] Preferably, the method further comprises: Deduplication processing is performed on a plurality of first data loading instructions from different threads, so that the first data is read from the external storage once.

[0010] In a second aspect, the present invention further provides a method for multicasting data in a multi-core computing cluster, comprising: Sending a data loading instruction; wherein the data loading instruction carries the address of the data, the thread ID of the thread to which the data loading instruction belongs, and the thread group mask; wherein the data loading instructions corresponding to the plurality of first threads have the same thread group mask, and the thread group mask also represents the number of the first threads, and the first thread refers to the thread that reuses the first data; This allows the receiving party to determine whether the data loading instructions of all first threads are received based on the received thread group mask and the number of identical thread group masks, and release the instructions related to the first data after determining that the data loading instructions of all first threads are received and the first data are written to each first thread.

[0011] Preferably, before sending the data loading instruction, the method further includes: According to the operation to be participated in by the matrix, a plurality of first threads that need to reuse the first data in the matrix are determined, and a same first thread group mask is generated for the plurality of first threads.

[0012] Preferably, generating the same first thread group mask for the multiple first threads specifically includes: Generate N bits; N is the total number of all threads; Each bit corresponds to a thread, the bit corresponding to the first thread is set to 1, and the bits corresponding to other threads are set to 0, to obtain the first thread group mask.

[0013] In a third aspect, the present invention further provides a data multicast device in a multi-core computing cluster, for implementing the data multicast method in a multi-core computing cluster described in the first aspect, the device comprising: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor to execute the data multicast method within a multi-core computing cluster described in the first aspect or the data multicast method within a multi-core computing cluster described in the second aspect.

[0014] In a fourth aspect, the present invention further provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, which are executed by one or more processors to complete the data multicast method in the multi-core computing cluster described in the first aspect. In an actual application scenario, the non-volatile computer storage medium may be a memory on the side of a central processing unit (CPU) or a memory on the side of a graphics processing unit (GPU), and the object for executing the computer executable instructions may be the CPU itself or the GPU itself, or other objects, such as a response multicast module.

[0015] In a fifth aspect, the present invention further provides a data multicast system in a multi-core computing cluster, comprising a first data multicast device and a second data multicast device; the first data multicast device and the second data multicast device are both the data multicast device in the multi-core computing cluster described in the third aspect; Each of them is used to execute the data multicast method within a multi-core computing cluster described in any one of the first aspect and the data multicast method within a multi-core computing cluster described in the second aspect, and complete the interaction between the first data multicast device and the second data multicast device.

[0016] The present invention adds a thread group mask to a data loading instruction, thereby using the thread group mask to monitor and manage the storage cycle of the corresponding data, that is, before receiving the data loading instructions of all the first threads, the read data is always stored in the cache until the data loading instructions of all the first threads are received, and the read data is written to each first thread, or, when the corresponding data loading instruction is received, the data is written to the corresponding first thread, and after receiving the data loading instructions of all the first threads, the instruction is released to ensure the normal data multicast multiplexing. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 It is a flowchart of a first data multicast method in a multi-core computing cluster provided by an embodiment of the present invention; Figure 2 is a schematic diagram of a first data multicast method within a multi-core computing cluster provided by an embodiment of the present invention; Figure 3It is a flowchart of a second method for multicasting data in a multi-core computing cluster provided by an embodiment of the present invention; Figure 4 It is a flowchart of a third method for multicasting data in a multi-core computing cluster provided by an embodiment of the present invention; Figure 5 It is a schematic diagram of a multicast request table in a data multicast method in a multi-core computing cluster provided by an embodiment of the present invention; Figure 6 is a schematic diagram of a second data multicast method within a multi-core computing cluster provided by an embodiment of the present invention; Figure 7 is a flowchart of a fourth method for multicasting data in a multi-core computing cluster provided by an embodiment of the present invention; Figure 8 This is a schematic diagram of an application scenario of a data multicast method within a multi-core computing cluster provided by an embodiment of the present invention; Fig. 9 is a schematic diagram of a third data multicast method within a multi-core computing cluster provided by an embodiment of the present invention; Fig.10 is a schematic diagram of a fourth data multicast method within a multi-core computing cluster provided by an embodiment of the present invention; Fig.11 This is a schematic diagram of an application scenario of another method for multicasting data in a multi-core computing cluster provided by an embodiment of the present invention; Fig.12 is a flowchart of a fifth method for multicasting data in a multi-core computing cluster provided by an embodiment of the present invention; Fig.13 is a flowchart of a sixth method for multicasting data in a multi-core computing cluster provided by an embodiment of the present invention; Fig.14 It is a schematic diagram of the architecture of a data multicast device in a multi-core computing cluster provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0020] Unless the context requires otherwise, throughout the specification and claims, the term "including" is to be interpreted as open inclusion, that is, "including, but not limited to". In the description of the specification, the terms "one embodiment", "some embodiments", "exemplary embodiments", "examples", "specific examples" or "some examples" and the like are intended to indicate that specific features, structures, materials or characteristics associated with the embodiment or example are included in at least one embodiment or example of the present disclosure. The schematic representation of the above terms does not necessarily refer to the same embodiment or example. In addition, the specific features, structures, materials or characteristics may be included in any one or more embodiments or examples in any appropriate manner, that is, although they may be carried in the embodiments or examples of the above terms due to reasons such as the order and position of appearance, it is not limited to that they can be carried in combination by one embodiment or example.

[0021] In the description of the present invention, the terms "first" and "second" are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, the meaning of "multiple" is two or more. In addition, for example, the same type of nouns may be described as two independent individuals by adding "A" and "B" at the end. In this case, the corresponding features defined as "A" and "B" are only used to distinguish the same type of individuals for description purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features.

[0022] When describing some embodiments, the expressions "coupling", "coupling" and "connection" and their derivatives may be used. For example, when describing some embodiments, the term "connection" may be used to indicate that two or more components are in direct physical or electrical contact with each other. For another example, when describing some embodiments, the term "coupling" may be used to indicate that two or more components are in direct physical or electrical contact. However, the terms "connection" or "coupling" may also refer to two or more components that are not in direct contact with each other, but still cooperate or interact with each other, such as "optical path coupling", "wireless connection", etc. The embodiments disclosed herein are not necessarily limited to the contents of the present invention.

[0023] In the description of the present invention, the expression "A and / or B" (where A and B are used to formally represent specific characteristic contents) is involved, and the corresponding expressions include the following three combinations: only A, only B, and a combination of A and B.

[0024] As used herein, "about," "substantially," or "approximately" includes the stated value and an average value that is within an acceptable range of deviation from the particular value as determined by one of ordinary skill in the art taking into account the measurements in question and the errors associated with the measurement of the particular quantity (i.e., the limitations of the measurement system).

[0025] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0026] The embodiment of the present invention provides a data multicast method in a multi-core computing cluster. The execution subject of the method is a multicast control module, such as Figure 1 As shown, including: In step 201, a data loading instruction is received from each thread; wherein the data loading instruction carries the address of the data, the thread ID and the thread group mask; wherein the data loading instructions corresponding to multiple first threads have the same thread group mask, and the thread group mask also represents the number of first threads, the first thread refers to the thread that reuses the first data; the size of the data read by the data loading instruction can be determined by a person skilled in the art according to the actual scenario requirements analysis, the first data does not represent specific data, but is used to refer to one of the multiple data, which can be understood as: all multiple threads that reuse the first data are called first threads, that is, multiple first threads all need the first data to participate in the calculation in the corresponding computing core, which can be understood as multiple first threads all need to read the first data from external storage. In a specific application scenario, the data can be an element or a matrix block in a matrix.

[0027] It should be emphasized here that the first data does not refer to a specific data, but is a word used to highlight a certain data from multiple data for the purpose of description. For example, it can be understood that: the matrix contains multiple data, each data is called the first data, the second data, ..., the nth data, and correspondingly, all threads that reuse the first data are called the first thread, and the data loading instructions of the multiple first threads have the same thread group mask, and the thread group mask is called the first thread group mask. All threads that reuse the second data are called the second thread, and the data loading instructions of the multiple second threads have the same thread group mask, and the thread group mask is called the second thread group mask. By analogy, all threads that reuse the nth data are called the nth thread, and the data loading instructions of the multiple nth threads have the same thread group mask, and the thread group mask is called the nth thread group mask; and each thread group mask is different, that is, the first thread group mask, the second thread group mask, ..., the nth thread group mask are different, such as Fig. 9As shown, taking matrix A as an example, A0 can be used as the first data, then T0, T1, T2 and T3 are all first threads, and the corresponding thread group mask is 00001111; A1 is used as the second data, T4, T5, T6 and T7 are all second threads, and the corresponding thread group mask is 11110000. Since the processing corresponding to the second data, the third data, ..., the nth data is implemented based on the same concept as the processing of the first data, it will not be described in detail.

[0028] The thread may be a thread similar to a CPU architecture, or a set of threads bundled and run in a single instruction stream multiple data stream (SIMD) structure in a GPU architecture. The thread described in this embodiment is an example of a thread in a thread group, which is also called a thread group.

[0029] In step 202, it is determined whether data loading instructions for all first threads are received based on the received thread group mask and the number of thread group masks with the same thread group mask; that is, when the number of first threads represented by the thread group mask is consistent with the number of received thread group mask, it is considered that the data loading instructions for all first threads corresponding to the thread group mask are received; otherwise, it is considered that the data loading instructions for all first threads corresponding to the thread group mask are not received, including the case where data loading instructions for some of all first threads corresponding to the thread group mask are received.

[0030] In step 203, after determining that the data loading instructions of all first threads are received and the first data is written to each first thread, the instructions related to the first data are released. The writing to the first thread is actually writing to the register of the first thread, each first thread is obtained by the thread ID carried in the data loading instruction, and the first data is obtained by reading the address of the data carried in the data loading instruction.

[0031] In actual use, the step of releasing the first data-related instructions specifically includes: releasing all data loading instructions of the first thread, that is, releasing all data loading instructions corresponding to the thread group mask.

[0032] There are multiple implementations of writing the first data into each first thread, including: The first method: before receiving the data loading instructions of all the first threads corresponding to the thread group mask, the first data read is stored in the cache, and the location of the first data is locked so as not to be overwritten. After determining that the data loading instructions of all the first threads are received, the first data in the cache is written to each first thread.

[0033] The second method: before receiving the data loading instructions of all the first threads corresponding to the thread group mask, write the first data into the corresponding first thread according to the received data loading instructions, and after determining that the data loading instructions of all the first threads have been received, release the data loading instructions corresponding to the thread group mask. The reading of the first data may be performed when the first data loading instruction of the thread group mask is received.

[0034] The thread group mask can be implemented in a variety of ways, including: Implementation method 1: Use the number of first threads and the corresponding ID number (i.e., the ID number of the thread group mask described below) to form a thread group mask, where different thread group masks have different ID numbers; for example, the first 4 bits can be preset to store the ID number, and the last 4 bits can be preset to store the number of first threads, thereby forming an 8-bit thread group mask. Multiple threads corresponding to a thread group mask can be uniformly regarded as a multicast group, or a multicast group can correspond to an ID number, and the ID number can be placed in the thread group mask for use.

[0035] Implementation method 2: According to the number of threads N in the thread group, N bits are pre-set, each bit represents a corresponding thread, and different values ​​are used to represent whether the corresponding thread is the first thread. Figure 2 For example, suppose there are 8 threads, namely T0, T1, T2, T3, T4, T5, T6 and T7, where T0, T1, T2 and T3 need to reuse the same data, T4, T5, T6 and T7 need to reuse another data, and the i-th bit in the thread group mask represents thread Ti. Then for T0, T1, T2 and T3, they have a common thread group mask of 11110000, and for T4, T5, T6 and T7, they have a common thread group mask of 00001111, where the i-th bit is 1, indicating that Ti is the first thread in the corresponding thread group mask. In another embodiment, it is also feasible to use the i-th bit as 0 to represent that Ti is the first thread in the corresponding thread group mask. In this case, T0, T1, T2 and T3 have a common thread group mask of 00001111, and T4, T5, T6 and T7 have a common thread group mask of 11110000.

[0036] This embodiment adds a thread group mask to the data loading instruction, thereby using the thread group mask to monitor and manage the storage cycle of the corresponding data, that is, before receiving the data loading instructions of all the first threads, the read data is always stored in the cache until the data loading instructions of all the first threads are received, and the read data is written to each first thread, or, when the corresponding data loading instruction is received, the data is written to the corresponding first thread, and after receiving the data loading instructions of all the first threads, the instruction is released to ensure the normal data multicast multiplexing.

[0037] When the above implementation mode 1 is adopted, determining whether the data loading instructions of all first threads are received according to the received thread group mask and the number of the same thread group mask specifically includes: A temporary variable is set for the ID number of each thread group mask, and the initial value of the temporary variable is 0. Each time a first data loading instruction is received, the corresponding temporary variable is found according to the ID number of the thread group mask in the first data loading instruction, and the value of the temporary variable is increased by 1. When the value of the temporary variable increases to the same number represented by the thread group mask corresponding to the ID number, it is determined that the data loading instructions of all first threads have been received.

[0038] Taking the above implementation mode 2 as an example, that is, when the thread group mask includes N bits, in the first thread group mask, the bits corresponding to all first threads are 1, and the other bits are 0.

[0039] The step of determining whether data loading instructions for all first threads are received is performed according to the received thread group mask and the number of thread group masks having the same thread group mask, such as Figure 3 As shown, specifically including: In step 301, each time a first data loading instruction is received, the corresponding bit in the first arrival mask is updated to 1 according to the thread ID in the first data loading instruction; wherein, in an initial state, all bits of the first arrival mask are 0; and the first data loading instruction is a data loading instruction carrying a first thread group mask.

[0040] In step 302, it is determined whether the first arrival mask is consistent with the first thread group mask. If it is determined that the first arrival mask is consistent with the first thread group mask, it is determined that data loading instructions for all first threads are received.

[0041] In actual use, when the thread group mask includes N bits, in the first thread group mask, the bits corresponding to all first threads are 0, and the other bits are 1; the above step 301 may also be: Each time a first data loading instruction is received, the corresponding bit in the first arrival mask is updated to 0 according to the thread ID in the first data loading instruction; wherein, in an initial state, all bits of the first arrival mask are 1; and the first data loading instruction is a data loading instruction carrying a first thread group mask.

[0042] Still taking the example that in the first thread group mask, the bits corresponding to all first threads are 1 and the other bits are 0, wherein the first arrival mask is established when the first data loading instruction is received for the first time, such as Figure 4 As shown, specifically including: In step 401, when a corresponding first data loading instruction is received, it is determined whether a first thread group mask carried in the first data loading instruction exists in a multicast request table.

[0043] In step 402, if the first thread group mask does not exist in the multicast request table, the first thread group mask is added to the multicast request table, and a first arrival mask is generated accordingly, where all bits of the first arrival mask are 0.

[0044] And according to the thread ID in the first data loading instruction, the corresponding bit in the first arrival mask is updated to 1.

[0045] In step 403, if the first thread group mask exists in the multicast request table, the first arrival mask is directly updated.

[0046] The multicast request table is as follows: Figure 5 As shown in , one thread group mask corresponds to one arrival mask. In actual use, Figure 6 As shown, there is also an optional implementation: using N bits as the thread ID of each thread, wherein each bit represents a thread, the i-th bit of the thread ID of the i-th thread is 1, and the other bits are 0.

[0047] Correspondingly, when the corresponding data loading instruction is received, the arrival mask corresponding to the thread group mask in the data loading instruction is found, and the thread ID in the data loading instruction is ORed with the arrival mask to obtain a new arrival mask. For example, the thread group mask carried in the corresponding data loading instruction is 00001111, and the thread ID carried is 00001000. In the multicast request table, the arrival mask corresponding to the thread group mask is 00000101. The arrival mask is ORed with the thread ID, that is, 00000101|00001000=00001101, and the new arrival mask 00001101 is updated to the multicast request table.

[0048] In actual use, the method further includes: performing deduplication processing on multiple first data loading instructions from different threads, so that the first data is read from the external storage once. For example, when the first data loading instruction is received for the first time, the first data is read from the external storage, the first data is stored in the cache, and after confirming that the data loading instructions of all the first threads are received, the first data is written to each thread. Considering that in actual use of the above method, when the number of first threads is large, it may cause the first data to occupy the cache for a long time, so on the basis of the above first method, this embodiment also provides a preferred implementation method: when the first data loading instruction is received for the nth time, the first data is read from the external storage, the first data is stored in the cache, and after confirming that the data loading instructions of all the first threads are received, the first data is written to each thread; wherein, the closer n is to the total number of the first threads, the shorter the time the first data occupies the cache, but correspondingly, when n is too close to the total number of the first threads, the last first data loading instruction may be received in the process of reading the first data, and at this time, the reading of the first data may not be completed, so the specific value of n can be obtained by those skilled in the art according to needs and experience analysis.

[0049] The above-mentioned data multicast method within a multi-core computing cluster is described with the multicast control module as the execution body. The embodiment of the present invention will further provide a data multicast method within a multi-core computing cluster, in order to make relevant explanations from the perspective of computing cores (i.e., threads) and further make relevant in-depth analysis on its design principles.

[0050] This embodiment also provides a data multicast method within a multi-core computing cluster, such as Figure 7 As shown, including: In step 501, a data loading instruction is sent; wherein the data loading instruction carries the address of the data, the thread ID of the thread to which the data loading instruction belongs, and the thread group mask; wherein the data loading instructions corresponding to multiple first threads have the same thread group mask, and the thread group mask also represents the number of first threads, and the first thread refers to the thread that reuses the first data.

[0051] In step 502, the receiving party determines whether the data loading instructions of all first threads are received according to the received thread group mask and the number of the same thread group mask, and writes the first data to each first thread after determining that the data loading instructions of all first threads are received. The receiving party is the multicast control module mentioned above.

[0052] In an actual application scenario, before sending the data loading instruction, the method further includes: According to the operation to be participated in by the matrix, a plurality of first threads that need to reuse the first data in the matrix are determined, and a same first thread group mask is generated for the plurality of first threads.

[0053] Taking the above-mentioned second implementation method as an example, generating the same first thread group mask for the multiple first threads specifically includes: generating N bits; N is the total number of all threads; making each bit correspond to a thread, setting the bit corresponding to the first thread to 1, and setting the bits corresponding to other threads to 0, to obtain the first thread group mask.

[0054] by Figure 8 The 2×1 matrix A is multiplied by the 1×4 matrix B to obtain the 2×4 matrix C. For example, the number of threads involved in the process is 2×4=8, that is, N=8. Figure 8 The T0, T1, T2, T3, T4, T5, T6 and T7 in the matrix A are collectively referred to as a thread group. During the calculation process, threads T0, T1, T2 and T3 need to access A0 in the matrix A. The threads that reuse A0 are T0, T1, T2 and T3. The thread group mask generated for A0 is 000011111, that is, in the data loading instructions sent by threads T0, T1, T2 and T3, the thread group mask carried is 000011111; threads T4, T5, T6 and T7 need to access A1 in the matrix A, and the thread group mask of A1 is 111100000, as shown in Fig. 9 shown.

[0055] Threads T0 and T4 need to use B0 in matrix B, and the thread group mask corresponding to B0 is 10001000; threads T1 and T5 need to use B1 in matrix B, and the thread group mask corresponding to B1 is 01000100; threads T2 and T6 need to use B2 in matrix B, and the thread group mask corresponding to B2 is 00100010; threads T3 and T7 need to use B3 in matrix B, and the thread group mask corresponding to B3 is 00010001, as shown in Fig.10 shown.

[0056] This embodiment also provides a method for generating a thread group mask for a multiplication operation, which specifically includes: When N threads are used to calculate the multiplication operation of the m×n first matrix M1 and the n×s second matrix M2, when reading the first matrix M1, in the thread group mask corresponding to the i-th thread, the [(i / s)×s+1, (i / s)×s+s]th bit is 1, and the other bits are 0; i is a positive integer, i is less than or equal to N.

[0057] When reading the second matrix M2, in the thread group mask corresponding to the i-th thread, the i%s+j×sth bit is 1, and the other bits are 0; j is an integer, and j is less than m. The order of the bits can be obtained by sorting from high to low, or from low to high.

[0058] In the above implementation, the address of the data required by the corresponding thread is calculated according to the storage method of the matrix in the external storage. When the storage method of the matrix is ​​row-major storage, the address of the jth data in the i-th row in the matrix is ​​addr[i,j]=base+(i×cols+j-1)×size; where base is the starting storage address of the matrix, size is the size of each data in the matrix, and cols is the number of columns in the matrix. When the storage method of the matrix is ​​column-major storage, the address of the jth data in the i-th row in the matrix is ​​addr[i,j]=base+((j-1)×rows+i-1)×size; where rows is the number of rows in the matrix.

[0059] The method described in this embodiment can be applied to a variety of scenarios, including being applied to a CPU, a GPU, or a neural processing unit (Neural Processing Unit, referred to as: NPU).

[0060] The following will combine specific application scenarios and use technical descriptions in related scenarios to explain the implementation process of the present invention in the characteristic scenarios.

[0061] This embodiment takes the example of multiple threads cooperating to complete a large matrix multiplication to describe how the user can use the load instruction (i.e. the above-mentioned data loading instruction) to control the hardware to complete the required multicast function and realize efficient data reuse. It should be noted that using the load instruction as the data loading instruction is only an optional implementation method. In actual use, it is also feasible to use other instructions representing data reading and writing functions as data loading instructions. For example, a custom martix load instruction can be used as a data loading instruction. The martix load instruction can be used to load all elements of a row in the matrix, all elements of a column, or read one or more elements in the matrix in other ways.

[0062] Still Figure 8For example, we need to complete the calculation process of matrix A × matrix B = matrix C, where matrix A can be divided into two small matrix blocks A0 and A1, matrix B can be divided into matrices B0, B1, B2 and B3, and matrix C can be divided into C0, C1, C2, C3, C4, C5, C6 and C7. In order to complete the matrix multiplication, assuming that a single computing cluster can support 8 threads running simultaneously, each small Ci can be mapped to the corresponding thread and marked as Ti, corresponding to T0, T1, T2, ..., T7. The calculations that each thread needs to complete are as follows: Thread T0 executes process C0=A0×B0, thread T1 executes process C1=A0×B1, thread T2 executes process C2=A0×B2, thread T3 executes process C3=A0×B3, thread T4 executes process C4=A1×B0, thread T5 executes process C5=A1×B1, thread T6 executes process C6=A1×B2, and thread T7 executes process C7=A1×B3. It can be seen that threads T0, T1, T2 and T3 can share data A0; threads T4, T5, T6 and T7 can share data A1; threads T0 and T4 can share data B0; threads T1 and T5 can share data B1; threads T2 and T6 can share data B2; threads T3 and T7 can share data B3.

[0063] The existing load instruction includes two parameters, namely, the source address load dst and the destination address src_addr. In order to support multicast control, we add multiple fields to the instruction, including thread id and thread group mask. Considering that in actual use, there may be some data that does not need to be reused. If all data is directly processed in a multiplexed manner, it may result in too many entries in the multicast request table, thereby occupying more space. Therefore, this embodiment also adds a multicast enable field to control whether the multicast function is enabled.

[0064] That is, the data loading instruction provided in this embodiment includes parameters including load dst, src_addr, multicast_en, thread_id and thread_group_mask, wherein multcast_en, thread_id and thread_group_mask are newly added control fields for displaying and controlling hardware multicast.

[0065] The multicast_en field is used to control whether the instruction requires multicast functionality.

[0066] The thread_id field is the thread identifier of the thread in the computing cluster, that is, the thread ID.

[0067] The thread_group_mask field is a multi-bit information, each bit of which corresponds to each thread in the thread group, and is used to indicate which threads in the thread group need to use the data of this data read instruction, that is, the thread group mask.

[0068] In order to realize the calculation of matrix A×matrix B=matrix C, each thread first sends a data loading instruction to the multicast control module, and specifically executes the following code: thread_id = get_thread_id(); / / Get the thread ID of the thread in the thread group / / Calculate the address of the matrix block corresponding to the thread. / / mat_a_stride is the size of each block, assuming A0, A1 are continuous, B0..B3 are stored continuously.

[0069] mat_a_addr = a_base_address + (thread_id / 4) mat_a_stride; mat_b_addr = b_base_addr + (thread_id%4) mat_b_stride; / / Generate multicast mask of matrix a if(thread_id / 4 == 0){ / / thread 0, 1, 2, 3 mat_a_multicast_mask = 0b00001111; / / threads with thread_id=0, 1, 2, 3 use thread group mask 0b00001111 } else { mat_a_multicast_mask = 0b11110000; / / threads with thread_id=4, 5, 6, 7 use thread group mask 0b11110000 } / / Generate multicast mask of matrix b switch (thread_id%4){ case 0: mat_b_multicast_mask = 0b00010001; / / threads with thread_id=0 and 4 use thread group mask 0b00010001 case 1: mat_b_multicast_mask = 0b00100010; / / threads with thread_id=1 and 5 use thread group mask 0b00100010 case 2: mat_b_multicast_mask = 0b01000100; / / threads with thread_id=2 and 6 use thread group mask 0b01000100 case 3: mat_b_multicast_mask = 0b10001000; / / threads with thread_id=3 and 7 use thread group mask 0b10001000 } load mat_a, mat_a_addr, MCAST_EN, thread_id, mat_a_multicast_mask; load mat_b, mat_b_addr, MCAST_en, thread_id, mat_b_multicast_mask; From the above code, we can see that each thread can calculate the address of the data mat_a and mat_b it needs based on its thread ID, and needs to further calculate the thread_group_mask of mat_a and mat_b to tell the hardware (i.e., the multicast control module) the multicast group where the mat_a and mat_b of this thread are located (i.e., multiple threads that reuse mat_a and multiple threads that reuse mat_b). When the hardware receives a load instruction with multicast enable, it can know which threads the data that this load instruction needs to read needs to be multicasted to based on the thread_group_mask in the instruction. The size of the data read by the load instruction can be determined according to the actual scenario. After the data of the load instruction is returned, the data is multicasted and written to the registers of the corresponding multiple threads to complete the multicast function.

[0070] like Fig.11 As shown in the figure, a computing core is the smallest computing unit, which corresponds to one thread. In a computing cluster, there are multiple computing cores. A computing cluster is responsible for processing one thread group, and each thread in the thread group is executed by a computing core in the computing cluster.

[0071] The multicast control module is responsible for receiving data loading instructions from the computing cluster. The multicast control module includes a request merging module and a response multicast module. The request merging module is responsible for the merging control of data loading instructions, that is, for data loading instructions with the same thread_group_mask, only one read instruction will be sent to the memory fabric. The response multicast module is responsible for multicast operations, that is, after receiving the returned data from the external storage, it will write the returned data to the register file of the corresponding thread according to the thread_group_mask information in the data loading instruction.

[0072] The functions performed by the request merging module are as follows: Fig.12 As shown, specifically including: Maintain an N-item multicast request table, each of which includes at least thread_group_mask and multicastarrive mask (i.e., arrival mask). In actual use, each item also includes a 1-bit valid field, thread groupid (i.e., thread group ID) and instruction id (i.e., multicast group ID).

[0073] The bit width of the thread group id is determined by the number of thread groups that need to be maintained in the specific hardware. In a computing cluster, multiple thread groups can usually run simultaneously. In order to distinguish each thread group, each thread group is assigned a thread group ID.

[0074] The bit width of the instruction id is also determined by the number of multicast instructions that the hardware needs to support running simultaneously. Each thread also runs multiple data load instructions at the same time. In order to distinguish each data load instruction, each data load instruction is also bound to an instruction ID.

[0075] The bit width of thread_group_mask and multicast arrive mask is also determined according to the number of threads in the thread group supported by the hardware.

[0076] The multicast arrive mask is used to mark the threads that have arrived in the multicast group.

[0077] Through the thread group ID, instruction ID and thread_group_mask, the hardware can uniquely identify each multicast group.

[0078] That is, based on the comparison between the thread group id, instruction id and thread_group_mask of the current instruction and all valid entries in the multicast request table, if there is an entry in the table whose thread group id, instruction id and thread_group_mask value are exactly the same as those of the instruction, then they belong to the same multicast group.

[0079] The load instruction of the same multicast group only needs to be sent once. If the newly received instruction finds the multicast group in the multicast request table, there is no need to send the request to the external storage. You only need to update the multicast arrive mask in the table entry. If the multicast arrive mask is the same as the thread_group_mask after the update, it means that all threads of the multicast request group have arrived, and the table entry can be released to wait for the data to be returned.

[0080] If the new multicast request instruction does not find the same group in the multicast request table, it means that the instruction is the first to arrive and the request needs to be sent to external storage. At the same time, an empty table entry is requested in the table, and the thread group id, instruction id and thread_group_mask of the instruction are filled in, and the multicast arrivemask is updated.

[0081] The functions performed by the response multicast module are as follows: Fig.13 As shown, specifically including: When the read data is returned from external storage, the data loading instruction-related information corresponding to the returned data is also stored inside the module, including the required destination register address (the logical address of all threads is the same), threadgroup id and thread_group_mask.

[0082] According to thread_group_mask, the response multicast module can know which threads in the thread group need this data, and the response multicast module writes data to each thread one by one to complete the data multicast response.

[0083] Assuming that the thread group size is 8, that is, there are at most 8 threads in the thread group, then the corresponding thread_group_mask has at most 8 bits, where bit0 corresponds to thread 0, bit1 corresponds to thread 1, bit2 corresponds to thread 2, and so on. Each bit corresponds to one of the threads.

[0084] The functionality of the response multicast module can be expressed through the following code: Multicast_resp_execute( return_data, instr_dst_reg_index, instr_multicast_en, instr_multicast_mask, instr_thread_id, instr_thread_group_id){ }{ If ( ~instr_multicast_en) { Write_register(instr_threadgroup_id, instr_thread_id, instr_dst_reg_index, retrun_data); } Else { For(int i=0; i<8; i++){ If( instr_multicast_mask[i] == 1){ Write_register( instr_thread_group_id, i, instr_dst_reg_index, retrun_data); / / Write the returned data to the threads whose bits in the thread group mask are 1 } } } } The embodiment of the present invention also provides a data multicast device in a multi-core computing cluster, such as Fig.14 , is a schematic diagram of the architecture of a data multicast device in a multi-core computing cluster according to an embodiment of the present invention. The data multicast device in a multi-core computing cluster according to this embodiment includes one or more processors 21 and a memory 22. Fig.14 A processor 21 is taken as an example.

[0085] The processor 21 and the memory 22 may be connected via a bus or other means. Fig.14 The example of connecting through bus is taken in the following.

[0086] The memory 22 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs and non-volatile computer executable programs, such as the data multicast method in the multi-core computing cluster described in the above embodiment. The processor 21 executes the data multicast method in the multi-core computing cluster by running the non-volatile software programs and instructions stored in the memory 22.

[0087] The memory 22 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 22 may optionally include a memory remotely arranged relative to the processor 21, and these remote memories may be connected to the processor 21 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0088] The program instructions / modules are stored in the memory 22, and when executed by the one or more processors 21, execute the first data multicast method within a multi-core computing cluster in the above embodiment, or the second data multicast method within a multi-core computing cluster in the above embodiment.

[0089] This embodiment also provides a data multicast system within a multi-core computing cluster, including a first data multicast device and a second data multicast device; the first data multicast device and the second data multicast device are both the data multicast devices within the multi-core computing cluster described in this embodiment.

[0090] Each of them is used to execute the first data multicast method within a multi-core computing cluster in the above embodiment and the second data multicast method within a multi-core computing cluster in the above embodiment, respectively, to complete the interaction between the first data multicast device and the second data multicast device.

[0091] It is worth noting that the information interaction, execution process, etc. between the modules and units within the above-mentioned devices and systems are based on the same concept as the processing method embodiment of the present invention. The specific contents can be found in the description of the method embodiment of the present invention and will not be repeated here.

[0092] A person skilled in the art may understand that all or part of the steps in the various methods of the embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0093] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A data multicast method in a multi-core computing cluster, characterized in that: include: Receive data loading instructions from each thread; wherein the data loading instructions carry the address of the data, the thread ID and the thread group mask; the data loading instructions corresponding to the multiple first threads have the same thread group mask, and the thread group mask also represents the number of first threads, and the first thread refers to the thread that reuses the first data; Determining whether data loading instructions of all first threads are received according to the received thread group mask and the number of thread group masks with the same thread group mask; After determining that the data loading instructions of all the first threads are received and the first data are written to each first thread, the instructions related to the first data are released.

2. The data multicast method in a multi-core computing cluster according to claim 1, characterized in that: The thread group mask includes N bits, and in the first thread group mask, the bits corresponding to all first threads are 1, and the other bits are 0; The determining, according to the received thread group mask and the number of the same thread group mask, whether the data loading instructions of all the first threads are received comprises: Each time a first data loading instruction is received, a corresponding bit in the first arrival mask is updated to 1 according to the thread ID in the first data loading instruction; wherein, in an initial state, all bits of the first arrival mask are 0; and the first data loading instruction is a data loading instruction carrying a first thread group mask; It is determined whether the first arrival mask is consistent with the first thread group mask. If it is determined that the first arrival mask is consistent with the first thread group mask, it is determined that data loading instructions for all first threads are received.

3. The data multicast method in a multi-core computing cluster according to claim 2, characterized in that: The first arrival mask is established when the first data loading instruction is received for the first time, and includes: When receiving the corresponding first data loading instruction, determining whether the first thread group mask carried in the first data loading instruction exists in the multicast request table; If the first thread group mask does not exist in the multicast request table, the first thread group mask is added to the multicast request table, and a first arrival mask is generated accordingly, where all bits of the first arrival mask are 0; And according to the thread ID in the first data loading instruction, the corresponding bit in the first arrival mask is updated to 1; If the first thread group mask exists in the multicast request table, the first arrival mask is directly updated.

4. The method for multicasting data in a multi-core computing cluster according to any one of claims 1 to 3, characterized in that: Also includes: Deduplication processing is performed on a plurality of first data loading instructions from different threads, so that the first data is read from the external storage once.

5. A data multicast method in a multi-core computing cluster, characterized in that: include: Sending a data loading instruction; wherein the data loading instruction carries the address of the data, the thread ID of the thread to which the data loading instruction belongs, and the thread group mask; wherein the data loading instructions corresponding to the plurality of first threads have the same thread group mask, and the thread group mask also represents the number of the first threads, and the first thread refers to the thread that reuses the first data; This allows the receiving party to determine whether the data loading instructions of all first threads are received based on the received thread group mask and the number of identical thread group masks, and release the instructions related to the first data after determining that the data loading instructions of all first threads are received and the first data are written to each first thread.

6. The method for multicasting data in a multi-core computing cluster according to claim 5, characterized in that: Before sending the data loading instruction, the method further includes: According to the operation to be participated in by the matrix, a plurality of first threads that need to reuse the first data in the matrix are determined, and a same first thread group mask is generated for the plurality of first threads.

7. The method for multicasting data in a multi-core computing cluster according to claim 6, characterized in that: Generating the same first thread group mask for the plurality of first threads comprises: Generate N bits; N is the total number of all threads; Each bit corresponds to a thread, the bit corresponding to the first thread is set to 1, and the bits corresponding to other threads are set to 0, to obtain the first thread group mask.

8. A data multicast device in a multi-core computing cluster, characterized in that: include: at least one processor; And, a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor to execute the data multicast method within a multi-core computing cluster as described in any one of claims 1 to 4, or to execute the data multicast method within a multi-core computing cluster as described in any one of claims 5 to 7.

9. A non-volatile computer storage medium, characterized in that: The computer storage medium stores computer executable instructions, which are executed by one or more processors to implement the data multicast method within a multi-core computing cluster as described in any one of claims 1 to 4 and / or the data multicast method within a multi-core computing cluster as described in any one of claims 5 to 7.

10. A data multicast system in a multi-core computing cluster, characterized in that: It comprises a first data multicast device and a second data multicast device; the first data multicast device and the second data multicast device are both data multicast devices in the multi-core computing cluster according to claim 8; Each of them is used to execute the data multicast method within a multi-core computing cluster described in any one of claims 1 to 4 and the data multicast method within a multi-core computing cluster described in any one of claims 5 to 7, respectively, to complete the interaction between the first data multicast device and the second data multicast device.

Images