An Accelerated Method for Triangular Matrix Multiplication Based on the Atlas 800 Platform

Abstract

The application relates to the field of high-performance computing and computing power optimization, and relates to a method for accelerating triangular matrix multiplication based on an atlas 800 platform, which comprises the following steps: cutting a result matrix into multiple task blocks, and presetting an equal division strategy based on the multiple task blocks; generating a mask matrix on a vector calculation unit, and storing the mask matrix back to a global memory; reading the matrix and the mask matrix from the global memory, implementing triangular masking of the matrix by using the vector calculation unit, and obtaining a triangular matrix A after triangular masking; based on the triangular matrix A after triangular masking, calling a cube calculation unit to complete matrix multiplication operation of all the task blocks according to the preset equal division strategy, and obtaining a result matrix. The mix method combining the vector calculation unit and the cube calculation unit is used to complete the triangular matrix multiplication, the calculation efficiency of the triangular matrix multiplication on the NPU is improved, the computing power of the Huawei atlas 800 NPU can be fully utilized, the triangular matrix multiplication calculation is accelerated, and computing power support is provided for the fields of deep learning and scientific calculation.

Description

Technical Field

[0001] This invention relates to the fields of high-performance computing and computing power optimization, specifically to an acceleration method for triangular matrix multiplication based on the atlas 800 platform. Background Technology

[0002] Triangular matrix multiplication refers to matrix multiplication operations performed when at least one of the two matrices is a triangular matrix. A triangular matrix is ​​a matrix with a specific form in which all elements are zero except for those on or below the diagonal (lower triangular) or above the diagonal (upper triangular). Triangular matrix multiplication has wide applications in scientific computing, engineering applications, and computer graphics. For example, in numerical computation, solving systems of linear equations is a common problem, and triangular matrix multiplication can be used to solve systems of linear equations involving sparse matrices; in signal processing, triangular matrix multiplication can be used to implement various filter operations. Triangular matrix multiplication has a relatively high computational cost, and many optimization techniques have been proposed, such as parallelization using multi-core processors and matrix rearrangement to improve data locality.

[0003] A Neural Processing Unit (NPU) is a processor specifically designed to accelerate computations in artificial neural networks. Unlike general-purpose processors (CPUs) and graphics processing units (GPUs), NPUs are highly optimized for hardware architecture, offering higher computational performance and energy efficiency within the same power consumption and area. NPUs emerged to meet the enormous computational demands of AI (Artificial Intelligence). With the rapid development of artificial intelligence technology, traditional CPUs and GPUs can no longer meet the increasingly complex computational needs, and NPUs are gradually becoming a new computational power option for many AI applications.

[0004] Huawei's Atlas 800 is a self-developed and manufactured NPU, with its computing core being the AI ​​Core, such as... Figure 1 The diagram shows the hardware architecture of the AI ​​Core computing core. The main computing units within the AI ​​Core include: Cube Unit, Vector Unit, and Scalar Unit, used to perform calculations on different types of data. The AI ​​Core has internal storage; it needs to load data from external storage into its internal storage to complete the corresponding calculations. The AI ​​Core's internal storage includes: L1 Buffer, L0 Buffer, Unified Buffer, and Scalar Buffer.

[0005] Currently, Huawei's NPU software ecosystem is not yet fully developed, with various acceleration libraries and numerical libraries under development. Among these, the Basic Linear Algebra Subprograms (BLAS) are a key component. BLAS involves numerous triangular matrix multiplication operations, such as trmm and trmv, making the optimization and acceleration of triangular matrix multiplication crucial. The Atlas 800 NPU does not support direct triangulation of matrices, requiring triangular masking to retain only the valid data. Using scalar operations with for loops to sequentially zero out invalid data in the original matrix incurs significant overhead, severely impacting computational efficiency. Therefore, efficient triangular masking operations, flexible task partitioning strategies, and concise triangular matrix multiplication calculation schemes should be devised to optimize the computational efficiency of triangular matrix multiplication on the NPU. Summary of the Invention

[0006] To address the technical problems existing in the prior art, this invention provides an acceleration method for triangular matrix multiplication based on the Atlas 800 platform. This method reduces the idle time of the computing core, decreases the computational load on the cube computing unit, and improves the computational efficiency of triangular matrix multiplication on the NPU. It can fully utilize the computing power of Huawei Atlas 800 NPU, accelerate triangular matrix multiplication calculation, has strong applicability, and can provide computing power support for fields such as deep learning and scientific computing.

[0007] The objective of this invention can be achieved by adopting the following technical solutions:

[0008] An acceleration method for triangular matrix multiplication based on the Atlas 800 platform includes the following steps:

[0009] S1. Divide the result matrix into multiple task blocks and pre-set a balanced partitioning strategy based on the multiple task blocks;

[0010] S2. Generate a mask matrix in the vector calculation unit and store the mask matrix back into global memory;

[0011] S3. Read the matrix and mask matrix from global memory, use the vector calculation unit to implement matrix triangular masking, and obtain the triangular matrix A after triangular masking;

[0012] S4. Based on the triangular matrix A after triangular masking, the cube computing unit is called according to the preset balanced partitioning strategy to complete the matrix multiplication operation of all task blocks and obtain the result matrix.

[0013] Specifically, step S1 includes:

[0014] S11. Determine the size of a task block based on the memory space of UB, and then divide the result matrix into multiple task blocks according to the size of the task blocks.

[0015] S12. Based on a preset balanced partitioning strategy for multiple task blocks, the preset balanced partitioning strategy includes: sorting and allocating multiple task blocks in descending order of computational load, and cyclically allocating multiple task blocks to each computing core of the atlas 800NPU.

[0016] Specifically, determining the size of a task block based on the memory space of the UB includes:

[0017] Assuming the memory space of UB is K bytes and the task block size is m×m, then the value of m is:

[0018]

[0019] Specifically, the process of sorting and allocating multiple task blocks according to their computational complexity from largest to smallest, and then cyclically distributing these task blocks to the various computing cores of the atlas 800 NPU, includes:

[0020] S121. The task blocks are allocated in descending order of computational complexity. The task blocks are allocated starting from the first atlas 800NPU computing core ID and then sequentially allocated to each computing core of the atlas 800NPU.

[0021] S122. If there are still unallocated task blocks when the last computing core is allocated, the remaining task blocks will be reallocated starting from the first computing core and sequentially allocated to each computing core of the atlas 800NPU.

[0022] S123. Repeat steps S121-S122 until all task blocks have been allocated.

[0023] Specifically, step S3 includes:

[0024] The original matrix and mask matrix are read from global memory. Each time a task block on the diagonal of the original matrix is ​​read, the vector calculation unit is called to multiply each column of the task block that needs to be triangularly masked with the corresponding column of the mask matrix element by element, so as to transform the original matrix into a triangular matrix after triangular masking.

[0025] Specifically, step S4 includes:

[0026] S41. Optimize the matrix multiplication calculation method by setting a judgment function to obtain the optimized matrix multiplication calculation method steps;

[0027] S42. Based on the triangular matrix A after triangular masking, the cube computing unit is called according to the preset balanced partitioning strategy. The matrix multiplication operation of all task blocks is completed through the optimized matrix multiplication calculation method and steps to obtain the result matrix.

[0028] Specifically, the optimized matrix multiplication calculation method includes the following steps:

[0029] For task block (x, y), perform matrix multiplication between the i-th block in row x from left to right in triangular matrix A and the i-th block in column y from top to bottom in general matrix B to obtain intermediate result C(x, y). i , where x and y are the row and column numbers of a task block in the result matrix C, respectively, and

[0030] Set the conditional function as follows:

[0031]

[0032] By performing matrix addition on all intermediate results and summing them together, the final result of this task block is obtained. Find all C(x,y) to obtain the complete result matrix C.

[0033] Specifically, the optimized matrix multiplication calculation method steps for completing matrix multiplication operations for all task blocks include:

[0034] When the task block is below the diagonal, the cube computing unit is invoked to complete the matrix multiplication operation at the task block granularity.

[0035] When a task block is located on the diagonal, the cube computing unit is invoked to perform matrix multiplication operations at the task block granularity.

[0036] When a task block is located in an invalid computation region, i.e., the upper triangular region, it is skipped without computation.

[0037] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0038] This invention provides an accelerated method for triangular matrix multiplication based on the Atlas 800 platform. The method divides the resulting matrix into multiple task blocks, and uses a pre-defined balanced partitioning strategy to allocate tasks in descending order of computational complexity. The execution time of subsequent smaller task blocks masks the execution time of earlier larger task blocks, fully utilizing the parallelism of the NPU to improve computational efficiency. This partitioning strategy maximizes load balancing, improves L2 cache hit rate, and reduces data transfer time. By pre-generating the mask matrix required for subsequent steps using a vector computation unit and storing it back in global memory, no additional operation is needed to generate the mask matrix each time it is used; it can simply be read from global memory. By setting a conditional function to optimize the matrix multiplication calculation method, some invalid data can be avoided, reducing the computational workload by approximately half and improving computational efficiency. This invention employs a mix method combining vector and cube computing units to perform triangular matrix multiplication. It proposes a triangular masking strategy based on vector computing units, which reduces the idle time of the computing cores, lowers the computational load on cube computing units, and improves the computational efficiency of triangular matrix multiplication on the NPU. This fully leverages the computing power of Huawei Atlas 800 NPU, accelerates triangular matrix multiplication, has strong applicability, and can provide computing power support for fields such as deep learning and scientific computing. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0040] Figure 1 This is a schematic diagram of the hardware architecture of the AI ​​Core computing core;

[0041] Figure 2 This is a step diagram of an acceleration method for triangular matrix multiplication based on the Atlas 800 platform in an embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram of lower triangular matrix multiplication in an embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram of the lower triangular matrix A completing the triangular masking operation in an embodiment of the present invention. Detailed Implementation

[0044] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments, and the implementation of the present invention is not limited thereto. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] Example 1:

[0046] Compared with traditional methods, the accelerated triangular matrix multiplication method based on the Atlas 800 platform described in this invention flexibly adopts a mix method that combines vector computing units and cube computing units to complete the triangular matrix multiplication. It also proposes a triangular masking strategy based on vector computing units, which improves the computational efficiency of triangular matrix multiplication on the NPU.

[0047] like Figure 2 As shown, the present invention provides an acceleration method for triangular matrix multiplication based on the Atlas 800 platform, comprising the following steps:

[0048] The proposed method for accelerating triangular matrix multiplication based on the Atlas 800 platform includes the following four steps:

[0049] S1. Divide the result matrix into multiple task blocks and pre-set a balanced partitioning strategy based on the multiple task blocks.

[0050] Since multiplying triangular matrices yields a result matrix, the computation task is divided into multiple task blocks based on the size of the result matrix. That is, each cube computation unit computes several task blocks in the result matrix.

[0051] Specifically, step S1 includes:

[0052] The memory space of S11 and UB determines the size of a task block, and then the result matrix is ​​divided into multiple task blocks according to the size of the task block.

[0053] Specifically, the size of the task block is determined based on the UB memory size. For example... Figure 3 The diagram illustrates lower triangular matrix multiplication. The general process is as follows: the original matrix A is transformed into a triangular matrix A by a triangular masking process. This triangular matrix A is then multiplied by another general matrix B, resulting in the final matrix C. Here, we take a lower triangular matrix A as an example, and the data type is single-precision. The final matrix C is divided into several task blocks for computation, the size of which is determined by the size of the memory UB.

[0054] Due to the special nature of triangular matrix multiplication, the triangular matrix A is a square matrix, and the task block size is also set to a square matrix for ease of subsequent calculations and operations. Therefore, assume the triangular matrix A is M×M, and the task block size is m×m. To perform matrix-matrix multiplication, in this embodiment, the UB memory needs to store two m×m matrices, where each element's data type is 4 bytes of single-precision data. This occupies 2m×m×4 bytes of memory. Assuming the UB memory space is K bytes, then m takes the value... Round down. To match the hardware architecture of Huawei's Atlas 800 NPU, it is generally rounded down to an integer that is a multiple of 32 bytes, i.e.:

[0055] Then, the result matrix C is segmented according to the task block size m×m. Assuming the result matrix size is M×N, the number of computational cores is c, and the computational core IDs are i∈{0,1,2,…,c-1}, then the total number of task blocks is... The computational workload for each computing core is:

[0056]

[0057] Here, tasks(i) represents the number of task blocks allocated to the computing core with id i.

[0058] S12. Based on a preset balanced partitioning strategy for multiple task blocks, the preset balanced partitioning strategy includes: sorting and allocating multiple task blocks in descending order of computational load, and cyclically allocating multiple task blocks to each computing core of the atlas 800NPU.

[0059] Specifically, multiple task blocks are sorted and allocated according to computational complexity from largest to smallest, and then cyclically distributed to the various computing cores of the atlas 800 NPU, including:

[0060] S121. The task blocks are allocated in descending order of computational complexity. The task blocks are allocated starting from the first atlas 800NPU computing core ID and then sequentially allocated to each computing core of the atlas 800NPU.

[0061] S122. If there are still unallocated task blocks when the last computing core is allocated, the remaining task blocks will be reallocated starting from the first computing core and sequentially allocated to each computing core of the atlas 800NPU.

[0062] S123. Repeat steps S121-S122 until all task blocks have been allocated.

[0063] In this embodiment, due to the special nature of the triangular matrix, the computational workload of each task block resulting from the matrix is ​​inconsistent. Therefore, the task blocks are sorted in descending order of computational workload, utilizing the parallelism of the NPU to mask the time taken by the computationally intensive task blocks. These task blocks are then cyclically distributed to each computing core of the Atlas 800 NPU, starting with the task blocks in descending order of computational workload. The task blocks are first distributed according to the computing core ID of the Atlas 800 NPU, starting from the first one, and then sequentially distributed to each computing core of the Atlas 800 NPU. If there are still unallocated task blocks when the last computing core is reached, the remaining task blocks will be reassigned to the first computing core. This cyclical distribution strategy ensures maximum load balancing.

[0064] Based on a pre-defined balanced partitioning strategy for multiple task blocks, task blocks are cyclically allocated to each computing core for parallel computation; that is, task block 0 is assigned to computing core 0, task block 1 is assigned to computing core 1, and so on. If task blocks remain after allocation to the last computing core, the above operation is repeated. However, since only the triangular portion of the triangular matrix is ​​computed, the computational load of each task block is inconsistent. Figure 3 The diagram illustrates lower triangular matrix multiplication. Taking lower triangular matrix A as an example, task blocks 0-4 have a significantly higher computational load than task blocks 15-19. Therefore, this invention allocates task blocks from highest to lowest computational load, using the execution time of subsequent smaller task blocks to mask the execution time of earlier larger task blocks, thus fully utilizing the parallelism of the NPU to improve computational efficiency. This partitioning strategy maximizes load balancing while improving L2 cache hit rate and reducing data transfer time.

[0065] S2. Generate a mask matrix in the vector calculation unit and store the mask matrix back into global memory.

[0066] The mask matrix required for subsequent steps is generated in advance using the vector calculation unit and stored back in global memory. Each time it is used, no additional operation is needed to generate the mask matrix; it can simply be read from global memory.

[0067] In this embodiment, the size of the task block is 160×160, so this step requires generating a 160×160 mask matrix:

[0068]

[0069] Then it is stored back in global memory, so that it does not need to be read repeatedly each time it is used.

[0070] In this embodiment, the data processing of the vector and scalar computing units by the atlas 800NPU can only be operated on the UB memory. Therefore, to generate the mask matrix, it is necessary to first allocate a space of the same size on both the UB memory and the global memory, which are used to generate the mask matrix and store it back, respectively.

[0071] like Figure 4 The diagram illustrates the triangular masking operation performed on the lower triangular matrix A. Taking the lower triangular matrix as an example, if the lower triangular matrix needs to mask the data in the upper triangular matrix while retaining the valid part in the lower triangular matrix, then the mask matrix of the lower triangular matrix is ​​a matrix in which the lower triangular matrix is ​​all 1 and the upper triangular matrix is ​​all 0. By multiplying the lower triangular matrix and the mask matrix element by element, the valid data part of the lower triangular matrix can be separated.

[0072] S3. Read the original matrix and mask matrix from global memory, and use the vector calculation unit to implement matrix triangular masking to obtain the triangular matrix A after triangular masking.

[0073] Specifically, the original matrix and mask matrix are first read from global memory. Each time a task block on the diagonal is read, the vector calculation unit is called to multiply each column of the task block that needs triangular masking by the corresponding column of the mask matrix element-wise, thus achieving triangular masking.

[0074]

[0075] Once all task blocks on the diagonal have been processed, the original matrix is ​​transformed into a triangular matrix A after triangular masking:

[0076]

[0077] like Figure 4 As shown, the task block requiring triangular masking and its corresponding mask matrix are first read from global memory and placed into UB memory. The mask matrix can be read once and used multiple times. Since Atlas 800 uses column-major storage, the vector computation unit is called to multiply each column of the task block element-wise with the corresponding column of the mask matrix, thus separating the effective data part of the lower triangle and achieving triangular masking. The vector computation unit can efficiently calculate the element-wise multiplication between two vectors. In the case of non-transpose, it is necessary to multiply the corresponding elements of each column of the matrix with the vector, and accumulate the results each time. Compared with the traditional method, this method involves an additional mask matrix transfer and uses an additional UB space, but it does not require scalar operations.

[0078] To transform a raw matrix into a triangular matrix A through triangular masking, the traditional method is to read the entire raw matrix into the memory of the byte array (UB), and then use a for loop based on scalar operations on the UB to set invalid triangular data parts to zero, thus achieving triangular masking. However, the scalar computing unit on the Atlas 800 has limited computing power, and a large number of scalar operations will inevitably reduce computational efficiency, failing to fully utilize the computing power of the Atlas 800. Therefore, a mask matrix method without scalar operations can achieve efficient triangular masking.

[0079] S4. Based on the triangular matrix A after triangular masking, the cube computing unit is called according to the preset balanced partitioning strategy to complete the matrix multiplication operation of all task blocks and obtain the result matrix.

[0080] Having obtained the triangular matrix A after triangular masking in step S3, the cube computing unit is now invoked to perform matrix multiplication according to the preset balanced partitioning strategy. Similar to general matrix multiplication, a task block of the result matrix C needs to perform matrix multiplication on a row of the task block of the triangular matrix A after triangular masking and a column of the task block of the general matrix B in turn, and then sum the results together. This completes the calculation of one task block of the result matrix C.

[0081] The traditional method for calculating matrix multiplication is:

[0082] For task block (x, y), perform matrix multiplication between the i-th block in row x from left to right in triangular matrix A and the i-th block in column y from top to bottom in general matrix B to obtain intermediate result C(x, y). i , where x and y are the row and column numbers of a task block in the result matrix C, respectively, and

[0083] Then, all the intermediate results are summed together using matrix addition to obtain the final result of this task block. Finding all C(x,y) will yield the complete result matrix C.

[0084] However, for triangular matrix multiplication, it's important to note that some data is invalid and can be omitted from calculation. Therefore, based on the spatial arrangement of the matrix, the steps of the traditional matrix multiplication calculation method described above are optimized and improved. A judgment function is set: i∈[0,x], then the final result of a certain task block is... The part i > x is in Figure 3 The area represented by the dashed lines in triangular matrix A does not require calculation. By setting a conditional function, invalid data can be excluded from calculation, reducing the amount of computation by approximately half and improving computational efficiency.

[0085] S41. By setting a judgment function to optimize the matrix multiplication calculation method, the optimized matrix multiplication calculation method steps are obtained:

[0086] For task block (x, y), perform matrix multiplication between the u-th block in row x from left to right in triangular matrix A and the i-th block in column y from top to bottom in general matrix B to obtain intermediate result C(x, y). i , where x and y are the row and column numbers of a task block in the result matrix C, respectively, and

[0087] Set the conditional function as follows:

[0088]

[0089] By performing matrix addition on all intermediate results and summing them together, the final result of this task block is obtained. Finding all C(x,y) will yield the complete result matrix C.

[0090] The following is an example of a triangular matrix, such as Figure 3 As shown, the triangular matrix completes the calculation of task blocks sequentially from bottom to top and from left to right. Utilizing the characteristic that the ID numbers of the computational cores range from 0 to 31, a preset balanced partitioning strategy can be applied. The task block number to be calculated in a given instance is... Then the task block is assigned to the one with the ID number. The core of the calculation is mod, where mod represents the remainder operation.

[0091] S42. Based on the triangular matrix A after triangular masking, the cube computing unit is called according to the preset balanced partitioning strategy. The matrix multiplication operation of all task blocks is completed through the optimized matrix multiplication calculation method and steps to obtain the result matrix.

[0092] In this embodiment, the original data of the i-th block (row x, from left to right) of triangular matrix A and the i-th block (column y, from top to bottom) of general matrix B are read from the hard disk and moved to global memory. Then, the data is moved from global memory to a unified buffer UB. The data is then used to place the two matrices into L0A and L0B buffers respectively via MTE and L1 buffers. Next, the data is moved to the Cube computing unit for matrix multiplication. Finally, the calculation result C(x,y) is stored in the L0C buffer. i Store back into the unified buffer UB, where Repeat the above steps Next, get An intermediate result C(x,y) i Then the final result of a certain task block is

[0093] According to the balanced partitioning strategy preset in step S1, each computing core is evenly divided into task blocks. The matrix multiplication operation of all task blocks is completed through an optimized matrix multiplication calculation method, which includes three cases as follows:

[0094] (1) When the task block is below the diagonal, the cube computing unit is called to complete the matrix multiplication operation at the task block granularity:

[0095]

[0096] (2) When the task block is located on the diagonal, the cube computing unit is called to complete the matrix multiplication operation at the task block granularity:

[0097]

[0098] (3) When the task block is located in the invalid calculation region, i.e., the upper triangular region, no calculation is needed, and it is skipped directly:

[0099]

[0100] Similar to general matrix multiplication, a task block of the result matrix C requires performing matrix multiplication on corresponding rows of the triangular matrix A and columns of the general matrix B, and then summing the results together. This completes the calculation of one task block of the result matrix C. When the calculation of each task block of the result matrix C is completed, the entire result matrix C is calculated.

[0101] In this embodiment, the atlas 800NPU has multiple AI Cores, typically 30 or 32 depending on the model. To maximize computational resource utilization, each AI Core needs to be kept running as much as possible during program execution, minimizing idle time. Therefore, computational tasks should be evenly distributed across each AI Core to distribute the computational load as evenly as possible, thereby fully utilizing all computational resources and improving computational parallelism.

[0102] Assuming the data type is single precision (a single single-precision data point occupies 4 bytes), let's take lower triangular matrix multiplication as an example (the same applies below, and the process is similar when handling upper triangular matrix multiplication). From the overall process, triangular matrix multiplication performs the following calculation: After processing, the original matrix A is transformed from a 1024×1024 square matrix into a 1024×1024 lower triangular matrix. Matrix B is a 1024×2048 general matrix, and the resulting matrix C is 1024×2048 in size. Finally, the triangular matrix multiplication AB = C is completed.

[0103]

[0104] In this embodiment, the atlas 800 platform with 32 computing cores is selected, and its UB memory space is 256KB, or 256×1024B. Based on the formula obtained above... The calculation shows that m is 160, so the task block size is 160×160. Given that the result matrix size is M×N=1024×2048, and the number of cores c=32, the total number of task blocks is... The task blocks are numbered starting from 0, i.e., from task block 0 to task block 90. ​​Due to the special nature of the triangular matrix, this embodiment assigns task blocks in descending order of computational complexity. According to the cyclic task partitioning strategy, cores with IDs 0 to 26 will be allocated 3 task blocks, and cores with IDs 27 to 31 will be allocated 2 task blocks. For example, core with ID 0 is allocated task blocks 0, 32, and 64; core with ID 26 is allocated task blocks 26, 58, and 90; core with ID 27 is allocated task blocks 27 and 59; and core with ID 31 is allocated task blocks 31 and 63.

[0105] On the Atlas 800 NPU, without using scalar computing units, it's impossible to perform triangular masking and zeroing on data in a contiguous memory space. However, using scalar units with a for loop for zeroing would impact overall computational efficiency. This invention utilizes vector computing units to generate a mask matrix, which is then multiplied element-wise with the original matrix to complete the triangular masking, retaining only valid data. Furthermore, based on the characteristics of triangular matrices, this solution designs an efficient task partitioning strategy and computational judgment criteria, employing a mix method combining vector and cube computing units to reduce idle time in the computing cores and lower the computational load on the cube computing units.

[0106] In summary, this invention provides an accelerated method for triangular matrix multiplication based on the Atlas 800 platform. By dividing the result matrix into multiple task blocks and using a pre-defined balanced partitioning strategy, tasks are allocated from highest to lowest computational load. The execution time of subsequent smaller task blocks masks the execution time of earlier larger task blocks, fully utilizing the parallelism of the NPU to improve computational efficiency. This partitioning strategy maximizes load balancing, improves L2 cache hit rate, and reduces data transfer time. By pre-generating the mask matrix required for subsequent steps using a vector computation unit and storing it back in global memory, no additional operation is needed to generate the mask matrix each time; it can simply be read from global memory. By optimizing the matrix multiplication calculation method with a judgment function, some invalid data can be avoided, reducing the computational load by approximately half and improving computational efficiency. This invention employs a mix method combining vector and cube computing units to perform triangular matrix multiplication. It proposes a triangular masking strategy based on vector computing units, which reduces the idle time of the computing cores, lowers the computational load on cube computing units, and improves the computational efficiency of triangular matrix multiplication on the NPU. This fully leverages the computing power of Huawei Atlas 800 NPU, accelerates triangular matrix multiplication, has strong applicability, and can provide computing power support for fields such as deep learning and scientific computing.

[0107] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for accelerating triangular matrix multiplication based on the Atlas 800 platform, characterized in that, Includes the following steps: S1. Divide the result matrix into multiple task blocks and pre-set a balanced partitioning strategy based on the multiple task blocks; S2. Generate a mask matrix in the vector computation unit and store the mask matrix back into global memory; S3. Read the original matrix and mask matrix from global memory, use a vector computing unit to implement matrix triangular masking, and obtain the triangular matrix A after triangular masking; S4. Based on the triangular matrix A after triangular masking, the cube computing unit is called according to the preset balanced partitioning strategy to complete the matrix multiplication operation of all task blocks and obtain the result matrix. Step S4 includes: S41. Optimize the matrix multiplication calculation method by setting a judgment function to obtain the optimized matrix multiplication calculation method steps; S42. Based on the triangular matrix A after triangular masking, the cube computing unit is called according to the preset balanced partitioning strategy. The matrix multiplication operation of all task blocks is completed through the optimized matrix multiplication calculation method and steps to obtain the result matrix. The optimized matrix multiplication calculation method includes the following steps: For task blocks ( x, y In triangular matrix A x The first row from left to right i In the blocks and the universal matrix B y The first in the column from top to bottom i Perform matrix multiplication on each block to obtain intermediate results. ,in x, y Let represent the row and column numbers of a task block in the result matrix C, respectively, and let the task block size be . m × m The resulting matrix C has a size of M × N Set the conditional function as follows: ； By performing matrix addition on all intermediate results and summing them together, the final result of this task block is obtained. Find all This yields the complete result matrix C.

2. The method for accelerating triangular matrix multiplication based on the Atlas 800 platform according to claim 1, characterized in that, Step S1 includes: S11. Determine the size of a task block based on the memory space of the unified buffer UB, and then divide the result matrix into multiple task blocks according to the size of the task blocks. S12. Based on a preset balanced partitioning strategy for multiple task blocks, the preset balanced partitioning strategy includes: sorting and allocating multiple task blocks in descending order of computational load, and cyclically allocating multiple task blocks to each computing core of the atlas 800 NPU.

3. The method for accelerating triangular matrix multiplication based on the Atlas 800 platform according to claim 2, characterized in that, The step of determining the size of a task block based on the memory space of the unified buffer UB includes: Assume the memory space of the unified buffer UB is K bytes, and the task block size is... m × m ,but m The value can be: 。 4. The method for accelerating triangular matrix multiplication based on the Atlas 800 platform according to claim 2, characterized in that, The process of sorting and allocating multiple task blocks according to their computational complexity from largest to smallest, and then cyclically distributing these task blocks to the various computing cores of the Atlas 800 NPU, includes: S121. Distribute the task blocks in descending order of computational complexity. Distribute the task blocks in descending order of computational complexity according to the computational core ID of the Atlas 800 NPU, starting from the first one, and then distribute them to each computational core of the Atlas 800 NPU in sequence. S122. If there are still unallocated task blocks when the last computing core is allocated, the remaining task blocks will be reallocated starting from the first computing core and sequentially allocated to each computing core of the atlas 800 NPU. S123. Repeat steps S121-S122 until all task blocks have been allocated.

5. The method for accelerating triangular matrix multiplication based on the Atlas 800 platform according to claim 4, characterized in that, Step S3 includes: The original matrix and mask matrix are read from global memory. Each time a task block on the diagonal of the original matrix is ​​read, the vector calculation unit is called to multiply each column of the task block that needs to be triangularly masked with the corresponding column of the mask matrix element by element, so as to transform the original matrix into a triangular matrix after triangular masking.

6. The method for accelerating triangular matrix multiplication based on the Atlas 800 platform according to claim 1, characterized in that, The optimized matrix multiplication calculation method completes the matrix multiplication operations for all task blocks, including: When the task block is below the diagonal, the cube computing unit is invoked to perform matrix multiplication operations at the task block granularity. When a task block is located on the diagonal, the cube computing unit is invoked to perform matrix multiplication operations at the task block granularity. When a task block is located in an invalid computation region, i.e., the upper triangular region, it is skipped without computation.

Images