Acceleration method and acceleration device for long and narrow matrix multiplication

Abstract

The invention provides an acceleration method and an acceleration device for long and narrow matrix multiplication. The method comprises the following steps: carrying out a padding operation on a long and narrow matrix in a unified buffer area of an operation platform to obtain a padding operation result, and accelerating by using double buffers in the process of carrying out the padding operation; averagely distributing a padding operation result as a calculation task to a matrix calculation unit in a calculation core of the calculation platform, carrying out matrix multiplication operation in the matrix calculation unit to obtain a matrix multiplication calculation result, and marking task blocks read into a first preset buffer area in the calculation core by using a double-buffer speed in the calculation process; scalar multiplication and matrix addition are carried out on a matrix multiplication calculation result in a vector calculation unit in the calculation core, meanwhile, double buffering is used for acceleration, and a calculation result of a long and narrow matrix is obtained. The method solves the problem of low calculation efficiency of long and narrow matrix multiplication in the prior art.

Description

Technical Field

[0001] The present application relates to the field of high performance computing, and in particular, to an acceleration method, an acceleration device, a computer-readable storage medium and an electronic device for narrow and long matrix multiplication. Background Art

[0002] Long and narrow matrices, i.e. matrices whose width is much greater than their height or whose height is much greater than their width, are common in many scientific computing, image processing, and machine learning applications. Long and narrow matrix multiplication calculations are extremely critical in modern science and technology. They become a bridge between theory and practical applications by accelerating computationally intensive tasks, optimizing resource utilization, promoting algorithm and model innovation, enhancing data processing capabilities, supporting high-performance computing, and improving numerical stability and accuracy. Especially in the fields of big data, machine learning, and high-performance computing, it has greatly improved processing speed, reduced costs, and promoted technological progress, showing a wide range of influence in efficiency, cost-effectiveness, and technological innovation. Therefore, it is necessary to efficiently process these matrix multiplications and design optimization techniques and algorithms based on the structural characteristics of long and narrow matrices.

[0003] In the prior art, the characteristics of the narrow matrix and the hardware structure of the NPU are not fully utilized, thus limiting the improvement of the computing speed. Summary of the invention

[0004] The main purpose of the present application is to provide an acceleration method, an acceleration device, a computer-readable storage medium and an electronic device for narrow and long matrix multiplication, so as to at least solve the problem of low computational efficiency of narrow and long matrix multiplication in the prior art.

[0005] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for accelerating narrow matrix multiplication is provided, comprising: a data processing step, performing a padding operation on a narrow matrix in a unified buffer of a computing platform to obtain a padding operation result, wherein double buffering is used for acceleration during the padding operation, and the padding operation includes modifying the arrangement of data and modifying the spacing between two columns at the same time; a first calculation step, distributing the padding operation result as a calculation task evenly to matrix calculation units in a computing core of the computing platform, and performing matrix multiplication operations in the matrix calculation units to obtain matrix multiplication calculation results, and double buffering is used for speed during the calculation process, and the task blocks read into the first preset buffer in the computing core are marked; a second calculation step, performing scalar multiplication operations and matrix addition operations on the matrix multiplication calculation results in a vector calculation unit in the computing core, and using double buffering for acceleration at the same time to obtain the calculation result of the narrow matrix.

[0006] Optionally, the data processing step includes: determining a padding method according to the dimension of the narrow matrix and the size of the unified buffer, the padding method being a single-column single padding operation or a multi-column single padding operation; assigning a padding calculation task to a plurality of the computing cores, and any of the computing cores calculating a position of a target task block in a first preset narrow matrix and a second preset narrow matrix according to the assigned padding calculation task, the block size and the padding method, the position indicating a position of the target task block in a global memory; reading the target task block from the global memory and writing the target task block into the unified buffer; writing the target task block back to the global memory to change the arrangement of the task block data.

[0007] Optionally, the narrow matrix multiplication is expressed as , where α and β are multiplication factors, and the narrow matrix multiplication is realized by matrix multiplication operation , scalar multiplication operation and matrix addition operation, the first calculation step implements the matrix multiplication operation , the process of executing the first calculation step includes: evenly distributing the calculation task to the matrix calculation unit in the calculation core of the computing platform, defining the dimension of matrix A as (M, K), the dimension of matrix B as (K, N), the dimension of matrix C as (M, N), the dimension of temporary matrix T as (M, N), the size of the block of the calculation task as (M0, N0), and obtaining the overall number of blocks , and the task block dimension of the matrix A is (M0, K), and the task block dimension of the matrix B is (K, N0); block division is performed in the K direction, and the task block dimensions of the matrix A and the matrix B are respectively transformed into (M0, K0) and (K0, N0), wherein the number of calculations required in the K direction is , wherein K0 is a block size determined according to the capacity of the first preset buffer.

[0008] Optionally, after the dimensions of the task blocks of the matrix A and the matrix B are respectively converted to (M0, K0) and (K0, N0), the first calculation step also includes: the computing core calculates the position of the corresponding task block in the matrix A and the matrix B according to the assigned computing task sequence number and the blocking in the K direction, and obtains the position of the task block in the global memory according to the position of the matrix A and the matrix B in the global memory; reads the task blocks of the matrix A and the task blocks of the matrix B from the global memory to the first preset buffer; performs a second blocking on the task blocks of the matrix A and the task blocks of the matrix B in the first preset buffer in the K direction, and converts the dimensions of the task blocks of the matrix A and the task blocks of the matrix B to (M0, K1) and (K1, N0) respectively, and the number of blocks is ; The block results of the task block of the matrix A and the task block of the matrix B are transferred from the first preset buffer to the second preset buffer and the third preset buffer in turn; the mad function is executed in the second preset buffer and the third preset buffer, the calculation results are accumulated in the fourth preset buffer, and the results in the fourth preset buffer are written back to the global memory.

[0009] Optionally, the task blocks read into the first preset buffer in the computing core are marked, including: marking each of the task blocks after segmentation with a serial number; after marking each of the task blocks after segmentation with a serial number, the method further includes: when executing the next computing task, if the serial number of the next task block is the same as the serial number of the task block that has been read, directly using the data of the task block that has been read for calculation.

[0010] Optionally, the narrow matrix multiplication is expressed as , where α and β are multiplication factors, and the narrow matrix multiplication is realized by matrix multiplication operation , scalar multiplication operation And matrix addition operation, the block dimensions of matrix T and matrix C are (M0, N0), and the number of blocks is , the second calculation step implements the scalar multiplication operation and the matrix addition operation, including: the computing core calculates the position of the target task block in the matrix C according to the sequence number of the assigned target task block, determines the position of the target task block in the global memory according to the position of the matrix C in the global memory, and reads the task block of the matrix C from the global memory to the unified buffer.

[0011] Optionally, after reading the task block of the matrix C from the global memory to the unified buffer, the second calculation step further includes: performing scalar multiplication of The calculation result of the scalar multiplication of is transferred to the position of the task block of the matrix C in the uniform buffer; the task block of the matrix T is read from the global memory to the uniform buffer; and the vector calculation unit in the calculation core is performed scalar multiplication of The result of the scalar multiplication of is stored in the position of the task block of the matrix T in the uniform buffer; the vector calculation unit in the calculation core is performed Matrix addition, the The calculation result of the matrix addition is stored in the position of the task block of the matrix C in the unified buffer; and the result of the task block of the matrix C in the unified buffer is written back to the global memory.

[0012] According to another aspect of the present application, there is provided an acceleration device for narrow matrix multiplication, comprising:

[0013] A data processing unit, used for performing a padding operation on the narrow matrix in a unified buffer of the computing platform to obtain a padding operation result, wherein a double buffer is used for acceleration during the padding operation, and the padding operation includes modifying the arrangement of data and modifying the spacing between two columns at the same time;

[0014] a first computing unit, configured to distribute the padding operation result as a computing task evenly to the matrix computing units in the computing core of the computing platform, and perform matrix multiplication in the matrix computing units to obtain matrix multiplication calculation results, and use double buffering to speed up the calculation process, and mark the task blocks read into the first preset buffer in the computing core;

[0015] The second computing unit is used to perform scalar multiplication and matrix addition operations on the matrix multiplication calculation result in the vector computing unit in the computing core, and use double buffering for acceleration to obtain the calculation result of the narrow matrix.

[0016] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute any one of the acceleration methods for narrow matrix multiplication.

[0017] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include an acceleration method for executing any one of the narrow matrix multiplications.

[0018] Applying the technical solution of the present application, first, padding operation is performed on the narrow matrix in the unified buffer of the computing platform to obtain the padding operation result, wherein double buffering is used for acceleration during the padding operation, and the padding operation includes modifying the arrangement of data and modifying the spacing between two columns at the same time; then, the padding operation result is evenly distributed as a computing task to the matrix computing unit in the computing core of the computing platform, and matrix multiplication operation is performed in the matrix computing unit to obtain the matrix multiplication calculation result, and double buffering is used for acceleration during the calculation process, and the task block read into the first preset buffer in the computing core is marked; finally, scalar multiplication operation and matrix addition operation are performed on the matrix multiplication calculation result in the vector computing unit in the computing core, and double buffering is used for acceleration at the same time to obtain the calculation result of the narrow matrix. In this solution, after padding operation is performed on the matrix, narrow matrix multiplication operation is performed in the matrix computing unit, and the vector computing unit is used to complete the scalar multiplication and matrix addition operation in parallel, and double buffering is used for acceleration at the same time, thereby solving the problem of low calculation efficiency of narrow matrix multiplication. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings constituting part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 A schematic flow chart of a method for accelerating narrow matrix multiplication provided according to an embodiment of the present application is shown;

[0021] Figure 2 A schematic diagram of block calculation in the K direction of a method for accelerating narrow matrix multiplication provided in an embodiment of the present application is shown;

[0022] Figure 3 A schematic diagram of task block reading marks of a method for accelerating narrow matrix multiplication provided according to an embodiment of the present application is shown;

[0023] Figure 4 A flow chart of parallel computing of an acceleration method for narrow matrix multiplication provided according to an embodiment of the present application is shown;

[0024] Figure 5The overall flow chart of a method for accelerating narrow matrix multiplication provided according to an embodiment of the present application is shown;

[0025] Figure 6 A structural block diagram of an acceleration device for narrow and long matrix multiplication provided according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0026] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0027] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] As introduced in the background technology, the prior art fails to fully utilize the characteristics of long and narrow matrices and the hardware structure of the NPU, thereby limiting the improvement of the computing speed. In order to solve the problem of low computational efficiency of long and narrow matrix multiplication, the embodiments of the present application provide an acceleration method, acceleration device, computer-readable storage medium and electronic device for long and narrow matrix multiplication.

[0030] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0031] Figure 1 FIG. 1 is a flow chart of a method for accelerating narrow matrix multiplication according to an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0032] Step S101, a data processing step, performing a padding operation on the narrow matrix in a unified buffer of a computing platform to obtain a padding operation result, wherein double buffering is used for acceleration during the padding operation, and the padding operation includes modifying the arrangement of data and modifying the spacing between two columns;

[0033] Specifically, in the data processing step, padding operation is performed on the narrow matrix to add a certain number of borders around the matrix for subsequent operations or processing. In this process, the original data is rearranged and the spacing between two columns is modified. In order to speed up the padding operation, double buffering technology is used. Double buffering is a technology that uses two buffers to perform read and write operations alternately, which can improve the efficiency and speed of data processing. In this process, one buffer is used to read the original data, and the other buffer is used to store the result of the padding operation. When the data in one buffer is processed, it can be immediately switched to another buffer for processing, thereby realizing parallel processing and improving the processing speed. The final padding operation result is that the original matrix is ​​modified, the arrangement of the data and the spacing between the two columns are changed. This operation can make the matrix more convenient and efficient in subsequent processing, and can also adapt to different types of processing requirements. Through the application of double buffering technology, this process can be accelerated and the efficiency and speed of data processing can be improved.

[0034] Step S102, a first calculation step, distributing the padding operation results as calculation tasks evenly to the matrix calculation units in the calculation core of the calculation platform, and performing matrix multiplication operations in the matrix calculation units to obtain matrix multiplication calculation results, and using double buffering to speed up the calculation process, and marking the task blocks read into the first preset buffer in the calculation core;

[0035] Specifically, in the first calculation step, matrix multiplication calculation is performed. First, the matrix data after padding operation is evenly distributed to the matrix calculation units in the calculation core of the calculation platform, and then the matrix multiplication operation is performed in the matrix calculation unit of each calculation core to obtain the matrix multiplication calculation result. In the calculation process, the double buffering technology can be used to improve the calculation speed, and the calculation waiting time can be reduced by alternating the two buffers. At the same time, the task blocks read into the first preset buffer in the calculation core are marked so that these data blocks can be correctly processed and tracked in the subsequent calculation steps to ensure the correctness and efficiency of the calculation. In this way, the computing resources can be effectively utilized and the computing efficiency and performance can be improved.

[0036] Step S103, the second calculation step, performs scalar multiplication and matrix addition operations on the above matrix multiplication calculation results in the vector calculation unit in the above calculation core, and uses double buffering for acceleration to obtain the calculation results of the above narrow matrix.

[0037] Specifically, scalar multiplication and matrix addition are performed on the calculation result of the above-mentioned matrix multiplication. Scalar multiplication is to multiply each element in the matrix by a scalar value, and matrix addition is to add the corresponding elements of two matrices to obtain a new matrix. In the vector calculation unit, double buffering can be used for acceleration. Double buffering refers to the use of two buffers during the calculation process, one for storing the result of the current calculation, and the other for storing the data required for the next calculation. By alternating the two buffers, data transmission and waiting time can be reduced, thereby improving calculation efficiency. By performing scalar multiplication and matrix addition operations in the vector calculation unit and using double buffering for acceleration, the final calculation result of the above-mentioned narrow matrix can be obtained. This method can effectively improve calculation efficiency and performance, making the matrix calculation process more efficient and faster.

[0038] In the above embodiment, first, a padding operation is performed on a narrow matrix in a unified buffer of a computing platform to obtain a padding operation result, wherein a double buffer is used for acceleration during the padding operation, and the padding operation includes modifying the arrangement of data and modifying the spacing between two columns; then, the padding operation result is evenly distributed as a computing task to the matrix computing unit in the computing core of the computing platform, and a matrix multiplication operation is performed in the matrix computing unit to obtain a matrix multiplication calculation result, and a double buffer is used for acceleration during the calculation process, and the task block read into the first preset buffer in the computing core is marked; finally, a scalar multiplication operation and a matrix addition operation are performed on the matrix multiplication calculation result in the vector computing unit in the computing core, and a double buffer is used for acceleration to obtain the calculation result of the narrow matrix. In this scheme, after the padding operation is performed on the matrix, a narrow matrix multiplication operation is performed in the matrix computing unit, and the scalar multiplication and matrix addition operations are completed in parallel by the vector computing unit, and a double buffer is used for acceleration, thereby solving the problem of low computational efficiency of narrow matrix multiplication.

[0039] In one embodiment of the present application, the data processing step includes: determining a padding method according to the dimension of the narrow matrix and the size of the unified buffer, the padding method being a single-column single padding operation or a multi-column single padding operation; assigning padding calculation tasks to the plurality of computing cores, and any of the computing cores calculating the position of the target task block in the first preset narrow matrix and the second preset narrow matrix according to the assigned padding calculation tasks, the block size and the padding method, the position indicating the position of the target task block in the global memory; reading the target task block from the global memory and writing the target task block into the unified buffer; writing the target task block back to the global memory to change the arrangement of the task block data.

[0040] Specifically, considering the architecture of the computing platform, when reading data, the reading speed will be the fastest when the interval between different columns of the matrix is ​​512B or its integer multiple. Therefore, a padding operation is designed before matrix multiplication. On the one hand, padding modifies the spacing between different data columns of the matrix; on the other hand, the storage mode of the data in the calculation block of the task block in the global memory is modified to continuous storage, and continuous reading is achieved by reading in blocks in the subsequent calculation process, thereby speeding up the reading speed of the data block. When padding, the multi-core characteristics of the NPU are fully utilized, and the block calculation in the padding process is divided into blocks and assigned to different cores for calculation. The size of the block is selected according to the size of the storage of the unified buffer UB. Since the entire operation process uses double buffering, the space of the unified buffer UB needs to be divided into two parts, which are used for padding adjustment. For matrices whose M direction is larger than half of the space of the unified buffer UB, the single column of the matrix is ​​divided into blocks when dividing the blocks. If it is a matrix with a smaller M direction, multiple columns of the matrix are used as a calculation block when dividing the blocks. The overall padding is to read the data into the unified buffer UB, and then write it back to the global memory through the unified buffer UB, but when writing it back, the data arrangement is modified, and the spacing between the two columns is modified. The padding calculation task is distributed to multiple computing cores, making full use of the multi-core computing power of the NPU.

[0041] In one embodiment of the present application, the above narrow matrix multiplication is expressed as , where α and β are multiplication factors. The above narrow matrix multiplication is realized by matrix multiplication operation , scalar multiplication operation and matrix addition operation, the first calculation step implements the above matrix multiplication operation , the process of executing the first calculation step includes: evenly distributing the calculation tasks to the matrix calculation units in the calculation cores of the computing platform, defining the dimensions of matrix A as (M, K), the dimensions of matrix B as (K, N), the dimensions of matrix C as (M, N), the dimensions of the temporary matrix T as (M, N), the size of the blocks of the calculation tasks as (M0, N0), and obtaining the overall number of blocks , and the task block dimension of the matrix A is (M0, K), and the task block dimension of the matrix B is (K, N0); divide the task blocks in the K direction, and transform the task block dimensions of the matrix A and the matrix B into (M0, K0) and (K0, N0) respectively, where the number of calculations required in the K direction is , wherein K0 is the block size determined according to the capacity of the first preset buffer.

[0042] Specifically, the main calculation process is divided into two parts: The matrix multiplication and Scalar multiplication and matrix addition. This part performs the matrix multiplication operation of the first part, which is the key part of the overall calculation. This part multiplies the padded matrices A and B, temporarily stores the calculation results in T, and writes them back to the global memory. For the A×B calculation process, the task needs to be divided into blocks first. Here, it is assumed that the dimensions of matrix A are (M, K), the dimensions of matrix B are (K, N), and the dimensions of matrix C are (M, N). The dimensions of the temporary matrix T obtained by calculation are (M, N), and the size of the calculation block is (M0, N0). The total number of blocks is , distribute the computing tasks to multiple computing cores, and make full use of the computing power of the multi-core of NPU. The above-mentioned block division is based on the result matrix. At this time, the task block dimension of matrix A is (M0, K), and the task block dimension of matrix B is (K, N0). Block division is performed in the K direction. The task block dimensions of matrix A and matrix B are transformed into (M0, K0) and (K0, N0) respectively. The number of calculations required in the K direction is .

[0043] In one embodiment of the present application, after the dimensions of the task blocks of the matrix A and the matrix B are respectively converted to (M0, K0) and (K0, N0), the first calculation step further includes: the computing core calculates the position of the corresponding task block in the matrix A and the matrix B according to the assigned computing task sequence number and the blocking in the K direction, and obtains the position of the task block in the global memory according to the position of the matrix A and the matrix B in the global memory; reads the task blocks of the matrix A and the matrix B from the global memory to the first preset buffer; performs a second blocking in the K direction on the task blocks of the matrix A and the matrix B in the first preset buffer, and converts the dimensions of the task blocks of the matrix A and the matrix B to (M0, K1) and (K1, N0), respectively, and the number of blocks is ; The block results of the task block of the above-mentioned matrix A and the task block of the above-mentioned matrix B are transferred from the above-mentioned first preset buffer to the second preset buffer and the third preset buffer in turn; the mad function is executed in the above-mentioned second preset buffer and the above-mentioned third preset buffer, the calculation results are accumulated in the fourth preset buffer, and the results in the above-mentioned fourth preset buffer are written back to the above-mentioned global memory.

[0044] Specifically, the computing core determines the corresponding task block positions in matrices A and B according to the assigned computing task sequence number and the block size in the K direction. This process depends on the initial layout of matrices A and B in the global memory, so as to accurately locate the specific positions of the task blocks in the global memory. Subsequently, these task blocks are read from the global memory into the first preset buffer, ready for the next step of refinement and calculation. In the first preset buffer, the task blocks of matrices A and B are further refined and divided in the K direction for the second time, and the dimensions are adjusted from (M0, K0) and (K0, N0) to (M0, K1) and (K1, N0). This block strategy can further decompose large tasks and facilitate parallel computing.

[0045] The matrix A and B task blocks that have been divided for the second time in the first preset buffer are transferred to the second preset buffer and the third preset buffer in turn, and the mad function is used to perform matrix multiplication calculations, and the results are accumulated in the fourth preset buffer. In this process, the mad function efficiently performs the core operation of matrix multiplication, multiplies and accumulates the elements of the two matrices, and generates an intermediate result of the multiplication. After completing the accumulation calculation in the fourth preset buffer, the final result in the fourth preset buffer is written back to the specified storage location in the global memory to complete the calculation of the specific task block. Since the entire calculation process makes full use of the double buffer mechanism, that is, the reading and writing of data in the first preset buffer, the second preset buffer, the third preset buffer, and the fourth preset buffer are all performed in parallel, this helps to speed up data processing, reduce waiting time, and thus improve calculation efficiency.

[0046] Specifically, the first block division in the K direction is because the previous calculation task is divided according to the result matrix, and the block size of the result matrix is ​​(M0, N0). However, during the calculation, the dimension in the K direction may be much larger than the first preset buffer memory. Since the size of the first preset buffer memory is limited, it is necessary to divide the blocks in the K dimension, such as Figure 2 As shown in the figure, a schematic diagram of block calculation in the K direction is shown, with matrix A 2,1 and B 1,2 For example, the large matrix is ​​divided into small blocks for calculation, and the resulting matrix T 2,2 The calculation formula is The second block division in the k direction is because the memory of the first preset buffer is several times that of the second preset buffer, the third preset buffer, and the fourth preset buffer. Therefore, further block division is required in the K dimension. 2,1 , and B 1,2 Further block division is performed, and the calculation formula is , and finally obtain the calculation of the final task block through two layers of loop nesting.

[0047] In one embodiment of the present application, the task blocks read into the first preset buffer in the above-mentioned computing core are marked, including: marking the serial numbers of the above-mentioned task blocks after the block division; after marking the serial numbers of the above-mentioned task blocks after the block division, the above-mentioned method also includes: when executing the next computing task, if the serial number of the next task block is the same as the serial number of the task block that has been read, directly use the data of the above-mentioned task block that has been read for calculation.

[0048] Specifically, the task block reading mark diagram is as follows Figure 3As shown in , when one dimension of M or K is larger and the other dimension is smaller, taking matrix A as an example, in the reading of the first and second task blocks, the blocks of matrix A read are the same. At this time, if the calculation method is the same as before, the repeated task block content will be read from the global memory to overwrite the data. For this, the task blocks after the blocks are marked with serial numbers. When reading in the next time, if the task block serial number is the same as before, the reading from the global memory this time will be skipped and the previously read data will be used directly. This method will improve the efficiency of matrix multiplication by reducing the overall number of memory accesses, such as Figure 3 As shown, , , at this time, the calculation of both tasks requires A 1,1 , and the order of the two operations is connected, so the second calculation does not require A 1,1 Read in.

[0049] In one embodiment of the present application, the above narrow matrix multiplication is expressed as , where α and β are multiplication factors. The above narrow matrix multiplication is realized by matrix multiplication operation , scalar multiplication operation And matrix addition operation, the block dimensions of matrix T and matrix C are (M0, N0), and the number of blocks is , the second calculation step implements the scalar multiplication operation and the matrix addition operation, including: the computing core calculates the position of the target task block in the matrix C according to the sequence number of the assigned target task block, determines the position of the target task block in the global memory according to the position of the matrix C in the global memory, and reads the task block of the matrix C from the global memory to the unified buffer.

[0050] Specifically, the dimensions of matrix T and matrix C are the same. Before calculation, T and C need to be divided into blocks with the dimension of (M0, N0). The number of blocks and the number of tasks are given by the formula The computing core calculates the position of the task block in the matrix C according to the assigned computing task sequence number. Depending on the position of the matrix C in the global memory, the position of the task block in the global memory can be determined, and the task block of the matrix C is read from the global memory to the unified buffer.

[0051] In one embodiment of the present application, after reading the task block of the matrix C from the global memory to the unified buffer, the second calculation step further includes: performing in the vector calculation unit in the calculation core: scalar multiplication, converting the above The calculation result of the scalar multiplication of is transferred to the position of the task block of the matrix C in the unified buffer; the task block of the matrix T is read from the global memory to the unified buffer; and the vector calculation unit in the calculation core is used to perform scalar multiplication, converting the above The result of the scalar multiplication of is stored in the position of the task block of the matrix T in the uniform buffer; the vector calculation unit in the calculation core is used to calculate Matrix addition, the above The calculation result of the matrix addition is stored in the position of the task block of the matrix C in the unified buffer; and the result of the task block of the matrix C in the unified buffer is written back to the global memory.

[0052] Specifically, in the vector computing unit, The scalar multiplication is performed, and the result is placed in the location of the matrix C task block in the uniform buffer. The matrix T task block is read from the global memory to the uniform buffer and is performed in the vector calculation unit. The scalar multiplication of the matrix T task block is placed in the uniform buffer and is performed in the vector calculation unit. Matrix addition, the calculation result is placed at the location of the matrix C task block in the uniform buffer, and the result in the matrix C task block in the uniform buffer is written back to the global memory.

[0053] In the above process, double buffering is used in both the unified buffer and the global memory, and the memory is divided into two parts to speed up the overall In the calculation of two scalar multiplications, first calculate This is because the temporary matrix T may not have been calculated yet. , fully realize the parallelism of vector calculation unit calculation and matrix calculation unit calculation, and improve the efficiency of overall narrow matrix multiplication. Each calculation core calculates the corresponding block, and writes the calculation result into global memory, finally obtains the calculation of the overall result matrix C, and then reads the result in matrix C to the host end to verify the final calculation result.

[0054] The operations on the matrix calculation unit and the vector calculation unit are performed in parallel, and double buffering is used for acceleration. The two are synchronized while being in parallel. The synchronization at this time is the synchronization of the task block calculation, not the synchronization of the overall calculation. Therefore, the size of the result matrix and the performance block in the first calculation step and the second calculation step are the same, and the dimensions are (M0, N0). The task block result calculated in the first calculation step is stored in the matrix T. When calculating the second calculation step, it is necessary to read the task block corresponding to the matrix T in the global memory. The calculation kernel synchronizes the first calculation step and the second calculation step through synchronization statements, which indicates that it is necessary to apply for an additional M0×N0 space in the global memory. In order to speed up the calculation rate, double buffering is used in the synchronization of the first calculation step and the second calculation step, that is, it is necessary to apply for 2×M0×N0 space. Figure 4 is a pipeline diagram for parallel computing, such as Figure 4 As shown in the figure, it shows how the matrix calculation unit and the vector calculation unit execute their respective tasks in parallel on the NPU of the computing platform, and how to accelerate the data handling and calculation process through double buffering technology, as well as how to synchronize the task block level between the matrix calculation unit and the vector calculation unit. The Cube (matrix) calculation unit calculates the task block T(i), writes the calculation result T of the task block T(i) back to the global memory, calculates the task block T(i+1), writes the calculation result T of the task block T(i+1) back to the global memory, and while the Cube calculation unit writes the calculation result T of the task block T(i) back to the global memory, the Vector (vector) calculation unit reads the task block C(i) into the unified buffer UB and calculates the scalar multiplication. While the Cube calculation unit calculates the task block T(i+1), the Vector calculation unit writes the task block T(i) back to the global memory. Read into the unified buffer UB and calculate scalar multiplication, complete the matrix addition of C(i) and T(i), and write them back to the global memory. At the same time, the Cube calculation unit writes the calculation result T of the task block T(i+1) back to the global memory. At the same time, the Vector calculation unit reads the task block C(i+1) into the unified buffer UB and calculates the scalar multiplication. Then, the Vector calculation unit reads the task block T(i+1) into the unified buffer UB and calculates the scalar multiplication. Finally, the Vector calculation unit completes the matrix addition of C(i+1) and T(i+1) and writes them back to the global memory.

[0055] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the acceleration method for narrow and long matrix multiplication of the present application will be described in detail below in combination with specific embodiments.

[0056] This embodiment relates to a specific method for accelerating narrow matrix multiplication. The overall flow chart is as follows: Figure 5 As shown, the following steps are included:

[0057] Step S1: performing a padding operation on the narrow matrix in a unified buffer of the computing platform to obtain a padding operation result.

[0058] Step S2: The padding operation results are evenly distributed as computing tasks to the matrix computing units in the computing core, and double buffering is used to accelerate the operation.

[0059] Step S3: When reading data in the first preset buffer, mark the task block.

[0060] Step S4: The vector computing unit in the computing core completes scalar multiplication and matrix addition operations, and uses double buffering to accelerate the operations.

[0061] Step S5: Write the operation result back to the global memory.

[0062] Specifically, considering the NPU architecture of the computing platform, when reading data, when the interval between different columns of the matrix is ​​512B or its integer multiple, the reading speed will be the fastest. Therefore, a padding operation is designed before matrix multiplication. On the one hand, padding modifies the spacing between different data columns of the matrix; on the other hand, the storage mode of the data in the calculation block of the task block in the global memory is modified to continuous storage, and continuous reading is achieved by reading in blocks in the subsequent calculation process, thereby speeding up the reading speed of the data block. When padding, the multi-core characteristics of the NPU are fully utilized, and the block calculation in the padding process is divided into blocks and assigned to different cores for calculation. The size of the block is selected according to the size of the unified buffer UB storage. Since the entire operation process uses double buffering, the space of the unified buffer UB needs to be divided into two parts, which are used for padding adjustment. For matrices whose M direction is larger than half of the space of the unified buffer UB, the single column of the matrix is ​​divided into blocks when dividing the blocks. If it is a matrix with a smaller M direction, multiple columns of the matrix are used as a calculation block when dividing the blocks. The overall padding is to read the data into the unified buffer UB, and then write it back to the global memory through the unified buffer UB, but when writing it back, the data arrangement is modified, and the spacing between the two columns is modified. The padding calculation task is distributed to multiple computing cores, making full use of the multi-core computing power of the NPU.

[0063] The target task block required for calculation is read from the global memory, and then written into the unified buffer UB, using the high-speed characteristics of the unified buffer UB to improve the data reading and processing speed. After completing the padding operation in the unified buffer UB, the modified data is written back to the global memory, and the arrangement of the data is adjusted to optimize the subsequent matrix reading efficiency. In this step, the arrangement of the data is changed to ensure that in subsequent calculations, the data can be read in a more efficient way, avoiding unnecessary data movement and buffer conflicts, and improving the overall calculation performance.

[0064] The main calculation process is divided into two parts. The matrix multiplication and Scalar multiplication and matrix addition. This part performs the matrix multiplication operation of the first part, which is the key part of the overall calculation. This part multiplies the padded matrices A and B, temporarily stores the calculation results in T, and writes them back to the global memory. For the A×B calculation process, the task needs to be divided into blocks first. Here, it is assumed that the dimensions of matrix A are (M, K), the dimensions of matrix B are (K, N), and the dimensions of matrix C are (M, N). The dimensions of the temporary matrix T obtained by calculation are (M, N), and the size of the calculation block is (M0, N0). The total number of blocks is , distribute the computing tasks to multiple computing cores, and make full use of the computing power of the multi-core of NPU. The above-mentioned block division is based on the result matrix. At this time, the task block dimension of matrix A is (M0, K), and the task block dimension of matrix B is (K, N0). Block division is performed in the K direction. The task block dimensions of matrix A and matrix B are transformed into (M0, K0) and (K0, N0) respectively. The number of calculations required in the K direction is .

[0065] The computing core determines the corresponding task block position in matrices A and B according to the assigned computing task sequence number and the block size in the K direction. This process depends on the initial layout of matrices A and B in the global memory, so as to accurately locate the specific position of the task block in the global memory. Subsequently, these task blocks are read from the global memory to the first preset buffer, ready for the next step of subdivision and calculation. In the first preset buffer, the task blocks of matrices A and B are further subdivided in the K direction for the second time, and the dimensions are adjusted from (M0, K0) and (K0, N0) to (M0, K1) and (K1, N0). This block strategy can further decompose large tasks and facilitate parallel computing.

[0066] The matrix A and B task blocks that have been divided for the second time in the first preset buffer are transferred to the second preset buffer and the third preset buffer in turn, and the mad function is used to perform matrix multiplication calculations, and the results are accumulated in the fourth preset buffer. In this process, the mad function efficiently performs the core operation of matrix multiplication, multiplies and accumulates the elements of the two matrices, and generates an intermediate result of the multiplication. After completing the accumulation calculation in the fourth preset buffer, the final result in the fourth preset buffer is written back to the specified storage location in the global memory to complete the calculation of the specific task block. Since the entire calculation process makes full use of the double buffer mechanism, that is, the reading and writing of data in the first preset buffer, the second preset buffer, the third preset buffer, and the fourth preset buffer are all performed in parallel, this helps to speed up data processing, reduce waiting time, and thus improve calculation efficiency.

[0067] The first block division in the K direction is because the previous calculation task is divided according to the result matrix. The block size of the result matrix is ​​(M0, N0). However, during the calculation, the dimension in the K direction may be much larger than the first preset buffer memory. Since the size of the first preset buffer memory is limited, it is necessary to divide the blocks in the K dimension. The result matrix T 2,2 The calculation formula is The second block division in the k direction is because the memory of the first preset buffer is several times that of the second preset buffer, the third preset buffer, and the fourth preset buffer. Therefore, further block division is required in the K dimension. 2,1 and B 1,2 Further block division is performed, and the calculation formula is , and finally obtain the calculation of the final task block through two layers of loop nesting.

[0068] When one dimension of M or K is larger and the other dimension is smaller, take matrix A as an example. In the first and second task block reading, the blocks of matrix A read are the same. At this time, if the calculation method is the same as before, the repeated task block content will be read from the global memory to overwrite the data. For this, the task blocks after the block division are marked with serial numbers. When reading in the next time, if the task block serial number is the same as before, the reading from the global memory will be skipped and the previously read data will be used directly. This method will improve the efficiency of matrix multiplication by reducing the overall number of memory accesses. , , at this time, the calculation of both tasks requires A 1,1 , and the order of the two operations is connected, so the second calculation does not require A 1,1 Read in.

[0069] The dimensions of matrix T and matrix C are the same. Before calculation, T and C need to be divided into blocks with the dimension of (M0, N0). The number of blocks and the number of tasks are given by the formula The computing core calculates the position of the task block in the matrix C according to the assigned computing task sequence number. Depending on the position of the matrix C in the global memory, the position of the task block in the global memory can be determined, and the task block of the matrix C is read from the global memory to the unified buffer.

[0070] Performed in the vector computing unit The scalar multiplication is performed, and the result is placed in the location of the matrix C task block in the uniform buffer. The matrix T task block is read from the global memory to the uniform buffer and is performed in the vector calculation unit. The scalar multiplication of the matrix T task block is placed in the uniform buffer and is performed in the vector calculation unit. Matrix addition, the calculation result is placed at the location of the matrix C task block in the uniform buffer, and the result in the matrix C task block in the uniform buffer is written back to the global memory.

[0071] In the above process, double buffering is used in both the unified buffer and the global memory, and the memory is divided into two parts to speed up the overall In the calculation of two scalar multiplications, first calculate This is because the temporary matrix T may not have been calculated yet. , fully realize the parallelism of vector calculation unit calculation and matrix calculation unit calculation, and improve the efficiency of overall narrow matrix multiplication. Each calculation core calculates the corresponding block, and writes the calculation result into global memory, finally obtains the calculation of the overall result matrix C, and then reads the result in matrix C to the host end to verify the final calculation result.

[0072] The operations on the matrix calculation unit and the vector calculation unit are performed in parallel, and double buffering is used for acceleration. The two are synchronized while being in parallel. The synchronization at this time is the synchronization of the task block calculation, not the synchronization of the overall calculation. Therefore, the size of the result matrix and the performance block in the first calculation step and the second calculation step are the same, and the dimensions are (M0, N0). The task block result calculated in the first calculation step is stored in the matrix T. When calculating the second calculation step, it is necessary to read the task block corresponding to the matrix T in the global memory. The calculation kernel synchronizes the first calculation step and the second calculation step through synchronization statements, which indicates that it is necessary to apply for an additional M0×N0 space in the global memory. In order to speed up the calculation rate, double buffering is used in the synchronization of the first calculation step and the second calculation step, that is, it is necessary to apply for 2×M0×N0 space.

[0073] The embodiment of the present application also provides an acceleration device for narrow and long matrix multiplication. It should be noted that the acceleration device for narrow and long matrix multiplication in the embodiment of the present application can be used to execute the acceleration method for narrow and long matrix multiplication provided in the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0074] The following is an introduction to the acceleration device for narrow matrix multiplication provided in an embodiment of the present application.

[0075] Figure 6 Schematic diagram of an acceleration device for narrow matrix multiplication according to an embodiment of the present application. Figure 6 As shown, the device comprises:

[0076] The data processing unit 10 is used to perform a padding operation on the narrow matrix in a unified buffer of the computing platform to obtain a padding operation result, wherein double buffering is used for acceleration during the padding operation, and the padding operation includes modifying the arrangement of data and modifying the spacing between two columns at the same time;

[0077] The first computing unit 20 is used to distribute the padding operation results as computing tasks to the matrix computing units in the computing core of the computing platform, and perform matrix multiplication operations in the matrix computing units to obtain matrix multiplication calculation results, and use double buffering to speed up the calculation process, and mark the task blocks read into the first preset buffer in the computing core;

[0078] The second computing unit 30 is used to perform scalar multiplication and matrix addition operations on the above matrix multiplication calculation results in the vector computing unit in the above computing core, and use double buffering for acceleration to obtain the calculation results of the above narrow matrix.

[0079] In one embodiment of the present application, the data processing unit includes:

[0080] A determination module, used to determine a padding method according to the dimension of the narrow matrix and the size of the unified buffer, wherein the padding method is to perform a single-column single padding operation or a multi-column single padding operation;

[0081] A first allocation module is used to allocate padding calculation tasks to the plurality of computing cores, and any computing core calculates the position of the target task block in the first preset narrow and long matrix and the second preset narrow and long matrix according to the allocated padding calculation task, the block size and the padding method, and the position indicates the position of the target task block in the global memory;

[0082] A first reading module, used for reading the target task block from the global memory and writing the target task block into the unified buffer;

[0083] The first write-back module is used to write the target task block back to the global memory to change the arrangement of the task block data.

[0084] Specifically, when reading data, when the interval between different columns of the matrix is ​​512B or its integer multiple, the reading speed will be the fastest. Therefore, a padding operation is designed before matrix multiplication. On the one hand, padding modifies the spacing between different data columns of the matrix; on the other hand, the storage mode of the data in the calculation block of the task block in the global memory is modified to continuous storage, and continuous reading is achieved by reading in blocks in the subsequent calculation process, thereby speeding up the reading speed of the data block. When padding, the multi-core characteristics of NPU are fully utilized, and the block calculation in the padding process is divided into blocks and assigned to different cores for calculation. The size of the block is selected according to the size of the unified buffer UB storage. Since the entire operation process uses double buffering, the space of the unified buffer UB needs to be divided into two parts, which are used for padding adjustment. For matrices whose M direction is larger than half of the space of the unified buffer UB, the single column of the matrix is ​​divided into blocks when dividing the blocks. If it is a matrix with a smaller M direction, multiple columns of the matrix are used as a calculation block when dividing the blocks. The overall padding is to read the data into the unified buffer UB, and then write it back to the global memory through the unified buffer UB, but when writing it back, the data arrangement is modified, and the spacing between the two columns is modified. The padding calculation task is distributed to multiple computing cores, making full use of the multi-core computing power of the NPU.

[0085] Specifically, the target task block required for calculation is read from the global memory, and then written into the unified buffer UB, using the high-speed characteristics of the unified buffer UB to improve the data reading and processing speed. After completing the padding operation in the unified buffer UB, the modified data is written back to the global memory, and the arrangement of the data is adjusted to optimize the subsequent matrix reading efficiency. In this step, the arrangement of the data is changed to ensure that in subsequent calculations, the data can be read in a more efficient way, avoiding unnecessary data movement and buffer conflicts, and improving the overall calculation performance.

[0086] In one embodiment of the present application, the above narrow matrix multiplication is expressed as , where α and β are multiplication factors. The above narrow matrix multiplication is realized by matrix multiplication operation , scalar multiplication operation and matrix addition operation, the first computing unit is used to perform the matrix multiplication operation , the first computing unit comprises:

[0087] The second allocation module is used to evenly allocate the above-mentioned computing tasks to the above-mentioned matrix computing units in the above-mentioned computing cores of the above-mentioned computing platform, define the dimensions of matrix A as (M, K), the dimensions of matrix B as (K, N), the dimensions of matrix C as (M, N), the dimensions of the temporary matrix T as (M, N), and the size of the blocks of the above-mentioned computing tasks as (M0, N0), and obtain the overall number of blocks , and the task block dimension of the above matrix A is (M0, K), and the task block dimension of the above matrix B is (K, N0);

[0088] The first block module is used to perform block division in the K direction, and transform the task block dimensions of the above matrix A and the above matrix B into (M0, K0) and (K0, N0) respectively, wherein the number of calculations required in the above K direction is , wherein K0 is the block size determined according to the capacity of the first preset buffer.

[0089] Specifically, the main calculation process is divided into two parts: The matrix multiplication and Scalar multiplication and matrix addition. This part performs the matrix multiplication operation of the first part, which is the key part of the overall calculation. This part multiplies the padded matrices A and B, temporarily stores the calculation results in T, and writes them back to the global memory. For the A×B calculation process, the task needs to be divided into blocks first. Here, it is assumed that the dimensions of matrix A are (M, K), the dimensions of matrix B are (K, N), and the dimensions of matrix C are (M, N). The dimensions of the temporary matrix T obtained by calculation are (M, N), and the size of the calculation block is (M0, N0). The total number of blocks is , distribute the computing tasks to multiple computing cores, and make full use of the computing power of the multi-core of NPU. The above-mentioned block division is based on the result matrix. At this time, the task block dimension of matrix A is (M0, K), and the task block dimension of matrix B is (K, N0). Block division is performed in the K direction. The task block dimensions of matrix A and matrix B are transformed into (M0, K0) and (K0, N0) respectively. The number of calculations required in the K direction is .

[0090] In one embodiment of the present application, the first calculation unit further includes:

[0091] The first calculation module is used for, after the task block dimensions of the matrix A and the matrix B are respectively converted into (M0, K0) and (K0, N0), the calculation core calculates the position of the corresponding task block in the matrix A and the matrix B according to the assigned calculation task sequence number and the block in the K direction, and obtains the position of the task block in the global memory according to the position of the matrix A and the matrix B in the global memory;

[0092] A second reading module is used to read the task blocks of the matrix A and the task blocks of the matrix B from the global memory to the first preset buffer;

[0093] The second block division module is used to divide the task blocks of the matrix A and the task blocks of the matrix B in the first preset buffer into blocks for the second time in the K direction, and transform the dimensions of the task blocks of the matrix A and the task blocks of the matrix B into (M0, K1) and (K1, N0) respectively. The number of blocks is ;

[0094] A first transfer module, used for transferring the block division results of the task block of the matrix A and the task block of the matrix B from the first preset buffer to the second preset buffer and the third preset buffer in sequence;

[0095] The accumulation module is used to execute the mad function in the second preset buffer and the third preset buffer, accumulate the calculation results into the fourth preset buffer, and write the results in the fourth preset buffer back to the global memory.

[0096] Specifically, the computing core determines the corresponding task block positions in matrices A and B according to the assigned computing task sequence number and the block size in the K direction. This process depends on the initial layout of matrices A and B in the global memory, so as to accurately locate the specific positions of the task blocks in the global memory. Subsequently, these task blocks are read from the global memory into the first preset buffer, ready for the next step of refinement and calculation. In the first preset buffer, the task blocks of matrices A and B are further refined and divided in the K direction for the second time, and the dimensions are adjusted from (M0, K0) and (K0, N0) to (M0, K1) and (K1, N0). This block strategy can further decompose large tasks and facilitate parallel computing.

[0097] The matrix A and B task blocks that have been divided for the second time in the first preset buffer are transferred to the second preset buffer and the third preset buffer in turn, and the mad function is used to perform matrix multiplication calculations, and the results are accumulated in the fourth preset buffer. In this process, the mad function efficiently performs the core operation of matrix multiplication, multiplies and accumulates the elements of the two matrices, and generates an intermediate result of the multiplication. After completing the accumulation calculation in the fourth preset buffer, the final result in the fourth preset buffer is written back to the specified storage location in the global memory to complete the calculation of the specific task block. Since the entire calculation process makes full use of the double buffer mechanism, that is, the reading and writing of data in the first preset buffer, the second preset buffer, the third preset buffer, and the fourth preset buffer are all performed in parallel, this helps to speed up data processing, reduce waiting time, and thus improve calculation efficiency.

[0098] Specifically, the first block division in the K direction is because the previous calculation task is divided according to the result matrix. The block size of the result matrix is ​​(M0, N0). However, during calculation, the dimension in the K direction may be much larger than the first preset buffer memory. Since the size of the first preset buffer memory is limited, it is necessary to divide the blocks in the K dimension. The calculation formula of the result matrix T2,2 is: The second block division in the k direction is because the memory of the first preset buffer is several times that of the second preset buffer, the third preset buffer, and the fourth preset buffer. Therefore, further block division is required in the K dimension. For A2,1, and B1,2, further block division is performed. The calculation formula is: , and finally obtain the calculation of the final task block through two layers of loop nesting.

[0099] In one embodiment of the present application, the task blocks read into the first preset buffer in the above-mentioned computing core are marked, including: marking the serial numbers of the above-mentioned task blocks after the block division; after marking the serial numbers of the above-mentioned task blocks after the block division, when executing the next computing task, if the serial number of the next task block is the same as the serial number of the task block that has been read, directly use the data of the above-mentioned task block that has been read for calculation.

[0100] Specifically, when one dimension of M or K is larger and the other dimension is smaller, taking matrix A as an example, in the reading of the first and second task blocks, the blocks of matrix A read are the same. At this time, if the calculation method is the same as before, the repeated task block content will be read from the global memory to overwrite the data. For this, the task blocks after the blocks are marked with serial numbers. When reading in the next time, if the task block serial number is the same as before, the reading from the global memory will be skipped and the previously read data will be used directly. This method will improve the efficiency of matrix multiplication by reducing the overall number of memory accesses. , , at this time, the calculation of both tasks requires A 1,1 , and the order of the two operations is connected, so the second calculation does not require A 1,1 Read in.

[0101] In one embodiment of the present application, the above narrow matrix multiplication is expressed as , where α and β are multiplication factors. The above narrow matrix multiplication is realized by matrix multiplication operation , scalar multiplication operation And matrix addition operation, the block dimensions of matrix T and matrix C are (M0, N0), and the number of blocks is , the second computing unit implements the scalar multiplication operation And the above matrix addition operations, including:

[0102] A second calculation module is used for the calculation core to calculate the position of the target task block in the matrix C according to the sequence number of the target task block assigned to it, and determine the position of the target task block in the global memory according to the position of the matrix C in the global memory;

[0103] The third reading module is used to read the task blocks of the matrix C from the global memory to the unified buffer.

[0104] Specifically, the dimensions of matrix T and matrix C are the same. Before calculation, T and C need to be divided into blocks with the dimension of (M0, N0). The number of blocks and the number of tasks are given by the formula The computing core calculates the position of the task block in the matrix C according to the assigned computing task sequence number. Depending on the position of the matrix C in the global memory, the position of the task block in the global memory can be determined, and the task block of the matrix C is read from the global memory to the unified buffer.

[0105] In one embodiment of the present application, the second calculation unit further includes:

[0106] The second transfer module is used to read the task block of the matrix C from the global memory to the unified buffer and then perform the transfer in the vector calculation unit in the calculation core. scalar multiplication, converting the above The calculation result of the scalar multiplication is transferred to the position of the task block of the above matrix C in the above unified buffer;

[0107] A fourth reading module, used for reading the task blocks of the matrix T from the global memory to the unified buffer;

[0108] The third transfer module is used to perform in the vector calculation unit in the above-mentioned calculation core scalar multiplication, converting the above The calculation result of the scalar multiplication of is stored in the position of the task block of the matrix T in the uniform buffer;

[0109] Addition module, used to perform in the vector calculation unit in the above calculation core Matrix addition, the above The calculation result of the matrix addition is stored in the position of the task block of the matrix C in the unified buffer;

[0110] The second write-back module is used to write the results in the task block of the matrix C in the unified buffer back to the global memory.

[0111] Specifically, in the vector computing unit, The scalar multiplication is performed, and the result is placed in the location of the matrix C task block in the uniform buffer. The matrix T task block is read from the global memory to the uniform buffer and is performed in the vector calculation unit. The scalar multiplication of the matrix T task block is placed in the uniform buffer and is performed in the vector calculation unit. Matrix addition, the calculation result is placed at the location of the matrix C task block in the uniform buffer, and the result in the matrix C task block in the uniform buffer is written back to the global memory.

[0112] Double buffering is used in both uniform buffers and global memory, and the memory is divided into two parts to speed up the overall In the calculation of two scalar multiplications, first calculate This is because the temporary matrix T may not have been calculated yet. , fully realize the parallelism of vector calculation unit calculation and matrix calculation unit calculation, and improve the efficiency of overall narrow matrix multiplication. Each calculation core calculates the corresponding block, and writes the calculation result into global memory, finally obtains the calculation of the overall result matrix C, and then reads the result in matrix C to the host end to verify the final calculation result.

[0113] The operations on the matrix calculation unit and the vector calculation unit are performed in parallel, and double buffering is used for acceleration. The two are synchronized while being in parallel. The synchronization at this time is the synchronization of the task block calculation, not the synchronization of the overall calculation. Therefore, the size of the result matrix and the performance block in the first calculation unit and the second calculation unit are the same, and the dimensions are (M0, N0). The task block results of the first calculation step unit are stored in the matrix T. When calculating, the second calculation unit needs to read the task block corresponding to the matrix T in the global memory. The calculation core uses synchronization statements to synchronize the first calculation unit and the second calculation unit, which means that it is necessary to apply for an additional M0×N0 space in the global memory. In order to speed up the calculation rate, double buffering is used in the synchronization of the first calculation unit and the second calculation unit, that is, 2×M0×N0 space needs to be applied.

[0114] The above-mentioned acceleration device for narrow matrix multiplication includes a processor and a memory, and the above-mentioned data processing unit, the first calculation unit, the second calculation unit, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions. The above-mentioned modules are all located in the same processor; or, the above-mentioned modules are located in different processors in any combination.

[0115] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0116] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the acceleration method of the narrow matrix multiplication.

[0117] An embodiment of the present invention provides a device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps of the acceleration method for the above-mentioned narrow matrix multiplication are implemented.

[0118] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0119] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0120] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0121] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0122] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0123] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0124] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0125] Computer readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0126] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0127] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for accelerating narrow matrix multiplication, characterized in that: include: A data processing step of performing a padding operation on the narrow matrix in a unified buffer of the computing platform to obtain a padding operation result, wherein double buffering is used for acceleration during the padding operation, and the padding operation includes modifying the arrangement of data and modifying the spacing between two columns at the same time; In a first calculation step, the padding operation result is evenly distributed as a calculation task to the matrix calculation units in the calculation core of the calculation platform, and a matrix multiplication operation is performed in the matrix calculation unit to obtain a matrix multiplication calculation result, and double buffering is used for acceleration during the calculation process, and the task blocks read into the first preset buffer in the calculation core are marked; In the second calculation step, a scalar multiplication operation and a matrix addition operation are performed on the matrix multiplication calculation result in the vector calculation unit in the calculation core, and double buffering is used for acceleration to obtain the calculation result of the narrow matrix.

2. The method according to claim 1, characterized in that: The data processing steps include: Determine a padding mode according to the dimension of the narrow matrix and the size of the unified buffer, wherein the padding mode is to perform a single-column single padding operation or a multi-column single padding operation; Assigning padding calculation tasks to the plurality of computing cores, and any computing core calculates the position of the target task block in the first preset narrow matrix and the second preset narrow matrix according to the assigned padding calculation task, the block size and the padding method, wherein the position indicates the position of the target task block in the global memory; Read the target task block from the global memory and write the target task block into the unified buffer; The target task block is written back to the global memory to change the arrangement of task block data.

3. The method according to claim 1, characterized in that The narrow matrix multiplication is expressed as , where α and β are multiplication factors, and the narrow matrix multiplication is realized by matrix multiplication operation , scalar multiplication operation and matrix addition operation, the first calculation step implements the matrix multiplication operation , the process of executing the first calculation step includes: The computing tasks are evenly distributed to the matrix computing units in the computing cores of the computing platform. The dimensions of matrix A are defined as (M, K), the dimensions of matrix B are defined as (K, N), the dimensions of matrix C are defined as (M, N), the dimensions of the temporary matrix T are defined as (M, N), the size of the blocks of the computing tasks is defined as (M0, N0), and the overall number of blocks is obtained. , and the task block dimension of the matrix A is (M0, K), and the task block dimension of the matrix B is (K, N0); Divide the matrix A and matrix B into blocks in the K direction, and transform the task block dimensions of the matrix A and the matrix B into (M0, K0) and (K0, N0) respectively. The number of calculations required in the K direction is , wherein K0 is a block size determined according to the capacity of the first preset buffer.

4. The method according to claim 3, characterized in that After the task block dimensions of the matrix A and the matrix B are converted into (M0, K0) and (K0, N0) respectively, the first calculation step further includes: The computing core calculates the position of the corresponding task block in the matrix A and the matrix B according to the assigned computing task sequence number and the block in the K direction, and obtains the position of the task block in the global memory according to the position of the matrix A and the matrix B in the global memory; Read the task blocks of the matrix A and the task blocks of the matrix B from the global memory to the first preset buffer; The task blocks of the matrix A and the task blocks of the matrix B in the first preset buffer are divided into blocks for the second time in the K direction, and the dimensions of the task blocks of the matrix A and the task blocks of the matrix B are respectively transformed into (M0, K1) and (K1, N0), and the number of blocks is ; Transferring the block division results of the task blocks of the matrix A and the task blocks of the matrix B from the first preset buffer to the second preset buffer and the third preset buffer in sequence; The mad function is executed in the second preset buffer and the third preset buffer, the calculation results are accumulated in the fourth preset buffer, and the results in the fourth preset buffer are written back to the global memory.

5. The method according to claim 1, characterized in that Marking the task blocks read into the first preset buffer in the computing core, including: marking the task blocks after being divided into blocks with serial numbers; After marking the serial numbers of the divided task blocks, the method further includes: when executing the next computing task, if the serial number of the next task block is the same as the serial number of the task block that has been read, directly using the data of the task block that has been read for calculation.

6. The method according to claim 1, characterized in that The narrow matrix multiplication is expressed as , where α and β are multiplication factors, and the narrow matrix multiplication is realized by matrix multiplication operation , scalar multiplication operation And matrix addition operation, the block dimensions of matrix T and matrix C are (M0, N0), and the number of blocks is , the second calculation step implements the scalar multiplication operation And the matrix addition operation includes: The computing core calculates the position of the target task block in the matrix C according to the sequence number of the assigned target task block, and determines the position of the target task block in the global memory according to the position of the matrix C in the global memory and the position of the matrix C in the global memory; A task block of the matrix C is read from the global memory to the unified buffer.

7. The method according to claim 6, characterized in that After reading the task blocks of the matrix C from the global memory to the unified buffer, the second calculation step further includes: In the vector computing unit in the computing core, scalar multiplication of The calculation result of the scalar multiplication is transferred to the position of the task block of the matrix C in the unified buffer; Reading a task block of the matrix T from the global memory to the unified buffer; In the vector computing unit in the computing core, scalar multiplication of The calculation result of the scalar multiplication of is stored in the position of the task block of the matrix T in the uniform buffer; In the vector computing unit in the computing core, Matrix addition, the The calculation result of the matrix addition is stored in the position of the task block of the matrix C in the unified buffer; The results of the task block of the matrix C in the uniform buffer are written back to the global memory.

8. An acceleration device for narrow matrix multiplication, characterized in that: include: A data processing unit, used for performing a padding operation on the narrow matrix in a unified buffer of the computing platform to obtain a padding operation result, wherein a double buffer is used for acceleration during the padding operation, and the padding operation includes modifying the arrangement of data and modifying the spacing between two columns at the same time; a first computing unit, configured to distribute the padding operation result as a computing task evenly to the matrix computing units in the computing core of the computing platform, and perform matrix multiplication in the matrix computing units to obtain matrix multiplication calculation results, and use double buffering to speed up the calculation process, and mark the task blocks read into the first preset buffer in the computing core; The second computing unit is used to perform scalar multiplication and matrix addition operations on the matrix multiplication calculation result in the vector computing unit in the computing core, and use double buffering for acceleration to obtain the calculation result of the narrow matrix.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the acceleration method for narrow matrix multiplication described in any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include an acceleration method for performing the narrow matrix multiplication described in any one of claims 1 to 7.

Images