Vector matrix multiplication acceleration method, unit, acceleration unit and system on chip

Abstract

Embodiments of the present application provide a vector matrix multiplication acceleration method, unit, acceleration unit and system on chip, the method is applied to a fifth generation reduced instruction set processor, the method comprises the following steps: obtaining the number N of elements of a first matrix stored in a vector register according to the data length of the elements of the first matrix and the length of the vector register, N is a positive integer; performing rearrangement processing on the first matrix according to the number N to obtain a third matrix which can process N data in parallel; and performing multiplication calculation on the vector and the third matrix by using a single instruction multiple data stream instruction. The present scheme can improve the operation speed of vector matrix multiplication.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the chip technical field, and in particular to a vector matrix multiplication acceleration method, unit, acceleration unit and system on chip. BACKGROUND

[0002] With the continuous development of chip technology, more and more fields need to use vector and matrix multiplication operation, which plays a crucial role. For example, in a typical application scenario, there is often a fully connected layer in a commonly used neural network, and the core operation of the fully connected layer is vector matrix multiplication operation. Therefore, the speed of vector matrix multiplication operation often determines the speed of some chip processing data. The fifth generation reduced instruction set (RISC-V) processor emerges as the chip technology develops, and it is increasingly applied to edge computing devices and IOT devices, so there is a higher requirement for the operation speed of vector matrix multiplication. Therefore, how to improve the speed of vector matrix multiplication operation has become a technical problem to be solved. SUMMARY

[0003] In view of this, embodiments of the present application provide a vector matrix multiplication acceleration method, unit, acceleration unit and system on chip to at least partially solve the above problems.

[0004] According to a first aspect of embodiments of the present application, a vector matrix multiplication acceleration method is provided, applied to a fifth generation reduced instruction set processor, and the method comprises:

[0005] According to the data length of the elements of the first matrix and the length of the vector register, the number N of elements of the first matrix stored in the vector register is obtained, N being a positive integer;

[0006] According to the number N, the first matrix is rearranged to obtain a third matrix that can process N data in parallel;

[0007] The single instruction multiple data stream instruction is used to perform multiplication calculation on the vector and the third matrix.

[0008] According to a second aspect of embodiments of the present application, a vector matrix multiplication acceleration unit is provided, applied to a fifth generation reduced instruction set processor, and the vector matrix multiplication acceleration unit comprises:

[0009] The first obtaining unit is configured to obtain the number N of elements of the first matrix stored in the vector register according to the data length of the elements of the first matrix and the length of the vector register, N being a positive integer;

[0010] The second obtaining unit is configured to rearrange the first matrix according to the number N to obtain a third matrix that can process N data in parallel;

[0011] The multiplication calculation unit is configured to perform multiplication calculation on the vector and the third matrix by using a single instruction multiple data stream instruction.

[0012] According to a third aspect of the embodiments of the present application, an acceleration unit is provided, comprising:

[0013] The vector matrix multiplication acceleration unit according to any one of the second aspect above;

[0014] The controller is configured to control the vector matrix multiplication acceleration unit to work.

[0015] According to a fourth aspect of the embodiments of the present application, an electronic device is provided, comprising:

[0016] The acceleration unit according to the third aspect above;

[0017] The scheduling unit is configured to schedule the acceleration unit to perform a vector matrix multiplication acceleration task.

[0018] According to a fifth aspect of the embodiments of the present application, a system on chip is provided, comprising the acceleration unit according to the third aspect above.

[0019] According to the vector matrix multiplication acceleration method provided by the embodiments of the present application, the number N of the vector register storing the elements of the first matrix can be obtained according to the data length of the elements of the first matrix and the length of the vector register, N being a positive integer, then the first matrix can be rearranged according to the number N to obtain the third matrix which can process N data in parallel, and finally the multiplication calculation on the vector and the third matrix can be performed by using a single instruction multiple data stream instruction, so that the third matrix which is more convenient to process is obtained by rearranging the first matrix in the embodiments of the present application, the storage structure of the first matrix in the memory is optimized, and the multiplication calculation on the vector and the third matrix is performed by using the single instruction multiple data stream instruction to obtain the result of the multiplication calculation on the vector and the first matrix, thereby the speed of the vector matrix multiplication calculation can be effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments described in the embodiments of the present application, and other drawings can also be obtained by those skilled in the art according to these drawings.

[0021] Figure 1 is a schematic diagram of an exemplary first matrix arranged in the memory according to an embodiment of the present application;

[0022] Figure 2 is a flowchart of an exemplary vector matrix multiplication acceleration method according to an embodiment of the present application;

[0023] Figure 3 is a schematic diagram of an exemplary second matrix arranged in memory according to an embodiment of the present application;

[0024] Figure 4 is a schematic diagram of an exemplary third matrix arranged in memory according to an embodiment of the present application;

[0025] Figure 5 is a structural block diagram of an exemplary vector-matrix multiplication acceleration unit according to an embodiment of the present application;

[0026] Figure 6 is a structural block diagram of an exemplary electronic device to which an embodiment of the present application is applied;

[0027] Figure 7 is a structural diagram of an exemplary scheduling unit and acceleration unit inside an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] The present application is described in the following based on embodiments, but the present application is not limited to these embodiments only. In the following detailed description of the present application, some specific details are described in detail. The present application can also be completely understood without the description of these details by those skilled in the art. In order to avoid obscuring the essence of the present application, well-known methods, processes, procedures are not described in detail. In addition, the drawings are not necessarily drawn to scale.

[0029] Vector matrix multiplication acceleration method

[0030] In order to embody the advantages of the vector-matrix multiplication acceleration method in the embodiments of the present application, before the vector-matrix multiplication acceleration method in the embodiments of the present application is described, a vector-matrix multiplication calculation scheme in the related art is introduced.

[0031] In the related art, vector-matrix multiplication is applied in many fields, for example, a fully connected layer in a neural network, the core operation of which is vector-matrix multiplication operation. In vector-matrix multiplication operation, a vector is stored as a 1-row H-column row matrix in the memory of a chip, and a matrix (referred to as a first matrix) for multiplication operation with the vector is stored as a W-row H-column matrix in the memory of the chip. The result of multiplication of the two is a W-row 1-column column matrix. The first matrix is stored in the memory by row, and with reference to Figure 1 , an exemplary first matrix B1 has W=8 rows and H=12 columns, and has 96 elements b0-b95 stored in the memory by row. The data type of the elements in the vector and the first matrix can be floating point or fixed point, and the present application takes floating point as an example.

[0032] With reference to Figure 1As shown, a conventional vector matrix multiplication operation mode is that the vector A is multiplied by the first matrix B1. Taking H=12 and W=8 as an example, when the vector matrix multiplication operation is performed, the element b0 of the first row and the first column of the first matrix is read from the memory by row in the first clock cycle, and is multiplied by the element a0 of the first column of the vector. Then the multiplication result is stored in the register. Then the elements from the second to the eighth of the first row of the first matrix b are read by row. When the result matrix (i.e. the result of the multiplication of the vector A and the first matrix B1, which has W elements) is calculated, each element of the vector A needs to be multiplied by the corresponding element of the first column of the first matrix B1 and then added (for example, refer to Figure 1 That is, a0 of the vector a is multiplied by b0 of the first matrix, a1 is multiplied by b8, a2 is multiplied by b16, a3 is multiplied by b24, a4 is multiplied by b32, a5 is multiplied by b40, a6 is multiplied by b48, a7 is multiplied by b56, a8 is multiplied by b64, a9 is multiplied by b72, a10 is multiplied by b80, and a11 is multiplied by b88. Then the 12 results are added, and the sum is the value of the first element of the result matrix.

[0033] Since when the value of each element in the result matrix is calculated, the elements of the first matrix B1 are read from the memory by row and multiplied by the corresponding elements in the vector A, multiple clock cycles are needed, and thus the reading and operation speed is slow. For example, refer to Figure 1 After a0 and b0 are multiplied, since the elements of the first matrix B1 are read from the memory by row, the time for reading b1-b7 is needed, until b8 is read. Then a1 is multiplied by b8. After that, the result of the multiplication of a1 and b8 is added to the result of the multiplication of a0 and b0. Similarly, a2 is multiplied by b16, and the time for reading b9-b15 is needed. a3 is multiplied by b24, a4 is multiplied by b32, a5 is multiplied by b40, a6 is multiplied by b48, a7 is multiplied by b56, a8 is multiplied by b64, a9 is multiplied by b72, a10 is multiplied by b80, and a11 is multiplied by b88. Thus, when the value of the first element of the result matrix is calculated, it can be seen that the reading and operation speed is slow, and the reading and operation of each element are slow.

[0034] Referring to the flowchart in Figure 2 , the application provides a vector matrix multiplication acceleration method applied to a fifth generation reduced instruction set processor. The method comprises the following steps S202, S204 and S206.

[0035] S202: According to the data length of the elements of the first matrix and the length of the vector register, the number N of the elements of the first matrix stored in the vector register is obtained, and N is a positive integer.

[0036] First of all, it needs to be pointed out that the vector matrix multiplication acceleration method in the present application is aimed at the form of the vector and the first matrix, which is no different from that in the related art, that is, the vector is stored as a 1-row H-column row matrix in the memory of the chip, and the matrix (referred to as the first matrix) which is subjected to multiplication operation with the vector is stored as a W-row H-column matrix in the memory of the chip. In the vector matrix multiplication operation, the result of multiplication of the two is a W-row 1-column column matrix. The first matrix is stored by row in the memory.

[0037] The fifth generation reduced instruction set processor is a processor based on the RSIC-V instruction set architecture, which is completely open source and free, separates benchmark instructions and extension instructions, and can customize modules and extensions through extension instructions. The RSIC-V instruction set includes SIMD instructions (single instruction multiple data stream instructions). The fifth generation reduced instruction set processor described in the embodiments of the present application can be used on computing devices such as computers, edge computing devices, and IOT devices. In the embodiments of the present application, the vector matrix multiplication can be accelerated by combining the SIMD instructions later, reducing the time consumed for calculation, and reducing the computing overhead of the processor.

[0038] In the embodiments, the data type of the elements in the first matrix is a floating point number or a fixed point number. The data length of the elements of the first matrix can refer to the data length of the floating point number or the fixed point number storing each element in the first matrix. For example, if the floating point number is of the float type, the data type of the elements of the first matrix can be one of 16-bit float data, 32-bit float data, and 64-bit float data, and the data length of the elements corresponding thereto is 16 bits, 32 bits, and 64 bits, respectively. For example, if the fixed point number is of the int type, the data type of the elements of the first matrix can include one of 8-bit int data, 16-bit int data, 32-bit int data, and 64-bit int data, and the data length of the elements corresponding thereto is 8 bits, 16 bits, 32 bits, and 64 bits, respectively. In order to facilitate the description of the embodiments, the data type of the elements of the first matrix is 32-bit float data (i.e., the data length of the elements of the first matrix is 32 bits) in the following embodiments.

[0039] In the embodiments, the data type of the elements of the vector subjected to the vector matrix multiplication is the same as the data type of the elements of the first matrix, that is, the data length of the elements of the vector is the same as the data length of the elements of the first matrix. For example, if the data type of the elements of the first matrix is 32-bit float data, the data type of each element in the vector is also 32-bit float data. In addition, each element of the result matrix of the vector matrix multiplication is also 32-bit float data.

[0040] In this embodiment, the vector register can store the result of multiplying the elements of the vector with the elements of the matrix, and the length of the vector register can refer to the number of bits of the vector register. When the data length of the elements of the first matrix is determined, the length of the vector register determines the number N of the elements of the first matrix stored in the vector register.

[0041] In one embodiment, the step S202 includes: dividing the length of the vector register by the data length of the elements of the first matrix to obtain the number N of the elements of the first matrix stored in the vector register. In this way, when the vector matrix multiplication is performed subsequently, the elements of the first matrix can be read from the memory row by row according to the maximum number of elements that the vector register can store, so that the computing resources can be utilized more fully.

[0042] For example, the vector register can be a SIMD register, which can support vector calculation of fixed-point numbers or floating-point numbers of different lengths. For example, the storage bit number (i.e., the length) of the vector register is 128 bits (i.e., 128 bits), if the data type of the elements of the first matrix is 32-bit float data, i.e., the data length of the elements of the first matrix is 32 bits, then N = 128 / 32 = 4, and the vector register can store at most 4 elements of the same data length as the elements of the first matrix. Similarly, if the data type of the elements of the first matrix is 16-bit float data, i.e., the data length of the elements of the first matrix is 16 bits, then N = 128 / 16 = 8; if the data type of the elements of the first matrix is 32-bit int data, i.e., the data length of the elements of the first matrix is 32 bits, then N = 128 / 32 = 4.

[0043] For the convenience of describing this embodiment, the vector register is taken as a SIMD register (with a length of 128 bits) in the following description, the data type of the elements of the first matrix is taken as 32-bit float data (e.g., the data length of the elements of the first matrix is 32 bits), and N = 4 is taken as an example.

[0044] S204: rearranging the first matrix according to the number N to obtain a third matrix that can process N data in parallel.

[0045] The rearrangement can refer to changing the storage structure of the first matrix in the memory, for example, the elements of the first matrix originally stored in the memory can be arranged into the third matrix, and the total number of elements of the third matrix is the same as that of the first matrix. For example, the N data that can be processed in parallel can refer to that N elements in the third matrix and the elements in the vector can be multiplied in parallel using a single instruction multiple data stream instruction (SIMD instruction).

[0046] In the related art, Single Instruction Multiple Data (SIMD) is a technology that uses one controller to control multiple processors to simultaneously perform the same operation on each of a group of data to achieve spatial parallelism. Using a SIMD instruction can accelerate the operation speed of vector and third matrix multiplication.

[0047] In this application, the first matrix is rearranged to optimize the storage structure of the first matrix in the memory to obtain a third matrix that can process N data in parallel, so that it can be read and operated more efficiently, reduce waiting time and delay, and reduce multiplication time.

[0048] In an embodiment of the application, the first matrix can be rearranged into a third matrix in different ways. For example, in the first case, the first matrix can be directly rearranged into a third matrix, and in the second case, the first matrix can be first rearranged into an intermediate matrix that meets a certain condition, and then the intermediate matrix can be rearranged into a third matrix.

[0049] For example, corresponding to the above-mentioned second case, in one embodiment, step S204 can include the following steps S2041 and S2042:

[0050] S2041: Rearranging the first matrix to obtain a second matrix that can be read continuously in a vector register.

[0051] S2042: Rearranging the second matrix according to the number N to obtain a third matrix that can process N data in parallel.

[0052] Specifically, when performing vector and second matrix multiplication, the second matrix can be read continuously by row. The difference between the second matrix and the first matrix is that when the vector is multiplied by the first matrix, the elements in the vector and the corresponding elements in the first matrix cannot be read continuously by row. Each time an element in the first matrix is multiplied by an element in the vector, it needs to wait for a period of time. Therefore, the first matrix is first rearranged to obtain a second matrix that can be read by row, reducing the waiting time and speeding up the reading speed, thereby improving the operation speed to a certain extent. Then, according to the number N, the second matrix is rearranged to obtain a third matrix that can process N data in parallel.

[0053] In one embodiment, step S2041 includes transposing the first matrix to obtain a second matrix, the first matrix being an H-row and W-column matrix, and the second matrix being a W-row and H-column matrix, W and H being positive integers.

[0054] Referring to Figure 1 and Figure 3 It is understood that Figure 1An example of the first matrix B1 is shown in the following table, Figure 3 An example of the second matrix B2 is shown in the following table, the first matrix B1 is an H row W column matrix, the second matrix B2 is a W row H column matrix, in this example, W=8, H=12, when transposing the first matrix B1, the H elements of the 1st column of the first matrix B1 in top-to-bottom order (i.e. 12 elements of the 1st column) are arranged in left-to-right order to the 1st row of the second matrix B2, the H elements of the 2nd column of the first matrix B1 in top-to-bottom order are arranged in left-to-right order to the 2nd row of the second matrix B2, the H elements of the 3rd column of the first matrix B1 in top-to-bottom order are arranged in left-to-right order to the 3rd row of the second matrix B2, the H elements of the 4th column of the first matrix B1 in top-to-bottom order are arranged in left-to-right order to the 4th row of the second matrix B2, the H elements of the 5th column of the first matrix B1 in top-to-bottom order are arranged in left-to-right order to the 5th row of the second matrix B2, the H elements of the 6th column of the first matrix B1 in top-to-bottom order are arranged in left-to-right order to the 6th row of the second matrix B2, the H elements of the 7th column of the first matrix B1 in top-to-bottom order are arranged in left-to-right order to the 7th row of the second matrix B2, the H elements of the 8th column of the first matrix B1 in top-to-bottom order are arranged in left-to-right order to the 8th row of the second matrix B2.

[0055] Further, if the vector A is multiplied with the second matrix B2, the elements read from the second matrix B2 can be read in row from left to right continuously, which reduces the waiting time and speeds up the reading compared with directly multiplying the vector A with the first matrix B1.

[0056] Although the second matrix B2 reduces the waiting time and speeds up the reading of the vector matrix multiplication calculation compared with the first matrix B1, it cannot be further accelerated using the SIMD instruction. Moreover, it still has obvious defects to be overcome, therefore, in the second case of the rearrangement processing of the present application, the second matrix is taken as an intermediate matrix, and then the second matrix is rearranged to obtain a third matrix.

[0057] In the following, the multiplication of the vector A with the corresponding elements of the 1st row of the second matrix B2 to obtain the first element of the result matrix is taken as an example to illustrate the defects of the second matrix B2, and the other calculation processes can be similarly deduced, so that the significant effect of the present application can be more obviously seen:

[0058] Specifically, when the vector A is multiplied by the second matrix B2, the first N (N=4) elements b0, b8, b16, b24 of the first row of the second matrix are read from the memory in row order, and are multiplied by the first N (N=4) elements a0, a1, a2, a3 of the vector A respectively to obtain four values: a0*b0, a1*b8, a2*b16, a3*b24, which are respectively stored in the N (N=4) positions of the vector register (as mentioned above, in this application, the data type of the first matrix is 32-bit float data, the length of the vector register is 128 bits, N=4, and a0*b0, a1*b8, a2*b16, a3*b24 are also 32-bit float data);

[0059] The middle N (N=4) elements b32, b40, b48, b56 of the first row of the second matrix are read from the memory in row order, and are multiplied by the middle N (N=4) elements a4, a5, a6, a7 of the vector A respectively to obtain four values: a4*b32, a5*b40, a6*b48, a7*b56, which are respectively stored in the N (N=4) positions of the vector register, and the already stored a0*b0, a1*b8, a2*b16, a3*b24 in the vector register are automatically added to the later stored a4*b32, a5*b40, a6*b48, a7*b56 respectively by using the accumulation function of the vector register, and the values in the vector register after the accumulation are respectively: a0*b0+a4*b32, a1*b8+a5*b40, a2*b16+a6*b48, a3*b24+a7*b56.

[0060] Then, the last N (N=4) elements b64, b72, b80, b88 of the first row of the second matrix are read from the memory in row order, and are multiplied by the last N (N=4) elements a8, a9, a10, a11 of the vector A respectively to obtain four values: a8*b64, a9*b72, a10*b80, a11*b88, which are respectively stored in the N (N=4) positions of the vector register, and the already stored a0*b0+a4*b32, a1*b8+a5*b40, a2*b16+a6*b48, a3*b24+a7*b56 in the vector register are automatically added to the later stored a8*b64, a9*b72, a10*b80, a11*b88 respectively by using the accumulation function of the vector register, and the values in the vector register after the accumulation are respectively: a0*b0+a4*b32+a8*b64, a1*b8+a5*b40+a9*b72, a2*b16+a6*b48+a10*b80, a3*b24+a7*b56+a11*b88.

[0061] Then, the four accumulated values of a0*b0+a4*b32+a8*b64, a1*b8+a5*b40+a9*b72, a2*b16+a6*b48+a10*b80, a3*b24+a7*b56+a11*b88 are read from the vector register, and the first element of the result matrix is obtained after adding the four accumulated values.

[0062] Directly multiplying the vector A with the second matrix B2 after transposing the first matrix B1 cannot be accelerated by the SIMD instruction, the performance of the fifth generation RISC processor is not fully utilized, and this process needs to add the N accumulated values to obtain the elements of the result matrix, which also increases the operation time of adding the N accumulated values.

[0063] In the present application, the second matrix is rearranged based on the number N to obtain the third matrix which can process N data in parallel to avoid this problem.

[0064] In one embodiment, the step S2042 can include: sequentially taking N rows from the first row to the Wth row of the second matrix, and arranging the data from the first column to the Hth column every N rows, and the obtained N*H data constitutes a row of data of the third matrix; until the first row to the W / Nth row data is obtained, the third matrix is constituted, and the third matrix is a W / N row and N*H column matrix.

[0065] Referring to Figure 3 and Figure 4 It is understood that Figure 3 An example of the second matrix B2 is shown in Figure 4 An example of the second matrix B2 is shown in the second matrix B1 is an H row and W column matrix, and the third matrix B2 is a W / N row and N*H column matrix. In this example, W=8, H=12, so the third matrix B2 is a 2 row and 48 column matrix (as mentioned above, in the present application, the data type of the elements of the first matrix is 32-bit float data, the length of the vector register is 128 bits, N=4, and the rearranged second matrix B2 and the third matrix B3 are also 32-bit float data).

[0066] In the process of rearranging the second matrix B2 into the third matrix B3, the first N (N=4) elements (b0, b1, b2, b3) of the first column of the second matrix B2 are arranged in order to the positions of the 1st to 4th elements of the first row of the third matrix, the first N (N=4) elements (b8, b9, b10, b11) of the second column of the second matrix B2 are arranged in order to the positions of the 5th to 8th elements of the first row of the third matrix, the first N (N=4) elements (b16, b17, b18, b19) of the third column of the second matrix B2 are arranged in order to the positions of the 9th to 12th elements of the first row of the third matrix, and so on, until the first N (N=4) elements (b88, b89, b90, b91) of the Hth (H=12) column of the second matrix B2 are arranged in order to the positions of the (N*H-3), (N*H-2), (N*H-1), N*H (45th to 48th) elements of the first row of the third matrix, and thus the rearrangement of the elements of the first row of the third matrix B3 is completed.

[0067] The last N (N=4) elements (b4, b5, b6, b7) of the first column of the second matrix B2 are arranged in order to the positions of the 1st to 4th elements of the second row of the third matrix, the last N (N=4) elements (b12, b13, b14, b15) of the second column of the second matrix B2 are arranged in order to the positions of the 5th to 8th elements of the second row of the third matrix, the last N (N=4) elements (b20, b21, b22, b23) of the third column of the second matrix B2 are arranged in order to the positions of the 9th to 12th elements of the second row of the third matrix, and so on, until the last N (N=4) elements (b92, b93, b94, b95) of the Hth (H=12) column of the second matrix B2 are arranged in order to the positions of the (N*H-3), (N*H-2), (N*H-1), N*H (45th to 48th) elements of the second row of the third matrix, and thus the rearrangement of the elements of the second row of the third matrix B3 is completed.

[0068] The number of rows of the third matrix B3 is W / N=2, and thus the first and second rows are rearranged to obtain the third matrix.

[0069] The first matrix is rearranged into the second matrix, and the second matrix is rearranged into the third matrix, which optimizes the storage structure of the first matrix in the memory. When the vector is multiplied with the third matrix to calculate the multiplication result of the vector and the first matrix, the reading time of the elements of the third matrix is much less than that of the first matrix, and the calculation process can be parallel processed for N data, which effectively speeds up the operation.

[0070] It can be understood that the number of rows and columns of the third matrix depends on the specific values of W, H and N, which are W=8, H=12 and N=4 in this example only, and are different in other cases. This example is only used for conveniently understanding the embodiment and is not limited herein.

[0071] For example, corresponding to the first case described above (i.e., directly rearranging the first matrix into the third matrix), step S204 can include: sequentially taking N columns from the first column to the Wth column of the first matrix, and arranging the data from the first row to the Hth row in each N columns, and the obtained N*H data constitutes a row of data of the third matrix; until the first row to the W / Nth row data are obtained, the third matrix is constituted, and the third matrix is a W / N row and N*H column matrix.

[0072] The specific rearrangement process is not described herein again, and the related content in this case can be understood with reference to the first matrix B1 in Figure 1 and the third matrix in Figure 4 .

[0073] Directly rearranging the first matrix into the third matrix optimizes the storage structure of the first matrix in the memory. When the vector and the third matrix are used for multiplication to calculate the multiplication result of the vector and the first matrix, the reading time of the elements of the third matrix is much smaller than the reading time of the elements of the first matrix when the first matrix is directly used for operation, and the calculation process can also be parallelly processed for N data, effectively accelerating the operation speed.

[0074] The multiplication process between the vector and the third matrix will be specifically introduced in step S206 below.

[0075] S206: performing multiplication calculation on the vector and the third matrix by using a single instruction multiple data stream instruction.

[0076] In the embodiment of the application, the number N of elements of the first matrix stored in the vector register can be obtained according to the data length of the elements of the first matrix and the length of the vector register, N is a positive integer, then the first matrix can be rearranged according to the number N to obtain the third matrix which can be parallelly processed for N data, and finally the multiplication calculation on the vector and the third matrix can be performed by using a single instruction multiple data stream instruction. Therefore, in the embodiment of the application, the third matrix which is more convenient to process is obtained by rearranging the first matrix, the storage structure of the first matrix in the memory is optimized, and the multiplication operation on the vector and the third matrix is performed by using a single instruction multiple data stream instruction to obtain the multiplication result of the vector and the first matrix, thereby effectively improving the speed of the vector-matrix multiplication operation.

[0077] Specifically, after the third matrix is obtained by rearranging the first matrix according to any of the above-mentioned manners in S204, the multiplication operation can be continued on the third matrix by using a single instruction multiple data stream instruction (SIMD instruction), and the third matrix can be used to process N pieces of data in parallel, which is fast and can fully utilize the performance of the fifth generation reduced instruction set processor.

[0078] In one embodiment, the step S206 can include performing multiplication calculation by using the formula , wherein,

[0079] Va={a[j], a[j],..., a[j]} represents a vector, wherein the number of a[j] is N;

[0080] Vb={b[N*j], b[N*j+1],..., b[N*j+N-1]} represents the third matrix, j is an integer from 0 to H, and i is an integer from 0 to W.

[0081] In this application, a[j] in Va, that is, one of the elements in the vector A. Vc[i] can represent each element in the multiplication result of the vector and the third matrix (that is, the multiplication result of the vector and the first matrix), that is, Vc[0] is the first element, Vc[1] is the second element, and the rest are similar.

[0082] Referring to Figure 4 , the formula can be understood as multiplying each element in the vector A in order with N elements in the third matrix B3, respectively, for example, Va={a0, a0, a0, a0}, Vb={b0, b1, b2, b3}; Va={a1, a1, a1, a1}, Vb={b8, b9, b10, b11}; Va={a2, a2, a2, a2}, Vb={b16, b17, b18, b19};...; Va={a11, a11, a11, a11}, Vb={b88, b89, b90, b91}.

[0083] The process of multiplying the vector A and the third matrix in Figure 4 according to the above formula by using a single instruction multiple data stream instruction (SIMD instruction) will be described below for easy understanding.

[0084] For example, taking the value of the first N elements of the result matrix obtained by multiplying each element of the vector A with the elements of the first row of the third matrix B3 (because the third matrix B3 is only 2 rows in this example, so it is the first N) as an example:

[0085] When the third matrix B3 is multiplied by the vector A, the first N (N=4) elements of the first row of the third matrix B3, i.e. the first to fourth elements b0, b1, b2, b3, are read from the memory in row order, a SIMD instruction is executed to multiply b0, b1, b2, b3 by a0 respectively, and then four values a0*b0, a0*b1, a0*b2, a0*b3 are obtained, which are respectively stored in the N (N=4) positions of the vector register;

[0086] Then the other N (N=4) elements of the first row of the third matrix B3, i.e. the fifth to eighth elements b8, b9, b10, b11, are read from the memory in row order, a SIMD instruction is executed to multiply b8, b9, b10, b11 by a1 respectively, and then four values a1*b8, a1*b9, a1*b10, a1*b11 are obtained, which are respectively stored in the N (N=4) positions of the vector register in which a0*b0, a0*b1, a0*b2, a0*b3 have been stored, and the vector register is automatically added with a1*b8, a1*b9, a1*b10, a1*b11 respectively corresponding to a0*b0, a0*b1, a0*b2, a0*b3, and the values in the vector register are added to a0*b0+a1*b8, a0*b1+a1*b9, a0*b2+a1*b10, a0*b3+a1*b11 respectively;

[0087] In sequence, until the last N (N=4) elements of the first row of the third matrix B3, i.e. the (N*H-3), (N*H-2), (N*H-1), N*H elements b88, b89, b90, b91, are read from the memory in row order, a SIMD instruction is executed to multiply b88, b89, b90, b91 by a11 (the Hth element of the vector A) respectively, and then four values a11*b88, a11*b89, a11*b90, a11*b91 are obtained, which are respectively stored in the N (N=4) positions of the aforementioned vector register, and the vector register is automatically added with a11*b88, a11*b89, a11*b90, a11*b91 respectively corresponding to the values already stored before, and the values in the vector register are added to a0*b0+a1*b8+...+a11*b88, a0*b1+a1*b9+...+a11*b89, a0*b2+a1*b10+...+a11*b90, a0*b3+a1*b11+...+a11*b91 respectively;

[0088] Similarly, multiplying the vector A with the elements of the 2nd row of the third matrix B3, the values of the last N (N=4) elements of the resultant matrix can be obtained (as the third matrix B3 is only 2 rows in this example, thus the last N), i.e.:

[0089] The first N (N=4) elements of the 2nd row of the third matrix B3, i.e. the 1st-4th elements b4, b5, b6, b7, are read from the memory in row order, and the SIMD instruction is executed to multiply b4, b5, b6, b7 with a0 respectively, and then four values a0*b4, a0*b5, a0*b6, a0*b7 are obtained, which are stored in the N (N=4) positions of another vector register respectively;

[0090] Then the other N (N=4) elements of the 2nd row of the third matrix B3, i.e. the 5th-8th elements b12, b13, b14, b15, are read from the memory in row order, and the SIMD instruction is executed to multiply b12, b13, b14, b15 with a1 respectively, and then four values a1*b12, a1*b13, a1*b14, a1*b15 are obtained, which are stored in the N (N=4) positions of the other vector register in which a0*b4, a0*b5, a0*b6, a0*b7 have been stored, and by using the accumulation function of the vector register, a0*b4, a0*b5, a0*b6, a0*b7 are automatically accumulated with a1*b12, a1*b13, a1*b14, a1*b15 respectively, and the values in the vector register are accumulated to be a0*b4+a1*b12, a0*b5+a1*b13, a0*b6+a1*b14, a0*b7+a1*b15 respectively;

[0091] Similarly, the last N (N=4) elements of the 2nd row of the third matrix B3, i.e., the (N*H-3), (N*H-2), (N*H-1), N*H elements b92, b93, b94, b95 are read from the memory in the row order, and the SIMD instruction is executed to multiply b92, b93, b94, b95 with a11 (the Hth element of the vector A) respectively, and then four values a11*b92, a11*b93, a11*b94, a11*b95 are obtained, which are stored in the N (N=4) positions of the aforementioned another vector register respectively, and the accumulation function of the vector register is used to automatically accumulate a11*b92, a11*b93, a11*b94, a11*b95 with the values already stored before respectively, and the values in the vector register after the accumulation are: a0*b4+a1*b12+...+a11*b92, a0*b5+a1*b13+...+a11*b93, a0*b6+a1*b14+...+a11*b94, a0*b7+a1*b15+...+a11*b95 respectively.

[0092] The four accumulated values are the values of the last N (N=4) elements of the result matrix, and the calculation of the values of the elements in the result matrix does not have the waiting time similar to the multiplication of the vector A and the first matrix B1, and does not need to additionally perform the step of adding the N accumulated values stored in the vector register once again as in the multiplication of the vector A and the second matrix B2, and the operation time of adding the N accumulated values is not increased.

[0093] Obviously, in the present application, by optimizing the storage structure of the first matrix B1 in the memory, the first matrix B1 is rearranged and processed into the third matrix B3, and through the multiplication of the vector A and the third matrix B3, the elements of the third matrix can be read in the row order without waiting time, the data reading speed is accelerated, and the parallel calculation can be performed by using the SIMD instruction, the performance of the fifth generation reduced instruction set processor can be fully utilized, and the accumulation function of the vector register is also fully utilized, so that the operation speed of the vector matrix multiplication is effectively improved.

[0094] It can be understood that the above embodiments and examples are only used as some optional embodiments or exemplary explanations of the present application, and do not limit any of the present application.

[0095] It should be noted that through sufficient experimental verification, compared with the scheme of directly multiplying the vector and the first matrix in the related art, the operation time of the vector matrix multiplication in the scheme can be reduced by more than [(1-X / Y)*100%], wherein X is the data length of the elements of the first matrix, and Y is the length of the vector register, so the acceleration effect of the scheme on the vector matrix multiplication is very significant. For example, for the aforementioned example, the data type of the elements of the first matrix is 32-bit float data (i.e., the data length of the elements of the first matrix is 32 bits), and the length of the vector register is 128 bits, compared with the scheme of directly multiplying the vector and the first matrix in the related art, the operation time of the vector matrix multiplication can be reduced by more than 75%, and the acceleration effect on the vector matrix multiplication is very significant.

[0096] In summary, the vector matrix multiplication acceleration method in the embodiment of the application can obtain the number N of the elements of the first matrix stored in the vector register according to the data length of the elements of the first matrix and the length of the vector register, N is a positive integer, then the first matrix can be rearranged according to the number N to obtain the third matrix that can process N data in parallel, and finally the multiplication of the vector and the third matrix can be calculated by using the single instruction multiple data stream instruction, so that the third matrix that is more convenient to process is obtained by rearranging the first matrix in the embodiment of the application, the storage structure of the first matrix in the memory is optimized, and the multiplication of the vector and the first matrix is calculated by using the single instruction multiple data stream instruction to obtain the result of the multiplication, which can effectively improve the speed of the vector matrix multiplication.

[0097] Vector matrix multiplication acceleration unit

[0098] Figure 5 is a structural diagram of a vector matrix multiplication acceleration unit 1000 of an embodiment of the application. The vector matrix multiplication acceleration unit 1000 is applied to a fifth generation reduced instruction set processor, and the vector matrix multiplication acceleration unit 1000 includes:

[0099] A first obtaining unit 1001 is configured to obtain the number N of the elements of the first matrix stored in the vector register according to the data length of the elements of the first matrix and the length of the vector register, N is a positive integer.

[0100] A second obtaining unit 1002 is configured to rearrange the first matrix according to the number N to obtain the third matrix that can process N data in parallel.

[0101] A multiplication calculating unit 1003 is configured to calculate the multiplication of the vector and the third matrix by using the single instruction multiple data stream instruction.

[0102] In one of the embodiments, the first obtaining unit is specifically configured to divide the length of the vector register by the data length of the elements of the first matrix to obtain the number N of the elements of the first matrix stored in the vector register.

[0103] In one of the embodiments, the second obtaining unit 1002 is specifically configured to perform rearrangement processing on the first matrix to obtain a second matrix which can be continuously read in the vector register, and perform rearrangement on the second matrix according to the number N to obtain a third matrix which can be processed in parallel with N data.

[0104] In one of the embodiments, the second obtaining unit 1002 is specifically configured to perform transposition on the first matrix to obtain the second matrix, the first matrix being an H-row and W-column matrix, and the second matrix being a W-row and H-column matrix, W and H being positive integers.

[0105] In one of the embodiments, the second obtaining unit 1002 is specifically configured to sequentially take N rows from the first row to the W-th row of the second matrix, and arrange the data from the first column to the H-th column every N rows, the N*H data obtained constituting a row of data of the third matrix; and until the first row to the W / N-th row data is obtained, the third matrix is constituted, the third matrix being a W / N-row and N*H-column matrix.

[0106] In one of the embodiments, the multiplication calculating unit 1003 is specifically configured to perform multiplication calculation by using the formula , wherein,

[0107] Va={a[j], a[j],..., a[j]} represents a vector, wherein the number of a[j] is N;

[0108] Vb={b[N*j], b[N*j+1],..., b[N*j+N-1]} represents the third matrix, j is an integer from 0 to H, and i is an integer from 0 to W.

[0109] In one of the embodiments, the second obtaining unit 1002 is specifically configured to sequentially take N columns from the first column to the W-th column of the first matrix, and arrange the data from the first row to the H-th row every N columns, the N*H data obtained constituting a row of data of the third matrix; and until the first row to the W / N-th row data is obtained, the third matrix is constituted, the third matrix being a W / N-row and N*H-column matrix.

[0110] Since the details of the vector matrix multiplication acceleration unit 1000 have been described in detail in the vector matrix multiplication acceleration method section of the above embodiments with reference to the accompanying drawings, the specific process can be found in the description of the aforementioned vector matrix multiplication acceleration method embodiments, and will not be repeated here.

[0111] Electronic device

[0112] Figure 6 This diagram illustrates the internal structure of an electronic device 100 according to an embodiment of the present application, such as... Figure 6 As shown, the electronic device 100 includes a bus-connected memory 110, a scheduling unit cluster 140, and an acceleration unit cluster 150. The scheduling unit cluster 140 includes multiple scheduling units 120. The acceleration unit cluster 150 includes multiple acceleration units 130. In this embodiment, the acceleration unit 130 is a specialized processing unit designed to accelerate vector matrix multiplication tasks. It can be a processing unit specifically designed for accelerating vector matrix multiplication operations, a graphics processing unit (e.g., GPU), an image display unit (e.g., DPU), an application-specific integrated circuit (ASIC), or a field-programmable gate array (FPGA). The scheduling unit is responsible for scheduling the acceleration units and allocating the sequence of instructions to be executed to each acceleration unit. It can take various forms, such as a processor (e.g., CPU), an application-specific integrated circuit (ASIC), or a field-programmable gate array (FPGA).

[0113] Traditional processing unit architectures often allocate a significant portion of space to control and storage units, leaving insufficient space for computation units. While efficient for logic control, this approach is inefficient for large-scale parallel computing. Therefore, various specialized acceleration units have been developed to more effectively improve computational speed for different functions and domains.

[0114] The acceleration unit 130 proposed in this application is a processing unit specifically designed to accelerate vector matrix multiplication operations. During vector matrix multiplication acceleration, the first acquisition unit 1001 within the vector matrix multiplication acceleration unit 1000 of the acceleration unit 130 can obtain the number N of elements of the first matrix stored in the vector register, where N is a positive integer, based on the data length of the elements of the first matrix and the length of the vector register; the second acquisition unit 1002 can rearrange the first matrix according to the number N to obtain a third matrix capable of processing N data points in parallel; and the multiplication calculation unit 1003 can use a single instruction multiple data stream instruction to perform multiplication calculations between the vector and the third matrix.

[0115] The acceleration unit 130 in the present application can accelerate the vector matrix multiplication acceleration task under the scheduling of the scheduling unit 120. For the vector matrix multiplication acceleration unit and the vector matrix multiplication acceleration method in the present application, the foregoing has been specifically described, and will not be repeated here. The foregoing can be referred to for understanding.

[0116] Internal structure of the scheduling unit and the acceleration unit

[0117] The following will be combined Figure 7 The internal structure of the scheduling unit 120 and the acceleration unit 130, and specifically describe how the scheduling unit 120 schedules the acceleration unit 130 to work.

[0118] As shown in Figure 7 The scheduling unit 120 includes a plurality of processor cores 121 and a cache 122 shared by the plurality of processor cores 121. Each processor core 121 includes an instruction fetch unit 123, an instruction decode unit 124, an instruction issue unit 125, an instruction execution unit 126, and a memory control unit 127.

[0119] The instruction fetch unit 123 is used to carry out the instructions to be executed from the memory 110 to the instruction register (which can be a register in the register file 128 shown) through the memory control unit 127, and to receive the next fetch address or to calculate the next fetch address according to the fetch algorithm, which includes, for example, incrementing or decrementing the address according to the instruction length. Figure 2

[0120] After fetching the instruction, the scheduling unit 120 enters the instruction decoding stage, and the instruction decode unit 124 decodes the fetched instruction according to the predetermined instruction format to obtain the operand acquisition information required by the fetched instruction, so as to prepare for the operation of the instruction execution unit 126. The operand acquisition information, for example, points to an immediate number, a register or other software / hardware that can provide a source operand.

[0121] The instruction issue unit 125 is located between the instruction decode unit 124 and the instruction execution unit 126, and is used for scheduling and control of the instructions to efficiently allocate each instruction to different instruction execution units 126, so that parallel operation of multiple instructions is possible.

[0122] ​After the instruction transmitting unit 125 transmits the instruction to the instruction executing unit 126, the instruction executing unit 126 starts to execute the instruction. But if the instruction executing unit 126 judges that the instruction should be executed by the acceleration unit, it forwards the instruction to the corresponding acceleration unit for execution. For example, if the instruction is an instruction of an acceleration vector matrix multiplication acceleration task, the instruction executing unit 126 no longer executes the instruction, but sends the instruction to the acceleration unit 130 through the bus for execution by the acceleration unit 130. The scheduling unit 120 schedules the acceleration unit 130 to execute the corresponding instruction.

[0123] The acceleration unit 130 comprises a controller 131 and a vector matrix multiplication acceleration unit 1000. The controller 131 is configured to control the vector matrix multiplication acceleration unit 1000 to work. The vector matrix multiplication acceleration unit 1000 is configured to obtain, according to a data length of elements of a first matrix and a length of a vector register, a number N of elements of the first matrix stored in the vector register, N being a positive integer; perform rearrangement processing on the first matrix according to the number N to obtain a third matrix which can be processed in parallel with N data; and perform multiplication calculation on a vector and the third matrix by using a single instruction multiple data stream instruction.

[0124] System on chip

[0125] The application also provides a system on chip, which comprises the acceleration unit 130 in any of the above embodiments.

[0126] Commercial value of embodiments of the present application

[0127] In the embodiments of the application, the number N of elements of the first matrix stored in the vector register can be obtained according to the data length of the elements of the first matrix and the length of the vector register, N being a positive integer, then the rearrangement processing can be performed on the first matrix according to the number N to obtain the third matrix which can be processed in parallel with N data, and finally the multiplication calculation can be performed on the vector and the third matrix by using the single instruction multiple data stream instruction, so that the third matrix which is more convenient to process is obtained by performing the rearrangement processing on the first matrix in the embodiments of the application, the storage structure of the first matrix in the memory is optimized, and the multiplication calculation is performed on the vector and the third matrix by using the single instruction multiple data stream instruction to obtain the result of the multiplication calculation of the vector and the first matrix, so that the speed of the vector matrix multiplication calculation can be effectively improved.

[0128] It should be noted that, through enough experimental verification, the vector matrix multiplication acceleration scheme of the embodiments of the present application can reduce the operation time of the vector matrix multiplication by more than [(1-X / Y)*100%] compared with the scheme of directly multiplying the vector and the first matrix in the related art, where X is the data length of the elements of the first matrix, and Y is the length of the vector register, so the acceleration effect of the scheme on the vector matrix multiplication is very significant. For example, for the aforementioned example where the data type of the elements of the first matrix is 32-bit float data (i.e., the data length of the elements of the first matrix is 32 bits), and the length of the vector register is 128 bits, compared with the scheme of directly multiplying the vector and the first matrix in the related art, the operation time of the vector matrix multiplication can be reduced by more than 75%, and the acceleration effect on the vector matrix multiplication is very significant.

[0129] It should be understood that each of the embodiments described in the specification are described in progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the difference from other embodiments. In particular, for the method embodiments, since they are basically similar to the methods described in the device and system embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the description of other embodiments.

[0130] It should be understood that the above describes specific embodiments of the present specification. Other embodiments are within the scope of the claims. In some cases, the actions or steps recited in the claims can be performed in an order different than the order in the embodiments and still achieve the desired result. In addition, the processes depicted in the figures do not necessarily require the particular order shown or sequential order to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous or possible.

[0131] It should be understood that the elements described herein in singular form or shown in the figures only as one do not represent the limitation of the number of the elements to one. In addition, the modules or elements described or shown herein as separate can be combined into a single module or element, and the modules or elements described or shown herein as single can be split into multiple modules or elements.

[0132] It should also be understood that the terms and expressions used herein are used only to describe and not to limit one or more embodiments of the present specification. The use of such terms and expressions does not mean that any illustration and description (or part thereof) is excluded from the equivalent features. It should be recognized that various modifications can exist and should be included in the scope of the claims. Other modifications, changes and replacements can also exist. Accordingly, the claims should be considered to cover all such equivalents.

Claims

1. A method for accelerating vector-matrix multiplication, applied to a fifth generation reduced instruction set computer (RISC-V) processor, the method comprising: obtaining a number N of elements of a first matrix stored in a vector register according to a data length of the elements of the first matrix and a length of the vector register, N being a positive integer; rearranging the first matrix according to the number N to obtain a third matrix in which N data can be processed in parallel; performing multiplication calculation on a vector and the third matrix by using a single instruction multiple data stream (SIMD) instruction; the rearranging the first matrix according to the number N to obtain the third matrix in which N data can be processed in parallel comprises: transposing the first matrix to obtain a second matrix, the first matrix being an H-row and W-column matrix, and the second matrix being a W-row and H-column matrix, H and W being positive integers; sequentially taking N rows from a first row to a W-th row of the second matrix, and arranging data from a first column to an H-th column in each N rows, N*H data obtained thereby constituting a row of data of the third matrix; and until a first row to a W / N-th row of data is obtained, a W / N-row and N*H-column third matrix is constituted; or sequentially taking N columns from a first column to a W-th column of the first matrix, and arranging data from a first row to an H-th row in each N columns, N*H data obtained thereby constituting a row of data of the third matrix; and until a first row to a W / N-th row of data is obtained, a W / N-row and N*H-column third matrix is constituted. the obtaining the number N of elements of the first matrix stored in the vector register according to the data length of the elements of the first matrix and the length of the vector register comprises:

2. The method of claim 1, wherein, dividing the length of the vector register by the data length of the elements of the first matrix to obtain the number N of elements of the first matrix stored in the vector register. the performing multiplication calculation on the vector and the third matrix by using the SIMD instruction comprises:

3. The method of claim 1, wherein, Va={a[j],a[j],...a[j]}, representing a vector, wherein the number of a[j] is N; The multiplication is performed using the formula wherein, Vb={b[N*j],b[N*j+1],...b[N*j+N-1]}, representing the third matrix, j being an integer from 0 to H, and i being an integer from 0 to W. 4.A vector-matrix multiplication acceleration unit, applied to a fifth generation reduced instruction set computer (RISC-V) processor, the vector-matrix multiplication acceleration unit comprising: a first obtaining unit, configured to obtain a number N of elements of a first matrix stored in a vector register according to a data length of the elements of the first matrix and a length of the vector register, N being a positive integer; a second obtaining unit, configured to rearrange the first matrix according to the number N to obtain a third matrix in which N data can be processed in parallel; a multiplication calculation unit, configured to perform multiplication calculation on a vector and the third matrix by using a single instruction multiple data stream (SIMD) instruction; the rearranging the first matrix according to the number N to obtain the third matrix in which N data can be processed in parallel comprises: ​ transposing the first matrix to obtain a second matrix, the first matrix being an H row and W column matrix, the second matrix being a W row and H column matrix, W and H being positive integers; sequentially taking N rows from a first row to a Wth row of the second matrix, and arranging data from a first column to an Hth column of each N rows, N*H data obtained thereby constituting a row of data of a third matrix; until a first row to a W / Nth row of data is obtained, constituting the third matrix, the third matrix being a W / N row and N*H column matrix; or, sequentially taking N columns from a first column to a Wth column of the first matrix, and arranging data from a first row to an Hth row of each N columns, N*H data obtained thereby constituting a row of data of a third matrix; until a first row to a W / Nth row of data is obtained, constituting the third matrix, the third matrix being a W / N row and N*H column matrix.

5. An acceleration unit, comprising: the vector matrix multiplication acceleration unit according to claim 4; a controller configured to control the vector matrix multiplication acceleration unit to work.

6. An electronic device, comprising: the acceleration unit according to claim 5; a scheduling unit configured to schedule the acceleration unit to perform a vector matrix multiplication acceleration task.

7. A system on chip, comprising the acceleration unit according to claim 5.

Images