Method and apparatus for executing general matrix vector multiplication, and computer device

Abstract

The embodiments disclosed in the present application belong to the technical field of computers, and particularly relate to a method and apparatus for executing general matrix vector multiplication, and a computer device. The method comprises: receiving a computing instruction, wherein the computing instruction comprises X groups of general matrix vector multiplication operations, each group of general matrix vector multiplication operations comprises a first matrix and a first vector, the first matrix comprises M rows and N columns, and the first vector comprises N elements; generating N groups of second vectors on the basis of X first matrices, wherein each group of second vectors comprises M second vectors, and the M second vectors in an ith group are respectively composed of M rows of X elements in an ith column in X matrices; generating N third vectors on the basis of X first vectors, wherein an ith third vector is composed of elements at ith positions in the X first vectors; and executing a vector multiplication operation on the M second vectors in the ith group and the ith third vector, so as to obtain operation results of the X groups of general matrix vector multiplication operations. By using the present application, the efficiency of computing general matrix vector multiplication can be improved.

Description

Method, device and computer device for performing matrix vector multiplication

[0001] The present application claims priority to the Chinese patent application No. 202410818682.0, filed on June 21, 2024, and entitled "Method, device and computer device for performing matrix vector multiplication", the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the technical field of computer, and in particular, to a method, device and computer device for performing matrix vector multiplication. BACKGROUND

[0003] General Matrix Vector Multiplication (GEMV) refers to the product of a matrix and a vector, which can be performed by a vector operation unit.

[0004] When performing the matrix vector multiplication, the vector operation unit can multiply each row of elements in the matrix with the elements in the vector respectively, and then add the plurality of vectors obtained by the multiplication to obtain the calculation result.

[0005] In the process of performing the matrix vector multiplication by the vector operation unit, if the size of the matrix is small, the number of elements to be multiplied or added is small, which causes the waste of the calculation performance of the vector operation unit. SUMMARY

[0006] The embodiments of the present application provide a method, device and computer device for performing matrix vector multiplication, which can improve the calculation efficiency of the matrix vector multiplication, and the corresponding technical solutions are as follows:

[0007] In a first aspect, a method for performing matrix vector multiplication is provided, which is applied to a processor including a vector operation unit, and the method includes:

[0008] receiving a calculation instruction sent by an application program, the calculation instruction indicating X groups of matrix vector multiplication operations, each group of matrix vector multiplication operations including a first matrix and a first vector, wherein the first matrix includes M rows and N columns, the first vector includes N elements, and X is greater than N; generating N groups of second vectors according to the X first matrices, each group of second vectors including M second vectors, wherein the M second vectors in the i-th group are respectively composed of X elements in the i-th column of M rows in the X matrices; generating N third vectors according to the X first vectors, wherein the i-th third vector is composed of elements at the i-th position in the X first vectors; performing vector multiplication operation on the M second vectors in the i-th group and the i-th third vector by using the vector operation unit; and determining the operation result of the X groups of matrix vector multiplication operations according to the operation result of the vector multiplication operation.

[0009] In the scheme shown in the present application, X M-row N-column first matrices can be converted into M*N second vectors, and X first vectors can be converted into N third vectors. Since X is greater than N, the number of converted second vectors (M*N) is less than the number of row vectors included in the X first matrices (M*X), and the number of converted third vectors (N) is less than the number of X first vectors (X). Therefore, by performing vector operations of the second vectors and the third vectors, the operation results of X sets of matrix-vector multiplication operations are obtained, which can improve the execution efficiency of the matrix-vector multiplication operations, compared to some embodiments in which the operation results of X sets of matrix-vector multiplication operations are obtained by performing vector operations of the row vectors in the first matrices and the first vectors, which can require a smaller number of vectors for operation.

[0010] In an implementable manner, performing vector multiplication operations on the M second vectors of the i-th group and the i-th third vector by using a vector operation unit includes: performing vector bit-by-bit multiplication operations on the j-th second vector of the i-th group and the i-th third vector by using the vector operation unit to obtain a j-th fourth vector corresponding to the i-th group, and performing vector bit-by-bit addition operations on the j-th fourth vector of each group to obtain a j-th fifth vector. Determining the operation results of the X sets of matrix-vector multiplication operations according to the operation results of the vector multiplication operations includes: determining the i-th element in each fifth vector as the operation result of the matrix-vector multiplication of the i-th group.

[0011] In the scheme shown in the present application, the fourth vectors can be obtained by performing vector bit-by-bit multiplication on each group of second vectors and corresponding third vectors, and the M fifth vectors can be obtained by performing vector bit-by-bit addition on the obtained fourth vectors. Among the M fifth vectors, the elements corresponding to the same positions are the operation results of a group of matrix-vector multiplication operations. In this way, the operation results of the X sets of matrix-vector multiplication operations can be obtained by reading the fifth vectors. It can be seen that in the scheme shown in the present application, the operation results of the X sets of matrix-vector multiplication operations can be obtained by performing a smaller number of vector operations on the second vectors and the corresponding third vectors, which can improve the efficiency of the matrix-vector multiplication operations.

[0012] In an implementable manner, the method further includes: in response to a calculation instruction sent by an application program, calling an intrinsic instruction package of a processor, and the intrinsic instruction package includes at least one vector loading instruction and at least one vector multiplication accumulation instruction. The at least one vector loading instruction is used to generate N groups of second vectors from X first matrices, and / or is used to generate N third vectors from X first vectors. The at least one vector multiplication accumulation instruction is used to perform vector multiplication operations on the M second vectors of the i-th group and the i-th third vector by using a vector operation unit.

[0013] In the scheme shown in the present application, the intrinsic instructions corresponding to the steps included in the method of performing matrix vector multiplication can be packaged into an intrinsic instruction package and added to the instruction set of the processor. In this way, the skilled person can directly implement the method of performing matrix vector multiplication provided in the present application by calling the intrinsic instruction package, which can avoid repeated programming by the skilled person and improve the development efficiency of the skilled person.

[0014] In an implementable manner, the vector operation unit included in the processor generates N groups of second vectors from X first matrices, including: starting from the rth element in the X first matrices stored in the memory, every t elements are stored in the s th vector register according to the vector loading instruction, where the initial values of r and s are 1, and t is equal to MxN-1. After storing X elements in the s th vector register, the values of r and s are both increased by 1, and the process of starting from the rth element in the X first matrices stored in the memory, every t elements are stored in the s th vector register according to the vector loading instruction is performed until the value of r or s is equal to MxN, where the vector stored in each vector register is a second vector.

[0015] In an implementable manner, the vector operation unit included in the processor generates N groups of second vectors from X first matrices, including: starting from the rth element in the X first matrices stored in the memory, every t elements are stored in the s th vector register according to the vector loading instruction, where the initial values of r and s are 1, and t is equal to MxN-1. After storing X elements in the s th vector register, the values of r and s are both increased by 1, and the process of starting from the rth element in the X first matrices stored in the memory, every t elements are stored in the s th vector register according to the vector loading instruction is performed until the value of r or s is equal to MxN, where the vector stored in each vector register is a second vector.

[0016] In an implementable manner, the vector operation unit performs vector bit-by-bit multiplication operation on the jth second vector in the ith group of second vectors and the ith third vector to obtain the jth fourth vector corresponding to the ith group, and performs vector bit-by-bit addition operation on the jth fourth vector of each group to obtain the jth fifth vector, including: according to the vector multiplication accumulation instruction, controlling the vector operation unit to perform vector bit-by-bit multiplication operation on the jth second vector in the ith group of second vectors and the ith third vector to obtain the jth fourth vector corresponding to the ith group, and performing vector bit-by-bit addition operation on the jth fourth vector of each group to obtain the jth fifth vector.

[0017] In an implementable manner, the vector multiplication operation of the M second vectors of the i-th group and the i-th third vector is performed by using the vector operation unit, including: compiling target program code to obtain a plurality of vector intrinsic basic instructions, the target program code being used to perform the vector multiplication operation of the M second vectors of the i-th group and the i-th third vector by using the vector operation unit. The plurality of vector intrinsic basic instructions are sequentially executed to perform the vector multiplication accumulation operation of the M second vectors of the i-th group and the i-th third vector by using the vector operation unit.

[0018] In the scheme shown in the present application, the program code for performing the matrix vector multiplication can be compiled into a plurality of vector intrinsic basic instructions by the vector automatic capability of the compiler, so that the method for performing the matrix vector multiplication provided in the present application can be implemented by executing the plurality of vector intrinsic basic instructions, that is, the matrix vector multiplication can be implemented by using larger size and fewer number of second vectors and third vectors, and the acceleration of the matrix vector multiplication is realized.

[0019] In a second aspect, a device for performing matrix vector multiplication is provided, and the device comprises:

[0020] The receiving module is configured to receive a calculation instruction sent by an application program, the calculation instruction comprising X groups of matrix vector multiplication operations, each group of matrix vector multiplication operations comprising a first matrix and a first vector, wherein the first matrix comprises M rows and N columns, the first vector comprises N elements, and X is greater than N.

[0021] The generating module is configured to generate N groups of second vectors according to the X first matrices, each group of second vectors comprising M second vectors, wherein the M second vectors in the i-th group are respectively composed of X elements of M rows and the i-th column in the X matrices; and generate N third vectors according to the X first vectors, wherein the i-th third vector is composed of elements at the i-th position in the X first vectors.

[0022] The operation module is configured to perform a vector multiplication operation of the M second vectors of the i-th group and the i-th third vector by using the vector operation unit.

[0023] The determining module is configured to determine an operation result of the X groups of matrix vector multiplication operations according to an operation result of the vector multiplication operation.

[0024] In an implementable manner, the operation module is configured to:

[0025] perform, according to the vector operation unit, a vector bit-by-bit multiplication operation of the j-th second vector in the i-th group of second vectors and the i-th third vector to obtain a j-th fourth vector corresponding to the i-th group, and perform a vector bit-by-bit addition operation on the j-th fourth vector of each group to obtain a j-th fifth vector.

[0026] The determining module is configured to determine the i-th element in each fifth vector as an operation result of matrix vector multiplication of the i-th group.

[0027] In an implementable manner, the apparatus further includes a calling module configured to, in response to the calculation instruction sent by the application program, call an intrinsic instruction package of the processor, the intrinsic instruction package including at least one vector loading instruction and at least one vector multiply-accumulate instruction. The at least one vector loading instruction is configured to implement generation of N groups of second vectors from X first matrices and / or generation of N third vectors from X first vectors, and the at least one vector multiply-accumulate instruction is configured to implement vector multiplication operation of the i-th group of M second vectors and the i-th third vector by using the vector operation unit.

[0028] In an implementable manner, the generating module is configured to, starting from the r-th element in the X first matrices stored in the memory, store, at intervals of t elements, elements after the interval of t elements to the s-th vector register according to the vector loading instruction, where the initial values of r and s are 1, and t is equal to M×N-1. After storing X elements in the s-th vector register, the values of r and s are both increased by 1, and the operation of storing, at intervals of t elements, elements after the interval of t elements to the s-th vector register, starting from the r-th element in the X first matrices stored in the memory, is performed until the value of r or s is equal to M×N, where the vector stored in each vector register is a second vector.

[0029] In an implementable manner, the generating module is configured to, starting from the u-th element in the X first vectors stored in the memory, store, at intervals of w elements, elements after the interval of w elements to the v-th vector register according to the vector loading instruction, where the initial values of u and v are 1, and w is equal to N-1. After storing X elements in the s-th vector register, the values of u and v are both increased by 1, and the operation of storing, at intervals of w elements, elements after the interval of w elements to the v-th vector register, starting from the u-th element in the X first vectors stored in the memory, is performed until the value of u or v is equal to N, where the vector stored in each vector register is a third vector.

[0030] In an implementable manner, the operation module is configured to, according to the vector multiply-accumulate instruction, control the vector operation unit to perform vector bit-by-bit multiplication operation on the j-th second vector in the i-th group of second vectors and the i-th third vector, and perform vector bit-by-bit addition operation on a fourth vector obtained by performing the vector bit-by-bit multiplication operation, to obtain the i-th fifth vector.

[0031] In an implementable manner, the operation module is configured to: compile the target program code to obtain a plurality of vector intrinsic basic instructions, the target program code being configured to perform vector multiplication operation on the M second vectors in the ith group and the ith third vector by using the vector operation unit; and sequentially execute the plurality of vector intrinsic basic instructions to perform vector multiplication accumulation operation on the M second vectors in the ith group and the ith third vector by using the vector operation unit.

[0032] In a third aspect, a computer device is provided, which includes a processor and a memory, the processor including a vector operation unit and a vector register, and the memory storing instructions, the processor being configured to execute the instructions stored in the memory to enable the computer device to perform the method according to the first aspect.

[0033] In a fourth aspect, a computer program product is provided, which includes instructions, the instructions, when executed by a processor of a computer device, enabling the processor in the computer device to perform the method according to the first aspect.

[0034] In a fifth aspect, a computer readable storage medium is provided, which includes computer program instructions, the instructions, when executed by a processor of a computer device, enabling the processor to perform the method according to the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0035] FIG. 1 is a schematic diagram of a conventional matrix vector multiplication;

[0036] FIG. 2 is a schematic diagram of a structure of a computer device according to an embodiment of the present application;

[0037] FIG. 3 is a flowchart of a method for performing matrix vector multiplication according to an embodiment of the present application;

[0038] FIG. 4 is a schematic diagram of a method for performing matrix vector multiplication according to an embodiment of the present application;

[0039] FIG. 5 is a flowchart of a method for performing matrix vector multiplication according to an embodiment of the present application;

[0040] FIG. 6 is a flowchart of a method for performing matrix vector multiplication according to an embodiment of the present application;

[0041] FIG. 7 is a schematic diagram of an apparatus for performing matrix vector multiplication according to an embodiment of the present application. DETAILED DESCRIPTION

[0042] To make the objectives, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0043] Matrix vector multiplication refers to a multiplication operation of matrix and vector, which has a wide range of applications in high performance computing (HPC), artificial intelligence (AI) and other fields. For example, in the field of HPC, the calculation of stress matrix, in the field of AI, the multiplication of input vector and weight matrix of neural network to obtain the activation data of neural network, etc.

[0044] FIG. 1 is a schematic diagram of a conventional matrix vector multiplication provided by an embodiment of the present application. As shown in FIG. 1, the matrix vector multiplication of matrix A and vector X can be represented as Y T = AX T , wherein X T is a transposed vector of vector X, Y is a result vector of the matrix vector multiplication of matrix A and vector X, and Y T is a transposed vector of the result vector.

[0045] In FIG. 1, four groups of matrix vector multiplication operations are shown. Taking the first group of matrix vector multiplication operations as an example, the first group of matrix vector multiplication operations includes the matrix vector multiplication of matrix and vector X1 = [w1 w2 w3]. The matrix vector multiplication of A1 and X1 can be converted into the vector inner product of the first row vector A 11 = [a1 a2 a3] included in A1 and column vector , the vector inner product of the second row vector A 12 = [a4 a5 a6] and column vector , and the vector inner product of the third row vector A 12 = [a7 a8 a9] and column vector . Wherein, the vector inner product result of row vector A 11 and column vector X1 T , i.e. the calculation result of a1w1+a2w2+a3w3, is the element value of the first element in column vector Y1 T ; the vector inner product result of row vector A 12 and column vector X1 T , i.e. the calculation result of a4w1+a5w2+a6w3, is the element value of the second element in column vector Y1 T ; the vector inner product result of row vector A 13 and column vector X1 T , i.e. the calculation result of a1w7+a2w8+a3w9, is the element value of the third element in column vector Y1 Tthe element value of the third element. It can be seen that, in order to perform the matrix vector multiplication of A1 and X1 once, at least 3 vector inner products need to be performed, each of which includes a vector pair bit multiplication operation and accumulation of the element in the vector inner product obtained by the vector pair bit multiplication operation. For the 4 groups of matrix vector multiplications shown in FIG. 1, including the matrix vector multiplication of A2 and X1 = [w1 w2 w3], the matrix vector multiplication of A3 and X2 = [x1 x2 x3], the matrix vector multiplication of A4 and X3 = [y1 y2 y3], the matrix vector multiplication of A5 and X4 = [z1 z2 z3]. Thus, for the 4 groups of matrix vector multiplications, at least 12 vector inner products need to be performed, that is, 12 vector pair bit multiplications and accumulations are needed. For the matrix vector multiplication of an X group of matrices with a size of M x N and a vector with a size of N, at least M x X vector inner products need to be performed, that is, M x X vector pair bit multiplications and accumulations are needed.

[0046] Scalable Vector Extension (SVE) is an instruction set provided for a vector operation unit to perform various vector operations, such as vector addition, vector subtraction, vector multiplication, vector multiplication and addition, and the like. The SVE instruction can instruct the vector operation unit in the processor to implement the above vector inner product operation.

[0047] Before the vector operation unit performs the vector inner product of two vectors, the two vectors to be operated can be stored in the vector register first. Then the vector operation unit performs the inner product operation on the two vectors stored in the vector register. In this process, the vector operation unit can perform parallel calculation on multiple elements in the two vectors when performing vector operation on the two vectors. Therefore, when the vector operation unit performs the matrix vector multiplication, if the size of the matrix to be operated is small, the number of elements in the two vectors for which the vector inner product is performed may be too small to fill the vector register. In this case, on the one hand, it will cause waste of the calculation performance of the vector operation unit. On the other hand, the number of vector operations performed each time is small, and the number of vector operations performed by the vector operation unit to implement the matrix vector multiplication is large, which also leads to low efficiency of the matrix vector multiplication. In addition, the more groups of small size matrices and vectors for which the matrix vector multiplication is performed, the lower the execution efficiency. The low efficiency of the matrix vector multiplication performed by the vector operation unit also leads to low efficiency of the processing involving matrix vector multiplication in the fields of HPC, AI, and the like.

[0048] The embodiment of the present application provides a method for performing matrix vector multiplication, in which the matrix vector multiplication of multiple groups of small-size matrices and vectors can be converted into large-size vector multiplication accumulation operation, the number of elements of each vector operation can be increased in the process of implementing the matrix vector multiplication by the vector operation unit, the number of times of vector operation performed by the vector operation unit can be reduced, the utilization rate of the vector operation unit can be improved, the execution efficiency of the matrix vector multiplication is further improved, and the processing related to the matrix vector multiplication operation in the field of HPC, AI and the like is further accelerated.

[0049] Fig. 2 is a schematic diagram of a computer device structure of the method for performing matrix vector multiplication provided by the embodiment of the present application. As shown in Fig. 2, the computer device 200 includes a bus 202, a processor 204, a memory 206 and a communication interface 208. The processor 204, the memory 206 and the communication interface 208 communicate through the bus 202. The computer device 200 can be a server or a terminal device. It should be understood that the number of processors and memories in the computer device 200 is not limited by the present application.

[0050] The bus 202 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one line is shown in Fig. 2, but it does not mean that there is only one bus or only one type of bus. The bus 202 can include a path for transmitting information between various components (for example, the memory 206, the processor 204, the communication interface 208) of the computer device 200.

[0051] The processor 204 can include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP) and the like. In an example, the processor 204 includes at least a vector register and a vector operation unit (such as an SVE register and an SVE operation unit), wherein the vector register can be used to implement the storage of a vector, and the vector operation unit can implement the vector operation on the vector stored in the vector register.

[0052] The memory 206 can include volatile memory, such as random access memory (RAM) including a cache area for the temporary storage of data. The memory 206 also can include non-volatile memory, such as read only memory (ROM), floppy, flexible, RAM, or solid state drive (SSD) such as flash memory.

[0053] The executable program code stored in the memory 206 is executed by the processor 204 to implement the method for performing matrix vector multiplication provided by the embodiments of the present application. For example, the processor 204 receives a calculation instruction sent by an application program, the calculation instruction including X sets of matrix vector multiplication operations. N sets of second vectors are generated according to X first matrices. N third vectors are generated according to X first vectors. Vector multiplication operations are performed on each set of M second vectors and corresponding third vectors by using a vector operation unit. The operation results of the X sets of matrix vector multiplication operations are determined according to the operation results of the vector multiplication operations.

[0054] The communication interface 208 uses a transceiver module such as, but not limited to, a network interface card, a transceiver, and the like to implement communication between the computer device 200 and other devices or communication networks.

[0055] FIG. 3 is a flowchart of a method for performing matrix vector multiplication provided by the embodiments of the present application. The method can be performed by the processor 204 included in the computer device 200 described above. As shown in FIG. 3, the method includes:

[0056] In step 301, a calculation instruction sent by an application program is received, the calculation instruction including X sets of matrix vector multiplication operations, each set of matrix vector multiplication operations including a first matrix and a first vector, wherein the first matrix includes M rows and N columns, the first vector includes N elements, and X is greater than N.

[0057] The application program can be any application program related to matrix vector multiplication, for example, the application program can be a high performance computing (HPC) application. Taking computational fluid dynamics as an example, the matrix to be operated on by the matrix vector multiplication operation can be a stiffness matrix, the vector to be operated on by the matrix vector multiplication operation can be a strain vector, and the operation result of the corresponding matrix vector multiplication can be a stress matrix.

[0058] In the embodiments of the present application, the matrix-vector multiplication of X groups of small-size (M rows and N columns) matrices and vectors can be converted into N groups of vector multiplication accumulation operations of large size (including X elements). In order to ensure the execution efficiency of the matrix-vector multiplication, the value of X can be greater than N. That is, the number of groups X of the first matrix and the first vector performing the matrix-vector multiplication is greater than the number of elements N of a row vector in the first matrix.

[0059] In an example, the value of X can be less than or equal to the number P of elements in a matrix (or a vector) that can be stored in a vector register at most, wherein the number P is equal to the ratio of the maximum bit width of the vector register and the bit width occupied by an element. If the total number of matrix-vector groups that need to perform the matrix-vector multiplication is greater than the number P, then the matrix-vector groups can be calculated in batches, wherein the number X of matrix-vector groups in each batch calculation can be less than or equal to the number P.

[0060] The operation process of the X groups of matrix-vector multiplication will be introduced below in combination with steps 302-305 and FIG. 4.

[0061] In step 302, N groups of second vectors are generated according to the X first matrices, and each group of the second vectors includes M second vectors, wherein the M second vectors in the i-th group are respectively composed of X elements of the i-th column of M rows in the X matrices.

[0062] FIG. 4 is a schematic diagram of a method for performing matrix-vector multiplication provided in the embodiments of the present application. The 4 groups of matrix-vector multiplication shown in FIG. 4 include first matrices A1, A2, A3 and A4 respectively. In step 302, X first matrices of size M×N can be converted into M×N second vectors of size X. Each first vector is composed of elements with the same row and column position in the X matrices, such as the elements in the first row and the first column of A1, A2, A3 and A4 in FIG. 4, which can form a second vector S1=[a1, b1, c1, d1]. The second vectors composed of elements with the same column position can be classified into a group of vectors, such as the second vectors S1, S2 and S3 composed of the elements in the first row and the first column of the four matrices in FIG. 4, which are the same group of second vectors. Thus, the number of second vectors included in each group of second vectors is equal to the number M of rows of the first matrix.

[0063] In step 303, N third vectors are generated according to the X first vectors, wherein the i-th third vector is composed of elements in the i-th position of the X first vectors.

[0064] In step 303, X first vectors of size N can be converted into N third vectors of size X. Each third vector is composed of elements of the same position in the X first vectors. For example, in FIG. 4, the elements of the first position in the first vectors X1, X2, X3, X4 can form a third vector T1 = [w1, x1, y1, z1].

[0065] In step 304, the vector multiplication operation can be performed on the M second vectors in the i-th group and the i-th third vector by the vector operation unit.

[0066] After obtaining the N groups of second vectors and the N third vectors, the vector operation can be performed on each group of second vectors and third vectors to obtain the operation results of the X groups of matrix vector multiplication operations.

[0067] In an example, the vector bit-wise multiplication operation can be performed on the j-th second vector in the i-th group of second vectors and the i-th third vector by the vector operation unit to obtain the j-th fourth vector corresponding to the i-th group, and the vector bit-wise addition operation can be performed on the j-th fourth vector in each group to obtain the j-th fifth vector.

[0068] The i-th group of second vectors can be a group of second vectors generated by the elements of the i-th column in each first matrix, and the j-th second vector in the i-th group of second vectors can be a second vector generated by the elements of the i-th row and the i-th column in each first matrix. The i-th third vector can be a third vector generated by the i-th position in the first vector.

[0069] In implementation, the vector bit-wise multiplication operation can be performed on the M second vectors included in each group and the corresponding third vector by the matrix operation unit to obtain N groups of fourth vectors, and each group of fourth vectors includes M fourth vectors. After the fourth vectors are calculated, the vector bit-wise addition operation can be performed on the fourth vectors with the same position in each group to obtain M fifth vectors of size X, respectively. The position of the fourth vector in the vector group is the row number of the first matrix in which the element of the second vector used to calculate the fourth vector is located.

[0070] As shown in FIG. 4, after obtaining each group of fourth vectors, the first fourth vector F1, F2, F3 included in each group of fourth vectors can be subjected to vector pair bit addition operation to obtain a fifth vector F4, and the calculation process of the fifth vector F5, F6 is similar, and the embodiments of the present application will not be described again. Wherein, F4 = [a1w1+a2w2+a3w3, b1x1+b2x2+b3x3, c1y1+c2y2+c3y3, d1z1+d2z2+d3z3], F5 = [a4w1+a5w2+a6w3, b4x1+b5x2+b6x3, c4y1+c5y2+c6y3, d4z1+d5z2+d6z3], F5 = [a7w1+a8w2+a9w3, b7x1+b8x2+b9x3, c7y1+c8y2+c9y3, d7z1+d8z2+d9z3].

[0071] In step 305, the operation result of the X group matrix vector multiplication operation is determined according to the operation result of the vector multiplication operation.

[0072] The operation result of the X group matrix vector multiplication operation is included in the operation result of the vector multiplication operation in step 304. In an example, the element corresponding to the i-th position in the plurality of fifth vectors is the operation result of the i-th group matrix vector multiplication operation.

[0073] It can be found in combination with FIG. 1 and FIG. 4 that the element in the first position in the plurality of fifth vectors shown in FIG. 4 is the operation result corresponding to the matrix vector multiplication of the first group in FIG. 1 and FIG. 4, the element in the second position in the plurality of fifth vectors is the operation result corresponding to the matrix vector multiplication of the second group in FIG. 1 and FIG. 4, the element in the third position in the plurality of fifth vectors is the operation result corresponding to the matrix vector multiplication of the third group in FIG. 1 and FIG. 4, and the element in the fourth position in the plurality of fifth vectors is the operation result corresponding to the matrix vector multiplication of the fourth group in FIG. 1 and FIG. 4. Therefore, after M fifth vectors are calculated in step 304, the operation result of each group matrix vector multiplication operation can be obtained by reading the elements in the same position of the M fifth vectors.

[0074] In the method for performing matrix-vector multiplication provided in the embodiments of the present application, when performing matrix-vector multiplication on X groups of small-size (MxN) matrices and vectors, the X M-row N-column first matrices can be first converted into MxN second vectors with a size of X, and the X first vectors can be converted into N third vectors with a size of X. Then, vector multiplication and accumulation operations are performed on the second vectors and the third vectors to obtain X groups of operation results of matrix-vector multiplication respectively corresponding to the second vectors and the third vectors. Since X is greater than N, the number (MxN) of the second vectors is less than the number (MxX) of row vectors in the X first matrices, the number (N) of the third vectors is less than the number (X) of the X first vectors, and the number (X) of elements included in the second vectors and the third vectors is greater than the number (N) of elements of the row vectors in the first matrices and the first vectors. In this way, the original MxX times of vector bit-by-bit multiplication and accumulation of elements in the vectors can be converted into MxN times of vector bit-by-bit multiplication and accumulation, thereby improving the utilization rate of the vector operation unit, improving the execution efficiency of matrix-vector multiplication, and further realizing acceleration of processing involving matrix-vector multiplication in the fields of HPC, AI, and the like.

[0075] The embodiments of the present application provide a method for implementing matrix-vector multiplication through intrinsic instructions, that is, the intrinsic instructions required by the processing of steps 302 to 304 can be packaged into an intrinsic instruction package. The processing of steps 302 to 304 can be implemented by calling the intrinsic instruction package. The intrinsic instruction package can include a plurality of vector loading instructions and a plurality of vector multiplication and accumulation instructions. The plurality of vector loading instructions can be used to implement the conversion of small-size matrices and vectors into large-size vectors in steps 302 and 303. The vector loading instructions can be svdup_f64, svldl_f64, etc. The plurality of vector multiplication and accumulation instructions can implement the vector multiplication and accumulation operations on large-size vectors involved in step 304. The vector multiplication and accumulation instructions can be svmla_x instructions.

[0076] The processing of step 302 can include:

[0077] Step 3021, starting from the rth element in the X first matrices stored in the memory, every t elements are stored in the st vector register according to the vector loading instruction, where the initial values of r and s are 1, and t is equal to MxN-1.

[0078] In implementation, the elements included in the X first matrices can be stored continuously in the memory. For example, when the memory stores the matrices in row-major order, the elements of each row of the X first matrices can be stored in the memory in sequence. In this way, the vector loading instruction can be used to read one element every t elements starting from the first element of the X first matrices, and load the read elements into a vector register. In this way, after the X elements are read into the vector register, a second vector composed of the elements in the first row and the first column of each first matrix can be obtained.

[0079] Step 3022, after storing the X elements in the s-th vector register, the values of r and s are both increased by 1, and the process of step 3021 is executed until the value of r or s is equal to MxN, wherein the vector stored in each vector register is a second vector.

[0080] After reading the X elements each time, the values of r and s can be increased by 1, and the process of step 3021 is executed again. After executing MxN times, the elements in the X first matrices can be stored in the form of second vectors into MxN vector registers, thereby realizing the conversion of the X first matrices into MxN second vectors.

[0081] The process of step 303 can include:

[0082] Step 3031, starting from the u-th element of the X first vectors stored in the memory, every w elements, the element after the interval of w elements is stored into the v-th vector register according to the vector loading instruction, wherein the initial values of u and v are 1, and w is equal to N-1.

[0083] In implementation, the elements included in the X first vectors can be stored continuously in the memory. In this way, the vector loading instruction can be used to read one element every w elements starting from the first element of the X first vectors, and load the read elements into a vector register. In this way, after the X elements are read into the vector register, a third vector composed of the first element of each first vector can be obtained.

[0084] Step 3032, after storing the X elements in the s-th vector register, the values of u and v are both increased by 1, and the process of step 3032 is executed until the value of u or v is equal to N, wherein the vector stored in each vector register is a third vector.

[0085] After reading the X elements each time, the values of u and v can be increased by 1, and the process of step 3031 is executed again. After executing N times, the elements in the X first vectors can be stored in the form of third vectors into N vector registers, thereby realizing the conversion of the X first matrices into N second vectors.

[0086] The processing of step 304 can include:

[0087] According to the vector multiply accumulate instruction, the control vector operation unit performs vector bit-wise multiplication operation on the jth second vector in the ith group of second vectors and the ith third vector, and performs vector bit-wise addition operation on the fourth vector obtained by performing the vector bit-wise multiplication operation, to obtain the ith fifth vector.

[0088] In implementation, the vector bit-wise multiplication operation can be performed on the second vector and the third vector at the same position in each second vector group by the vector multiply accumulate instruction in sequence, and then the vector bit-wise addition operation is performed on the fourth vector obtained each time to obtain the corresponding fifth vector. In this way, the matrix vector multiplication of the matrix with size MxN in the X group and the vector can be converted into MxN times of vector multiply accumulate, and the execution efficiency of the matrix vector multiplication can be improved.

[0089] FIG. 5 is a flowchart of a method for implementing matrix vector multiplication by intrinsic instruction according to an embodiment of the present application. In the method shown in FIG. 5, the processing of step 304 described above can be implemented by intrinsic instruction, as shown in FIG. 5, the method includes:

[0090] Step 501, read the dimensions M, N and the number X of the first matrix.

[0091] Step 502, apply 2 groups of SVE registers AX and KX with length X.

[0092] Step 503, store the first matrix data in the SVE register AX every MxN-1 elements, a total of MxN times, and store the elements of the first vector in the KX every N-1 elements, a total of N times.

[0093] Step 504, perform vector multiply accumulate operation on the SVE register AX and KX by using the svmla instruction.

[0094] Step 505, perform N times of multiply-add svmla.

[0095] Step 506, apply 1 SVE register with size X to store the values after the multiply-add operation, and assign the value of the register to the result vector.

[0096] Step 507, determine whether the execution times Iter of step 506 is equal to M, if Iter is equal to M, output the operation result. If less than M, go to step 505.

[0097] In the embodiments of the present application, the intrinsic instructions corresponding to the steps included in the method of performing matrix vector multiplication can be packaged into an intrinsic instruction package and added to the instruction set of the processor. In this way, the skilled person can directly implement the method of performing matrix vector multiplication provided by the present application by calling the intrinsic instruction package, which can avoid repeated programming by the skilled person and improve the development efficiency of the skilled person.

[0098] The embodiments of the present application provide a method of implementing matrix vector multiplication in a vector automated manner, i.e., using the automatic vectorization capability of the compiler, converting the program code for performing matrix vector multiplication provided by the present application into a plurality of vector intrisic basic instructions, and implementing the method of performing matrix vector multiplication provided by the present application by executing the plurality of vector intrisic basic instructions.

[0099] In implementation, the compiler can compile the program code for implementing step 304, and use the automatic vectorization capability of the compiler to compile the program code into a plurality of vector intrisic basic instructions for sequential execution. The processor can implement the calculation method shown in step 304 by executing the plurality of vector intrisic basic instructions.

[0100] FIG. 6 is a flowchart of a method of implementing matrix vector multiplication in a vector automated manner provided by an embodiment of the present application. As shown in FIG. 6, the method includes the following steps:

[0101] Referring to FIG. 6, another method of matrix operation provided by an embodiment of the present application is shown. The method can include the following steps:

[0102] Step 601: Read the dimensions M, N and the number X of the first matrix.

[0103] Step 602: Apply 2 groups of SVE registers AX and KX of length X.

[0104] Step 603: Store the first matrix data into the SVE register AX every MxN-1 elements, and execute MxN times. Store the elements of the first vector into the KX every N-1 elements, and execute N times.

[0105] Step 604: Perform matrix vector multiplication operation using automatic vectorization.

[0106] In the embodiments of the present application, the program code for performing matrix-vector multiplication can be compiled into a plurality of vector intrinsic basic instructions through the vector automatic capability of the compiler. Thus, the method for performing matrix-vector multiplication provided by the present application can be implemented by executing the plurality of vector intrinsic basic instructions, that is, the matrix-vector multiplication can be implemented by using a second vector and a third vector with a larger size and a smaller number, and the acceleration of the matrix-vector multiplication can be achieved.

[0107] FIG. 7 is a schematic diagram of an apparatus for performing matrix-vector multiplication according to an embodiment of the present application. The apparatus includes:

[0108] The receiving module 710 is configured to receive a calculation instruction sent by an application program, the calculation instruction including X groups of matrix-vector multiplication operations, each group of matrix-vector multiplication operations including a first matrix and a first vector, wherein the first matrix includes M rows and N columns, the first vector includes N elements, and X is greater than N. The receiving module 710 can be specifically configured to implement the receiving function of step 301 and other implicit steps.

[0109] The generating module 720 is configured to generate N groups of second vectors according to the X first matrices, each group of second vectors including M second vectors, wherein the M second vectors in the ith group are respectively composed of X elements of M rows and the ith column in the X matrices; and generate N third vectors according to the X first vectors, wherein the ith third vector is composed of elements at the ith position in the X first vectors. The generating module 720 can be specifically configured to implement the generating function of step 302 and other implicit steps.

[0110] The operation module 730 is configured to perform vector multiplication operation on the M second vectors in the ith group and the ith third vector by using the vector operation unit. The operation module 730 can be specifically configured to implement the operation function of step 303 and other implicit steps.

[0111] The determining module 740 is configured to determine the operation result of the X groups of matrix-vector multiplication operations according to the operation result of the vector multiplication operation. The determining module 740 can be specifically configured to implement the determining function of step 304 and other implicit steps.

[0112] In an implementable manner, the operation module 730 is configured to:

[0113] perform vector bit-by-bit multiplication operation on the jth second vector in the ith group of second vectors and the ith third vector by using the vector operation unit, to obtain the jth fourth vector corresponding to the ith group, and perform vector bit-by-bit addition operation on the jth fourth vector of each group to obtain the jth fifth vector;

[0114] The determining module 740 is configured to determine the ith element in each fifth vector as the operation result of the matrix-vector multiplication operation of the ith group.

[0115] In an implementable manner, the apparatus further comprises a calling module, configured to:

[0116] in response to a calculation instruction sent by the application program, call an intrinsic instruction package of the processor, the intrinsic instruction package comprising at least one vector loading instruction and at least one vector multiply-accumulate instruction; wherein,

[0117] the at least one vector loading instruction is used to implement the generation of N groups of second vectors from X first matrices, and / or is used to implement the generation of N third vectors from X first vectors;

[0118] the at least one vector multiply-accumulate instruction is used to implement the execution of vector multiplication operation on the ith group of M second vectors and the ith third vector by using the vector operation unit.

[0119] In an implementable manner, the generating module 720 is configured to:

[0120] starting from an rth element in X first matrices stored in the memory, every t elements are stored in an sth vector register according to a vector loading instruction, wherein the initial values of r and s are 1, and t is equal to MxN-1;

[0121] after X elements are stored in the sth vector register, the values of r and s are both increased by 1, and then the process of starting from an rth element in X first matrices stored in the memory, every t elements are stored in an sth vector register according to a vector loading instruction is performed until the value of r or s is equal to MxN, wherein the vector stored in each vector register is a second vector.

[0122] In an implementable manner, the generating module 720 is configured to:

[0123] starting from a u th element in X first vectors stored in the memory, every w elements are stored in a v th vector register according to a vector loading instruction, wherein the initial values of u and v are 1, and w is equal to N-1;

[0124] after X elements are stored in the sth vector register, the values of u and v are both increased by 1, and then the process of starting from an rth element in X first matrices stored in the memory, every t elements are stored in an sth vector register according to a vector loading instruction is performed until the value of r or s is equal to MxN, wherein the vector stored in each vector register is a second vector.

[0125] In an implementable manner, the operation module 730 is configured to:

[0126] According to the vector multiply-accumulate instruction, the vector operation unit is controlled to perform a vector bit-wise multiplication operation on the jth second vector in the ith group of second vectors and the ith third vector, and perform a vector bit-wise addition operation on a fourth vector obtained by performing the vector bit-wise multiplication operation to obtain an ith fifth vector.

[0127] In an implementable manner, the operation module 730 is configured to:

[0128] The target program code is compiled to obtain a plurality of vector intrisic basic instructions, and the target program code is used to perform a vector multiplication operation on M second vectors in an ith group and an ith third vector by using the vector operation unit.

[0129] The plurality of vector intrisic basic instructions are sequentially executed to implement, by using the vector operation unit, a vector multiply-accumulate operation on the M second vectors in the ith group and the ith third vector.

[0130] The device for performing matrix vector multiplication provided in the embodiments of the present application can be used to perform the method for performing matrix vector multiplication provided in the embodiments of the present application. For the specific execution process, refer to the description of the method for performing matrix vector multiplication in the above embodiments, which will not be repeated here. The device for performing matrix vector multiplication provided in the embodiments of the present application can convert matrix vector multiplication of multiple groups of small-size matrices and vectors into large-size vector multiply-accumulate operations. The device can increase the number of elements in each vector operation in the process of implementing matrix vector multiplication by the vector operation unit, reduce the number of times of vector operation performed by the vector operation unit, improve the utilization rate of the vector operation unit, and thus improve the execution efficiency of matrix vector multiplication, thereby accelerating the processing of matrix vector multiplication involved in the fields of HPC, AI and the like.

[0131] The embodiments of the present disclosure further provide a computer program product containing instructions. The computer program product can be software or a program product containing instructions, which can run on a computer device or be stored in any available medium. When the computer program product runs on at least one computer device, the processor of the at least one computer device is caused to perform the method for matrix vector multiplication.

[0132] The embodiment of the present disclosure further provides a computer readable storage medium. The computer readable storage medium can be any available medium or data storage device that can be used to store instructions that can be executed by a computer device, or a data center and the like. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a digital video disk (DVD)), or a semiconductor medium (for example, a solid state disk) and the like. The computer readable storage medium includes instructions that can instruct the processor of the computer device to execute the method of matrix vector multiplication.

[0133] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or by program to instruct relevant hardware to complete, and the program can be stored in a computer readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk.

[0134] In the present application, the terms "first", "second" and the like are used to distinguish between elements or items having substantially the same function and similar items, and it should be understood that there is no logical or time sequence between "first" and "second", and the number and execution order are not limited. It should also be understood that although the following description uses the terms first, second, and the like to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish one element from another. In the present application, the term "at least one" means one or more, and the term "multiple" in the present application means two or more.

[0135] The above description is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any skilled person in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for performing matrix-vector multiplication, characterized in that, The method is applied to a processor, the processor including a vector operation unit, and the method includes: The system receives calculation instructions sent by the application. The calculation instructions include X sets of matrix-vector multiplication operations. Each set of matrix-vector multiplication operations includes a first matrix and a first vector. The first matrix has M rows and N columns, and the first vector has N elements. X is greater than N. N sets of second vectors are generated based on X first matrices. Each set of second vectors includes M second vectors, where the M second vectors in the i-th set are respectively composed of the X elements in the M-th row and i-th column of the X matrices. N third vectors are generated based on X first vectors, wherein the i-th third vector is composed of the element at the i-th position of the X first vectors; The vector operation unit is used to perform vector multiplication on the M second vectors of the i-th group and the i-th third vector; The result of the vector multiplication operation is determined based on the result of the vector multiplication operation of the X groups of matrices.

2. The method according to claim 1, characterized in that, The step of performing vector multiplication on the M second vectors and the i-th third vector of the i-th group using the vector operation unit includes: According to the vector operation unit, a vector-positional multiplication operation is performed on the j-th second vector and the i-th third vector in the i-th group of second vectors to obtain the j-th fourth vector corresponding to the i-th group, and a vector-positional addition operation is performed on the j-th fourth vector of each group to obtain the j-th fifth vector. Determining the result of the vector multiplication operation of the X groups of matrices based on the result of the vector multiplication operation includes: The i-th element in each fifth vector is determined as the result of the matrix-vector multiplication operation of the i-th group.

3. The method according to claim 1 or 2, characterized in that, The method further includes: In response to a computation instruction sent by the application, the processor's intrinsic instruction package is invoked, the intrinsic instruction package including at least one vector load instruction and at least one vector multiply-accumulate instruction; wherein, The at least one vector loading instruction is used to implement the generation of N sets of second vectors based on X first matrices, and / or to implement the generation of N third vectors based on X first vectors; The at least one vector multiplication and accumulation instruction is used to implement the vector multiplication operation performed by the vector operation unit on the M second vectors of the i-th group and the i-th third vector.

4. The method according to claim 3, characterized in that, The processor also includes a vector register, and the step of generating N sets of second vectors based on X first matrices includes: Starting from the r-th element of the X first matrices stored in memory, every t elements, according to the vector loading instruction, the elements after the t-th element are stored in the s-th vector register, where the initial values ​​of r and s are 1, and t equals M×N-1; After storing X elements in the s-th vector register, the values ​​of r and s are both incremented by 1, and execution is switched to start from the r-th element of the X first matrices stored in memory. Every t-elements, according to the vector loading instruction, the elements after the t-elements interval are stored in the s-th vector register until the value of r or s is equal to M×N. The vector stored in each vector register is the second vector.

5. The method according to claim 3, characterized in that, The processor also includes a vector register, and the step of generating N third vectors based on X first vectors includes: Starting from the u-th element of the X first vectors stored in memory, every w elements, according to the vector loading instruction, the elements after the w-th element interval are stored in the v-th vector register, where the initial values ​​of u and v are 1, and w equals N-1; After storing X elements in the s-th vector register, increment the values ​​of u and v by 1, and then proceed to execute the operation starting from the u-th element of the X first vectors stored in memory. Every w elements, according to the vector loading instruction, store the elements after the w-element interval into the v-th vector register until the value of u or v equals N. The vector stored in each vector register is the third vector.

6. The method according to any one of claims 3 to 5, characterized in that, The step of performing a vector-by-vector multiplication on the j-th second vector and the i-th third vector in the i-th group of second vectors, according to the vector operation unit, to obtain the j-th fourth vector corresponding to the i-th group, and performing a vector-by-vector addition on the j-th fourth vector of each group to obtain the j-th fifth vector, includes: According to the vector multiplication and accumulation instruction, the vector operation unit is controlled to perform a vector-positional multiplication operation on the j-th second vector and the i-th third vector in the i-th group of second vectors to obtain the j-th fourth vector corresponding to the i-th group, and to perform a vector-positional addition operation on the j-th fourth vector of each group to obtain the j-th fifth vector.

7. The method according to claim 1 or 2, characterized in that, The step of performing vector multiplication on the M second vectors and the i-th third vector of the i-th group using the vector operation unit includes: The target program code is compiled to obtain multiple vector intrinsic basic instructions. The target program code is used to perform vector multiplication operations on the M second vectors of the i-th group and the i-th third vector using the vector operation unit. The plurality of vector intrinsic basic instructions are executed sequentially to utilize the vector operation unit to perform vector multiplication and accumulation operations on the M second vectors of the i-th group and the i-th third vector.

8. An apparatus for performing matrix-vector multiplication, characterized in that, The device includes: The receiving module is used to receive calculation instructions sent by the application. The calculation instructions include X sets of matrix-vector multiplication operations. Each set of matrix-vector multiplication operations includes a first matrix and a first vector. The first matrix includes M rows and N columns, and the first vector includes N elements. X is greater than N. The generation module is used to generate N sets of second vectors based on X first matrices, each set of second vectors including M second vectors, wherein the M second vectors in the i-th set are respectively composed of the X elements in the M rows and i columns of the X matrices; and to generate N third vectors based on X first vectors, wherein the i-th third vector is composed of the element at the i-th position of the X first vectors. The operation module is used to perform vector multiplication on the M second vectors and the i-th third vector in the i-th group using the vector operation unit; The determining module is used to determine the result of the vector multiplication operation of the X groups of matrices based on the result of the vector multiplication operation.

9. The apparatus according to claim 8, characterized in that, The computing module is used for: According to the vector operation unit, a vector-positional multiplication operation is performed on the j-th second vector and the i-th third vector in the i-th group of second vectors to obtain the j-th fourth vector corresponding to the i-th group, and a vector-positional addition operation is performed on the j-th fourth vector of each group to obtain the j-th fifth vector. The determining module is used to: determine the i-th element in each fifth vector as the result of the matrix-vector multiplication operation of the i-th group.

10. The apparatus according to claim 8 or 9, characterized in that, The device further includes a calling module for: In response to a computation instruction sent by the application, the processor's intrinsic instruction package is invoked, the intrinsic instruction package including at least one vector load instruction and at least one vector multiply-accumulate instruction; wherein, The at least one vector loading instruction is used to implement the generation of N sets of second vectors based on X first matrices, and / or to implement the generation of N third vectors based on X first vectors; The at least one vector multiplication and accumulation instruction is used to implement the vector multiplication operation performed by the vector operation unit on the M second vectors of the i-th group and the i-th third vector.

11. The apparatus according to claim 10, characterized in that, The generation module is used for: Starting from the r-th element of the X first matrices stored in memory, every t elements, according to the vector loading instruction, the elements after the t-th element are stored in the s-th vector register, where the initial values ​​of r and s are 1, and t equals M×N-1; After storing X elements in the s-th vector register, the values ​​of r and s are both incremented by 1, and execution is switched to start from the r-th element of the X first matrices stored in memory. Every t-elements, according to the vector loading instruction, the elements after the t-elements interval are stored in the s-th vector register until the value of r or s is equal to M×N. The vector stored in each vector register is the second vector.

12. The apparatus according to claim 10, characterized in that, The generation module is used for: Starting from the u-th element of the X first vectors stored in memory, every w elements, according to the vector loading instruction, the elements after the w-th element interval are stored in the v-th vector register, where the initial values ​​of u and v are 1, and w equals N-1; After storing X elements in the s-th vector register, increment the values ​​of u and v by 1, and then proceed to execute the operation starting from the u-th element of the X first vectors stored in memory. Every w elements, according to the vector loading instruction, store the elements after the w-element interval into the v-th vector register until the value of u or v equals N. The vector stored in each vector register is the third vector.

13. The apparatus according to any one of claims 10 to 12, characterized in that, The computing module is used for: According to the vector multiplication and accumulation instruction, the vector operation unit is controlled to perform a vector-positional multiplication operation on the j-th second vector and the i-th third vector in the i-th group of second vectors, and to perform a vector-positional addition operation on the fourth vector obtained by performing the vector-positional multiplication operation to obtain the i-th fifth vector.

14. The apparatus according to claim 8 or 9, characterized in that, The computing module is used for: The target program code is compiled to obtain multiple vector intrinsic basic instructions. The target program code is used to perform vector multiplication operations on the M second vectors of the i-th group and the i-th third vector using the vector operation unit. The plurality of vector intrinsic basic instructions are executed sequentially to utilize the vector operation unit to perform vector multiplication and accumulation operations on the M second vectors of the i-th group and the i-th third vector.

15. A computer device, characterized in that, The computer device includes a processor and a memory, the processor includes a vector operation unit and a vector register, and the memory stores instructions. The processor is configured to execute instructions stored in the memory to cause the computer device to perform the method as described in any one of claims 1 to 8.

16. A computer program product, characterized in that, The computer program product includes instructions that, when executed by a processor of a computer device, cause the processor to perform the method according to any one of claims 1 to 7.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer program instructions that, when executed by a processor of a computer device, cause the processor to perform the method according to any one of claims 1 to 7.

Images