Method and apparatus for low-bit-width data matrix-vector multiplication of vector processors

Abstract

The application discloses a low-bit-width data matrix vector multiplication implementation method and device of a vector processor, and relates to the technical field of vector processors. The method comprises the following steps: storing low-bit-width vector data and low-bit-width matrix data in a double data rate synchronous dynamic random access memory; calling a direct memory access operation to load the low-bit-width matrix data from the double data rate synchronous dynamic random access memory to an on-chip array memory space of each computing core of a multi-core vector processor, load the low-bit-width vector data from the double data rate synchronous dynamic random access memory to a global shared memory, and then load the low-bit-width vector data to an on-chip scalar memory space of each computing core of the multi-core vector processor and perform vectorization processing on the low-bit-width vector data to obtain target vector data; and performing matrix vector multiplication on the target vector data and the low-bit-width matrix data in the array memory space to obtain a calculation result. In this way, the matrix vector multiplication calculation of low-bit-width data can be efficiently implemented.

Description

Technical Field

[0001] This invention relates to the field of vector processor technology, and in particular to a method and apparatus for implementing vector multiplication of low-bit-width data matrices in a vector processor. Background Technology

[0002] Matrix-vector multiplication is a relatively small-scale operator commonly used in scientific computing, deep learning, and other fields. Its flexibility makes it well-suited for various tasks, facilitating computation. Therefore, optimizing matrix-vector multiplication can significantly improve the efficiency of various tasks. For example, in models crucial to natural language processing such as Recurrent Neural Networks (RNNs), Long Short-Term Memory (LSTM) networks, and Gate Recurrent Units (GRUs), most of the computation comes from matrix-vector multiplication. Thus, matrix-vector multiplication directly determines the computational efficiency and performance of these models.

[0003] In existing technologies, matrix-vector multiplication implementations for other architectures perform poorly on vector processors or cannot be executed on vector processors. Therefore, matrix multiplication is sometimes used as an alternative to matrix-vector multiplication. However, doing so generally leads to a reduction in computational efficiency.

[0004] Therefore, how to efficiently implement matrix-vector multiplication for vector processors is a problem that urgently needs to be solved. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method and apparatus for implementing low-bit-width data matrix vector multiplication in a vector processor. Targeting vector processors and optimized for their architectural characteristics, this invention enables efficient vectorized computation of low-bit-width data matrix vector multiplication. The specific solution is as follows:

[0006] In a first aspect, this application discloses a method for implementing low-bit-width data matrix-vector multiplication in a vector processor, including:

[0007] Low-bit-width vector data and low-bit-width matrix data are stored in double-rate synchronous dynamic random access memory;

[0008] Invoking a direct memory access operation loads the low-bit-width matrix data from the double-rate synchronous dynamic random access memory into the on-chip array memory space of each computing core of the multi-core vector processor;

[0009] The direct memory access operation is invoked to load the low-bit-width vector data from the double-rate synchronous dynamic random access memory into the global shared memory, so that the global shared memory loads the low-bit-width vector data into the on-chip scalar memory space of each computing core of the multi-core vector processor, and performs vectorization processing on the low-bit-width vector data to obtain the target vector data.

[0010] The target vector data and the low-width matrix data are multiplied by a matrix-vector multiplication in the array memory space to obtain the calculation result.

[0011] Optionally, the dimension of the low-bit-width vector data x is 1×k; the dimension of the low-bit-width matrix data A is k×n; the size of the global shared memory is gsm_sizeMB; the size of the scalar memory space is sm_sizeKB; and the size of the array memory space is am_sizeKB.

[0012] Optionally, the process of invoking the direct memory access operation to load the low-bit-width matrix data from the double-rate synchronous dynamic random access memory into the on-chip array memory space of each computing core of the multi-core vector processor further includes:

[0013] Determine whether the size of the low-width matrix data occupying the array memory space exceeds half the size of the array memory space.

[0014] Optionally, the invocation of the direct memory access operation to load the low-bit-width matrix data from the double-rate synchronous dynamic random access memory into the on-chip array memory space of each computing core of the multi-core vector processor includes:

[0015] Invoking the direct memory access operation, when the low-width matrix data occupies no more than half the size of the array memory space, the low-width matrix data A is divided into n-dimensional blocks to obtain the first sub-matrix A′ of k×block_n; where block_n = 3×2×L, and L is the number of vector processing units in the vector processor. Group;

[0016] If group_n%core_num=0, each computation core processes A first submatrix A′ is loaded from the double-rate synchronous dynamic random access memory into the on-chip array memory space of each computing core of the multi-core vector processor;

[0017] If group_n % core_num! = 0, the first group_n % core_num compute cores each process The first submatrix A′, and the remaining core_num-group_n%core_num computational cores each process A first submatrix A′ is loaded from the double-rate synchronous dynamic random access memory into the on-chip array memory space of each computation core of the multi-core vector processor; where core_num is the number of computation cores on the vector processor.

[0018] Optionally, the invocation of the direct memory access operation to load the low-bit-width matrix data from the double-rate synchronous dynamic random access memory into the on-chip array memory space of each computing core of the multi-core vector processor includes:

[0019] When the direct memory access operation is invoked, if the low-width matrix data occupies more than half of the array memory space, the first submatrix A′ is further divided into k-dimensional blocks to obtain the second submatrix A″ of block_k×block_n.

[0020] point Next, the second submatrix A″ is loaded from the double-rate synchronous dynamic random access memory into the on-chip array memory space of each computing core of the multi-core vector processor; where n k The number of the second submatrix A″;

[0021] Accordingly, the invocation of the direct memory access operation to load the low-bit-width vector data from the double-rate synchronous dynamic random access memory into the global shared memory, so that the global shared memory loads the low-bit-width vector data into the on-chip scalar memory space of each computing core of the multi-core vector processor, includes:

[0022] Invoke the direct memory access operation to load the low-bit-width vector data x from the double-rate synchronous dynamic random access memory into the global shared memory;

[0023] The low-width vector data x is divided into k-dimensional blocks to obtain a 1×block_k subvector x′;

[0024] point The subvector x′ is then loaded from the global shared memory into the on-chip scalar memory space of each computational core of the multi-core vector processor; where n k Let x' be the number of the subvectors x'.

[0025] Optionally, the step of performing matrix-vector multiplication on the target vector data and the low-width matrix data in the array memory space to obtain the calculation result includes:

[0026] Step 1: Let j = 0, where j is the j-th second submatrix A″. j The number of indices, 0 ≤ j < n k ;

[0027] Step 2: Set the first block_k low-width data in SM0 to 0; where SM0 is the starting address of the scalar memory space;

[0028] Step 3: The j-th second submatrix A j "Load into AM0, x" j Load from GSM0 to SM0; AM0 is the starting address of the array memory space; GSM0 is the starting address of the global shared memory space;

[0029] Step 4: Use the immediate assignment instruction of the vector register to set the data in vector registers VR40 to VR63 to 0;

[0030] Step 5: Using the vector load instruction of the vector register, from AM output Start loading 1×block_n low-bit-width data into vector registers VR6 to VR8 as half-word vectors;

[0031] Among them, AM output The address is AM0 after offsetting off sets of low-width data. output The first 1×block_n low-width data bits are zero; offset = block_n×block_k;

[0032] Step 6: When k < block_k, let k′ = k; when k ≥ block_k, k′ = block_k;

[0033] Step 7: Let l = 0, where l is the index of the loop on k′;

[0034] Step 8: Let c = 0, where c represents the index of loop_c (the number of times the loop is unrolled); loop_c = 8;

[0035] Step 9: Using the vector loading instruction, starting from AM0, load block_n low-width data into vector registers VR0 to VR2 in half-word vector format, and increment AM0 by the size of block_n low-width data.

[0036] Step 10: Using the scalar load instruction of the vector register, load a low-width data starting from SM0 into the target scalar register R30, and increment SM0 by 1 bit of low-width data size;

[0037] Step 11: Using the low-bit single-precision extension instruction of the vector register, copy and extend the low-width data in the lower 32 bits of the 64-bit scalar register R30 to the scalar register R40.

[0038] Step 12: Broadcast the data in the scalar register R40 to the vector register VR30 via a broadcast operation;

[0039] Step 13: Based on the vector multiply-add instructions of the vector registers, multiply the data of the vector registers VR0 to VR2 and the data of the vector register VR30 respectively, and then add them to the data of the vector registers VR(40+3×c) to VR(42+3×c), and save the corresponding first calculation result to the vector registers VR(40+3×c) to VR(42+3×c);

[0040] Step 14: Let c = c + 1. If c < loop_c, then jump to step 9; if c ≥ loop_c, execute step 15.

[0041] Step 15: Let l = l + 1, if Then proceed to step 8; if Then proceed to step 16;

[0042] Step 16: Let d = 1; d is the offset index of the vector register where the results of the last loop_c-1 iterations will be stored;

[0043] Step 17: Using the vector multiply-add instruction, multiply the data in vector registers VR40 to VR42 by the immediate value 1, then add it to the data in vector registers VR(40+3×d) to VR(42+3×d), and save the corresponding second calculation result to vector registers VR40 to VR42.

[0044] Step 18: Let d = d + 1. If d < loop_c, jump to step 17; if d ≥ loop_c, execute step 19.

[0045] Step 19: Using the vector multiply-add instruction, multiply the data in vector registers VR40 to VR42 by the immediate value 1, then add it to the data in vector registers VR6 to VR8, and save the corresponding third calculation result to vector registers VR6 to VR8.

[0046] Step 20: Using the vector storage instruction of the vector register, store the data from vector registers VR6 to VR8 back to the AM in half-word vector format.output ;

[0047] Step 21: Restore the addresses of AM0 and SM0, let j = j + 1, if j < n k If j≥n, then proceed to step 2; k Then proceed to step 22;

[0048] Step 22: Set the current AM output The data is stored in the low-width output data of the corresponding computing core of the multi-core vector processor, and after each computing core of the multi-core vector processor obtains the corresponding low-width output data, the low-width output data are synchronized to obtain the target calculation result.

[0049] Secondly, this application discloses a low-bit-width data matrix vector multiplication implementation device for a vector processor, comprising:

[0050] The data storage module is used to store low-bit-width vector data and low-bit-width matrix data in a double-rate synchronous dynamic random access memory.

[0051] The first loading module is used to invoke direct memory access operations to load the low-bit-width matrix data from the double-rate synchronous dynamic random access memory into the on-chip array memory space of each computing core of the multi-core vector processor.

[0052] The second loading module is used to call the direct memory access operation to load the low-bit-width vector data from the double-rate synchronous dynamic random access memory into the global shared memory, so that the global shared memory loads the low-bit-width vector data into the on-chip scalar memory space of each computing core of the multi-core vector processor, and performs vectorization processing on the low-bit-width vector data to obtain the target vector data.

[0053] The matrix-vector multiplication calculation module is used to perform matrix-vector multiplication between the target vector data and the low-width matrix data in the array memory space to obtain the calculation result.

[0054] This application provides a method for implementing low-bit-width data matrix-vector multiplication in a vector processor, comprising: storing low-bit-width vector data and low-bit-width matrix data in double-rate synchronous dynamic random access memory; invoking a direct memory access operation to load the low-bit-width matrix data from the double-rate synchronous dynamic random access memory into the on-chip array memory space of each computing core of the multi-core vector processor; invoking the direct memory access operation to load the low-bit-width vector data from the double-rate synchronous dynamic random access memory into global shared memory, so that the global shared memory loads the low-bit-width vector data into the on-chip scalar memory space of each computing core of the multi-core vector processor, and performs vectorization processing on the low-bit-width vector data to obtain target vector data; and performing matrix-vector multiplication on the target vector data and the low-bit-width matrix data in the array memory space to obtain the calculation result. The beneficial technical effects of this application are as follows: For vector and matrix data with low bit width and high precision, the low-bit-width vector and matrix data are first stored in double-rate synchronous dynamic random access memory (DRAM). Furthermore, due to the high memory access bandwidth of global shared memory, to improve bandwidth utilization, the vectors are first transferred to global shared memory and then loaded into on-chip scalar memory to improve memory access efficiency. For vector processors, the on-chip space of the vector processor is fully utilized for maximum vectorization, and combined with the architectural characteristics of the vector processor, efficient matrix-vector multiplication is performed. Compared with traditional algorithms of this type, the technical solution of this application is more adapted to the architectural characteristics of vector processors and can handle matrix-vector multiplication functions of various scales; at the same time, it can make better use of vector computing components and effectively tap the computational efficiency of the vector processor. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0056] Figure 1 This is a schematic diagram of the architecture of a vector processor disclosed in this application.

[0057] Figure 2 This is a flowchart of a low-bit-width data matrix vector multiplication implementation method for a vector processor disclosed in this application;

[0058] Figure 3 This is a schematic diagram illustrating the grouping of low-bit-width matrix data A according to the number of computational cores disclosed in this application;

[0059] Figure 4This is a schematic diagram of the first submatrix A′ and the low-width vector data x disclosed in this application, divided into k-dimensional blocks.

[0060] Figure 5 This is a schematic diagram of the low-width data in the spatial start address of an array memory disclosed in this application;

[0061] Figure 6 This is a schematic diagram of the low-width data in the starting address of a scalar memory disclosed in this application;

[0062] Figure 7 This is a schematic diagram of the initial low-bit width data in VR6-VR8 disclosed in this application;

[0063] Figure 8 This is a schematic diagram of low bit width data in VR0-VR2 disclosed in this application;

[0064] Figure 9 This is a schematic diagram of low bit width data in R30 disclosed in this application;

[0065] Figure 10 This is a schematic diagram of low bit width data in R40 disclosed in this application;

[0066] Figure 11 This is a schematic diagram of low bit width data in VR30 disclosed in this application;

[0067] Figure 12 This is a schematic diagram of low bit width data in VR40-VR63 disclosed in this application;

[0068] Figure 13 This is a schematic diagram of low bit width data in VR40-VR42 disclosed in this application;

[0069] Figure 14 This is a schematic diagram of low-bit-width data in a loop process of VR6-VR8 disclosed in this application;

[0070] Figure 15 This is a schematic diagram of the structure of a low-bit-width data matrix vector multiplication implementation device for a vector processor disclosed in this application. Detailed Implementation

[0071] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0072] Vector processors are a novel architecture. For example... Figure 1As shown, it includes a Scalar Processing Unit (SPU) for scalar operations, a Vector Processing Unit (VPU) for vector operations, and a Direct Memory Access (DMA) unit responsible for data transfer. The Scalar Processing Unit consists of a Scalar Processing Element (SPE) and Scalar Memory (SM). The Vector Processing Unit consists of L Vector Processing Elements (VPEs) and an Array Memory (AM). The L Vector Processing Elements operate collaboratively in a Single Instruction Multiple Data (SIMD) manner, supporting the enabling and disabling of specific Vector Processing Elements, but not supporting data interaction between multiple Vector Processing Elements. A single Vector Processing Element can process one 8-byte data unit (e.g., FP64, Int64) or two 4-byte data units (e.g., FP32, Int32) at a time.

[0073] Currently, for vector processors, the matrix-vector multiplication implementations of other architectures have poor performance or cannot be executed on vector processors. Sometimes, matrix multiplication is used as a substitute for matrix-vector multiplication, which leads to a reduction in computational efficiency.

[0074] To this end, this application provides a vector processor-based implementation scheme for low-bit-width data matrix vector multiplication, which is optimized for vector processors and can efficiently realize vectorized computation of low-bit-width data matrix vector multiplication.

[0075] This invention discloses a method for implementing low-bit-width data matrix-vector multiplication in a vector processor. See [link to relevant documentation]. Figure 2 As shown, the method includes:

[0076] Step S11: Store the low-bit-width vector data and low-bit-width matrix data in a double-rate synchronous dynamic random access memory.

[0077] In this embodiment of the application, when performing matrix-vector multiplication on low-bit-width data of a vector processor, the low-bit-width vector data and the low-bit-width matrix data are first stored in Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM).

[0078] Let x represent low-bit-width vector data, and let A represent low-bit-width matrix data. In this embodiment, through... This allows us to determine the result of the matrix-vector multiplication. Here, x is the input vector, and A is the input matrix. Furthermore, we set the dimension of the input low-bit-width vector x to 1×k; and the dimension of the input low-bit-width matrix A to k×n. It can be understood that the output vector y of the matrix-vector multiplication has a dimension of 1×n.

[0079] Furthermore, in this embodiment, the high memory access bandwidth of Global Shared Memory (GSM) is utilized to transfer data to GSM, thereby improving memory access efficiency. Therefore, multiple vector processor cores can share and exchange data across cores via GSM. It is understood that a vector processing unit consists of L vector processing units and an array memory. A single vector processing unit can process one 8-byte data unit or two 4-byte data units at a time. The minimum granularity of direct memory access operations is also 8 bytes; therefore, the operation unit that can access direct memory can handle data transfer between scalar memory, GSM, and the array memory.

[0080] Furthermore, let the size of the global shared memory be gsm_size MB, which is... A low-bit-width data; let the size of the scalar memory space be sm_sizeKB, which is... A low-bit-width data; let the size of the array memory space be am_size KB, which is... Low-width data.

[0081] Step S12: Invoke the direct memory access operation to load the low-bit-width matrix data from the double-rate synchronous dynamic random access memory into the on-chip array memory space of each computing core of the multi-core vector processor.

[0082] In this embodiment, the operation unit via direct memory access is responsible for data transfer between scalar memory, global shared memory, and array memory. Low-bit-width data is loaded from double-rate synchronous dynamic random access memory (DRAM) into the on-chip array memory space of each computing core of the multi-core vector processor. First, the size of the space occupied by the low-bit-width matrix data is determined, specifically whether the size of the array memory space occupied by the low-bit-width matrix data exceeds half the size of the array memory space.

[0083] In one specific implementation, when the low-width matrix data occupies no more than half the size of the array memory space, the low-width matrix data A is divided into n-dimensional blocks to obtain a first sub-matrix A′ of dimension k×block_n. Each core of the vector processor processes the first sub-matrix A′ of dimension k×block_n; where block_n = 3×2×L, is the grouping size in the n-dimensional space, and L is the number of vector processing units in the vector processor. Group; It represents rounding up.

[0084] Furthermore, if group_n % core_num = 0, each computation core processes... A first submatrix A′ is loaded from the double-rate synchronous dynamic random access memory into the on-chip array memory space of each computation core of the multi-core vector processor; if group_n % core_num! = 0, the first group_n % core_num computation cores each process The first submatrix A′, and the remaining core_num-group_n%core_num computational cores each process A first submatrix A′ is loaded from the double-rate synchronous dynamic random access memory (DRAM) into the on-chip array memory space of each computation core of the multi-core vector processor; where core_num is the number of computation cores on the vector processor. The final low-bit-width output data computed by each computation core is denoted as y. i , 0 ≤ i < core_num. Taking core_num = 4 and n = 6 × block_n as an example, the first submatrix A′ obtained by grouping is as follows: Figure 3 As shown.

[0085] In another specific implementation, when the low-width matrix data occupies more than half the size of the array memory space, the first submatrix A′ is further divided into k-dimensional blocks to obtain a second submatrix A″ of block_k×block_n, where, point Next, the second submatrix A″ is loaded from the double-rate synchronous dynamic random access memory into the on-chip array memory space of each computing core of the multi-core vector processor; where n k Let A be the number of the second submatrix A″.

[0086] It is understandable that, since the dimension of the low-bit-width vector data x is 1×k, the low-bit-width vector data x is also divided into k-dimensional blocks to obtain 1×block_k sub-vectors x′; the number of sub-vectors x′ is also... Taking k = 4 × block_k as an example, the first submatrix A′ and the subvector x′ are divided into k-dimensional blocks, such as... Figure 4 As shown.

[0087] Step S13: Invoke the direct memory access operation to load the low-bit-width vector data from the double-rate synchronous dynamic random access memory into the global shared memory, so that the global shared memory loads the low-bit-width vector data into the on-chip scalar memory space of each computing core of the multi-core vector processor, and performs vectorization processing on the low-bit-width vector data to obtain the target vector data.

[0088] In this embodiment, corresponding to the two implementations of step S12, if the low-bit-width matrix data A does not need to be divided into k-dimensional blocks, then after loading the first sub-matrix A′ into the array memory space, based on direct memory access operations, the low-bit-width vector data is directly loaded from the double-rate synchronous dynamic random access memory into the global shared memory, and then loaded from the global shared memory into the on-chip scalar memory space of each computing core of the multi-core vector processor. If the low-bit-width matrix data A needs to be divided into k-dimensional blocks, then the low-bit-width vector data x is also divided into k-dimensional blocks. After loading the second sub-matrix A″ into the array memory space, based on direct memory access operations, the low-bit-width vector data x is divided into k-dimensional blocks. The subvector x′ is then loaded from the global shared memory into the on-chip scalar memory space of each computational core of the multi-core vector processor.

[0089] Step S14: Multiply the target vector data and the low-width matrix data by matrix-vector in the array memory space to obtain the calculation result.

[0090] In this embodiment of the application, matrix-vector multiplication is performed on the data loaded into the on-chip array memory space of each computing core of the multi-core vector processor in the array memory space.

[0091] First, let offset = block_n × block_k, and AM0 be the starting address of the array memory space; AM output The address is after offsetting AM0 by offset bits of low-width data; AM output The first 1×block_n low-width data bits are zero; let SM0 be the starting address of the scalar memory space; let GSM0 be the starting address of the global shared memory space.

[0092] The specific implementation process of matrix-vector multiplication is explained below:

[0093] Step 1: Let j = 0, where j is the j-th second submatrix A j The index of the number of "″, 0≤j<n k .

[0094] Step 2: Set the first block_k low-width data in SM0 to 0; where SM0 is the starting address of the scalar memory space;

[0095] Setting the first block_k low-width data in SM0 to zero ensures that the final matrix-vector multiplication result is not affected when k is not divisible by block_k.

[0096] Step 3: The j-th second submatrix A j "Load into AM0, x" j Load from GSM0 to SM0; AM0 is the starting address of the array memory space; GSM0 is the starting address of the global shared memory space;

[0097] In the aforementioned steps, each calculation requires calling the direct memory access operation to transfer the data again. Therefore, the second submatrix A″ is loaded into AM0, the low-width vector data x is loaded into GSM0, and then the subvector x... j Loading from GSM0 to SM0; taking L=2, k=16, block_k=8 as an example, n k =2, the data in AM0 and SM0 under different j are as follows: Figure 5 , Figure 6 As shown.

[0098] Step 4: Use the immediate assignment instruction of the vector register to set the data in vector registers VR40 to VR63 to 0;

[0099] There are 64 registers with serial numbers 0-63. Vector register VR40 represents the vector register with serial number 40, and vector register VR63 represents the vector register with serial number 63.

[0100] Step 5: Using the vector load instruction of the vector register, from AM output Start loading 1×block_n low-bit-width data into vector registers VR6 to VR8 as half-word vectors;

[0101] like Figure 7 The figure shows the data for VR6-VR8 under different j.

[0102] Step 6: When k < block_k, let k′ = k; when k ≥ block_k, k′ = block_k;

[0103] Step 7: Let l = 0, where l is the index of the loop on k′;

[0104] Step 8: Set c = 0, where c represents the index of loop_c, the number of times the loop is unrolled;

[0105] In this embodiment of the application, the loop unrolling count is set to loop_c = 8.

[0106] Step 9: Using the vector loading instruction, starting from AM0, load block_n low-width data into vector registers VR0 to VR2 in half-word vector format, and increment AM0 by the size of block_n low-width data.

[0107] like Figure 8 The figure shows the data in VR0-VR2 under different j and c.

[0108] Step 10: Using the scalar load instruction of the vector register, load a low-width data starting from SM0 into the target scalar register R30, and increment SM0 by 1 bit of low-width data size;

[0109] like Figure 9 The figure shows the data in R30 under different j.

[0110] Step 11: Using the low-bit single-precision extension instruction of the vector register, copy and extend the low-width data in the lower 32 bits of the 64-bit scalar register R30 to the scalar register R40.

[0111] like Figure 10 The figure shows the data in R40 under different j.

[0112] Step 12: Broadcast the data in the scalar register R40 to the vector register VR30 via a broadcast operation;

[0113] Among them, L vector processing units store the same data; such as Figure 11 The figure shows the data in VR30 under different j.

[0114] Step 13: Based on the vector multiply-add instructions of the vector registers, multiply the data of the vector registers VR0 to VR2 and the data of the vector register VR30 respectively, and then add them to the data of the vector registers VR(40+3×c) to VR(42+3×c), and save the corresponding first calculation result to the vector registers VR(40+3×c) to VR(42+3×c);

[0115] like Figure 12 The figure shows the data for VR40-VR63 under different j and c.

[0116] Step 14: Let c = c + 1. If c < loop_c, then jump to step 9; if c ≥ loop_c, execute step 15.

[0117] Step 15: Let l = l + 1, if Then proceed to step 8; if Then proceed to step 16;

[0118] Step 16: Let d = 1; d is the offset index of the vector register where the results of the last loop_c-1 iterations will be stored;

[0119] When d = 1, vector registers VR(40 + 3 × d) to VR(42 + 3 × d) represent VR43-VR45, used to accumulate the results of loop_c. Figure 13 The figure shows the data for VR40-VR42 under different j and d.

[0120] Step 17: Using the vector multiply-add instruction, multiply the data in vector registers VR40 to VR42 by the immediate value 1, then add it to the data in vector registers VR(40+3×d) to VR(42+3×d), and save the corresponding second calculation result to vector registers VR40 to VR42.

[0121] Step 18: Let d = d + 1. If d < loop_c, jump to step 17; if d ≥ loop_c, execute step 19.

[0122] Step 19: Using the vector multiply-add instruction, multiply the data in vector registers VR40 to VR42 by the immediate value 1, then add it to the data in vector registers VR6 to VR8, and save the corresponding third calculation result to vector registers VR6 to VR8.

[0123] like Figure 14 The figure shows the data for VR6-VR8 under different j.

[0124] Step 20: Using the vector storage instruction of the vector register, store the data from vector registers VR6 to VR8 back to the AM in half-word vector format. output ;

[0125] Step 21: Restore the addresses of AM0 and SM0, let j = j + 1, if j < n k If j≥n, then proceed to step 2; k Then proceed to step 22;

[0126] Step 22: Set the current AM output The data is stored in the low-width output data of the corresponding computing core of the multi-core vector processor, and after each computing core of the multi-core vector processor obtains the corresponding low-width output data, the low-width output data are synchronized to obtain the target calculation result.

[0127] Waiting for each core of the multi-core processor to complete y i The computation, after synchronization of all cores, performs matrix-vector multiplication to obtain y. It is evident that the space of scalar memory and array memory is fully utilized, as are the loading, storage, and computation components, and the VR register, achieving maximum vectorization. Expanding lpop_c on k′ perfectly matches the instruction latency of vector processors and is well-suited for commonly used RNN models.

[0128] This application provides a method for implementing low-bit-width data matrix-vector multiplication in a vector processor, comprising: storing low-bit-width vector data and low-bit-width matrix data in double-rate synchronous dynamic random access memory; invoking a direct memory access operation to load the low-bit-width matrix data from the double-rate synchronous dynamic random access memory into the on-chip array memory space of each computing core of the multi-core vector processor; invoking the direct memory access operation to load the low-bit-width vector data from the double-rate synchronous dynamic random access memory into global shared memory, so that the global shared memory loads the low-bit-width vector data into the on-chip scalar memory space of each computing core of the multi-core vector processor, and performs vectorization processing on the low-bit-width vector data to obtain target vector data; and performing matrix-vector multiplication on the target vector data and the low-bit-width matrix data in the array memory space to obtain the calculation result. The beneficial technical effects of this application are as follows: For vector and matrix data with low bit width and high precision, the low-bit-width vector and matrix data are first stored in double-rate synchronous dynamic random access memory (DRAM). Furthermore, due to the high memory access bandwidth of global shared memory, to improve bandwidth utilization, the vectors are first transferred to global shared memory and then loaded into on-chip scalar memory to improve memory access efficiency. For vector processors, the on-chip space of the vector processor is fully utilized for maximum vectorization, and combined with the architectural characteristics of the vector processor, efficient matrix-vector multiplication is performed. Compared with traditional algorithms of this type, the technical solution of this application is more adapted to the architectural characteristics of vector processors and can handle matrix-vector multiplication functions of various scales; at the same time, it can make better use of vector computing components and effectively tap the computational efficiency of the vector processor.

[0129] Accordingly, embodiments of this application also disclose a vector processor implementation device for low-bit-width data matrix vector multiplication, see [link to relevant documentation]. Figure 15 As shown, the device includes:

[0130] Data storage module 11 is used to store low-bit-width vector data and low-bit-width matrix data in double-rate synchronous dynamic random access memory;

[0131] The first loading module 12 is used to call a direct memory access operation to load the low-bit-width matrix data from the double-rate synchronous dynamic random access memory into the on-chip array memory space of each computing core of the multi-core vector processor.

[0132] The second loading module 13 is used to call the direct memory access operation to load the low-bit-width vector data from the double-rate synchronous dynamic random access memory into the global shared memory, so that the global shared memory loads the low-bit-width vector data into the on-chip scalar memory space of each computing core of the multi-core vector processor, and performs vectorization processing on the low-bit-width vector data to obtain the target vector data.

[0133] The matrix-vector multiplication calculation module 14 is used to perform matrix-vector multiplication between the target vector data and the low-width matrix data in the array memory space to obtain the calculation result.

[0134] For more detailed information on the working process of each of the above modules, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0135] Therefore, the above-described scheme in this embodiment includes: storing low-bit-width vector data and low-bit-width matrix data in double-rate synchronous dynamic random access memory; invoking a direct memory access operation to load the low-bit-width matrix data from the double-rate synchronous dynamic random access memory into the on-chip array memory space of each computing core of the multi-core vector processor; invoking the direct memory access operation to load the low-bit-width vector data from the double-rate synchronous dynamic random access memory into global shared memory, so that the global shared memory loads the low-bit-width vector data into the on-chip scalar memory space of each computing core of the multi-core vector processor, and performs vectorization processing on the low-bit-width vector data to obtain target vector data; and performing matrix-vector multiplication on the target vector data and the low-bit-width matrix data in the array memory space to obtain the calculation result. The beneficial technical effects of this application are as follows: For vector and matrix data with low bit width and high precision, the low-bit-width vector and matrix data are first stored in double-rate synchronous dynamic random access memory (DRAM). Furthermore, due to the high memory access bandwidth of global shared memory, to improve bandwidth utilization, the vectors are first transferred to global shared memory and then loaded into on-chip scalar memory to improve memory access efficiency. For vector processors, the on-chip space of the vector processor is fully utilized for maximum vectorization, and combined with the architectural characteristics of the vector processor, efficient matrix-vector multiplication is performed. Compared with traditional algorithms of this type, the technical solution of this application is more adapted to the architectural characteristics of vector processors and can handle matrix-vector multiplication functions of various scales; at the same time, it can make better use of vector computing components and effectively tap the computational efficiency of the vector processor.

[0136] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0137] The present invention provides a detailed description of a method and apparatus for implementing vector multiplication of a low-bit-width data matrix in a vector processor. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for implementing low-bit-width data matrix-vector multiplication in a vector processor, characterized in that, include: Low-bit-width vector data and low-bit-width matrix data are stored in double-rate synchronous dynamic random access memory; Invoking a direct memory access operation loads the low-bit-width matrix data from the double-rate synchronous dynamic random access memory into the on-chip array memory space of each computing core of the multi-core vector processor; The direct memory access operation is invoked to load the low-bit-width vector data from the double-rate synchronous dynamic random access memory into the global shared memory, so that the global shared memory loads the low-bit-width vector data into the on-chip scalar memory space of each computing core of the multi-core vector processor, and performs vectorization processing on the low-bit-width vector data to obtain the target vector data. The target vector data and the low-width matrix data are multiplied by a matrix-vector multiplication in the array memory space to obtain the calculation result.

2. The method for implementing low-bit-width data matrix-vector multiplication in a vector processor according to claim 1, characterized in that, The dimension of the low-width vector data x is The dimension of the low-width matrix data A is... The size of the global shared memory is The size of the scalar memory space is The size of the array memory space is .

3. The method for implementing low-bit-width data matrix-vector multiplication in a vector processor according to claim 2, characterized in that, The process of calling the direct memory access operation to load the low-bit-width matrix data from the double-rate synchronous dynamic random access memory into the on-chip array memory space of each computing core of the multi-core vector processor further includes: Determine whether the size of the low-width matrix data occupying the array memory space exceeds half the size of the array memory space.

4. The method for implementing low-bit-width data matrix-vector multiplication in a vector processor according to claim 3, characterized in that, The invocation of the direct memory access operation to load the low-bit-width matrix data from the double-rate synchronous dynamic random access memory into the on-chip array memory space of each computing core of the multi-core vector processor includes: When the direct memory access operation is invoked, if the size of the low-width matrix data occupying the array memory space does not exceed half the size of the array memory space, the low-width matrix data A is divided into n-dimensional blocks to obtain... First submatrix ;in, L represents the number of vector processing units in the vector processor. Group; if Each computational core processes The first submatrix , so as to make the first submatrix Loaded from the double-rate synchronous dynamic random access memory into the on-chip array memory space of each computing core of the multi-core vector processor; if ,forward Each computing core processes... The first submatrix The remaining Each computing core processes... The first submatrix , so as to make the first submatrix The on-chip array memory space of each computational core of the multi-core vector processor is loaded from the double-rate synchronous dynamic random access memory; wherein... This refers to the number of computing cores on the vector processor.

5. The method for implementing low-bit-width data matrix-vector multiplication in a vector processor according to claim 4, characterized in that, The invocation of the direct memory access operation to load the low-bit-width matrix data from the double-rate synchronous dynamic random access memory into the on-chip array memory space of each computing core of the multi-core vector processor includes: When the direct memory access operation is invoked, if the low-bit-width matrix data occupies more than half the size of the array memory space, then the first sub-matrix is... Continue partitioning in the k-dimensional space to obtain... The second submatrix ; point Next, the second submatrix The on-chip array memory space of each computational core of the multi-core vector processor is loaded from the double-rate synchronous dynamic random access memory; wherein... The second submatrix The number of; Accordingly, the invocation of the direct memory access operation to load the low-bit-width vector data from the double-rate synchronous dynamic random access memory into the global shared memory, so that the global shared memory loads the low-bit-width vector data into the on-chip scalar memory space of each computing core of the multi-core vector processor, includes: The direct memory access operation is invoked to transfer the low-bit-width vector data. Load from the double-rate synchronous dynamic random access memory into the global shared memory; The low-width vector data x is divided into k-dimensional blocks to obtain subvectors ; point The subvector will be used next The on-chip scalar memory space of each computational core of the multi-core vector processor is loaded from the global shared memory; wherein... For the subvector The number of.

6. The method for implementing low-bit-width data matrix-vector multiplication in a vector processor according to claim 5, characterized in that, The step of performing matrix-vector multiplication on the target vector data and the low-width matrix data in the array memory space to obtain the calculation result includes: Step 1: Let , j is the j-th second submatrix The number of indexes, ; Step 2: Center front The low-width data is set to 0; among them... This is the starting address of the scalar memory space; Step 3: The j-th second submatrix Load to ,Will from Load to ; This is the starting address of the array memory space; This is the starting address of the globally shared memory; Step 4: Use the immediate assignment instruction of the vector register to set the data in vector registers VR40 to VR63 to 0; Step 5: Using the vector load instruction of the vector register, from Start Each low-bit-width data is loaded into vector registers VR6 to VR8 as a half-word vector; in, for Offset The address following the low-width data. Center front The lowest bit width data is zero; ; Step 6: When season ;when hour, ; Step 7: Let ,l is The index of the previous loop; Step 8: Let c represents the number of times the loop unrolling is used. Index; ; Step 9: Using the vector loading instruction, from Start Each low-bit width data segment is loaded into vector registers VR0 through VR2 as a half-word vector. Self-incrementing Size of each low-width data segment; Step 10: Using the scalar load instruction of the vector register, from Begin loading a low-width data bit into the target scalar register R30. Increment by 1 bit in the low-width data size; Step 11: Using the low-bit single-precision extension instruction of the vector register, copy and extend the low-width data in the lower 32 bits of the 64-bit scalar register R30 to the scalar register R40. Step 12: Broadcast the data in the scalar register R40 to the vector register VR30 via a broadcast operation; Step 13: Based on the vector multiply-add instructions of the vector register, perform multiplication operations using the data from vector registers VR0 to VR2 and the data from vector register VR30, respectively, and then multiply the results with the data from the vector register. To vector register The data are added together, and the corresponding first calculation result is saved to the vector register. to the vector register ; Step 14: Let ,when If so, proceed to step 9; when Then, proceed to step 15; Step 15: Let ,if If so, proceed to step 8; Then proceed to step 16; Step 16: Let ;d represents the final result. The offset index of the vector register where the result is stored; Step 17: Using the vector multiply-add instruction, multiply the data in vector registers VR40 to VR42 with the immediate value 1, and then multiply by the vector register... To vector register The data are added together, and the corresponding second calculation result is saved to the vector registers VR40 to VR42; Step 18: Let ,when When, jump to step 17; when Then, proceed to step 19; Step 19: Using the vector multiply-add instruction, multiply the data in vector registers VR40 to VR42 by the immediate value 1, then add it to the data in vector registers VR6 to VR8, and save the corresponding third calculation result to vector registers VR6 to VR8. Step 20: Using the vector storage instruction of the vector register, store the data from vector registers VR6 to VR8 back to the vector register in half-word vector format. ; Step 21: Restore the With the The address, ,if If so, proceed to step 2; if Then proceed to step 22; Step 22: Set the current The data is stored in the low-width output data of the corresponding computing core of the multi-core vector processor, and after each computing core of the multi-core vector processor obtains the corresponding low-width output data, the low-width output data are synchronized to obtain the target calculation result.

7. A vector processor apparatus for implementing low-bit-width data matrix-vector multiplication, characterized in that, include: The data storage module is used to store low-bit-width vector data and low-bit-width matrix data in a double-rate synchronous dynamic random access memory. The first loading module is used to invoke direct memory access operations to load the low-bit-width matrix data from the double-rate synchronous dynamic random access memory into the on-chip array memory space of each computing core of the multi-core vector processor. The second loading module is used to call the direct memory access operation to load the low-bit-width vector data from the double-rate synchronous dynamic random access memory into the global shared memory, so that the global shared memory loads the low-bit-width vector data into the on-chip scalar memory space of each computing core of the multi-core vector processor, and performs vectorization processing on the low-bit-width vector data to obtain the target vector data. The matrix-vector multiplication calculation module is used to perform matrix-vector multiplication between the target vector data and the low-width matrix data in the array memory space to obtain the calculation result.

Images