A method for complex matrix-vector multiplication based on the Ascend Atlas matrix computation unit.

Abstract

This invention belongs to the field of high-performance computing technology and is a method for complex matrix-vector multiplication based on the Ascend Atlas matrix computation unit. The method includes the following steps: The partitioned computational tasks are assigned to the various matrix computation cores of the Atlas NPU for parallel computation; the vector computation unit recombines and arranges the complex vectors to obtain two column vectors with real-negative imaginary part spacing and imaginary-real part spacing, and combines these two vectors into a column-major matrix; the matrix operation unit is invoked to perform matrix-vector multiplication, obtaining the real and imaginary parts of the complex result vector; the real and imaginary parts of the complex result vector are merged by the vector computation unit, synchronizing the matrix operation unit and the vector operation unit at the computation core granularity to obtain the final complex result vector. This invention eliminates the need for real-imaginary separation of the complex matrix, reducing computational performance consumption, accelerating complex matrix-vector multiplication, and improving the computational efficiency of complex matrix-vector multiplication on the NPU.

Description

Technical Field

[0001] This invention relates to the field of high-performance computing technology, specifically to a method for complex matrix-vector multiplication based on the Ascend Atlas matrix computing unit. Background Technology

[0002] Matrix-vector multiplication (GEMV) is a fundamental operation in linear algebra with wide applications in machine learning, large-scale models, and other fields. For example, in machine learning, GEMV is widely used in the forward and backward propagation processes of neural networks; in large-scale models, it is extensively applied to the inference process. In signal processing, GEMV is used in tasks such as filter design and signal reconstruction. The computational complexity of GEMV is O(n log n). 2 ), where n represents the dimension of the matrix. Therefore, when the matrix is ​​large, its computational cost can be very high. To accelerate the computation of matrix-vector multiplication, many optimization techniques have been proposed, such as cache-based optimization, vectorized instruction-based optimization, and parallel computing-based optimization.

[0003] NPU stands for Neural Processing Unit, a processor specifically designed to accelerate computation in artificial neural networks. Unlike general-purpose processors (CPUs) and graphics processing units (GPUs), NPUs employ highly optimized hardware architectures, delivering higher computing performance and energy efficiency within the same power consumption and area. The emergence of NPUs was primarily to meet the extremely high demands of artificial intelligence applications for computing performance and energy efficiency. With the rapid development of artificial intelligence technology, traditional CPUs and GPUs can no longer meet the increasingly complex computational needs, and NPUs have gradually become the computing power foundation for many artificial intelligence applications.

[0004] Huawei Ascend Atlas is a self-developed NPU by Huawei, with its computing core being the AI ​​Core. The hardware architecture of each AI Core is as follows: Figure 1 As shown, the main computational units in the AI ​​Core include: Cube Unit, Vector Unit, and Scalar Unit, used to perform calculations on different types of data. The AI ​​Core has internal storage; it needs to load data from external storage into its internal storage to complete the corresponding calculations. The AI ​​Core's internal storage includes: L1 Buffer, L0 Buffer, Unified Buffer, and Scalar Buffer.

[0005] Currently, Huawei's NPU software ecosystem is not as mature as that of general-purpose GPUs, making the optimization and acceleration of some fundamental operators of great significance. Performing complex matrix-vector multiplication on the Atlas NPU presents certain challenges because it does not support direct complex number calculations. The complex number needs to be split into its real and imaginary parts for computation, and this splitting process is computationally intensive on the Atlas NPU. Furthermore, the separation of the real and imaginary parts of the complex matrix increases the computational overhead for complex matrix-vector multiplication. Summary of the Invention

[0006] To address the technical problems existing in the prior art, this invention proposes a method for complex matrix-vector multiplication based on the Ascend Atlas matrix computation unit. This method eliminates the need for separating the real and virtual complex matrices, reduces computational performance consumption, and accelerates the calculation of complex matrix-vector multiplication.

[0007] The objective of this invention can be achieved by adopting the following technical solutions:

[0008] The method for complex matrix-vector multiplication based on the Ascend Atlas matrix computation unit includes the following steps:

[0009] S1. Divide the computation task of complex matrix-vector multiplication according to the complex result vector, and assign the divided computation task to each matrix computation core of Atlas NPU for parallel computation.

[0010] S2. Read the complex vector from global memory. The vector calculation unit recombines and arranges the complex vector to obtain two column vectors with real part-negative imaginary part interval and imaginary part-real part interval. Combine the two vectors into a column-major matrix.

[0011] S3. Read the complex matrix from global memory. Based on the complex matrix and the column-major matrix formed by combining the two vectors, call the matrix operation unit to perform matrix-vector multiplication to obtain the real and imaginary parts of the complex result vector.

[0012] S4. By merging the real and imaginary parts of the complex result vector through the vector calculation unit, the matrix operation unit and the vector operation unit are synchronized at the core granularity of calculation to obtain the final complex result vector.

[0013] Specifically, step S1 includes:

[0014] The complex number result vector is divided into multiple task segments for computation. The size of a task segment is calculated based on the size of the L1 memory on the computing core chip. The complex number result vector is then segmented according to the size of the task segment to obtain multiple task segments.

[0015] Based on the number of computing cores, all task segments are evenly distributed to the computing cores for parallel computing according to the cyclic task partitioning strategy.

[0016] Specifically, step S2 includes:

[0017] The complex vector X is separated into its real and imaginary parts using the vector operation unit. r And the imaginary part X i , will the real part X r And the imaginary part X i Then, the two vectors are rearranged into a combination of real and negative imaginary parts and a combination of imaginary and real parts, and the two vectors are combined into a column-priority matrix.

[0018] Specifically, the column priority matrix can be represented as:

[0019]

[0020] Among them, X r1 X i1 These are the real and imaginary parts of element X1, respectively, and -X i1 It is the negative imaginary part of element X1.

[0021] Specifically, step S3 includes:

[0022] In the case of matrix transpose, matrix-vector multiplication can be represented as:

[0023] Y T =X T A

[0024] Y represents the output vector, X represents the input vector, A represents the input matrix, and T represents the transpose of the matrix.

[0025] Specifically, step S4 includes:

[0026] Once the matrix computation unit obtains the real and imaginary parts of the complex result vector for a task segment, the vector computation unit immediately begins to combine the real and imaginary parts. At the same time, the matrix computation unit calculates the data for the next task segment, allowing the vector computation unit and the matrix computation unit to perform parallel computations.

[0027] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0028] This invention proposes a method for complex matrix-vector multiplication based on the Ascend Atlas matrix computation unit. By rearranging complex vectors to obtain two column vectors with real-negative imaginary part spacing and imaginary-real part spacing, the two vectors are combined into a column-priority matrix. This eliminates the need for separating the real and imaginary parts of the complex matrix, reducing computational performance consumption and accelerating complex matrix-vector multiplication. It can fully utilize the computing power of Huawei's Ascend Atlas NPU to accelerate complex matrix-vector multiplication, providing computing power support for fields such as deep learning and scientific computing, and has good practical value. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0030] Figure 1 This is an AI Core architecture diagram in Ascend Atlas according to an embodiment of the present invention;

[0031] Figure 2 This is a flowchart of a complex matrix-vector multiplication calculation method based on the Ascend Atlas matrix calculation unit in an embodiment of the present invention;

[0032] Figure 3 This is a schematic diagram of the matrix-vector multiplication task partitioning in an embodiment of the present invention;

[0033] Figure 4 This is a schematic diagram of the complex matrix-vector multiplication calculation scheme in an embodiment of the present invention;

[0034] Figure 5 This is a schematic diagram illustrating the communication between the matrix calculation unit and the vector calculation unit in an embodiment of the present invention. Detailed Implementation

[0035] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments, and the implementation of the present invention is not limited thereto. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] Example 1:

[0037] This invention discloses a method for complex matrix-vector multiplication in the Ascend Atlas matrix computation unit. It employs the matrix computation unit as the primary unit and the vector computation unit as a secondary unit to complete all computational steps of matrix-vector multiplication. This invention reassembles and arranges complex vectors to obtain two column vectors with real-negative imaginary part intervals and imaginary-real part intervals. These two vectors are then combined into a column-priority matrix, eliminating the need for real-imaginary separation of the complex matrix, reducing computational performance consumption, accelerating complex matrix-vector multiplication calculations, and improving the computational efficiency of complex matrix-vector multiplication on the NPU.

[0038] like Figure 2 The diagram shows a flowchart of a complex matrix-vector multiplication calculation method based on the Ascend Atlas matrix calculation unit. The complex matrix-vector multiplication calculation method based on the Ascend Atlas matrix calculation unit described in this invention includes the following steps:

[0039] S1. Task partitioning: The computation task of complex matrix-vector multiplication is divided into complex result vectors, and the partitioned computation tasks are assigned to the various matrix computation cores of the Atlas NPU for parallel computation.

[0040] Complex matrix-vector multiplication results in a complex vector. Tasks are divided according to this complex result vector, meaning each computing core computes one or more segments of the complex result vector. Based on the number of computing cores and the size of their on-chip cache, the number of complex result vector segments each core needs to compute and the number of segments each core can compute in a given time are calculated. The computational task is then divided into multiple task segments, and all task segments are evenly distributed to the computing cores for parallel computation using a cyclic task partitioning strategy. Ascend Atlas has multiple AI Cores, typically 20 or 24. Computational tasks need to be evenly distributed across each AI Core to ensure balanced computational load, fully utilize all computing resources, and maximize parallelism.

[0041] The complex number result vector is divided into multiple task segments for computation, and the size of each task segment is calculated based on the size of the L1 memory on the computation core chip. For example... Figure 3 As shown, the complex number result vector Y needs to be divided into multiple task segments for computation. First, the size of one computation task segment needs to be determined, which is based on the size of the L1 memory. Assuming the size of the task segment is m, since it involves complex number computation, in this embodiment, L1 needs to store two m×m matrices and two m-sized vectors, each element being single-precision, occupying 2×(m×m+m)×8 bytes of memory. Assuming the storage space of memory L1 is K bytes and double buffering is used, then m must satisfy... The maximum value.

[0042] Then, the complex result vector Y is segmented according to the size m of the task segment, resulting in multiple task segments. Assuming the size of the complex result vector is M, the number of task segments is... Assuming the number of computation cores is c, and the computation core numbers are i∈{0,1,2,…,c-1}, then the number of computational task segments for each core is:

[0043]

[0044] Based on the number of computing cores, all task segments are evenly distributed to the computing cores for parallel computation using a cyclic task partitioning strategy. That is, task segment 0 is assigned to computing core 0, task segment 1 is assigned to computing core 1, and so on. When there are still task segments remaining on the last computing core, the allocation restarts from computing core 0. This block partitioning strategy maximizes load balancing, improves L2 cache hit rate, and reduces data transfer time.

[0045] Assuming matrix A is nontranspose, the matrix-vector multiplication of matrix A and vector X is as follows:

[0046]

[0047] In this embodiment, the Ascend Atlas platform has 20 matrix computation cores and 512KB of L1 storage. If dual cache is used, the space used each time is 256KB. That is, 2(m×m+m)×8≤256×1024. Due to platform limitations, m must be a multiple of 8, so m is taken as 120. The resulting matrix has 4096 elements, which can be divided into There are 20 computing cores in each task segment. According to the cyclic task allocation strategy, the 0th core is assigned to task segments 0 and 20, the 1st core is assigned to task segments 1 and 21, the 2nd core is assigned to task segments 2 and 22, the 3rd core is assigned to task segment 3, and so on, with the 19th computing core assigned to task segment 19.

[0048] S2. The complex vector is read from global memory. The vector computation unit recombines and arranges the complex vector to obtain two column vectors with real part-negative imaginary part intervals and imaginary part-real part intervals, and combines the two vectors into a column-major matrix. The vector computation unit arranges the complex vector into two column vectors with real part-negative imaginary part intervals and imaginary part-real part intervals, which facilitates the calculation of complex matrix multiplication by the real matrix computation unit.

[0049] In GPUs, virtual-to-real separation and virtual-to-real combination involve data movement within registers and do not generate additional computation. However, on the Ascend Atlas platform, virtual-to-real separation can only be performed by the vector computation unit, which requires additional computation. If computation is performed using the vector computation unit after virtual-to-real separation, the powerful computing capabilities of the matrix operation unit cannot be utilized. If computation is performed using the matrix operation unit, the vector computation unit must first write the data to global memory, and then the matrix operation unit must read it before computation, increasing data access to global memory. Therefore, these methods cannot fully utilize the hardware performance of the Ascend Atlas platform.

[0050] In this embodiment, a complex vector is read from global memory, and the vector operation unit is used to arrange the data of the complex vector. Then, the complex vector is arranged into two column vectors with alternating real and negative imaginary parts, and with alternating imaginary and real parts, which facilitates the matrix calculation unit to calculate the complex result. There is no need to separate the real and imaginary parts of the complex matrix, which reduces the consumption of computational performance and accelerates the calculation of complex matrix-vector multiplication.

[0051] Specifically, the complex vector X is rearranged, and the complex vector X = [X] is processed by the vector operation unit. r1 ,X i1 ,X r2 ,X i2 …X r120 ,X i120 The real and virtual parts are separated into real parts X. r =[X r1 ,X r2 ,…X r120 ] and imaginary part X i =[X i1 ,X i2 ,…X i120 ], and then rearrange them into real parts and negative imaginary parts [X]. r1 ,-X i1 ,X r2 ,-X i2 …X r120 ,-X i120 The combination of the imaginary and real parts [X] i1 ,X r1 ,X i2 ,X r2 …X i120 ,X r120 Given two vectors, combine them into a column-major matrix. Specifically, a column-major matrix can be represented as:

[0052]

[0053] Among them, X r1 X i1 These are the real and imaginary parts of element X1, respectively, and -Xi1 It is the negative imaginary part of element X1.

[0054] S3. Read the complex matrix from global memory. Based on the complex matrix and the column-major matrix formed by combining the two vectors, call the matrix operation unit to perform matrix-vector multiplication to obtain the real and imaginary parts of the complex result vector.

[0055] After step S2, the complex vectors have been arranged into two new vectors. This step completes part of the matrix-vector multiplication operation for one task segment. For example... Figure 3 As shown, calculating the result of a task segment requires iteratively reading a row block of the complex matrix A, then reading a segment corresponding to the complex vector X, completing a small block of matrix-vector multiplication, and finally accumulating the results to complete a partial matrix-vector multiplication of a task segment. The matrix operation unit can efficiently calculate matrix multiplication. Since matrix data is stored in memory in row-major order, the two vectors are treated as a matrix with two columns, and the matrix calculation result consists of one row of real parts and one row of imaginary parts. The matrix operation unit is used to perform the following matrix-vector multiplication operations:

[0056]

[0057] After performing the matrix-vector multiplication operation described above by the matrix operation unit, the real part vector [Y] of the complex result vector can be obtained. r1 ,Y r2 ,…Y r120 ] and the imaginary part vector [Y i1 ,Y r2 ,…Y r120 Then, using vector computation units, the imaginary part is combined to obtain the final imaginary part result [Y]. r1 ,Y i1 ,Y r2 ,Y i2 …Y r120 ,Y i120 ]. Where Y rk Y ik It is a complex element Y k The real and imaginary parts of , k = 1, 2, ..., 120.

[0058] The matrix-vector multiplication of a segment of complex matrix A and its corresponding segment of complex vector X is processed in two ways: matrix transpose and non-transpose. The choice between transpose and non-transpose determines whether to use a vector computation unit for real-imaginary separation. In the non-transpose case, the matrix calculation result is already arranged in complex order, requiring no additional real-imaginary separation, and the result data can be directly output. In the transpose case, the matrix calculation result consists of real and imaginary vectors, requiring the use of a vector computation unit for real-imaginary separation before output.

[0059] In the case of an untransposed matrix, since the matrix data is stored in row-major order, the data in each row is stored contiguously in L1, so the data is read into L1. The layout is then modified according to the requirements of the matrix calculation unit.

[0060] Specifically, in the case of matrix transpose, the computation process is simplified. The matrix-vector multiplication operation for the general case of matrix transpose can be represented as follows:

[0061] Y = A T X

[0062] Y represents the output vector, X represents the input vector, A represents the input matrix, and T represents the transpose of the matrix.

[0063] Given Y=A T Taking the transpose of both sides of X, the matrix-vector multiplication operation can be represented as:

[0064] Y T =X T A

[0065] Vector transpose does not require any actual operation, so the transposed and non-transposed cases are the same; you only need to multiply the matrix on the right.

[0066] like Figure 4 As shown, the multiplication of a 4x2 complex matrix and a complex vector of length 2 is calculated. Taking the calculation of the complex element Y1 as an example, the formula for Y1 is Y1 = A. 11 ×X1+A 12 ×X2. Hardware cannot directly calculate complex number multiplication; it needs to be converted to real part multiplication. The formula for calculating the real part of Y1 is Y... r1 =A r11 ×X r1 -A i11 ×X i1 +A r12 ×X r2 -A i12 ×X i2 The formula for calculating the imaginary part is Y. i1 =A i11 ×X r1 +A r11 ×X i1 +A i12 ×X r2 +A r12 ×X i2 By arranging complex vectors, the multiplication and addition of complex vectors are converted into the multiplication and addition of real part vectors. The formula for calculating the real part of Y1 after the conversion is Y... r1 =A r11 ×X r1 +A i11 ×(-Xi1 )+A r12 ×X r2 +A i12 ×(-X i2 The formula for calculating the imaginary part is Y. i1 =A r11 ×X i1 +A i11 ×X r1 +A r12 ×X i2 +A i12 ×X r2 Subtraction in complex multiplication is converted into addition after negating the elements, which utilizes the powerful computing power of the matrix operation unit and reduces memory access requests for separating the virtual and real matrices in the vector calculation unit.

[0067] S4. By merging the real and imaginary parts of the complex result vector through the vector calculation unit, the matrix operation unit and the vector operation unit are synchronized at the core granularity of calculation to obtain the final complex result vector.

[0068] Specifically, after the matrix computation unit obtains the real and imaginary parts of the complex result vector for a task block, the vector computation unit immediately begins combining the real and imaginary parts, while the matrix computation unit calculates the data for the next task block, allowing the vector computation unit and the matrix computation unit to perform parallel computation. After obtaining the imaginary and real part complex result vectors in step S3, these two complex result vectors need to be merged into a single complex result vector. The vector computation unit merges the real part vector [Y]. r1 ,Y r2 ,…Y r120 ] and the imaginary part vector [Y i1 ,Y r2 ,…Y r120 The final complex vector [Y] is obtained. r1 ,Y i1 ,Y r2 ,Y i2 …Y r120 ,Y i120 The vector computation unit can directly complete the merging of the real and imaginary complex number result vectors using the vector instruction. The matrix computation unit and the vector computation unit perform synchronization at the core computational granularity; the vector computation unit reads the intermediate results obtained by the matrix computation unit and performs the real-imaginary-real combination. If the matrix computation unit calculates all complex matrices into vectors before performing the real-imaginary-real combination, it will increase the operator execution time, and the matrix computation unit and the vector computation unit are not parallel. In Ascend Atlas, the matrix computation unit and the vector computation unit have a data path.

[0069] like Figure 5 As shown, after the matrix computation unit obtains the real and imaginary parts of the complex result vector for a task segment, the vector computation unit immediately begins combining the real and imaginary parts. Simultaneously, the matrix computation unit computes a task block of data, allowing the vector and matrix computation units to run in parallel. This masks the overhead of the vector computation unit. After the 20 computation cores complete their respective task segments in parallel, they write the computation results back to the corresponding locations in global memory, thus completing the overall matrix-vector multiplication.

[0070] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for complex matrix-vector multiplication based on the Ascend Atlas matrix computation unit, characterized in that, Includes the following steps: S1. Divide the computation task of complex matrix-vector multiplication according to the complex result vector, and assign the divided computation task to each matrix computation core of Atlas NPU for parallel computation. S2. Read the complex vector from global memory. The vector calculation unit recombines and arranges the complex vector to obtain two column vectors with real part-negative imaginary part interval and imaginary part-real part interval. Combine the two vectors into a column-major matrix. Step S2 includes: separating the real and imaginary parts of the complex vector X into its real part X using a vector operation unit. r And the imaginary part X i , will the real part X r And the imaginary part X i Then, recombine and arrange the two vectors into a combination of real part and negative imaginary part and a combination of imaginary part and real part, and combine the two vectors into a column-priority matrix; The column priority matrix can be represented as: ； Among them, X r1 X i1 These are the real and imaginary parts of element X1, respectively, and -X i1 It is the negative imaginary part of element X1; S3. Read the complex matrix from global memory. Based on the complex matrix and the column-major matrix formed by combining the two vectors, call the matrix operation unit to perform matrix-vector multiplication to obtain the real and imaginary parts of the complex result vector. S4. By merging the real and imaginary parts of the complex result vector through the vector computation unit, the matrix operation unit and the vector operation unit are synchronized at the core granularity of computation to obtain the final complex result vector.

2. The complex matrix-vector multiplication calculation method based on the Ascend Atlas matrix calculation unit according to claim 1, characterized in that, Step S1 includes: The complex number result vector is divided into multiple task segments for computation. The size of a task segment is calculated based on the size of the L1 memory on the computing core chip. The complex number result vector is then segmented according to the size of the task segment to obtain multiple task segments. Based on the number of computing cores, all task segments are evenly distributed to the computing cores for parallel computing according to the cyclic task partitioning strategy.

3. The complex matrix-vector multiplication calculation method based on the Ascend Atlas matrix calculation unit according to claim 1, characterized in that, Step S3 includes: In the case of matrix transpose, matrix-vector multiplication can be represented as: Y T = X T A； Y represents the output vector, X represents the input vector, A represents the input matrix, and T represents the transpose of the matrix.

4. The complex matrix-vector multiplication calculation method based on the Ascend Atlas matrix calculation unit according to claim 3, characterized in that, Step S4 includes: Once the matrix computation unit obtains the real and imaginary parts of the complex result vector for a task segment, the vector computation unit immediately begins combining the real and imaginary parts. Simultaneously, the matrix computation unit calculates the data for the next task segment, allowing the vector computation unit and the matrix computation unit to perform parallel computations.

Images