Complex general matrix multiplication calculation method based on separation architecture sublimation NPU

Abstract

The invention relates to the technical field of computing power optimization, in particular to a complex number general matrix multiplication calculation method based on the mercuric chloride NPU of a separation architecture, which comprises the following steps: S1, dividing a matrix multiplication calculation task into a plurality of task blocks, and distributing the task blocks to each calculation core in the mercuric chloride NPU; s2, reading data of corresponding task blocks from the global memory, performing virtual-real separation operation on the read data by a vector computing unit AIV, and writing the data subjected to the virtual-real separation operation back to the global memory; s3, performing matrix multiplication operation on the data after the virtual-real separation operation through a matrix calculation unit AIC; and S4, respectively carrying out operation of matrix vector multiplication, accumulation and virtual-real combination through a vector calculation unit AIV to obtain a final result of the corresponding task block. According to the method, the number of times of carrying data among different operation units can be reduced, the time expenditure caused by data access is reduced, the time for data access is covered, and the operation efficiency of general matrix multiplication in a mercuration NPU environment is improved.

Description

Technical Field

[0001] The present invention relates to the field of computing power optimization technology, and in particular to a complex general matrix multiplication calculation method based on a separated architecture Ascend NPU. Background Art

[0002] The operation performed by General Matrix Multiplication (GEMM) is C = αAB + βC, where α and β are scalars, and A, B, and C are dense matrices of size M×K, K×N, and M×N, respectively. This operation involves dense multiplication and addition operations, and has high computational complexity. Therefore, cutting the matrix into several sub-blocks to give full play to the hardware architecture capabilities and improve computational parallelism is an effective means to improve computational efficiency.

[0003] As a basic and powerful tool, general matrix multiplication widely supports and accelerates the core algorithms in many fields such as scientific computing, engineering applications, and machine learning. In fact, in many scientific and engineering calculations, whether it is solving linear equations, performing matrix decomposition, realizing transformation matrices, or performing complex calculations in simulations such as physics, chemistry, and biology, matrix multiplication is indispensable. In addition, in the field of artificial intelligence, especially deep learning, general matrix multiplication is the core of the forward propagation and backpropagation algorithms of neural networks. Each layer in the neural network involves a large number of matrix multiplication operations, which are used to calculate the product of weights and inputs and the output of activation functions. It can be said to be the cornerstone of artificial intelligence. It is worth noting that matrix multiplication not only occupies an important position in scientific and engineering computing, but is also widely used in many fields such as image processing, signal processing, data analysis, and financial modeling.

[0004] NPU (Neural Processing Unit) is a hardware chip specially designed to accelerate deep learning and artificial intelligence applications. It aims to solve the performance bottleneck and power consumption bottleneck of traditional computing architecture (general-purpose processor CPU and image processor GPU) when processing artificial intelligence (AI) tasks. Through dedicated hardware design optimization, NPU has demonstrated significant advantages in energy efficiency, real-time processing capabilities and overall performance, meeting the growing computing needs and increasingly complex model algorithms in AI tasks. NPU provides high-performance deep learning computing capabilities at a relatively low cost, lowering the threshold for enterprises to deploy AI computing and promoting the widespread application and popularization of AI technology in various fields.

[0005] Ascend NPU is an NPU independently developed by Huawei. It is based on the Da Vinci architecture. Each NPU consists of several computing cores, such as Figure 1As shown. Under the separation architecture, the computing units in the AI ​​Core include the CubeUnit (AIC), the Vector Unit (AIV) and the Scalar Unit (AIS), which are used for matrix multiplication, matrix-vector multiplication and matrix-scalar multiplication respectively. Multi-level caches are designed in the AI ​​Core, including L0 Buffer, L1 Buffer, Unified Buffer and Scalar Buffer. The AI ​​Cores, as well as the AIC and AIV in each AI Core, are connected to the shared global memory (GM), and the data transfer engine (MTE) is used to efficiently transfer data between the GM and the cache, as well as between caches at different levels.

[0006] In order to give full play to the hardware performance, reduce the cost of programming difficulty, and enrich the software ecosystem, major hardware manufacturers have launched a series of basic linear algebra subprograms (BLAS), such as AMD's ROCm and Nvidia's cuBLAS. In contrast, the software ecosystem in the Ascend NPU environment is not yet perfect, and the operators in the BLAS library have not yet achieved ideal performance. General matrix multiplication includes matrix multiplication, matrix-vector multiplication, and matrix-scalar multiplication operations. In order to give full play to the hardware computing power, in the Ascend NPU environment, the above three operations should be respectively assigned to AIC, AIV, and AIS. However, there is no data exchange link between AIC and AIV, and data transfer must pass through GM, which has an adverse effect on data access efficiency. At the same time, complex matrix multiplication requires the separation and combination of the real and imaginary parts of the data, and due to hardware limitations, this operation can only be completed on AIV, which further increases the number of times data is transferred through GM, bringing greater potential performance bottlenecks. Summary of the invention

[0007] In order to solve the technical problems existing in the prior art, the present invention proposes a complex general matrix multiplication calculation method based on the separated architecture Ascend NPU. The method fully considers the hardware characteristics of Ascend NPU, effectively improves the operation parallelism and hardware resource utilization, increases the data volume of a single operation instruction to reduce the total number of data transfers, optimizes the order of operation instructions, and achieves the masking of data access time while ensuring the correctness of the results, thereby improving the operation efficiency and shortening the operation time. It can solve the problems of low efficiency of general matrix multiplication operations and insufficient utilization of hardware resources in the current Ascend NPU environment.

[0008] The purpose of the present invention can be achieved by adopting the following technical solutions:

[0009] The complex general matrix multiplication calculation method based on the separated architecture Ascend NPU includes the following steps:

[0010] S1. Divide the matrix multiplication calculation task of matrix A and matrix B into several task blocks, assign the divided task blocks to each computing core AI Core in Ascend NPU, and obtain the corresponding relationship between all task blocks and each computing core AI Core;

[0011] S2. According to the correspondence between all task blocks and each AI Core, the data of the corresponding task blocks are read from the global memory, so that all AI Cores are busy at the same time. The vector computing unit AIV performs a virtual-real separation operation on the read data and writes the data after the virtual-real separation operation back to the global memory.

[0012] S3, read the data after the virtual-real separation operation from the global memory, perform matrix multiplication operation on the data after the virtual-real separation operation through the matrix calculation unit AIC, and write the intermediate result obtained by the matrix multiplication operation back to the global memory;

[0013] S4. Read the intermediate results from the global memory, and perform matrix-vector multiplication, accumulation, and virtual-real combination operations through the vector calculation unit AIV to obtain the final result of the corresponding task block; if there are still task blocks that have not participated in the operation, return to step S2, otherwise the operation ends.

[0014] Specifically, the step S1 includes:

[0015] Determine the size of a single task block according to the space size of the L1 cache, L0A cache, L0B cache, and L0C cache;

[0016] The number of task blocks obtained by decomposing the matrix multiplication calculation is determined based on the size of a single task block. A circular task allocation strategy is adopted to allocate all task blocks to the various computing cores of the Ascend NPU, and the corresponding relationship between all task blocks and each computing core AI Core is obtained.

[0017] Specifically, the step S2 includes:

[0018] The vector computing unit AIV reads the real and imaginary parts of matrix A and the real and imaginary parts of matrix B respectively through the interval reading instruction. The real and imaginary parts of matrix A are recorded as A r , A i , the real and imaginary parts of the matrix B are denoted as B r , B i ;

[0019] Before the data after the virtual-real separation operation is written back to the global memory GM, padding operation is performed on the real and imaginary parts of matrix A and the real and imaginary parts of matrix B, and all data are continuously stored in the global memory GM.

[0020] Specifically, step S3 includes:

[0021] Through multiple matrix calculation units AIC, four rounds of matrix multiplication are performed on the real and imaginary parts of matrix A and the real and imaginary parts of matrix B to obtain the intermediate results C rr , C ri , C ir , C ii ; where C rr =A r B r , C ri =A r B i , C ir =A i B r , C ii =A i B i ;

[0022] The intermediate result C rr , C ri , C ir , C ii Store to the L0C cache, and then write the intermediate results in the L0C cache back to the corresponding continuous address space in the global memory GM.

[0023] Specifically, the step S4 includes:

[0024] The vector computing unit AIV of the computing core AI Core performs matrix-vector multiplication, accumulation, and virtual-real combination operations on the intermediate results read from the global memory; in the process of data pipeline transfer, double buffering is used for acceleration, and multiple AI Cores are used for parallel computing; after multiple AI Cores complete the block calculation of the tasks they are responsible for in parallel, the calculation results are written back to the corresponding locations in the global memory.

[0025] Specifically, the vector computing unit AIV of the computing core AI Core performs matrix-vector multiplication, accumulation, and virtual-real combination operations on the intermediate results read from the global memory, including:

[0026] Perform the following operations:

[0027] C r =α(C rr -C ii )+βC r , C i =α(Cir +C ri )+βC i ,

[0028] Among them, α and β are scalars, C r , C i are the real and imaginary parts of the result matrix C respectively;

[0029] Read a task block from global memory GM C rr , C ii And store it in UB, perform C rr -C ii Operations of

[0030] Perform matrix-vector multiplication to obtain α(C rr -C ii ) and βC r ;

[0031] Perform atomic operations and calculate the result: C r =α(C rr -C ii )+βC r , temporarily store the result in UB general cache;

[0032] Read a task block of C from global memory GM ir , C ri And store it in UB general cache for C ir +C ri Operations of

[0033] Perform matrix-vector multiplication to obtain: α(C ir +C ri ) and βC i ;

[0034] Perform atomic operations and calculate the result: C i =α(C ir +C ri )+βC i , temporarily store the result in UB general cache;

[0035] C r , C i Perform virtual and real combination operation, and C r , C i The interval is stored in the corresponding space of the global memory GM to obtain the final result of the result matrix C of the current task block

[0036] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0037] The present invention proposes a complex general matrix multiplication calculation method based on the separated architecture Ascend NPU. According to the correspondence between all task blocks and each computing core AI Core, the data of the corresponding task block is read from the global memory, so that all computing cores AI Core are busy at the same time. The vector computing unit AIV performs a virtual-real separation operation on the read data, and writes the data after the virtual-real separation operation back to the global memory, thereby increasing the amount of data for a single instruction operation, reducing the number of times data is transferred between different computing units, and reducing the time overhead caused by data memory access while fully utilizing hardware resources. At the same time, the order of operation instructions is optimized, and the time spent on data memory access is covered while ensuring the correctness of the results, thereby improving the operation efficiency, shortening the operation time, and effectively improving the operation efficiency of general matrix multiplication in the Ascend NPU environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0039] Figure 1 : is the AI ​​Core architecture diagram of the separated architecture Ascend NPU in an embodiment of the present invention;

[0040] Figure 2 is a flowchart of a complex general matrix multiplication calculation method based on a separated architecture Ascend NPU in an embodiment of the present invention;

[0041] Figure 3 It is a schematic diagram of the general matrix multiplication block parallel operation in an embodiment of the present invention;

[0042] Figure 4 It is a logic diagram of the masking part operation in the embodiment of the present invention. DETAILED DESCRIPTION

[0043] The technical solution of the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is obvious that the described embodiments are part of the embodiments of the present invention, not all of the embodiments, and the implementation of the present invention is not limited to this. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0044] Embodiment 1:

[0045] The present invention proposes a complex general matrix multiplication calculation method based on the separated architecture Ascend NPU. The method fully considers the hardware characteristics of the Ascend NPU, optimizes the order of operation instructions, and achieves the masking of data access time while ensuring the correctness of the results, thereby improving the operation efficiency, operation parallelism and hardware resource utilization.

[0046] like Figure 2 As shown, the complex general matrix multiplication calculation method based on the separated architecture Ascend NPU of the present invention includes the following steps:

[0047] S1. Divide the matrix multiplication calculation task of matrix A and matrix B into several task blocks, assign the divided task blocks to each computing core AI Core in Ascend NPU, and obtain the corresponding relationship between all task blocks and each computing core AI Core.

[0048] Specifically, the size of a single task block is determined according to the space size of the L1 cache, the L0A cache, the L0B cache, and the L0C cache. Figure 3 As shown in the figure, it is a general matrix multiplication block parallel operation principle diagram. A result matrix block C is of size M0×N0, which is obtained by multiplying two strip matrix blocks A and B of size M0×K and K×N0 respectively. Considering the limitation of cache space size, the strip matrix blocks are often split into The resulting matrix blocks can be converted to The accumulation of block products of block matrices (sizes M0×K0 and K0×N0 respectively). Assuming that the cache size of L0A, L0B, and L0C is X bytes, the data type is single-precision complex, and a double buffering strategy is adopted, then the block sizes M0, N0, and K0 need to meet the following restrictions at the same time:

[0049] The number of task blocks obtained by decomposing the matrix multiplication calculation is determined according to the size of a single task block. A task allocation strategy of circular allocation is adopted to allocate all task blocks to each computing core AI Core of the NPU, and the corresponding relationship between each task block and each computing core AI Core is obtained. Assuming that the number of AI Cores is R, and the numbers are i∈{0, 1, 2, ..., R-1}, the tasks will be cyclically allocated to each computing core, that is, task 0 will be allocated to computing core AI Core 0, task 1 will be allocated to computing core AI Core 1, and so on. When there are still task blocks that have not been allocated to the last computing core, the computing tasks will be allocated again from AI Core 0. The task allocation strategy of circular allocation can improve the L2 cache hit rate of Ascend NPU, improve the efficiency of data transfer, and maximize the workload of each computing core, which helps to improve the parallelism of calculations.

[0050] Assuming that matrices A, B, and C are all non-transposed matrices with dimensions M=N=K=1024, the matrix multiplication has the following representation:

[0051]

[0052] In this embodiment, the Atlas 300T Pro training card based on the Ascend NPU architecture (hereinafter referred to as the training card) is taken as an example. The training card has 30 computing cores AI Core. Here, the block size of M0=N0=K0=128 is taken as an example. This block size meets the cache size limit and also meets the data alignment requirements. It can be concluded that under the current block, the result matrix C can be divided into tasks, apply for 30 AI Cores to work, and distribute these 64 tasks to 30 AI Cores in a loop. When distributing tasks in a loop, task 0 will be assigned to AI Core 0, task 1 will be assigned to AI Core 1, and so on. Task 29 will be assigned to AI Core 29, and then tasks 30-59 will be assigned from AI Core 0 again in sequence, and tasks 60-63 will be assigned from AI Core 0 again in sequence. The result matrix of task 0 and task 1 is divided into blocks as follows:

[0053]

[0054] S2. According to the correspondence between all task blocks and each AI Core, the data of the corresponding task blocks are read from the global memory, so that all AI Cores are busy at the same time. The vector computing unit AIV performs virtual-real separation operation on the read data and writes the data after virtual-real separation operation back to the global memory. By increasing the single operation, all AI Cores can work, increase the single operation data volume, and avoid idle resources.

[0055] S21, according to the corresponding relationship between the divided computing tasks and each computing core AI Core, read the data of the corresponding number of task blocks from the global memory, and the vector computing unit AIV of the AI ​​Core reads the real part and imaginary part of the matrix A and the real part and imaginary part of the matrix B respectively through the interval reading instruction. The real part and imaginary part of the matrix A are respectively recorded as A r , A i , the real and imaginary parts of the matrix B are denoted as B r , B i .

[0056] S22. Before writing the data after the virtual-real separation operation back to the global memory GM, perform padding operation on the real and imaginary parts of matrix A and the real and imaginary parts of matrix B, and store all data continuously in the global memory GM.

[0057] In this embodiment, the padding operation includes: modifying the spacing between different data columns of the matrix so that the data involved in the calculation are aligned to the nearest integer multiple of 512 bytes, which can significantly improve the data handling efficiency, and at the same time converting the discrete matrix storage into 512-byte aligned continuous storage, so that the storage mode of the data after the virtual-real separation operation in the global memory GM is modified to continuous storage, so that the block reading can be realized continuously in the subsequent calculation process, thereby speeding up the reading speed of the data block. The specific steps include: calculating the difference between the data size contained in the current matrix and the nearest integer multiple of 512 bytes; adding a corresponding number of 0s at the end of the matrix data so that the matrix can be aligned to 512 bytes, so as to improve the handling efficiency without affecting the correctness of the calculation results. After completing the padding operation, all data will be stored continuously in the global memory GM.

[0058] In this embodiment, the 30 AI Cores of the training card will read matrices A and B in sequence and perform virtual-real separation operations, read the real part and the imaginary part respectively through interval reading instructions, and perform padding operations on the data at the same time, and store them in a continuous global memory space in sequence.

[0059] S3. Read the data after the virtual-real separation operation from the global memory, perform matrix multiplication operation on the data after the virtual-real separation operation through the matrix calculation unit AIC, and write the intermediate result obtained by the matrix multiplication operation back to the global memory.

[0060] The matrix calculation unit AIC of the computing core AI Core is responsible for performing matrix multiplication operations on the separated real and imaginary data. A total of four rounds of matrix multiplication are performed, and the block matrix is ​​read from the global memory. After multiple transfers from the global memory to the L1 cache and then to the L0 cache, the calculation is performed to obtain the corresponding intermediate results, and the results are continuously stored in the global memory GM for fast reading in subsequent steps. Multiple computing cores AI Core are used simultaneously during the multiplication process, and multiple computing cores AI Core parallel calculations increase computing efficiency. After multiple computing cores AI Core parallelly calculate and complete the block calculations for their respective tasks, the calculation results are written back to the corresponding locations in the global memory, and the overall matrix multiplication calculation is completed.

[0061] Specifically, the matrix calculation unit AIC of the computing core AI Core is responsible for performing four rounds of matrix multiplication operations on the separated real and imaginary data to obtain the intermediate results C rr , C ri , C ir , C ii , where C rr =A r B r , C ri =A r B i , C ir =A i B r , C ii =A i B i ; C rr represents the product of the real part of matrix A and the real part of matrix B, C ri represents the product of the real part of matrix A and the imaginary part of matrix B, C ir represents the product of the imaginary part of matrix A and the real part of matrix B, C ii Represents the product of the imaginary part of matrix A and the imaginary part of matrix B.

[0062] C rr =A r B r As an example, first we need to solve the C rr The task block number is used to offset the address. r With B r Load from global memory into L0A and L0B cache respectively, and then A r With B r Perform matrix multiplication and store the result in the L0C cache; finally write the data in the L0C cache back to the global memory GM rr The corresponding continuous address space. ri , C ir, C ii The steps are similar to those of rr , C ri , C ir , C ii They will be stored continuously in GM to facilitate quick reading in subsequent steps.

[0063] C rr =A r B r Taking task 0 in the operation as an example, the computational task that needs to be completed can be expressed as:

[0064]

[0065] Specifically, considering the cache space limitation, the matrix A r , B r It is also necessary to further decompose in the K direction Each C rr Each task block performs 8 matrix multiplication accumulations.

[0066] S4. Read the intermediate results from the global memory, and perform matrix-vector multiplication, accumulation, and virtual-real combination operations through the vector calculation unit AIV to obtain the final result of the corresponding task block; if there are still task blocks that have not participated in the operation, return to step S2, otherwise the operation ends.

[0067] Specifically, the vector computing unit AIV of the computing core AI Core performs matrix-vector multiplication, accumulation, and virtual-real combination operations on the intermediate results read from the global memory; in the process of data pipeline transportation, double buffering is used for acceleration, and multiple AI Cores are used for parallel computing. Multiple AI Core parallel computing can increase computing efficiency; after multiple AI Cores complete the task block calculations for which they are responsible, the calculation results are written back to the corresponding locations in the global memory, which means that the complex matrix multiplication calculations currently executed synchronously by all AI Cores are completed. At this time, if there are still task blocks that have not participated in the calculation, return to step S2; otherwise, the whole process ends.

[0068] In this embodiment, the AIV of the AI ​​Core is responsible for performing matrix-vector multiplication, accumulation, and virtual-real combination operations on the intermediate results obtained in step S3, and performing the following main operations:

[0069] C r =α(C rr -C ii )+βC r ;

[0070] C i =α(C ir +Cri )+βC i ;

[0071] Among them, α and β are scalars, C r , C i are the real and imaginary parts of the result matrix C respectively.

[0072] The specific steps include:

[0073] S41, read a task block C from the global memory GM rr , C ii And store it in UB, perform C rr -C ii Operations of

[0074] S42, perform matrix-vector multiplication to obtain α(C rr -C ii ) and βC r ;

[0075] S43, perform atomic operation and get result: C r =α(C rr -C ii )+βC r , temporarily store the result in UB general cache;

[0076] S44, read a task block C from the global memory GM ir , C ri And store it in UB general cache for C ir +C ri Operations of

[0077] S45, perform matrix-vector multiplication to obtain α(C ir +C ri ) and βC i ;

[0078] S46, perform atomic operation and get result: C i =α(C ir +C ri )+βC i , temporarily store the result in UB general cache;

[0079] S47, to C r , C i Perform virtual and real combination operation, and C r , C i The intervals are stored in the corresponding space of the global memory GM to obtain the final result of the result matrix C of the current task block; at this time, if there are still task blocks that have not participated in the calculation, return to step S2; otherwise, the whole process ends.

[0080] Atomic operation is a commonly used synchronization method, the main purpose of which is to ensure the correctness of the results. Atomic operation allows each memory address to be accessed by only one thread at a time, serializing the access of parallel threads, and avoiding calculation errors caused by multiple threads executing simultaneously reading or writing the same memory address.

[0081] like Figure 4 As shown in the figure, it is a logical diagram for masking the time taken for some operations. When the amount of calculation is large, the number of physical cores determines that the chip cannot complete all the calculation tasks at one time. In this case, it is possible to consider parallelizing some of the calculation processes to mask the time taken for some operations. Specifically, (i) the vector calculation unit AIV first performs virtual-real separation and padding operations on a part of the matrix. After the operation is completed, it sends the synchronization instruction sync(0), and then immediately performs virtual-real separation and padding operations on another part of the matrix. After completing the next part, it sends the synchronization instruction sync(2). (ii) When the matrix calculation unit AIC receives the synchronization instruction sync(0), it starts the matrix multiplication operation of the corresponding part and obtains C rr , C ri , C ir , C ii Intermediate result, after completing the operation, send the synchronization instruction sync(1). Similarly, when receiving the synchronization instruction sync(2), AIC will start to execute the corresponding matrix multiplication operation of another part after completing the previous calculation task, and send the synchronization instruction sync(3) after completion. (iii) After the vector calculation unit AIV receives the sync(1) instruction, it executes the series of operations in step S4 on the completed matrix multiplication part, waits for the synchronization instruction sync(3) after the operation is completed, and then performs the series of operations in step S4 on the other part of the matrix multiplication result. In this way, AIC executes the matrix multiplication operation of the first part and AIV executes the virtual-real separation padding operation of the second part in complete parallel, and AIC executes the matrix multiplication operation of the second part and AIV executes the step S4 series of operations of the first part in complete parallel, so as to cover the time taken by some operations during the operation.

[0082] The present invention adopts a fusion strategy of the matrix computing unit AIC and the vector computing unit AIV, and the matrix multiplication and matrix-vector multiplication operations in the general matrix multiplication are respectively handed over to the matrix computing unit AIC and the vector computing unit AIV to complete, ensuring that the hardware computing power of the Ascend NPU is fully utilized. By increasing the amount of data in a single instruction operation, the number of times data is transferred between different operation units is reduced, while fully utilizing hardware resources, the time overhead caused by data memory access is reduced, and the adverse effects of data memory access delay on operation efficiency are avoided. The present invention also optimizes the order of different operation instructions to cover up the time spent on data memory access while ensuring the correctness of the results, reduce the calculation time, further improve the operation efficiency of general matrix multiplication, and further avoid the adverse effects of data memory access delay on operation efficiency.

[0083] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A complex general matrix multiplication calculation method based on a separated architecture Ascend NPU, characterized in that: The following steps are involved: S1. Divide the matrix multiplication calculation task of matrix A and matrix B into several task blocks, assign the divided task blocks to each computing core AI Core in Ascend NPU, and obtain the corresponding relationship between all task blocks and each computing core AI Core; S2. According to the correspondence between all task blocks and each AI Core, the data of the corresponding task blocks are read from the global memory, so that all AI Cores are busy at the same time. The vector computing unit AIV performs a virtual-real separation operation on the read data and writes the data after the virtual-real separation operation back to the global memory. S3, read the data after the virtual-real separation operation from the global memory, perform matrix multiplication operation on the data after the virtual-real separation operation through the matrix calculation unit AIC, and write the intermediate result obtained by the matrix multiplication operation back to the global memory; S4. Read the intermediate results from the global memory, and perform matrix-vector multiplication, accumulation, and virtual-real combination operations through the vector calculation unit AIV to obtain the final result of the corresponding task block; determine whether there are still task blocks that have not been calculated. If there are still task blocks that have not been calculated, return to step S2; otherwise, the calculation ends.

2. The complex general matrix multiplication calculation method based on the separated architecture Ascend NPU according to claim 1 is characterized in that: The step S1 comprises: Determine the size of a single task block according to the space size of the L1 cache, L0A cache, L0B cache, and L0C cache; The number of task blocks obtained by decomposing the matrix multiplication calculation is determined based on the size of a single task block. A circular task allocation strategy is adopted to allocate all task blocks to the various computing cores of the Ascend NPU, and the corresponding relationship between all task blocks and each computing core AI Core is obtained.

3. The complex general matrix multiplication calculation method based on the separated architecture Ascend NPU according to claim 2 is characterized in that: The step S2 comprises: The vector computing unit AIV reads the real and imaginary parts of matrix A and the real and imaginary parts of matrix B respectively through the interval reading instruction. The real and imaginary parts of matrix A are recorded as A r , A i , the real and imaginary parts of the matrix B are denoted as B r , B i ; Before the data after the virtual-real separation operation is written back to the global memory GM, padding operation is performed on the real and imaginary parts of matrix A and the real and imaginary parts of matrix B, and all data are continuously stored in the global memory GM.

4. The complex general matrix multiplication calculation method based on the separated architecture Ascend NPU according to claim 3 is characterized in that: The padding operation is performed on the real and imaginary parts of the matrix A and the real and imaginary parts of the matrix B, and all data are continuously stored in the global memory GM, including: Modify the spacing between different data columns of the matrix so that the data involved in the calculation are aligned to the nearest integer multiple of 512 bytes; The discrete matrix storage is converted into 512-byte aligned continuous storage, so that the data after the virtual-real separation operation is stored in the global memory GM in a continuous manner.

5. The complex general matrix multiplication calculation method based on the separated architecture Ascend NPU according to claim 3 is characterized in that: The step S3 comprises: Through multiple matrix calculation units AIC, four rounds of matrix multiplication are performed on the real and imaginary parts of matrix A and the real and imaginary parts of matrix B to obtain intermediate results C rr , C ri , C ir , C ii Among them, C rr =A r B r , C ri =A r B i , C ir =A i B r , C ii =A i B i ; The intermediate result C rr , C ri , C ir , C ii Store to the L0C cache, and then write the intermediate results in the L0C cache back to the corresponding continuous address space in the global memory GM.

6. The complex general matrix multiplication calculation method based on the separated architecture Ascend NPU according to claim 5 is characterized in that: The step S4 comprises: The vector computing unit AIV of the computing core AI Core performs matrix-vector multiplication, accumulation, and virtual-real combination operations on the intermediate results read from the global memory; in the process of data pipeline transfer, double buffering is used for acceleration, and multiple computing cores AICore are used for parallel computing; after multiple computing cores AICore complete the block calculation of the task they are responsible for in parallel, the calculation results are written back to the corresponding locations in the global memory.

7. The complex general matrix multiplication calculation method based on the separated architecture Ascend NPU according to claim 6 is characterized in that: The vector computing unit AIV of the computing core AI Core performs matrix-vector multiplication, accumulation, and virtual-real combination operations on the intermediate results read from the global memory, including: The following operations are performed through the vector computing unit AIV of the computing core AICore: C r =α(C rr -C ii )+βC r 、C i =α(C ir +C ri )+βC i , Among them, α and β are scalars, C r , C i are the real and imaginary parts of the result matrix C respectively; Read a task block of C from global memory GM rr , C ii And store it in UB, perform C rr -C ii Operations of Perform matrix-vector multiplication to obtain: α(C rr -C ii ) and βC r ; Perform atomic operations and calculate the result: C r =α(C rr -C ii )+βC r , temporarily store the result in UB general cache; Read a task block of C from global memory GM ir , C ri And store it in UB general cache for C ir +C ri Operations of Perform matrix-vector multiplication to obtain: α(C ir +C ri ) and βC i ; Perform atomic operations and calculate the result: C i =α(C ir +C ri )+βC i , temporarily store the result in UB general cache; C r , C i Perform virtual and real combination operation, and C r , C i The intervals are stored in the corresponding space of the global memory GM to obtain the final result of the result matrix C of the current task block.

Images