Shared memory allocation method, device, medium, equipment and product for matrix calculation

Abstract

The application discloses a shared memory allocation method and device for matrix calculation, a medium, equipment and products, and the method comprises the following steps: when the second storage body with smaller storage capacity is sufficient to accommodate the first multiplier matrix, allocating the first multiplier matrix with larger memory requirement to the second storage body, and allocating the second multiplier matrix to the first storage body; checking whether the storage body with the largest remaining space is sufficient to accommodate the bias matrix; if yes, allocating the bias matrix to the storage body with the largest remaining space and performing matrix fusion calculation; if not, if the memory requirement of the bias matrix is not greater than the total remaining space, dividing the bias matrix into two sub-matrices, and allocating the two sub-matrices to the first storage body and the second storage body respectively to perform matrix fusion calculation; and allocating the result matrix obtained through matrix fusion calculation to the memory occupation space of the bias matrix through multiplexing operation. The application can maximize the utilization of the remaining space of the shared memory, and significantly improve the success rate of memory allocation.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a shared memory allocation method, apparatus, computer-readable storage medium, electronic device, and computer program product for matrix calculation. Background Technology

[0002] The implementation of General Matrix Multiply (GEMM) operators (including post-processing operations) typically employs a tiling strategy, breaking down large-scale matrix operations into multiple smaller sub-operations performed in shared memory.

[0003] As the complexity of the model increases, the size of the matrix blocks also increases accordingly. Due to the inherent hardware resources of shared memory, memory allocation failures are very likely to occur, thus preventing the smooth execution of matrix operations. Summary of the Invention

[0004] The purpose of this invention is to provide a shared memory allocation method, apparatus, computer-readable storage medium, electronic device, and computer program product for matrix computation. By logically continuating the remaining space of the storage and reusing the memory space occupied by the bias matrix, the remaining space of the shared memory can be maximized and the memory allocation success rate can be significantly improved.

[0005] A first aspect of the present invention provides a shared memory allocation method for matrix computation, wherein the shared memory is divided into a first storage bank and a second storage bank; wherein the storage capacity of the first storage bank is greater than the storage capacity of the second storage bank; the method includes:

[0006] When the memory requirement of the first multiplier matrix is ​​not greater than the storage capacity of the second storage bank, the first multiplier matrix is ​​allocated to the starting position of the second storage bank, and the second multiplier matrix is ​​allocated to the starting position of the first storage bank; wherein, the memory requirement of the first multiplier matrix is ​​greater than or equal to the memory requirement of the second multiplier matrix;

[0007] Check if the largest remaining storage space is sufficient to accommodate the bias matrix;

[0008] If so, the bias matrix is ​​allocated to the storage volume with the largest remaining space and matrix fusion calculation is performed;

[0009] If not, compare whether the memory requirement of the bias matrix is ​​greater than the total remaining space of the shared memory, and if the memory requirement of the bias matrix is ​​not greater than the total remaining space, divide the bias matrix into two sub-matrices and allocate them to the first storage bank and the second storage bank respectively to perform the matrix fusion calculation.

[0010] Based on the matrix fusion calculation, the corresponding result matrix is ​​obtained, and through a reuse operation, the result matrix is ​​allocated to the memory space occupied by the bias matrix.

[0011] Optionally, after comparing whether the memory requirement of the bias matrix is ​​greater than the total remaining space of the shared memory, the method further includes:

[0012] If the memory requirement of the bias matrix exceeds the total remaining space, the memory allocation is deemed to have failed, and the execution of the matrix fusion calculation is stopped.

[0013] Optionally, before checking whether the storage bank with the largest remaining space is sufficient to accommodate the bias matrix, the method further includes:

[0014] When the memory requirement of the first multiplier matrix is ​​greater than the storage capacity of the second storage bank, check whether the first storage bank is sufficient to hold the first multiplier matrix, and check whether the second storage bank is sufficient to hold the second multiplier matrix.

[0015] If all are true, then the first multiplier matrix is ​​assigned to the beginning of the first storage block, and the second multiplier matrix is ​​assigned to the beginning of the second storage block;

[0016] If at least one of them is not true, then memory allocation is deemed to have failed, and the execution of the matrix fusion calculation is stopped.

[0017] Optionally, the matrix fusion calculation includes: matrix multiplication between the first multiplier matrix and the second multiplier matrix, and cumulative calculation related to N bias matrices; wherein, N≥1.

[0018] Optionally, when N > 1, the cumulative calculation related to the N bias matrices is performed through the following steps:

[0019] By reusing the memory space multiple times, each bias matrix is ​​loaded in a preset order to perform the corresponding cumulative calculations in sequence; wherein, the capacity of the memory space is equal to the memory requirement of any bias matrix.

[0020] Optionally, the first and second storage banks are each configured with independent memory access channels.

[0021] A second aspect of the present invention provides a shared memory allocation device for matrix computation, wherein the shared memory is divided into a first storage bank and a second storage bank; wherein the storage capacity of the first storage bank is greater than the storage capacity of the second storage bank; the device includes:

[0022] The first allocation module is used to allocate the first multiplier matrix to the starting position of the second storage bank and the second multiplier matrix to the starting position of the first storage bank when the memory requirement of the first multiplier matrix is ​​not greater than the storage capacity of the second storage bank; wherein the memory requirement of the first multiplier matrix is ​​greater than or equal to the memory requirement of the second multiplier matrix.

[0023] The capacity check module is used to check whether the largest remaining storage space is sufficient to accommodate the bias matrix;

[0024] The second allocation module is used to allocate the bias matrix to the storage volume with the largest remaining space and perform matrix fusion calculation if the condition is met.

[0025] The third allocation module is used to compare whether the memory requirement of the bias matrix is ​​greater than the total remaining space of the shared memory if the memory requirement of the bias matrix is ​​not greater than the total remaining space, and to divide the bias matrix into two sub-matrices and allocate them to the first storage bank and the second storage bank respectively to perform the matrix fusion calculation.

[0026] The fourth allocation module is used to obtain the corresponding result matrix based on the matrix fusion calculation, and allocate the result matrix to the memory space occupied by the bias matrix through a reuse operation.

[0027] A third aspect of the present invention provides a computer-readable storage medium comprising a stored computer program; wherein, when the computer program is executed, it controls the device in which the computer-readable storage medium is located to perform the shared memory allocation method for matrix calculation as described in any embodiment of the first aspect.

[0028] A fourth aspect of the present invention provides a computer program product including computer instructions that, when executed by a processor, implement the shared memory allocation method for matrix calculation as described in any embodiment of the first aspect.

[0029] A fifth aspect of the present invention provides an electronic device including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the shared memory allocation method for matrix calculation as described in any embodiment of the first aspect.

[0030] Compared with the prior art, the embodiments of the present invention provide a shared memory allocation method, apparatus, computer-readable storage medium, electronic device, and computer program product for matrix computation. The embodiments of the present invention first attempt to allocate the second multiplier matrix (smaller matrix) to a first storage bank with a larger capacity, so that the first storage bank can reserve more contiguous remaining space for the bias matrix, thereby reducing the probability of storing the bias matrix in multiple storage banks; by logically integrating the total remaining space of shared memory, allocation can be carried out as long as the total remaining space is greater than or equal to the memory requirement of the bias matrix, which can significantly improve the success rate of shared memory allocation for GEMM operators under the large block Tiling strategy and avoid resource waste caused by space fragmentation; in addition, the result matrix reuses the memory space occupied by the bias matrix, which can significantly reduce the resource occupation of shared memory and further improve memory utilization efficiency, thereby reserving more memory space for matrix computation with larger blocks. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of an embodiment of the preference storage alternation mechanism provided by the present invention;

[0032] Figure 2 This is a schematic diagram of another embodiment of the preference storage alternation mechanism provided by the present invention;

[0033] Figure 3 This is a flowchart illustrating an embodiment of the shared memory allocation method for matrix computation provided by the present invention.

[0034] Figure 4 This is a flowchart illustrating another embodiment of the shared memory allocation method for matrix computation provided by the present invention;

[0035] Figure 5 This is a schematic diagram of an embodiment of the shared memory allocation device for matrix calculation provided by the present invention;

[0036] Figure 6 This is a schematic diagram of the structure of an embodiment of the electronic device provided by the present invention;

[0037] Figure 7 This is a schematic diagram of the structure of an embodiment of the artificial intelligence processor provided by the present invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] It should be noted that matrix multiplication (D = A × B) is a fundamental operation in many computational fields. General Matrix Multiply (GEMM) extends this fundamental operation to a fused multiplication and addition operation, such as D = α × A × B + β × C; where A and B are multiplier matrices, C is the bias matrix, D is the result matrix, and α and β are scalar coefficients. By introducing scalar coefficients and the bias matrix, the efficiency and flexibility of computational tasks can be improved. Matrices A, B, C, and D all require shared memory (GSM) storage space; therefore, reasonable allocation of shared memory resources is necessary for successful execution of GEMM operators.

[0040] A certain artificial intelligence chip has a shared memory space of 224KB, physically divided into a low-address bank and a high-address bank, with a 128KB boundary. The low bank has a storage capacity of 128KB, and the high bank has a storage capacity of 96KB. During the research process, the inventors discovered that due to hardware characteristics, placing matrices A and B in different banks yields better performance benefits. Considering the capacity difference between the low and high banks, and the fact that matrices C and D also need to be placed in shared memory, the inventors initially used a "preferred bank alternation" mechanism for shared memory allocation. However, they found that this mechanism had technical defects such as low memory allocation success rate and insufficient space utilization in GEMM operation scenarios with large block tiling strategies.

[0041] Specifically, the memory requirements of matrix A and matrix B are first compared. The matrix with the larger memory requirement is placed in the low bank (at address 0), and the other matrix is ​​placed in the high bank (at address 128KB). That is, the larger matrix is ​​placed in the lower bank with greater capacity. After allocating banks for matrices A and B and updating the remaining space in each bank, a preferred bank identifier (denoted as `prefer`) is introduced. Initially, the preference is set to the lower bank (i.e., `prefer=0`). For the allocation of matrix C, it is first checked whether the remaining space in the current preferred bank meets its memory requirements. If it does, matrix C is allocated to that preferred bank; otherwise, it is allocated to another bank. Simultaneously, to achieve "alternating preferred banks" allocation, after the position of matrix C is determined, the preferred bank of the subsequent matrix to be allocated (such as matrix D) is set to the opposite bank of the bank containing matrix C. For example, if matrix C is placed in the lower bank, the preferred bank of matrix D automatically switches to the high bank (i.e., `prefer=1` for matrix D). In particular, if the remaining space in both banks is insufficient to accommodate matrix C alone, the memory allocation is considered a failure. The allocation logic of matrix D is the same as that of matrix C, and it also follows the above-mentioned "preference bank alternation" mechanism. When the remaining space of any bank cannot meet its memory requirements alone, it is also judged as a memory allocation failure.

[0042] like Figure 1 The diagram shown is a schematic representation of an embodiment of the preferred storage alternation mechanism provided by this invention. It illustrates a successful memory allocation. The memory spaces occupied by matrix A (denoted as buf_A) and matrix C (denoted as buf_C) are both located in the low bank; the memory spaces occupied by matrix B (denoted as buf_B) and matrix D (denoted as buf_D) are both located in the high bank, and both the low and high banks have some remaining space. Figure 2 The diagram shown is a schematic of another embodiment of the preferred storage bank alternation mechanism provided by the present invention, which is an example of memory allocation failure. In the scenario of large-block matrix computation, after allocating matrices A, B, and C, the remaining space in both the low bank and high bank is insufficient to independently meet the memory requirements of matrix D, but the total remaining space is sufficient to allocate matrix D. This is a phenomenon of wasted memory space.

[0043] Based on this, the inventors further proposed another shared memory allocation method for matrix calculations, which can maximize the use of the remaining space of shared memory and significantly improve the memory allocation success rate.

[0044] See Figure 3This is a flowchart illustrating an embodiment of the shared memory allocation method for matrix computation provided by the present invention.

[0045] A first aspect of the present invention provides a shared memory allocation method for matrix computation, wherein the shared memory is divided into a first storage bank and a second storage bank; wherein the storage capacity of the first storage bank is greater than the storage capacity of the second storage bank; the method includes steps S1 to S5:

[0046] Step S1: When the memory requirement of the first multiplier matrix is ​​not greater than the storage capacity of the second storage bank, the first multiplier matrix is ​​allocated to the starting position of the second storage bank, and the second multiplier matrix is ​​allocated to the starting position of the first storage bank; wherein, the memory requirement of the first multiplier matrix is ​​greater than or equal to the memory requirement of the second multiplier matrix;

[0047] Step S2: Check whether the storage cell with the largest remaining space is sufficient to accommodate the bias matrix;

[0048] Step S3: If yes, then allocate the bias matrix to the storage volume with the largest remaining space and perform matrix fusion calculation;

[0049] Step S4: If not, compare whether the memory requirement of the bias matrix is ​​greater than the total remaining space of the shared memory, and if the memory requirement of the bias matrix is ​​not greater than the total remaining space, divide the bias matrix into two sub-matrices and allocate them to the first storage bank and the second storage bank respectively to perform the matrix fusion calculation.

[0050] Step S5: Based on the matrix fusion calculation, the corresponding result matrix is ​​obtained, and through a reuse operation, the result matrix is ​​allocated to the memory space occupied by the bias matrix.

[0051] It should be noted that in a certain artificial intelligence chip, the first storage bank is the low bank (storage capacity of 128KB), and the second storage bank is the high bank (storage capacity of 96KB). In this embodiment of the invention, matrix A and matrix B are still placed in different banks, and the remaining space of the low bank and the high bank is integrated into a logically contiguous space (total remaining space of GSM). Allocation is no longer based on the remaining space of a single bank. As long as the total remaining space is greater than or equal to the memory requirement of the bias matrix, the allocation is successful. In addition, buf_C is used to store the bias matrix loaded from global memory to shared memory, and buf_D is used to store the result matrix D written from the register to shared memory. Since the use of buf_C and buf_D is sequential (the result matrix is ​​written to shared memory only after the bias matrix has been calculated), and their memory capacity requirements are exactly the same, they can reuse the same shared memory space.

[0052] Specifically, such as Figure 4 The diagram shown is a flowchart of another embodiment of the shared memory allocation method for matrix computation provided by this invention, illustrating the implementation process of matrix fusion computation for GEMM D = α × A × B + β × C. First, the sizes (memory requirements) of matrices A and B are determined to identify the smaller and larger matrices. Next, it is checked whether the high bank is sufficient to accommodate the larger matrix; if so, the larger matrix (i.e., the first multiplier matrix) is placed at the beginning of the high bank, and the smaller matrix (i.e., the second multiplier matrix) is placed at the beginning of the low bank; otherwise, the larger matrix is ​​placed at the beginning of the low bank, and the smaller matrix is ​​placed at the beginning of the high bank. This embodiment of the invention prioritizes "deploying the smaller matrix in the low bank," thus fully utilizing the large capacity of the low bank and reserving more contiguous remaining space for subsequent bias matrices (such as matrix C), thereby reducing the probability of the bias matrix being stored across multiple banks and improving memory access efficiency.

[0053] After allocating multiplier matrices A and B, the remaining space in the low bank, high bank, and GSM is updated. Next, it is checked whether the bank with the largest remaining space is sufficient to accommodate the bias matrix (e.g., matrix C). If so, the bias matrix is ​​directly allocated to that bank, and matrix fusion calculation is performed. If not, the total remaining space is further checked to ensure it is sufficient to accommodate the bias matrix. If the memory requirement of the bias matrix is ​​no greater than the total remaining space, the bias matrix is ​​divided into two sub-matrices and allocated to the remaining space in the first and second memory banks respectively. This avoids physical cross-bank contiguous access while utilizing fragmented space for allocation.

[0054] Finally, after obtaining the result matrix D based on matrix fusion calculation, the result matrix D is directly allocated to the memory space occupied by the bias matrix through space reuse operation (since the bias matrix has completed the operation, its memory space can be reused by the result matrix), which can further improve the utilization efficiency of shared memory. Specifically, since the bias matrix and the result matrix have the same data type and the same matrix size in GEMM operation, this provides the basic condition for them to share a shared memory space. In related technologies, the bias matrix and the result matrix adopt independent memory space design and are independently controlled; however, in this embodiment of the invention, the bias matrix and the result matrix share the same memory space and are controlled by a unified synchronization signal. Only after the result matrix is ​​unloaded from shared memory to global memory can the next round of loading operation of the bias matrix be executed. This ensures the data security of the bias matrix and the result matrix when sharing space (avoiding calculation errors caused by data overwriting).

[0055] As can be seen from the above, compared with the "preference bank alternation" mechanism, on the one hand, the embodiments of the present invention first attempt to allocate the smaller matrix (the second multiplier matrix) to the larger first storage bank (low bank), so that the first storage bank can reserve more contiguous remaining space for the bias matrix, thereby reducing the probability of the bias matrix being stored in multiple banks; on the other hand, by logically integrating the total remaining space, allocation can be made as long as the total remaining space is greater than or equal to the memory requirement of the bias matrix, which can significantly improve the success rate of shared memory allocation for GEMM operators under the large block Tiling strategy, avoid resource waste caused by space fragmentation, and does not require additional maintenance of preference bank identifiers; in addition, the serial reuse of buf_C and buf_D can significantly reduce the resource occupation of shared memory without affecting the operation process, and further improve memory utilization efficiency, thereby reserving more memory space for matrix operations with larger blocks.

[0056] In an optional embodiment, after comparing whether the memory requirement of the bias matrix is ​​greater than the total remaining space of the shared memory, the method further includes:

[0057] If the memory requirement of the bias matrix exceeds the total remaining space, the memory allocation is deemed to have failed, and the execution of the matrix fusion calculation is stopped.

[0058] It should be noted that if the total remaining space of GSM obtained by integrating all the remaining space in the low bank and high bank is still insufficient to accommodate any bias matrix involved in the matrix fusion calculation, it is directly determined that the shared memory allocation has failed, and the execution process of the matrix fusion calculation is terminated in a timely manner.

[0059] In an optional embodiment, before checking whether the storage bank with the largest remaining space is sufficient to accommodate the bias matrix, the method further includes:

[0060] When the memory requirement of the first multiplier matrix is ​​greater than the storage capacity of the second storage bank, check whether the first storage bank is sufficient to hold the first multiplier matrix, and check whether the second storage bank is sufficient to hold the second multiplier matrix.

[0061] If all are true, then the first multiplier matrix is ​​assigned to the beginning of the first storage block, and the second multiplier matrix is ​​assigned to the beginning of the second storage block;

[0062] If at least one of them is not true, then memory allocation is deemed to have failed, and the execution of the matrix fusion calculation is stopped.

[0063] It should be noted that when the memory requirement of the first multiplier matrix (the larger matrix) exceeds the storage capacity of the high bank, a double check is performed—first, it checks whether the low bank is large enough to fully accommodate the first multiplier matrix (the larger matrix), and second, it checks whether the high bank is large enough to fully accommodate the second multiplier matrix (the smaller matrix). If both double checks result in "yes", then according to the fallback principle of "matching large matrices with large-capacity storage", the first multiplier matrix is ​​allocated to the beginning position of the low bank, and the second multiplier matrix is ​​allocated to the beginning position of the high bank to ensure the complete storage of core computational data. If at least one of the double checks results in "no", then it is directly determined that the shared memory allocation has failed, and the execution of subsequent matrix fusion calculations is immediately stopped.

[0064] In an optional embodiment, the matrix fusion calculation includes: matrix multiplication between the first multiplier matrix and the second multiplier matrix, and cumulative calculation related to N bias matrices; wherein, N≥1.

[0065] Furthermore, when N > 1, the cumulative calculation related to the N bias matrices is performed through the following steps:

[0066] By reusing the memory space multiple times, each bias matrix is ​​loaded in a preset order to perform the corresponding cumulative calculations in sequence; wherein, the capacity of the memory space is equal to the memory requirement of any bias matrix.

[0067] It should be noted that, when calculating the bias matrix C in stages, the formula for matrix fusion is D = α × A × B + β1 × C1 + β2 × C2 + … + β N ×C N ; where β1~β N For scalar coefficients, C1~C N This is the bias matrix. When multiple bias matrices need to be accumulated, the specific implementation process of this embodiment is as follows:

[0068] First, perform matrix multiplication between the first and second multiplier matrices to obtain the basic product and store it in the accumulator `acc`. Simultaneously, determine the memory space (i.e., `buf_C`) used to load the bias matrix. Then, through timing synchronization control, ensure that after the previous bias matrix completes its accumulation calculation, the current bias matrix data in `buf_C` is cleared (or `buf_C` is directly reused via overwriting), before loading the next bias matrix and performing the accumulation calculation (i.e., `acc = acc + β`). k+1 ×C k+1(k+1 is the index of the next bias matrix). After all N bias matrices have been loaded and accumulated, the accumulator acc stores the result matrix D of the matrix fusion calculation. buf_C is reused again to unload the result matrix D from the register to the GSM.

[0069] In an optional embodiment, the first memory bank and the second memory bank are each configured with independent memory access channels.

[0070] It's important to note that in a certain AI chip, read / write operations in the low bank and high bank can be executed in parallel through their respective dedicated memory access channels, without needing to compete for the same channel resources. Therefore, in scenarios where the bias matrix needs to be stored across multiple banks, the computing unit can read one sub-matrix of the bias matrix through the low bank's memory access channel and simultaneously read another sub-matrix through the high bank's memory access channel. This does not affect the efficiency of matrix fusion computation and avoids the performance loss caused by cross-bank access.

[0071] See Figure 5 This is a schematic diagram of an embodiment of the shared memory allocation device for matrix calculation provided by the present invention.

[0072] A second aspect of the present invention provides a shared memory allocation device for matrix computation, wherein the shared memory is divided into a first storage bank and a second storage bank; wherein the storage capacity of the first storage bank is greater than the storage capacity of the second storage bank; the device includes:

[0073] The first allocation module 11 is used to allocate the first multiplier matrix to the starting position of the second storage bank and allocate the second multiplier matrix to the starting position of the first storage bank when the memory requirement of the first multiplier matrix is ​​not greater than the storage capacity of the second storage bank; wherein the memory requirement of the first multiplier matrix is ​​greater than or equal to the memory requirement of the second multiplier matrix.

[0074] Capacity checking module 12 is used to check whether the storage cell with the largest remaining space is sufficient to accommodate the bias matrix;

[0075] The second allocation module 13 is used to allocate the bias matrix to the storage volume with the largest remaining space and perform matrix fusion calculation if the condition is met.

[0076] The third allocation module 14 is used to compare whether the memory requirement of the bias matrix is ​​greater than the total remaining space of the shared memory if the memory requirement of the bias matrix is ​​not greater than the total remaining space, and to divide the bias matrix into two sub-matrices and allocate them to the first storage bank and the second storage bank respectively to perform the matrix fusion calculation.

[0077] The fourth allocation module 15 is used to obtain the corresponding result matrix based on the matrix fusion calculation, and to allocate the result matrix to the memory space occupied by the bias matrix through a reuse operation.

[0078] It should be noted that the shared memory allocation device for matrix calculation provided in the second aspect embodiment of the present invention can implement all the processes of the shared memory allocation method for matrix calculation described in any of the first aspect embodiments. The functions and technical effects of each module and unit in the device are the same as those of the shared memory allocation method for matrix calculation described in any of the first aspect embodiments, and will not be repeated here.

[0079] A third aspect of the present invention provides a computer-readable storage medium comprising a stored computer program; wherein, when the computer program is executed, it controls the device in which the computer-readable storage medium is located to perform the shared memory allocation method for matrix calculation described in any of the first aspects of the present invention.

[0080] A fourth aspect of the present invention provides a computer program product including computer instructions that, when executed by a processor, implement the shared memory allocation method for matrix calculation described in any of the first aspects above.

[0081] See Figure 6 This is a schematic diagram of an embodiment of the electronic device provided by the present invention.

[0082] A fifth aspect of the present invention provides an electronic device including a processor 21, a memory 22, and a computer program stored in the memory 22 and configured to be executed by the processor 21, wherein the processor 21, when executing the computer program, implements the shared memory allocation method for matrix calculation described in any of the first aspects of the present invention.

[0083] Preferably, the computer program can be divided into one or more modules / units (such as computer program one, computer program two, ...), and the one or more modules / units are stored in the memory 22 and executed by the processor 21 to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device.

[0084] The processor 21 can be any one of a CPU (Central Processing Unit), GPU (Graphics Processing Unit), TPU (Tensor Processing Unit), NPU (Neural Network Processing Unit), DPU (Deep Learning Processing Unit), APU (Accelerated Processing Unit), and GPGPU (General-Purpose Computing on Graphics Processing Unit). The processor 21 is the control center of the electronic device, connecting various parts of the electronic device via various interfaces and lines.

[0085] The memory 22 mainly includes a program storage area and a data storage area. The program storage area can store the operating system, applications required for at least one function, etc., and the data storage area can store related data, etc. In addition, the memory 22 can be a high-speed random access memory, or a non-volatile memory, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, and a flash card, etc., or the memory 22 can also be other volatile solid-state storage devices.

[0086] It should be noted that the aforementioned electronic devices may include, but are not limited to, processors and memory, as will be understood by those skilled in the art. Figure 6 The structural block diagram shown is merely a structural example of the above-described electronic device and does not constitute a limitation on the structure of the above-described electronic device. The above-described electronic device may include more or fewer components than shown, or combine certain components, or different components.

[0087] See Figure 7 This is a schematic diagram of an embodiment of the artificial intelligence processor provided by the present invention.

[0088] The artificial intelligence processor provided in this embodiment of the invention includes multiple computing units (CUs). The CU is the core component for performing computing tasks and can run multiple threads in parallel. Each computing unit can execute the shared memory allocation method for matrix calculation described in any of the embodiments of the first aspect above.

[0089] In addition, the CU also has other hardware resources, such as shared memory and thread-level registers. During matrix fusion calculation, the first multiplier matrix, the second multiplier matrix, and the bias matrix are loaded from global memory to shared memory. When executing specific matrix multiply-accumulate (MMA) instructions, either the first multiplier matrix or the second multiplier matrix is ​​loaded into the thread-level register to complete the matrix multiplication operation between the first and second multiplier matrices, obtain the basic product result, and store it in the accumulator acc. Through the accumulator acc and the bias matrix cached in shared memory, the accumulation calculation is performed to obtain the result matrix of the matrix fusion calculation. This result matrix is ​​then unloaded from the thread-level register to shared memory, and then further unloaded from shared memory to global memory.

[0090] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A shared memory allocation method for matrix computation, characterized by, The shared memory is divided into a first memory bank and a second memory bank; wherein the storage capacity of the first memory bank is greater than the storage capacity of the second memory bank; the method comprises: when the memory requirement of the first multiplier matrix is not greater than the storage capacity of the second memory bank, the first multiplier matrix is allocated to the starting position of the second memory bank, and the second multiplier matrix is allocated to the starting position of the first memory bank; wherein the memory requirement of the first multiplier matrix is greater than or equal to the memory requirement of the second multiplier matrix; checking whether the memory bank with the largest remaining space is sufficient to accommodate the bias matrix; if yes, the bias matrix is allocated to the memory bank with the largest remaining space and the matrix fusion calculation is performed; if no, comparing whether the memory requirement of the bias matrix is greater than the total remaining space of the shared memory, and in the case that the memory requirement of the bias matrix is not greater than the total remaining space, the bias matrix is divided into two sub-matrices, which are allocated to the first memory bank and the second memory bank respectively to perform the matrix fusion calculation; based on the matrix fusion calculation, a corresponding result matrix is obtained, and the result matrix is allocated to the memory occupation space of the bias matrix through multiplexing operation.

2. The shared memory allocation method for matrix computation of claim 1, wherein, After the comparison of whether the memory requirement of the bias matrix is greater than the total remaining space of the shared memory, the method further comprises: in the case that the memory requirement of the bias matrix is greater than the total remaining space, it is determined that the memory allocation fails, and the execution of the matrix fusion calculation is stopped.

3. The shared memory allocation method for matrix computation of claim 1, wherein, Before the checking of whether the memory bank with the largest remaining space is sufficient to accommodate the bias matrix, the method further comprises: when the memory requirement of the first multiplier matrix is greater than the storage capacity of the second memory bank, checking whether the first memory bank is sufficient to accommodate the first multiplier matrix and checking whether the second memory bank is sufficient to accommodate the second multiplier matrix; if all, the first multiplier matrix is allocated to the starting position of the first memory bank, and the second multiplier matrix is allocated to the starting position of the second memory bank; if at least one is not, it is determined that the memory allocation fails, and the execution of the matrix fusion calculation is stopped.

4. The shared memory allocation method for matrix computation of claim 1, wherein, The matrix fusion calculation comprises matrix multiplication between the first multiplier matrix and the second multiplier matrix, and accumulation calculation related to N bias matrices; wherein N≥1.

5. The shared memory allocation method for matrix computation of claim 4, wherein, In the case that N>1, the accumulation calculation related to N bias matrices is performed by the following steps: each bias matrix is loaded in a preset order through multiple multiplexing operations of the memory occupation space to sequentially perform the corresponding accumulation calculation; wherein the capacity of the memory occupation space is equal to the memory requirement of any bias matrix.

6. The shared memory allocation method for matrix computation of claim 1, wherein, The first memory bank and the second memory bank are respectively configured with independent memory access channels.

7. An apparatus for shared memory allocation for matrix computation, characterized by, The shared memory is divided into a first memory bank and a second memory bank; wherein the storage capacity of the first memory bank is greater than the storage capacity of the second memory bank; the device comprises: a first allocation module, configured to, when the memory requirement of the first multiplier matrix is not greater than the storage capacity of the second memory bank, allocate the first multiplier matrix to the starting position of the second memory bank, and allocate the second multiplier matrix to the starting position of the first memory bank; wherein the memory requirement of the first multiplier matrix is greater than or equal to the memory requirement of the second multiplier matrix; A capacity checking module is configured to check whether a memory bank with the largest remaining space is sufficient to accommodate the bias matrix; A second allocation module is configured to, if yes, allocate the bias matrix to the memory bank with the largest remaining space and perform matrix fusion calculation; A third allocation module is configured to, if no, compare whether the memory requirement of the bias matrix is greater than the total remaining space of the shared memory, and in the case that the memory requirement of the bias matrix is not greater than the total remaining space, divide the bias matrix into two sub-matrices, and allocate the two sub-matrices to the first memory bank and the second memory bank respectively to perform the matrix fusion calculation; A fourth allocation module is configured to, based on the matrix fusion calculation, obtain a corresponding result matrix, and allocate the result matrix to the memory occupation space of the bias matrix through multiplexing operation.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium comprises a stored computer program; wherein the computer program, when running, controls a device where the computer readable storage medium is located to perform the shared memory allocation method for matrix calculation according to any one of claims 1 to 6.

9. A computer program product, characterised in that, The computer program product comprises computer instructions, which, when executed by a processor, implement the shared memory allocation method for matrix calculation according to any one of claims 1 to 6.

10. An electronic device, comprising: The computer program product comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the shared memory allocation method for matrix calculation according to any one of claims 1 to 6.

Images