Pe array, processor, related device and multiplication matrix operation processing method

Abstract

The application discloses a PE array, a processor, related equipment and a multiplication matrix operation processing method, and belongs to the electronic technical field.The PE array comprises: M rows of N column processing units PE, which are divided into a first triangular area PE and a second triangular area PE; an i-th row j-th column PE of the PE array is connected with an i+1-th row j+1-th column PE, adjacent column PEs in an M-th row PE of the first triangular area PE are connected, adjacent column PEs in a first row PE of the second triangular area PE are connected, and adjacent row PEs of each column PE of the PE array are connected; the first triangular area PE is used for performing matrix multiplication operation on an effective first triangular area or an effective second triangular area of a first intermediate matrix, and / or the second triangular area PE is used for performing matrix multiplication operation on an effective second triangular area or an effective first triangular area of a second intermediate matrix; the first intermediate matrix is the same as or different from the second intermediate matrix, the first intermediate matrix is a multiplication operation matrix of a first matrix and a second matrix, and the second intermediate matrix is a multiplication operation matrix of a third matrix and a fourth matrix.

Description

Technical Field

[0001] This application belongs to the field of electronic technology, specifically relating to a PE array, processor, related devices, and a multiplication matrix operation processing method. Background Technology

[0002] Many large models utilize attention mechanisms during inference. For example, the autoregressive inference process of a Large Language Model (LLM) involves matrix multiplication, such as calculating Q⋅K. T In these matrix operations, due to the causal attention constraint, the current token can only depend on itself and the keys and values ​​of its previous positions, and cannot depend on future tokens. Therefore, only the lower triangular or upper triangular part of the resulting matrix contains valid values, such as Q⋅K. T In the resulting matrix, only the lower triangular portion (including the diagonal) is valid, while the upper triangular portion is masked as invalid. Traditional General Matrix Multiply (GEMM) hardware architectures typically perform calculations on the complete matrix, which results in a large number of invalid multiply-accumulate (MAC) operations on the invalid upper triangular or first triangular regions, leading to invalid memory accesses and power consumption.

[0003] It is evident that existing matrix operation schemes contain a large number of invalid calculations, which easily lead to a waste of computing resources and power consumption. Summary of the Invention

[0004] The purpose of this application is to provide a PE array, processor, related devices, and a multiplication matrix operation processing method, which can solve the problem that existing matrix operation schemes have a large number of invalid calculations, which easily lead to a waste of computing resources and power consumption.

[0005] In a first aspect, embodiments of this application provide a PE array, comprising: The M-row N-column processing unit PE is divided into a first triangular region PE and a second triangular region PE according to the target diagonal. The target diagonal is composed of PEs with the same row number and column number in the M-row N-column PE. M and N are both integers greater than 1. When M is less than or equal to N, the first triangular region PE includes the target diagonal. When M is greater than N, the first triangular region PE does not include the target diagonal. The PE array in the i-th row and j-th column is connected to the PE array in the (i+1)-th row and j+1-th column. The PE array in the M-th row of the first triangular region is connected to the adjacent columns of PE. The PE array in the 1-th row of the second triangular region is connected to the adjacent columns of PE. The PE array in the column is connected to the adjacent rows of PE. i is an integer between 1 and M-1, and j is an integer between 1 and N-1. The first triangular region PE is used to perform matrix multiplication operations on the effective first triangular region or effective second triangular region of the first intermediate matrix, and / or the second triangular region PE is used to perform matrix multiplication operations on the effective second triangular region or effective first triangular region of the second intermediate matrix; wherein, the first intermediate matrix and the second intermediate matrix are the same or different, the first intermediate matrix is ​​the multiplication operation matrix of the first matrix and the second matrix, the second intermediate matrix is ​​the multiplication operation matrix of the third matrix and the fourth matrix, and the dimensions of the first intermediate matrix and the second intermediate matrix are adapted to the PE array.

[0006] Secondly, embodiments of this application provide a processor, including: the PE array described in the first aspect above, and further including a control module, a first data loading module, a second data loading module, a data writing module, and a static random access memory (SRAM); The control module is connected to the first data loading module, the second data loading module, and the write data module, respectively. The first data loading module, the second data loading module, and the write data module are all connected to the PE array and the SRAM. The control module is used to configure matrix operation parameters for the first data loading module, the second data loading module, and the data writing module respectively based on the configured matrix operation task information; The first data loading module is configured to read the row vector of the first matrix from the SRAM based on the matrix operation parameters, and inject the read row vector of the first matrix into the first triangular region PE of the PE array; the second data loading module is configured to read the column vector of the second matrix from the SRAM based on the matrix operation parameters, and inject the read column vector of the second matrix into the first triangular region PE of the PE array; and / or, the first data loading module is configured to read the row vector of the third matrix from the SRAM based on the matrix operation parameters, and inject the read row vector of the third matrix into the second triangular region PE of the PE array; the second data loading module is configured to read the column vector of the fourth matrix from the SRAM based on the matrix operation parameters, and inject the read column vector of the fourth matrix into the second triangular region PE of the PE array; The PE array is used to perform matrix multiplication operations on the first matrix and the second matrix through the first triangular region PE, and / or to perform matrix multiplication operations on the third matrix and the fourth matrix through the second triangular region PE; The data writing module is used to write the matrix multiplication result of the PE array into the SRAM based on the matrix operation parameters.

[0007] Thirdly, embodiments of this application provide an electronic device, which includes a central processing unit (CPU), a memory, and the processor described in the second aspect above, wherein a control module in the processor is connected to the CPU via a configuration bus, and an SRAM in the processor is connected to the memory via a data bus.

[0008] Fourthly, embodiments of this application provide a matrix multiplication operation processing method, applied to the processor described in the second aspect, the method comprising: The control module in the processor configures matrix operation parameters for the processor's first data loading module, second data loading module, and write data module based on the configured matrix operation task information. The first data loading module reads the row vector of a first matrix from the SRAM of the processor based on the matrix operation parameters, and injects the read row vector of the first matrix into the first triangular region PE of the PE array; the second data loading module reads the column vector of a second matrix from the SRAM based on the matrix operation parameters, and injects the read column vector of the second matrix into the first triangular region PE of the PE array; and / or, the first data loading module reads the row vector of a third matrix from the SRAM based on the matrix operation parameters, and injects the read row vector of the third matrix into the second triangular region PE of the PE array; the second data loading module reads the column vector of a fourth matrix from the SRAM based on the matrix operation parameters, and injects the read column vector of the fourth matrix into the second triangular region PE of the PE array; The PE array performs matrix multiplication on the first matrix and the second matrix through the first triangular region PE, and / or performs matrix multiplication on the third matrix and the fourth matrix through the second triangular region PE; The data writing module writes the matrix multiplication result of the PE array into the SRAM based on the matrix operation parameters.

[0009] Fifthly, embodiments of this application provide an electronic device including a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions, when executed by the processor, implementing the steps of the method described in the fourth aspect.

[0010] In a sixth aspect, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the fourth aspect.

[0011] In a seventh aspect, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method described in the fourth aspect.

[0012] Eighthly, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the fourth aspect.

[0013] In this embodiment, a PE array consisting of M rows and N columns of PEs is designed, and the PE array is divided into two triangular regions PE along the diagonal. The connection relationship between each PE unit in the PE array is established according to the data flow direction, and the data of the matrix to be operated is loaded based on the PE connection relationship. This makes the PE array aligned with the effective triangular region of the multiplication operation matrix at the structural and data flow levels. Thus, it is possible to use different triangular regions PE of the PE array to specifically calculate the effective triangular region in the multiplication operation matrix, and even to calculate the effective triangular region in two different multiplication operation matrices in parallel. In this way, this embodiment can reduce the calculation and data access of a large number of invalid triangular regions, improve the efficiency of matrix multiplication operations, and save computing resources and power consumption. Attached Figure Description

[0014] Figure 1 This is a diagram of the PE array structure provided in the embodiments of this application; Figure 2 This is a schematic diagram of matrix calculation provided in an embodiment of this application; Figure 3 This is a structural diagram of the PE unit provided in the embodiments of this application; Figure 4 This is a schematic diagram of the DP-MM data flow provided in an embodiment of this application; Figure 5 This is a schematic diagram of the DP-MM remapping data flow provided in an embodiment of this application; Figure 6 This is a schematic diagram of the architecture of the DP-MM accelerated processor provided in an embodiment of this application; Figure 7This is a schematic diagram of the PE data flow in the lower triangular region provided in an embodiment of this application. Detailed Implementation

[0015] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0016] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0017] To make the embodiments of this application clearer, the relevant technical knowledge involved in the embodiments of this application will be introduced as follows: When using block-based computation (such as FlashAttention-style block-based computation), the effective regions in the diagonal edge blocks (boundary tiles) are shaped like lower or upper triangles. If rectangular computation arrays are still used to process these blocks, approximately 50% of the computation will be locally invalid within the block. In scenarios such as Sliding Window Attention (SWA), the blocks contain three effective region forms simultaneously: lower triangles, upper triangles, and rectangles. If rectangular arrays are used to process them uniformly, it will lead to a decrease in MAC utilization and energy loss.

[0018] In existing technologies, software optimizations such as block sparsity and FlashAttention mainly reduce memory access and macroscopic computation. However, due to the underlying hardware being a regular rectangular computing array, there is still invalid computation and power waste on the diagonal edge blocks. Although hardware solutions include sparse matrix multiplication, reconfigurable arrays, and dedicated attention units, they mostly adopt scheduling methods that disable some PEs on the square array through indexing or masking, rather than designing the array physical topology as a triangular mesh and coordinating data flow and vector identification mapping specifically for the triangular effective domain.

[0019] Existing solutions typically have the following shortcomings: 1) Invalid computation and power consumption: The upper triangle result in causal attention is invalid. Traditional square array pulse arrays still perform MAC on this area, resulting in wasted computation and power consumption.

[0020] 2) Low utilization of block edges: In diagonal edge blocks, the effective area is a triangle. The MAC utilization of rectangular arrays on this type of block is about 50%, which is obviously wasteful.

[0021] 3) Mismatch between array topology and computational domain geometry: General-purpose systolic arrays use a rectangular mesh topology, with data flow advancing in a rectangular wavefront manner; however, the effective computational domain for causal attention is the lower triangular geometry. When the rectangular data flow covers the triangular computational domain, the PE in the upper triangular region generates idle cycles (pipeline bubbles) due to the lack of effective computational tasks, resulting in a decrease in the overall array utilization and energy efficiency.

[0022] 4) Masking solutions are not a fundamental solution: Existing solutions often disable some PEs on the matrix by indexing or masking. Although this can reduce some computation, it still retains the rectangular topology and rectangular data flow. The physical hardware of the invalid area still exists, and it cannot eliminate invalid transistor switching and wiring overhead from the source.

[0023] 5) Insufficient or costly scalability / reconfigurability support: Maintaining high utilization and avoiding complex mask / index scheduling overhead remains a challenge when sequence length and block size vary.

[0024] To overcome the problems of existing technologies, this application proposes a novel matrix operation technique, defined as Diagonal-Partitioned Matrix Multiply (DP-MM). Specifically, this application provides a DP-MM acceleration architecture and method for causal attention, aligning the DP-MM hardware with the effective domains of the upper / lower triangular matrices at the structural and dataflow levels. This reduces invalid computations and memory accesses, improves the utilization of block edges and energy efficiency, and supports adaptation to different sequence lengths and patterns.

[0025] The technical solution of this application differs from the "block sparse / mask-aware scheduling" in the following ways: by using a defined triangular PE arrangement, fixed injection points and propagation directions, and a one-to-one mapping between vector identifiers and PE coordinates, the hardware natively implements multiplication and accumulation only for the effective regions of the lower or upper triangles, without relying on general sparse scheduling or software-issued block-by-block activation maps. This results in a structure consistent with the effective field shape of causal attention, reducing invalid MACs and power consumption, and adapting to different sequence lengths and block partitioning strategies.

[0026] The PE array and processor provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0027] Please see Figure 1 , Figure 1 The PE array structure diagram provided in the embodiments of this application is as follows: Figure 1 As shown, the PE array includes: The M-row N-column processing element (PE) is divided into a first triangular region PE and a second triangular region PE according to the target diagonal. The target diagonal is composed of PEs with the same row and column numbers in the M-row N-column PE. M and N are both integers greater than 1. When M is less than or equal to N, the first triangular region PE includes the target diagonal. When M is greater than N, the first triangular region PE does not include the target diagonal. The PE array in the i-th row and j-th column is connected to the PE array in the (i+1)-th row and j+1-th column. The PE array in the M-th row of the first triangular region is connected to the adjacent columns of PE. The PE array in the 1-th row of the second triangular region is connected to the adjacent columns of PE. The PE array in the column is connected to the adjacent rows of PE. i is an integer between 1 and M-1, and j is an integer between 1 and N-1. The first triangular region PE is used to perform matrix multiplication operations on the effective first triangular region or effective second triangular region of the first intermediate matrix, and / or the second triangular region PE is used to perform matrix multiplication operations on the effective second triangular region or effective first triangular region of the second intermediate matrix; wherein, the first intermediate matrix and the second intermediate matrix are the same or different, the first intermediate matrix is ​​the multiplication operation matrix of the first matrix and the second matrix, the second intermediate matrix is ​​the multiplication operation matrix of the third matrix and the fourth matrix, and the dimensions of the first intermediate matrix and the second intermediate matrix are adapted to the PE array.

[0028] The PE array provided in this embodiment is a causal attention systolic computation PE array, such as... Figure 1 As shown, the PE array comprises M rows and N columns of PEs, with a total of M×N PE elements. Specifically, when M equals N, the PE array is a square PE array structure. For ease of understanding, this application embodiment mainly uses M=N as an example to illustrate the PE array structure in this application embodiment.

[0029] In this embodiment, the PE array can be divided into two parts along the target diagonal, namely the first triangular region PE and the second triangular region PE. The target diagonal can be assumed to be the diagonal line where the PEs with the same row and column numbers are located in the PE array. The first triangular region PE and the second triangular region PE can be the lower triangular region and the upper triangular region of the PE array, respectively. Figure 1As shown, based on the four PEs PE00, PE11, PE22 and PE33 on the diagonal, the system is divided into a lower triangular PE array and an upper triangular PE array. When M is less than or equal to N, the lower triangular PE can be defined to include the target diagonal. When M is greater than N, the lower triangular PE can be defined to exclude the target diagonal.

[0030] For example, such as Figure 1 As shown, the PE array consists of 16 PE units in 4 rows and 4 columns. Each PE is used to calculate the multiplication and accumulation results of matrix vectors and accumulate them in a loop to output the final calculation result. This belongs to the output stationary (OS) data stream mode.

[0031] Compared to traditional pulsating arrays, the pulsating PE array in this embodiment can be divided into two parts, a lower triangular region C_L and an upper triangular region C_U, based on the calculated output matrix element positions. The connection relationships and data flow between each PE unit are as follows: Figure 1 The arrows indicate the path: long dashed arrows represent row vector injection paths, short dashed arrows represent column vector injection paths, and solid arrows represent the path for outputting the result matrix.

[0032] Each PE unit in the PE array, i.e., PE(i,j), is responsible for calculating and outputting the element C[i][j] of the result matrix. Each PE participates in the generation of only one element of the result matrix. Each PE performs the multiplication and accumulation calculation of two input vectors (row vector and column vector, respectively), accumulates and updates the data in the register (Reg), performs cyclic accumulation, and finally outputs the calculation result. For example, the structure of the PE unit is as follows: Figure 3 As shown, each PE unit contains 8 multipliers, 1 adder tree, 1 accumulator, and 1 register, enabling the multiplication and accumulation of two vectors. The computation circuitry in the PE employs a multi-step pipelined architecture.

[0033] In this embodiment, the effective lower triangular region of the multiplication matrix can be calculated using the lower triangular region PE of the PE array, or the effective upper triangular region of the multiplication matrix can be mapped to the lower triangular region and then calculated using the lower triangular region PE. Alternatively, the effective upper triangular region of a certain multiplication matrix can be calculated using the upper triangular region PE of the PE array, or the effective lower triangular region of the multiplication matrix can be mapped to the upper triangular region and then calculated using the upper triangular region PE.

[0034] In this embodiment, the effective lower triangular region (PE) and effective upper triangular region (PE) of the PE array can be used to calculate the effective lower triangular region and effective upper triangular region of the same multiplication matrix, respectively. Alternatively, the effective lower triangular region and effective upper triangular region of different multiplication matrices can be calculated using the lower triangular region and effective upper triangular region of the PE array, respectively. Furthermore, the effective lower triangular region of different multiplication matrices can be calculated using the lower triangular region and effective upper triangular region of the PE array, with the effective lower triangular region of one of the multiplication matrices mapped to the upper triangular region before calculation. Or, the effective upper triangular region of different multiplication matrices can be calculated using the lower triangular region and effective upper triangular region of the PE array, with the effective upper triangular region of one of the multiplication matrices mapped to the lower triangular region before calculation.

[0035] It should be noted that the multiplication matrix in the embodiments of this application can refer to the intermediate matrix used in the multiplication of two matrices, or the result matrix of the multiplication of two matrices, such as... Figure 2 (b) shows the matrix C before the matrix operation results have been calculated.

[0036] For ease of understanding, Figure 2 A schematic diagram of matrix calculation is provided. Figure 2 In (a), matrix A has 4 row vectors, and matrix B has 4 column vectors, both with a length of 4. C = A·B is a matrix operation, which involves multiplying and adding the row vectors of A and the column vectors of B to obtain the resulting matrix C. The labels within each element of matrix C represent the row vectors of matrix A and the column vectors of matrix B involved in the operation; gray numbers represent the row vector indices of matrix A, and black numbers represent the column vector indices of matrix B. During matrix operations, each row vector and each column vector participates in 4 vector multiplication and addition operations. Matrix operations can be divided diagonally into two parts: a lower triangular region C_L and an upper triangular region C_U, as shown below. Figure 2 (b) Each box corresponds to a PE unit, meaning each box is processed by the PE unit at the corresponding position in the PE array, forming a rectangular computation array. This rectangular computation array has four flexible mapping methods: 1) The upper triangular PE and the lower triangular PE calculate the result matrix C of the same matrix operation, i.e., calculate C_U+C_L, which is equivalent to classical matrix computation; 2) The upper triangular PE and the lower triangular PE calculate the upper triangular matrix C_U and the lower triangular matrix C_L of different matrix operations in parallel; 3) Change the mapping method, and the lower triangular PE is reused to calculate the upper triangular matrix, i.e., calculate the upper triangular matrix C_U1 and C_U2 of different matrix operations in parallel; 4) Change the mapping method, and the upper triangular PE is reused to calculate the lower triangular matrix, i.e., calculate the lower triangular matrix C_L1 and C_L2 of different matrix operations in parallel.

[0037] Figure 1 Schematic PE array and Figure 2 The multiplication operation matrix in (b) corresponds to this.

[0038] It should also be noted that the first intermediate matrix and the second intermediate matrix in the embodiments of this application can be the same matrix or different matrices.

[0039] Specifically, when the third matrix is ​​the same as the first matrix, and the fourth matrix is ​​the same as the second matrix, the first intermediate matrix and the second intermediate matrix are the same, meaning they are intermediate matrices of the same multiplication operation matrix. This applies to different triangular regions of the PE array where the same multiplication operation matrix is ​​calculated separately in the upper and lower triangular regions.

[0040] The third matrix differs from the first matrix, or the fourth matrix differs from the second matrix, or both the third and first matrices differ, and the first and second intermediate matrices differ as well. This means they are intermediate matrices of different multiplication operation matrices, or intermediate matrices of different parts of the same multiplication operation matrix. This situation is applicable when calculating the effective triangular regions of different multiplication operation matrices in the upper and lower triangular regions of a PE array, or when calculating different effective regions of a relatively large multiplication operation matrix. In this way, the multiplication operations of multiple matrices can be calculated at once, or multiple effective parts of a large matrix can be calculated simultaneously.

[0041] In this embodiment, the sizes of the first intermediate matrix and the second intermediate matrix are adapted to the PE array. This can mean that the sizes of the first intermediate matrix and the second intermediate matrix are the same as or smaller than the size of the PE array, so that the effective triangular regions of the first intermediate matrix and the second intermediate matrix can be fully processed in any triangular region of the PE array at once. If the original multiplication matrix size is larger than the PE array size, the multiplication matrix needs to be adapted and cut according to the PE array size before being injected into the PE array for operation. In this case, the first intermediate matrix and the second intermediate matrix can refer to two sub-matrices adapted to the PE size obtained after matrix partitioning a multiplication matrix according to the PE array size.

[0042] Optionally, the PE array includes multiple computing modes, wherein, When the PE array uses the first calculation mode, the first triangular region PE is used to perform matrix multiplication on the effective first triangular region of the first intermediate matrix, and the second triangular region PE is used to perform matrix multiplication on the effective second triangular region of the second intermediate matrix. The second intermediate matrix is ​​the same as the first intermediate matrix. Alternatively, when the PE array uses the second calculation mode, the first triangular region PE is used to perform matrix multiplication on the effective first triangular region of the first intermediate matrix, and the second triangular region PE is used to perform matrix multiplication on the effective second triangular region of the second intermediate matrix, wherein the first intermediate matrix is ​​different from the second intermediate matrix. Alternatively, when the PE array uses a third calculation mode, the first triangular region PE is used to perform matrix multiplication on the effective second triangular region of the first intermediate matrix, and the second triangular region PE is used to perform matrix multiplication on the effective second triangular region of the second intermediate matrix, wherein the first intermediate matrix is ​​different from the second intermediate matrix. Alternatively, when the PE array uses the fourth calculation mode, the first triangular region PE is used to perform matrix multiplication on the effective first triangular region of the first intermediate matrix, and the second triangular region PE is used to perform matrix multiplication on the effective first triangular region of the second intermediate matrix, wherein the first intermediate matrix is ​​different from the second intermediate matrix.

[0043] In some embodiments, the upper and lower triangular regions of the PE array can be reused for computation. Specifically, by adjusting the matrix data arrangement, the lower triangular region of the PE array can be used to calculate the result of the upper triangular region, and vice versa. Therefore, the PE array in the embodiments of this application can support multiple different computation modes: 1) First calculation mode: Upper and lower triangular calculation mode of the same matrix; in this calculation mode, the lower triangular region PE and the upper triangular region PE are used to calculate the effective lower triangular region and effective upper triangular region of the same multiplication operation matrix, respectively.

[0044] 2) Second calculation mode: Upper and lower triangular calculation mode for different matrices; in this calculation mode, the lower triangular region PE and the upper triangular region PE are used to calculate the effective lower triangular region and effective upper triangular region of different multiplication operation matrices, respectively.

[0045] 3) Third calculation mode: double upper triangular calculation mode; in this calculation mode, the lower triangular region PE and the upper triangular region PE are used to calculate the effective upper triangular region of different multiplication operation matrices.

[0046] 4) Fourth calculation mode: Double lower triangular calculation mode; In this calculation mode, the lower triangular region PE and the upper triangular region PE are used to calculate the effective lower triangular region of different multiplication operation matrices.

[0047] The above four computing modes can meet the computing needs of different scenarios and can maintain the PE array at full load.

[0048] In this way, different computing modes of the PE array can be used to compute the effective triangular region of the multiplication matrix in parallel using different triangular regions of PE, thereby reducing the computation of invalid regions, improving the efficiency of matrix multiplication, and saving computing resources and power consumption.

[0049] Optionally, the first PE in the first triangular region PE is used to receive the first target row vector of the first matrix and to receive the second target row vector of the first matrix, and to pass the second target row vector to the corresponding row PE. The first PE is the PE in the g-th row and 1-th column of the first triangular region PE. Wherein, the first target row vector is the g-th row vector of the first matrix, and the second target row vector includes the (g+1)-th row vector to the M-th row vector of the first matrix. When M is less than or equal to N, g is 1, and when M is greater than N, g is 2. Alternatively, the first target row vector is the h-th row vector of the first matrix, and the second target row vector includes the 1-th row vector to the (h-1)-th row vector of the first matrix. When M is less than or equal to N, h is M, and when M is greater than N, h is M-1. The second PE in the first triangular region PE is used to receive the third target column vector of the second matrix and to receive the fourth target column vector of the second matrix, and to pass the fourth target column vector to the corresponding column PE. The second PE is the PE in the M-th row and e-th column of the first triangular region PE. The third target column vector is the e-th column vector of the second matrix, and the fourth target column vector includes the 1st to e-1st column vectors of the second matrix. When M is less than N, e is N-1, and when M is greater than or equal to N, e is N. Alternatively, the third target column vector is the f-th column vector of the first matrix, and the fourth target column vector includes the f+1st to Nth column vectors of the second matrix. When M is less than N, f is 2, and when M is greater than or equal to N, f is 1. And / or, The third PE in the second triangular region PE is used to receive the fifth target row vector of the third matrix and to receive the sixth target row vector of the third matrix, and to pass the sixth target row vector to the corresponding row PE. The third PE is the k-th row and N-th column PE in the second triangular region PE. The fifth target row vector is the k-th row vector of the third matrix, and the sixth target row vector includes the 1st to k-1st row vectors of the third matrix. When M is greater than or equal to N, k is M-1, and when M is less than N, k is M. Alternatively, the fifth target row vector is the r-th row vector of the third matrix, and the sixth target row vector includes the r+1th to M-th row vectors of the third matrix. When M is less than N, r is 1, and when M is greater than or equal to N, r is 2. The fourth PE of the second triangular region PE is used to receive the seventh target column vector of the fourth matrix and to receive the eighth target column vector of the fourth matrix, and to pass the eighth target column vector to the corresponding column PE. The fourth PE is the PE in the first row and s column of the second triangular region PE. The seventh target column vector is the s column vector of the fourth matrix, and the eighth target column vector includes the (s+1)th to the Nth column vector of the fourth matrix. When M is greater than N, s is 1, and when M is less than or equal to N, s is 2. Alternatively, the seventh target column vector is the t column vector of the fourth matrix, and the eighth target column vector includes the first to the (t-1)th column vector of the fourth matrix. When M is less than or equal to N, t is N-1, and when M is greater than N, t is N.

[0050] In some embodiments, for the lower triangular region PE of the PE array, it can be configured to receive row vectors loaded from the first matrix from one PE cell, such as the upper left PE, and column vectors loaded from the second matrix from another PE cell, such as the lower right PE. Based on the connection relationship and data flow between PE cells in the lower triangular region PE, each row vector loaded from the first matrix can be passed to the corresponding row PE, and each column vector loaded from the second matrix can be passed to the corresponding column PE.

[0051] like Figure 4 The diagram illustrates the data flow of matrices A, B, and C during calculations in the upper and lower triangular regions of the same multiplication operation matrix, also known as the DP-MM data flow. This diagram is a representation of... Figure 1 A diagram illustrating the splitting of the data stream, and related to... Figure 2 (b) Correspondingly, the box represents PE.

[0052] See Figure 4 When calculating C_L in the lower triangular region PE, such as Figure 4 As shown in the lower left corner of (a), the row vectors of matrix A are injected into the PE array from the upper left PE and propagated towards the lower right corner, sequentially injecting the 3rd, 2nd, 1st, and 0th row vectors. When a PE connects to two other PEs, it will copy the data twice and then pass it to these two PEs, as shown below. Figure 4 In (a), PE00 passes the same data to PE10 and PE11.

[0053] like Figure 4 As shown in the lower left corner of (b), the column vectors of matrix B are injected from the lower right corner PE and passed to the upper left corner, injecting the 0th, 1st, 2nd and 3rd column vectors in sequence. The PE copying behavior is the same as above.

[0054] Similarly, for the upper triangular region PE of the PE array, it can be set to receive row vectors loaded from the first matrix from one PE cell, such as the lower right PE, and column vectors loaded from the second matrix from another PE cell, such as the upper left PE. Based on the connection relationship and data flow between PE cells in the upper triangular region PE, each row vector loaded from the first matrix can be passed to the corresponding row PE, and each column vector loaded from the second matrix can be passed to the corresponding column PE.

[0055] See Figure 4 When calculating C_U in the upper triangular region PE, such as Figure 4 As shown in the upper right corner of (a), the row vectors of matrix A are injected into the PE array from the lower right corner PE and passed to the upper left corner, injecting the 0th, 1st and 2nd row vectors in sequence. The PE copying behavior is the same as above.

[0056] like Figure 4 As shown in the upper right corner of (b), the column vectors of matrix B are injected from the upper left corner PE and passed to the lower right corner, injecting the 3rd, 2nd and 1st column vectors in sequence. The PE copying behavior is the same as above.

[0057] In some embodiments, the upper triangular region C_U and the lower triangular region C_L of the multiplication matrix to be calculated can be remapped, and then injected into the corresponding lower triangular region PE and upper triangular region PE for calculation. For example, C_L' obtained after remapping the upper triangular region C_U is calculated in the lower triangular region PE of the PE array, and C_U' obtained after remapping the lower triangular region C_L is calculated in the upper triangular region PE of the PE array. That is, the calculations of the upper and lower triangular regions are interchanged.

[0058] like Figure 5 The diagram shows a remapping representation of DP-MM matrix operations. The leftmost sub-diagram represents the diagonal division of the matrix remapping. In this case, the upper right triangular portion of matrix C includes the diagonal and is used as the lower triangular region C_L' after mapping, while the lower left triangular portion of matrix C does not include the diagonal and is used as the upper triangular region C_U' after mapping.

[0059] Figure 5 The right-hand diagram shows the PE array and data flow after the upper and lower triangular regions are remapped. The row vectors of matrix A and the column vectors of matrix B calculated in the PE array have changed compared to the calculations before the mapping.

[0060] When calculating C_L' in the lower triangular region PE, such as Figure 5 As shown in the lower left corner of (a), the row vectors of matrix A are injected into the PE array from the upper left corner PE, and then propagated towards the lower right corner, injecting the 0th, 1st, 2nd, and 3rd row vectors in sequence, as shown in (a). Figure 5 As shown in the lower left corner of (b), the column vectors of matrix B are injected from the lower right corner PE, and the 3rd, 2nd, 1st and 0th column vectors are injected in sequence, and then propagated to the upper left corner.

[0061] When calculating C_U' in the upper triangular region PE, such as Figure 5 As shown in the upper right corner of (a), the row vectors of matrix A are injected into the PE array from the lower right corner PE, and then propagated towards the upper left corner, sequentially injecting the 3rd, 2nd, and 1st row vectors, as follows. Figure 5 As shown in the upper right corner of (b), the column vectors of matrix B are injected from the upper left corner PE and passed to the lower right corner, injecting the 0th, 1st and 2nd column vectors in sequence.

[0062] In this way, the matrix data to be calculated can be accurately and orderly loaded into the corresponding calculation area of ​​the PE array, ensuring the effective triangular area of ​​the PE array for efficient parallel calculation of the multiplication matrix.

[0063] The PE array in this embodiment includes: an M-row N-column processing unit PE, wherein the M-row N-column PE is divided into a first triangular region PE and a second triangular region PE according to a target diagonal. The target diagonal is composed of PEs with the same row and column numbers in the M-row N-column PE, where M and N are both integers greater than 1. When M is less than or equal to N, the first triangular region PE includes the target diagonal; when M is greater than N, the first triangular region PE does not include the target diagonal. The PE in the i-th row and j-th column of the PE array is connected to the PE in the (i+1)-th row and j+1-th column. Adjacent columns of PE in the M-th row of the first triangular region PE are connected to each other, and adjacent columns of PE in the 1-th row of the second triangular region PE are connected to each other. Next, adjacent rows of PEs in each column of the PE array are connected, where i is an integer between 1 and M-1, and j is an integer between 1 and N-1; the first triangular region PE is used to perform matrix multiplication operations on the effective first triangular region or effective second triangular region of the first intermediate matrix, and / or, the second triangular region PE is used to perform matrix multiplication operations on the effective second triangular region or effective first triangular region of the second intermediate matrix; wherein, the first intermediate matrix and the second intermediate matrix are the same or different, the first intermediate matrix is ​​the multiplication operation matrix of the first matrix and the second matrix, the second intermediate matrix is ​​the multiplication operation matrix of the third matrix and the fourth matrix, and the size of the first intermediate matrix and the second intermediate matrix is ​​adapted to the PE array. In this way, by designing a PE array consisting of M rows and N columns of PEs, and dividing the PE array into two triangular regions PE along the diagonal, establishing connection relationships between each PE unit in the PE array according to the data flow direction, and loading the matrix data to be operated on based on the PE connection relationships, the PE array is aligned with the effective triangular region of the multiplication matrix at both the structural and data flow levels. This allows for the targeted calculation of the effective triangular region in the multiplication matrix using different triangular regions PE of the PE array, and even the parallel calculation of the effective triangular regions in two different multiplication matrices. Thus, the embodiments of this application can reduce the calculation and data access of a large number of invalid triangular regions, improve the efficiency of matrix multiplication operations, and save computing resources and power consumption.

[0064] Please see Figure 6 , Figure 6 A processor architecture diagram for accelerating matrix multiplication provided in this application embodiment is shown below. Figure 6 As shown, the processor includes: The PE array 601 described in the above embodiments also includes a control module 602, a first data loading module 6031, a second data loading module 6032, a data writing module 604, and a static random-access memory (SRAM) 605. The control module is connected to the first data loading module, the second data loading module, and the write data module, respectively. The first data loading module, the second data loading module, and the write data module are all connected to the PE array and the SRAM. The control module is used to configure matrix operation parameters for the first data loading module, the second data loading module, and the data writing module respectively based on the configured matrix operation task information; The first data loading module is configured to read the row vector of the first matrix from the SRAM based on the matrix operation parameters, and inject the read row vector of the first matrix into the first triangular region PE of the PE array; the second data loading module is configured to read the column vector of the second matrix from the SRAM based on the matrix operation parameters, and inject the read column vector of the second matrix into the first triangular region PE of the PE array; and / or, the first data loading module is configured to read the row vector of the third matrix from the SRAM based on the matrix operation parameters, and inject the read row vector of the third matrix into the second triangular region PE of the PE array; the second data loading module is configured to read the column vector of the fourth matrix from the SRAM based on the matrix operation parameters, and inject the read column vector of the fourth matrix into the second triangular region PE of the PE array; The PE array is used to perform matrix multiplication operations on the first matrix and the second matrix through the first triangular region PE, and / or to perform matrix multiplication operations on the third matrix and the fourth matrix through the second triangular region PE; The data writing module is used to write the matrix multiplication result of the PE array into the SRAM based on the matrix operation parameters.

[0065] In this embodiment, the third matrix may be the same as or different from the first matrix, and the fourth matrix may be the same as or different from the second matrix. In some embodiments, the third matrix is ​​the same as the first matrix, and the fourth matrix is ​​the same as the second matrix. This corresponds to application scenarios where different triangular regions of the PE array are used to process the upper and lower triangular regions of the same multiplication matrix, or where the upper or lower triangular region of the PE array is used to process the effective triangular region of a multiplication matrix. In other embodiments, the third matrix is ​​different from the first matrix, and the fourth matrix is ​​different from the second matrix. This corresponds to application scenarios where different triangular regions of the PE array are used to process the effective triangular regions of different multiplication matrices, or where the upper triangular region of the PE array is used to process the effective triangular region of a multiplication matrix, or where the lower triangular region of the PE array is used to process the effective triangular region of another multiplication matrix.

[0066] Figure 6 This is a schematic diagram of the overall architecture of the accelerated processor for DP-MM oriented towards causal attention, provided in an embodiment of this application. Figure 6 As shown, the processor includes a control module 602 (CTRL), a PE array 601 supporting DP-MM, two data loading modules, a write data module (C Store) 604, and an SRAM 605. The two data loading modules are the first data loading module 6031, referred to as A Loader, and the second data loading module 6032, referred to as B Loader.

[0067] The control module 602 receives computation task information configured by the CPU, i.e., matrix operation task information, via the configuration bus (CFG_BUS). This information may include the computation mode, computational load, and read / write addresses of matrices A and B to be operated on. Based on the computation task information configured by the CPU, the control module 602 parses the computation task and configures the corresponding computation mode, matrix size, and other relevant parameters for the data loading module and the data writing module. During runtime, the control module 602 is responsible for coordinating data interaction and pipeline between other modules.

[0068] The first data loading module 6031, such as A Loader, is used to generate a corresponding SRAM read request based on the matrix operation parameters configured by the control module 602, such as the calculation mode and matrix size, and inject the row vector data of the read A matrix into the PE array.

[0069] The second data loading module 6032, such as B Loader, is used to generate a corresponding SRAM read request based on the matrix operation parameters configured by the control module 602, such as the calculation mode and matrix size, and inject the column vector data of the read B matrix into the PE array.

[0070] The PE array 601 is the core computing unit. For example, the PE array 601 can be constructed from 4 rows and 4 columns of PEs to form an upper and lower triangular data flow structure. It supports four computing modes: 1) upper and lower triangular computing mode for the same matrix; 2) upper and lower triangular computing mode for different matrices; 3) double lower triangular computing mode; and 4) double upper triangular computing mode. The PE array 601 can perform matrix multiplication operations on the row vectors of the loaded matrix A and the column vectors of matrix B through the lower triangular PEs, and it can also perform matrix multiplication operations on the row vectors of the loaded matrix A and the column vectors of matrix B through the upper triangular PEs.

[0071] The write data module 604, such as C Store, is used to generate corresponding SRAM write requests based on the calculation mode, matrix size, and other information configured by the control module 602, and write the data output from the PE array back to the SRAM. In some embodiments, the write data module 604 also includes a data mapping submodule (C remap). When calculating the remapped upper and lower triangular matrices in the lower or upper triangular PE region, the data mapping submodule C remap in the write data module 604 can be used to adjust the data format according to the calculation mode before writing it back to the SRAM.

[0072] The SRAM605 is an on-chip buffer that supports high-bandwidth read and write data through the first data loading module 6031 A Loader, the second data loading module 6032 B Loader, and the write data module 604 C Store. It can exchange data with external storage via the data bus.

[0073] Optionally, the matrix operation task information includes matrix information to be multiplied, the matrix information including information of the target intermediate matrix and the valid and / or invalid triangular regions in the target intermediate matrix, the target intermediate matrix being the multiplication matrix of the first target matrix and the second target matrix to be multiplied; The control module is used to determine a target calculation mode for the target intermediate matrix based on the valid and / or invalid triangular regions in the target intermediate matrix. The target calculation mode is used to instruct the valid triangular regions in the target intermediate matrix to perform operations on the target triangular region PE in the PE array. The target triangular region PE is one of the first triangular region PE and the second triangular region PE. The matrix operation parameters include the calculation mode and the matrix information. The target calculation mode is one of the multiple calculation modes supported by the PE array. Wherein, when the size of the target intermediate matrix is ​​equal to the size of the PE array, the first target matrix and the second target matrix are respectively the first matrix and the second matrix, the target intermediate matrix is ​​the first intermediate matrix, and the first intermediate matrix and the second intermediate matrix are the same matrix; Alternatively, if the size of the target intermediate matrix is ​​larger than the size of the PE array, the control module is further configured to divide the target intermediate matrix into multiple sub-intermediate matrices according to the size of the PE array, wherein the first intermediate matrix and the second intermediate matrix are any two of the multiple sub-intermediate matrices, the first intermediate matrix and the second intermediate matrix are different matrices, the first matrix and the third matrix are sub-matrices in the first target matrix, and the second matrix and the fourth matrix are sub-matrices in the second target matrix.

[0074] In some embodiments, the matrix operation task information configured by the CPU includes matrix information to be multiplied, specifically including information of the target intermediate matrix and the valid and / or invalid triangular regions in the target intermediate matrix. The target intermediate matrix is ​​the multiplication matrix of the first target matrix and the second target matrix to be multiplied.

[0075] The control module 602 can determine which calculation mode to use for the target intermediate matrix based on the valid and / or invalid triangular regions in the target intermediate matrix. Therefore, based on the target calculation mode, it can determine which triangular region (PE) in the PE array to place the valid triangular regions of the target intermediate matrix for computation. It should be noted that in some embodiments, the CPU can directly specify which calculation mode the control module 602 should use.

[0076] When the size of the target intermediate matrix is ​​comparable to that of the PE array, the matrix can be directly divided into upper and lower triangular regions along the diagonal, and the corresponding regions in the PE array can be processed. When the target intermediate matrix is ​​a large matrix, such as one much larger than the size of the PE array, the control module 602 can cut the target intermediate matrix according to the size of the PE array, divide it into multiple sub-intermediate matrices, and then process the different effective triangular regions of the multiple intermediate sub-matrices in parallel through the corresponding triangular regions PE of the PE array.

[0077] In this way, by adapting and cutting the multiplication matrix to be operated on according to the size of the PE array, the multiplication efficiency of the PE array for large-sized matrices can be improved, and multiple matrix block multiplication operations can be run simultaneously.

[0078] Optionally, the first data loading module is configured to determine a first injection order for the row vectors of the first matrix based on the target calculation mode, and sequentially read the row vectors of the first matrix from the SRAM according to the first injection order, and inject the row vectors of the first matrix read each time into the first triangular region PE; the second data loading module is configured to determine a second injection order for the column vectors of the second matrix based on the target calculation mode, and sequentially read the column vectors of the second matrix from the SRAM according to the second injection order, and inject the column vectors of the second matrix read each time into the first triangular region PE; And / or, the first data loading module is configured to determine a third injection order for the row vectors of the third matrix based on the target calculation mode, and based on the third injection order, sequentially read the row vectors of the third matrix from the SRAM, and inject the row vectors of the third matrix read each time into the second triangular area PE; the second data loading module is configured to determine a fourth injection order for the column vectors of the fourth matrix based on the target calculation mode, and based on the fourth injection order, sequentially read the column vectors of the fourth matrix from the SRAM, and inject the column vectors of the fourth matrix read each time into the second triangular area PE; The data writing module is used to determine the target output order of the matrix multiplication results of the PE array based on the target calculation mode, output the matrix multiplication results according to the target output order, and write the output matrix multiplication results into the SRAM in sequence.

[0079] In some embodiments, the first data loading module may determine the first injection order of the row vectors of the first matrix based on the calculation mode configured by the control module, and read the row vectors of the first matrix from the SRAM in sequence according to the first injection order, and inject them into the corresponding row PE of the lower triangular region PE in batches.

[0080] The second data loading module can also determine the second injection order of the column vectors of the second matrix based on the calculation mode configured by the control module, and read the column vectors of the second matrix from the SRAM in sequence according to the second injection order, and inject them into the corresponding column PE of the lower triangular region PE in batches.

[0081] The first data loading module can also determine the third injection order of the row vectors of the third matrix based on the calculation mode configured by the control module, and read the row vectors of the third matrix from the SRAM in sequence according to the third injection order, and inject them into the corresponding row PE of the upper triangular region PE in batches.

[0082] The second data loading module can also determine the fourth injection order of the column vectors of the fourth matrix based on the calculation mode configured by the control module, and read the row vectors of the fourth matrix from the SRAM in sequence according to the fourth injection order, and inject them into the corresponding column PE of the upper triangular region PE in batches.

[0083] Optionally, when the target calculation mode is the first calculation mode, the first injection order is the order in which vectors from the Mth row to the gth row of the first matrix are injected sequentially; the second injection order is the order in which vectors from the 1st column to the eth column of the second matrix are injected sequentially; the third injection order is the order in which vectors from the 1st row to the kth row of the first matrix are injected sequentially; the fourth injection order is the order in which vectors from the Nth column to the sth column of the second matrix are injected sequentially; the target output order is the order in which vectors from the 1st row to the Mth row and from the 1st column to the Mth column of the operation result matrix are output sequentially; and the operation result matrix is ​​the matrix corresponding to the matrix multiplication operation result. When the target calculation mode is the second calculation mode, the first injection order is the order in which vectors from the Mth row to the gth row of the first matrix are injected sequentially; the second injection order is the order in which vectors from the 1st column to the eth column of the second matrix are injected sequentially; the third injection order is the order in which vectors from the 1st row to the kth row of the third matrix are injected sequentially; the fourth injection order is the order in which vectors from the Nth column to the sth column of the fourth matrix are injected sequentially; and the target output order is the order in which vectors from the 1st row to the Mth row and from the 1st column to the Mth column of the result matrix are output sequentially. When the target calculation mode is the third calculation mode, the first injection order is the order in which vectors from the 1st row to the hth row of the first matrix are injected sequentially; the second injection order is the order in which vectors from the Nth column to the fth column of the second matrix are injected sequentially; the third injection order is the order in which vectors from the 1st row to the kth row of the third matrix are injected sequentially; and the fourth injection order is the order in which vectors from the Nth column to the sth column of the fourth matrix are injected sequentially. The target output order is: the first triangular area of ​​the operation result matrix is ​​output sequentially from the hth row to the 1st row and from the Nth column to the fth column; and the second triangular area of ​​the operation result matrix is ​​output sequentially from the 1st row to the kth row and from the sth column to the Nth column. When the target calculation mode is the fourth calculation mode, the first injection order is the order in which vectors from the Mth row to the gth row of the first matrix are injected sequentially; the second injection order is the order in which vectors from the 1st column to the eth column of the second matrix are injected sequentially; the third injection order is the order in which vectors from the rth row to the 1st row of the third matrix are injected sequentially; and the fourth injection order is the order in which vectors from the 1st column to the tth column of the fourth matrix are injected sequentially. The target output order is: the first triangular area of ​​the operation result matrix is ​​output sequentially from the gth row to the Mth row and from the 1st column to the eth column; and the second triangular area of ​​the operation result matrix is ​​output sequentially from the rth row to the 1st row and from the tth column to the 1st column. Where M is less than or equal to N, g is 1, s is 2, h is M, and t is N-1; when M is greater than N, g is 2, s is 1, h is M-1, and t is N. When M is less than N, e is N-1, k is M, f is 2, and r is 1; when M is greater than or equal to N, e is N, k is M-1, f is 1, and r is 2.

[0084] Specifically, in combination Figure 4 and Figure 5 Examples illustrating the vector injection order under each computation mode are provided below: like Figure 4 The diagram shows the data flow of matrices A, B, and C during the calculation of the upper and lower triangular regions of the same matrix, which is also the data flow of DP-MM.

[0085] When calculating the lower triangular region C_L of the matrix PE in the lower triangular region, such as Figure 4 As shown in the lower left corner of (a), the row vectors of matrix A are injected into the PE array from the upper left corner and propagated to the lower right corner. The injection order of the row vectors is 3, 2, 1, 0, that is, from row 4 to row 1, which takes 4 clock cycles to complete. Figure 4As shown in the lower left corner of (b), the column vectors of matrix B are injected from the lower right corner and passed to the upper left corner. The order of column vector injection is 0, 1, 2, 3, that is, from column 1 to column 4 in sequence. It takes 4 cycles to complete the loading, which is the same as the number of cycles for injecting the row vectors of matrix A.

[0086] When calculating the upper triangular region C_U of the matrix PE in the upper triangular region, such as Figure 4 As shown in the upper right corner of (a), the row vectors of matrix A are injected into the PE array from the lower right corner and propagated towards the upper left corner. The injection order of the row vectors is 0, 1, 2, that is, from row 1 to row 3, which takes 3 clock cycles to complete. Figure 4 As shown in the upper right corner of (b), the column vectors of matrix B are injected from the upper left corner and passed to the lower right corner. The order of column vector injection is 3, 2, 1, that is, from the 4th column to the 2nd column in sequence, which is the opposite of the injection order of the row vectors of matrix A. It takes 3 cycles to complete the loading, which is the same as the number of cycles for the row vector injection of matrix A.

[0087] The injected data is a vector. After matrices A and B are loaded into the PE array, each PE calculates the product and sum of the row vectors of matrix A and the column vectors of matrix B, and adds the sum to the original register in the PE. The final calculated result, matrix C, is output step-by-step from the top of the PE array, requiring 4 steps. The output rows of matrix C are in the order of rows 0, 1, 2, and 3, and the elements within each row are in the order of columns 0, 1, 2, and 3.

[0088] like Figure 5 The diagram illustrates the remapping of DP-MM matrix operations. The leftmost sub-diagram shows the diagonal partitioning method for matrix remapping. In this case, the upper right part of matrix C includes the diagonal and serves as the lower triangle C_L' after mapping; the lower left part of matrix C does not include the diagonal and serves as the upper triangle C_U' after mapping. After remapping, C_L' is calculated in the lower left corner of the PE array, while C_U' is calculated in the upper right corner of the PE array, meaning the calculations for the upper and lower triangles are interchanged.

[0089] Figure 5 The diagram on the right shows the PE array and data flow after the upper and lower triangular remapping. The row vectors of matrix A and the column vectors of matrix B calculated in PE have changed compared to the calculations before the mapping.

[0090] When calculating C_L' in the lower triangular region PE, such as Figure 5 As shown in the lower left corner of (a), the row vectors of matrix A are injected in the order of 0, 1, 2, 3, that is, from row 1 to row 4. Figure 5 As shown in the lower left corner of (b), the column vectors of matrix B are injected in the order of 3, 2, 1, 0, that is, from the 4th column to the 1st column, which is the opposite of the order of the row vectors of matrix A. It takes 4 cycles to complete the loading, which is the same as the number of cycles for injecting the row vectors of matrix A.

[0091] When calculating C_U' in the upper triangular region PE, such as Figure 5 As shown in the upper right corner of (a), the row vectors of matrix A are injected in the order of 3, 2, 1, that is, from row 4 to row 2, requiring 3 clock cycles to complete the loading. Figure 5 As shown in the upper right corner of (b), the column vectors of matrix B are injected in the order of 0, 1, 2, that is, from column 1 to column 3 in sequence, which is the opposite of the order of the row vectors of matrix A. It takes 3 cycles to complete the loading, which is the same as the number of cycles for injecting the row vectors of matrix A.

[0092] At this point, the C' matrix after the calculations of C_L' and C_U' is a matrix rotated 180 degrees from the original C matrix, meaning the C matrix is ​​first flipped vertically and then horizontally. The output row order of the C' matrix is ​​3, 2, 1, 0, meaning it is injected sequentially from row 4 to row 1, and the internal element order of each row is 3, 2, 1, 0, meaning it is injected sequentially from column 4 to column 1. Therefore, this data flow scheme for remapping the upper and lower triangular regions can still complete the correct GEMM calculation.

[0093] It should be noted that the matrix operations in this example are intended to illustrate that calculations of the upper and lower triangular regions can be interchanged within the same rectangular PE array composed of upper and lower triangles. In practice, calculations within the same matrix do not require this mapping mode, but when calculating the upper and lower triangular regions of different matrices, an appropriate mapping method can be selected.

[0094] The above embodiments illustrate the correctness and feasibility of the upper and lower triangular region remapping. The implementation of upper and lower triangular region remapping is simple: it only requires changing the loading order of the row vectors of matrix A and the column vectors of matrix B into the PE matrix, and changing the order of the output rows and elements within those rows of matrix C. This example demonstrates that upper and lower triangular region calculations can be mutually converted. It also proves that this array structure can support four calculation modes, and that PE computing resources maintain high utilization across different calculation modes.

[0095] Optionally, the first data loading module is connected to the first PE of the first triangular region PE and the second PE of the second triangular region PE, wherein the first PE is the PE in the g-th row and 1-th column of the first triangular region PE, and the second PE is the PE in the M-th row and e-th column of the first triangular region PE. The second data loading module is connected to the third PE of the lower corner area PE and the fourth PE of the second triangular area PE. The third PE is the PE in the kth row and Nth column of the second triangular area PE, and the fourth PE is the PE in the 1st row and sth column of the second triangular area PE. The data writing module is connected to the first row of PEs in the PE array; The first data loading module is used to inject the row vector of the read first matrix from the first PE into the first triangular region PE, and the second data loading module is used to inject the column vector of the read second matrix from the third PE into the first triangular region PE; and / or, the first data loading module is used to inject the row vector of the read third matrix from the second PE into the second triangular region PE, and the second data loading module is used to inject the column vector of the read fourth matrix from the fourth PE into the second triangular region PE.

[0096] In some embodiments, such as Figure 1 and Figure 6 As shown, the first data loading module 601 can be connected to the upper left corner PE of the lower triangular area PE and to the lower right corner PE of the upper triangular area PE, the second data loading module 602 can be connected to the lower right corner PE of the lower triangular area PE and to the upper left corner PE of the upper triangular area PE, and the write data module 605 can be connected to the first row PE of the PE array.

[0097] This ensures that the row vectors of the first matrix read from SRAM are sequentially injected into the corresponding rows of the lower triangular region PE from the top left corner PE, the column vectors of the second matrix read from SRAM are sequentially injected into the corresponding columns of the lower triangular region PE from the bottom right corner PE, the row vectors of the first matrix read from SRAM are sequentially injected into the corresponding rows of the upper triangular region PE from the bottom right corner PE, and the column vectors of the second matrix read from SRAM are sequentially injected into the corresponding columns of the upper triangular region PE from the top left corner PE, and guarantees that the final matrix operation results are output in order from the first row PE.

[0098] Optionally, the first data loading module is used to inject the row vectors of the first matrix read from the SRAM K times sequentially from the first PE into rows M to M-K+1 of the first triangular region PE. Specifically, the first data loading module is used to inject the row vectors of the first matrix read from the SRAM K-1 times sequentially from the first PE into rows M to M-K+2 of the first triangular region PE, and to inject the row vector of the first matrix read from the SRAM K times into the first PE, where K is the number of times the first data loading module reads the row vectors of the first matrix from the SRAM, and K is a positive integer less than or equal to M. The second data loading module is used to inject the column vectors of the second matrix read from the SRAM L times sequentially from the third PE into the first column to the Lth column of the first triangular region PE. Specifically, the second data loading module is used to inject the column vectors of the second matrix read from the SRAM L-1 times sequentially from the third PE into the first column to the L-1th column of the first triangular region PE, and to inject the column vector of the second matrix read from the SRAM Lth time into the third PE, where L is the number of times the second data loading module reads the column vectors of the second matrix from the SRAM, and L is a positive integer less than or equal to N. And / or, The first data loading module is used to inject the row vectors of the third matrix read from the SRAM P times sequentially from the second PE into rows 1 to P of the second triangular area PE. Specifically, the first data loading module is used to inject the row vectors of the third matrix read from the SRAM P-1 times sequentially from the second PE into rows 1 to P-1 of the second triangular area PE, and to inject the row vector of the third matrix read from the SRAM P times into the second PE. P is the number of times the first data loading module reads the row vectors of the third matrix from the SRAM, and P is a positive integer less than or equal to M. The second data loading module is used to inject the column vectors of the fourth matrix read from the SRAM Q times sequentially from the fourth PE into columns N to N-Q+1 of the second triangular region PE. Specifically, the second data loading module is used to inject the column vectors of the fourth matrix read from the SRAM Q-1 times sequentially from the fourth PE into columns N to N-Q+2 of the second triangular region PE, and to inject the column vector of the fourth matrix read from the SRAM Q times into the fourth PE. Q is the number of times the second data loading module reads the column vectors of the fourth matrix from the SRAM, and Q is a positive integer less than or equal to N.

[0099] In some embodiments, both the first data loading module and the second data loading module can inject the row / column vectors of the corresponding matrix read each time into the PE at the corresponding row position according to their respective SRAM data reading order, so as to ensure the accuracy and effectiveness of matrix data loading.

[0100] Optionally, the PE array adopts a pipelined design, and the number of pipelines in the PE array is less than the number of cycles required for the PE array to load data. The number of cycles includes the number of injections required to inject the row vectors of the first matrix into the PE array and the number of injections required to inject the column vectors of the second matrix.

[0101] In some embodiments, to ensure high processing efficiency, the data loading module and the data writing module can guarantee the data bit width required by the PE array. The PE array 601 contains a computation pipeline, and the number of pipeline stages is no greater than the number of cycles required for data loading.

[0102] Specifically, during calculations in the upper and lower triangular regions, the PE array needs to wait for the data to finish loading before starting the calculation, and only outputs the final result after the cumulative calculation is complete. Since the lower triangular region PE has only one pair of input ports, such as the row vector of matrix A and the column vector of matrix B, ... Figure 1 and Figure 6 Taking a 4×4 array as an example, the data loading time interval for the lower triangular region PE is 4 clock cycles. Since the PE array uses a pipelined design, the number of pipeline stages must not exceed the number of clock cycles required for data loading to complete. To maintain high utilization of the PE array, the vector length loaded by the lower triangular region PE each time must be 4 times the vector length that the PE array pipeline can process per clock cycle. During PE operation, the data is injected into the computation pipeline in 4 clock cycles to ensure uninterrupted computation and to match the data loading interval. Similarly, the data loading time interval for the upper triangular region PE is 3 clock cycles. The vector lengths of matrices A and B loaded each time must be 3 times the vector length that the PE array pipeline can process. During PE operation, the data is injected into the computation pipeline in 3 clock cycles.

[0103] by Figure 7 The example shown illustrates the PE calculation in the lower triangular region. The PE array contains a pipeline consisting of four MACs, meaning the vector length processed per cycle is 4. For example... Figure 7 As shown in (a), when 4 data points of PE are loaded per cycle, the pipeline is underutilized, with only 25% utilization, or 1 / 4. Figure 7 As shown in (b), when 16 data points of PE are loaded per tick, the pipeline is fully pipelined and has 100% utilization.

[0104] In summary, the embodiments of this application have the following major improvements: 1) Triangle-sensing pulsating array geometry By employing a physical arrangement of lower triangular and / or upper triangular physical transformers (PEs), the number of PEs is consistent with the effective computation area, avoiding physical computation in invalid regions. Compared to a square array of the same size, the number of PEs is reduced by approximately 50%. Structurally, PEs are not placed in the upper triangular invalid region, thereby avoiding the physical MAC and transistor flipping in that region.

[0105] 2) Data flow and mapping rules specifically for the triangular effective domain The injection point, propagation direction, beat order, and propagation distance rules are designed and defined for the lower and upper triangles, respectively. Through a fixed mapping relationship between vector identifiers and PE positions, each PE only receives and participates in the multiplication and accumulation of the row vector of its unique corresponding matrix A and the column of matrix B. The resulting matrix elements correspond one-to-one with the PEs, and there is no redundant calculation. This data stream and mapping are specifically designed for the effective field of the triangle, which is different from the rectangular wavefront and sparse scheduling of general square matrices.

[0106] 3) Complete DP-MM is achieved through dual-triangle collaboration. The lower triangular array outputs the lower triangular part of the result matrix, and the upper triangular array outputs the upper triangular part. Logically, the two can be merged into a complete matrix multiplication result. This supports the calculation of the effective region of the lower triangular part of causal attention while also taking into account the calculation needs of the upper triangular block in SWA block division, thereby improving the overall MAC utilization.

[0107] 4) Flexible configuration capability of homogeneous arrays Decoupling of physical array type (upper / lower triangular shape) from computational target (C_L / C_U): Homogeneous triangular arrays (lower or upper triangular arrays) can flexibly switch computation modes through data flow reconfiguration. Three deployment models are supported: Double lower triangle: Two lower triangular arrays can be used to calculate two C_L values ​​simultaneously (batch processing), or one array can calculate C_L and the other can calculate C_U (by configuring the upper triangular data stream). Double upper triangle: Two upper triangular arrays can be used to calculate two C_U values ​​simultaneously, or one array can calculate C_U and the other can calculate C_L (by configuring the lower triangular data stream). A lower triangular array plus an upper triangular array: can collaboratively compute the complete result of the same matrix, or process the upper and lower triangular calculations of different matrices in parallel.

[0108] This allows for adaptation to the computing needs of different application scenarios. For example, pure causal attention can use a double lower triangle to improve throughput, while SWA scenarios can use a hybrid deployment to ensure flexibility.

[0109] This application also provides a matrix multiplication operation processing method, which is applied to... Figure 6 The processor shown specifically includes the following steps: Step 801: Based on the configured matrix operation task information, the control module configures matrix operation parameters for the first data loading module, the second data loading module, and the data writing module respectively; Step 802: The first data loading module reads the row vector of the first matrix from the SRAM based on the matrix operation parameters, and injects the read row vector of the first matrix into the first triangular region PE of the PE array; the second data loading module reads the column vector of the second matrix from the SRAM based on the matrix operation parameters, and injects the read column vector of the second matrix into the first triangular region PE of the PE array. And / or, the first data loading module reads the row vector of the third matrix from the SRAM based on the matrix operation parameters, and injects the read row vector of the third matrix into the second triangular region PE of the PE array; the second data loading module reads the column vector of the fourth matrix from the SRAM based on the matrix operation parameters, and injects the read column vector of the fourth matrix into the second triangular region PE of the PE array. Step 803: The PE array performs matrix multiplication on the first matrix and the second matrix through the first triangular region PE, and / or performs matrix multiplication on the third matrix and the fourth matrix through the second triangular region PE; Step 804: The data writing module writes the matrix multiplication result of the PE array into the SRAM based on the matrix operation parameters.

[0110] Optionally, the matrix operation task information includes matrix information to be multiplied, the matrix information including information of the target intermediate matrix and the valid and / or invalid triangular regions in the target intermediate matrix, the target intermediate matrix being the multiplication matrix of the first target matrix and the second target matrix to be multiplied; Before step 801, the method further includes: The control module determines a target calculation mode for the target intermediate matrix based on the valid and / or invalid triangular regions in the target intermediate matrix. The target calculation mode is used to instruct the valid triangular regions in the target intermediate matrix to perform operations on the target triangular region PE in the PE array. The target triangular region PE is one of the first triangular region PE and the second triangular region PE. The matrix operation parameters include the calculation mode and the matrix information. The target calculation mode is one of the multiple calculation modes supported by the PE array. Wherein, when the size of the target intermediate matrix is ​​equal to the size of the PE array, the first target matrix and the second target matrix are respectively the first matrix and the second matrix, the target intermediate matrix is ​​the first intermediate matrix, and the first intermediate matrix and the second intermediate matrix are the same matrix; Alternatively, if the size of the target intermediate matrix is ​​larger than the size of the PE array, the method further includes: the control module dividing the target intermediate matrix into multiple sub-intermediate matrices according to the size of the PE array, wherein the first intermediate matrix and the second intermediate matrix are any two of the multiple sub-intermediate matrices, the first intermediate matrix and the second intermediate matrix are different matrices, the first matrix and the third matrix are sub-matrices in the first target matrix, and the second matrix and the fourth matrix are sub-matrices in the second target matrix.

[0111] Optionally, step 802 includes: The first data loading module determines a first injection order for the row vectors of the first matrix based on the target calculation mode, and sequentially reads the row vectors of the first matrix from the SRAM according to the first injection order, and injects the row vectors of the first matrix read each time into the first triangular region PE; the second data loading module determines a second injection order for the column vectors of the second matrix based on the target calculation mode, and sequentially reads the column vectors of the second matrix from the SRAM according to the second injection order, and injects the column vectors of the second matrix read each time into the first triangular region PE; And / or, the first data loading module determines a third injection order for the row vectors of the third matrix based on the target calculation mode, and sequentially reads the row vectors of the third matrix from the SRAM based on the third injection order, and injects the row vectors of the third matrix read each time into the second triangular area PE; the second data loading module determines a fourth injection order for the column vectors of the fourth matrix based on the target calculation mode, and sequentially reads the column vectors of the fourth matrix from the SRAM based on the fourth injection order, and injects the column vectors of the fourth matrix read each time into the second triangular area PE; Step 804 includes: The data writing module determines the target output order of the matrix multiplication results of the PE array based on the target calculation mode, outputs the matrix multiplication results according to the target output order, and writes the output matrix multiplication results into the SRAM sequentially.

[0112] Optionally, when the target calculation mode is the first calculation mode, the first injection order is the order in which vectors from the Mth row to the gth row of the first matrix are injected sequentially; the second injection order is the order in which vectors from the 1st column to the eth column of the second matrix are injected sequentially; the third injection order is the order in which vectors from the 1st row to the kth row of the first matrix are injected sequentially; the fourth injection order is the order in which vectors from the Nth column to the sth column of the second matrix are injected sequentially; the target output order is the order in which vectors from the 1st row to the Mth row and from the 1st column to the Mth column of the operation result matrix are output sequentially; and the operation result matrix is ​​the matrix corresponding to the matrix multiplication operation result. When the target calculation mode is the second calculation mode, the first injection order is the order in which vectors from the Mth row to the gth row of the first matrix are injected sequentially; the second injection order is the order in which vectors from the 1st column to the eth column of the second matrix are injected sequentially; the third injection order is the order in which vectors from the 1st row to the kth row of the third matrix are injected sequentially; the fourth injection order is the order in which vectors from the Nth column to the sth column of the fourth matrix are injected sequentially; and the target output order is the order in which vectors from the 1st row to the Mth row and from the 1st column to the Mth column of the result matrix are output sequentially. When the target calculation mode is the third calculation mode, the first injection order is the order in which vectors from the 1st row to the hth row of the first matrix are injected sequentially; the second injection order is the order in which vectors from the Nth column to the fth column of the second matrix are injected sequentially; the third injection order is the order in which vectors from the 1st row to the kth row of the third matrix are injected sequentially; and the fourth injection order is the order in which vectors from the Nth column to the sth column of the fourth matrix are injected sequentially. The target output order is: the first triangular area of ​​the operation result matrix is ​​output sequentially from the hth row to the 1st row and from the Nth column to the fth column; and the second triangular area of ​​the operation result matrix is ​​output sequentially from the 1st row to the kth row and from the sth column to the Nth column. When the target calculation mode is the fourth calculation mode, the first injection order is the order in which vectors from the Mth row to the gth row of the first matrix are injected sequentially; the second injection order is the order in which vectors from the 1st column to the eth column of the second matrix are injected sequentially; the third injection order is the order in which vectors from the rth row to the 1st row of the third matrix are injected sequentially; and the fourth injection order is the order in which vectors from the 1st column to the tth column of the fourth matrix are injected sequentially. The target output order is: the first triangular area of ​​the operation result matrix is ​​output sequentially from the gth row to the Mth row and from the 1st column to the eth column; and the second triangular area of ​​the operation result matrix is ​​output sequentially from the rth row to the 1st row and from the tth column to the 1st column. Where M is less than or equal to N, g is 1, s is 2, h is M, and t is N-1; when M is greater than N, g is 2, s is 1, h is M-1, and t is N. When M is less than N, e is N-1, k is M, f is 2, and r is 1; when M is greater than or equal to N, e is N, k is M-1, f is 1, and r is 2.

[0113] Optionally, the first data loading module is connected to the first PE of the first triangular region PE and the second PE of the second triangular region PE, wherein the first PE is the PE in the g-th row and 1-th column of the first triangular region PE, and the second PE is the PE in the M-th row and e-th column of the first triangular region PE. The second data loading module is connected to the third PE of the lower corner area PE and the fourth PE of the second triangular area PE. The third PE is the PE in the kth row and Nth column of the second triangular area PE, and the fourth PE is the PE in the 1st row and sth column of the second triangular area PE. The data writing module is connected to the first row of PEs in the PE array; Step 802 includes: The first data loading module injects the row vectors of the first matrix read from the first PE into the first triangular region PE; the second data loading module injects the column vectors of the second matrix read from the third PE into the first triangular region PE. And / or, the first data loading module injects the row vector of the read third matrix from the second PE into the second triangular region PE; the second data loading module is used to inject the column vector of the read fourth matrix from the fourth PE into the second triangular region PE.

[0114] Optionally, step 802 includes: The first data loading module injects the row vectors of the first matrix read from the SRAM in the first K-1 consecutive reads into the first PE from row M to row M-K+2 of the first triangular region PE, and injects the row vector of the first matrix read from the SRAM in the first PE into the first PE; K is the number of times the first data loading module reads the row vectors of the first matrix from the SRAM, and K is an integer less than or equal to M; The second data loading module injects the column vectors of the second matrix read from the SRAM in the first L-1 times sequentially into the first column of the first triangular region PE from the first column to the L-1th column of the PE, and injects the column vector of the second matrix read from the SRAM in the Lth time into the third PE. L is the number of times the second data loading module reads the column vector of the second matrix from the SRAM, and L is an integer less than or equal to N. And / or, The first data loading module injects the row vectors of the third matrix read from the SRAM in the first P-1 times from the second PE into the first row to the P-1th row of the second triangular region PE, and injects the row vector of the third matrix read from the SRAM in the Pth time into the second PE. P is the number of times the first data loading module reads the row vectors of the third matrix from the SRAM, and P is an integer less than or equal to M. The second data loading module injects the column vectors of the fourth matrix read from the SRAM in the first Q-1 times from the fourth PE into the Nth to N-Q+2th columns of the second triangular region PE, respectively, and injects the column vector of the fourth matrix read from the SRAM in the Qth time into the fourth PE. Q is the number of times the second data loading module reads the column vectors of the fourth matrix from the SRAM, and Q is an integer less than or equal to N.

[0115] Optionally, the PE array adopts a pipelined design, and the number of pipelines in the PE array is less than the number of cycles required for the PE array to load data. The number of cycles includes the number of injections required to inject the row vectors of the first matrix and the number of injections required to inject the column vectors of the second matrix into the PE array.

[0116] This application embodiment serves as a comparison with... Figure 6 The method implementation corresponding to the processor embodiment shown can be found in the relevant descriptions in the foregoing embodiments, and can achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0117] The embodiments of this application have the following beneficial effects: 1) Eliminating Physical Computation and Power Waste in Ineffective Regions: This application employs a triangle-aware pulsating array geometry, physically arranging the physical pairs (PEs) in a lower or upper triangular shape, ensuring the number of PEs matches the effective computation area. Compared to a square array of the same size, this application structurally avoids arranging PEs in the upper triangular ineffective region, preventing the execution of physical MAC operations and transistor switching in that region, thereby eliminating computational and power waste in the upper triangular ineffective region of causal attention at the hardware level.

[0118] 2) Improved utilization of diagonal edge blocks: This application employs dedicated data flow and mapping rules for the effective triangular domain, designing specific injection points, propagation directions, and beat sequences for the lower and upper triangles. Through a fixed mapping relationship between vector identifiers and PE positions, each PE receives and participates only in the calculation of its unique corresponding matrix element, resulting in a one-to-one correspondence between the matrix elements and PEs with no redundant calculations. When processing diagonal edge blocks, the MAC utilization of the triangular array approaches 100%, significantly outperforming the approximately 50% utilization of the rectangular array.

[0119] 3) Eliminating pipeline bubbles and improving array energy efficiency: The triangular PE arrangement and DP-MM data flow in this application are specifically designed for the triangular effective domain, and the data flow advancement method is consistent with the geometry of the triangular computation domain. During lower triangular computation, row vector A is injected from the upper left corner and passed to the lower right corner, while column vector B is injected from the lower right corner and passed to the upper left corner; during upper triangular computation, the data injection point and transmission direction are adjusted accordingly. This data flow, which matches the geometry of the effective domain, eliminates pipeline bubbles generated when rectangular data flows cover the triangular computation domain, thereby improving the overall array utilization and energy efficiency.

[0120] 4) Eliminating Invalid Transistor Toggling and Routing Overhead at the Source: This application fundamentally eliminates the physical hardware presence of invalid regions by redesigning the array topology at the physical level and directly adopting a triangular mesh instead of a rectangular mesh. Unlike schemes that disable some PEs on a square array through indexing or masking, this invention does not require retaining the physical hardware of invalid regions, avoiding invalid transistor toggling and routing overhead, and achieving the elimination of invalid computation at the source.

[0121] 5) Reduced scheduling overhead and flexible adaptation to different computing needs: This application decouples the physical array type from the computing target through the flexible configuration capability of homogeneous arrays. The homogeneous triangular array can flexibly switch computing modes through data flow reconfiguration, supporting various deployment forms such as double lower triangle, double upper triangle, and lower-upper-lower triangle. When the sequence length and block size change, there is no need for complex mask or index scheduling; different computing needs can be adapted simply by changing the vector injection order, reducing scheduling overhead while maintaining high utilization.

[0122] This application also provides an electronic device, including a central processing unit (CPU), a memory, and the processor provided in the foregoing embodiments, wherein the control module in the processor is connected to the CPU via a configuration bus, and the SRAM in the processor is connected to the memory via a data bus.

[0123] This application embodiment serves as a comparison with... Figure 6 The specific implementation of the electronic device side embodiment corresponding to the processor embodiment shown can be found in the relevant descriptions in the foregoing embodiments. To avoid repetition, it will not be repeated here.

[0124] The electronic device in this application embodiment can be a terminal, or it can be any other device besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, handheld computer, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not make specific limitations.

[0125] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0126] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0127] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0128] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art, under the guidance of this application, can make many modifications without departing from the spirit and scope of the claims, all of which fall within the protection scope of this application. The technical solutions in the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application fall within the protection scope of this application.

[0129] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0130] To make the embodiments of this application clearer, the relevant technical knowledge involved in the embodiments of this application will be introduced as follows: When using block-based computation (such as FlashAttention-style block-based computation), the effective regions in the diagonal edge blocks (boundary tiles) are shaped like lower or upper triangles. If rectangular computation arrays are still used to process these blocks, approximately 50% of the computation will be locally invalid within the block. In scenarios such as Sliding Window Attention (SWA), the blocks contain three effective region forms simultaneously: lower triangles, upper triangles, and rectangles. If rectangular arrays are used to process them uniformly, it will lead to a decrease in MAC utilization and energy loss.

[0131] In existing technologies, software optimizations such as block sparsity and FlashAttention mainly reduce memory access and macroscopic computation. However, due to the underlying hardware being a regular rectangular computing array, there is still invalid computation and power waste on the diagonal edge blocks. Although hardware solutions include sparse matrix multiplication, reconfigurable arrays, and dedicated attention units, they mostly adopt scheduling methods that disable some PEs on the square array through indexing or masking, rather than designing the array physical topology as a triangular mesh and coordinating data flow and vector identification mapping specifically for the triangular effective domain.

[0132] Existing solutions typically have the following shortcomings: 1) Invalid computation and power consumption: The upper triangle result in causal attention is invalid. Traditional square array pulse arrays still perform MAC on this area, resulting in wasted computation and power consumption.

[0133] 2) Low utilization of block edges: In diagonal edge blocks, the effective area is a triangle. The MAC utilization of rectangular arrays on this type of block is about 50%, which is obviously wasteful.

[0134] 3) Mismatch between array topology and computational domain geometry: General-purpose systolic arrays use a rectangular mesh topology, with data flow advancing in a rectangular wavefront manner; however, the effective computational domain for causal attention is the lower triangular geometry. When the rectangular data flow covers the triangular computational domain, the PE in the upper triangular region generates idle cycles (pipeline bubbles) due to the lack of effective computational tasks, resulting in a decrease in the overall array utilization and energy efficiency.

[0135] 4) Masking solutions are not a fundamental solution: Existing solutions often disable some PEs on the matrix by indexing or masking. Although this can reduce some computation, it still retains the rectangular topology and rectangular data flow. The physical hardware of the invalid area still exists, and it cannot eliminate invalid transistor switching and wiring overhead from the source.

[0136] 5) Insufficient or costly scalability / reconfigurability support: Maintaining high utilization and avoiding complex mask / index scheduling overhead remains a challenge when sequence length and block size vary.

[0137] To overcome the problems of existing technologies, this application proposes a novel matrix operation technique, defined as Diagonal-Partitioned Matrix Multiply (DP-MM). Specifically, this application provides a DP-MM acceleration architecture and method for causal attention, aligning the DP-MM hardware with the effective domains of the upper / lower triangular matrices at the structural and dataflow levels. This reduces invalid computations and memory accesses, improves the utilization of block edges and energy efficiency, and supports adaptation to different sequence lengths and patterns.

[0138] The technical solution of this application differs from the "block sparse / mask-aware scheduling" in the following ways: by using a defined triangular PE arrangement, fixed injection points and propagation directions, and a one-to-one mapping between vector identifiers and PE coordinates, the hardware natively implements multiplication and accumulation only for the effective regions of the lower or upper triangles, without relying on general sparse scheduling or software-issued block-by-block activation maps. This results in a structure consistent with the effective field shape of causal attention, reducing invalid MACs and power consumption, and adapting to different sequence lengths and block partitioning strategies.

[0139] The PE array and processor provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0140] Please see Figure 1 , Figure 1 The PE array structure diagram provided in the embodiments of this application is as follows: Figure 1 As shown, the PE array includes: The M-row N-column processing element (PE) is divided into a first triangular region PE and a second triangular region PE according to the target diagonal. The target diagonal is composed of PEs with the same row and column numbers in the M-row N-column PE. M and N are both integers greater than 1. When M is less than or equal to N, the first triangular region PE includes the target diagonal. When M is greater than N, the first triangular region PE does not include the target diagonal. The PE array in the i-th row and j-th column is connected to the PE array in the (i+1)-th row and j+1-th column. The PE array in the M-th row of the first triangular region is connected to the adjacent columns of PE. The PE array in the 1-th row of the second triangular region is connected to the adjacent columns of PE. The PE array in the column is connected to the adjacent rows of PE. i is an integer between 1 and M-1, and j is an integer between 1 and N-1. The first triangular region PE is used to perform matrix multiplication operations on the effective first triangular region or effective second triangular region of the first intermediate matrix, and / or the second triangular region PE is used to perform matrix multiplication operations on the effective second triangular region or effective first triangular region of the second intermediate matrix; wherein, the first intermediate matrix and the second intermediate matrix are the same or different, the first intermediate matrix is ​​the multiplication operation matrix of the first matrix and the second matrix, the second intermediate matrix is ​​the multiplication operation matrix of the third matrix and the fourth matrix, and the dimensions of the first intermediate matrix and the second intermediate matrix are adapted to the PE array.

[0141] The PE array provided in this embodiment is a causal attention systolic computation PE array, such as... Figure 1 As shown, the PE array comprises M rows and N columns of PEs, with a total of M×N PE elements. Specifically, when M equals N, the PE array is a square PE array structure. For ease of understanding, this application embodiment mainly uses M=N as an example to illustrate the PE array structure in this application embodiment.

[0142] In this embodiment, the PE array can be divided into two parts along the target diagonal, namely the first triangular region PE and the second triangular region PE. The target diagonal can be assumed to be the diagonal line where the PEs with the same row and column numbers are located in the PE array. The first triangular region PE and the second triangular region PE can be the lower triangular region and the upper triangular region of the PE array, respectively. Figure 1 As shown, based on the four PEs PE00, PE11, PE22 and PE33 on the diagonal, the system is divided into a lower triangular PE array and an upper triangular PE array. When M is less than or equal to N, the lower triangular PE can be defined to include the target diagonal. When M is greater than N, the lower triangular PE can be defined to exclude the target diagonal.

[0143] For example, such as Figure 1 As shown, the PE array consists of 16 PE units in 4 rows and 4 columns. Each PE is used to calculate the multiplication and accumulation results of matrix vectors and accumulate them in a loop to output the final calculation result. This belongs to the output stationary (OS) data stream mode.

[0144] Compared to traditional pulsating arrays, the pulsating PE array in this embodiment can be divided into two parts, a lower triangular region C_L and an upper triangular region C_U, based on the calculated output matrix element positions. The connection relationships and data flow between each PE unit are as follows: Figure 1 The arrows indicate the path: long dashed arrows represent row vector injection paths, short dashed arrows represent column vector injection paths, and solid arrows represent the path for outputting the result matrix.

[0145] Each PE unit in the PE array, i.e., PE(i,j), is responsible for calculating and outputting the element C[i][j] of the result matrix. Each PE participates in the generation of only one element of the result matrix. Each PE performs the multiplication and accumulation calculation of two input vectors (row vector and column vector, respectively), accumulates and updates the data in the register (Reg), performs cyclic accumulation, and finally outputs the calculation result. For example, the structure of the PE unit is as follows: Figure 3 As shown, each PE unit contains 8 multipliers, 1 adder tree, 1 accumulator, and 1 register, enabling the multiplication and accumulation of two vectors. The computation circuitry in the PE employs a multi-step pipelined architecture.

[0146] In this embodiment, the effective lower triangular region of the multiplication matrix can be calculated using the lower triangular region PE of the PE array, or the effective upper triangular region of the multiplication matrix can be mapped to the lower triangular region and then calculated using the lower triangular region PE. Alternatively, the effective upper triangular region of a certain multiplication matrix can be calculated using the upper triangular region PE of the PE array, or the effective lower triangular region of the multiplication matrix can be mapped to the upper triangular region and then calculated using the upper triangular region PE.

[0147] In this embodiment, the effective lower triangular region (PE) and effective upper triangular region (PE) of the PE array can be used to calculate the effective lower triangular region and effective upper triangular region of the same multiplication matrix, respectively. Alternatively, the effective lower triangular region and effective upper triangular region of different multiplication matrices can be calculated using the lower triangular region and effective upper triangular region of the PE array, respectively. Furthermore, the effective lower triangular region of different multiplication matrices can be calculated using the lower triangular region and effective upper triangular region of the PE array, with the effective lower triangular region of one of the multiplication matrices mapped to the upper triangular region before calculation. Or, the effective upper triangular region of different multiplication matrices can be calculated using the lower triangular region and effective upper triangular region of the PE array, with the effective upper triangular region of one of the multiplication matrices mapped to the lower triangular region before calculation.

[0148] It should be noted that the multiplication matrix in the embodiments of this application can refer to the intermediate matrix used in the multiplication of two matrices, or the result matrix of the multiplication of two matrices, such as... Figure 2 (b) shows the matrix C before the matrix operation results have been calculated.

[0149] For ease of understanding, Figure 2 A schematic diagram of matrix calculation is provided. Figure 2In (a), matrix A has 4 row vectors, and matrix B has 4 column vectors, both with a length of 4. C = A·B is a matrix operation, which involves multiplying and adding the row vectors of A and the column vectors of B to obtain the resulting matrix C. The labels within each element of matrix C represent the row vectors of matrix A and the column vectors of matrix B involved in the operation; gray numbers represent the row vector indices of matrix A, and black numbers represent the column vector indices of matrix B. During matrix operations, each row vector and each column vector participates in 4 vector multiplication and addition operations. Matrix operations can be divided diagonally into two parts: a lower triangular region C_L and an upper triangular region C_U, as shown below. Figure 2 (b) Each box corresponds to a PE unit, meaning each box is processed by the PE unit at the corresponding position in the PE array, forming a rectangular computation array. This rectangular computation array has four flexible mapping methods: 1) The upper triangular PE and the lower triangular PE calculate the result matrix C of the same matrix operation, i.e., calculate C_U+C_L, which is equivalent to classical matrix computation; 2) The upper triangular PE and the lower triangular PE calculate the upper triangular matrix C_U and the lower triangular matrix C_L of different matrix operations in parallel; 3) Change the mapping method, and the lower triangular PE is reused to calculate the upper triangular matrix, i.e., calculate the upper triangular matrix C_U1 and C_U2 of different matrix operations in parallel; 4) Change the mapping method, and the upper triangular PE is reused to calculate the lower triangular matrix, i.e., calculate the lower triangular matrix C_L1 and C_L2 of different matrix operations in parallel.

[0150] Figure 1 Schematic PE array and Figure 2 The multiplication operation matrix in (b) corresponds to this.

[0151] It should also be noted that the first intermediate matrix and the second intermediate matrix in the embodiments of this application can be the same matrix or different matrices.

[0152] Specifically, when the third matrix is ​​the same as the first matrix, and the fourth matrix is ​​the same as the second matrix, the first intermediate matrix and the second intermediate matrix are the same, meaning they are intermediate matrices of the same multiplication operation matrix. This applies to different triangular regions of the PE array where the same multiplication operation matrix is ​​calculated separately in the upper and lower triangular regions.

[0153] The third matrix differs from the first matrix, or the fourth matrix differs from the second matrix, or both the third and first matrices differ, and the first and second intermediate matrices differ as well. This means they are intermediate matrices of different multiplication operation matrices, or intermediate matrices of different parts of the same multiplication operation matrix. This situation is applicable when calculating the effective triangular regions of different multiplication operation matrices in the upper and lower triangular regions of a PE array, or when calculating different effective regions of a relatively large multiplication operation matrix. In this way, the multiplication operations of multiple matrices can be calculated at once, or multiple effective parts of a large matrix can be calculated simultaneously.

[0154] In this embodiment, the sizes of the first intermediate matrix and the second intermediate matrix are adapted to the PE array. This can mean that the sizes of the first intermediate matrix and the second intermediate matrix are the same as or smaller than the size of the PE array, so that the effective triangular regions of the first intermediate matrix and the second intermediate matrix can be fully processed in any triangular region of the PE array at once. If the original multiplication matrix size is larger than the PE array size, the multiplication matrix needs to be adapted and cut according to the PE array size before being injected into the PE array for operation. In this case, the first intermediate matrix and the second intermediate matrix can refer to two sub-matrices adapted to the PE size obtained after matrix partitioning a multiplication matrix according to the PE array size.

[0155] Optionally, the PE array includes multiple computing modes, wherein, When the PE array uses the first calculation mode, the first triangular region PE is used to perform matrix multiplication on the effective first triangular region of the first intermediate matrix, and the second triangular region PE is used to perform matrix multiplication on the effective second triangular region of the second intermediate matrix. The second intermediate matrix is ​​the same as the first intermediate matrix. Alternatively, when the PE array uses the second calculation mode, the first triangular region PE is used to perform matrix multiplication on the effective first triangular region of the first intermediate matrix, and the second triangular region PE is used to perform matrix multiplication on the effective second triangular region of the second intermediate matrix, wherein the first intermediate matrix is ​​different from the second intermediate matrix. Alternatively, when the PE array uses a third calculation mode, the first triangular region PE is used to perform matrix multiplication on the effective second triangular region of the first intermediate matrix, and the second triangular region PE is used to perform matrix multiplication on the effective second triangular region of the second intermediate matrix, wherein the first intermediate matrix is ​​different from the second intermediate matrix. Alternatively, when the PE array uses the fourth calculation mode, the first triangular region PE is used to perform matrix multiplication on the effective first triangular region of the first intermediate matrix, and the second triangular region PE is used to perform matrix multiplication on the effective first triangular region of the second intermediate matrix, wherein the first intermediate matrix is ​​different from the second intermediate matrix.

[0156] In some embodiments, the upper and lower triangular regions of the PE array can be reused for computation. Specifically, by adjusting the matrix data arrangement, the lower triangular region of the PE array can be used to calculate the result of the upper triangular region, and vice versa. Therefore, the PE array in the embodiments of this application can support multiple different computation modes: 1) First calculation mode: Upper and lower triangular calculation mode of the same matrix; in this calculation mode, the lower triangular region PE and the upper triangular region PE are used to calculate the effective lower triangular region and effective upper triangular region of the same multiplication operation matrix, respectively.

[0157] 2) Second calculation mode: Upper and lower triangular calculation mode for different matrices; in this calculation mode, the lower triangular region PE and the upper triangular region PE are used to calculate the effective lower triangular region and effective upper triangular region of different multiplication operation matrices, respectively.

[0158] 3) Third calculation mode: double upper triangular calculation mode; in this calculation mode, the lower triangular region PE and the upper triangular region PE are used to calculate the effective upper triangular region of different multiplication operation matrices.

[0159] 4) Fourth calculation mode: Double lower triangular calculation mode; In this calculation mode, the lower triangular region PE and the upper triangular region PE are used to calculate the effective lower triangular region of different multiplication operation matrices.

[0160] The above four computing modes can meet the computing needs of different scenarios and can maintain the PE array at full load.

[0161] In this way, different computing modes of the PE array can be used to compute the effective triangular region of the multiplication matrix in parallel using different triangular regions of PE, thereby reducing the computation of invalid regions, improving the efficiency of matrix multiplication, and saving computing resources and power consumption.

[0162] Optionally, the first PE in the first triangular region PE is used to receive the first target row vector of the first matrix and to receive the second target row vector of the first matrix, and to pass the second target row vector to the corresponding row PE. The first PE is the PE in the g-th row and 1-th column of the first triangular region PE. Wherein, the first target row vector is the g-th row vector of the first matrix, and the second target row vector includes the (g+1)-th row vector to the M-th row vector of the first matrix. When M is less than or equal to N, g is 1, and when M is greater than N, g is 2. Alternatively, the first target row vector is the h-th row vector of the first matrix, and the second target row vector includes the 1-th row vector to the (h-1)-th row vector of the first matrix. When M is less than or equal to N, h is M, and when M is greater than N, h is M-1. The second PE in the first triangular region PE is used to receive the third target column vector of the second matrix and to receive the fourth target column vector of the second matrix, and to pass the fourth target column vector to the corresponding column PE. The second PE is the PE in the M-th row and e-th column of the first triangular region PE. The third target column vector is the e-th column vector of the second matrix, and the fourth target column vector includes the 1st to e-1st column vectors of the second matrix. When M is less than N, e is N-1, and when M is greater than or equal to N, e is N. Alternatively, the third target column vector is the f-th column vector of the first matrix, and the fourth target column vector includes the f+1st to Nth column vectors of the second matrix. When M is less than N, f is 2, and when M is greater than or equal to N, f is 1. And / or, The third PE in the second triangular region PE is used to receive the fifth target row vector of the third matrix and to receive the sixth target row vector of the third matrix, and to pass the sixth target row vector to the corresponding row PE. The third PE is the k-th row and N-th column PE in the second triangular region PE. The fifth target row vector is the k-th row vector of the third matrix, and the sixth target row vector includes the 1st to k-1st row vectors of the third matrix. When M is greater than or equal to N, k is M-1, and when M is less than N, k is M. Alternatively, the fifth target row vector is the r-th row vector of the third matrix, and the sixth target row vector includes the r+1th to M-th row vectors of the third matrix. When M is less than N, r is 1, and when M is greater than or equal to N, r is 2. The fourth PE of the second triangular region PE is used to receive the seventh target column vector of the fourth matrix and to receive the eighth target column vector of the fourth matrix, and to pass the eighth target column vector to the corresponding column PE. The fourth PE is the PE in the first row and s column of the second triangular region PE. The seventh target column vector is the s column vector of the fourth matrix, and the eighth target column vector includes the (s+1)th to the Nth column vector of the fourth matrix. When M is greater than N, s is 1, and when M is less than or equal to N, s is 2. Alternatively, the seventh target column vector is the t column vector of the fourth matrix, and the eighth target column vector includes the first to the (t-1)th column vector of the fourth matrix. When M is less than or equal to N, t is N-1, and when M is greater than N, t is N.

[0163] In some embodiments, for the lower triangular region PE of the PE array, it can be configured to receive row vectors loaded from the first matrix from one PE cell, such as the upper left PE, and column vectors loaded from the second matrix from another PE cell, such as the lower right PE. Based on the connection relationship and data flow between PE cells in the lower triangular region PE, each row vector loaded from the first matrix can be passed to the corresponding row PE, and each column vector loaded from the second matrix can be passed to the corresponding column PE.

[0164] like Figure 4 The diagram illustrates the data flow of matrices A, B, and C during calculations in the upper and lower triangular regions of the same multiplication operation matrix, also known as the DP-MM data flow. This diagram is a representation of... Figure 1 A diagram illustrating the splitting of the data stream, and related to... Figure 2 (b) Correspondingly, the box represents PE.

[0165] See Figure 4 When calculating C_L in the lower triangular region PE, such as Figure 4 As shown in the lower left corner of (a), the row vectors of matrix A are injected into the PE array from the upper left PE and propagated towards the lower right corner, sequentially injecting the 3rd, 2nd, 1st, and 0th row vectors. When a PE connects to two other PEs, it will copy the data twice and then pass it to these two PEs, as shown below. Figure 4 In (a), PE00 passes the same data to PE10 and PE11.

[0166] like Figure 4 As shown in the lower left corner of (b), the column vectors of matrix B are injected from the lower right corner PE and passed to the upper left corner, injecting the 0th, 1st, 2nd and 3rd column vectors in sequence. The PE copying behavior is the same as above.

[0167] Similarly, for the upper triangular region PE of the PE array, it can be set to receive row vectors loaded from the first matrix from one PE cell, such as the lower right PE, and column vectors loaded from the second matrix from another PE cell, such as the upper left PE. Based on the connection relationship and data flow between PE cells in the upper triangular region PE, each row vector loaded from the first matrix can be passed to the corresponding row PE, and each column vector loaded from the second matrix can be passed to the corresponding column PE.

[0168] See Figure 4 When calculating C_U in the upper triangular region PE, such as Figure 4 As shown in the upper right corner of (a), the row vectors of matrix A are injected into the PE array from the lower right corner PE and passed to the upper left corner, injecting the 0th, 1st and 2nd row vectors in sequence. The PE copying behavior is the same as above.

[0169] like Figure 4 As shown in the upper right corner of (b), the column vectors of matrix B are injected from the upper left corner PE and passed to the lower right corner, injecting the 3rd, 2nd and 1st column vectors in sequence. The PE copying behavior is the same as above.

[0170] In some embodiments, the upper triangular region C_U and the lower triangular region C_L of the multiplication matrix to be calculated can be remapped, and then injected into the corresponding lower triangular region PE and upper triangular region PE for calculation. For example, C_L' obtained after remapping the upper triangular region C_U is calculated in the lower triangular region PE of the PE array, and C_U' obtained after remapping the lower triangular region C_L is calculated in the upper triangular region PE of the PE array. That is, the calculations of the upper and lower triangular regions are interchanged.

[0171] like Figure 5 The diagram shows a remapping representation of DP-MM matrix operations. The leftmost sub-diagram represents the diagonal division of the matrix remapping. In this case, the upper right triangular portion of matrix C includes the diagonal and is used as the lower triangular region C_L' after mapping, while the lower left triangular portion of matrix C does not include the diagonal and is used as the upper triangular region C_U' after mapping.

[0172] Figure 5 The right-hand diagram shows the PE array and data flow after the upper and lower triangular regions are remapped. The row vectors of matrix A and the column vectors of matrix B calculated in the PE array have changed compared to the calculations before the mapping.

[0173] When calculating C_L' in the lower triangular region PE, such as Figure 5 As shown in the lower left corner of (a), the row vectors of matrix A are injected into the PE array from the upper left corner PE, and then propagated towards the lower right corner, injecting the 0th, 1st, 2nd, and 3rd row vectors in sequence, as shown in (a). Figure 5 As shown in the lower left corner of (b), the column vectors of matrix B are injected from the lower right corner PE, and the 3rd, 2nd, 1st and 0th column vectors are injected in sequence, and then propagated to the upper left corner.

[0174] When calculating C_U' in the upper triangular region PE, such as Figure 5 As shown in the upper right corner of (a), the row vectors of matrix A are injected into the PE array from the lower right corner PE, and then propagated towards the upper left corner, sequentially injecting the 3rd, 2nd, and 1st row vectors, as follows. Figure 5 As shown in the upper right corner of (b), the column vectors of matrix B are injected from the upper left corner PE and passed to the lower right corner, injecting the 0th, 1st and 2nd column vectors in sequence.

[0175] In this way, the matrix data to be calculated can be accurately and orderly loaded into the corresponding calculation area of ​​the PE array, ensuring the effective triangular area of ​​the PE array for efficient parallel calculation of the multiplication matrix.

[0176] The PE array in this embodiment includes: an M-row N-column processing unit PE, wherein the M-row N-column PE is divided into a first triangular region PE and a second triangular region PE according to a target diagonal. The target diagonal is composed of PEs with the same row and column numbers in the M-row N-column PE, where M and N are both integers greater than 1. When M is less than or equal to N, the first triangular region PE includes the target diagonal; when M is greater than N, the first triangular region PE does not include the target diagonal. The PE in the i-th row and j-th column of the PE array is connected to the PE in the (i+1)-th row and j+1-th column. Adjacent columns of PE in the M-th row of the first triangular region PE are connected to each other, and adjacent columns of PE in the 1-th row of the second triangular region PE are connected to each other. Next, adjacent rows of PEs in each column of the PE array are connected, where i is an integer between 1 and M-1, and j is an integer between 1 and N-1; the first triangular region PE is used to perform matrix multiplication operations on the effective first triangular region or effective second triangular region of the first intermediate matrix, and / or, the second triangular region PE is used to perform matrix multiplication operations on the effective second triangular region or effective first triangular region of the second intermediate matrix; wherein, the first intermediate matrix and the second intermediate matrix are the same or different, the first intermediate matrix is ​​the multiplication operation matrix of the first matrix and the second matrix, the second intermediate matrix is ​​the multiplication operation matrix of the third matrix and the fourth matrix, and the size of the first intermediate matrix and the second intermediate matrix is ​​adapted to the PE array. In this way, by designing a PE array consisting of M rows and N columns of PEs, and dividing the PE array into two triangular regions PE along the diagonal, establishing connection relationships between each PE unit in the PE array according to the data flow direction, and loading the matrix data to be operated on based on the PE connection relationships, the PE array is aligned with the effective triangular region of the multiplication matrix at both the structural and data flow levels. This allows for the targeted calculation of the effective triangular region in the multiplication matrix using different triangular regions PE of the PE array, and even the parallel calculation of the effective triangular regions in two different multiplication matrices. Thus, the embodiments of this application can reduce the calculation and data access of a large number of invalid triangular regions, improve the efficiency of matrix multiplication operations, and save computing resources and power consumption.

[0177] Please see Figure 6 , Figure 6 A processor architecture diagram for accelerating matrix multiplication provided in this application embodiment is shown below. Figure 6 As shown, the processor includes: The PE array 601 described in the above embodiments also includes a control module 602, a first data loading module 6031, a second data loading module 6032, a data writing module 604, and a static random-access memory (SRAM) 605. The control module is connected to the first data loading module, the second data loading module, and the write data module, respectively. The first data loading module, the second data loading module, and the write data module are all connected to the PE array and the SRAM. The control module is used to configure matrix operation parameters for the first data loading module, the second data loading module, and the data writing module respectively based on the configured matrix operation task information; The first data loading module is configured to read the row vector of the first matrix from the SRAM based on the matrix operation parameters, and inject the read row vector of the first matrix into the first triangular region PE of the PE array; the second data loading module is configured to read the column vector of the second matrix from the SRAM based on the matrix operation parameters, and inject the read column vector of the second matrix into the first triangular region PE of the PE array; and / or, the first data loading module is configured to read the row vector of the third matrix from the SRAM based on the matrix operation parameters, and inject the read row vector of the third matrix into the second triangular region PE of the PE array; the second data loading module is configured to read the column vector of the fourth matrix from the SRAM based on the matrix operation parameters, and inject the read column vector of the fourth matrix into the second triangular region PE of the PE array; The PE array is used to perform matrix multiplication operations on the first matrix and the second matrix through the first triangular region PE, and / or to perform matrix multiplication operations on the third matrix and the fourth matrix through the second triangular region PE; The data writing module is used to write the matrix multiplication result of the PE array into the SRAM based on the matrix operation parameters.

[0178] In this embodiment, the third matrix may be the same as or different from the first matrix, and the fourth matrix may be the same as or different from the second matrix. In some embodiments, the third matrix is ​​the same as the first matrix, and the fourth matrix is ​​the same as the second matrix. This corresponds to application scenarios where different triangular regions of the PE array are used to process the upper and lower triangular regions of the same multiplication matrix, or where the upper or lower triangular region of the PE array is used to process the effective triangular region of a multiplication matrix. In other embodiments, the third matrix is ​​different from the first matrix, and the fourth matrix is ​​different from the second matrix. This corresponds to application scenarios where different triangular regions of the PE array are used to process the effective triangular regions of different multiplication matrices, or where the upper triangular region of the PE array is used to process the effective triangular region of a multiplication matrix, or where the lower triangular region of the PE array is used to process the effective triangular region of another multiplication matrix.

[0179] Figure 6 This is a schematic diagram of the overall architecture of the accelerated processor for DP-MM oriented towards causal attention, provided in an embodiment of this application. Figure 6 As shown, the processor includes a control module 602 (CTRL), a PE array 601 supporting DP-MM, two data loading modules, a write data module (C Store) 604, and an SRAM 605. The two data loading modules are the first data loading module 6031, referred to as A Loader, and the second data loading module 6032, referred to as B Loader.

[0180] The control module 602 receives computation task information configured by the CPU, i.e., matrix operation task information, via the configuration bus (CFG_BUS). This information may include the computation mode, computational load, and read / write addresses of matrices A and B to be operated on. Based on the computation task information configured by the CPU, the control module 602 parses the computation task and configures the corresponding computation mode, matrix size, and other relevant parameters for the data loading module and the data writing module. During runtime, the control module 602 is responsible for coordinating data interaction and pipeline between other modules.

[0181] The first data loading module 6031, such as A Loader, is used to generate a corresponding SRAM read request based on the matrix operation parameters configured by the control module 602, such as the calculation mode and matrix size, and inject the row vector data of the read A matrix into the PE array.

[0182] The second data loading module 6032, such as B Loader, is used to generate a corresponding SRAM read request based on the matrix operation parameters configured by the control module 602, such as the calculation mode and matrix size, and inject the column vector data of the read B matrix into the PE array.

[0183] The PE array 601 is the core computing unit. For example, the PE array 601 can be constructed from 4 rows and 4 columns of PEs to form an upper and lower triangular data flow structure. It supports four computing modes: 1) upper and lower triangular computing mode for the same matrix; 2) upper and lower triangular computing mode for different matrices; 3) double lower triangular computing mode; and 4) double upper triangular computing mode. The PE array 601 can perform matrix multiplication operations on the row vectors of the loaded matrix A and the column vectors of matrix B through the lower triangular PEs, and it can also perform matrix multiplication operations on the row vectors of the loaded matrix A and the column vectors of matrix B through the upper triangular PEs.

[0184] The write data module 604, such as C Store, is used to generate corresponding SRAM write requests based on the calculation mode, matrix size, and other information configured by the control module 602, and write the data output from the PE array back to the SRAM. In some embodiments, the write data module 604 also includes a data mapping submodule (C remap). When calculating the remapped upper and lower triangular matrices in the lower or upper triangular PE region, the data mapping submodule C remap in the write data module 604 can be used to adjust the data format according to the calculation mode before writing it back to the SRAM.

[0185] The SRAM605 is an on-chip buffer that supports high-bandwidth read and write data through the first data loading module 6031 A Loader, the second data loading module 6032 B Loader, and the write data module 604 C Store. It can exchange data with external storage via the data bus.

[0186] Optionally, the matrix operation task information includes matrix information to be multiplied, the matrix information including information of the target intermediate matrix and the valid and / or invalid triangular regions in the target intermediate matrix, the target intermediate matrix being the multiplication matrix of the first target matrix and the second target matrix to be multiplied; The control module is used to determine a target calculation mode for the target intermediate matrix based on the valid and / or invalid triangular regions in the target intermediate matrix. The target calculation mode is used to instruct the valid triangular regions in the target intermediate matrix to perform operations on the target triangular region PE in the PE array. The target triangular region PE is one of the first triangular region PE and the second triangular region PE. The matrix operation parameters include the calculation mode and the matrix information. The target calculation mode is one of the multiple calculation modes supported by the PE array. Wherein, when the size of the target intermediate matrix is ​​equal to the size of the PE array, the first target matrix and the second target matrix are respectively the first matrix and the second matrix, the target intermediate matrix is ​​the first intermediate matrix, and the first intermediate matrix and the second intermediate matrix are the same matrix; Alternatively, if the size of the target intermediate matrix is ​​larger than the size of the PE array, the control module is further configured to divide the target intermediate matrix into multiple sub-intermediate matrices according to the size of the PE array, wherein the first intermediate matrix and the second intermediate matrix are any two of the multiple sub-intermediate matrices, the first intermediate matrix and the second intermediate matrix are different matrices, the first matrix and the third matrix are sub-matrices in the first target matrix, and the second matrix and the fourth matrix are sub-matrices in the second target matrix.

[0187] In some embodiments, the matrix operation task information configured by the CPU includes matrix information to be multiplied, specifically including information of the target intermediate matrix and the valid and / or invalid triangular regions in the target intermediate matrix. The target intermediate matrix is ​​the multiplication matrix of the first target matrix and the second target matrix to be multiplied.

[0188] The control module 602 can determine which calculation mode to use for the target intermediate matrix based on the valid and / or invalid triangular regions in the target intermediate matrix. Therefore, based on the target calculation mode, it can determine which triangular region (PE) in the PE array to place the valid triangular regions of the target intermediate matrix for computation. It should be noted that in some embodiments, the CPU can directly specify which calculation mode the control module 602 should use.

[0189] When the size of the target intermediate matrix is ​​comparable to that of the PE array, the matrix can be directly divided into upper and lower triangular regions along the diagonal, and the corresponding regions in the PE array can be processed. When the target intermediate matrix is ​​a large matrix, such as one much larger than the size of the PE array, the control module 602 can cut the target intermediate matrix according to the size of the PE array, divide it into multiple sub-intermediate matrices, and then process the different effective triangular regions of the multiple intermediate sub-matrices in parallel through the corresponding triangular regions PE of the PE array.

[0190] In this way, by adapting and cutting the multiplication matrix to be operated on according to the size of the PE array, the multiplication efficiency of the PE array for large-sized matrices can be improved, and multiple matrix block multiplication operations can be run simultaneously.

[0191] Optionally, the first data loading module is configured to determine a first injection order for the row vectors of the first matrix based on the target calculation mode, and sequentially read the row vectors of the first matrix from the SRAM according to the first injection order, and inject the row vectors of the first matrix read each time into the first triangular region PE; the second data loading module is configured to determine a second injection order for the column vectors of the second matrix based on the target calculation mode, and sequentially read the column vectors of the second matrix from the SRAM according to the second injection order, and inject the column vectors of the second matrix read each time into the first triangular region PE; And / or, the first data loading module is configured to determine a third injection order for the row vectors of the third matrix based on the target calculation mode, and based on the third injection order, sequentially read the row vectors of the third matrix from the SRAM, and inject the row vectors of the third matrix read each time into the second triangular area PE; the second data loading module is configured to determine a fourth injection order for the column vectors of the fourth matrix based on the target calculation mode, and based on the fourth injection order, sequentially read the column vectors of the fourth matrix from the SRAM, and inject the column vectors of the fourth matrix read each time into the second triangular area PE; The data writing module is used to determine the target output order of the matrix multiplication results of the PE array based on the target calculation mode, output the matrix multiplication results according to the target output order, and write the output matrix multiplication results into the SRAM in sequence.

[0192] In some embodiments, the first data loading module may determine the first injection order of the row vectors of the first matrix based on the calculation mode configured by the control module, and read the row vectors of the first matrix from the SRAM in sequence according to the first injection order, and inject them into the corresponding row PE of the lower triangular region PE in batches.

[0193] The second data loading module can also determine the second injection order of the column vectors of the second matrix based on the calculation mode configured by the control module, and read the column vectors of the second matrix from the SRAM in sequence according to the second injection order, and inject them into the corresponding column PE of the lower triangular region PE in batches.

[0194] The first data loading module can also determine the third injection order of the row vectors of the third matrix based on the calculation mode configured by the control module, and read the row vectors of the third matrix from the SRAM in sequence according to the third injection order, and inject them into the corresponding row PE of the upper triangular region PE in batches.

[0195] The second data loading module can also determine the fourth injection order of the column vectors of the fourth matrix based on the calculation mode configured by the control module, and read the row vectors of the fourth matrix from the SRAM in sequence according to the fourth injection order, and inject them into the corresponding column PE of the upper triangular region PE in batches.

[0196] Optionally, when the target calculation mode is the first calculation mode, the first injection order is the order in which vectors from the Mth row to the gth row of the first matrix are injected sequentially; the second injection order is the order in which vectors from the 1st column to the eth column of the second matrix are injected sequentially; the third injection order is the order in which vectors from the 1st row to the kth row of the first matrix are injected sequentially; the fourth injection order is the order in which vectors from the Nth column to the sth column of the second matrix are injected sequentially; the target output order is the order in which vectors from the 1st row to the Mth row and from the 1st column to the Mth column of the operation result matrix are output sequentially; and the operation result matrix is ​​the matrix corresponding to the matrix multiplication operation result. When the target calculation mode is the second calculation mode, the first injection order is the order in which vectors from the Mth row to the gth row of the first matrix are injected sequentially; the second injection order is the order in which vectors from the 1st column to the eth column of the second matrix are injected sequentially; the third injection order is the order in which vectors from the 1st row to the kth row of the third matrix are injected sequentially; the fourth injection order is the order in which vectors from the Nth column to the sth column of the fourth matrix are injected sequentially; and the target output order is the order in which vectors from the 1st row to the Mth row and from the 1st column to the Mth column of the result matrix are output sequentially. When the target calculation mode is the third calculation mode, the first injection order is the order in which vectors from the 1st row to the hth row of the first matrix are injected sequentially; the second injection order is the order in which vectors from the Nth column to the fth column of the second matrix are injected sequentially; the third injection order is the order in which vectors from the 1st row to the kth row of the third matrix are injected sequentially; and the fourth injection order is the order in which vectors from the Nth column to the sth column of the fourth matrix are injected sequentially. The target output order is: the first triangular area of ​​the operation result matrix is ​​output sequentially from the hth row to the 1st row and from the Nth column to the fth column; and the second triangular area of ​​the operation result matrix is ​​output sequentially from the 1st row to the kth row and from the sth column to the Nth column. When the target calculation mode is the fourth calculation mode, the first injection order is the order in which vectors from the Mth row to the gth row of the first matrix are injected sequentially; the second injection order is the order in which vectors from the 1st column to the eth column of the second matrix are injected sequentially; the third injection order is the order in which vectors from the rth row to the 1st row of the third matrix are injected sequentially; and the fourth injection order is the order in which vectors from the 1st column to the tth column of the fourth matrix are injected sequentially. The target output order is: the first triangular area of ​​the operation result matrix is ​​output sequentially from the gth row to the Mth row and from the 1st column to the eth column; and the second triangular area of ​​the operation result matrix is ​​output sequentially from the rth row to the 1st row and from the tth column to the 1st column. Where M is less than or equal to N, g is 1, s is 2, h is M, and t is N-1; when M is greater than N, g is 2, s is 1, h is M-1, and t is N. When M is less than N, e is N-1, k is M, f is 2, and r is 1; when M is greater than or equal to N, e is N, k is M-1, f is 1, and r is 2.

[0197] Specifically, in combination Figure 4 and Figure 5 Examples illustrating the vector injection order under each computation mode are provided below: like Figure 4 The diagram shows the data flow of matrices A, B, and C during the calculation of the upper and lower triangular regions of the same matrix, which is also the data flow of DP-MM.

[0198] When calculating the lower triangular region C_L of the matrix PE in the lower triangular region, such as Figure 4 As shown in the lower left corner of (a), the row vectors of matrix A are injected into the PE array from the upper left corner and propagated to the lower right corner. The injection order of the row vectors is 3, 2, 1, 0, that is, from row 4 to row 1, which takes 4 clock cycles to complete. Figure 4As shown in the lower left corner of (b), the column vectors of matrix B are injected from the lower right corner and passed to the upper left corner. The order of column vector injection is 0, 1, 2, 3, that is, from column 1 to column 4 in sequence. It takes 4 cycles to complete the loading, which is the same as the number of cycles for injecting the row vectors of matrix A.

[0199] When calculating the upper triangular region C_U of the matrix PE in the upper triangular region, such as Figure 4 As shown in the upper right corner of (a), the row vectors of matrix A are injected into the PE array from the lower right corner and propagated towards the upper left corner. The injection order of the row vectors is 0, 1, 2, that is, from row 1 to row 3, which takes 3 clock cycles to complete. Figure 4 As shown in the upper right corner of (b), the column vectors of matrix B are injected from the upper left corner and passed to the lower right corner. The order of column vector injection is 3, 2, 1, that is, from the 4th column to the 2nd column in sequence, which is the opposite of the injection order of the row vectors of matrix A. It takes 3 cycles to complete the loading, which is the same as the number of cycles for the row vector injection of matrix A.

[0200] The injected data is a vector. After matrices A and B are loaded into the PE array, each PE calculates the product and sum of the row vectors of matrix A and the column vectors of matrix B, and adds the sum to the original register in the PE. The final calculated result, matrix C, is output step-by-step from the top of the PE array, requiring 4 steps. The output rows of matrix C are in the order of rows 0, 1, 2, and 3, and the elements within each row are in the order of columns 0, 1, 2, and 3.

[0201] like Figure 5 The diagram illustrates the remapping of DP-MM matrix operations. The leftmost sub-diagram shows the diagonal partitioning method for matrix remapping. In this case, the upper right part of matrix C includes the diagonal and serves as the lower triangle C_L' after mapping; the lower left part of matrix C does not include the diagonal and serves as the upper triangle C_U' after mapping. After remapping, C_L' is calculated in the lower left corner of the PE array, while C_U' is calculated in the upper right corner of the PE array, meaning the calculations for the upper and lower triangles are interchanged.

[0202] Figure 5 The diagram on the right shows the PE array and data flow after the upper and lower triangular remapping. The row vectors of matrix A and the column vectors of matrix B calculated in PE have changed compared to the calculations before the mapping.

[0203] When calculating C_L' in the lower triangular region PE, such as Figure 5 As shown in the lower left corner of (a), the row vectors of matrix A are injected in the order of 0, 1, 2, 3, that is, from row 1 to row 4. Figure 5 As shown in the lower left corner of (b), the column vectors of matrix B are injected in the order of 3, 2, 1, 0, that is, from the 4th column to the 1st column, which is the opposite of the order of the row vectors of matrix A. It takes 4 cycles to complete the loading, which is the same as the number of cycles for injecting the row vectors of matrix A.

[0204] When calculating C_U' in the upper triangular region PE, such as Figure 5 As shown in the upper right corner of (a), the row vectors of matrix A are injected in the order of 3, 2, 1, that is, from row 4 to row 2, requiring 3 clock cycles to complete the loading. Figure 5 As shown in the upper right corner of (b), the column vectors of matrix B are injected in the order of 0, 1, 2, that is, from column 1 to column 3 in sequence, which is the opposite of the order of the row vectors of matrix A. It takes 3 cycles to complete the loading, which is the same as the number of cycles for injecting the row vectors of matrix A.

[0205] At this point, the C' matrix after the calculations of C_L' and C_U' is a matrix rotated 180 degrees from the original C matrix, meaning the C matrix is ​​first flipped vertically and then horizontally. The output row order of the C' matrix is ​​3, 2, 1, 0, meaning it is injected sequentially from row 4 to row 1, and the internal element order of each row is 3, 2, 1, 0, meaning it is injected sequentially from column 4 to column 1. Therefore, this data flow scheme for remapping the upper and lower triangular regions can still complete the correct GEMM calculation.

[0206] It should be noted that the matrix operations in this example are intended to illustrate that calculations of the upper and lower triangular regions can be interchanged within the same rectangular PE array composed of upper and lower triangles. In practice, calculations within the same matrix do not require this mapping mode, but when calculating the upper and lower triangular regions of different matrices, an appropriate mapping method can be selected.

[0207] The above embodiments illustrate the correctness and feasibility of the upper and lower triangular region remapping. The implementation of upper and lower triangular region remapping is simple: it only requires changing the loading order of the row vectors of matrix A and the column vectors of matrix B into the PE matrix, and changing the order of the output rows and elements within those rows of matrix C. This example demonstrates that upper and lower triangular region calculations can be mutually converted. It also proves that this array structure can support four calculation modes, and that PE computing resources maintain high utilization across different calculation modes.

[0208] Optionally, the first data loading module is connected to the first PE of the first triangular region PE and the second PE of the second triangular region PE, wherein the first PE is the PE in the g-th row and 1-th column of the first triangular region PE, and the second PE is the PE in the M-th row and e-th column of the first triangular region PE. The second data loading module is connected to the third PE of the lower corner area PE and the fourth PE of the second triangular area PE. The third PE is the PE in the kth row and Nth column of the second triangular area PE, and the fourth PE is the PE in the 1st row and sth column of the second triangular area PE. The data writing module is connected to the first row of PEs in the PE array; The first data loading module is used to inject the row vector of the read first matrix from the first PE into the first triangular region PE, and the second data loading module is used to inject the column vector of the read second matrix from the third PE into the first triangular region PE; and / or, the first data loading module is used to inject the row vector of the read third matrix from the second PE into the second triangular region PE, and the second data loading module is used to inject the column vector of the read fourth matrix from the fourth PE into the second triangular region PE.

[0209] In some embodiments, such as Figure 1 and Figure 6 As shown, the first data loading module 601 can be connected to the upper left corner PE of the lower triangular area PE and to the lower right corner PE of the upper triangular area PE, the second data loading module 602 can be connected to the lower right corner PE of the lower triangular area PE and to the upper left corner PE of the upper triangular area PE, and the write data module 605 can be connected to the first row PE of the PE array.

[0210] This ensures that the row vectors of the first matrix read from SRAM are sequentially injected into the corresponding rows of the lower triangular region PE from the top left corner PE, the column vectors of the second matrix read from SRAM are sequentially injected into the corresponding columns of the lower triangular region PE from the bottom right corner PE, the row vectors of the first matrix read from SRAM are sequentially injected into the corresponding rows of the upper triangular region PE from the bottom right corner PE, and the column vectors of the second matrix read from SRAM are sequentially injected into the corresponding columns of the upper triangular region PE from the top left corner PE, and guarantees that the final matrix operation results are output in order from the first row PE.

[0211] Optionally, the first data loading module is used to inject the row vectors of the first matrix read from the SRAM K times sequentially from the first PE into rows M to M-K+1 of the first triangular region PE. Specifically, the first data loading module is used to inject the row vectors of the first matrix read from the SRAM K-1 times sequentially from the first PE into rows M to M-K+2 of the first triangular region PE, and to inject the row vector of the first matrix read from the SRAM K times into the first PE, where K is the number of times the first data loading module reads the row vectors of the first matrix from the SRAM, and K is a positive integer less than or equal to M. The second data loading module is used to inject the column vectors of the second matrix read from the SRAM L times sequentially from the third PE into the first column to the Lth column of the first triangular region PE. Specifically, the second data loading module is used to inject the column vectors of the second matrix read from the SRAM L-1 times sequentially from the third PE into the first column to the L-1th column of the first triangular region PE, and to inject the column vector of the second matrix read from the SRAM Lth time into the third PE, where L is the number of times the second data loading module reads the column vectors of the second matrix from the SRAM, and L is a positive integer less than or equal to N. And / or, The first data loading module is used to inject the row vectors of the third matrix read from the SRAM P times sequentially from the second PE into rows 1 to P of the second triangular area PE. Specifically, the first data loading module is used to inject the row vectors of the third matrix read from the SRAM P-1 times sequentially from the second PE into rows 1 to P-1 of the second triangular area PE, and to inject the row vector of the third matrix read from the SRAM P times into the second PE. P is the number of times the first data loading module reads the row vectors of the third matrix from the SRAM, and P is a positive integer less than or equal to M. The second data loading module is used to inject the column vectors of the fourth matrix read from the SRAM Q times sequentially from the fourth PE into columns N to N-Q+1 of the second triangular region PE. Specifically, the second data loading module is used to inject the column vectors of the fourth matrix read from the SRAM Q-1 times sequentially from the fourth PE into columns N to N-Q+2 of the second triangular region PE, and to inject the column vector of the fourth matrix read from the SRAM Q times into the fourth PE. Q is the number of times the second data loading module reads the column vectors of the fourth matrix from the SRAM, and Q is a positive integer less than or equal to N.

[0212] In some embodiments, both the first data loading module and the second data loading module can inject the row / column vectors of the corresponding matrix read each time into the PE at the corresponding row position according to their respective SRAM data reading order, so as to ensure the accuracy and effectiveness of matrix data loading.

[0213] Optionally, the PE array adopts a pipelined design, and the number of pipelines in the PE array is less than the number of cycles required for the PE array to load data. The number of cycles includes the number of injections required to inject the row vectors of the first matrix into the PE array and the number of injections required to inject the column vectors of the second matrix.

[0214] In some embodiments, to ensure high processing efficiency, the data loading module and the data writing module can guarantee the data bit width required by the PE array. The PE array 601 contains a computation pipeline, and the number of pipeline stages is no greater than the number of cycles required for data loading.

[0215] Specifically, during calculations in the upper and lower triangular regions, the PE array needs to wait for the data to finish loading before starting the calculation, and only outputs the final result after the cumulative calculation is complete. Since the lower triangular region PE has only one pair of input ports, such as the row vector of matrix A and the column vector of matrix B, ... Figure 1 and Figure 6 Taking a 4×4 array as an example, the data loading time interval for the lower triangular region PE is 4 clock cycles. Since the PE array uses a pipelined design, the number of pipeline stages must not exceed the number of clock cycles required for data loading to complete. To maintain high utilization of the PE array, the vector length loaded by the lower triangular region PE each time must be 4 times the vector length that the PE array pipeline can process per clock cycle. During PE operation, the data is injected into the computation pipeline in 4 clock cycles to ensure uninterrupted computation and to match the data loading interval. Similarly, the data loading time interval for the upper triangular region PE is 3 clock cycles. The vector lengths of matrices A and B loaded each time must be 3 times the vector length that the PE array pipeline can process. During PE operation, the data is injected into the computation pipeline in 3 clock cycles.

[0216] by Figure 7 The example shown illustrates the PE calculation in the lower triangular region. The PE array contains a pipeline consisting of four MACs, meaning the vector length processed per cycle is 4. For example... Figure 7 As shown in (a), when 4 data points of PE are loaded per cycle, the pipeline is underutilized, with only 25% utilization, or 1 / 4. Figure 7 As shown in (b), when 16 data points of PE are loaded per tick, the pipeline is fully pipelined and has 100% utilization.

[0217] In summary, the embodiments of this application have the following major improvements: 1) Triangle-sensing pulsating array geometry By employing a physical arrangement of lower triangular and / or upper triangular physical transformers (PEs), the number of PEs is consistent with the effective computation area, avoiding physical computation in invalid regions. Compared to a square array of the same size, the number of PEs is reduced by approximately 50%. Structurally, PEs are not placed in the upper triangular invalid region, thereby avoiding the physical MAC and transistor flipping in that region.

[0218] 2) Data flow and mapping rules specifically for the triangular effective domain The injection point, propagation direction, beat order, and propagation distance rules are designed and defined for the lower and upper triangles, respectively. Through a fixed mapping relationship between vector identifiers and PE positions, each PE only receives and participates in the multiplication and accumulation of the row vector of its unique corresponding matrix A and the column of matrix B. The resulting matrix elements correspond one-to-one with the PEs, and there is no redundant calculation. This data stream and mapping are specifically designed for the effective field of the triangle, which is different from the rectangular wavefront and sparse scheduling of general square matrices.

[0219] 3) Complete DP-MM is achieved through dual-triangle collaboration. The lower triangular array outputs the lower triangular part of the result matrix, and the upper triangular array outputs the upper triangular part. Logically, the two can be merged into a complete matrix multiplication result. This supports the calculation of the effective region of the lower triangular part of causal attention while also taking into account the calculation needs of the upper triangular block in SWA block division, thereby improving the overall MAC utilization.

[0220] 4) Flexible configuration capability of homogeneous arrays Decoupling of physical array type (upper / lower triangular shape) from computational target (C_L / C_U): Homogeneous triangular arrays (lower or upper triangular arrays) can flexibly switch computation modes through data flow reconfiguration. Three deployment models are supported: Double lower triangle: Two lower triangular arrays can be used to calculate two C_L values ​​simultaneously (batch processing), or one array can calculate C_L and the other can calculate C_U (by configuring the upper triangular data stream). Double upper triangle: Two upper triangular arrays can be used to calculate two C_U values ​​simultaneously, or one array can calculate C_U and the other can calculate C_L (by configuring the lower triangular data stream). A lower triangular array plus an upper triangular array: can collaboratively compute the complete result of the same matrix, or process the upper and lower triangular calculations of different matrices in parallel.

[0221] This allows for adaptation to the computing needs of different application scenarios. For example, pure causal attention can use a double lower triangle to improve throughput, while SWA scenarios can use a hybrid deployment to ensure flexibility.

[0222] This application also provides a matrix multiplication operation processing method, which is applied to... Figure 6 The processor shown specifically includes the following steps: Step 801: Based on the configured matrix operation task information, the control module configures matrix operation parameters for the first data loading module, the second data loading module, and the data writing module respectively; Step 802: The first data loading module reads the row vector of the first matrix from the SRAM based on the matrix operation parameters, and injects the read row vector of the first matrix into the first triangular region PE of the PE array; the second data loading module reads the column vector of the second matrix from the SRAM based on the matrix operation parameters, and injects the read column vector of the second matrix into the first triangular region PE of the PE array. And / or, the first data loading module reads the row vector of the third matrix from the SRAM based on the matrix operation parameters, and injects the read row vector of the third matrix into the second triangular region PE of the PE array; the second data loading module reads the column vector of the fourth matrix from the SRAM based on the matrix operation parameters, and injects the read column vector of the fourth matrix into the second triangular region PE of the PE array. Step 803: The PE array performs matrix multiplication on the first matrix and the second matrix through the first triangular region PE, and / or performs matrix multiplication on the third matrix and the fourth matrix through the second triangular region PE; Step 804: The data writing module writes the matrix multiplication result of the PE array into the SRAM based on the matrix operation parameters.

[0223] Optionally, the matrix operation task information includes matrix information to be multiplied, the matrix information including information of the target intermediate matrix and the valid and / or invalid triangular regions in the target intermediate matrix, the target intermediate matrix being the multiplication matrix of the first target matrix and the second target matrix to be multiplied; Before step 801, the method further includes: The control module determines a target calculation mode for the target intermediate matrix based on the valid and / or invalid triangular regions in the target intermediate matrix. The target calculation mode is used to instruct the valid triangular regions in the target intermediate matrix to perform operations on the target triangular region PE in the PE array. The target triangular region PE is one of the first triangular region PE and the second triangular region PE. The matrix operation parameters include the calculation mode and the matrix information. The target calculation mode is one of the multiple calculation modes supported by the PE array. Wherein, when the size of the target intermediate matrix is ​​equal to the size of the PE array, the first target matrix and the second target matrix are respectively the first matrix and the second matrix, the target intermediate matrix is ​​the first intermediate matrix, and the first intermediate matrix and the second intermediate matrix are the same matrix; Alternatively, if the size of the target intermediate matrix is ​​larger than the size of the PE array, the method further includes: the control module dividing the target intermediate matrix into multiple sub-intermediate matrices according to the size of the PE array, wherein the first intermediate matrix and the second intermediate matrix are any two of the multiple sub-intermediate matrices, the first intermediate matrix and the second intermediate matrix are different matrices, the first matrix and the third matrix are sub-matrices in the first target matrix, and the second matrix and the fourth matrix are sub-matrices in the second target matrix.

[0224] Optionally, step 802 includes: The first data loading module determines a first injection order for the row vectors of the first matrix based on the target calculation mode, and sequentially reads the row vectors of the first matrix from the SRAM according to the first injection order, and injects the row vectors of the first matrix read each time into the first triangular region PE; the second data loading module determines a second injection order for the column vectors of the second matrix based on the target calculation mode, and sequentially reads the column vectors of the second matrix from the SRAM according to the second injection order, and injects the column vectors of the second matrix read each time into the first triangular region PE; And / or, the first data loading module determines a third injection order for the row vectors of the third matrix based on the target calculation mode, and sequentially reads the row vectors of the third matrix from the SRAM based on the third injection order, and injects the row vectors of the third matrix read each time into the second triangular area PE; the second data loading module determines a fourth injection order for the column vectors of the fourth matrix based on the target calculation mode, and sequentially reads the column vectors of the fourth matrix from the SRAM based on the fourth injection order, and injects the column vectors of the fourth matrix read each time into the second triangular area PE; Step 804 includes: The data writing module determines the target output order of the matrix multiplication results of the PE array based on the target calculation mode, outputs the matrix multiplication results according to the target output order, and writes the output matrix multiplication results into the SRAM sequentially.

[0225] Optionally, when the target calculation mode is the first calculation mode, the first injection order is the order in which vectors from the Mth row to the gth row of the first matrix are injected sequentially; the second injection order is the order in which vectors from the 1st column to the eth column of the second matrix are injected sequentially; the third injection order is the order in which vectors from the 1st row to the kth row of the first matrix are injected sequentially; the fourth injection order is the order in which vectors from the Nth column to the sth column of the second matrix are injected sequentially; the target output order is the order in which vectors from the 1st row to the Mth row and from the 1st column to the Mth column of the operation result matrix are output sequentially; and the operation result matrix is ​​the matrix corresponding to the matrix multiplication operation result. When the target calculation mode is the second calculation mode, the first injection order is the order in which vectors from the Mth row to the gth row of the first matrix are injected sequentially; the second injection order is the order in which vectors from the 1st column to the eth column of the second matrix are injected sequentially; the third injection order is the order in which vectors from the 1st row to the kth row of the third matrix are injected sequentially; the fourth injection order is the order in which vectors from the Nth column to the sth column of the fourth matrix are injected sequentially; and the target output order is the order in which vectors from the 1st row to the Mth row and from the 1st column to the Mth column of the result matrix are output sequentially. When the target calculation mode is the third calculation mode, the first injection order is the order in which vectors from the 1st row to the hth row of the first matrix are injected sequentially; the second injection order is the order in which vectors from the Nth column to the fth column of the second matrix are injected sequentially; the third injection order is the order in which vectors from the 1st row to the kth row of the third matrix are injected sequentially; and the fourth injection order is the order in which vectors from the Nth column to the sth column of the fourth matrix are injected sequentially. The target output order is: the first triangular area of ​​the operation result matrix is ​​output sequentially from the hth row to the 1st row and from the Nth column to the fth column; and the second triangular area of ​​the operation result matrix is ​​output sequentially from the 1st row to the kth row and from the sth column to the Nth column. When the target calculation mode is the fourth calculation mode, the first injection order is the order in which vectors from the Mth row to the gth row of the first matrix are injected sequentially; the second injection order is the order in which vectors from the 1st column to the eth column of the second matrix are injected sequentially; the third injection order is the order in which vectors from the rth row to the 1st row of the third matrix are injected sequentially; and the fourth injection order is the order in which vectors from the 1st column to the tth column of the fourth matrix are injected sequentially. The target output order is: the first triangular area of ​​the operation result matrix is ​​output sequentially from the gth row to the Mth row and from the 1st column to the eth column; and the second triangular area of ​​the operation result matrix is ​​output sequentially from the rth row to the 1st row and from the tth column to the 1st column. Where M is less than or equal to N, g is 1, s is 2, h is M, and t is N-1; when M is greater than N, g is 2, s is 1, h is M-1, and t is N. When M is less than N, e is N-1, k is M, f is 2, and r is 1; when M is greater than or equal to N, e is N, k is M-1, f is 1, and r is 2.

[0226] Optionally, the first data loading module is connected to the first PE of the first triangular region PE and the second PE of the second triangular region PE, wherein the first PE is the PE in the g-th row and 1-th column of the first triangular region PE, and the second PE is the PE in the M-th row and e-th column of the first triangular region PE. The second data loading module is connected to the third PE of the lower corner area PE and the fourth PE of the second triangular area PE. The third PE is the PE in the kth row and Nth column of the second triangular area PE, and the fourth PE is the PE in the 1st row and sth column of the second triangular area PE. The data writing module is connected to the first row of PEs in the PE array; Step 802 includes: The first data loading module injects the row vectors of the first matrix read from the first PE into the first triangular region PE; the second data loading module injects the column vectors of the second matrix read from the third PE into the first triangular region PE. And / or, the first data loading module injects the row vector of the read third matrix from the second PE into the second triangular region PE; the second data loading module is used to inject the column vector of the read fourth matrix from the fourth PE into the second triangular region PE.

[0227] Optionally, step 802 includes: The first data loading module injects the row vectors of the first matrix read from the SRAM in the first K-1 consecutive reads into the first PE from row M to row M-K+2 of the first triangular region PE, and injects the row vector of the first matrix read from the SRAM in the first PE into the first PE; K is the number of times the first data loading module reads the row vectors of the first matrix from the SRAM, and K is an integer less than or equal to M; The second data loading module injects the column vectors of the second matrix read from the SRAM in the first L-1 times sequentially into the first column of the first triangular region PE from the first column to the L-1th column of the PE, and injects the column vector of the second matrix read from the SRAM in the Lth time into the third PE. L is the number of times the second data loading module reads the column vector of the second matrix from the SRAM, and L is an integer less than or equal to N. And / or, The first data loading module injects the row vectors of the third matrix read from the SRAM in the first P-1 times from the second PE into the first row to the P-1th row of the second triangular region PE, and injects the row vector of the third matrix read from the SRAM in the Pth time into the second PE. P is the number of times the first data loading module reads the row vectors of the third matrix from the SRAM, and P is an integer less than or equal to M. The second data loading module injects the column vectors of the fourth matrix read from the SRAM in the first Q-1 times from the fourth PE into the Nth to N-Q+2th columns of the second triangular region PE, respectively, and injects the column vector of the fourth matrix read from the SRAM in the Qth time into the fourth PE. Q is the number of times the second data loading module reads the column vectors of the fourth matrix from the SRAM, and Q is an integer less than or equal to N.

[0228] Optionally, the PE array adopts a pipelined design, and the number of pipelines in the PE array is less than the number of cycles required for the PE array to load data. The number of cycles includes the number of injections required to inject the row vectors of the first matrix and the number of injections required to inject the column vectors of the second matrix into the PE array.

[0229] This application embodiment serves as a comparison with... Figure 6 The method implementation corresponding to the processor embodiment shown can be found in the relevant descriptions in the foregoing embodiments, and can achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0230] The embodiments of this application have the following beneficial effects: 1) Eliminating Physical Computation and Power Waste in Ineffective Regions: This application employs a triangle-aware pulsating array geometry, physically arranging the physical pairs (PEs) in a lower or upper triangular shape, ensuring the number of PEs matches the effective computation area. Compared to a square array of the same size, this application structurally avoids arranging PEs in the upper triangular ineffective region, preventing the execution of physical MAC operations and transistor switching in that region, thereby eliminating computational and power waste in the upper triangular ineffective region of causal attention at the hardware level.

[0231] 2) Improved utilization of diagonal edge blocks: This application employs dedicated data flow and mapping rules for the effective triangular domain, designing specific injection points, propagation directions, and beat sequences for the lower and upper triangles. Through a fixed mapping relationship between vector identifiers and PE positions, each PE receives and participates only in the calculation of its unique corresponding matrix element, resulting in a one-to-one correspondence between the matrix elements and PEs with no redundant calculations. When processing diagonal edge blocks, the MAC utilization of the triangular array approaches 100%, significantly outperforming the approximately 50% utilization of the rectangular array.

[0232] 3) Eliminating pipeline bubbles and improving array energy efficiency: The triangular PE arrangement and DP-MM data flow in this application are specifically designed for the triangular effective domain, and the data flow advancement method is consistent with the geometry of the triangular computation domain. During lower triangular computation, row vector A is injected from the upper left corner and passed to the lower right corner, while column vector B is injected from the lower right corner and passed to the upper left corner; during upper triangular computation, the data injection point and transmission direction are adjusted accordingly. This data flow, which matches the geometry of the effective domain, eliminates pipeline bubbles generated when rectangular data flows cover the triangular computation domain, thereby improving the overall array utilization and energy efficiency.

[0233] 4) Eliminating Invalid Transistor Toggling and Routing Overhead at the Source: This application fundamentally eliminates the physical hardware presence of invalid regions by redesigning the array topology at the physical level and directly adopting a triangular mesh instead of a rectangular mesh. Unlike schemes that disable some PEs on a square array through indexing or masking, this invention does not require retaining the physical hardware of invalid regions, avoiding invalid transistor toggling and routing overhead, and achieving the elimination of invalid computation at the source.

[0234] 5) Reduced scheduling overhead and flexible adaptation to different computing needs: This application decouples the physical array type from the computing target through the flexible configuration capability of homogeneous arrays. The homogeneous triangular array can flexibly switch computing modes through data flow reconfiguration, supporting various deployment forms such as double lower triangle, double upper triangle, and lower-upper-lower triangle. When the sequence length and block size change, there is no need for complex mask or index scheduling; different computing needs can be adapted simply by changing the vector injection order, reducing scheduling overhead while maintaining high utilization.

[0235] This application also provides an electronic device, including a central processing unit (CPU), a memory, and the processor provided in the foregoing embodiments, wherein the control module in the processor is connected to the CPU via a configuration bus, and the SRAM in the processor is connected to the memory via a data bus.

[0236] This application embodiment serves as a comparison with... Figure 6 The specific implementation of the electronic device side embodiment corresponding to the processor embodiment shown can be found in the relevant descriptions in the foregoing embodiments. To avoid repetition, it will not be repeated here.

[0237] The electronic device in this application embodiment can be a terminal, or it can be any other device besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, handheld computer, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not make specific limitations.

[0238] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0239] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0240] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0241] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A PE array, characterized in that, The system includes an M-row N-column processing unit PE. The M-row N-column PE is divided into a first triangular region PE and a second triangular region PE according to the target diagonal. The target diagonal is composed of PEs with the same row and column numbers in the M-row N-column PE. M and N are both integers greater than 1. When M is less than or equal to N, the first triangular region PE includes the target diagonal. When M is greater than N, the first triangular region PE does not include the target diagonal. The PE array in the i-th row and j-th column is connected to the PE array in the (i+1)-th row and j+1-th column. The PE array in the M-th row of the first triangular region is connected to the adjacent columns of PE. The PE array in the 1-th row of the second triangular region is connected to the adjacent columns of PE. The PE array in the column is connected to the adjacent rows of PE. i is an integer between 1 and M-1, and j is an integer between 1 and N-1. The first triangular region PE is used to perform matrix multiplication operations on the effective first triangular region or effective second triangular region of the first intermediate matrix, and / or the second triangular region PE is used to perform matrix multiplication operations on the effective second triangular region or effective first triangular region of the second intermediate matrix; wherein, the first intermediate matrix and the second intermediate matrix are the same or different, the first intermediate matrix is ​​the multiplication operation matrix of the first matrix and the second matrix, the second intermediate matrix is ​​the multiplication operation matrix of the third matrix and the fourth matrix, and the dimensions of the first intermediate matrix and the second intermediate matrix are adapted to the PE array.

2. The PE array according to claim 1, characterized in that, The PE array includes multiple computing modes, among which, When the PE array uses the first calculation mode, the first triangular region PE is used to perform matrix multiplication on the effective first triangular region of the first intermediate matrix, and the second triangular region PE is used to perform matrix multiplication on the effective second triangular region of the second intermediate matrix. The second intermediate matrix is ​​the same as the first intermediate matrix. Alternatively, when the PE array uses the second calculation mode, the first triangular region PE is used to perform matrix multiplication on the effective first triangular region of the first intermediate matrix, and the second triangular region PE is used to perform matrix multiplication on the effective second triangular region of the second intermediate matrix, wherein the first intermediate matrix is ​​different from the second intermediate matrix. Alternatively, when the PE array uses a third calculation mode, the first triangular region PE is used to perform matrix multiplication on the effective second triangular region of the first intermediate matrix, and the second triangular region PE is used to perform matrix multiplication on the effective second triangular region of the second intermediate matrix, wherein the first intermediate matrix is ​​different from the second intermediate matrix. Alternatively, when the PE array uses the fourth calculation mode, the first triangular region PE is used to perform matrix multiplication on the effective first triangular region of the first intermediate matrix, and the second triangular region PE is used to perform matrix multiplication on the effective first triangular region of the second intermediate matrix, wherein the first intermediate matrix is ​​different from the second intermediate matrix.

3. The PE array according to claim 1, characterized in that, The first PE in the first triangular region PE is used to receive the first target row vector of the first matrix and to receive the second target row vector of the first matrix, and to pass the second target row vector to the corresponding row PE. The first PE is the PE in the g-th row and 1-th column of the first triangular region PE. The first target row vector is the g-th row vector of the first matrix, and the second target row vector includes the (g+1)-th row vector of the first matrix to the M-th row vector. When M is less than or equal to N, g is 1, and when M is greater than N, g is 2. Alternatively, the first target row vector is the h-th row vector of the first matrix, and the second target row vector includes the 1-th row vector of the first matrix to the (h-1)-th row vector. When M is less than or equal to N, h is M, and when M is greater than N, h is M-1. The second PE in the first triangular region PE is used to receive the third target column vector of the second matrix and to receive the fourth target column vector of the second matrix, and to pass the fourth target column vector to the corresponding column PE. The second PE is the PE in the M-th row and e-th column of the first triangular region PE. The third target column vector is the e-th column vector of the second matrix, and the fourth target column vector includes the 1st to e-1st column vectors of the second matrix. When M is less than N, e is N-1, and when M is greater than or equal to N, e is N. Alternatively, the third target column vector is the f-th column vector of the first matrix, and the fourth target column vector includes the f+1st to Nth column vectors of the second matrix. When M is less than N, f is 2, and when M is greater than or equal to N, f is 1. And / or, The third PE in the second triangular region PE is used to receive the fifth target row vector of the third matrix and to receive the sixth target row vector of the third matrix, and to pass the sixth target row vector to the corresponding row PE. The third PE is the k-th row and N-th column PE in the second triangular region PE. The fifth target row vector is the k-th row vector of the third matrix, and the sixth target row vector includes the 1st to k-1st row vectors of the third matrix. When M is greater than or equal to N, k is M-1, and when M is less than N, k is M. Alternatively, the fifth target row vector is the r-th row vector of the third matrix, and the sixth target row vector includes the r+1th to M-th row vectors of the third matrix. When M is less than N, r is 1, and when M is greater than or equal to N, r is 2. The fourth PE of the second triangular region PE is used to receive the seventh target column vector of the fourth matrix and to receive the eighth target column vector of the fourth matrix, and to pass the eighth target column vector to the corresponding column PE. The fourth PE is the PE in the first row and s column of the second triangular region PE. The seventh target column vector is the s column vector of the fourth matrix, and the eighth target column vector includes the (s+1)th to the Nth column vector of the fourth matrix. When M is greater than N, s is 1, and when M is less than or equal to N, s is 2. Alternatively, the seventh target column vector is the t column vector of the fourth matrix, and the eighth target column vector includes the first to the (t-1)th column vector of the fourth matrix. When M is less than or equal to N, t is N-1, and when M is greater than N, t is N.

4. A processor, characterized in that, The PE array includes any one of claims 1 to 3, and further includes a control module, a first data loading module, a second data loading module, a data writing module, and a static random access memory (SRAM). The control module is connected to the first data loading module, the second data loading module, and the write data module, respectively. The first data loading module, the second data loading module, and the write data module are all connected to the PE array and the SRAM. The control module is used to configure matrix operation parameters for the first data loading module, the second data loading module, and the data writing module respectively based on the configured matrix operation task information; The first data loading module is configured to read the row vector of the first matrix from the SRAM based on the matrix operation parameters, and inject the read row vector of the first matrix into the first triangular region PE of the PE array; the second data loading module is configured to read the column vector of the second matrix from the SRAM based on the matrix operation parameters, and inject the read column vector of the second matrix into the first triangular region PE of the PE array; and / or, the first data loading module is configured to read the row vector of the third matrix from the SRAM based on the matrix operation parameters, and inject the read row vector of the third matrix into the second triangular region PE of the PE array; the second data loading module is configured to read the column vector of the fourth matrix from the SRAM based on the matrix operation parameters, and inject the read column vector of the fourth matrix into the second triangular region PE of the PE array; The PE array is used to perform matrix multiplication operations on the first matrix and the second matrix through the first triangular region PE, and / or to perform matrix multiplication operations on the third matrix and the fourth matrix through the second triangular region PE; The data writing module is used to write the matrix multiplication result of the PE array into the SRAM based on the matrix operation parameters.

5. The processor according to claim 4, characterized in that, The matrix operation task information includes matrix information to be multiplied, the matrix information including information of the target intermediate matrix and the valid and / or invalid triangular regions in the target intermediate matrix, the target intermediate matrix being the multiplication matrix of the first target matrix and the second target matrix to be multiplied; The control module is used to determine a target calculation mode for the target intermediate matrix based on the valid and / or invalid triangular regions in the target intermediate matrix. The target calculation mode is used to instruct the valid triangular regions in the target intermediate matrix to perform operations on the target triangular region PE in the PE array. The target triangular region PE is one of the first triangular region PE and the second triangular region PE. The matrix operation parameters include the calculation mode and the matrix information. The target calculation mode is one of the multiple calculation modes supported by the PE array. Wherein, when the size of the target intermediate matrix is ​​equal to the size of the PE array, the first target matrix and the second target matrix are respectively the first matrix and the second matrix, the target intermediate matrix is ​​the first intermediate matrix, and the first intermediate matrix and the second intermediate matrix are the same matrix; Alternatively, if the size of the target intermediate matrix is ​​larger than the size of the PE array, the control module is further configured to divide the target intermediate matrix into multiple sub-intermediate matrices according to the size of the PE array, wherein the first intermediate matrix and the second intermediate matrix are any two of the multiple sub-intermediate matrices, the first intermediate matrix and the second intermediate matrix are different matrices, the first matrix and the third matrix are sub-matrices in the first target matrix, and the second matrix and the fourth matrix are sub-matrices in the second target matrix.

6. The processor according to claim 5, characterized in that, The first data loading module is used to determine a first injection order for the row vectors of the first matrix based on the target calculation mode, and sequentially read the row vectors of the first matrix from the SRAM according to the first injection order, and inject the row vectors of the first matrix read each time into the first triangular area PE; the second data loading module is used to determine a second injection order for the column vectors of the second matrix based on the target calculation mode, and sequentially read the column vectors of the second matrix from the SRAM according to the second injection order, and inject the column vectors of the second matrix read each time into the first triangular area PE; And / or, the first data loading module is configured to determine a third injection order for the row vectors of the third matrix based on the target calculation mode, and based on the third injection order, sequentially read the row vectors of the third matrix from the SRAM, and inject the row vectors of the third matrix read each time into the second triangular area PE; the second data loading module is configured to determine a fourth injection order for the column vectors of the fourth matrix based on the target calculation mode, and based on the fourth injection order, sequentially read the column vectors of the fourth matrix from the SRAM, and inject the column vectors of the fourth matrix read each time into the second triangular area PE; The data writing module is used to determine the target output order of the matrix multiplication results of the PE array based on the target calculation mode, output the matrix multiplication results according to the target output order, and write the output matrix multiplication results into the SRAM in sequence.

7. The processor according to claim 6, characterized in that, When the target calculation mode is the first calculation mode, the first injection order is the order in which vectors from the Mth row to the gth row of the first matrix are injected sequentially; the second injection order is the order in which vectors from the 1st column to the eth column of the second matrix are injected sequentially; the third injection order is the order in which vectors from the 1st row to the kth row of the first matrix are injected sequentially; the fourth injection order is the order in which vectors from the Nth column to the sth column of the second matrix are injected sequentially; the target output order is the order in which vectors from the 1st row to the Mth row and from the 1st column to the Mth column of the operation result matrix are output sequentially; and the operation result matrix is ​​the matrix corresponding to the matrix multiplication operation result. When the target calculation mode is the second calculation mode, the first injection order is the order in which vectors from the Mth row to the gth row of the first matrix are injected sequentially; the second injection order is the order in which vectors from the 1st column to the eth column of the second matrix are injected sequentially; the third injection order is the order in which vectors from the 1st row to the kth row of the third matrix are injected sequentially; the fourth injection order is the order in which vectors from the Nth column to the sth column of the fourth matrix are injected sequentially; and the target output order is the order in which vectors from the 1st row to the Mth row and from the 1st column to the Mth column of the result matrix are output sequentially. When the target calculation mode is the third calculation mode, the first injection order is the order in which vectors from the 1st row to the hth row of the first matrix are injected sequentially; the second injection order is the order in which vectors from the Nth column to the fth column of the second matrix are injected sequentially; the third injection order is the order in which vectors from the 1st row to the kth row of the third matrix are injected sequentially; and the fourth injection order is the order in which vectors from the Nth column to the sth column of the fourth matrix are injected sequentially. The target output order is: the first triangular area of ​​the operation result matrix is ​​output sequentially from the hth row to the 1st row and from the Nth column to the fth column; and the second triangular area of ​​the operation result matrix is ​​output sequentially from the 1st row to the kth row and from the sth column to the Nth column. When the target calculation mode is the fourth calculation mode, the first injection order is the order in which vectors from the Mth row to the gth row of the first matrix are injected sequentially; the second injection order is the order in which vectors from the 1st column to the eth column of the second matrix are injected sequentially; the third injection order is the order in which vectors from the rth row to the 1st row of the third matrix are injected sequentially; and the fourth injection order is the order in which vectors from the 1st column to the tth column of the fourth matrix are injected sequentially. The target output order is: the first triangular area of ​​the operation result matrix is ​​output sequentially from the gth row to the Mth row and from the 1st column to the eth column; and the second triangular area of ​​the operation result matrix is ​​output sequentially from the rth row to the 1st row and from the tth column to the 1st column. Where M is less than or equal to N, g is 1, s is 2, h is M, and t is N-1; when M is greater than N, g is 2, s is 1, h is M-1, and t is N. When M is less than N, e is N-1, k is M, f is 2, and r is 1; when M is greater than or equal to N, e is N, k is M-1, f is 1, and r is 2.

8. The processor according to any one of claims 4 to 7, characterized in that, The first data loading module is connected to the first PE of the first triangular region PE and the second PE of the second triangular region PE. The first PE is the PE in the g-th row and 1-th column of the first triangular region PE, and the second PE is the PE in the M-th row and e-th column of the first triangular region PE. The second data loading module is connected to the third PE of the lower corner area PE and the fourth PE of the second triangular area PE. The third PE is the PE in the kth row and Nth column of the second triangular area PE, and the fourth PE is the PE in the 1st row and sth column of the second triangular area PE. The data writing module is connected to the first row of PEs in the PE array; The first data loading module is used to inject the row vector of the read first matrix from the first PE into the first triangular region PE, and the second data loading module is used to inject the column vector of the read second matrix from the third PE into the first triangular region PE; and / or, the first data loading module is used to inject the row vector of the read third matrix from the second PE into the second triangular region PE, and the second data loading module is used to inject the column vector of the read fourth matrix from the fourth PE into the second triangular region PE.

9. The processor according to claim 8, characterized in that, The first data loading module is used to inject the row vectors of the first matrix read from the SRAM K times sequentially from the first PE into rows M to M-K+1 of the first triangular region PE, where K is the number of times the first data loading module reads the row vectors of the first matrix from the SRAM, and K is a positive integer less than or equal to M. The second data loading module is used to inject the column vectors of the second matrix, which are read from the SRAM L times in sequence, from the third PE into the first column to the Lth column of the first triangular region PE, respectively. L is the number of times the second data loading module reads the column vectors of the second matrix from the SRAM, and L is a positive integer less than or equal to N. And / or, The first data loading module is used to inject the row vectors of the third matrix read from the SRAM P times sequentially from the second PE into the first row to the Pth row of the second triangular area PE, where P is the number of times the first data loading module reads the row vectors of the third matrix from the SRAM, and P is a positive integer less than or equal to M. The second data loading module is used to inject the column vectors of the fourth matrix, which are read from the SRAM Q times in sequence, from the fourth PE into the Nth to the (N-Q+1)th columns of the second triangular region PE, respectively. Q is the number of times the second data loading module reads the column vectors of the fourth matrix from the SRAM, and Q is a positive integer less than or equal to N.

10. The processor according to any one of claims 4 to 9, characterized in that, The PE array adopts a pipelined design, and the number of pipelines in the PE array is less than the number of cycles required for the PE array to load data. The number of cycles includes the number of injections required to inject the row vectors of the first matrix and the number of injections required to inject the column vectors of the second matrix into the PE array.

11. An electronic device, characterized in that, It includes a central processing unit (CPU), a memory, and the processor as described in claims 4 to 10, wherein the control module in the processor is connected to the CPU via a configuration bus, and the SRAM in the processor is connected to the memory via a data bus.

12. A matrix multiplication operation processing method, characterized in that, Applied to the processor of any one of claims 4 to 10, the method comprises: The control module in the processor configures matrix operation parameters for the processor's first data loading module, second data loading module, and write data module based on the configured matrix operation task information. The first data loading module reads the row vector of a first matrix from the SRAM of the processor based on the matrix operation parameters, and injects the read row vector of the first matrix into the first triangular region PE of the PE array; the second data loading module reads the column vector of a second matrix from the SRAM based on the matrix operation parameters, and injects the read column vector of the second matrix into the first triangular region PE of the PE array; and / or, the first data loading module reads the row vector of a third matrix from the SRAM based on the matrix operation parameters, and injects the read row vector of the third matrix into the second triangular region PE of the PE array; the second data loading module reads the column vector of a fourth matrix from the SRAM based on the matrix operation parameters, and injects the read column vector of the fourth matrix into the second triangular region PE of the PE array; The PE array performs matrix multiplication on the first matrix and the second matrix through the first triangular region PE, and / or performs matrix multiplication on the third matrix and the fourth matrix through the second triangular region PE; The data writing module writes the matrix multiplication result of the PE array into the SRAM based on the matrix operation parameters.

13. The method according to claim 12, characterized in that, The matrix operation task information includes matrix information to be multiplied, the matrix information including information of the target intermediate matrix and the valid and / or invalid triangular regions in the target intermediate matrix, the target intermediate matrix being the multiplication matrix of the first target matrix and the second target matrix to be multiplied; Before the control module configures matrix operation parameters for the first data loading module, the second data loading module, and the data writing module based on the configured matrix operation task information, the method further includes: The control module determines a target calculation mode for the target intermediate matrix based on the valid and / or invalid triangular regions in the target intermediate matrix. The target calculation mode is used to instruct the valid triangular regions in the target intermediate matrix to perform operations on the target triangular region PE in the PE array. The target triangular region PE is one of the first triangular region PE and the second triangular region PE. The matrix operation parameters include the calculation mode and the matrix information. The target calculation mode is one of the multiple calculation modes supported by the PE array. Wherein, when the size of the target intermediate matrix is ​​equal to the size of the PE array, the first target matrix and the second target matrix are respectively the first matrix and the second matrix, the target intermediate matrix is ​​the first intermediate matrix, and the first intermediate matrix and the second intermediate matrix are the same matrix; Alternatively, if the size of the target intermediate matrix is ​​larger than the size of the PE array, the method further includes: the control module dividing the target intermediate matrix into multiple sub-intermediate matrices according to the size of the PE array, wherein the first intermediate matrix and the second intermediate matrix are any two of the multiple sub-intermediate matrices, the first intermediate matrix and the second intermediate matrix are different matrices, the first matrix and the third matrix are sub-matrices in the first target matrix, and the second matrix and the fourth matrix are sub-matrices in the second target matrix.

14. The method according to claim 13, characterized in that, The first data loading module reads the row vectors of the first matrix from the SRAM based on the matrix operation parameters, and injects the read row vectors of the first matrix into the first triangular region PE of the PE array, including: The first data loading module determines the first injection order of the row vectors of the first matrix based on the target calculation mode, and reads the row vectors of the first matrix sequentially from the SRAM according to the first injection order, and injects the row vectors of the first matrix read each time into the first triangular region PE. The second data loading module, based on the matrix operation parameters, reads the column vectors of the second matrix from the SRAM and injects the read column vectors of the second matrix into the first triangular region PE of the PE array, including: The second data loading module determines the second injection order of the column vectors of the second matrix based on the target calculation mode, and reads the column vectors of the second matrix from the SRAM in sequence according to the second injection order, and injects the column vectors of the second matrix read each time into the first triangular region PE; And / or, The first data loading module reads the row vectors of the third matrix from the SRAM based on the matrix operation parameters, and injects the read row vectors of the third matrix into the second triangular region PE of the PE array, including: The first data loading module determines the third injection order of the row vectors of the third matrix based on the target calculation mode, and reads the row vectors of the third matrix sequentially from the SRAM based on the third injection order, and injects the row vectors of the third matrix read each time into the second triangular region PE; The second data loading module, based on the matrix operation parameters, reads the column vector of the fourth matrix from the SRAM and injects the read column vector of the fourth matrix into the second triangular region PE of the PE array, including: The second data loading module determines the fourth injection order of the column vectors of the fourth matrix based on the target calculation mode, and reads the column vectors of the fourth matrix sequentially from the SRAM based on the fourth injection order, and injects the column vectors of the fourth matrix read each time into the second triangular region PE; The data writing module writes the matrix multiplication result of the PE array into the SRAM based on the matrix operation parameters, including: The data writing module determines the target output order of the matrix multiplication results of the PE array based on the target calculation mode, outputs the matrix multiplication results according to the target output order, and writes the output matrix multiplication results into the SRAM sequentially.

15. The method according to claim 14, characterized in that, When the target calculation mode is the first calculation mode, the first injection order is the order in which vectors from the Mth row to the gth row of the first matrix are injected sequentially; the second injection order is the order in which vectors from the 1st column to the eth column of the second matrix are injected sequentially; the third injection order is the order in which vectors from the 1st row to the kth row of the first matrix are injected sequentially; the fourth injection order is the order in which vectors from the Nth column to the sth column of the second matrix are injected sequentially; the target output order is the order in which vectors from the 1st row to the Mth row and from the 1st column to the Mth column of the operation result matrix are output sequentially; and the operation result matrix is ​​the matrix corresponding to the matrix multiplication operation result. When the target calculation mode is the second calculation mode, the first injection order is the order in which vectors from the Mth row to the gth row of the first matrix are injected sequentially; the second injection order is the order in which vectors from the 1st column to the eth column of the second matrix are injected sequentially; the third injection order is the order in which vectors from the 1st row to the kth row of the third matrix are injected sequentially; the fourth injection order is the order in which vectors from the Nth column to the sth column of the fourth matrix are injected sequentially; and the target output order is the order in which vectors from the 1st row to the Mth row and from the 1st column to the Mth column of the result matrix are output sequentially. When the target calculation mode is the third calculation mode, the first injection order is the order in which vectors from the 1st row to the hth row of the first matrix are injected sequentially; the second injection order is the order in which vectors from the Nth column to the fth column of the second matrix are injected sequentially; the third injection order is the order in which vectors from the 1st row to the kth row of the third matrix are injected sequentially; and the fourth injection order is the order in which vectors from the Nth column to the sth column of the fourth matrix are injected sequentially. The target output order is: the first triangular area of ​​the operation result matrix is ​​output sequentially from the hth row to the 1st row and from the Nth column to the fth column; and the second triangular area of ​​the operation result matrix is ​​output sequentially from the 1st row to the kth row and from the sth column to the Nth column. When the target calculation mode is the fourth calculation mode, the first injection order is the order in which vectors from the Mth row to the gth row of the first matrix are injected sequentially; the second injection order is the order in which vectors from the 1st column to the eth column of the second matrix are injected sequentially; the third injection order is the order in which vectors from the rth row to the 1st row of the third matrix are injected sequentially; and the fourth injection order is the order in which vectors from the 1st column to the tth column of the fourth matrix are injected sequentially. The target output order is: the first triangular area of ​​the operation result matrix is ​​output sequentially from the gth row to the Mth row and from the 1st column to the eth column; and the second triangular area of ​​the operation result matrix is ​​output sequentially from the rth row to the 1st row and from the tth column to the 1st column. Where M is less than or equal to N, g is 1, s is 2, h is M, and t is N-1; when M is greater than N, g is 2, s is 1, h is M-1, and t is N. When M is less than N, e is N-1, k is M, f is 2, and r is 1; when M is greater than or equal to N, e is N, k is M-1, f is 1, and r is 2.

16. The method according to any one of claims 12 to 15, characterized in that, The first data loading module is connected to the first PE of the first triangular region PE and the second PE of the second triangular region PE. The first PE is the PE in the g-th row and 1-th column of the first triangular region PE, and the second PE is the PE in the M-th row and e-th column of the first triangular region PE. The second data loading module is connected to the third PE of the lower corner area PE and the fourth PE of the second triangular area PE. The third PE is the PE in the kth row and Nth column of the second triangular area PE, and the fourth PE is the PE in the 1st row and sth column of the second triangular area PE. The data writing module is connected to the first row of PEs in the PE array; The first data loading module injects the row vectors of the first matrix it reads into the first triangular region PE of the PE array, including: the first data loading module injects the row vectors of the first matrix it reads from the first PE into the first triangular region PE; The second data loading module injects the column vectors of the second matrix it reads into the first triangular region PE of the PE array, including: the second data loading module injects the column vectors of the second matrix it reads from the third PE into the first triangular region PE; And / or, The first data loading module injects the row vectors of the first matrix it reads into the first triangular region PE of the PE array, including: the first data loading module injects the row vectors of the third matrix it reads from the second PE into the second triangular region PE; The second data loading module injects the column vectors of the second matrix into the first triangular region PE of the PE array, including: the second data loading module is used to inject the column vectors of the fourth matrix from the fourth PE into the second triangular region PE.

17. The method according to claim 16, characterized in that, The first data loading module reads the row vectors of the first matrix from the SRAM based on the matrix operation parameters, and injects the read row vectors of the first matrix into the first triangular region PE of the PE array, including: The first data loading module injects the row vectors of the first matrix read from the SRAM in the first K-1 consecutive reads into the first PE from row M to row M-K+2 of the first triangular region PE, and injects the row vector of the first matrix read from the SRAM in the first PE into the first PE; K is the number of times the first data loading module reads the row vectors of the first matrix from the SRAM, and K is an integer less than or equal to M; The second data loading module, based on the matrix operation parameters, reads the column vectors of the second matrix from the SRAM and injects the read column vectors of the second matrix into the first triangular region PE of the PE array, including: The second data loading module injects the column vectors of the second matrix read from the SRAM in the first L-1 times sequentially into the first column of the first triangular region PE from the first column to the L-1th column of the PE, and injects the column vector of the second matrix read from the SRAM in the Lth time into the third PE. L is the number of times the second data loading module reads the column vector of the second matrix from the SRAM, and L is an integer less than or equal to N. And / or, The first data loading module reads the row vectors of the third matrix from the SRAM based on the matrix operation parameters, and injects the read row vectors of the third matrix into the second triangular region PE of the PE array, including: The first data loading module injects the row vectors of the third matrix read from the SRAM in the first P-1 times from the second PE into the first row to the P-1th row of the second triangular region PE, and injects the row vector of the third matrix read from the SRAM in the Pth time into the second PE. P is the number of times the first data loading module reads the row vectors of the third matrix from the SRAM, and P is an integer less than or equal to M. The second data loading module, based on the matrix operation parameters, reads the column vector of the fourth matrix from the SRAM and injects the read column vector of the fourth matrix into the second triangular region PE of the PE array, including: The second data loading module injects the column vectors of the fourth matrix read from the SRAM in the first Q-1 times from the fourth PE into the Nth to N-Q+2th columns of the second triangular region PE, respectively, and injects the column vector of the fourth matrix read from the SRAM in the Qth time into the fourth PE. Q is the number of times the second data loading module reads the column vectors of the fourth matrix from the SRAM, and Q is an integer less than or equal to N.

18. The method according to any one of claims 12 to 17, characterized in that, The PE array adopts a pipelined design, and the number of pipelines in the PE array is less than the number of cycles required for the PE array to load data. The number of cycles includes the number of injections required to inject the row vectors of the first matrix and the number of injections required to inject the column vectors of the second matrix into the PE array.

19. A chip comprising a processor according to any one of claims 4 to 10 and a communication interface, the communication interface being coupled to the processor, the processor being configured to run a program or instructions to implement the method according to any one of claims 12 to 18.

Images