A method and system for evaluating the performance of a systolic array accelerator

Abstract

The present invention discloses a method and system for evaluating the performance of a systolic array accelerator. The present invention includes an input workload, data stream, and architecture parameters of the systolic array. The architecture parameters include the scale of the systolic array, the scale of on-chip storage, and the transmission bandwidth. According to the types of the workload and data stream, corresponding relationship-centered expression symbols are determined. A mapping space composed of mapping schemes that meet the hardware resource constraints is constructed. Each point in the mapping space represents a mapping scheme that meets the hardware resource constraints. A performance evaluation model for evaluating the performance metrics of each mapping scheme is established, and the mapping scheme with the optimal performance metrics is searched in the mapping space. The present invention can improve the expression ability of performance evaluation and the accuracy of simulation results, and can more quickly evaluate the performance of various mappings and find the best mapping scheme than a cycle-accurate simulator.

Description

Technical Field

[0001] The present invention belongs to the field of performance detection of systolic arrays, and particularly relates to a method and system for evaluating the performance of a systolic array accelerator. Background Art

[0002] Currently, a general method for constructing an efficient DNN (Deep Neural Network) accelerator is to use a regular array of multiply-accumulate (MAC) units to achieve efficient computing by leveraging data reuse within the array. Due to its unique structural characteristics, the systolic array has become one of the most popular mainstream DNN accelerator architectures. Figure 1 Shows the basic architecture of a systolic array accelerator. The accelerator mainly consists of a systolic array and several on-chip buffers. Each PE in the systolic array has two registers to store different data. The A, B, and C buffers employ double-buffer technology. In addition, for floating-point data where the multiply-add operation cannot be computed in a single cycle, the pipeline for filling and emptying the PE array requires a longer time to ensure the correctness of the computation results. Given that the architecture is basically predetermined, different data streams and mappings will continue to have a significant impact on hardware performance. These data streams define how operations are partitioned and scheduled for computation, as well as the way data is staged in the memory hierarchy. However, even when the workload, architecture, and data stream are determined, different mapping schemes can still have a significant impact on computational efficiency. In this paper, (m, k, n) is used to represent the mapping scheme. Figure 2 Gives the mapping scheme for general matrix multiplication. The sizes of the two matrices involved in the computation are M*K and K*N, while the size of the systolic array is x*x. The mapping scheme first divides the matrices according to the size of the on-chip buffers to obtain several blocks. This corresponds to the process of dividing matrix A into small matrices of m*k scale. For each block, the mapping scheme performs a second cut based on the scale of the systolic array. In the figure, matrix A1 and matrix B1 are the blocks obtained through the second cut. As the most basic blocks, they are sequentially sent to the systolic array to participate in the computation. Due to the differences in data reuse and hardware utilization generated by the data stream, the data stream has been proven to have a significant impact on performance and energy efficiency. The current mainstream method for finding the most efficient architecture and data stream is to build simulators to evaluate different architectures.

[0003] The current mainstream simulators mainly use three methods: The MAESTRO method adopts a data - centric modeling method, which is very convenient for calculating data reuse, but its expression ability is relatively poor and it cannot describe the systolic array structure. The SCALE - SIM method adopts a computation - centric method for modeling; although this method provides great flexibility in describing how to execute computations in an imperative programming manner, it makes it difficult to accurately analyze data reuse. The TENET method adopts a relationship - centric modeling method with strong expression ability. However, when simulating the performance of the systolic array, it still has some drawbacks. All in all,. Existing accelerator simulators emphasize the exploration of specific applications in the hardware design space. However, these simulators cannot simulate the results of different application mappings under a specific architecture, which is also extremely important for performance verification and software development. Traditional cycle - accurate simulators can achieve this, but they are difficult to develop and have a long simulation time.

[0004] For ease of expression, we use pe_latency to represent the number of cycles required for a processing element to complete a multiply - accumulate operation. First, they only model the scenario where pe_latency is 1 and do not support simulation in other cases (such as floating - point calculations). Since pipelining introduces performance optimization, this is not a simple task of multiplying by a factor. Second, these methods do not support the performance simulation of different mappings. The hardware will adopt different scheduling operations and data partitioning during the calculation process, which we call different mappings. Mapping the workload to a real systolic array requires block - level and block - level partitioning. These models simplify this process, so when evaluating the architecture or data flow, they can only obtain an approximately optimal solution in an ideal situation. Traditional cycle - accurate simulators can achieve this goal, but they require a long development time and are not flexible. In addition, these simulators need to simulate the calculation process of different data flows, which requires a large amount of code work. Summary of the Invention

[0005] The technical problem to be solved by the present invention: Aiming at the above - mentioned problems of the prior art, the present invention provides a method and system for evaluating the performance of a systolic array accelerator. The present invention can improve the expression ability of performance evaluation and the accuracy of simulation results, and can evaluate the performance of various mappings faster than cycle - accurate simulators and find the best mapping scheme.

[0006] To solve the above - mentioned technical problems, the technical solution adopted by the present invention is:

[0007] A method for evaluating the performance of a systolic array accelerator, comprising:

[0008] S1, input the workload, data flow, and architecture parameters of the systolic array, where the architecture parameters of the systolic array include the scale of the systolic array, the scale of on - chip storage, and the transmission bandwidth;

[0009] S2. Determine the corresponding relationship - centered expression symbols according to the type of workload and data flow;

[0010] S3. According to the scale of the workload and the scale of on - chip storage, construct a mapping space composed of mapping schemes that meet the hardware resource constraints for the relationship - centered expression symbols. Each point in the mapping space represents a mapping scheme that meets the hardware resource constraints;

[0011] S4. Establish a performance evaluation model for evaluating the performance indicators of each mapping scheme, and search for the mapping scheme with the optimal performance indicators in the mapping space according to the performance evaluation model.

[0012] Optionally, the type of the workload input in step S1 is general matrix multiplication GEMM or convolution operation CONV, and the type of the data flow input in step S1 is input - stationary IS, weight - stationary WS, or output - stationary OS.

[0013] Optionally, the relationship - centered expression symbols determined in step S2 include:

[0014] When the type of the workload is general matrix multiplication GEMM and the type of the data flow is input - stationary IS, the relationship - centered expression symbols are:

[0015] {S[i,j,k]→PE[k%8,j%8]}

[0016] {S[i,j,k]→T[fl(k / 8),fl(j / 8),i + j%8 + k%8*pe_latency]}

[0017] When the type of the workload is general matrix multiplication GEMM and the type of the data flow is weight - stationary WS, the relationship - centered expression symbols are:

[0018] {S[i,j,k]→PE[i%8,k%8]}

[0019] {S[i,j,k]→T[fl(i / 8),fl(k / 8),j + i%8*pe_latency + k%8]}

[0020] When the type of the workload is general matrix multiplication GEMM and the type of the data flow is output - stationary OS, the relationship - centered expression symbols are:

[0021] {S[i,j,k]→PE[i%8,j%8]}

[0022] {S[i,j,k] → T[fl(i / 8),fl(j / 8),i % 8 * pe_latency + j % 8 + k]}

[0023] When the type of the workload is the convolution operation CONV and the type of the data stream is input stationary IS, the relationship-centered expression symbol is:

[0024] {S[k1,c,ox,oy,rx,ry] → PE[(c * rx * ry) % 8,(ox * oy) % 8]}

[0025] {S[k1,c,ox,oy,rx,ry] →

[0026] T[fl((c * rx * ry) / 8),fl((ox * oy) / 8),k1 + (ox * oy) % 8 + (c * rx * ry) % 8 * pe_latency]}

[0027] When the type of the workload is the convolution operation CONV and the type of the data stream is weight stationary WS, the relationship-centered expression symbol is:

[0028] {S[k1,c,ox,oy,rx,ry] → PE[k1 % 8,(c * rx * ry) % 8]}

[0029] {S[k1,c,ox,oy,rx,ry] →

[0030] T[fl(k1 / 8),fl((c * rx * ry) / 8),ox * oy + (c * rx * ry) % 8 + k1 % 8 * pe_latency]}

[0031] When the type of the workload is the convolution operation CONV and the type of the data stream is output stationary OS, the relationship-centered expression symbol is:

[0032] {S[k1,c,ox,oy,rx,ry] → PE[k1 % 8,(ox * oy) % 8]}

[0033] {S[k1,c,ox,oy,rx,ry] →

[0034] T[fl(k1 / 8),fl((ox * oy) / 8),c * rx * ry + (ox * oy) % 8 + k1 % 8 * pe_latency]}

[0035] Wherein, S represents a given loop instance, PE represents the allocated processing unit, "→" represents the mapping relationship between the given loop instance on the left and the allocated processing unit on the right, T represents the timestamp, pe_latency is the number of cycles required for the processing unit to complete the multiply-accumulate operation, fl represents the floor function, % represents the modulus, and i, j, k, c, ox, oy, rx, ry are parameters in the general matrix multiplication GEMM operation. i*k and k*j are the scales of two matrices; k1, c, ox, oy, rx, ry are parameters in the convolution operation CONV. k1 represents the number of convolution kernels, c represents the number of channels of the input image, the scale of a single-channel image in the output image is ox*oy, and the scale of a single-channel convolution kernel in the convolution kernel is rx*ry.

[0036] Optionally, in step S3, constructing the mapping space composed of mapping schemes that meet the hardware resource constraints specifically refers to, for two matrices of size M*K and K*N as the workload, respectively using different block partitioning methods for block partitioning under the premise of meeting the hardware resource constraints. Each block partitioning method obtains a mapping scheme that meets the hardware resource constraints, so as to obtain a mapping space composed of multiple mapping schemes corresponding to multiple block partitioning methods that meet the hardware resource constraints.

[0037] Optionally, in the performance evaluation model established in step S4 for evaluating the performance indicators of each mapping scheme, the performance indicators include the total calculation time T of a single block of the mapping scheme and the utilization rate of the processing unit in the systolic array accelerator.

[0038] Optionally, the calculation function expression of the total calculation time T of a single block of the mapping scheme is:

[0039] T = T1 + T2 + T3 + T4,

[0040] In the above formula, T1 is the calculation time of the last dimension of the multi-dimensional timestamp, T2 is the time cost corresponding to the floor function fl(j), T3 is the time cost corresponding to the floor function fl(k), T4 is the additional time overhead caused by pauses, and there are:

[0041] T1 = m - 1 + (n - 1) % x + (k - 1) % x * pe_latency,

[0042]

[0043] T3 = m * n * (k - 1) / (x * x),

[0044] T4 = (x * pe_latency - m) * k * n / (2 * x * x),

[0045] In the above formula, m*k represents the scale of one matrix in the block matrix multiplication operation, n*k represents the scale of another matrix in the block matrix multiplication operation, k is the common dimension of the two matrices in the block matrix multiplication operation, x*x is the scale of the systolic array, and pe_latency is the number of cycles required for the processing unit to complete the multiply-accumulate operation.

[0046] Optionally, the calculation function expression for the utilization rate of the processing unit in the systolic array accelerator is:

[0047] Utuluzation = M*K*N / (T total *x*x),

[0048] In the above formula, Utilization represents the utilization rate of the processing unit in the systolic array accelerator, M*K*N represents the number of multiply-accumulate operations required when two matrices of size M*K and K*N are multiplied on a systolic array of size x*x, and T total represents the time required when two matrices of size M*K and K*N are multiplied on a systolic array of size x*x.

[0049] Optionally, when searching for the mapping scheme with the optimal performance index in the mapping space according to the performance evaluation model in step S4, if the scale of the workload is less than the set value, the method of traversing is used to search for the mapping scheme with the optimal performance index in the mapping space according to the performance evaluation model, otherwise the heuristic search method is used to search for the mapping scheme with the optimal performance index in the mapping space according to the performance evaluation model.

[0050] In addition, the present invention also provides a system for evaluating the performance of a systolic array accelerator, including a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to execute the method for evaluating the performance of the systolic array accelerator.

[0051] In addition, the present invention also provides a computer-readable storage medium, in which a computer program is stored, and the computer program is used to be programmed or configured by a microprocessor to execute the method for evaluating the performance of the systolic array accelerator.

[0052] Compared with the prior art, the present invention mainly has the following advantages:

[0053] 1. The present invention can improve the expression ability of performance evaluation and the accuracy of simulation results, and can evaluate the performance of various mappings faster than a cycle-accurate simulator and find the best mapping scheme.

[0054] 2. The present invention uses a relationship - centered representation to describe different data streams, fully considers the adverse effects that may be caused by resource conflicts during computation, and can handle the performance models of multiple data streams and arbitrary processing element (PE) latencies under a systolic array structure, having stronger expressive power and being able to provide more accurate evaluation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a schematic diagram of the basic architecture of an existing systolic array accelerator.

[0056] Figure 2 It is a schematic diagram of the principle of the mapping scheme of an existing general matrix multiplication.

[0057] Figure 3 It is a schematic diagram of the basic process of the method of the embodiment of the present invention.

[0058] Figure 4 It is an example of using the relationship - centered representation method in the embodiment of the present invention to analyze matrix multiplication using the IS data stream on a 2×2 - scale systolic array.

[0059] Figure 5 It is an example of the calculation process of the GEMM workload using the IS data stream in the embodiment of the present invention.

[0060] Figure 6 It is an example of legal mapping and illegal mapping in the embodiment of the present invention.

[0061] Figure 7 It is a comparison of the time overhead of GEMM and CONV workloads in the embodiment of the present invention, and pe_latency = 1.

[0062] Figure 8 It is a comparison between the method (TILE - SIM) of the embodiment of the present invention and the hardware simulation platform under GEMM and CONV workloads, and pe_latency = 6. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0063] As Figure 3 shown, the method for evaluating the performance of a systolic array accelerator in this embodiment includes:

[0064] S1. Input the workload, data stream, and the architecture parameters of the systolic array. The architecture parameters of the systolic array include the scale of the systolic array, the scale of on - chip storage, and the transmission bandwidth;

[0065] S2. Determine the corresponding relationship - centered expression symbol according to the types of the workload and the data stream;

[0066] S3. According to the scale of the workload and the scale of on-chip storage, for the relation-centered expression symbols, construct a mapping space composed of mapping schemes that meet the hardware resource constraints, where each point in the mapping space represents a mapping scheme that meets the hardware resource constraints;

[0067] S4. Establish a performance evaluation model for evaluating the performance metrics of each mapping scheme, and search for the mapping scheme with the optimal performance metrics in the mapping space according to the performance evaluation model.

[0068] TILE-SIM supports multiple workloads, including general matrix multiplication (GEMM), convolutional layer (CONV), and fully connected layer. In this embodiment, the type of the workload input in step S1 is general matrix multiplication GEMM or convolutional operation CONV, and the type of the data stream input in step S1 is input stationary IS, weight stationary WS, or output stationary OS. Generally speaking, the data stream latency includes communication latency and computation latency. In the case of using pipelining technology and double buffering technology to hide latency, the total latency is equal to the maximum of the two. Since the communication latency can be calculated based on the available data volume and bandwidth, in order to more accurately simulate the computing process, this paper assumes sufficient bandwidth during modeling; this helps to exclude the influence of bandwidth on the simulation results and enables this paper to make a more fair comparison with other works.

[0069] To improve the generality of the model and make it applicable to a wider range of application scenarios, this paper uses the relation-centered representation method for modeling. Specifically, the relation-centered representation method assigns a spatial timestamp PE and a temporal timestamp T to a given loop instance S. The spatial timestamp provides the PE coordinates where S will be executed, while the temporal timestamp determines the execution order of S. To ensure the correctness of the temporal timestamp, the relation-centered notation assumes that the systolic array can obtain sufficient data supply during the computation. Figure 4Shows an example of mapping GEMM on a systolic array when pe_latency is 2. At this time, a single pe can simultaneously execute the calculation of multiple groups of data by adopting a pipelined calculation method within the pe. In this example, the loop instance S[i, j, k] is executed on PE[k, j] and is assigned a one-dimensional timestamp (i + j + k * 2). The data that meets the conditions will be sent to the systolic array for calculation at the appropriate time; after two cycles, the calculation will be completed and the registers within the pe will be released. For example, A[0, 0] and B[0, 00] are calculated at PE[0, 0] with timestamp T[0] and completed at timestamp T[2]. At the same time, A[2, 0] and B[0, 0] will start a new calculation at PE[0, 0]. TILE - SIM delays the time when A[0, 1] and B[1, 0] enter PE[1, 0] based on the PE_latency value. The reason is that A[0, 1] and B[1, 0] need to use the results of calculating A[0.0] and B[0, 0] as partial sums to participate in the calculation. By extending the filling and discharging time of the PE array, the method (TILE - SIM) in this embodiment for evaluating the performance of the systolic array accelerator ensures that data in different rows is sequentially transmitted to the systolic array at fixed time intervals, thereby achieving vertical multiplexing of data. Therefore, the pe_latency in the timestamp is related to the longitudinal coordinate of the 2D systolic array. In Figure 4 , the k - dimension corresponds to the ordinate of the 2D systolic array, so it is multiplied by pe_latency. In the WS and OS data streams, the i - th dimension corresponds to the ordinate of the 2D systolic array, so pe_latency is multiplied by i. It can be observed from the examples in this article that the timestamp not only determines the execution order of S but also records the specific moment of execution. By constructing a function with the timestamp as a variable, this article can quickly determine the time required to complete the calculation. In this embodiment, the relationship - centered expression symbols determined in step S2 include:

[0070] When the type of workload is general matrix multiplication GEMM and the type of data stream is input - stationary IS, the relationship - centered expression symbols are:

[0071] {S[i,j,k]→PE[k%8,j%8]}

[0072] {S[i,j,k]→T[fl(k / 8),fl(j / 8),i+j%8+k%8*pe_latency]}

[0073] When the type of workload is general matrix multiplication GEMM and the type of data stream is weight - stationary WS, the relationship - centered expression symbols are:

[0074] {S[i,j,k]→PE[i%8,k%8]}

[0075] {S[i,j,k] → T[fl(i / 8),fl(k / 8),j + i % 8 * pe_latency + k % 8]}

[0076] When the type of the workload is General Matrix Multiplication (GEMM) and the type of the data stream is Output Stationary (OS), the relation - centered expression symbol is:

[0077] {S[i,j,k] → PE[i % 8,j % 8]}

[0078] {S[i,j,k] → T[fl(i / 8),fl(j / 8),i % 8 * pe_latency + j % 8 + k]}

[0079] When the type of the workload is Convolution (CONV) and the type of the data stream is Input Stationary (IS), the relation - centered expression symbol is:

[0080] {S[k1,c,ox,oy,rx,ry] → PE[(c * rx * ry) % 8,(ox * oy) % 8]}

[0081] {S[k1,c,ox,oy,rx,ry] →

[0082] T[fl((c * rx * ry) / 8),fl((ox * oy) / 8),k1+(ox * oy) % 8+(c * rx * ry) % 8 * pe_latency]}

[0083] When the type of the workload is Convolution (CONV) and the type of the data stream is Weight Stationary (WS), the relation - centered expression symbol is:

[0084] {S[k1,c,ox,oy,rx,ry] → PE[k1 % 8,(c * rx * ry) % 8]}

[0085] {S[k1,c,ox,oy,rx,ry] →

[0086] T[fl(k1 / 8),fl((c * rx * ry) / 8),ox * oy+(c * rx * ry) % 8 + k1 % 8 * pe_latency]}

[0087] When the type of the workload is Convolution (CONV) and the type of the data stream is Output Stationary (OS), the relation - centered expression symbol is:

[0088] {S[k1,c,ox,oy,rx,ry] → PE[k1 % 8,(ox * oy) % 8]}

[0089] {S[k1, c, ox, oy, rx, ry] ->

[0090] T[fl(k1 / 8), fl((ox * oy) / 8), c * rx * ry + (ox * oy) % 8 + k1 % 8 * pe_latency]}

[0091] Among them, S represents a given loop instance, PE represents the allocated processing unit, "->" represents the mapping relationship between the given loop instance on the left and the allocated processing unit on the right, T represents the timestamp, pe_latency is the number of cycles required for the processing unit to complete the multiply-accumulate operation, fl represents the floor function, % represents the modulus, i, j, k, c, ox, oy, rx, ry are parameters in the general matrix multiplication GEMM operation, i * k and k * j are the scales of two matrices; k1, c, ox, oy, rx, ry are parameters in the convolution operation CONV, k1 represents the number of convolution kernels, c represents the number of channels of the input image, the size of a single-channel image in the output image is ox * oy, and the size of a single-channel convolution kernel is rx * ry.

[0092] When the user finishes the input, the mapper adopted in this embodiment can construct a mapping space according to the workload and architecture parameters. The method of this embodiment refers to different scheduling operations and data partitioning adopted by the hardware during the calculation as different mappings. According to the scale of the workload and the size of the on-chip storage, the method of this embodiment can filter out all mapping schemes that meet the hardware resource constraints. Each scheme corresponds to a point in the mapping space, and together they constitute the entire mapping space. Briefly speaking, this process is to regard the workload as two matrices, and a mapping scheme is a partitioning method of dividing the matrix into blocks. Different-shaped blocks correspond to different mapping schemes. Meeting the hardware resource constraints means that the size of each block should be smaller than the scale of the on-chip storage, so as to ensure that the block can be stored on the chip. Specifically, in step S3 of this embodiment, constructing the mapping space composed of mapping schemes that meet the hardware resource constraints specifically means that for two matrices of size M * K and K * N as the workload, different block partitioning methods are respectively adopted under the premise of meeting the hardware resource constraints for block partitioning. Each block partitioning method obtains a mapping scheme that meets the hardware resource constraints, so as to obtain a mapping space composed of multiple mapping schemes corresponding to multiple block partitioning methods that meet the hardware resource constraints.

[0093] In this embodiment, in the performance evaluation model established in step S4 for evaluating the performance indicators of each mapping scheme, the performance indicators include the total calculation time T of a single block of the mapping scheme and the utilization rate of the processing units in the systolic array accelerator. When the user inputs a relationship-centered expression symbol, the method of this embodiment will evaluate according to relevant formulas and give the results. Specifically, for a given mapping scheme, the method of this embodiment first performs block analysis. According to the mapping scheme, the model can quickly analyze the order and corresponding sizes of the blocks. Thereafter, the performance model can quickly calculate the calculation time overhead of the corresponding block by calling the corresponding formula according to the relationship-centered expression symbol. By calling the formula multiple times to calculate the calculation time of each block and finally accumulating, the total time T can be obtained. In addition, according to the scale of the workload, the model can calculate the total amount of operations required to complete the calculation. According to the scale of the systolic array, the model can calculate the shortest calculation time in the ideal case. Dividing it by the total time T calculated by the model gives the PE utilization rate of the systolic array.

[0094] A common method for simulating the computational delay involves estimating using the total number of loop instances and the average number of active PEs. The existing TENET method also adopts a relationship-centered representation and is modeled by this method. However, this method requires calling the ISL library for complex modeling; in addition, it can only simulate the scenario where pe_latency is 1. The method (TILE-SIM) for evaluating the performance of the systolic array accelerator in this embodiment mathematically models the computational process and derives relevant formulas based on the fact that the computational and data movement patterns during the calculation of the accelerator are largely deterministic. Figure 5 Shows the computational process of a general matrix multiplication with a stationary input data stream. During the calculation process, a1 is calculated successively using b11, b12, and b13, and then a2 is calculated successively using b21, b22, and b23. It can be intuitively observed from the figure that when the pipelining technique is adopted, the time overhead of most matrix calculations running at full capacity on the systolic array is m. Combining with the expression of the multi-dimensional timestamp, it can be found in this paper that each dimension of the timestamp corresponds to the corresponding calculation time.

[0095] T1 corresponds to the calculation time of the last dimension of the multi-dimensional timestamp. When the loop is fully unrolled, i, j, and k all take their maximum values. Therefore, T1 can be obtained as follows:

[0096] T1 = m - 1 + (n - 1) % x + (k - 1) % x * pe_latency,

[0097] T2 represents the time cost corresponding to the function fl(j), which describes Figure 4The time cost of matrix calculations of the same type in. Taking b21, b22, and b23 as examples, the time spent on each matrix calculation is m, and the number of matrices is determined by the maximum value of j and the size of the systolic array. From this, the present paper can obtain T2:

[0098]

[0099] T3 corresponds to the floor function fl(k). It describes Figure 4 the number of matrices of different types in, which is again determined by the maximum value of k and the size of the systolic array. Therefore, the present paper can obtain T3 in the following way:

[0100] In addition, the present paper also observes that due to the limitations of the PE registers and the mapping scheme, the systolic array may pause during calculations, which introduces additional time overhead. The present paper refers to the mapping scheme that enables the systolic array to operate at full capacity as a legal mapping, and the rest are regarded as illegal mappings. Pipeline technology is usually used to ensure that the hardware can perform efficient calculations; however, this also adds constraints to the mapping scheme. Figure 6 The a in shows a legal mapping scheme. Before starting the calculation of A3, A1 must complete all calculations and leave the systolic array, and the data related to the A3 calculation must be stored in the registers of the processing elements. When m is large enough, this condition can be met. However, when m is small, bubbles will appear in the pipeline, which will reduce the calculation efficiency of the hardware. Figure 6 The b in shows this situation. When m is small, after the first row of data of A2 is transferred into the systolic array, the first row of data of A3 cannot be transferred immediately. Since each PE has only two registers for storing the data required for calculations, when both A1 and A2 are performing calculations, the data required for the A3 calculation cannot enter the systolic array. Therefore, A3 needs to wait for the calculation of A1 to complete and the new data to be stored in the PE registers before it can start calculating. Current mainstream simulators, such as TENET and SCALE-SIM, do not consider this problem. Therefore, when it is impossible to assume sufficient data supply, it is difficult for their models to ensure that the simulation results are correct. When x * pe_latency is greater than m, the systolic array calculation will pause. T4 represents the additional time overhead generated due to the pause. Since the number of PE registers is two, a pause occurs whenever two sub-blocks complete their calculations. The duration of the pause is x * pe_latency - m. From this, the present paper can obtain T4:

[0101] T4 = (x * pe_latency - m) * k * n / (2 ** x),

[0102] In summary, the calculation delay T = T1 + T2 + T3 + T4 can be obtained in this paper. For the WS and OS data streams, the corresponding calculation delays can also be derived according to the symbols in Table 1. Therefore, the calculation function expression of the total calculation time T of a single block in the mapping scheme of this embodiment is:

[0103] T = T1 + T2 + T3 + T4,

[0104] In the above formula, T1 is the calculation time of the last dimension of the multi-dimensional timestamp, T2 is the time cost corresponding to the floor function fl(j), T3 is the time cost corresponding to the floor function fl(k), T4 is the additional time overhead caused by pauses, and there are:

[0105] T1 = m - 1 + (n - 1) % x + (k - 1) % x * pe_latency,

[0106]

[0107] T3 = m * n * (k - 1) / (x * x),

[0108] T4 = (x * pe_latency - m) * k * n / (2 * x * x),

[0109] In the above formula, m * k represents the scale of one matrix in the block matrix multiplication operation, n * k represents the scale of the other matrix in the block matrix multiplication operation, k is the common dimension of the two matrices in the block matrix multiplication operation, x * x is the scale of the systolic array, and pe_latency is the number of cycles required for the processing unit to complete the multiply-accumulate operation.

[0110] It should be emphasized here that the formula T represents the calculation time of a single block. For a complete workload, the complete delay can be simply obtained in this paper by calculating the calculation times of different blocks using the formula. When searching for the optimal mapping scheme, the method (TILE-SIM) for evaluating the performance of the systolic array accelerator in this embodiment uses a performance model to evaluate each mapping and finally finds the optimal mapping. This evaluation uses the formula derived from the relationship-centered representation, so it can produce faster estimation results compared with the cycle-accurate simulator.

[0111] The utilization rate of the processing unit (PE utilization rate) is an important indicator for evaluating the performance of the accelerator. In this paper, the PE utilization rate is obtained according to the number of MAC operations that need to be executed for the calculation, the size of the systolic array, and the calculation time. Specifically, the calculation function expression of the utilization rate of the processing unit in the systolic array accelerator is:

[0112] Utilization = M * K * N / (T total * x * x),

[0113] In the above formula, Utilization represents the utilization rate of the processing unit in the systolic array accelerator, M*K*N represents the number of multiply-accumulate operations required when multiplying two matrices of sizes M*K and K*N on a systolic array of size x*x, and T total represents the time required when multiplying two matrices of sizes M*K and K*N on a systolic array of size x*x.

[0114] In this embodiment, when searching for the mapping scheme with the optimal performance metrics in the mapping space according to the performance evaluation model in step S4, if the scale of the workload is smaller than the set value, the traversal method is used to search for the mapping scheme with the optimal performance metrics in the mapping space according to the performance evaluation model; otherwise, the heuristic search method is used to search for the mapping scheme with the optimal performance metrics in the mapping space according to the performance evaluation model. For small-scale workloads, the traversal method can be used to search the mapping space to finally obtain the optimal mapping scheme. For large-scale workloads, the heuristic method can be used for searching.

[0115] The method (TILE-SIM) for evaluating the performance of the systolic array accelerator in this embodiment supports multiple data streams and different workloads, and is applicable to systolic arrays of different sizes. For the convenience of comparison with other works, unless otherwise specified, an 8*8 systolic array with a single on-chip buffer size of 512KB is used in the following experiments in this paper. Since this paper uses pipeline and double-buffer optimization techniques, this paper assumes that the accelerator has sufficient bandwidth, which means that the time required for data communication is hidden by the computing time. This assumption enables this paper to exclude the influence of bandwidth on the final result, thereby helping this paper more accurately evaluate the modeling results of different simulators. For the workloads in the experiments of this paper, the convolutional layers in AlexNet and several representative layers in the following widely used contemporary natural language processing models are used: GNMT3, DeepSpeech2, and Neural Collaborative Filtering (NCF1). The matrix dimensions corresponding to these workloads are listed in detail in Table 1.

[0116] Table 1: Matrix dimensions of the language model workloads mapped to M, K, and N.

[0117] Name M K N GNMT3 2048 32 4096 DB1 35 2560 4096 NCF1 256 2048 256

[0118] When pe_latency is 1, two workloads are tested in the method (TILE-SIM) for evaluating the performance of the systolic array accelerator in this embodiment and the existing TENET method, SCALE-SIM method, and hardware simulation platform (Emulator) in this paper. Figure 7Shows the simulation results on GEMM and CONV using the IS data stream, where the bars Figure 1 represent the hardware emulation platform (Emulator), the bars Figure 2 represent TENET, the bars Figure 3 represent SCALE - SIM, the bars Figures 4 - 6 represent the related method of the method of this embodiment (TILE - SIM), the bars Figure 4 represent the best mapping scheme found by the method of this embodiment (TILE - SIM), while the bars Figure 5 and the bars Figure 6 correspond to the invariant mapping scheme respectively. Specifically, the bars Figure 5 represent the mapping scheme with a size of (30, 30, 10), the bars Figure 6 represent the mapping method with a size of (30, 15, 20). The bars Figure 4 represent the best mapping scheme found by the method of this embodiment (TILE - SIM), and thus varies with the workload. In this embodiment, the hardware emulation time can be regarded as the golden result. For other models, the closer the estimated time is to this result, the better the performance of the scheme. Figure 7 In, C1 - C5 represent the five convolutional layers of AlexNet, and DB1, GNMT3, NCF1 represent the three simulated systolic arrays.

[0119] From Figure 7It can be seen that the simulation results obtained by TENET are the smallest. The reason is that TENET uses the maximum number of active PEs to estimate the computing time. Therefore, TENET assumes that the systolic array is always at full capacity when estimating the required time. However, in reality, this assumption is difficult to meet. The best mapping scheme M1 of TILE-SIM is always close to the golden result. Compared with TENET, which can only provide the best estimate value but not a specific mapping scheme, TILE-SIM can provide more specific and reliable technical guidance for accelerator users, enabling them to maximize the hardware performance. For its part, SCALE-SIM does not consider the overlap between data filling time and computing time; therefore, the gap between its result and the golden result depends on the workload and may vary greatly in some cases. When simulating DB1, the systolic array will not be fully loaded because M is not a multiple of 8, which will reduce the computing efficiency and cause SCALE-SIM to produce results far from the golden result. When simulating GNMT3, due to the small K, large M and N, the workload is often tiled when mapped to the systolic array. At this time, the gap between SCALE-SIM and the golden result is also large. As shown in the NCF1 simulation, only when neither of the above two cases occurs, the results of SCALE-SIM are more accurate. During the evaluation using SCALE-SIM, the parameters of the convolutional layer in this paper are converted into the parameters of GEMM for testing. The experimental results show that the simulation results of SCALE-SIM are close to the golden result, while the optimized mapping results of TILE-SIM are better than SCALE-SIM. The reason is that the obtained K value is large after the transformation of the convolutional layer parameters, while the M and N values are small, which means that the estimated value of SCALE-SIM is more accurate. However, since SCALE-SIM is only a solution in the mapping space, the estimated results of SCALE-SIM are not as accurate as the optimal solution obtained by the method (TILE-SIM) for evaluating the performance of the systolic array accelerator in this embodiment.

[0120] Table 2 shows the results of three simulators using different data streams to simulate the first convolutional layer of AlexNet.

[0121] Table 2: Comparison of the results of TILE-SIM, TENET and SCALE-SIM under three data streams (PE_latency = 1).

[0122] Data flow TILE-SIM TENET SCALE-SIM IS 1,651,347 cycles 1,647,112 cycles 1,750,980 cycles WS 1,647,342 cycles 1,647,112 cycles 1,750,980 cycles OS 1,651,442 cycles 1,647,112 cycles 1,750,980 cycles

[0123] As can be seen from Table 2, SCALE-SIM does not consider the case where the time required to fill the data into the systolic array can be masked by the computation time, and proposes an expression applicable to three data streams (input stationary IS, weight stationary WS, or output stationary OS); therefore, it is insensitive to the data stream when simulating the computation time. TENET proposes a scheme to estimate the computation time using the total number of loop instances and the average number of active PEs. However, after examining its code, it is found in this paper that TENET actually uses the maximum number of active PEs instead of the average number of active PEs in the estimation. This results in TENET obtaining the same computation time when simulating the same workload with different data streams. In contrast, TILE-SIM supports the simulation of different data streams; it fully considers the impact of various factors on the latency during the modeling process and can find the optimal mapping.

[0124] This paper also constructs a register transfer level hardware simulation platform to evaluate the case where pe_latency is not 1. Specifically, this paper uses a 16*16 scale systolic array and tests different GEMM and CONV workloads with pe_latency of 6. C1-C5 represent five convolutional layers of AlexNet, and FC1-FC3 represent three fully connected layers of AlexNet. The method (TILE-SIM) for evaluating the performance of the systolic array accelerator in this embodiment explores the optimal mapping of different workloads and tests using the mapping scheme on the hardware simulation platform. From Figure 8 It can be seen that the simulation results of the method (TILE-SIM) for evaluating the performance of the systolic array accelerator in this embodiment are close to those of the hardware simulation platform, which further proves the accuracy and generality of the method (TILE-SIM) for evaluating the performance of the systolic array accelerator in this embodiment. Since the hardware will be affected by insufficient bandwidth and fetch conflicts during the computation, the actual measurement time will be longer than the model estimation time. The experimental results also show that under the optimal mapping scheme, all layers except C1 have a high PE utilization rate. The PE utilization rate of the C1 layer is not high because the number of convolutional kernels in this layer is small, which means that the M dimension of the matrix obtained after the im2col transformation of the data corresponding to this layer is small. In this case, the systolic array will pause during the computation. During the computation of C1, T4 accounts for 24% of the total time, which also leads to a reduction in the PE utilization rate.

[0125] In summary, the matrix engine based on the systolic array provides high performance and energy efficiency, and thus is widely used in AI or HPC chips. Existing accelerator simulators emphasize application-specific exploration of the hardware design space. However, these simulators cannot simulate the results of different application mappings under a specific architecture, which is also extremely important for performance verification and software development. Traditional cycle-accurate simulators can achieve this, but they are difficult to develop and have long simulation times. To solve these problems, the method for evaluating the performance of the systolic array accelerator in this embodiment uses a relationship-centered representation to describe different data streams, and fully considers the adverse effects that may be caused by resource conflicts during computation. It can handle a performance model of multiple data streams and arbitrary processing element (PE) delays under the systolic array structure. Compared with the current state-of-the-art similar technologies, the method for evaluating the performance of the systolic array accelerator in this embodiment has stronger expressive power and can provide more accurate evaluation results for users. Generally speaking, the method for evaluating the performance of the systolic array accelerator in this embodiment is a practical and widely applicable systematic method, which can help people better understand the impact of hardware resources, workloads, and data streams on the mapping scheme; in addition, it can also be used to systematically evaluate the mapping scheme to achieve the goal of fully maximizing the hardware computing performance.

[0126] In addition, this embodiment also provides a system for evaluating the performance of a systolic array accelerator, including a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to execute the method for evaluating the performance of the systolic array accelerator.

[0127] In addition, this embodiment also provides a computer-readable storage medium, in which a computer program is stored, and the computer program is used to be programmed or configured by a microprocessor to execute the method for evaluating the performance of the systolic array accelerator.

[0128] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code. The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in the process Figure 1 one process or multiple processes and / or blocks Figure 1 These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in the process Figure 1 one process or multiple processes and / or blocks Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in the process Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0129] The above is only the preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions falling within the idea of the present invention belong to the protection scope of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements should also be regarded as within the protection scope of the present invention.

Claims

1. A method for evaluating the performance of a systolic array accelerator, characterized in that, Including: S1, input the workload, data stream and architecture parameters of the systolic array, where the architecture parameters of the systolic array include the scale of the systolic array, the scale of on-chip storage, and the transmission bandwidth; S2, determine the corresponding relationship-centered expression symbols according to the types of the workload and the data stream; S3, according to the scale of the workload and the scale of on-chip storage, construct a mapping space composed of mapping schemes that meet the hardware resource constraints for the relationship-centered expression symbols, and each point in the mapping space represents a mapping scheme that meets the hardware resource constraints; S4, establish a performance evaluation model for evaluating the performance indicators of each mapping scheme, and search for the mapping scheme with the optimal performance indicators in the mapping space according to the performance evaluation model; The type of the workload input in step S1 is general matrix multiplication GEMM or convolution operation CONV, and the type of the data stream input in step S1 is input stationary IS, weight stationary WS, or output stationary OS; Specifically, constructing the mapping space composed of mapping schemes that meet the hardware resource constraints in step S3 means that for two matrices of size M*K and K*N as the workload, different block partitioning methods are respectively used for block partitioning under the premise of meeting the hardware resource constraints, and each block partitioning method obtains a mapping scheme that meets the hardware resource constraints, so as to obtain a mapping space composed of multiple mapping schemes corresponding to multiple block partitioning methods that meet the hardware resource constraints; In the performance evaluation model established in step S4 for evaluating the performance indicators of each mapping scheme, the performance indicators include the total calculation time of a single block of the mapping scheme T and the utilization rate of the processing units in the systolic array accelerator.

2. The method for evaluating the performance of a systolic array accelerator according to claim 1, wherein The relationship-centered expression symbols determined in step S2 include: When the type of the workload is general matrix multiplication GEMM and the type of the data stream is input stationary IS, the relationship-centered expression symbol is: {S[i,j,k] → PE[k%8,j%8]} {S[i,j,k] → T[fl(k / 8),fl(j / 8),i+j%8+k%8*pe_latency]} When the type of the workload is general matrix multiplication GEMM and the type of the data stream is weight stationary WS, the relationship-centered expression symbol is: {S[i,j,k] → PE[i%8,k%8]} {S[i,j,k] → T[fl(i / 8),fl(k / 8),j+i%8*pe_latency+k%8]} When the type of the workload is general matrix multiplication GEMM and the type of the data stream is output stationary OS, the relationship-centered expression symbol is: {S[i,j,k] → PE[i%8,j%8]} {S[i,j,k] → T[fl(i / 8),fl(j / 8),i%8* pe_latency+j%8+k]} When the type of the workload is convolution operation CONV and the type of the data stream is input stationary IS, the relationship-centered expression symbol is: {S[k1,c,ox,oy,rx,ry] → PE[(c*rx*ry)%8,(ox*oy)%8]} {S[k1,c,ox,oy,rx,ry] → T[fl((c*rx*ry) / 8),fl((ox*oy) / 8),k1+(ox*oy)%8+(c*rx*ry)%8*pe_latency]} When the type of workload is convolution operation CONV and the type of data stream is weight stationary WS, the relationship-centered expression symbol is: {S[k1,c,ox,oy,rx,ry] → PE[k1%8,(c*rx*ry)%8]} {S[k1,c,ox,oy,rx,ry] → T[fl(k1 / 8),fl((c*rx*ry) / 8),ox*oy+(c*rx*ry)%8+k1%8*pe_latency]} When the type of workload is convolution operation CONV and the type of data stream is output stationary OS, the relationship-centered expression symbol is: {S[k1,c,ox,oy,rx,ry] → PE[k1%8,(ox*oy)%8]} {S[k1,c,ox,oy,rx,ry]→ T[fl(k1 / 8),fl((ox*oy) / 8),c*rx*ry+(ox*oy)%8+k1%8*pe_latency]} Among them, S represents a given loop instance, PE represents the allocated processing unit, "→" represents the mapping relationship between the given loop instance on the left and the allocated processing unit on the right, T represents the timestamp, pe_latency is the number of cycles required for the processing unit to complete the multiply-accumulate operation, fl represents the floor function, % represents the modulus, i, j, k, c, ox, oy, rx, ry are parameters in the general matrix multiplication GEMM operation, i*k and k*j are the scales of two matrices; k1, c, ox, oy, rx, ry are parameters in the convolution operation CONV, k1 represents the number of convolution kernels, c represents the number of channels of the input image, the scale of a single-channel image in the output image is ox*oy, and the scale of a single-channel convolution kernel is rx*ry.

3. The method for evaluating the performance of a systolic array accelerator according to claim 2, wherein The total calculation time of a single block of the mapping scheme T The calculation function expression is as follows: ， In the above formula, is the calculation time of the last dimension of the multi-dimensional timestamp, is the time cost corresponding to the floor function fl(j), is the time cost corresponding to the floor function fl(k), is the additional time overhead caused by the pause, and there is: ， ， ， ， In the above formula, represents the scale of one matrix in the block matrix multiplication operation, represents the scale of another matrix in the block matrix multiplication operation, is the common dimension of the two matrices in the block matrix multiplication operation, is the scale of the systolic array, pe_ latency is the number of cycles required for the processing unit to complete the multiply-accumulate operation.

4. The method for evaluating the performance of a systolic array accelerator according to claim 1, wherein The calculation function expression of the utilization rate of the processing unit in the systolic array accelerator is: ， In the above formula, represents the utilization rate of the processing unit in the systolic array accelerator, and M*K*N represents the number of multiply-accumulate operations required when two matrices of sizes M*K and K*N are multiplied on a systolic array of size . represents the time required when two matrices of sizes M*K and K*N are multiplied on a systolic array of size .

5. The method for evaluating the performance of a systolic array accelerator according to claim 1, wherein When searching for the mapping scheme with the optimal performance index in the mapping space according to the performance evaluation model in step S4, if the scale of the workload is less than the set value, the method of traversing is used to search for the mapping scheme with the optimal performance index in the mapping space according to the performance evaluation model, otherwise the heuristic search method is used to search for the mapping scheme with the optimal performance index in the mapping space according to the performance evaluation model.

6. A system for evaluating the performance of a systolic array accelerator, comprising a microprocessor and a memory connected to each other, characterized in that, The microprocessor is programmed or configured to execute the method for evaluating the performance of the systolic array accelerator according to any one of claims 1 to 5.

7. A computer-readable storage medium storing a computer program, characterized in that, The computer program is used to be programmed or configured by the microprocessor to execute the method for evaluating the performance of the systolic array accelerator according to any one of claims 1 to 5.

Images