Method for improving sparse calculation efficiency of in-memory calculation array

Abstract

The invention discloses a method for improving sparse calculation efficiency of an in-memory calculation array, which comprises the following steps of: preprocessing data to participate in calculation, and performing non-zero value aggregation at a bit level; during calculation, the calculation data subjected to non-zero value aggregation and the data stored in the corresponding sub-array in the in-memory calculation array are subjected to operation. According to the method, the in-memory calculation efficiency can be improved based on bit-level sparseness.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of integrated circuits, in particular to a method for improving the efficiency of sparse computation of an in-memory computing array. BACKGROUND

[0002] In-memory computing is an emerging computing paradigm that can reduce the transmission overhead of data from a storage module to a computing module. However, the data flow flexibility of an in-memory computing array is low, and it is good at processing dense computation but not good at processing sparse computation. When the sparsity of the model to be accelerated by in-memory computing is high, the efficiency of the in-memory computing array will be low. Existing in-memory computing arrays are mostly designed for dense computation. Even if in-memory computing considers sparsity, there is a problem of low optimization granularity. SUMMARY

[0003] The present application provides a method for improving the efficiency of sparse computation of an in-memory computing array, which can improve the efficiency of in-memory computing based on bit-level sparsity.

[0004] The present application provides a method for improving the efficiency of sparse computation of an in-memory computing array, which comprises:

[0005] The data to be involved in the computation is preprocessed, non-zero value aggregation is performed at the bit level, and the address of each row subarray of the in-memory computing array is configured according to the data exchange condition in the non-zero value aggregation;

[0006] During computation, the computation data after non-zero value aggregation is operated with the data stored in the corresponding subarray of the in-memory computing array.

[0007] The present application preprocesses the data to be involved in the computation, performs non-zero value aggregation at the bit level, and configures the address of each row subarray of the in-memory computing array according to the data exchange condition in the non-zero value aggregation, so that each input channel can transmit an independent subarray address signal, realizing the flexibility of adjusting the in-memory computing array computation according to the sparsity. By using this flexibility, the computation amount of the in-memory computing array is reduced during computation without affecting the computation result by performing non-zero value aggregation on data with high sparsity. Especially when the load sparsity is high enough, a full-zero computation word is easily generated, and such a computation word can be skipped for computation, thereby reducing the computation delay and power consumption. Therefore, the present application can improve the support of the in-memory computing array for sparsity, reduce the delay when processing sparse computation, reduce the computation power consumption, and ultimately improve the efficiency of the in-memory computing array when processing sparse computation. BRIEF DESCRIPTION OF DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0009] Figure 1 A method flow chart for improving the sparse computing efficiency of an in-memory computing array is provided for the embodiments of the present application.

[0010] Figure 2 A schematic diagram of an in-memory computing array with independent input channels is provided for the embodiments of the present application.

[0011] Figure 3 A computing schematic diagram for different computing words in the same input channel to exchange input data is provided for the embodiments of the present application.

[0012] Figure 4 A bit-level non-zero value aggregation method schematic diagram is provided for the embodiments of the present application.

[0013] Figure 5 A Figure 4 Input data input and computing schematic diagram without non-zero value aggregation in the embodiments of the present application.

[0014] Figure 6 Input data input and computing schematic diagram after non-zero value aggregation. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solutions and advantages of the present application clearer, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0016] The existing in-memory computing array is mostly designed for dense computing. When the sparsity of the model is high, the problem of low computing efficiency of the in-memory computing array occurs. Therefore, the present application is based on a method for improving the sparse computing efficiency of an in-memory computing array. By configuring each sub-array with an independent input channel, and inputting the sparse data participating in the computation after bit-level non-zero aggregation into the corresponding in-memory computing sub-array for operation, the amount of computation is reduced, thereby reducing the delay and power consumption during computation.

[0017] In analogy to the concept of "word" of memory, the present application newly defines the concept of "compute word", the definition of which is that all input channels of the compute-in-memory array that participate in the computation simultaneously are collectively referred to as a compute word. For example, if a compute-in-memory array has 4 rows of subarrays, each of which inputs 1 bit at the same time for bit-by-bit serial computation, then a compute word includes 4 bits.

[0018] To make the technical solutions of the present application clearer, the embodiments of the present application are described in detail below in combination with the drawings.

[0019] Figure 1 A flowchart of a method for improving the sparse computation efficiency of a compute-in-memory array is provided for the embodiments of the present application, Figure 2 A schematic diagram of a compute-in-memory array with independent input channels is used for the embodiments of the present application, Figure 3 A computation schematic diagram for exchanging input data of different compute words in the same input channel in the embodiments of the present application is shown as follows, Figure 1 The method of the present embodiment includes the following steps:

[0020] Step 1: Preprocessing the data to be involved in the computation, aggregating the non-zero values at the bit level, and configuring the address of each row of subarrays of the compute-in-memory array according to the data exchange condition in the non-zero value aggregation.

[0021] In order to enable the compute-in-memory array to efficiently compute model data with high sparsity, the present embodiment configures each subarray of the compute-in-memory array to have an independent input channel, Figure 2 As shown in the figure, each subarray of the compute-in-memory array in the present embodiment has an independent input channel, Figure 2 As shown in the figure, the compute-in-memory array developed based on SRAM is taken as an example, but the scheme proposed by the present application is not limited to this. The so-called input channel refers to the row of compute-in-memory. When computing, each row of the compute-in-memory array will input different input data. When the storage-to-computation ratio of the compute-in-memory array is greater than 1, each subarray will contain multiple addresses A

a:0

[0022] By configuring independent addresses for input channels, additional flexibility can be introduced into in-memory computing arrays. Due to the independent nature of the input channels, data from multiple input channels can be multiplied with different weights stored at different addresses within the subarray. This design provides an opportunity to leverage sparsity; that is, within the same input channel, exchanging input data in different computation words can be done by changing the address selection signal of that input channel to ensure the correctness of the computation result. Figure 3 As shown, by adjusting the input data to aggregate non-zero data, during calculation, the corresponding subarray address can be controlled to ensure that the corresponding weights participate in the multiplication operation.

[0023] Step 2: During calculation, the calculation data after non-zero value aggregation is used to perform operations on the data stored in the corresponding subarray in the in-memory calculation array.

[0024] The present invention achieves the independence of input channels in the above steps, and makes it possible to reduce the amount of computation by using non-zero value aggregation in the computational mapping.

[0025] The following example illustrates the bit-level non-zero value aggregation method. Figure 4 This is a schematic diagram of a bit-level non-zero value aggregation method provided in an embodiment of the present invention. Figure 5 for Figure 4 A schematic diagram illustrating the input and calculation of data without non-zero value aggregation. Figure 6 This is a schematic diagram illustrating the input and calculation of data after aggregation of non-zero values. For example... Figures 4-6 As shown in the illustration, this only illustrates an example with two input groups; the invention is not limited to this, and the method is applicable even with multiple groups. Here, we take an in-memory computation array with four subarrays, each row receiving 1 bit of input data simultaneously. For such an array, processing INT8 calculations requires the input data to be divided into eight 1-bit segments, input over eight clock cycles. Assume the input data to be processed consists of 16 INT8 data points, all belonging to the same row of the input matrix. These data need to be divided into 16 computational words: {1, 0, 0, 0}, {1, 0, 1, 0}, {0, 0, 1, 0}, {1, 0, 1, 0}, {1, 0, 0, 0}, {0, 1, 0, 0}, {1, 0, 0, 0}; {0, 1, 0, 0}, {0, 1, 0, 0}, {0, 0, 0, 1}, {0, 0, 0, 1}, {0, 0, 0, 1}, {0, 1, 0, 1}, {0, 1, 0, 0}, with each group consisting of 8 computational words. Without optimization, such as... Figure 5 As shown, it takes 16 clock cycles to complete the calculation.

[0026] In order to reduce the calculation time, the input data is aggregated by non-zero value, and the aggregation method is as follows: by exchanging the same position bits of the data belonging to the same input channel in the same sequence number calculation word of different groups, the non-zero value is aggregated to a specific calculation word. That is, the calculation word to be calculated can be changed to {1, 1, 0, 0}, {1, 1, 1, 0}, {0, 1, 1, 0}, {1, 0, 1, 1}, {1, 0, 0, 1}, {0, 1, 0, 1}, {1, 1, 0, 1}, {1, 1, 0, 0}; {0, 0, 0, 0}, {0, 0, 0, 0}, {0, 0, 0, 0}, {0, 0, 0, 0}, {0, 0, 0, 0}, {0, 0, 0, 0}, {0, 0, 0, 0}, {0, 0, 0, 0}. In this way, 8 all-zero calculation words are generated, and only 8 non-zero calculation words are left, and the calculation can be completed in 8 clock cycles, as shown in the following table. Figure 6

[0027] The embodiment of the present application aggregates the data to be involved in the calculation at the bit level by preprocessing the data to be involved in the calculation, and configures the address of each row subarray of the in-memory calculation array according to the data exchange condition in the non-zero value aggregation, so that each input channel can transmit an independent subarray address signal, and the flexibility of the in-memory calculation array calculation according to the sparsity is realized; by aggregating the data with high sparsity, the scheme of reducing the in-memory calculation array calculation amount without affecting the calculation result is realized, especially when the load sparsity is high enough, the all-zero calculation word is easily generated, and such calculation word can be skipped for calculation, thereby reducing the calculation delay and power consumption. Therefore, the present application can improve the support of the in-memory calculation array to the sparsity, reduce the delay when processing the sparse calculation, reduce the calculation power consumption, and finally improve the efficiency of the in-memory calculation array when processing the sparse calculation.

[0028] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.​

Claims

1. A method for improving sparse computation efficiency of an in-memory computing array, comprising: The application comprises the following steps: Preprocessing data to be involved in calculation, non-zero value aggregation at bit level, and configuring the address of each row subarray of the in-memory computing array according to the data exchange condition in non-zero value aggregation; During calculation, the data after non-zero value aggregation is operated with the data stored in the corresponding subarray in the in-memory computing array.

2. The method of claim 1, wherein, The non-zero value aggregation is specifically performed by exchanging data between different calculation words in the same input channel to aggregate non-zero values to specific calculation words, which are input data of all input channels of the in-memory computing array participating in calculation at the same time.

3. The method according to claim 1 or 2, characterized in that, The in-memory computing array adopts SRAM.

Images