PE unit and PE array based on ripple counter

Abstract

The invention discloses a PE unit and a PE array based on a ripple counter, the PE unit comprises a 4b + 4b unsigned adder and a 16b ripple counter, and the working process of the PE unit comprises the following steps: obtaining the weight and activation input in a single period; adding bias to the activation; adding bias to the activation; acquiring a corresponding accumulated value according to the weight; a 16b-based ripple counter records the occurrence frequency of each weight; and subtracting the number of times of occurrence of the weight from the data subjected to pre-addition processing, and multiplying and summing with a corresponding possible value of the weight to obtain a final output. The PE array comprises a plurality of PE units as described above. According to the invention, the problems of multiplication redundancy and overhigh power consumption of the adder when the traditional INT4MAC is used for processing the overlong vector multiplication can be solved. The method can be widely applied to the field of high-resolution imaging.

Description

Technical Field

[0001] This invention relates to the field of chip cell technology, and in particular to a PE cell and PE array based on a ripple counter. Background Technology

[0002] In recent years, with the development of deep neural networks, large language models have demonstrated outstanding performance in various fields, but their deployment on edge devices still faces challenges. Due to the large number of parameters in large language models and the limited resources of edge devices, model compression has become crucial. Quantization methods have enabled large language models to be efficiently quantized into 4-bit (4-bit) form, becoming a mainstream solution for edge deployment.

[0003] However, as the precision of the multiply-accumulate (MAC) unit further decreases to 4 bits, the traditional optimization approach of improving energy efficiency by simply reducing the data bit width begins to face a bottleneck. That is, the energy efficiency improvement brought by the reduction from 8-bit MAC unit to 4-bit MAC unit is far less than the performance improvement brought by the reduction from 16-bit MAC unit to 8-bit MAC unit, especially in the case of ultra-long vectors in large language models.

[0004] While reducing the MAC (Macro-Macro-Input Matrix) precision from INT8 to INT4 can reduce energy consumption and area, the energy and area optimization benefits are not as significant as those from INT16 to INT8, especially in very long vector operations. This is mainly due to two factors: First, there is the problem of multiplication redundancy—taking the Llama2-7B model as an example, completing vector multiplication requires 4096 multiplication operations, while INT4 multiplication can only generate 256 independent data results. This means there is serious multiplication redundancy (4096...). (256), thus leading to energy waste. Secondly, in the INT4 MAC, the accumulator module accounts for as much as 60% of the power consumption. When the precision is reduced from INT8 to INT4, since the MAC output bit width remains unchanged, the register bit width in the accumulator remains the same; at the same time, the multiplier output bit width is reduced from 16 bits to 8 bits, so the adder in the accumulator changes from 16b + 24b to 8b + 24b. This shows that in the process of INT8 to INT4, the accumulator module in the MAC only adjusts the bit width of one input port of the adder, which means that the optimization of the precision of simple MAC operations is relatively limited. Summary of the Invention

[0005] In view of this, in order to solve the problems of multiplication redundancy and excessive power consumption of adders in traditional INT4 MAC when processing ultra-long vector multiplication, the present invention proposes a PE unit based on a ripple counter, which replaces a 4b+20b unsigned adder with a 4b+4b unsigned adder and a 16b ripple counter. The working process is as follows.

[0006] In a single cycle, the PE unit retrieves the weight W and activation X, which have been quantized into 4 bits, from memory;

[0007] Adding a bias (+8) to X yields , Always greater than 0; Based on the weight W, one of the 16 accumulated values ​​stored in cell PE is accumulated, with an accumulation intensity of . The accumulation operation is implemented by a 4b+4b unsigned adder and a 16b ripple counter; Among them, the 16 accumulated values ​​in the PE unit correspond to the 16 possible values ​​of the weight W.

[0008] Use 16 16-bit ripple counters to record the number of occurrences of each weight W; Subtract the occurrence count of each of the 16 weights from the 16 cumulative values, then multiply each of the 16 cumulative values ​​by the corresponding possible weight values ​​and sum them to obtain the final output.

[0009] That is, each accumulated value corresponds to a weight count and a possible weight value. After calculating the corresponding values ​​one by one, the results are summed.

[0010] In some embodiments, to reduce wasted area, the design is further expanded based on the PE unit described above, and includes: Before X is input into the adder, X is preprocessed again according to the range of w values, thereby compressing the original range of w values ​​and thus reducing the number of storage units required in unit R30. Specifically: when w < 0, utilize When x is the opposite of w, x remains unchanged when w > 0.

[0011] Secondly, a PE array includes multiple PE units as described above.

[0012] Based on the above scheme, this invention provides a PE cell and PE array based on a ripple counter. The PE cell is designed by introducing a ripple counter and an improved formula. This PE cell can effectively solve the problems of multiplication redundancy and excessive adder power consumption in traditional INT4 MAC when processing very long vector multiplications. Furthermore, based on this PE cell, x is preprocessed again to compress the value range of W, thereby reducing storage units and saving area overhead. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the structure of unit S16 in specific embodiment 1 of the present invention; Figure 2This is a schematic diagram of a traditional 4b MAC structure; Figure 3 This is a schematic diagram of the ripple counter structure; Figure 4 This is a schematic diagram illustrating the process of splitting a 4b + 20b unsigned adder into a 4b + 4b unsigned adder and a 16b counter in a specific embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of the R30 PE unit in specific embodiment 2 of the present invention; Figure 6 This is a schematic diagram of the structure of unit R16 in specific embodiment 3 of the present invention; Figure 7 This is a schematic diagram of the multiply-accumulate unit; Figure 8 This is a schematic diagram of the matrix multiplication circuit implemented by unit S16 in a specific embodiment of the present invention; Figure 9 This is a schematic diagram of the Converter circuit corresponding to R30 and R16 in a specific embodiment of the present invention; Figure 10 It is a 4-bit matrix multiplication pulsating array composed of R16 units in a specific embodiment of the present invention; Figure 11 This is a schematic diagram illustrating the application scenario of the R16 systolic array of the present invention in a coprocessor. Detailed Implementation

[0014] To address the conflict between ultra-long vector multiplication in large language model inference and INT4 multiplication, which only generates 256 independent results, statistical methods can be used to reduce the number of multiplications in long vector multiplication. The specific formula is as follows: As can be seen from the formula, it can be done in advance based on... right By performing pre-addition processing, the number of multiplication operations required was reduced from N to only 16. Statistically speaking, this reduces the number of multiplication operations needed. The preprocessing performed can be seen as a process of finding The distribution of the distribution is considered, and the subsequent 16 multiplications can be viewed as a summation of this distribution. In this invention, this is called the statistical weighted distribution method. However, there is currently no MAC circuit specifically designed based on this idea; therefore, based on this idea, this invention designs several PE units.

[0015] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0016] It should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0017] It should be understood that the terms "system," "apparatus," "unit," and / or "module" used in this application are a method of distinguishing different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0018] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "a," and / or "the" are not specifically singular and may include the plural. Generally, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements. An element defined by the phrase "comprising an..." does not exclude the presence of other identical elements in the process, method, product, or apparatus that includes the element.

[0019] In the description of the embodiments of this application, "a plurality of" refers to two or more. The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0020] Furthermore, flowcharts are used in this application to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, the steps can be processed in reverse order or simultaneously. Additionally, other operations can be added to these processes, or one or more steps can be removed from them. Specific Implementation Example 1: First, by Therefore, to perform matrix multiplication, we need to first... To perform pre-processing, a basic PE cell is designed for this step, based on the SRAM S16 cell: Cell S16 is a simple circuit consisting of a 16-bit SRAM and a 4-bit + 20-bit signed adder, such as... Figure 1 As shown. The SRAM depth is 16 for ease of use. Addressing is performed. The circuit works as follows: First, the single-cycle input is addressed... and , we through As the input address of the SRAM, make the SRAM output similar to the current... Corresponding cumulative value Then, this cumulative value and Add to complete the expression. Pre-processing.

[0022] Compared to the S16 unit Figure 2 For the traditional INT4 MAC shown, the S16 unit consumes less power. However, this S16 unit design is not optimal because it contains a 4b+20b signed adder, which causes relatively high power consumption. Therefore, a ripple counter is used to optimize the 4b+20b signed adder.

[0023] The ripple counter is an asynchronous circuit, consisting of multiple D flip-flops connected in series, such as... Figure 3 As shown. When an external pulse arrives, the first flip-flop (Q0) toggles according to its state. However, the second flip-flop (Q1) is only triggered when a rising edge occurs at the inverted output of Q0. The inverted output of Q1 then triggers the third flip-flop. The ripple counter has a partial switching characteristic, meaning that during the counting process, the least significant bit flips once with each external pulse, the next least significant bit flips once every two pulses, and so on. The most significant bit flips the least frequently (every 2...). n (Once per cycle, n is the number of bits). Therefore, compared to a synchronous counter where all bits are connected to the same clock (where the clock inputs of all flip-flops switch on every valid clock edge), in a ripple counter, only the clock inputs of the flip-flop that needs to be toggled and its subsequent flip-flops switch. The clock network and internal nodes of the high-order flip-flops that have not received a valid clock edge remain relatively stationary, theoretically reducing dynamic power consumption and thus achieving low power consumption.

[0024] To introduce a ripple counter into the PE cell, two premises need to be understood: First, the ripple counter can only perform +1 operations, while the adder in the S16 cell is a 4-bit + 20-bit signed adder, therefore the ripple counter cannot be directly replaced by the adder in the S16 cell. Second, the ripple counter is composed of flip-flops, thus it has the ability to store data; therefore, the ripple counter can be considered to replace the SRAM in the S16 cell. Based on these two premises, an R30 cell based on a ripple counter was designed, and the specific approach is as follows. Specific Implementation Example 2: A PE unit includes a 4b+4b unsigned adder and a 16b ripple counter.

[0026] pass Figure 4 In this way, a 4b+20b unsigned adder is replaced with a 4b+4b unsigned adder and a 16b counter. This 16b counter is used to counter the overflow bits of the 4b+4b, and can then be implemented using a 16b ripple counter.

[0027] The working process of this PE unit is as follows: Obtain the W and X values ​​of a single cycle input, where X is an INT4 value ranging from -8 to 7.

[0028] First, add a bias (+8) to X to get X'. At this time, the value range of X' is 0~15. Then, input X' into the adder for calculation. Furthermore, since the ripple counter itself has a storage function, the SRAM in the S16 unit is further replaced by the ripple counter. Subsequently, the number of times each w is biased is recorded using a ripple counter. Finally, after the pre-processing is completed, these biases are subtracted.

[0029] The mathematical expression for this process is as follows: In summary, the PE unit designed based on the ripple counter in this embodiment, such as... Figure 5 As shown. Compared to Figure 1 The 4b + 20b signed adder in unit S16 is replaced by a 4b + 4b unsigned adder, and the memory unit is replaced by a ripple counter and a register set. Specific Implementation Example 3: To further reduce wasted space, an R16 unit circuit was designed based on the PE unit in specific embodiment 2, as follows: Figure 6 As shown.

[0031] The number of storage units in units S16 and R30 depends on the value range of w. The number of storage units in unit S16 is equal to the value range of w, while the number of storage units in unit R30 is equal to twice the value range of w. Since the value range of w is [-8, 7], unit S16 needs to store 16 data points, and unit R30 needs to store 32 data points. Therefore, reducing the number of storage units essentially means compressing the value range of w. Based on this process, before x and w are input into unit R30, x is preprocessed again according to the value range of w, thus compressing the original value range of w from [-8, 7] to [0, 8], thereby reducing the number of storage units required in unit R30. The mathematical essence of this idea is that when w < 0, using... When x is the opposite of w, x remains unchanged when w > 0.

[0032] Finally, the R30 cell was designed by introducing a ripple counter and an improved formula. This cell effectively solves the problems of multiplication redundancy and excessive power consumption of adders in traditional INT4 MAC when processing very long vector multiplications. Furthermore, the R16 cell, based on the R30 cell, reduces the number of memory cells by almost half, thereby saving area overhead.

[0033] Based on the above embodiments, three PE units are proposed for pre-addition processing: S16 unit, R30 unit, and R16 unit. These units are assumed to be equivalent to MAC units, but after pre-addition processing of x, the pre-addition result still needs to be processed. The following content describes the processing circuit for the pre-addition result and explains why the overhead of this circuit can be ignored.

[0034] for Figure 1 Unit S16 requires 16 more multiply-accumulate operations, therefore an additional multiply-accumulate unit needs to be added, as detailed below. Figure 7 As shown. Figure 7 The QREG variable stores all the values ​​of w, so QREG is similar to an array [-8, -7, -6, -5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5, 6, 8]. It's worth noting that... Figure 7 The circuitry within, except for QREG, is identical in structure to a traditional MAC. This means that if... Figure 1 add Figure 7 Implement a 4b ultralong vector multiplication in this way ( The PE unit of the MAC will have one more unit than the traditional MAC. Figure 1 The S16 unit.

[0035] Analysis of the formulas and circuits shows that traditional MAC requires N executions, or N clock cycles, to perform ultra-long vector multiplication; while here... Figure 1 + Figure 7 When the circuit performs very long vector multiplication, Figure 7 The multiply-accumulate circuit in the code only needs to perform 16 operations, or 16 clock cycles. The actual number of operations required is N. Figure 1 The circuit in, i.e. Figure 1 The circuit in the circuit needs to execute N clock cycles. Because... Figure 1 and Figure 7 The two circuits have different execution cycles, which gives us the possibility of reuse, namely, the circuit with the shorter single execution cycle ( Figure 7 It can be multiplexed by multiple circuits with long execution cycles. Figure 1 ),like Figure 8 As shown.

[0036] exist Figure 8 In this context, the multiplexer multiply-accumulate unit is called a Converter module. Through multiplexing, the overhead of a single Converter can be distributed across multiple S16 units. The number of S16 units corresponding to one multiply-accumulate unit depends on the design requirements. In the most extreme case, there can be... One S16 unit corresponds to one S16 unit, and the Converter overhead amortized to each S16 unit is: The overhead of a single Converter. Since N >> 256 in long vector multiplication in large language models, the overhead of a single S16 unit of the Converter is negligible.

[0037] The converter circuits corresponding to units R30 and R16 can be derived from... Figure 9 Similar to the Converter corresponding to unit S16, the Converter corresponding to unit R30 (or R16) can be reused by multiple units R30 (or R16), thus the overhead of this Converter in large-scale multiplication operations can be ignored. However, it is also necessary to consider that the Converter circuits of R30 and R16 also need to complete... This step involves a relatively complex circuit structure.

[0038] Based on the above scheme, the present invention also conducted comparative experiments, and the relevant experimental data are as follows: To demonstrate the superiority of the statistical weight distribution method in ultra-long vector multiplication, and also to prove the advantages of the R30 and R16 units designed in this invention, tests were conducted on INT4 MAC, S16 PE, S30 PE, and R16PE with two random vectors of dimension 4096 as inputs.

[0039] As shown in Table 1, the areas of the S16 PE, R30 PE, and R16 PE are all larger than that of the INT4 MAC, while their power consumption is lower than that of the INT4 MAC. This indicates that the statistical weight distribution method is essentially a trade-off between area and power consumption. The data suggests that the statistical weight distribution method offers significant power savings when calculating very long vector multiplications.

[0040] Comparing the various units, the lower power consumption of the S16 PE unit compared to the INT4 MAC demonstrates that reducing the number of multiplications in ultra-long vector multiplication does indeed bring about a certain improvement in power consumption. However, the power consumption of R30 PE and R16 PE is even lower than that of S16 PE, indicating that our invention can bring even better power consumption improvements. Meanwhile, the area of ​​R30 PE is nearly twice that of S16 PE, which indeed indicates that the R30 PE unit suffers from excessive area overhead. Conversely, the area of ​​R16 PE is close to that of S16 PE, demonstrating that the R16 PE unit, an optimization based on R30 PE, offers significant area gains. Therefore, whether considering power consumption or area, the R16 unit is a good solution to the multiplication redundancy and excessive adder power consumption issues that arise in traditional INT4 MAC when processing ultra-long vector multiplication.

[0041] Table 1. Power consumption and area data for the three PE cells. This invention addresses two problems encountered by traditional INT4 MAC-based systolic arrays when processing very long vector multiplications: multiplication redundancy and excessive adder power consumption, thereby saving energy consumed in the systolic array. This invention designs three PE units based on a statistical weight distribution method. The S16 unit is the most direct solution for implementing the statistical weight distribution method, while the S30 and R16 units are improvements on the S16 unit design. The S30 and R16 units are inventions, and their circuit structures have been described above. The application of these two PE units in the system will be given below.

[0042] Unit R16 passes through Figure 10This method replaces the traditional systolic array of INT4 MAC in the system, and the R30 unit can also replace the systolic array in the system using this method. Compared to the traditional INT4 MAC systolic array, this method of constructing a systolic array uses the preprocessing of x as the actual operation in the systolic array. In very long vector multiplication, the execution time of the Converter is very short, and since it is multiplexed by the PE unit, its power consumption per PE unit is almost negligible. Since the power consumption of the R30 unit and R16 unit is lower than that of the INT4 MAC, the power consumption of the array constructed by them is lower than that of the systolic array constructed by INT4 MAC. Figure 11 It is a coprocessor built on an R16-cell systolic array, and a coprocessor based on an R30-cell systolic array can also be built in this way.

[0043] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A ripple counter based PE unit, characterized by, The 4b+4b unsigned adder and 16b ripple counter work as follows: Obtain the weight and activation of the input of a single cycle; Add a bias to the activation, and the activation after adding the bias is always greater than 0; Perform an accumulation operation based on the 4b+4b unsigned adder and the 16b ripple counter: accumulate one of the 16 accumulated values stored in the PE unit according to the weight with the bias-added activation as the accumulation strength; Record the number of occurrences of each of the weights based on the 16b ripple counter; Subtract the 16 accumulated values from the 16 weight occurrence numbers respectively, and then multiply each by the corresponding 16 possible weight values and sum them up to obtain the final output.

2. The PE unit based on ripple counter according to claim 1, wherein, Further comprising: Preprocess the activation according to the value range of the weight.

3. The PE unit based on ripple counter of claim 1, wherein, The step of preprocessing the activation according to the value range of the weight specifically comprises: When the weight is less than 0, take the opposite of the activation; When the weight is greater than 0, keep the activation unchanged.

4. The PE unit based on ripple counter of claim 1, wherein, The expression of the final output is represented as follows: where y denotes the final output, j denotes the activation index, w ' j denotes the activation at the j index after biasing, and w is the weight.

5. A PE array, characterized in that, The PE unit comprises a plurality of PE units as claimed in claim 1.

6. A coprocessor, comprising: The PE array comprises a PE unit as claimed in claim 5.

Images