A reconfigurable approximate multiply-accumulate unit based on neural network activation distribution

Abstract

The application discloses a reconfigurable approximate multiply-accumulate unit based on a neural network activation distribution, and specifically comprises a multiplication module, an encoder, a partial product generator, a first adder, an addition module and a second adder.The encoder performs radix-8 booth coding on an activation input and a weight input of a neural network to generate corresponding partial product selection signals.The partial product generator generates partial products according to the partial product selection signals.The first adder accumulates the partial products in a top-down order to obtain a multiplication result.The addition module comprises a classifier, a compressor and a second adder.The classifier classifies high bits of the multiplication result into large numbers and small numbers.The compressor accurately compresses the large numbers and approximately compresses the small numbers.The second adder adds the compressed results to obtain a convolution result.The application effectively reduces the power consumption and the delay of a calculation unit in a neural network without losing too much accuracy, can dynamically adapt to different network types, and has good practical application value.

Description

Technical Field

[0001] This invention relates to the field of MAC (Multiplicative Accumulation) units, and in particular to a reconfigurable approximate multiplicative accumulation unit based on the activation distribution of a neural network. Background Technology

[0002] The high computational demands of neural networks pose a significant performance bottleneck and critical power consumption problem for embedded systems. The core of the inference process in neural networks is the MAC (multiply-accumulate) operation, and advanced neural networks integrate thousands of MACs, resulting in enormous energy consumption. Most previous approximate MACs were designed for unsigned multipliers, thus exhibiting low accuracy when computing signed or unevenly distributed data.

[0003] The weights of neural networks all follow a Gaussian distribution, while the activations follow a Gaussian-like distribution. Prior to this, some works considered the input weight distribution but not the activation distribution. Other works considered the activation distribution but did not integrate it with the bootstrap multiplier and the approximate compression properties.

[0004] The radix-8 booth multiplier generates fewer partial products than traditional signed multipliers, resulting in less latency. Furthermore, the generation of even-numbered partial products in the radix-8 booth multiplier can be achieved through bit shifting. However, compared to ordinary signed multipliers, the radix-8 booth multiplier generates odd-numbered partial products in its partial product generator, leading to higher power consumption. Summary of the Invention

[0005] Purpose of the invention: To provide a reconfigurable approximate multiply-accumulate unit based on neural network activation distribution to solve the above-mentioned problems existing in the prior art.

[0006] Technical solution: A reconfigurable approximate multiply-accumulate unit based on neural network activation distribution, comprising: The multiplication module includes: The encoder performs radix-8 booth encoding on the activation and weight inputs of the neural network to generate the corresponding partial product selection signal; A partial product generator generates a partial product based on a partial product selection signal. The first adder accumulates the partial products in top-to-bottom order to obtain the multiplication result; The addition module includes: The classifier classifies the multiplication result based on its high-order bits. If the high-order bits of the multiplication result are not all zeros, it is classified as a large number; if the high-order bits of the multiplication result are all zeros, it is classified as a small number. A compressor that precisely compresses large numbers and approximates the compression of small numbers; The second adder adds the compressed results together to obtain the convolution result.

[0007] According to one aspect of this application, the partial product generator includes an exact partial product generator and an approximate partial product generator, specifically: When all the high-order bits of the input activation are 0, the approximate partial product generator uses approximate encoding to generate the partial products of the last few rows with higher weights; when the high-order bits of the input activation are not all 0, the exact partial product generator uses exact encoding to generate the partial products of the last few rows with higher weights, and uses approximate encoding to generate the partial products of the first few rows with lower weights.

[0008] According to one aspect of this application, the compressor includes: 3-2 compressors for precise compression; and 4-2 compressors for approximate compression.

[0009] According to one aspect of this application, the second adder is a carry-lookahead adder.

[0010] According to one aspect of this application, the activation input and weight input are also represented using low-bit-width integers or fixed-point numbers.

[0011] According to one aspect of this application, it also includes a generative adversarial network or a variational autoencoder, specifically using a generative adversarial network or a variational autoencoder to post-process the convolution result for enhancing visual effects.

[0012] Beneficial effects: This invention effectively reduces the power consumption and latency of computing units in neural networks without sacrificing too much accuracy, and can dynamically adapt to different network types, thus having good practical application value. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the weight distribution in a convolutional neural network.

[0014] Figure 2 This is a schematic diagram of the activation distribution of a convolutional neural network.

[0015] Figure 3 This is a schematic diagram of the multiplier encoding of the present invention.

[0016] Figure 4 This is a schematic diagram of the multiplier structure of the present invention.

[0017] Figure 5 This is a schematic diagram of the adder structure of the present invention.

[0018] Figure 6 This is a schematic diagram of the MAC unit structure of the present invention.

[0019] Figure 7 This is a schematic diagram illustrating the formula for calculating the inference accuracy of a convolutional neural network.

[0020] Figure 8 This is a schematic diagram illustrating the inference accuracy of the present invention tested under different networks.

[0021] Figure 9 A schematic diagram comparing the simulation power consumption and area of ​​this invention with that of a precise multiply-accumulate unit. Detailed Implementation

[0022] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0023] like Figure 1 and Figure 2 The diagram shows the weights and activation distributions of a common 16-bit quantized neural network. It's easy to see that the weights in the neural network follow a Gaussian distribution; the activation values ​​are all positive and also follow a Gaussian-like distribution. Therefore, the highest bit of all activation inputs is 0, and the probability of the highest 3 bits being 1 is less than 20%. This invention addresses this characteristic by improving upon traditional radix-8 booth multipliers and adders. Traditional radix-8 booth multipliers generate a corresponding number of partial products through an encoding table, forming a partial product tree from top to bottom, and finally accumulating them.

[0024] In this invention, a multiplication module and an addition module are specifically included, wherein the multiplication module includes: The encoder performs radix-8 booth encoding on the activation and weight inputs of the neural network to generate the corresponding partial product selection signal; A partial product generator generates a partial product based on a partial product selection signal. The first adder accumulates the partial products in top-to-bottom order to obtain the multiplication result.

[0025] The partial product generator includes an exact partial product generator and an approximate partial product generator. Specifically, when all the high-order bits of the input activation are 0, the approximate partial product generator uses an approximate encoding method to generate the partial products of the last few rows with higher weights; when the high-order bits of the input activation are not all 0, the exact partial product generator uses an exact encoding method to generate the partial products of the last few rows with higher weights, and uses an approximate encoding method to generate the partial products of the first few rows with lower weights.

[0026] In the multiplication module, the structure of the multiplier is adjusted according to the activation input distribution of the neural network. The neural network can tolerate small errors. Therefore, this invention first determines the relative size of the activation by high bits, and then decides whether to generate a high-weight partial product in an approximate or precise manner. This can reduce power consumption while ensuring high accuracy.

[0027] The multiplication module of this invention fully considers the activation input distribution characteristics of neural networks. It generates corresponding multiplier units based on the activation input features of the neural network. The multiplier module includes an encoder, a partial product generator, and an adder. The partial product generator includes a reconfigurable precise partial product generator and an approximate partial product generator. The partial product generator can dynamically adjust the number of rows of the approximate partial product based on the activation magnitude of the input. Since approximating the first few rows of partial products in the precise radix-8 booth multiplier has almost no impact on the final inference accuracy, this invention mainly considers the last few rows of partial products with higher weights in the radix-8 booth multiplier. Furthermore, it correlates the magnitude of the input activation with the approximation of the last few rows of partial products in the partial product tree. When the input activation is small, all high-order bits of the activation are 0, allowing the last few rows of partial products with higher weights to be generated using approximate encoding. When the input activation is large, the last few rows of partial products with higher weights are generated using precise encoding, while the first few rows of partial products with lower weights are generated using approximate encoding.

[0028] The multiplication module of this invention can also flexibly adjust the approximate or precise working mode according to different networks or different activation inputs, so it can adapt to networks of any size at the same time.

[0029] The addition module includes: The classifier classifies the multiplication result based on its high-order bits. If the high-order bits of the multiplication result are not all zeros, it is classified as a large number; if the high-order bits of the multiplication result are all zeros, it is classified as a small number. A compressor that performs precise compression on large numbers and approximate compression on small numbers; the compressor includes: 3-2 compressors for precise compression; and 4-2 compressors for approximate compression.

[0030] The carry-lookahead adder adds the compressed results together to obtain the convolution result.

[0031] This invention incorporates a 3-2 compressor (full adder) and an approximate 4-2 compressor in the addition module. The multiplication results are categorized based on their size: large numbers are compressed using a 3-2 method, and small numbers are compressed using a 4-2 method. Finally, the compressed results are combined into the final convolution result using a carry-lookahead adder. This achieves a better balance between power consumption, latency, and accuracy.

[0032] Furthermore, it includes using low-bit-width integers or fixed-point numbers to represent the activation and weight inputs. Using 8-bit integers or fixed-point numbers to represent the activation and weight inputs can save 75% of storage space and computing resources compared to using 32-bit floating-point numbers. This reduces storage space and computational complexity, improves operational efficiency, and saves energy.

[0033] Further methods include using generative adversarial networks (GANs) or variational autoencoders (VAEs) to post-process the convolution results. Using a GAN to denoise the convolution results leverages adversarial learning between the generator and discriminator to generate denoised results closer to the real image. Using a VAE to denoise the convolution results utilizes the reconstruction error and prior distribution of latent variables between the encoder and decoder to generate denoised results with more structural information and texture details. This enhances visual effects and improves image quality and fidelity.

[0034] In a further embodiment, such as Figure 3 The diagram illustrates the radix-8 booth multiplier encoding of this invention. This multiplier module improves upon the classic radix-8 booth multiplier encoding, where the input D is grouped into overlapping four-bit groups (di+2, di+1, di, di-1). The corresponding partial product is selected from the set shown (±4C, ±3C±2C, ±C, or ±0). It should be noted that calculating the ±3C multiplier results in higher latency and energy consumption compared to others; therefore, this invention borrows from existing approximation methods, approximating ±3C to the closest ±2C. Assuming the activation input follows a Gaussian-like distribution, the errors introduced by this approximation method can cancel each other out.

[0035] In a further embodiment, such as Figure 4 The diagram shows a schematic of the multiplier module of this invention. Taking a 16-bit multiply-accumulate unit as an example, this multiplier module improves upon the classic radix-8 booth multiplier module. Since the activation inputs of a neural network are all integers and follow a Gaussian-like distribution, the highest bit Msb of the activation is always 0, and the probability of the second highest bit Smsb and the third highest bit Tmsb being 1 is less than 20%. Therefore, this invention, according to the encoding grouping on the right side of the diagram, determines whether to generate the partial product of the column with the largest weight in the partial product tree (i.e., the partial product of the bottom row) by using the 15th bit of the activation. The 15th and 14th bits of the activation are then ORed. If the output is 0, it indicates that the activation is a small number, which will introduce a small error. Therefore, the partial product of the fifth row of this activation can also be used. Figure 3 Approximate encoding APP-PP iGenerate; otherwise, use precise ACC-PP encoding. i Due to the fault tolerance of neural networks, this invention uses approximate encoding APP-PP in the remaining partial product generation stages. i By approximating multiplication by +3 and -3 as multiplication by +2 and -2 to reduce power consumption and latency, multiplier performance can be improved by 20%-30% while only losing about 1% of inference accuracy.

[0036] In a further embodiment, such as Figure 5 The diagram shown is a schematic of the adder module of the present invention. The adder module adopts a classification compression module, which determines whether the multiplication result is a larger or smaller number by whether the 32nd and 31st bits of the addend are zero. The larger addend is compressed using 3-2 exact compression, and the smaller addend is compressed using 4-2 approximate compression. The final 3-2 compression and 4-2 compression results are added by a carry-lookahead adder and used as the output result of the convolution.

[0037] In a further embodiment, such as Figure 6 The diagram shows the overall module of the present invention. In the approximate multiplier of the present invention, the activation and weights are first judged by the high-order values. If they are 0, the partial products are generated using the approximate method. If they are not 0, the partial products of the last few rows with larger weights are generated using the exact method. After summing the partial products, the multiplication result is obtained and then sent to the adder for high-order value judgment. If all the high-order values ​​are 0, they are sent to the 4-2 compressor with error. If not all the high-order values ​​are 0, they are sent to the 3-2 compressor without error. The compression is repeated until two rows of results are obtained (the current and carry outputs, respectively). Finally, the four rows of results output by the two compressors are added using the carry-lookahead adder cla to obtain the final result of the convolution.

[0038] like Figure 7 The figure shows the calculation formula for Accurate, the precision of neural network inference in this invention. It can be expressed as the ratio of the accuracy of inference in approximate calculation mode to the accuracy of inference in precise calculation mode. The accuracy of inference can be calculated by dividing the number of correctly inferred images by the total number of input images.

[0039] like Figure 8 and Figure 9 The diagram shows a comparison of the accuracy, power consumption, and area of ​​this invention and the precise multiply-accumulate unit. This invention uses a small-scale neural network, AlexNet, and a large-scale network, VGG16, to test a 16-bit multiplier. The approximation row count is set to two rows, meaning the last two rows are determined based on the high-order bits of the input activation to determine whether an approximation technique was used. The test dataset is CIFAR-10. It can be seen that this approximate multiply-accumulate unit achieves a 21.3% area saving and a 31.7% power saving at the cost of nearly 1% accuracy in both networks.

[0040] The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the specific details of the above embodiments. Within the scope of the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all fall within the protection scope of the present invention.

Claims

1. A reconfigurable approximate multiply-accumulate unit based on neural network activation distribution, characterized in that, include: The multiplication module includes: The encoder performs radix-8 booth encoding on the activation and weight inputs of the neural network to generate the corresponding partial product selection signal; A partial product generator generates a partial product based on a partial product selection signal. The first adder accumulates the partial products in top-to-bottom order to obtain the multiplication result; The addition module includes: The classifier classifies the multiplication result based on its high-order bits. If the high-order bits of the multiplication result are not all zeros, it is classified as a large number; if the high-order bits of the multiplication result are all zeros, it is classified as a small number. A compressor that precisely compresses large numbers and approximates the compression of small numbers; The second adder sums the compressed results to obtain the convolution result; the partial product generator includes an exact partial product generator and an approximate partial product generator, specifically: When all the high-order bits of the input activation are 0, the approximate partial product generator uses approximate encoding to generate the partial products of the last few rows with higher weights; when the high-order bits of the input activation are not all 0, the exact partial product generator uses exact encoding to generate the partial products of the last few rows with higher weights, and uses approximate encoding to generate the partial products of the first few rows with lower weights.

2. The reconfigurable approximate multiply-accumulate unit based on neural network activation distribution according to claim 1, characterized in that, The compressor includes: 3-2 compressors for precise compression; and 4-2 compressors for approximate compression.

3. The reconfigurable approximate multiply-accumulate unit based on neural network activation distribution according to claim 1, characterized in that, The second adder is a carry-lookahead adder.

4. The reconfigurable approximate multiply-accumulate unit based on neural network activation distribution according to claim 1, characterized in that, It also includes using low-bit-width integers or fixed-point numbers to represent the activation input and weight input.

5. The reconfigurable approximate multiply-accumulate unit based on neural network activation distribution according to claim 1, characterized in that, It also includes generative adversarial networks or variational autoencoders, specifically using generative adversarial networks or variational autoencoders to post-process the convolution results to enhance visual effects.

Images