Low power approximate multiplier for convolutional neural networks

Abstract

The application provides a low-power approximate multiplier for a convolutional neural network, comprising: an input preprocessing circuit, a weight decoding circuit, an approximate multiplication circuit based on partial product speculation and an output post-processing circuit; the input preprocessing circuit pre-processes the input of the multiplier, and the processed result is connected with the approximate multiplication circuit based on partial product speculation and the output post-processing circuit respectively; the weight decoding circuit decodes the input weight, and the decoding result is connected with the approximate multiplication circuit based on partial product speculation and the output post-processing circuit respectively; the approximate multiplication circuit based on partial product speculation realizes approximate multiplication operation on the input, and the 16-bit output is connected with the output post-processing circuit. The application uses the correlation between adjacent partial products of the multiplier, uses the higher weight partial product to speculate the overall value of the lower weight partial product, realizes the generation and compression process of the approximate partial product, and greatly reduces the power consumption, area and delay of the multiplier.

Description

Technical Field

[0001] This invention relates to the field of convolutional neural networks and approximate multipliers, and more specifically, to a low-power approximate multiplier for convolutional neural networks. Background Technology

[0002] Existing approximate multiplier design methods can be categorized into three typical types based on different approximation stages: approximate partial product generation, approximate partial product compression, and approximate logarithmic multiplication. Among these three types of approximate multipliers, there is currently no method that uses partial product prediction to generate approximate partial products. The closest technical point to this invention is to approximate the lower-weighted partial products as constants. For example, in the literature [F. Sabbetzadeh, MH Moaiyeri and M. Ahmadinejad, "An Ultra-Efficient Approximate Multiplier With Error Compensation for Error-Resilient Applications," in IEEE Transactions on Circuits and Systems II: Express Briefs, vol. 70, no. 2, pp. 776-780, Feb. 2023.], it is mentioned that to achieve ultra-high hardware efficiency, a portion of the partial product array with lower weights is fixed as a constant, while the middle portion uses a four-input OR gate structure to compress the partial product array. Although this design method has low power consumption, it leads to higher errors when the input values ​​are small. Therefore, this design approach is only suitable for simple neural networks, and its accuracy is low in complex neural network applications.

[0003] A similar technical aspect to this invention includes using a lead-1 detection circuit to preprocess the multiplier input. For example, in the literature [S. Hashemi, R.I. Bahar, and S. Reda, "DRUM: A dynamic range unbiased multiplier for approximate applications," in IEEE / ACM International Conference on Computer-Aided Design, pp. 418-425, 2015.], a lead-1 detection circuit is used to detect the position of the lead-1 in the input operand, then the lead-1 and several bits to its right are selected, and the least significant bit is compensated with 1 as the input to a lower-width accurate multiplier. Finally, the output of the accurate multiplier is shifted right by a certain number of bits to obtain an approximate multiplication result. This design ensures that the multiplier error is unbiased, but its average absolute error is much larger than that of this design. In addition, its hardware overhead is large when high accuracy is required.

[0004] Patent document CN111221499A discloses an approximate multiplier and calculation method based on approximate 6-2 and 4-2 compressors. This approximate multiplier includes a partial product generation module, a partial product tree compression module, and a carry adder module. The partial product generation module is an AND gate array, where each bit of the multiplier and multiplicand is ANDed to obtain the corresponding partial product. The partial product tree compression module includes a precise compression unit, an approximate compression unit, and a truncated unit. The carry propagation adder is used to add the output signals of the precise compression unit and the approximate compression unit, and the resulting output signal is combined with the output signal of the truncated unit to obtain the multiplication result. However, this patent cannot solve the technical problem of high errors when the input value is small, nor can it meet the requirements of the convolutional neural network targeted by this invention. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this invention is to provide a low-power approximate multiplier for convolutional neural networks.

[0006] The low-power approximate multiplier for convolutional neural networks provided by the present invention includes: an input preprocessing circuit, a weight decoding circuit, an approximate multiplication circuit based on partial product inference, and an output postprocessing circuit.

[0007] The input preprocessing circuit preprocesses the input of the multiplier, and the processed result is connected to the approximate multiplication circuit based on partial product inference and the output postprocessing circuit, respectively.

[0008] The weight decoding circuit decodes the input weights, and the decoding result is connected to the approximate multiplication circuit based on partial product inference and the output post-processing circuit, respectively.

[0009] The approximate multiplication circuit based on partial product inference performs approximate multiplication on its input, and its 16-bit output is connected to the output post-processing circuit to obtain the final approximate multiplication calculation result.

[0010] Preferably, the input preprocessing circuit includes: a pilot 1 detection circuit, a left shifter, and an encoder;

[0011] The leader 1 detection circuit is used to detect the position of the leader 1 of the input 8-bit operand and input the result to the left shifter and encoder. According to the output result of the leader 1 detection circuit, the left shifter performs a shift operation on the input 8-bit operand and inputs the result to the approximate multiplication circuit based on partial product inference.

[0012] The encoder encodes the output of the pilot 1 detection circuit into a left-shifted number of bits, and the result is connected to the output post-processing circuit.

[0013] Preferably, the weight decoding circuit includes a decoder, wherein the input is a pre-encoded weight, the higher bits of which are connected to an output post-processing circuit, the decoder decodes the pre-encoded weight, and its output is connected to an approximate multiplication circuit based on partial product inference.

[0014] Preferably, the approximate multiplication circuit based on partial product inference includes: a partial product array generation circuit, a partial product compression circuit, and a 7-bit ripple carry adder;

[0015] The partial product array generation circuit includes 41 two-input AND gates, which generate 41 partial products and are connected to the partial product compression circuit.

[0016] The partial product compression circuit consists of four stages of parallel accumulation circuits, including 24 full adders, 8 half adders, and a sixth column partial product compressor. Its output of two rows of partial products is connected to a 7-bit ripple carry adder. The sixth column partial product compressor includes 5 two-input AND gates, 3 two-input OR gates, and 1 three-input OR gate. Through different logical combinations of the 6 inputs, it obtains a speculative value to replace the sum of the ungenerated partial products in the 5th column, as well as 2 carry and 2 summation signals to represent the approximate addition result of these 6 inputs. In addition, the product non-zero flag is generated by a product non-zero judgment circuit, which is independent of the other three parts. It consists of 2 eight-input OR gates and 1 two-input AND gate. The inputs of the OR gates are 8-bit signals of the input and weight, respectively. The two output signals are connected to the two-input AND gate to realize the judgment of whether both inputs of the multiplier are non-zero. If either of them is 0, the final product is 0.

[0017] Preferably, the output post-processing circuit includes: an adder and a right shifter;

[0018] The adder receives the output of the encoder and the higher bits of the weights from the input preprocessing circuit. Its output is connected to a right shifter, the other input of which is the output of an approximate multiplication circuit based on partial product inference.

[0019] Preferably, the working process is as follows: First, the input preprocessing circuit preprocesses the input data. The lead 1 detection circuit detects the position of the lead 1 in the input data. Based on the position of the lead 1, the left shifter shifts the input to the left by the corresponding number of bits to retain the lead 1 and the bits to its right, and inputs them to the approximate multiplication circuit based on partial product inference. The encoder encodes the position of the lead 1 into the number of bits to be shifted to the left, and inputs it to the output postprocessing circuit. At the same time, the weight decoder decodes the encoded weights to obtain the lead 1 and the bits to its right, and inputs them to the approximate multiplication circuit based on partial product inference. The higher bits of the weights are the number of bits to be shifted, and are connected to the output postprocessing circuit.

[0020] Preferably, different approximations of the multiplier can be achieved by setting the maximum number of bits to shift to the left;

[0021] When shifting left by a maximum of 1 bit, the bit width of shift_w is 1 bit, and the bit width of mult_w is 7 bits. shift_w = 0 means shifting left by 0 bits. At this time, the leading 1 is in the highest bit of input w, the high 6 bits of mult_w are the high 6 bits of w[7:2], and the lowest bit of mult_w is the result of the OR of the low 2 bits of w. shift_w = 1 means shifting left by 1 bit. At this time, the leading 1 is not in the highest bit of w, and mult_w is the low 7 bits of w[6:0].

[0022] When shifting left by a maximum of 2 bits, the bit width of shift_w is 2 bits, and the bit width of mult_w is 6 bits. shift_w = 00 means shifting left by 0 bits. At this time, the leading 1 is in the highest bit of input w, the high 5 bits of mult_w are w[6:2], and the lowest bit of mult_w is the result of the OR of the low 2 bits of w. shift_w = 01 means shifting left by 1 bit. At this time, the leading 1 is in the second highest bit of w[6], and mult_w is the low 6 bits of w[5:0]. shift_w = 10 means shifting left by 2 bits. At this time, the leading 1 is not in the high 2 bits of w. For the convenience of decoding, the high 5 bits of mult_w are the low 5 bits of w[4:0], and the lowest bit of mult_w is w[5]. When shift_w = 00 and 01, the hidden 1 in the highest bit of mult_w is omitted by default.

[0023] When shifting left by a maximum of 3 bits, the bit width of shift_w is 2 bits, and the bit width of mult_w is 6 bits. shift_w = 00 indicates a left shift of 0 bits, at which point the leading 1 is in the highest bit of input w, the high 5 bits of mult_w are w[6:2], and the lowest bit of mult_w is the result of the OR operation of the lower 2 bits of w. shift_w = 01 indicates a left shift of 1 bit, at which point the leading 1 is in the second highest bit of w, w[6], and mult_w is the lower 6 bits of w, w[5:0]. shift_w = 10 indicates a left shift of 2 bits. When the leading 1 is in the 5th bit of w, i.e., w[5] = 1, the high 5 bits of mult_w are the low 5 bits of w w[4:0], and the least significant bit of mult_w is 0; shift_w = 11 means shifting left by 3 bits. At this time, the leading 1 is not in the high 3 bits of w, the high 4 bits of mult_w are the low 4 bits of w w[3:0], mult_w[1] = 0, and the least significant bit of mult_w is w[4]; when shift_w = 00, 01 and 10, the hidden 1 of the highest bit is omitted in mult_w by default;

[0024] Where shift_w represents the number of bits shifted left, and mult_w represents the valid bits after encoding w.

[0025] Preferably, the partial product generation principle is as follows: each bit of one operand is ANDed with each bit of the other operand to obtain the partial product at the corresponding position. The lowest row and the lowest six columns are omitted, thus requiring 41 two-input AND gates. When neither operand is 0, the partial product is approximately (110000)2, that is, the lower four bits are 0, and the fourth and fifth bits are the product non-zero flags. The product non-zero flags are generated by two eight-input OR gates and one two-input AND gate. The input of an eight-input OR gate is an 8-bit signal of an operand, used to determine whether the operand is 0. The outputs of the two OR gates are connected to a two-input AND gate. The AND gate output is 1, indicating that the product is non-zero.

[0026] Preferably, the partial product compression principle is as follows: the partial product generated in the sixth column is approximately compressed, and the sum of the partial products in the fifth column is inferred using the partial product of the sixth column. The inference formula is as follows:

[0027] spec_flag=(pp0&pp1)|(pp1&pp2)|…|(pp n-2 &pp n-1 )

[0028] When spec_flag is 1, it indicates that at least one partial product with a value of 1 exists in the speculated column; when spec_flag is 0, it indicates that the sum of the speculated columns is very likely to be 0. Here, spec_flag is considered as a compensation value for the speculated column, and this part approximates the compression itself, as shown in the following formula:

[0029] s1 = pp0

[0030] s2 = pp3

[0031] c1 = pp1 & pp2

[0032] c1 = pp5|pp4

[0033] Therefore, the partial product compression circuit of the sixth column requires one two-input AND gate and one two-input OR gate, which means that the six partial products are approximately compressed into two carry signals (c1 and c2) and two sum signals (s1 and s2).

[0034] Preferably, for the remaining partial products of the array, the Wallace tree method is used for accurate compression, which requires a total of 24 full adders and 8 half adders. After 4 stages, the product is compressed into two rows of outputs and sent to a 7-bit ripple carry adder. The 7-bit ripple carry adder adds the two rows of results to obtain the result of an approximate multiplication circuit.

[0035] The output post-processing circuit performs post-processing on the output of the approximate multiplication circuit based on partial product inference. Since the input operands are left-shifted before the multiplication operation, the multiplication result of the approximate multiplication circuit based on partial product inference needs to be right-shifted to restore its value. The number of shifts is the sum of the left-shifted numbers of the input preprocessing circuit and the weight decoder output.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] (1) This invention proposes an 8×8 unsigned approximate multiplier with configurable precision. By utilizing the correlation between adjacent partial products of the multiplier, the overall value of the partial product with lower weight is inferred using the partial product with higher weight. This realizes the generation and compression process of the approximate partial product, which greatly reduces the power consumption, area and delay of the multiplier. In addition, an input preprocessing and weight encoding scheme is proposed to ensure the accuracy of multiplication with small operands, thereby realizing the configurable precision of the multiplier.

[0038] (2) By adopting a data preprocessing and weight encoding scheme, this invention solves the problem of poor performance of approximate multipliers when processing complex datasets, achieves compatibility with neural networks of different complexity, improves applicability, and makes them more suitable for processing complex datasets.

[0039] (3) By employing speculation and approximate compression mechanisms in the design of the approximate multiplier, especially the approximate calculation and approximate compression of the sixth column, this invention solves the problem of large hardware overhead in the partial product compression of the multiplier and significantly reduces the hardware overhead of the multiplication circuit. Attached Figure Description

[0040] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0041] Figure 1 This is a schematic diagram of the low-power approximate multiplier structure for convolutional neural networks according to the present invention;

[0042] Figure 2 (a), (b), and (c) represent different weighting schemes. Detailed Implementation

[0043] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0044] Example

[0045] like Figure 1 This invention provides an 8×8 unsigned approximate multiplier for convolutional neural networks, comprising: an input preprocessing circuit 1, a weight decoding circuit 2, an approximate multiplication circuit based on partial product prediction 3, and an output post-processing circuit 4. The input preprocessing circuit 1 preprocesses the input of the multiplier, and the processed result is connected to the approximate multiplication circuit based on partial product prediction 3 and the output post-processing circuit 4, respectively. The weight decoding circuit 2 decodes the input weights, and its decoding result is connected to the approximate multiplication circuit based on partial product prediction 3 and the output post-processing circuit 4, respectively. The approximate multiplication circuit based on partial product prediction 3 performs approximate multiplication operations on its input, and its 16-bit output is connected to the output post-processing circuit 4, thereby obtaining the final approximate multiplication calculation result.

[0046] The input preprocessing circuit 1 includes a leader 1 detection circuit, a left shifter, and an encoder. The leader 1 detection circuit is used to detect the position of the leader 1 of the input 8-bit operand and input the result to the left shifter and the encoder. According to the output result of the leader 1 detection circuit, the left shifter performs a shift operation on the input 8-bit operand and inputs the result to the approximate multiplication circuit 3 based on partial product inference. The encoder encodes the output result of the leader 1 detection circuit into the number of left shifts, and the result is connected to the output postprocessing circuit 4.

[0047] The weight decoding circuit 2 includes a decoder, wherein the input is a pre-encoded weight, the higher bits of which are connected to the output post-processing circuit 4, the decoder decodes the pre-encoded weight, and its output is connected to the approximate multiplication circuit 3 based on partial product inference.

[0048] The approximate multiplication circuit 3 based on partial product prediction includes: a partial product array generation circuit, a partial product compression circuit, and a 7-bit ripple carry adder. Specifically: the partial product array generation circuit contains 41 two-input AND gates, generating 41 partial products connected to the partial product compression circuit; the partial product compression circuit consists of four stages of parallel accumulation circuits, including 24 full adders, 8 half adders, and a sixth column partial product compressor, whose output of two rows of partial products is connected to the 7-bit ripple carry adder; the sixth column partial product compressor includes 5 two-input AND gates, 3 two-input OR gates, and 1 three-input OR gate. Through different logical combinations of the 6 inputs, a speculative value is obtained to replace the sum of the ungenerated partial products in the 5th column, along with 2 carry and 2 summation signals to represent the approximate addition result of these 6 inputs. In addition, the product non-zero flag is generated by the product non-zero judgment circuit, which is independent of the other three parts. It consists of 2 eight-input OR gates and 1 two-input AND gate. The inputs of the OR gates are the 8-bit signals of the input and the weight, respectively. The output signals of the two OR gates are connected to the two-input AND gate to realize the judgment of whether both inputs of the multiplier are non-zero. If either of them is 0, the final product is 0.

[0049] The output post-processing circuit 4 includes an adder and a right shifter. The adder receives the output of the encoder and the higher bits of the weights in the input pre-processing circuit 1. Its output is connected to the right shifter. The other input of the right shifter is the output of the approximate multiplication circuit 3 based on partial product inference.

[0050] This 8×8 unsigned approximate multiplier is designed for convolutional neural networks. Its two input operands are both 8 bits, denoted as input and weight, respectively. The input is an 8-bit unsigned binary integer, and the weight is 8-bit data that has been pre-trained, re-encoded, and stored in a memory cell. The encoding scheme of the weight is consistent with the input preprocessing scheme.

[0051] This embodiment operates through the following steps:

[0052] First, the input preprocessing circuit 1 preprocesses the input data. The leader 1 detection circuit detects the position of the leader 1 in the input data. Based on the leader 1 position, a left shifter shifts the input to the left by the corresponding number of bits to preserve the bits to the right of the leader 1. This left shifter is then fed into the approximate multiplication circuit 3, which is based on partial product inference. The encoder encodes the leader 1 position into the number of bits to be shifted left and inputs this into the output post-processing circuit 4. Simultaneously, the weight decoder decodes the encoded weights to obtain the leader 1 and the bits to its right, which are then input into the approximate multiplication circuit 3. The higher bits of the weights represent the number of bits to be shifted and are connected to the output post-processing circuit 4.

[0053] Specific weighting encoding schemes (i.e., input preprocessing schemes) are as follows: Figure 2 As shown:

[0054] Different degrees of approximation can be achieved by setting the maximum number of bits to shift to the left. Figure 2 (a), (b), and (c) in the figure give the encoding schemes when shifting left by a maximum of 1, 2, and 3 bits, respectively, where shift_w represents the number of bits to shift left and mult_w represents the effective bits after encoding.

[0055] When shifting left by a maximum of 1 bit, the bit width of shift_w is 1 bit, and the bit width of mult_w is 7 bits. shift_w = 0 means shifting left by 0 bits. At this time, the leading 1 is in the highest bit of input w, the high 6 bits of mult_w are the high 6 bits of w w[7:2], and the lowest bit of mult_w is the result of the OR of the low 2 bits of w. shift_w = 1 means shifting left by 1 bit. At this time, the leading 1 is not in the highest bit of w, and mult_w is the low 7 bits of w w[6:0].

[0056] When shifting left by a maximum of 2 bits, the bit width of shift_w is 2 bits, and the bit width of mult_w is 6 bits. shift_w = 00 means shifting left by 0 bits. At this time, the leading 1 is in the highest bit of input w, the high 5 bits of mult_w are w[6:2], and the lowest bit of mult_w is the result of the OR of the lower 2 bits of w. shift_w = 01 means shifting left by 1 bit. At this time, the leading 1 is in the second highest bit of w[6], and mult_w is the lower 6 bits of w[5:0]. shift_w = 10 means shifting left by 2 bits. At this time, the leading 1 is not in the high 2 bits of w. For decoding convenience, the high 5 bits of mult_w are the lower 5 bits of w[4:0], and the lowest bit of mult_w is w[5]. Note: When shift_w = 00 and 01, the hidden 1 in the highest bit of mult_w is omitted by default.

[0057] When shifting left by a maximum of 3 bits, the bit width of shift_w is 2 bits, and the bit width of mult_w is 6 bits. shift_w = 00 indicates a left shift of 0 bits. At this time, the leading 1 is in the highest bit of input w, the high 5 bits of mult_w are w[6:2], and the lowest bit of mult_w is the result of the OR operation of the lower 2 bits of w. shift_w = 01 indicates a left shift of 1 bit. At this time, the leading 1 is in the second highest bit of w, w[6], and mult_w is the lower 6 bits of w, w[5:0]. `hift_w = 10` means shifting left by 2 bits. At this time, the leading 1 is in the 5th bit of `w`, i.e., `w[5] = 1`, the high 5 bits of `mult_w` are the low 5 bits of `w`, `w[4:0]`, and the least significant bit of `mult_w` is 0. `shift_w = 11` means shifting left by 3 bits. At this time, the leading 1 is not in the high 3 bits of `w`. For ease of decoding, the high 4 bits of `mult_w` are the low 4 bits of `w`, `w[3:0]`, `mult_w[1] = 0`, and the least significant bit of `mult_w` is `w[4]`. Note: When `shift_w = 00`, `01`, and `10`, the hidden 1 in the highest bit of `mult_w` is omitted by default.

[0058] The input to the approximate multiplication circuit 3, based on partial product deduction, is two 8-bit operands. The principle of this circuit is as follows:

[0059] The principle of partial product generation: A partial product is obtained by ANDing each bit of one operand with each bit of the other operand. The least significant bit (the lowest row and the lowest six columns) is omitted, thus requiring 41 two-input AND gates. When neither operand is zero, this partial product is approximately (110000)², meaning the lowest four bits are 0, and the fourth and fifth bits are a non-zero product flag. The non-zero product flag is generated by two eight-input OR gates and one two-input AND gate. The input of an eight-input OR gate is an 8-bit signal of an operand, used to determine if the operand is zero. The outputs of the two OR gates are connected to a two-input AND gate; a result of 1 indicates a non-zero product.

[0060] Partial product compression principle: Approximate compression is performed on the partial products generated in the sixth column, and the sum of the partial products in the fifth column is inferred from the partial products in the sixth column. The inference formula is as follows:

[0061] spec_flag=(pp0&pp1)|(pp1&pp2)|…|(pp n-2 &pp n -1)

[0062] When spec_flag is 1, it indicates that at least one partial product with a value of 1 exists in the speculated column; when spec_flag is 0, it indicates that the sum of the speculated columns is very likely to be 0. Therefore, spec_flag is considered a compensation value for the speculated columns. This part also approximates the compression of the speculated column itself, as shown in the following formula:

[0063] s1 = pp0

[0064] s2 = pp3

[0065] c1 = pp1 & pp2

[0066] c1 = pp5|pp4

[0067] Therefore, the partial product compression circuit of the sixth column only needs one two-input AND gate and one two-input OR gate to approximately compress the six partial products into two carry signals (c1 and c2) and two summation signals (s1 and s2), which greatly reduces the hardware overhead.

[0068] For the remaining partial products of the array, due to their high weights, the Wallace tree method is used for accurate compression, requiring a total of 24 full adders and 8 half adders. After 4 stages, they are compressed into two rows of outputs to a 7-bit ripple carry adder. The ripple carry adder adds the two rows of results to obtain the result of an approximate multiplication circuit.

[0069] The output post-processing circuit 4 performs post-processing on the output of the approximate multiplication circuit 3 based on partial product inference. Since the input operands were left-shifted before the multiplication operation, the multiplication result of the approximate multiplication circuit 3 based on partial product inference needs to be right-shifted to restore its value. The number of shifts is the sum of the left-shifted numbers output by the input preprocessing circuit 1 and the weight decoder 2.

[0070] In the description of this application, it should be understood that the terms "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0071] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0072] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A low-power approximate multiplier for convolutional neural networks, characterized in that, include: Input preprocessing circuit (1), weight decoding circuit (2), approximate multiplication circuit based on partial product inference (3), and output postprocessing circuit (4); The input preprocessing circuit (1) preprocesses the input of the multiplier, and the processed result is connected to the approximate multiplication circuit (3) based on partial product inference and the output postprocessing circuit (4), respectively. The weight decoding circuit (2) decodes the input weights, and its decoding result is connected to the approximate multiplication circuit (3) based on partial product inference and the output post-processing circuit (4), respectively. The approximate multiplication circuit (3) based on partial product prediction performs approximate multiplication on its input, and its 16-bit output is connected to the output post-processing circuit (4) to obtain the final approximate multiplication calculation result. The approximate multiplication circuit (3) based on partial product prediction includes: a partial product array generation circuit, a partial product compression circuit, and a 7-bit ripple carry adder; The partial product array generation circuit includes 41 two-input AND gates, which generate 41 partial products and are connected to the partial product compression circuit. The partial product compression circuit consists of four stages of parallel accumulation circuits, including 24 full adders, 8 half adders, and a sixth column partial product compressor. Its output of two rows of partial products is connected to a 7-bit ripple carry adder. The sixth column partial product compressor includes five two-input AND gates, three two-input OR gates, and one three-input OR gate. Through different logical combinations of the six inputs, it obtains a speculative value to replace the sum of the ungenerated partial products in the fifth column, along with two carry signals and two summation signals to represent the approximate addition result of these six inputs. Furthermore, the product non-zero flag is generated by a non-zero product judgment circuit, which is independent of the other three parts. It consists of two eight-input OR gates and one two-input AND gate. The inputs of the OR gates are the 8-bit signals of the input and weight, respectively. The output signals of the two OR gates are connected to the two-input AND gate to determine whether both inputs of the multiplier are non-zero. If either input is zero, the final product is zero. Partial product generation principle: Each bit of one operand is ANDed with each bit of the other operand to obtain the partial product at the corresponding position. The lowest row and the lowest six columns are omitted, so 41 two-input AND gates are needed. When neither operand is 0, the part is approximately (110000)2, that is, the lower four bits are 0, and the fourth and fifth bits are the product non-zero flag. The product non-zero flag is generated by two eight-input OR gates and one two-input AND gate. The input of an eight-input OR gate is an 8-bit signal of an operand, which is used to determine whether the operand is 0. The outputs of the two OR gates are connected to a two-input AND gate. The output of the AND gate is 1, which indicates that the product is non-zero.

2. The low-power approximate multiplier for convolutional neural networks according to claim 1, characterized in that, The input preprocessing circuit (1) includes: a pilot 1 detection circuit, a left shifter, and an encoder; The lead 1 detection circuit is used to detect the position of the lead 1 of the input 8-bit operand and input the result to the left shifter and encoder. According to the output result of the lead 1 detection circuit, the left shifter performs a shift operation on the input 8-bit operand and inputs the result to the approximate multiplication circuit (3) based on partial product inference. The encoder encodes the output of the pilot 1 detection circuit into a left-shifted number of bits, and the result is connected to the output post-processing circuit (4).

3. The low-power approximate multiplier for convolutional neural networks according to claim 1, characterized in that, The weight decoding circuit (2) includes a decoder, wherein the input is a pre-encoded weight, the higher bits of which are connected to the output post-processing circuit (4), the decoder decodes the pre-encoded weight, and its output is connected to the approximate multiplication circuit (3) based on partial product inference.

4. The low-power approximate multiplier for convolutional neural networks according to claim 1, characterized in that, The output post-processing circuit (4) includes: an adder and a right shifter; The adder receives the output of the encoder and the higher bits of the weights in the input preprocessing circuit (1), and its output is connected to a right shifter. The other input of the right shifter is the output of the approximate multiplication circuit (3) based on partial product inference.

5. The low-power approximate multiplier for convolutional neural networks according to claim 1, characterized in that, The working process is as follows: First, the input preprocessing circuit (1) preprocesses the input data. The pilot 1 detection circuit detects the position of the pilot 1 of the input data. According to the position of the pilot 1, the left shifter shifts the input to the left by the corresponding number of bits to retain the pilot 1 and the bits to its right and inputs them to the approximate multiplication circuit (3) based on partial product inference. The encoder encodes the position of the pilot 1 into the number of bits to be shifted to the left and inputs it to the output postprocessing circuit (4). At the same time, the weight decoder decodes the weight after encoding, obtains the pilot 1 and the bits to its right and inputs them to the approximate multiplication circuit (3) based on partial product inference. The higher bits of the weight are the number of bits to be shifted and are connected to the output postprocessing circuit (4).

6. The low-power approximate multiplier for convolutional neural networks according to claim 1, characterized in that, Different approximations can be achieved by setting the maximum number of bits to shift to the left; When shifting left by a maximum of 1 bit, the bit width of shift_w is 1 bit, and the bit width of mult_w is 7 bits. shift_w=0 means shifting left by 0 bits. At this time, the leading 1 is in the highest bit of input w, the high 6 bits of mult_w are the high 6 bits of w[7:2], and the lowest bit of mult_w is the result of the OR of the low 2 bits of w. shift_w=1 means shifting left by 1 bit. At this time, the leading 1 is not in the highest bit of w, and mult_w is the low 7 bits of w[6:0]. When shifting left by a maximum of 2 bits, the bit width of shift_w is 2 bits, and the bit width of mult_w is 6 bits. shift_w=00 means shifting left by 0 bits. At this time, the leading 1 is in the highest bit of input w, the high 5 bits of mult_w are w[6:2], and the lowest bit of mult_w is the result of the OR of the lower 2 bits of w. shift_w=01 means shifting left by 1 bit. At this time, the leading 1 is in the second highest bit of w w[6], and mult_w is the lower 6 bits of w w[5:0]. shift_w=10 means shifting left by 2 bits. At this time, the leading 1 is not in the high 2 bits of w. For the convenience of decoding, the high 5 bits of mult_w are the lower 5 bits of w w[4:0], and the lowest bit of mult_w is w[5]. When shift_w=00 and 01, the hidden 1 in the highest bit of mult_w is omitted by default. When shifting left by a maximum of 3 bits, the bit width of shift_w is 2 bits, and the bit width of mult_w is 6 bits. shift_w=00 indicates a left shift of 0 bits, at which point the leading 1 is in the highest bit of input w, the high 5 bits of mult_w are w[6:2], and the lowest bit of mult_w is the result of the OR operation of the lower 2 bits of w; shift_w=01 indicates a left shift of 1 bit, at which point the leading 1 is in the second highest bit of w w[6], and mult_w is the lower 6 bits of w w[5:0]; shift_w=10 indicates a left shift of 2 bits. When the leading 1 is in the 5th bit of w, i.e. w[5]=1, the high 5 bits of mult_w are the low 5 bits of w w[4:0], and the least significant bit of mult_w is 0; shift_w=11 means shifting left by 3 bits. At this time, the leading 1 is not in the high 3 bits of w, the high 4 bits of mult_w are the low 4 bits of w w[3:0], mult_w[1]=0, and the least significant bit of mult_w is w[4]; when shift_w=00, 01 and 10, the hidden 1 of the highest bit is omitted in mult_w by default; Where shift_w represents the number of bits shifted left, and mult_w represents the valid bits after encoding w.

7. The low-power approximate multiplier for convolutional neural networks according to claim 1, characterized in that, Partial product compression principle: Approximate compression is performed on the partial products generated in the sixth column, and the sum of the partial products in the fifth column is inferred from the partial products in the sixth column. The inference formula is as follows: spec_flag=(pp0&pp1)|(pp1&pp2)|…|(pp n-2 &pp n-1 ) When spec_flag is 1, it means that at least one partial product with a value of 1 exists in the speculated column; when spec_flag is 0, it means that the sum of the speculated columns is very likely to be 0. Here, spec_flag is regarded as a compensation value for the speculated column, and this part approximates the compression itself, as shown in the following formula: s1=pp0 s2=pp3 c1=pp1&pp2 c1=pp5|pp4 Therefore, the partial product compression circuit of the sixth column requires one two-input AND gate and one two-input OR gate, which means that the six partial products are approximately compressed into two carry signals c1 and c2, and two summation signals s1 and s2.

8. The low-power approximate multiplier for convolutional neural networks according to claim 1, characterized in that, The output result of the approximate multiplication circuit (3) based on partial product prediction is post-processed by the output post-processing circuit (4). Since the input operands are left-shifted before the multiplication operation, the multiplication result of the approximate multiplication circuit (3) based on partial product prediction needs to be right-shifted to restore its value. The number of shifts is the sum of the number of left shifts output by the input preprocessing circuit (1) and the weight decoder 2.

Images