Approximate precoding convolution operation method and system based on booth algorithm

Abstract

The application provides a kind of approximate precoding convolution operation method and system based on booth algorithm, comprising: step S1: multiplier and multiplicand input multiplier are separated by approximate precoding module to partial product, and the partial product related to the multiplicand after separation is input to addition tree module to complete compression, and the partial product related to the multiplicand after compression and the partial product related to the multiplier are added to obtain multiplier result;Step S2: the multiplier result is extended to sign bit, and the extended multiplier result is input into accumulation module to carry out shift and addition operation, and the shift addition operation result is carried out carry compensation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular, to an approximate pre-encoding convolution operation method and system based on Booth algorithm, more particularly, to a multiply-accumulate unit based on convolutional neural network (CNN), and to a multiply-accumulate unit (MAC) based on Booth algorithm with 8-bit precision, which is designed based on field programmable gate array (FPGA) and application specific integrated circuit (ASIC), and which realizes a convolution operation unit circuit with multiple precision, low latency, small area and low power consumption. BACKGROUND

[0002] Convolutional neural network (CNN) has been widely used in image recognition and other fields. However, the increasing size of CNN model puts higher requirements on memory size, bandwidth and calculation. Therefore, many researchers use model quantization to reduce the storage and calculation requirements of convolutional neural network. Generally, the weights in the CNN model are trained and saved in 32-bit single-precision floating-point format, and the floating-point model is converted into a fixed-point model for calculation during hardware deployment. A large number of experiments have shown that using 8-bit precision to represent weights and activation values does not bring significant accuracy degradation to the compressed model.

[0003] By model quantization of CNN model, the storage-intensive problem is alleviated to some extent. However, the traditional convolution operation array cannot effectively utilize the advantages of model quantization to solve the problem of dense calculation. Most of the multiply-accumulate units of the convolution operation array use fixed high-precision multipliers for operation, but the multiply-accumulate operation after quantization of the convolution layer does not require such high precision, and a large amount of logic is idle in hardware resources. In addition, different application scenarios have different precision requirements for convolution operation units, and fixed-precision convolution operation units lack flexibility.

[0004] Therefore, researchers have begun to explore how to design an adjustable precision operation unit with 8-bit as the highest precision. The traditional scheme is to decompose a high-precision multiplier into several low-precision multipliers, and the convolution operation unit can be configured to different calculation precisions. Since a high-precision multiplication is decomposed into several low-precision multiplication operations, compared with the fixed-precision operation unit, the throughput and energy efficiency of this scheme are effectively improved.

[0005] However, the design of the operation unit with adjustable precision also has great challenges. On the one hand, with the improvement of performance, the scheme of splicing using a low-precision multiplication unit requires complex configurable logic, which increases the area and reduces the timing margin. On the other hand, with the reduction of precision, the demand for bandwidth of activation values and weights is high, and the logic resources of the configuration data path become further complex. How to meet the multi-precision multiplication and accumulation operation requirements, reduce the logic complexity of the convolution operation unit, and effectively utilize the bandwidth resources has become a key issue.

[0006] The multiplier circuit in the conventional convolution operation unit uses the bit splicing method to complete the multi-precision operation, which increases the complexity of the addition tree circuit, and the configurable logic becomes the main area consumption and critical path of the convolution operation unit. How to optimize the logic complexity of the Booth encoding logic, the addition tree and the configurable logic, and improve the timing of the multiplication and accumulation circuit has become a major problem. SUMMARY

[0007] In view of the defects in the prior art, the purpose of the present application is to provide an approximate pre-coding convolution operation method and system based on Booth algorithm.

[0008] The approximate pre-coding convolution operation method based on Booth algorithm provided by the present application comprises:

[0009] Step S1: the multiplier and the multiplicand are input into the multiplier, and the partial products are separated by the approximate pre-coding module, the separated multiplicand-related partial products are input into the addition tree module to complete compression, the compressed multiplicand-related partial products and the multiplier-related partial products are added to obtain the multiplier result;

[0010] Step S2: the sign bit of the multiplier result is expanded, and the expanded multiplier result is input into the accumulation module for shifting and adding operation, and the result of the shifting and adding operation is compensated for carry.

[0011] Preferably, the step S1 adopts:

[0012] Step S1.1: the weight of the multiplicand is encoded by the base-4 Booth multiplier;

[0013] Step S1.2: the multiplier and the encoded multiplicand are decoded to generate partial products;

[0014] Step S1.3: the generated partial products are separated into multiplicand-related partial products and multiplier-related partial products by the approximate pre-coding module;

[0015] Step S1.4: the multiplicand-related partial products are compressed by the addition tree module to obtain the compressed multiplicand-related partial products; and the compressed multiplicand-related partial products and the multiplier-related partial products are added to obtain the multiplier result.

[0016] Preferably, the step S1.2 employs:

[0017] The multiplied operands after encoding operation are decoded to generate partial products based on an approximate calculation mode or the multiplied operands after encoding operation are decoded to generate partial products based on an accurate calculation mode;

[0018] The approximate calculation mode is that the hardware circuit generates partial products by shifting;

[0019] The accurate calculation mode is that the hardware circuit generates partial products by adding.

[0020] Preferably, the approximate pre-encoding module employs:

[0021] Step S1.3.1: The generated partial products are extended using sign bits;

[0022] Step S1.3.2: The partial products after sign bit extension are separated into multiplier-related partial products and multiplicand-related partial products; the multiplier-related partial products are unsigned numbers determined by weights; the multiplicand-related partial products are partial products that vary with activation values.

[0023] Preferably, the addition tree module employs:

[0024] The effective bits of each row of partial products of the multiplicand are extracted, and when the effective bits are 2 bits, the multiplier result is directly outputted; when the effective bits are not 2 bits, the generation mode of the current partial product is determined;

[0025] When the generation mode of the current partial product is the approximate calculation mode, the multiplicand partial products are divided into right upper, right lower, left upper and left lower four parts according to a preset requirement, the partial products of the four parts are added two by two to generate a first layer result; the first layer result includes the partial products after addition and a carry; the partial products after addition and the carry are added through an addition chain to obtain the multiplier result;

[0026] When the generation mode of the current partial product is the accurate calculation mode, the first row and the second row of partial products are XORed to obtain a result after XOR operation, and then the result after XOR operation is added to the remaining partial products to generate a first layer result; the first layer result includes the partial products after addition and a carry, and the partial products after addition and the carry are added to obtain the multiplier result.

[0027] Preferably, the step S2 employs: one w-bit signed number and one a-bit signed number; after N product accumulations, if the N product accumulations are represented as one q-bit signed number, the q accumulated data bit width satisfies the following:

[0028]

[0029] The accumulation module performs the extraction of the calculation result after completing the N times of multiplication and accumulation operations.

[0030] According to the present application, a system for approximate pre-coding convolution operation based on Booth algorithm is provided, comprising:

[0031] Module M1: the multiplier and the multiplicand are input into the multiplier through the approximate pre-coding module to separate the partial products, the separated multiplicand-related partial products are input into the addition tree module to complete compression, the compressed multiplicand-related partial products and the multiplier-related partial products are added to obtain the multiplier result;

[0032] Module M2: the multiplier result is extended by sign bits, and the extended multiplier result is input into the accumulation module to perform shift and addition operations, and the result of the shift and addition operations is compensated by carry.

[0033] Preferably, the module M1 adopts:

[0034] Module M1.1: the weight of the multiplicand is coded by the base-4 Booth multiplier;

[0035] Module M1.2: the multiplier and the coded multiplicand are decoded to generate partial products;

[0036] Module M1.3: the generated partial products are separated into multiplicand-related partial products and multiplier-related partial products by the approximate pre-coding module;

[0037] Module M1.4: the multiplicand-related partial products are compressed by the addition tree module to obtain the compressed multiplicand-related partial products; the compressed multiplicand-related partial products and the multiplier-related partial products are added to obtain the multiplier result;

[0038] The module M1.2 adopts:

[0039] The coded multiplicand and the multiplier are decoded to generate partial products based on an approximate calculation mode, or the coded multiplicand and the multiplier are decoded to generate partial products based on an accurate calculation mode;

[0040] The approximate calculation mode is that the hardware circuit generates partial products by shifting;

[0041] The accurate calculation mode is that the hardware circuit generates partial products by adding;

[0042] The approximate pre-coding module adopts:

[0043] Module M1.3.1: the generated partial products are extended by sign bits;

[0044] The module M1.3.2 separates the partial products after the sign bit extension into a multiplier-related partial product and a multiplicand-related partial product; wherein the multiplier-related partial product is an unsigned number determined by a weight; and the multiplicand-related partial product is a partial product that varies with an activation value.

[0045] Preferably, the addition tree module adopts:

[0046] The effective bits of each row of the multiplicand partial products are extracted, and when the effective bits are 2 bits, the multiplier result is directly output; when the effective bits are not 2 bits, the generation mode of the current partial product is judged;

[0047] When the generation mode of the current partial product is the approximate calculation mode, the multiplicand partial products are divided into right-up, right-down, left-up and left-down four parts according to a preset requirement, the partial products of the four parts are added two by two to generate a first layer result; the first layer result includes the partial products after the addition and a carry; and the partial products after the addition and the carry are added through an addition chain to obtain the multiplier result;

[0048] When the generation mode of the current partial product is the accurate calculation mode, the first row and the second row of the partial products are XORed to obtain a result after the XOR operation, and then the result after the XOR operation is added to the remaining partial products to generate a first layer result; the first layer result includes the partial products after the addition and a carry, and the partial products after the addition and the carry are added to obtain the multiplier result.

[0049] Preferably, the module M2 adopts: a w-bit signed number and an a-bit signed number; after N multiplication and accumulation operations, if the N multiplication and accumulation operations are represented as a q-bit signed number, then the q accumulation data bit width satisfies the following condition:

[0050]

[0051] The accumulation module performs the extraction of the calculation result after completing N multiplication and accumulation operations.

[0052] Compared with the prior art, the present application has the following beneficial effects: the overall structure of the multiplier reduces the logical resources required by the addition tree by adopting the Booth algorithm; the approximate pre-encoding module of the multiplier not only removes the encoding logic required by the traditional Booth multiplier, but also separates the multiplicand partial products, which simultaneously reduces the area and calculation delay of the multiplier; the addition tree structure of the multiplier adapts to various operation scenarios; and the configurable accumulation module of the convolution operation unit multiplexes the multiplier resources. The present application realizes a low-power, small-area and highly expandable multiplication and accumulation unit based on a convolutional neural network. BRIEF DESCRIPTION OF DRAWINGS

[0053] Other features, objects, and advantages of the application will become more apparent from the following detailed description of non-limiting embodiments thereof, when read in conjunction with the accompanying drawings:

[0054] Figure 1 Fig. 1 is a schematic diagram of the overall structure of the circuit.

[0055] Figure 2 Fig. 4 is a schematic diagram of the parallel operation of the multiplier.

[0056] Figure 3 Fig. 5 is a schematic diagram of the structure of the pre-encoding module.

[0057] Figure 4 Fig. 6 is a schematic diagram of the structure of the base-16 Booth approximation calculation.

[0058] Figure 5 Fig. 7 is a schematic diagram of the module circuit of the addition tree.

[0059] Figure 6 Fig. 8 is a schematic diagram of the configurable logic accumulation module. DETAILED DESCRIPTION

[0060] The application will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the application, but do not limit the application in any form. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the application. These are within the scope of the application.

[0061] Example 1

[0062] According to the approximate pre-encoding convolution operation method based on the Booth algorithm provided by the application, comprising:

[0063] Step S1: the multiplier and the multiplicand are input into the multiplier, and the partial products are separated by the approximate pre-encoding module, the separated multiplicand-related partial products are input into the addition tree module to complete compression, the compressed multiplicand-related partial products and the multiplier-related partial products are added to obtain the multiplier result;

[0064] Step S2: the multiplier result is extended by a sign bit, and the extended multiplier result is input into the accumulation module for shifting and adding operation, and the result of the shifting and adding operation is compensated for carry.

[0065] The Booth algorithm-based approximate precoding convolution operation method is applied to the actual application of image classification, a convolution operation unit (PE) based on the Booth algorithm is applied to the field of picture classification, the feature map and the weight of each layer of network are cached into the multiplier and the multiplicand ports of the PE unit, the control signal is cached into the mode configuration unit through the CPU, and the PE unit completes the operations of encoding, separation and accumulation, and the output result is sent to the output cache; until all network operations are completed, the output results of all PE units are sent to the processing unit to complete image classification.

[0066] Specifically, the step S1 employs:

[0067] Step S1.1: the weight of the multiplicand is subjected to an encoding operation through a base-4 Booth multiplier;

[0068] Step S1.2: the multiplier and the multiplicand subjected to the encoding operation are subjected to a decoding operation to generate partial products;

[0069] Step S1.3: the generated partial products are separated into multiplicand-related partial products and multiplier-related partial products through an approximate precoding module;

[0070] Step S1.4: the multiplicand-related partial products are compressed through an addition tree module to obtain compressed multiplicand-related partial products; and the compressed multiplicand-related partial products and the multiplier-related partial products are added to obtain the result of the multiplier.

[0071] Specifically, the step S1.2 employs:

[0072] The multiplicand subjected to the encoding operation and the multiplier are subjected to a decoding operation to generate partial products based on an approximate calculation mode, or the multiplicand subjected to the encoding operation and the multiplier are subjected to a decoding operation to generate partial products based on an accurate calculation mode;

[0073] The approximate calculation mode is that the hardware circuit generates the partial products through shifting;

[0074] The accurate calculation mode is that the hardware circuit generates the partial products through addition.

[0075] Specifically, the approximate precoding module employs:

[0076] Step S1.3.1: the generated partial products are expanded using a sign bit;

[0077] Step S1.3.2: the partial products subjected to the sign bit expansion are separated into multiplicand-related partial products and multiplier-related partial products; wherein the multiplier-related partial products are unsigned numbers determined by the weight; and the multiplicand-related partial products are partial products that vary with the activation value.

[0078] Specifically, the addition tree module employs:

[0079] extracting the significant bits of each row of the partial products of the multiplicand, when the significant bits are 2 bits, then directly outputting the multiplier result; when the significant bits are not 2 bits, judging the generation mode of the current partial product;

[0080] when the generation mode of the current partial product is the approximate calculation mode, then dividing the partial product of the multiplicand into right upper, right lower, left upper and left lower four parts according to the preset requirement, adding the partial products of the four parts two by two to generate the first layer result; the first layer result includes the partial product and the carry after addition; adding the partial product and the carry after addition through the addition chain to obtain the multiplier result;

[0081] when the generation mode of the current partial product is the accurate calculation mode, then performing an exclusive OR operation on the first row and the second row of the partial product to obtain the result after the exclusive OR operation, and then performing an addition operation on the result after the exclusive OR operation and the remaining partial product to generate the first layer result; the first layer result includes the partial product and the carry after addition, and the partial product and the carry after addition are added to obtain the multiplier result.

[0082] Specifically, the step S2 adopts: a w-bit signed number, an a-bit signed number; after N times of multiplication and accumulation, if the multiplication and accumulation result is represented as a q-bit signed number, then the q accumulation data bit width satisfies the following:

[0083]

[0084] The accumulation module performs the extraction of the calculation result after completing the N times of multiplication and accumulation operation.

[0085] According to the approximate pre-coding convolution operation system based on the Booth algorithm provided by the application, comprising:

[0086] Module M1: the multiplier and the multiplicand input the multiplier, and the partial products are separated through the approximate pre-coding module; the multiplicand related partial products after separation are input into the addition tree module to complete compression; the multiplicand related partial products after compression and the multiplier related partial products are added to obtain the multiplier result;

[0087] Module M2: the multiplier result is extended in sign bit, and the extended multiplier result is input into the accumulation module for shift and addition operation; the result of the shift and addition operation is compensated for carry.

[0088] Specifically, the module M1 adopts:

[0089] Module M1.1: the weight of the multiplicand is coded through the base-4 Booth multiplier;

[0090] Module M1.2: the multiplier and the coded multiplicand are decoded to generate partial products;

[0091] Module M1.3: the generated partial products are separated into a multiplicand-related partial product and a multiplier-related partial product by an approximate precoding module;

[0092] Module M1.4: the multiplicand-related partial product is compressed by an addition tree module to obtain a compressed multiplicand-related partial product; and the compressed multiplicand-related partial product and the multiplier-related partial product are added to obtain a multiplier result.

[0093] Specifically, the module M1.2 employs:

[0094] The multiplicand and the multiplier after the encoding operation are decoded to generate partial products based on an approximate calculation mode, or the multiplicand and the multiplier after the encoding operation are decoded to generate partial products based on an exact calculation mode.

[0095] The approximate calculation mode is that a hardware circuit generates partial products by shifting.

[0096] The exact calculation mode is that a hardware circuit generates partial products by adding.

[0097] Specifically, the approximate precoding module employs:

[0098] Module M1.3.1: the generated partial products are expanded using a sign bit.

[0099] Module M1.3.2: the partial products after the sign bit expansion are separated into a multiplicand-related partial product and a multiplier-related partial product; the multiplier-related partial product is a weight-determined unsigned number; and the multiplicand-related partial product is a partial product that varies with an activation value.

[0100] Specifically, the addition tree module employs:

[0101] The effective bits of each row of partial products of the multiplicand are extracted; when the effective bits are 2 bits, the multiplier result is directly output; when the effective bits are not 2 bits, the generation mode of the current partial product is determined.

[0102] When the generation mode of the current partial product is the approximate calculation mode, the multiplicand partial products are divided into upper right, lower right, upper left, and lower left four parts according to a preset requirement; the partial products of the four parts are added two by two to generate a first layer result; the first layer result includes the partial products after the addition and a carry; and the partial products after the addition and the carry are added by an addition chain to obtain the multiplier result.

[0103] When the generation mode of the current partial product is the exact calculation mode, the first row and the second row of partial products are XORed to obtain a result after the XOR operation; and the result after the XOR operation and the remaining partial products are added to generate a first layer result; the first layer result includes the partial products after the addition and a carry; and the partial products after the addition and the carry are added to obtain the multiplier result.

[0104] Specifically, the module M2 adopts: a w-bit signed number, an a-bit signed number; after N times of product accumulation, if it is represented as a q-bit signed number, then the q accumulation data bit width satisfies the following:

[0105]

[0106] The accumulation module performs calculation result extraction after completing N times of multiplication and accumulation operation.

[0107] Example 2

[0108] Embodiment 2 is a preferred example of embodiment 1

[0109] The present application is directed to a multi-precision neural network application, and an 8-bit approximate pre-biased convolution operation circuit based on Booth algorithm is designed, the present application can also be used to design 16-bit and 32-bit convolution operation units, and can be applied to FPGA and ASIC. The circuit mainly includes a multiplier module, an approximate pre-encoding module, an addition tree module and an accumulation circuit module. The convolution operation circuit supports three precision multiplication operations of 4 2-bit, 2 4-bit and 1 8-bit. By multiplexing the partial product, the addition tree and the accumulation unit, the resource utilization rate is high; by removing the encoding logic to reduce the overall delay and area of the circuit, a general addition tree and accumulation module are designed to be compatible with more scene applications, realize multi-precision, low delay, small area and low power consumption of the convolution operation unit circuit.

[0110] The 8-bit operation based on Booth algorithm proposed in the present application can be for signed operation or unsigned operation, and the parallel computing mode is divided into 8-bit multiplication, 2 4-bit multiplication and 4 2-bit multiplication. n When the operation is performed, 2 2 n-1 bit multiplication, 4 2 n-2 bit multiplication can be selected.

[0111] The design method requires that the accumulation data bit width q in different modes satisfies 2 q-1 ≥ N*2 w-1 (2 a -1)+1. Wherein, N is the number of accumulations in different modes, w is the weight bit width, and a is the activation value bit width.

[0112] After the encoding operation of the Booth algorithm is completed, the predictable value generated by the multiplier of the N times of pre-encoding module is extracted in advance for accumulation. When the first multiplication and accumulation is performed, the accumulation value of the predictable value is input to the accumulation module for summation. The pre-encoding algorithm proposed in the present application can adapt to any size of convolution kernel and is compatible with different parallel degree mode calculations.

[0113] The accumulation result is the sum of N partial products and N predictable values, in order to prevent the overflow of the result, the conventional calculation circuit will expand one bit m as a sign bit. But in the present application, sum1 is taken as an unsigned number. When the pre-encoding module is extracted, one signed number is split into one positive number and one signed number, and the following cases shown in the table can occur. Only when the extracted partial product is positive and the result is negative, positive overflow can occur, and in this case, the negative number obtained by adding the positive numbers is the final result.

[0114] Final result Partial product after extraction Accumulation of predictable values Explanation Positive Positive Positive Two positives are small, addition does not overflow Positive Negative Positive Positive and negative are added, there is no overflow Negative Negative Positive Positive and negative are added, there is no overflow Negative Positive Positive Positive and positive are added, sign bit overflows 1

[0115] In order to extend the circuit structure used in the present application to 16-bit, 32-bit and 64-bit multipliers, the present application uses decoding selection to generate partial products for different precision multiplication operations. The input of the decoding selection is selected according to different modes.

[0116] The addition tree structure used in the present application divides the partial product into four parts. When the precision is low, the resources required for addition output are more, so direct output is considered. When the precision is two highest, the resources consumed by addition output are obviously less than direct output. Therefore, the present application proposes to divide the partial product into left upper, left lower, right upper and right lower four parts, and use direct output in the four parts, and then add and output the last 4 rows of results.

[0117] Example 3

[0118] Embodiment 3 is a preferred example of embodiment 1

[0119] According to the 8-bit approximate pre-encoding convolution operation unit based on the Booth algorithm provided by the present application, the method is suitable for various processes and various models of FPGA, and the overall structure of the circuit is as shown in Figure 1 The circuit mainly includes a multiplier module, an approximate pre-encoding module, an addition tree module and an accumulation circuit module. The circuit design method mainly includes a mode configuration method, a pre-encoding algorithm and a circuit structure planning.

[0120] Multiplier module

[0121] The base-4 Booth multiplier is composed of an encoding module and a decoding selection module, and the overall structure of the multiplier is as shown in Figure 1The 8-bit multiplier is shown in the center of the frame. After the encoding module, each 3-bit multiplier encoding generates a symbol bit, a multiply-1 signal and a multiply-2 signal, a total of 4 groups of signals, each group of signals occupies 3 bits, a total of 12 bits. The 4 groups of signals are sequentially negated, multiplied by 1 or multiplied by 2 with respect to the multiplicand, and the partial products are generated by the decoding selection logic on the circuit. For different precision multiplication operations, the inputs of the decoding selection unit are selected according to different modes, and the multiplication operations of different modes multiplex the decoding selection unit. The 4 groups of partial products generated by the decoding logic are shifted and added pairwise to synthesize 2 groups of partial products; the 2 groups of partial products are shifted to synthesize a 16-bit output result.

[0122] The multiplier proposed in the application is mainly applied to multi-precision multiplication operation, and the supported parallelism is mainly 4 2-bit operations, 2 4-bit operations and 1 8-bit operation, and the multiplier and the multiplicand are multiplied as shown in the Figure 2

[0123] When the mode is 0, the multiplier performs 1 8-bit operation. The 4 groups of partial products are added, and the output result is filled into a 16-bit signal;

[0124] When the mode is 1, the multiplier performs 2 4-bit operations. 2 4-bit inputs are placed in the high 4 bits of the two inputs of the multiplier, and the other 2 inputs are placed in the low 4 bits of the multiplier. After the 4 groups of partial products are added pairwise, 2 8-bit output results are filled into a 16-bit signal;

[0125] When the mode is 2, the multiplier performs 4 2-bit multiplication operations. 2 2-bit inputs are placed in the high 2 bits of the two inputs of the multiplier, and the subsequent multiplications are placed in the next high bit, the next low bit and the low 2 bits of the multiplier in turn. The 4 4-bit output results are directly output to the 16-bit signal without addition.

[0126] Approximate precoding module

[0127] During the operation of the multiplier, the multiplier needs to be encoded to reduce the number of partial products of the multiplicand. When the multiplier is applied to the convolution unit of the neural network, the weight is a "constant" determined after training, and the activation value is a "variable" that changes for different input vectors. The encoding module can select to encode the activation value or the weight, and then the multiplicand generates the partial product. If the "constant" is encoded, the result of the encoding logic can be calculated in advance during the entire convolution process. By using this characteristic, the four encoding logics in the 8-bit multiplier are taken out, as shown in the gray module in Figure 3 , and the results of the 12-bit encoding are directly input.

[0128] ​By calculating the encoding logic in advance, the present application removes the encoding module on the chip, which reduces the area and increases the timing margin. The cost is that more memory and bandwidth are needed to store the pre-encoded weight values. Compared with the activation value, the bit sparsity of the weight is less sensitive, which is reflected in the result of mixed precision quantization. The weight bit width will be smaller and the compression ratio will be larger. Therefore, the higher weight storage requirement caused by the pre-encoding logic is acceptable.

[0129] The Booth multiplier can complete the overall operation through sign bit extension in partial sum calculation. In high-bit multiplication operation, the area and power consumption will be worse due to sign bit extension, but the loss caused by sign bit extension is not large for 2-bit and 4-bit operation. The present application uses the sign bit extension technology, inverts the sign bit of the partial sum, and adds 1 in front of each row.

[0130] In addition to removing the encoding logic, the present application also pre-processes the sign extension bit of the partial product, and calculates the unsigned number generated by the encoding logic in the partial product in advance. When calculating the convolution block of the convolutional neural network, the results of 3*3 or 5*5 need to be accumulated together. Since the weight is pre-trained, the accumulation in the convolution block can be divided into two parts, as shown in the following formula 2, the first part is the partial product which changes with the activation value, and the second part is the unsigned number determined by the weight. Figure 3 The present application proposes that the pre-encoding logic only needs to calculate the result generated by the first part, and the unsigned number generated by the second part is input into the accumulation module of the convolution operation unit after being accumulated in advance in other devices, so as to effectively utilize the accumulation resources of the convolution operation unit.

[0131] The following table shows the accumulation results of the predictable values in different modes, where s is the sign bit after the encoding of the multiplier. N predictable values are unsigned numbers, and the accumulation result of the N predictable values is an m-bit unsigned number, with a value of [0, 2 m -1]. The accumulation result of the N partial products is an n-bit signed number, with a value of [-2 n-1 , 2 n-1 -1], and n is necessarily greater than m. The reason is that the size of n is obtained by accumulating the N largest 16-bit signed numbers, and the predictable value is smaller than the largest 16-bit signed number.

[0132] Mode Operation mode Predictable value 0 8-bit multiplication N{0101_0110_0s0s_0s0s} 1 4-bit multiplication N{0110_0s0s_0110_0s0s} 2 2-bit multiplication N{000s_000s_0000_000s}

[0133] The multiplier proposed in the present application is based on the original accurate calculation, and a configurable approximate calculation mode is proposed, as shown in the following formula 3. In common multiplication operation scenarios such as image processing and convolution operation, reasonable approximate calculation has been proved to have little effect on the result. Figure 4 The present application uses the mixed Booth encoding method to encode the lower 4 bits of the multiplier in base-16, and the result after encoding is shown in the following formula 1, where N is the binary representation of an integer, n-1 Default is 0, formula 2 is base-4 encoding formula, formula 3 is base-16 encoding formula. However, the exact base-16 encoding logic cannot be used with the original base-4 gating logic, so it is considered to use approximate base-16 encoding. By limiting the encoded result to a power of 2, as shown in formula 4, the hardware circuit can generate partial products by shifting. In the present application, the base-16 partial product generation circuit is equivalent to a low two-row partial product bitwise or logic output.

[0134]

[0135]

[0136]

[0137]

[0138] Addition tree module

[0139] Since the multiplier supports both approximate and exact calculation modes, after generating 4 rows of partial products through pre-encoding logic, there are two addition tree operation sequences, and the calculation sequence is as shown in formula 4. Figure 5

[0140] The exact calculation sequence is as follows:

[0141] In the first operation period, 4-bit significant bits of each row of partial products are extracted, corresponding to the multiplication result of the multiplier and the multiplicand in formula 4. Figure 2 The 16-bit result is the output result of 4 2-bit multiplications. This result is directly output through mode selection, which consumes 16 muxes at this time.

[0142] In the second operation period, the upper right, lower right, upper left, and lower left four partial products are calculated at the same time, and two rows of partial products of the four partial products are added two by two. At this time, instead of directly outputting 2 8-bit, the upper left and lower right 10 partial products are muxed with 0. This operation consumes 10 muxes, saving 6 muxes compared to directly outputting 16-bit signals.

[0143] In the third operation period, the sum of the last 2 rows of partial products is completed. When the mode is 4-bit operation, the output result of the second operation period is added to 0 for output; when the mode is 8-bit operation, 2 rows of partial products and 2 carries are added through an addition chain for output.

[0144] For the approximate multiplication mode:

[0145] The first operation period result remains unchanged, and 16 muxes are needed for 4-bit result direct output.

[0146] ​The second operation period simultaneously completes the first row and the second row XOR operation, 3 partial addition operations, compared with the accurate calculation mode, the approximate calculation saves the adder resource required for the upper left partial addition operation.

[0147] The third operation period outputs 2 rows of partial product and 1 carry addition.

[0148] In summary, the approximate calculation is compatible with the accurate calculation structure, and the timing and area are superior to the accurate calculation.

[0149] In the addition tree structure, the 2-bit multiplication result is directly output, and the 4-bit multiplication and 8-bit multiplication results are output through partial accumulation. This selection is based on the consideration of mux consumption. Taking 64-bit multiplication supporting 5 kinds of precision as an example, the mux resource consumption is shown in the following table, and p is the mode parallel degree.

[0150] Precision Direct output consumes mux resources Addition output consumes mux resources 2-bit 4p 28p 4-bit 8p 24p 8-bit 16p 16p 16-bit 32p 8p 32-bit 64p 8p

[0151] Accumulation circuit module

[0152] In order to prevent overflow of the convolutional neural network accumulation result, the 16-bit result of the multiplier needs to be split and placed in the accumulation circuit, as shown in Figure 1 .

[0153] The present application proposes a kind of hardware compatible accumulation circuit for mixed precision multiplier, and the structure is as shown in Figure 6 . 16-bit output result is first expanded according to configuration signal, then is accumulated with the signal inputted from outside. Taking 4 4-bit and 1 16-bit output result as an example:

[0154] 4 4-bit output results are filled psum_out[47:36], psum_out[35:24], psum_out[23:12], psum_out[11:0] respectively, enter 4 12-bit adders, and are accumulated with the input C[47:36], C[35:24], C[23:12], C[11:0], C is the accumulation value of circuit, at this time, carry is not considered;

[0155] 1 16-bit output result is filled psum_out[47:0], and after being divided into 4 parts, it also enters 4 12-bit adders, and is also accumulated with the input 4 parts of accumulation value. Configuration unit will control carry of last result to add and output current result.

[0156] Configuration unit controls carry to complete accumulation calculation in different modes, and 4 12-bit adder resources are multiplexed.

[0157] The accumulated data bit width in different modes satisfies the following requirements. A w-bit signed number, an a-bit signed number. After N such products are accumulated, if they are represented as a q-bit signed number, q should satisfy formula 5.

[0158] Then the accumulation module can perform the extraction of the calculation result after completing the N times of multiplication and accumulation operations.

[0159]

[0160] Those skilled in the art know that, in addition to implementing the system, device and each module thereof provided by the present application in the form of pure computer readable program code, the same program can also be implemented in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers, etc. by logically programming the method steps. Therefore, the system, device and each module thereof provided by the present application can be considered as a hardware component, and the modules included therein for implementing various programs can also be considered as structures in the hardware component; the modules for implementing various functions can also be considered as both software programs for implementing methods and structures in the hardware component.

[0161] The specific embodiments of the present application are described above. It needs to be understood that the present application is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essential content of the present application. The embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily without conflict.

Claims

1. A method for approximate pre-coding convolution operation based on Booth algorithm, characterized in that, The method comprises the steps of: Step S1: the multiplier and the multiplicand are input into the multiplier, and the partial products are separated by the approximate pre-encoding module, the multiplicand-related partial products are input into the addition tree module for compression, and the compressed multiplicand-related partial products and the multiplier-related partial products are added to obtain the multiplier result; Step S2: the multiplier result is extended by a sign bit, and the extended multiplier result is input into the accumulation module for shifting and adding operation, and the result of the shifting and adding operation is compensated by carry; The approximate pre-encoding module adopts: Step S1.3.1: the generated partial product is extended by a sign bit; Step S1.3.2: the partial product extended by the sign bit is separated into the multiplicand-related partial product and the multiplier-related partial product; wherein the multiplier-related partial product is an unsigned number determined by a weight; and the multiplicand-related partial product is a partial product varying with an active value; The addition tree module adopts: The effective bits of each row of partial products of the multiplicand are extracted, when the effective bits are 2 bits, the multiplier result is directly output; when the effective bits are not 2 bits, the generation mode of the current partial product is judged; When the generation mode of the current partial product is the approximate calculation mode, the multiplicand partial product is divided into right upper, right lower, left upper and left lower four parts according to the preset requirement, the partial products of the four parts are added two by two to generate the first layer result; the first layer result comprises the partial product after addition and the carry; the partial product after addition and the carry are added through the addition chain to obtain the multiplier result; When the generation mode of the current partial product is the accurate calculation mode, the first row and the second row of the partial product are operated to obtain the result after the operation, and then the result after the operation is added to the remaining partial product to generate the first layer result; the first layer result comprises the partial product after addition and the carry, and the partial product after addition and the carry are added to obtain the multiplier result.

2. The method of claim 1, wherein the Booth algorithm-based approximate precoded convolution operation is performed by a processor. The step S1 adopts: Step S1.1: the weight of the multiplicand is encoded by the base-4 Booth multiplier; Step S1.2: the multiplier and the encoded multiplicand are decoded to generate partial products; Step S1.3: the generated partial products are separated into the multiplicand-related partial products and the multiplier-related partial products by the approximate pre-encoding module; Step S1.4: the multiplicand-related partial products are compressed by the addition tree module to obtain the compressed multiplicand-related partial products; and the compressed multiplicand-related partial products and the multiplier-related partial products are added to obtain the multiplier result.

3. The method of claim 2, wherein the Booth algorithm-based approximate pre-encoding convolution operation is performed by a processor. The step S1.2 adopts: The encoded multiplicand and the multiplier are decoded to generate partial products based on the approximate calculation mode, or the encoded multiplicand and the multiplier are decoded to generate partial products based on the accurate calculation mode; The approximate calculation mode is that the hardware circuit generates partial products by shifting; The accurate calculation mode is that the hardware circuit generates partial products by adding.

4. The method of claim 1, wherein the Booth algorithm based approximate pre-encoding convolution operation is performed by a processor. The step S2 adopts: one w-bit signed number and one a-bit signed number; after N times of multiplication and accumulation, if the product is represented as one q-bit signed number, then the q accumulation data bit width satisfies the following condition: The accumulation module extracts the calculation result after completing N times of multiplication and accumulation operation.

5. A system for approximate pre-coding convolution operation based on Booth's algorithm, characterized in that, The method comprises the steps of: Module M1: the multiplier and the multiplicand input the multiplier through the approximate pre-coding module to separate the partial product, the separated multiplicand related partial product is input to the addition tree module to complete compression, the compressed multiplicand related partial product and the multiplier related partial product are added to obtain the multiplier result; Module M2: the multiplier result is sign bit extended, and the extended multiplier result is input to the accumulation module to perform shifting and adding operation, and the result of the shifting and adding operation is carried out to compensate for the carry; The approximate pre-coding module adopts: Module M1.3.1: the generated partial product is extended using a sign bit; Module M1.3.2: the sign bit extended partial product is separated into a multiplicand related partial product and a multiplier related partial product; wherein the multiplier related partial product is an unsigned number determined by a weight; the multiplicand related partial product is a partial product that varies with an active value; The addition tree module adopts: The effective bits of each row of partial products of the multiplicand are extracted, when the effective bits are 2 bits, the multiplier result is directly output; when the effective bits are not 2 bits, the generation mode of the current partial product is judged; When the generation mode of the current partial product is the approximate calculation mode, the multiplicand partial product is divided into right upper, right lower, left upper and left lower four parts according to the preset requirement, the partial products of the four parts are added two by two to generate the first layer result; the first layer result includes the partial product after addition and the carry; the partial product after addition and the carry are added through the addition chain to obtain the multiplier result; When the generation mode of the current partial product is the accurate calculation mode, the first row and the second row of the partial product are operated to obtain the result after the operation, and then the result after the operation is added with the remaining partial product to generate the first layer result; the first layer result includes the partial product after addition and the carry, and the partial product after addition and the carry are added to obtain the multiplier result.

6. The system for approximate pre-coding convolution operation based on Booth algorithm according to claim 5, wherein, The module M1 adopts: Module M1.1: the weight of the multiplicand is coded by the base-4 Booth multiplier; Module M1.2: the multiplier and the coded multiplicand are decoded to generate partial products; Module M1.3: the generated partial products are separated into multiplicand related partial products and multiplier related partial products by the approximate pre-coding module; Module M1.4: the multiplicand related partial products are compressed by the addition tree module to obtain the compressed multiplicand related partial products; the compressed multiplicand related partial products and the multiplier related partial products are added to obtain the multiplier result; The module M1.2 adopts: The coded multiplicand and the multiplier are decoded to generate partial products based on the approximate calculation mode, or the coded multiplicand and the multiplier are decoded to generate partial products based on the accurate calculation mode; The approximate calculation mode is that the hardware circuit generates partial products by shifting; The accurate calculation mode is that the hardware circuit generates partial products by adding.

7. The system for approximate pre-coding convolution operation based on Booth algorithm according to claim 5, wherein, The module M2 adopts: a w-bit signed number and an a-bit signed number; after N times of multiplication and accumulation, if the product is represented as a q-bit signed number, the q accumulation data bit width satisfies the following: The accumulation module extracts the calculation result after completing N times of multiplication and accumulation operation.

Images