A multiplier for retraining approximate neural networks and its retraining algorithm

Abstract

The present invention provides a multiplier for approximate neural network retraining and a retraining algorithm thereof, wherein the multiplier includes multiple approximate Booth encoding modules, multiple partial product generation modules, and an adder; each approximate Booth encoding module receives multi-bit data corresponding to a weight input position and encodes the multi-bit data; each partial product generation module receives the multi-bit data encoding result output by the corresponding approximate Booth encoding module and operates the multi-bit data encoding result with feature map data to obtain partial products; the adder sums all partial products to obtain a final output result. The present invention reduces the number of partial products by using high-radix Booth encoding and introduces approximation in the approximate Booth encoding and partial product generation modules, thereby reducing the area and energy consumption of the multiplier.

Description

Technical Field

[0001] The present invention belongs to the technical field of approximate calculation and digital operation devices, and in particular relates to a multiplier for approximate neural network retraining and a retraining algorithm thereof. Background Art

[0002] Convolutional neural networks (CNNs) are now widely used in computer vision tasks such as image classification and object detection. Many edge hardware devices, such as smart cameras, require real-time processing of captured images at the edge to accomplish tasks such as face recognition and vehicle detection. However, strict area and energy constraints at the edge pose significant challenges to the design and implementation of neural network acceleration hardware. Approximate computing is an important solution to this problem.

[0003] Approximate computing leverages the inherent fault tolerance of applications like digital signal processing and neural networks. By introducing a certain amount of error into the computing components, it reduces hardware area and energy consumption while keeping application performance degradation within acceptable limits. In neural networks, multiplication operations have the highest utilization and energy consumption, and the area of ​​multipliers is much larger than other computing components like adders. Therefore, designing appropriate approximate multipliers to replace exact multipliers can better adapt acceleration hardware to edge scenarios.

[0004] Directly replacing exact multipliers with approximate ones can significantly reduce the accuracy of neural networks, especially for complex tasks. Retraining by simulating the computational behavior of approximate multipliers can compensate for the accuracy loss caused by approximate multipliers. However, because the multiplication process introduces irregular errors (such as the generation and accumulation of partial products), these errors can only be simulated during training using lookup tables and are difficult to accelerate using GPUs. Therefore, the design of current retraining methods is limited to smaller networks and relatively simple tasks. To circumvent the difficulties of retraining, some existing work begins by analyzing the error tolerance of each layer in the CNN and then assigning different approximate multipliers to each layer, even using exact multipliers in some key layers to reduce retraining overhead by minimizing accuracy loss. However, this exhaustive design space exploration approach is overly sensitive to network structure; even slight changes to the CNN structure can cause the approximate multiplier array to fail, thus losing accuracy. In order to ensure the versatility of approximate neural network acceleration hardware, some work has also combined multiple possible approximate multipliers into different computing arrays, which are then selected by each layer of routers; this leads to a significant increase in area, while still resulting in a high degree of accuracy degradation in complex tasks. Summary of the Invention

[0005] In view of the deficiencies in the prior art, the present invention provides a multiplier for retraining an approximate neural network and a retraining algorithm thereof.

[0006] In a first aspect, the present invention provides a multiplier for retraining an approximate neural network, comprising:

[0007] Multiple approximate Booth encoding modules, multiple partial product generation modules and adders;

[0008] Each of the approximate Booth encoding modules receives multi-bit data at a corresponding position in the weight input and encodes the multi-bit data;

[0009] Each of the partial product generation modules receives the multi-bit data encoding result output by the corresponding approximate Booth encoding module, and operates the multi-bit data encoding result with the feature map data to obtain a partial product;

[0010] The adder accumulates and adds all the parts to obtain the final output result.

[0011] Furthermore, multiple high-radix Booth codes are used to reduce the number of partial products.

[0012] Furthermore, the adopting of multiple high-radix Booth codes to reduce the number of the partial products includes:

[0013] Starting from the tail of the weight input, base-8 Booth encoding is performed sequentially;

[0014] If encoding into the weight input header requires sign bit extension for base-8 Booth encoding, the header is encoded using a lower base than base-8 Booth encoding.

[0015] Furthermore, each of the approximate Booth encoding modules sets all encoding results of +3 or -3 in the base-8 Booth encoding truth table to 0; after removing the encoding results of +3 and -3, the products of the remaining encoding results and the feature map input in each of the partial product generation modules are generated by shifting.

[0016] Furthermore, each group of 4 bits in the weight is re-encoded into 3 bits, and the 3-bit encoded weight is stored; the input of each of the approximate Booth coding modules is reduced from 4 bits to 3 bits to reduce the overhead of each of the approximate Booth coding modules.

[0017] In a second aspect, the present invention provides a retraining algorithm for a multiplier for retraining an approximate neural network as described in the first aspect, comprising:

[0018] Set the state where the encoding result of the integer type weight input is ±3 to an invalid state, and sparsely quantize the floating-point weight to the nearest valid integer type state;

[0019] The sparse quantization-aware training method is used for retraining so that all weights are in a valid state with correct Booth encoding.

[0020] Furthermore, the retraining using the sparse quantization-aware training method to make all weights in a valid state where Booth encoding is correct includes:

[0021] The weight W of each layer in the neural network is obtained by the proposed pseudo sparse quantization module to obtain the sparse quantization perception weight The pseudo-sparse quantization module first sparsely quantizes the floating-point weights to the nearest valid integer type, and then dequantizes them back to floating-point type to achieve sparse quantization and training of the weights W;

[0022] The bias parameter b of each layer in the neural network is quantized by the pseudo-quantization module to obtain the perceptual bias. The pseudo quantization module quantizes the floating-point type bias to integer type first, and then dequantizes it back to floating-point type to achieve quantization and training of the bias parameter b;

[0023] use and Perform forward calculations on each layer and pass the calculation results through the pseudo-quantization module to obtain the output results of each layer;

[0024] The back propagation algorithm is used to update the weights and bias parameters of each layer in the neural network.

[0025] The present invention provides a multiplier for approximate neural network retraining and its implementation method, wherein the multiplier includes multiple approximate Booth encoding modules, multiple partial product generation modules and an adder; each of the approximate Booth encoding modules receives multi-bit data at a corresponding position in the weight input and encodes the multi-bit data; each of the partial product generation modules receives the multi-bit data encoding result output by the corresponding approximate Booth encoding module, and operates the multi-bit data encoding result with feature map data to obtain partial products; the adder adds all the partial products to obtain the final output result. The present invention reduces the number of partial products by using high-radix Booth encoding and introduces approximation in the approximate Booth encoding and partial product generation modules, thereby reducing the area and energy consumption of the multiplier. At the same time, by proposing a retraining algorithm that matches this approximate multiplier, efficient retraining can be achieved on the approximate neural network using this multiplier. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0027] Figure 1 The Booth multiplier provided in the embodiment of the present invention completes the operation diagram of multiplying the 8-bit feature map by the 6-bit weight.

[0028] Figure 2 A schematic diagram of Booth coding division provided in an embodiment of the present invention;

[0029] Figure 3 A schematic diagram of the structure of an 8-bit × 6-bit approximate multiplier provided in an embodiment of the present invention;

[0030] Figure 4 A schematic diagram of a pseudo quantization module provided in an embodiment of the present invention;

[0031] Figure 5 A schematic diagram of a pseudo-sparse quantization module provided in an embodiment of the present invention;

[0032] Figure 6 A schematic diagram of sparse quantization perception training provided by an embodiment of the present invention;

[0033] Figure 7 A flow chart of a retraining algorithm for a multiplier for approximate neural network retraining provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0035] In one embodiment, if Figure 3 As shown, an embodiment of the present invention provides a multiplier for approximate neural network retraining, which is applicable to different input bit widths and includes multiple approximate Booth encoding modules, multiple partial product generation modules and adders.

[0036] Each of the approximate Booth encoding modules receives multi-bit data at a corresponding position in the weight input and encodes the multi-bit data.

[0037] Each of the partial product generation modules receives the multi-bit data encoding result output by the corresponding approximate Booth encoding module, and operates the multi-bit data encoding result with the feature map data to obtain a partial product.

[0038] Use multiple high-radix Booth codes to reduce the number of partial products. Radix-8 Booth codes are applied sequentially starting from the tail of the weight input. If the weight input header requires sign bit extension for radix-8 Booth coding, encode the header using a lower radix than radix-8 Booth coding. For example, these header bits are encoded using a lower radix such as radix-4 Booth coding.

[0039] For example, Figure 1 The figure shows how the Booth multiplier performs the computation between an 8-bit feature map and a 6-bit weight. (a) shows the radix-4 Booth encoding scheme, and (b) shows the radix-8 Booth encoding scheme. The radix-8 Booth encoding scheme reduces one partial product compared to the traditional radix-4 Booth encoding scheme.

[0040] like Figure 2 As shown, (a) is the base-4 Booth encoding under 7-bit weight, (b) is the proposed encoding division under 7-bit weight, (c) is the base-4 Booth encoding under 8-bit weight, and (d) is the proposed encoding division under 8-bit weight, and e in the figure represents the sign bit extension bit. The number of partial products generated when the weight input of the embodiment of the present invention is 7-bit and 8-bit can be reduced by 1. This shows that the present invention can be applied to different weight input bit widths to reduce the number of partial products, thereby reducing the area and energy consumption of the approximate multiplier.

[0041] As shown in Table 1, each of the approximate Booth encoding modules sets all the encoding results of +3 or -3 in the base-8 Booth encoding truth table to 0; after removing the encoding results of +3 and -3, the remaining encoding results are multiplied by the feature map input in each of the partial product generation modules by shifting.

[0042] The weight encoding method is to re-encode each group of 4 bits in the weight into 3 bits and store the 3-bit encoded weight; reduce the input of each of the approximate Booth encoding modules from 4 bits to 3 bits to reduce the overhead of each of the approximate Booth encoding modules.

[0043] As shown in Table 1, the proposed approximate Booth coding results in only 7 possible outcomes, which can be fully represented using only 3 bits. Therefore, according to Table 1, each group of 4 bits is first encoded into 3 bits and stored, and then these 3 bits are mapped to the corresponding results in the approximate Booth coding module of the multiplier. This can further simplify the approximate Booth coding module of the multiplier, and at the same time, the first step of coding is pre-implemented before storage, without incurring overhead. Figure 2 As shown, before Booth encoding, the weight needs to be padded with 0, so the base-8 approximate booth encoding of the tail is directly applied to the 3-bit input.

[0044] Table 1 Radix-8 Booth coding and proposed weight coding method

[0045]

[0046] The adder accumulates and adds all the parts to obtain the final output result.

[0047] like Figure 3 As shown, an 8-bit × 6-bit approximate multiplier is provided. Prior to use, the weights, quantized to 6-bit fixed-point integers, are encoded using the proposed weight encoding method and stored. During use, the lower 3 bits (2-0) and upper 3 bits (5-3) of the re-encoded weights are sequentially passed through the approximate Booth encoder and the partial product generation module, respectively. The two partial products are then summed to produce the final result.

[0048] In another embodiment, Figure 7 As shown, an embodiment of the present invention provides a retraining algorithm for a multiplier for retraining an approximate neural network, including:

[0049] Step 101: Set the state in which the encoding result of the integer type weight input is ±3 to an invalid state, and sparsely quantize the floating-point weight to the nearest valid integer type state.

[0050] Step 102: retraining is performed using a sparse quantization-aware training method so that all weights are in a valid state where Booth encoding is correct.

[0051] For example, Figure 4 、 Figure 5 and Figure 6 As shown, the weight W of each layer in the neural network is obtained by the proposed pseudo sparse quantization module to obtain the sparse quantization perception weight The pseudo-sparse quantization module first sparsely quantizes the floating-point weights to the nearest valid integer state, and then dequantizes them back to the floating-point type to achieve sparse quantization and training of the weights W.

[0052] The bias parameter b of each layer in the neural network is quantized by the pseudo-quantization module to obtain the perceptual bias. The pseudo-quantization module first quantizes the floating-point type bias to an integer type, and then dequantizes it back to a floating-point type to achieve quantization and training of the bias parameter b.

[0053] use and Perform forward calculations on each layer, and pass the calculation results through the pseudo-quantization module to obtain the output results of each layer.

[0054] The weights and bias parameters of each layer in the neural network are updated through the back-propagation algorithm. During the back-propagation process, the gradient bypasses the pseudo-quantization module and the pseudo-sparse quantization module to solve the problem of non-differentiable operations in these two modules.

[0055] The present invention has been described in detail above with reference to specific embodiments and exemplary examples. However, these descriptions should not be construed as limiting the present invention. Those skilled in the art will appreciate that various equivalent substitutions, modifications, or improvements may be made to the technical solutions and implementations of the present invention without departing from the spirit and scope of the present invention, all of which fall within the scope of the present invention. The scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A multiplier for retraining an approximate neural network, characterized in that: It includes multiple approximate Booth encoding modules, multiple partial product generation modules and adders; Each of the approximate Booth encoding modules receives multi-bit data at a corresponding position in the weight input and encodes the multi-bit data; Each of the partial product generation modules receives the multi-bit data encoding result output by the corresponding approximate Booth encoding module, and operates the multi-bit data encoding result with the feature map data to obtain a partial product; The adder accumulates all the parts and obtains the final output result; Using multiple high-radix Booth codes to reduce the number of partial products; include: Starting from the tail of the weight input, base-8 Booth encoding is performed sequentially; If the encoding into the weight input header requires sign bit extension for base-8 Booth encoding, the header is encoded using a lower base than base-8 Booth encoding; Each of the approximate Booth encoding modules sets all encoding results of +3 or -3 in the base-8 Booth encoding truth table to 0; after removing the encoding results of +3 and -3, the remaining encoding results are multiplied by the feature map input in each of the partial product generation modules by shifting.

2. The multiplier for approximate neural network retraining according to claim 1, characterized in that: Each group of 4 bits in the weight is re-encoded into 3 bits, and the 3-bit encoded weight is stored; the input of each of the approximate Booth encoding modules is reduced from 4 bits to 3 bits to reduce the overhead of each of the approximate Booth encoding modules.

3. A retraining algorithm for a multiplier for retraining an approximate neural network according to any one of claims 1 to 2, characterized in that: include: Set the state where the encoding result of the integer type weight input is ±3 to an invalid state, and sparsely quantize the floating-point weight to the nearest valid integer type state; Use sparse quantization-aware training to retrain so that all weights are in a valid state with correct Booth encoding. The retraining using the sparse quantization perception training method to make all weights in a valid state with correct Booth encoding includes: The weight W of each layer in the neural network is obtained by the proposed pseudo sparse quantization module to obtain the sparse quantization perception weight The pseudo-sparse quantization module first sparsely quantizes the floating-point weights to the nearest valid integer state, and then dequantizes them back to floating-point type to achieve sparse quantization and training of the weights W. The bias parameter b of each layer in the neural network is quantized by the pseudo-quantization module to obtain the perceptual bias. ,The pseudo quantization module quantizes the floating-point type bias to integer type first, and then dequantizes it back to floating-point type to achieve quantization and training of the bias parameter b; use and Perform forward calculations on each layer and pass the calculation results through the pseudo-quantization module to obtain the output results of each layer; The back propagation algorithm is used to update the weights and bias parameters of each layer in the neural network.

Images