A hardware accelerator for robotic grasping

Abstract

The application provides a hardware accelerator for robot grabbing, comprising a top controller, n computing array units, a buffer unit and a router unit; the top controller is used for managing the timing logic of the top layer; the buffer unit is controlled by reading and writing through enabling and addressing signals; each computing array unit is composed of m high-bit multiplication unit groups, and m high-bit multiplication unit groups of the same computing array unit share activation, and weights are shared between different computing array units; the computing array unit is responsible for processing tile-based convolution or input feature mapping deconvolution, and outputs through an output buffer as the input of the next layer. The hardware accelerator of the application has lower processing delay, lower power consumption and area, higher peak energy efficiency and peak area efficiency when processing quantized grabbing detection networks; supports more different bit-width quantization schemes; reduces resource overhead, calculation and storage complexity without losing accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of integrated circuits, and in particular to a hardware accelerator for robot grasping. BACKGROUND

[0002] Robot grasping tasks require cameras to obtain environmental information and rely on grasping detection networks to analyze and identify objects to be grasped. However, grasping detection networks often have high complexity, and low-bit quantization techniques are needed to reduce network complexity. For low-bit quantization techniques, many hardware accelerators have been derived to specifically accelerate the convolution and deconvolution operations of the quantized grasping detection network. Considering the various quantization strategies of low-bit quantization techniques, hardware accelerators are developing towards precision scalability. The main design idea of hardware accelerators relies on bit-serial and multiplication decomposition.

[0003] Bit-serial-based accelerators obtain precision scalability by adjusting processing cycles and serially processing activations or weights; multiplication decomposition-based accelerators obtain precision scalability by decomposing high-precision fixed-point multiplication.

[0004] However, bit-serial-based accelerators require additional cycles, and multiplication decomposition-based accelerators have significant hardware redundancy. At the same time, the precision-scalable hardware accelerator still has two problems that have not been solved, namely, the hardware redundancy of the basic MAC unit and its spatial composition, and although the existing technology introduces a simplified Bit Brick based on input-output analysis, there is still significant resource waste in common precision modes. SUMMARY

[0005] To overcome the deficiencies of the prior art, the present application provides a hardware accelerator for robot grasping.

[0006] The technical solution of the present application is: a hardware accelerator for robot grasping, comprising a top-level controller, n computing array units, a buffer unit, and a router unit; the top-level controller is used to manage the timing logic of the top level; and the buffer unit is controlled by enabling and addressing signals for reading and writing; each computing array unit is composed of m high-bit multiplication unit groups MMU Group, and the m high-bit multiplication unit groups MMU Group of the same computing array share activations, and different computing arrays share weights; the computing array unit is responsible for processing tile-based convolution or input feature mapping deconvolution, and outputs through the output buffer as the input of the next layer.

[0007] As preferred, the buffer unit comprises input buffer and weight buffer; the router unit comprises input router and weight router; the top controller reads input and weight from the input buffer and weight buffer through enable and address signals; then the input router and weight router rearrange the data sent by the input buffer and weight buffer according to various calculation modes.

[0008] As preferred, the accelerator further comprises accumulator ACC; the top controller supervises the accumulator ACC to accumulate and clear, and writes the result processed by the calculation array unit into the output buffer through the accumulator ACC as the input of the next layer.

[0009] As preferred, each high-bit multiplication unit group MMU Group is composed of k multi-bit multiplication units MMU.

[0010] As preferred, each multi-bit multiplication unit MMU is composed of multiple bit-level multiplication units BMU.

[0011] As preferred, the bit-level multiplication unit BMU has four calculation conditions of signed number x signed number, signed number x unsigned number, unsigned number x signed number and unsigned number x unsigned number, and is expressed by the result of a 3b x 2b signed number multiplier.

[0012] As preferred, the four calculation conditions of the bit-level multiplication unit BMU are respectively expressed as:

[0013]

[0014]

[0015]

[0016]

[0017] In the formula, A2, A1 and A0 respectively represent the highest bit, the first bit and the lowest bit of the input value; W1 and W0 respectively represent the highest bit and the lowest bit of the weight; represents the highest bit of the input value taking inversion; represents the highest bit of the weight taking inversion;

[0018] Among them, formula (2) to formula (4) increase high-bit inversion and bit deviation relative to formula (1).

[0019] As preferred, the four calculation conditions of the bit-level multiplication unit BMU are uniformly expressed as:

[0020]

[0021] wherein S A , S W represents an extra sign bit, 1 for signed number, 0 for unsigned number.

[0022] As preferred, each of the bit-level multiplication units BMU comprises two half-adders and two selectors.

[0023] As preferred, the multi-bit-width multiplication units MMU are formed by combining a plurality of bit-level multiplication units BMU and shift-add logic according to different calculation modes.

[0024] As preferred, the multi-bit-width multiplication units MMU implement five precision calculation modes, i.e. 3b x 2b, 4b x 2b, 4b x 4b, 6b x 2b and 6b x 4b; wherein 3bit input is a signed number, and 4bit input is a non-negative signed number, input is 12bits of activation and 8bits of weight, and output is multiplication result less than or equal to 10bits.

[0025] As preferred, the calculation parallelism in each of the multi-bit-width multiplication units MMU is given to the input channel dimension, and the partial sum accumulation of the plurality of bit-level multiplication units BMU in each multi-bit-width multiplication unit MMU is performed in the input channel dimension.

[0026] As preferred, the input activation of the high-bit multiplication unit group MMU Group comes from k input channels; then the multi-bit-width multiplication units MMU are reorganized in the high-bit multiplication unit group MMU Group; and since the bit-level multiplication units BMU in the same position in different multi-bit-width multiplication units MMU have the same shift logic; the results of the bit-level multiplication units BMU in the same position in different multi-bit-width multiplication units MMU are added, and then sent to the shift logic to obtain the output.

[0027] The beneficial effects of the present application are:

[0028] 1. The hardware accelerator of the present application has lower processing delay for the quantized grasp detection network, and only needs 4.5ms, which is substantially improved compared with the prior art of more than 20ms, and the acceleration ratio is 4.4 times;

[0029] 2. The hardware accelerator of the present application has lower power consumption and area, and the multi-bit-width multiplication unit MMU of the present application reduces the area by 62-83% and the power consumption by 12-74% compared with the prior art;

[0030] 3. The hardware accelerator of the present invention has higher peak energy efficiency and peak area efficiency, improving peak energy efficiency by 2-2.2 times and peak area efficiency by 1.8-2.0 times compared with the prior art;

[0031] 4. The hardware accelerator of this invention supports more quantization schemes with different bit widths. This invention supports at least six quantization schemes: 3b×2b, 4b×2b, 3b×4b, 4b×4b, 6b×2b and 6b×4b.

[0032] 5. This invention reduces resource consumption, computational and storage complexity without sacrificing accuracy. Attached Figure Description

[0033] Figure 1 This is a structural framework diagram of the accelerator of the present invention;

[0034] Figure 2 This is a structural framework diagram of the bit-level multiplication unit (BMU) in an embodiment of the present invention;

[0035] Figure 3 This is a structural framework diagram of the multi-bit wide multiplication unit (MMU) in an embodiment of the present invention;

[0036] Figure 4 This is a structural framework diagram of the high-bit multiplication unit group (MMU Group) in an embodiment of the present invention. Detailed Implementation

[0037] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings:

[0038] Example 1

[0039] like Figure 1 As shown, this invention provides a hardware accelerator for robot grasping, including a top-level controller, n computing array units, an input buffer, a weight buffer, an input router, a weight router, an accumulator (ACC), and an output buffer. The top-level controller manages the top-level timing logic, supervises the accumulator module for accumulation and clearing, and controls the input buffer, weight buffer, and output buffer through read and write operations using enable and address signals. The top-level controller reads the input and weights from the input and weight buffers through enable and address signals. Then, the input and weight routers rearrange the data fed into the input and weight buffers according to various computing modes. The computing array units are responsible for processing tile-based convolutions or deconvolutions of input feature maps. The results are written to the output buffer after passing through the accumulator (ACC) as the input for the next layer.

[0040] As preferred in the embodiment, the accelerator of the embodiment has 8 computing array units; each of the computing array units is composed of 32 high-bit multiplication unit groups MMU Group, the input of each of the computing array units is the activation of the same position of 16 input channels and the weight of 32 output channels, and the output is the multiply-accumulate result of 32 output channels. The 32 high-bit multiplication unit groups MMU Group of the same computing array share the activation, and different computing arrays share the weight; each of the high-bit multiplication unit groups MMU Group is composed of 16 multi-bit multiplication units MMU. Each of the multi-bit multiplication units MMU is composed of 4 bit-level multiplication units BMU.

[0041] As preferred in the embodiment, the bit-level multiplication unit BMU has four calculation conditions of signed number x signed number, signed number x unsigned number, unsigned number x signed number, and unsigned number x unsigned number, and is uniformly expressed by the result of a 3b x 2b signed number multiplier. The input is 3 bits of activation and 2 bits of weight, and the output is 5 bits of signed number.

[0042] As preferred in the embodiment, the four calculation conditions of the bit-level multiplication unit BMU are respectively expressed as:

[0043]

[0044]

[0045]

[0046]

[0047] In the formula, A2, A1, A0 respectively represent the highest bit, the first bit and the lowest bit of the input value; W1, W0 respectively represent the highest bit and the lowest bit of the weight; represents the highest bit of the input value taking inversion; represents the highest bit of the weight taking inversion.

[0048] In the formula, A2, A1, A0 respectively represent the highest bit, the first bit and the lowest bit of the input value; W1, W0 respectively represent the highest bit and the lowest bit of the weight;

[0049] As preferred in the embodiment, the four calculation conditions of the bit-level multiplication unit BMU are uniformly expressed as:

[0050]

[0051] In the formula, S A , S W represents an extra sign bit, which is 1 for a signed number and 0 for an unsigned number.

[0052] As preferred in the embodiment, as shown in Figure 2 Each of the bit-level multiplication units BMU includes two half adders HA and two selectors, and the embodiment further reduces the hardware redundancy of the bit-level multiplication units BMU.

[0053] As preferred in the embodiment, as shown in Figure 3 The multi-bit-width multiplication unit MMU is formed by combining four bit-level multiplication units BMU and shift-add logic according to different calculation modes, as shown in Figure 3 A H A L is the high / low 3 bits of the activation, W H W L is the high / low 2 bits of the weight.

[0054] As preferred in the embodiment, the multi-bit-width multiplication unit MMU implements five precision calculation modes, i.e. 3b×2b, 4b×2b, 4b×4b, 6b×2b and 6b×4b; wherein the 3bit input is a signed number, and the 4bit input is a non-negative signed number, the input is 12bits of activation and 8bits of weight, and the output is a multiplication result less than or equal to 10bits, as shown in Table 1.

[0055]

[0056] 6b×4b: Four BMUs generate a 10-bit result, wherein Figure 3 BMU0 is left shifted by 6 bits, BMU1 is left shifted by 3 bits, BMU2 is left shifted by 4 bits, and BMU3 is left shifted by 0 bit.

[0057] 6b×2b: Two BMUs of each row respectively obtain an 8-bit result, and the calculation parallelism is 2. BMU0 and BMU2 are left shifted by 3 bits, and BMU1 and BMU3 are left shifted by 0 bit.

[0058] 3b(4b)×4b: Two BMUs of each column respectively obtain a 7-bit result, and the calculation parallelism is 2. BMU0 and BMU1 are left shifted by 2 bits, and BMU2 and BMU3 are left shifted by 0 bit.

[0059] 3b(4b)×2b: Four BMUs respectively obtain a 5(6)-bit result, and the calculation parallelism is 4. No shift is needed.

[0060] To solve the problem of unbalanced output data, the four bit-level multiplication units BMU in the multi-bit multiplication unit MMU of the embodiment accumulate the partial sum in the dimension of input channel. In this way, the output data of all calculation modes can be less than or equal to 10 bits, and the pressure of data storage can be reduced. To solve the problem of unbalanced input data, in the high-precision calculation mode, the data is read from the buffer buffer every two cycles, half of which is sent to the calculation array for current calculation, and the other half is saved through the register as the input value of the next cycle. In this way, the input data of all calculation modes can be 20 bits, and the pressure of data transmission can be reduced.

[0061] As preferred in the embodiment, the calculation parallelism in each multi-bit multiplication unit MMU is given to the dimension of input channel, and the partial sum of the plurality of bit-level multiplication units BMU in each multi-bit multiplication unit MMU is accumulated in the dimension of input channel.

[0062] As preferred in the embodiment, the input of the high-bit multiplication unit group MMU Group is activated from 16 input channels; then the multi-bit multiplication units MMU in the high-bit multiplication unit group MMU Group are reorganized; and because the bit-level multiplication units BMU in the same position in different multi-bit multiplication units MMU have the same shift logic, the results of the bit-level multiplication units BMU in the same position in different multi-bit multiplication units MMU are added and then sent to the shift logic to obtain the output, as shown in Figure 4 To reduce the hardware overhead of shift-add logic in the multi-bit multiplication unit MMU, the embodiment reorganizes 16 multi-bit multiplication units MMU in the high-bit multiplication unit group MMU GROUP; for example, when 6b x 4b, the bit-level multiplication unit BMU0 of each multi-bit multiplication unit MMU needs to be left shifted by 6 bits. Therefore, the embodiment adds the results of the bit-level multiplication units BMU in the same position in different multi-bit multiplication units MMU and then sends them to the shift logic to obtain the output. In this way, the multi-bit multiplication units MMU in the high-bit multiplication unit group MMU GROUP can share the cost of shift-add logic.

[0063] Embodiment 2

[0064] To demonstrate the support of the hardware accelerator designed in Embodiment 1 for the robot grasping task, in this embodiment, the grasping detection network GR-ConvNet is mixed with low-bit quantization in the supported scheme, including all the calculation modes that the calculation array can support. Considering the balance between accuracy and calculation complexity, the first convolutional layer is quantized to 6-bit activation and 4-bit weight, the second convolutional layer is quantized to 6-bit activation and 2-bit weight, and the third convolutional layer is quantized to 4-bit activation and 2-bit weight. The intermediate layers are all quantized to 6-bit activation and 4-bit weight. The output layer is quantized to 4-bit activation and 4-bit weight. This embodiment uses the Cornell grasping dataset to evaluate the performance of the grasping detection network, and the input resolution is 640x480. The experimental results show that the training accuracy before and after quantization is 0.96, and the influence of the proposed quantization scheme on performance can be ignored.

[0065] The hardware accelerator in this embodiment reduces the calculation delay from 20ms (before quantization) to 4.5ms, achieving a speedup of 4.4x. The peak energy efficiency is 10.74TOPS / W, and the peak area efficiency is 1.41TOPS / mm 2 ).

[0066] Table 2 compares with other processing units in the 4bx4b calculation mode

[0067] Work Area (pm 2 )]]> Power consumption (pW) FU[5] 768 70.1913 C_PE[3] 334 20.7826 Ours 125 18.1529

[0068] The MMU of Embodiment 1 has superior performance in terms of area and power consumption. Compared with the C-PE, the area of the MMU of Embodiment 1 is reduced by 62.6%, and the power consumption is reduced by 12.7%. Compared with the FU, the area of the MMU of Embodiment 1 is reduced by 83.7%, and the power consumption is reduced by 74.2%.

[0069] As shown in Table 3, this embodiment is compared with other variable precision hardware accelerators

[0070] Table 3 compares the accelerator of Embodiment 1 with other variable precision hardware accelerators

[0071]

[0072] The 4b input is a non-negative signed number, and the 3b input is a signed number.

[0073] As can be seen from Table 3, the accelerator of the present embodiment has higher energy efficiency while supporting more different bit-width calculation modes. Among them, the energy efficiency of 4bx2b is equal to the peak energy efficiency of 3bx2b, and the energy efficiency of 6bx2b is equal to the peak energy efficiency of 4bx4b. The present embodiment improves the peak energy efficiency by 2.0 times and 2.2 times, respectively, compared to the prior art in the case of a larger input bit width, and significantly improves the peak area efficiency by 2.0 times and 1.8 times, respectively.

[0074] The above embodiments and descriptions described in the specification are only to illustrate the principles and best embodiments of the present application, and various changes and improvements can be made to the present application without departing from the spirit and scope of the present application, and these changes and improvements all fall within the scope of the claimed present application.

Claims

1. A hardware accelerator for robotic grasping, characterized in that, It comprises a top controller, n computing array units, a buffer unit and a router unit; the top controller is used for managing the timing logic of the top layer and controlling the buffer unit through reading and writing of enabling and addressing signals; each computing array unit is composed of m high-bit multiplication unit groups (MMU Group), and m high-bit multiplication unit groups of the same computing array unit share activation, and weights are shared between different computing array units; The computing array unit is responsible for processing tile-based convolution or input feature mapping deconvolution and outputs through an output buffer as the input of the next layer. Each high-bit multiplication unit group (MMU Group) is composed of k multi-bit multiplication units (MMU); Each multi-bit multiplication unit (MMU) is formed by combining a plurality of bit-level multiplication units (BMU) and shift-add logic according to different calculation modes, and each bit-level multiplication unit (BMU) comprises two half adders and two selectors; The input activation of the high-bit multiplication unit group (MMU Group) comes from k input channels, then the multi-bit multiplication units (MMU) are reorganized in the high-bit multiplication unit group (MMU Group), and because the bit-level multiplication units (BMU) in the same position in different multi-bit multiplication units (MMU) have the same shift logic, the results of the bit-level multiplication units (BMU) in the same position in different multi-bit multiplication units (MMU) are added, and then sent to the shift logic to obtain the output; The calculation parallelism of each multi-bit multiplication unit (MMU) is given to the input channel dimension, and the plurality of bit-level multiplication units (BMU) in each multi-bit multiplication unit (MMU) perform partial sum accumulation in the input channel dimension; The bit-level multiplication unit (BMU) has four calculation conditions of signed number x signed number, signed number x unsigned number, unsigned number x signed number and unsigned number x unsigned number, and is uniformly represented by the result of a 3b x 2b signed number multiplier; the four calculation conditions of the bit-level multiplication unit (BMU) are uniformly represented as: wherein, , represents an extra sign bit, 1 for signed number and 0 for unsigned number; respectively represent the most significant bit, the first bit and the least significant bit of the input value; respectively represent the most significant bit and the least significant bit of the weight.

2. The hardware accelerator for robotic grasping of claim 1, wherein: The buffer unit comprises an input buffer and a weight buffer, the router unit comprises an input router and a weight router, the top controller reads input and weight from the input buffer and the weight buffer through enabling and addressing signals, and then the input router and the weight router rearrange the data sent by the input buffer and the weight buffer according to various calculation modes.

3. The hardware accelerator for robotic grasping of claim 2, wherein: The accelerator further comprises an accumulator (ACC), and the top controller is further used for supervising the accumulator (ACC) to accumulate and clear, and writing the result processed by the computing array unit into the output buffer through the accumulator (ACC) as the input of the next layer.

4. The hardware accelerator for robotic grasping of claim 1, wherein: The multi-bit multiplication unit MMU realizes five precision calculation modes, namely 3b×2b, 4b×2b, 4b×4b, 6b×2b and 6b×4b; wherein the 3bit input is a signed number, and the 4bit input is a non-negative signed number, the input is 12bits of activation and 8bits of weight, and the output is a multiplication result less than or equal to 10bits.

5. The hardware accelerator for robotic grasping of claim 1, wherein: The four calculation conditions of the bit-level multiplication unit BMU are respectively represented as: (1) (2) (3) (4) wherein respectively represent the most significant bit, the first bit and the least significant bit of the input value; respectively represent the most significant bit and the least significant bit of the weight; represents the most significant bit of the input value negated; represents the most significant bit of the weight negated; In formula (2) to formula (4), high bit inversion and bit deviation are added relative to formula (1).

Images