Deep learning convolution acceleration method and processor using bit-level sparsity

Abstract

The present invention proposes a deep learning convolution acceleration method and processor utilizing bit-level sparsity, comprising: obtaining multiple groups of data pairs to be convolved, summing the exponents of the activation values ​​and original weights in each group of data pairs to obtain the exponential sum of each group of data pairs, and selecting the exponential sum with the largest value from all data pairs as the maximum exponent; arranging the mantissas of the original weights in a calculation order to form a weight matrix, and uniformly aligning each row in the weight matrix to the maximum exponent to obtain an alignment matrix; removing slack bits in the alignment matrix to obtain a simplified matrix, and filling the vacancies in the basic bits of each column of the simplified matrix in a calculation order to form an intermediate matrix, removing the empty rows of the intermediate matrix, setting the empty positions in the matrix to 0 to obtain an interleaved weight matrix, sending the mantissas of the weight segments and corresponding activation values ​​in each row of the interleaved weight matrix to an addition tree for summation processing, and performing shift addition on the processing results to obtain an output feature map as the convolution result of the multiple groups of data pairs.

Description

Technical Field

[0001] The present invention relates to the field of deep learning accelerator design, and in particular to a deep learning acceleration method and an intelligent processor utilizing bit-level sparsity. Background Art

[0002] To achieve higher accuracy, deep learning models are constantly increasing in size. Correspondingly, the performance of deep learning accelerators should keep pace. However, due to limitations such as battery life, power budget, and cost, hardware designers are reluctant to invest more computing resources in response to the development of DNNs (deep neural networks), especially on embedded devices such as robots, drones, and smartphones. Therefore, improving accelerator efficiency is highly desirable in both high-performance and low-power scenarios.

[0003] Previous research has focused on maximizing the potential of weight / activation sparsity and executing as many efficient MACs (Multiply-Accumulate operations) in parallel as possible. However, sparsity is not always sufficient and varies across different models, and even across layers within the same model. For example, activation values ​​exhibit increased sparsity due to nonlinear activation functions. However, for weights, sparsity is typically very low, except when trained using the L1 criterion. Furthermore, even activation values ​​can only produce zero values ​​when passed through functions such as ReLU or PReLu. To address this challenge, some work has created more sparse space for pruning by identifying near-zero values ​​in the operand set or implementing tedious sparse (re)training.

[0004] Past work has proposed a series of bit-serial accelerators that exploit the rich bit-level sparsity to varying degrees. Figure 1 The computational paradigms of three types of accelerators (PEs) are compared through examples. Figure 1 a) and the bit-serial accelerator use bit-level arithmetic with the same value to compute inner products. For example, decomposing an 8b×8b product into 8 1b×8b products is done serially ( Figure 1 Step (b) ) organization and input weights to produce the same results. Figure 1 (c) is a calculation example of the present invention. Figure 1Comparison of the computational distribution of the bit-interleaved PE in fixed-point mode with the previous bit-parallel / serial PE. The white background marks sparse bits (0 bits), and the gray background marks basic bits (1 bits). In (a) bit-parallel PE, Step 1 organizes the weights in parallel; Step 2 performs the MAC. In (b) bit-serial PE, Step 1 organizes the weights in serial; Step 2 synchronizes the values ​​of the necessary bits; and Step 3 performs a "bit-serial" MAC. In (c) bit-interleaved PE, Step 1 organizes the weights in parallel, but Step 2 performs a serial MAC along the values ​​of each bit without synchronization.

[0005] However, the space for exploring value sparsity has reached its limit. From a software perspective, if lossless accuracy is the primary design priority, there is a significant margin that the compression rate cannot exceed. Regardless of the pruning method used, a considerable amount of time is required to explore this margin to balance model accuracy and size. From a hardware implementation perspective, exploiting value sparsity also inevitably leads to more complex accelerator designs. For example, expanding its storage system to accommodate the growing exponential size comes at the cost of increased memory accesses and reduced peak computational throughput.

[0006] At the same time, there are other problems with existing technologies. Figure 1 (a), in order to release the maximum potential of bit sparsity, it is best to skip as many zero bits as possible. However, the position of the zero bit in each 8-bit operand is difficult to predict, especially after fixed-point quantization. The reason is that after quantization, the limited bit width is fully utilized to represent the numerical range, making the zero bits arbitrarily interleaved with the necessary 1 bits. In order to fully utilize the bit sparsity of the parameters themselves, careful synchronization operations need to be performed, such as Figure 1 Steps in (b) As shown, in the final determination step Before the bit-serial MAC in the MAC is executed, synchronization must be performed.

[0007] Previously used synchronization methods include intermediate dense scheduling and hardware-level Booth coding. However, the key weakness of these methods stems from the difficulty in determining a unified pattern to describe the location of synchronization sparsity. A direct consequence is that the ongoing MAC operation must be stopped to adjust the importance of the bit, which comes at the cost of reduced throughput compared to the bit-parallel counterparts. For example, in Figure 1In (b), the MAC must be completed before the MAC is complete. Hardware implementation complexity also increases, as Booth encoding requires additional circuitry to encode and store the weight bits. Another weakness is that this serialized organization cannot support floating-point operations, severely limiting the use cases of bit-serial accelerators and preventing their deployment in a wider range of scenarios. Summary of the Invention

[0008] The purpose of this invention is to solve the efficiency and versatility problems of the above-mentioned existing deep learning accelerator design. It proposes a computing method that utilizes bit-sparse parallelism - the "bit interleaving" computing method, and designs a hardware accelerator - Bitlet that implements the "bit interleaving" computing method.

[0009] To address the shortcomings of the existing technology, the present invention proposes a deep learning convolution acceleration method that utilizes bit-level sparsity, including:

[0010] Step 1: Get multiple data pairs to be convolved, each data pair consists of an activation value and its corresponding original weight, and both the activation value and the original weight are floating point numbers;

[0011] Step 2: Sum the exponents of the activation values ​​and the original weights in each data pair to obtain the exponential sum of each data pair, and select the exponential sum with the largest value from all data pairs as the maximum exponent;

[0012] Step 3: Arrange the mantissas of the original weights in the order of calculation to form a weight matrix, and align each row in the weight matrix to the maximum exponent to obtain an alignment matrix;

[0013] Step 4: Remove the slack bits in the alignment matrix to obtain a simplified matrix with vacancies, and fill the vacancies with the basic bits of each column of the simplified matrix in the calculation order to form an intermediate matrix. After removing the empty rows of the intermediate matrix, set the empty positions in the matrix to 0 to obtain an interleaved weight matrix, and use each row in the interleaved weight matrix as the required weight.

[0014] Step 5. According to the correspondence between the activation value and the basic bit in the original weight, the position information of each activation value corresponding to the necessary weight is obtained, and the necessary weight is sent to the split accumulator. The split accumulator divides the necessary weight into multiple weight segments by bit. According to the position information, the weight segment and the mantissa of the corresponding activation value are sent to the addition tree for summation processing. By performing shift addition on the processing results, the output feature map is obtained as the convolution result of the multiple data pairs.

[0015] The deep learning convolution acceleration method utilizing bit-level sparsity, wherein the activation value is the pixel value of the image.

[0016] The present invention also proposes a processor for implementing the above-mentioned deep learning convolution acceleration method utilizing bit-level sparsity.

[0017] The processor includes:

[0018] A preprocessing module is used to obtain multiple data pairs to be convolved, where each data pair consists of an activation value and its corresponding original weight, and both the activation value and the original weight are floating-point numbers; and the exponential sum of the activation value and the original weight in each data pair is summed to obtain the exponential sum of each data pair, and the exponential sum with the largest value is selected from all data pairs as the maximum exponent;

[0019] An exponent alignment module is used to arrange the mantissas of the original weights in the order of calculation to form a weight matrix, and to align each row in the weight matrix to the maximum exponent to obtain an alignment matrix;

[0020] A weight interleaving module is configured to remove slack bits from the alignment matrix to obtain a reduced matrix with vacancies, and to fill the vacancies with the basic bits of each column of the reduced matrix in the calculation order to form an intermediate matrix. After removing the empty rows of the intermediate matrix, the empty positions in the matrix are set to 0 to obtain an interleaved weight matrix, and each row in the interleaved weight matrix is ​​used as a necessary weight.

[0021] A circular register is used to extract the basic bit in the required weight and obtain the position information of the corresponding activation value of each bit in the required weight from the mantissa of all activation values;

[0022] A split accumulator is used to split the necessary weight into multiple weight segments by bit. According to the position information, the weight segment and the mantissa of the corresponding activation value are sent to the addition tree for summation processing. By performing shift addition on the processing results, the output feature map is obtained as the convolution result of the multiple groups of data pairs.

[0023] The processor, wherein the activation value is a pixel value of the image.

[0024] From the above scheme, it can be seen that the advantages of the present invention are:

[0025] (1) Compared with the latest high-performance GPUs, the energy efficiency of training / inference is improved by 81 times / 21 times respectively;

[0026] (2) Compared with the most advanced fixed-point accelerator, the speed and efficiency are improved by 15 times and 8 times respectively;

[0027] (3) The designed accelerator has an area of ​​1.5 mm2. Under TSMC 28nm process, the accelerator area is 0.039 square millimeters, and the power consumption is 570 mW (32-bit floating point mode), 432 mW (16-bit fixed point mode), and 365 mW (8-bit fixed point mode).

[0028] (4) The accelerator is highly configurable. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Comparison chart of bit-interleaved PE and bit-parallel / serial PE calculation distribution in fixed-point mode;

[0030] Figure 2 It is a schematic diagram of sparse parallelism;

[0031] Figure 3 This is a conceptual diagram of bit interleaving;

[0032] Figure 4 This is the structural diagram of the BCE module;

[0033] Figure 5 This is the structural diagram of the Bitlet accelerator. DETAILED DESCRIPTION

[0034] The weaknesses of the aforementioned techniques are primarily due to the sparsity of the exploited values. During the research for this invention, we discovered that "bit sparsity" is an inherently finer sparsity, targeting "zero bits" in each operand rather than coarse-grained zero values. Using floating-point or fixed-point numbers to represent weights or activations, the percentage of zero bits can reach 45% to 77% across different DNN models. Skipping zero bits in operands does not affect the results, which means that if efficient computation is performed at the bit level, speedups can be achieved directly without any software effort. Therefore, this invention leverages the richness of bit-level sparse parallelism to accelerate both training and inference, serving general-purpose deep learning in the cloud and edge.

[0035]

[0036]

[0037] Table 1

[0038] Table 1 above categorizes state-of-the-art sparsity-based accelerators. Early bit-parallel accelerators, such as Cambricon and SCNN, focused their sparsity research solely on numeric values. Software-level pruning creates a wider space for zero sparsity, unlocking the potential of these accelerators. Given that bit sparsity is more abundant in weights and activations, recent research on bit-serial accelerators has focused on sparsity at the bit level. Laconic, a recent implementation, uses "terms" after Booth encoding to serially extract necessary bits and proposes a low-cost LPE to mitigate the increased power consumption caused by frequent encoding / decoding. Tactical addresses sparsity at the bit level of weights and activations. Its design concept is similar to Pragmatic, both utilizing zero-bit skipping to optimize inefficient operations. However, Tactical relies on a data-independent front-end to skip zero weights and a software scheduler to maximize the probability of weight skipping. Several sparsity design patterns also follow the bit-serial approach. For example, Stripes and UNPU implement bit-serialization of fixed-point operands without circumventing sparsity. Bit-fusion supports faster spatial and temporal combinations to accelerate bit serialization, but still cannot make good use of the sparsity of bits.

[0039] While previous work has demonstrated that bit-level sparsity is rich, previous work only focused on exploring strategies for skipping zero bits within a specific weight, without exploring sparsity across weights.

[0040] like Figure 2 As shown in the figure, each point represents the proportion of zero bits in the bit lane of all weights in this convolution kernel. The figure shows that approximately 50% of the bits in all convolution kernels are 0. On the x-axis of the figure, sparsity only includes the mantissa (23 / 10 bits in floating point 32 / 16 bits). In the int8 bit precision representation, only 7 significant bits are included, excluding the sign bit. Figure 2The bit sparsity of different convolution kernels is shown, and the weight sparsity is consistent across all bit values. The X-axis represents the mantissa bit value, totaling 23 bits, excluding the hidden 1 bit in the standard floating-point 32 format. Each dot represents the percentage of zero bits at that exponent within a convolution kernel. For example, ResNet152 and MobileNetV2 show a significant clustering in the first half of the mantissa (bits 0 to 16), indicating that the number of 0s and 1s at this bit value is nearly equal. This facilitates parallel loading of weights into the accelerator and serial computation. Furthermore, starting from bits 17 to 23, these points largely overlap at 100% on the Y-axis (the long tail in fp32 numbers), indicating that the majority of bits are zero. Because floating-point multipliers are designed to cover any operand case, they do not distinguish between these suboptimal cases. This is the fundamental reason why floating-point multiply-add operations and convolution operations (MACs) are difficult to accelerate.

[0041] Despite the success of fixed-point representation for efficient DNN inference, accelerators designed for fixed-point precision are limited to inference, making them difficult to use in general scenarios. For example, DNN training still relies on floating-point backpropagation to ensure that the model is scaled to floating point, but still needs to meet real-time requirements, especially when fixed-point precision cannot meet the corresponding accuracy. Ideally, an accelerator should be applicable to most use cases and provide sufficient convenience and flexibility for end-user collaboration.

[0042] Based on the above exploration, the present invention proposes a sparse parallel design pattern based on bit interleaving. The advantage of the bit serial accelerator is that it effectively utilizes the sparsity of bits. However, the throughput provided by the bit serial accelerator is relatively lower than that of its corresponding bit parallel accelerator. Based on these two design concepts, the present invention proposes a bit interleaving design, which combines the advantages of the above designs and avoids the disadvantages. This design pattern can significantly surpass the previous bit serial / parallel pattern. The accelerator Bitlet adopts the design concept of bit interleaving, and also supports multiple precisions including floating point and fixed point. This configurable feature makes Bitlet suitable for both high performance and low power consumption scenarios.

[0043] In order to make the above features and effects of the present invention more clearly understood, embodiments are given below and described in detail with reference to the accompanying drawings.

[0044] The following is a detailed explanation of the present invention:

[0045] 1. "Bit Interleaving"

[0046] Without loss of generality, a floating-point operand consists of three parts: a sign bit, a mantissa, and an exponent. This follows the IEEE 754 standard, which is the most commonly used floating-point standard in the industry. If we use single-precision floating-point numbers (fp32), the mantissa is 23 bits wide, the exponent is 8 bits wide, and the remaining one bit is the sign bit. A single-precision floating-point weight can be expressed as: fp = (-1) s 1.m×2 e-127 , the size of e is the actual position of the floating point decimal point plus 127. We use a series of floating point 32-bit single precision MACs to calculate the partial sums of the convolution.

[0047]

[0048] Formula 1: W i Converted to fp32 representation, where and for and Simplified expression of . Including the hidden mantissa 1, but actually in memory, according to the IEEE-754 standard, this bit is hidden. Is the mantissa, fixed width - a total of 24 bits, so further decomposition You can get the partial sum represented by bits:

[0049]

[0050] in yes The bth bit of the binary representation. Represent A using the IEEE-754 binary format. i , Formula 2 can be rewritten as Formula 3. In addition, let Then Formula 3 can be rewritten as

[0051]

[0052] According to Equation 5, it can be inferred that the N fp32 MAC results are equivalent to a series of bit-level operations on the corresponding mantissa. Specifically, if Then the sum of N MACs is converted into N signed The sum of ( ), and on this basis, move left (right)

[0053] The above analysis shows that: considering the sparsity, the partial sum of floating point numbers can be converted into a bit-level operation. The product is mainly composed of the mantissa But whether it contributes to the product is determined by the formula in 5 This bit-level sparsity can also be exploited in bit interleaving. There is a significant proportion of zero bits in each bit value, so if But another weight W on the same bit value b j If it is bit 1, you can use replace The different weight bits are interleaved on the same bit row. In the same cycle, the mantissa and Participate in partial sum operations, that is, use sparsity to speed up the calculation.

[0054] This calculation theory also includes fixed-point accuracy. In formula 5, E max and E i -E max This is not necessary because fixed-point precision representation does not have an exponent. This invention will describe in detail how bit interleaving works under floating-point 32-bit precision weights, as well as the design details of a Bitlet accelerator that supports multiple precisions.

[0055] Figure 1 (c) illustrates the bit interleaving process for an 8-bit fixed-point MAC, with a step-by-step demonstration. However, in practice, floating-point MACs are not as easily exploited as fixed-point MACs because of the presence of a special part in the binary operands—the exponent—and different operands have different exponents. To maximize the potential of floating-point sparsity, bit interleaving, based on Equation 5, consists of three separate but sequential steps.

[0056] Preprocessing

[0057] Figure 3 (a) Using an example as an illustration, six ordinary 32-bit floating point weights are arranged in a row, and the exponent and mantissa of each weight are arbitrary. The triangle mark indicates the actual position of the binary decimal point. For simplicity, the actual 32-bit floating point representation stored in memory is not shown in binary format, but a more representative representation method is used to express the value. For example, 0.012 with E5=-2 represents 0.25 (W5) in decimal. This step is the same as Figure 1 Steps in c The interleaving is similar, except that the 32-bit floating-point weights are organized in parallel. These binary weights are then preprocessed to obtain their respective exponents, further determining the "maximum" exponent (E6 in this example). The mantissa is also stored for later MAC calculation. To simplify the presentation, the tail bits of each mantissa (bits 9-23) are omitted.

[0058] :Dynamic index alignment:

[0059] The exponent indicates the position of the decimal point in the binary representation. Traditionally, this involves the "exponent alignment" step in floating-point addition. However, in bit-interleaving, we align the exponents of a group of floating-point numbers by aligning them to a maximum value (E6 in this case) rather than doing it one by one. This step is called "dynamic exponent alignment" and Figure 1 This step is not involved in (c) because fixed-point values ​​have no exponent.

[0060] Looking back at Equation 5, in actual implementation, the two summations can be performed in parallel. The outer summation represents Figure 3 The vertical dimension in (a) is the N weights and their corresponding activation values; the internal summation represents the horizontal dimension, which is the different bit widths of the mantissa. From this perspective, the key concept of Formula 5 is along Figure 3 The two dimensions in (a) calculate all of

[0061] Since our ultimate goal is to calculate This involves the calculation of N weights and activation values. Therefore, all indices are aligned to their maximum values ​​in each execution instead of matching them one by one. Figure 3 As can be seen in (b), all six weights are aligned to the largest exponent, W6. For example, W5 needs to be right-shifted 8 bits to align with W6. The advantage of this is that the exponent alignment of all six weights only needs to be performed once, saving time and resources for efficient hardware implementation.

[0062] :Necessary Bit Extraction

[0063] At present, the key is how to use the necessary bits to obtain accurate partial sums and further obtain better inference speed. Considering the sparse parallelism mentioned above, this step uses this feature to extract the necessary bits, which is similar to Figure 1 Steps in c Exactly the same.

[0064] like Figure 3 As shown in (c), if we efficiently extract the necessary bit 1, the total computational effort can be reduced from a 6-operand MAC to only 3. Still taking W6 as an example, the W6 index is 6, and the first bit (b=0) is the necessary bit 1. Inspired by Formula 5, the bit Value equal to 2 6 , which means that the bit is located at the 7th position before the binary point. For W1 to W5, after alignment 2 6 If the first bit of W6 is moved up to replace the position of the same vertical track in W1, then A can be calculated at the same time. 6 ×2 6+A 1 ×2 3 The necessary bits belonging to other weights can also be operated in the same way, and the extracted weights are finally seen Figure 3 (c). In summary, these two steps accelerate MACs for floating-point 32-bit precision computations in two ways: (1) avoiding the computationally expensive exponent alignment operation; and (2) eliminating the ineffective computation caused by 0 bits by leveraging sparse parallelism.

[0065] 2. Bitlet Accelerator

[0066] To perform bit interleaving, we designed a new accelerator, named Bitlet. In this section, we will describe the key hardware design modules of Bitlet, including the microarchitecture that supports multi-precision computing engines and the overall architecture for efficient memory access.

[0067] Key Module 1 - Preprocessing Module. First, the present invention designs a component involving two steps in the "bit interleaving" operation. Bitlet inputs multiple pairs of weights and activation values, Figure 4 In the Bitlet Computing Component (BCE), W0 to W N-1 is the original weight, and A0~A N-1 is the corresponding activation. The preprocessing module converts each W i and A i Decompose into two parts: mantissa and exponent, and perform for each A / W pair After that, select the maximum exponent E max And stored in the register for subsequent dynamic index alignment operations. max back, Moved left (right) bit, so that its exponent is equal to E max Consistent. Figure 3 As an example, the weight in E max E6=6 in bit W6, and all other weights are aligned with E6, i.e. Will be shifted 6-0=6 bits, such as Figure 4 At the same time, the left shift position will be automatically filled with 0, because the mantissa is 24 bits long, so the tail exceeding b=23 will be discarded.

[0068] Key module 2 - Line coordinator. After dynamic exponent alignment, we get a 24-bit shifted mantissa, which is The mantissa will be further fed into another module, Figure 4 It is called "line coordinator" in , which is used to group the same bit values ​​together and reorganize the lines to output the matrix in columns. The output of the coordinator is expressed as The range of b is 0 to 23. This module does not contain any combinational or sequential logic and only performs aggregation and transposition operations on the aligned mantissa. Therefore, intuitively, this module does not introduce significant power consumption.

[0069] Key module 3 - Circular register RR-reg. RR-reg extracts the necessary bit 1 (basic bit) in the interleaved weight and selects the output of BCE from N activation mantissas. Each RR-reg has its own internal clock and is connected to the accelerator clock tree. Figure 4 The pseudo code shown represents the specific procedure: RR-reg first extracts the necessary bits 1 in sequence according to the input bits. The "select" signal instructs the decoding component to configure the activation value path and output O i If the necessary bit 1 is not detected, RR-reg will activate the "fill 0" signal. i The output will also be 0. The "fill 0" signal operation is applicable to the case where all bits in a bit row are 0, that is, Figure 3 (c) Scenario with b=1 or 2.

[0070] BCE has three key advantages: 1. This architecture does not incur precision loss, as the dynamic exponent alignment described above is identical to floating-point operations in IEEE 754. The rightmost bits after the shift are discarded, but these bits are negligible due to their small value and have no impact on accuracy. 2. BCE does not require any preprocessing to account for parameter sparsity. Figure 4 The preprocessing module in is only responsible for converting the activation value of the weight into the corresponding mantissa and exponent. In the actual RTL implementation, each RR-reg will implement a sliding window to achieve automatic interleaving and extract the necessary bits. Benefiting from the favorable conditions of sparse parallelism, the processing can be completed almost simultaneously in each RR-reg. ③ In addition to RR-reg, BCE is mainly composed of combinational circuits, but does not involve complex circuits that may cause the critical path delay to become longer. Each RR-reg generates an output O in each clock cycle. i However, compared with the traditional one-to-one MAC, the total cycle of calculating the partial sum is greatly optimized. N is the only design parameter in BCE. A larger N is beneficial for extracting more bit 1s.

[0071] 3. Accelerator Architecture

[0072] PE: Bitlet is composed of mesh-connected PEs. Figure 5 As shown, each PE consists of a BCE and an adder tree. The BCE connects the on-chip cache and the adder tree. Each PE serially inputs N weights and activations and generates a partial sum Oi As the input of the adder tree. Since the BCE output is limited to 24-bit mantissa, the input of the adder tree is also 24. PE is multiplied by (Note that b is a negative number) to finalize the result to ensure the correctness of the result. Can be decomposed into a fixed part b and a common part E max , used to generate BCE output. The fixed part is executed by a fixed number of shifts. max Simply perform the calculation on the result in the accumulator. i Only fixed-point addition of the mantissa of the activation value is required, without any multiplication, which means that the arithmetic complexity and power consumption are also optimized accordingly.

[0073] Memory System: To achieve high throughput, the Bitlet Accelerator provides separate DMA channels for activations and weights. Figure 5 As shown, the local cache stores data retrieved from DDR3 memory and provides sufficient bandwidth for access by the corresponding Bitlet PE. In the RTL implementation, the bandwidth per channel between the memory and the local cache reaches 12.8 GB / s, and the PE array can utilize a total of 25.6 GB / s of bandwidth to retrieve activation and weight data from the local cache. In terms of data flow, the Bitlet utilizes weight fixation and activation broadcast mechanisms to reduce main memory accesses.

[0074] 4. Bitlet Flexibility

[0075] The Bitlet accelerator supports multi-precision operations. It can be easily configured in fixed-point mode, providing sufficient flexibility for end users. For example, if you want to use 16-bit fixed-point precision, you can use the pre-processing module (attached) that performs exponent alignment and shifting. Figure 4 in ">> ) part of the gate, and let the input W i Directly connected to the line coordinator. Bitlet was originally designed to support 24-bit mantissa, so if 16-bit fixed-point precision is used, only RR-reg0~RR-reg 15 Other RR-regs can be safely disabled or left empty. INT8 quantization or any other target precision (i.e., INT4, INT9, etc.) is handled similarly. Therefore, end users do not need to rely on other precision-specific accelerators to accommodate different use cases. Users can freely configure DNNs on this platform to meet the trade-off between accuracy goals and power / performance.

[0076] The following is a system embodiment corresponding to the above method embodiment. This embodiment can be implemented in conjunction with the above embodiment. The relevant technical details mentioned in the above embodiment are still valid in this embodiment and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiment.

[0077] The present invention also proposes a processor for implementing the above-mentioned deep learning convolution acceleration method utilizing bit-level sparsity.

[0078] The processor includes:

[0079] A preprocessing module is used to obtain multiple data pairs to be convolved, where each data pair consists of an activation value and its corresponding original weight, and both the activation value and the original weight are floating-point numbers; and the exponential sum of the activation value and the original weight in each data pair is summed to obtain the exponential sum of each data pair, and the exponential sum with the largest value is selected from all data pairs as the maximum exponent;

[0080] An exponent alignment module is used to arrange the mantissas of the original weights in the order of calculation to form a weight matrix, and to align each row in the weight matrix to the maximum exponent to obtain an alignment matrix;

[0081] A weight interleaving module is configured to remove slack bits from the alignment matrix to obtain a reduced matrix with vacancies, and to fill the vacancies with the basic bits of each column of the reduced matrix in the calculation order to form an intermediate matrix. After removing the empty rows of the intermediate matrix, the empty positions in the matrix are set to 0 to obtain an interleaved weight matrix, and each row in the interleaved weight matrix is ​​used as a necessary weight.

[0082] A circular register is used to extract the basic bit in the required weight and obtain the position information of the corresponding activation value of each bit in the required weight from the mantissa of all activation values;

[0083] A split accumulator is used to split the necessary weight into multiple weight segments by bit. According to the position information, the weight segment and the mantissa of the corresponding activation value are sent to the addition tree for summation processing. By performing shift addition on the processing results, the output feature map is obtained as the convolution result of the multiple groups of data pairs.

[0084] The processor, wherein the activation value is a pixel value of the image.

Claims

1. A deep learning convolution acceleration method using bit-level sparsity, characterized in that: include: Step 1: Get multiple data pairs to be convolved, each data pair consists of an activation value and its corresponding original weight, and both the activation value and the original weight are floating point numbers; Step 2: Sum the exponents of the activation values ​​and the original weights in each data pair to obtain the exponential sum of each data pair, and select the exponential sum with the largest value from all data pairs as the maximum exponent; Step 3: Arrange the mantissas of the original weights in the order of calculation to form a weight matrix, and align each row in the weight matrix to the maximum exponent to obtain an alignment matrix; Step 4: Remove the slack bits in the alignment matrix to obtain a simplified matrix with vacancies, and fill the vacancies with the basic bits of each column of the simplified matrix in the calculation order to form an intermediate matrix. After removing the empty rows of the intermediate matrix, set the empty positions in the matrix to 0 to obtain an interleaved weight matrix, and use each row in the interleaved weight matrix as the required weight. Step 5: According to the correspondence between the activation value and the basic bit in the original weight, the position information of each corresponding activation value in the necessary weight is obtained, and the necessary weight is sent to the split accumulator. The split accumulator divides the necessary weight into multiple weight segments by bit. According to the position information, the weight segment and the mantissa of the corresponding activation value are sent to the addition tree for summation processing. By performing shift addition on the processing results, the output feature map is obtained as the convolution result of the multiple data pairs; The activation value is the pixel value of the image.

2. A processor, characterized in that: Used to implement the deep learning convolution acceleration method using bit-level sparsity as claimed in claim 1.

3. The processor according to claim 2, wherein: The processor includes: A preprocessing module is used to obtain multiple data pairs to be convolved, where each data pair consists of an activation value and its corresponding original weight, and both the activation value and the original weight are floating-point numbers; and the exponential sum of the activation value and the original weight in each data pair is summed to obtain the exponential sum of each data pair, and the exponential sum with the largest value is selected from all data pairs as the maximum exponent; An exponent alignment module is used to arrange the mantissas of the original weights in a calculation order to form a weight matrix, and to align each row in the weight matrix to the maximum exponent to obtain an alignment matrix; A weight interleaving module is configured to remove slack bits from the alignment matrix to obtain a reduced matrix with vacancies, and to fill the vacancies with the basic bits of each column of the reduced matrix in the calculation order to form an intermediate matrix. After removing the empty rows of the intermediate matrix, the empty positions in the matrix are set to 0 to obtain an interleaved weight matrix, and each row in the interleaved weight matrix is ​​used as a necessary weight. A circular register is used to extract the basic bit in the required weight and obtain the position information of the corresponding activation value of each bit in the required weight from the mantissa of all activation values; A split accumulator is used to split the necessary weight into multiple weight segments by bit. According to the position information, the weight segment and the mantissa of the corresponding activation value are sent to the addition tree for summation processing. By performing shift addition on the processing results, the output feature map is obtained as the convolution result of the multiple groups of data pairs.

4. The processor according to claim 3, wherein: The activation value is the pixel value of the image.

Images