A mixed precision arithmetic unit, method and array

Abstract

The application relates to a mixed precision operation unit, a method and an array, and belongs to the field of operation unit design. The mixed precision operation unit comprises four bit series-based mixed precision multipliers, three adders and a shifter; the mixed precision multiplier is used for multiplying and adding an activation value with 2-bit precision and a weight value with variable 1-8-bit continuous precision; the shifter is based on the bit precision of the activation value of the mixed precision multiplier and performs corresponding shifting; the three adders are used for performing relevant multiplication and addition operations. The mixed precision operation unit supports multiplication and addition operations of the activation value with variable 2, 4 and 8-bit precision and the weight value with variable 1-8-bit continuous precision through splicing and fusion of the four mixed precision multipliers, and can be expanded into an operation array to improve the parallelism of data operation. While meeting the demand of different precision operations, the mixed precision operation unit greatly reduces the calculation bit width redundancy, power consumption and area overhead, and realizes more fine division of variable bit width precision.

Description

Technical Field

[0001] This invention belongs to the field of arithmetic unit design, and particularly relates to a mixed-precision arithmetic unit, method and array. Background Technology

[0002] In recent years, neural networks have been widely deployed in various deep learning applications. Current CNN convolutional neural networks require billions or even tens of billions of computations and hundreds of megabytes of parameters during a single inference process. Therefore, the urgent need to achieve massive computational demands with relatively limited hardware resources poses a greater challenge to the design of computational units in neural network accelerators.

[0003] Based on existing research findings, there are two main design schemes for arithmetic units: single-precision arithmetic unit design and mixed-precision arithmetic unit design.

[0004] Single-precision arithmetic unit (ALU) design is the most traditional ALU design scheme, supporting only single-precision operations, and the bit width of the ALU must always remain consistent. The advantage of this design is its relative simplicity, but its disadvantages are: for single-precision ALUs using higher bit widths, when processing data in network layers with low precision requirements, significant computational bit width redundancy occurs, and this redundant, ineffective bit width incurs additional area and power consumption overhead; for single-precision ALUs using lower bit widths, when processing data in network layers with high precision requirements, computational precision is lost.

[0005] There are two main approaches to designing mixed-precision computation units (MCUs): The first approach uses a single-bit precision for activation values ​​and continuously variable precision across the entire bit width for weight values. The advantage of this design is that it can handle computations with weight values ​​of any precision across the entire bit width. However, its disadvantage is that the activation value's bit width precision cannot be changed; it must be a single bit width. For example, using an 8-bit activation value and continuously variable 1-8 bit weight values, when processing convolutional layers in the deep convolutional neural network AlexNet, since some convolutional layers in AlexNet only require 4-bit precision activation values, the activation value bit width in the mixed-precision computation unit becomes redundant. This redundant, ineffective bit width introduces additional area and power consumption overhead to the computation unit.

[0006] The second type of mixed-precision computation unit uses discontinuous, variable bit width precision (2, 4, 8 bits or 2, 4, 8, 16 bits) for both activation and weight values. The advantage of this design is that the bit width precision of both activation and weight values ​​is variable. However, its disadvantage is that the bit width precision of both can only vary within a certain range and cannot be continuously variable. For example, when processing convolution operations in a binary network, since it only requires a 1-bit precision weight value, the weight value bit width in the mixed-precision computation unit becomes redundant. This redundant, ineffective bit width introduces additional area and power consumption overhead to the computation unit.

[0007] In summary, single-precision arithmetic units cannot be compatible with operations on data with varying bit widths across different network layers. While current mixed-precision arithmetic units can handle some bit widths across different network layers, their coarse-grained design and insufficient fine-grained division of variable bit widths prevent them from meeting finer-grained mixed-precision arithmetic requirements, such as some convolutional layers requiring only 4-bit activation values ​​or binarized networks requiring only 1-bit weights. Both approaches result in excessive bit width redundancy in the arithmetic units, leading to excessive power consumption and area overhead. Summary of the Invention

[0008] Based on the above analysis, this invention aims to provide a mixed-precision arithmetic unit, method, and array. Based on bit serialization and bit fusion, it supports multiply-accumulate operations with variable precision of 2, 4, and 8 bits for activation values ​​and continuously variable precision of 1-8 bits for weight values. Furthermore, it can be expanded into an arithmetic array to improve the parallelism of data processing. While meeting the needs of different precision operations, it significantly reduces computational bit-width redundancy, power consumption, and area overhead, achieving a more refined division of variable bit-width precision.

[0009] On one hand, the present invention provides a mixed-precision arithmetic unit, including four bit-serial based mixed-precision multipliers, a first, second, and third adder, and a first shifter;

[0010] Each of the mixed-precision multipliers is used to perform a multiplication and addition operation on the activation value and the weight value with 2-bit precision; the first shifter performs a corresponding shift based on the bit precision of the activation value input by the mixed-precision arithmetic unit;

[0011] The first adder has two input terminals connected to the output terminals of two mixed-precision multipliers, and its output terminal connected to one input terminal of the third adder.

[0012] The second adder has two input terminals connected to the output terminals of two other mixed-precision multipliers, and its output terminal connected to the other input terminal of the third adder after passing through the first shifter.

[0013] The third adder is used to output the calculation result of the mixed precision arithmetic unit.

[0014] Furthermore, the first shifter performs a corresponding shift based on the bit precision of the activation value input by the mixed precision arithmetic unit, including: when the bit precision of the activation value input by the mixed precision arithmetic unit is 2 or 4, the shift value of the first shifter is 0; when the bit precision of the activation value input by the mixed precision arithmetic unit is 8, the shift value of the first shifter is 4.

[0015] Furthermore, each of the mixed-precision multipliers is used for multiplying and adding 2-bit precision activation values ​​and weight values. The four mixed-precision multipliers perform multiplying and adding operations on 2, 4, or 8-bit precision activation values ​​and weight values ​​input to the mixed-precision operation unit by grouping them together.

[0016] When the activation value input to the mixed precision arithmetic unit is 2 bits, each mixed precision multiplier is a separate group, performing a multiplication and addition operation on a 2-bit activation value and a weight value respectively. The four mixed precision multipliers complete the multiplication and addition operation on four groups of data in parallel.

[0017] When the activation value input to the mixed precision arithmetic unit has a precision of 4 bits, each pair of mixed precision multipliers forms a group to complete a multiplication and addition operation of a group of data. In each group, one mixed precision multiplier completes a multiplication and addition operation of the lower 2 bits of the activation value and the weight value, and the other mixed precision multiplier completes a multiplication and addition operation of the higher 2 bits of the activation value and the weight value.

[0018] When the activation value input to the mixed-precision arithmetic unit has a precision of 8 bits, the four mixed-precision multipliers are concatenated and merged into one group to jointly complete the multiplication and addition operation of a set of data. Among them, the fourth mixed-precision multiplier completes the multiplication and addition operation of the highest 2-bit activation value and weight value, the third mixed-precision multiplier completes the multiplication and addition operation of the next 2-bit activation value and weight value, the second mixed-precision multiplier completes the multiplication and addition operation of the next 2-bit activation value and weight value, and the first mixed-precision multiplier completes the multiplication and addition operation of the lowest 2-bit activation value and weight value.

[0019] Furthermore, the weight value is a variable weight value with a full width of 1-8 bits.

[0020] Furthermore, the mixed-precision multiplier includes a mux data selector, a fourth adder, a register, a second shifter, and a third shifter;

[0021] The mux data selector is used to perform a selection operation based on the input activation value and weight value in each clock cycle to obtain a partial product;

[0022] The two inputs of the fourth adder are connected to the mux data selector and the high two bits of the register, respectively, and are used to accumulate the partial product and output it to the high three bits of the register.

[0023] The three most significant bits of the register are connected to the output of the fourth adder and are used to cache the accumulated result of the partial product each clock cycle.

[0024] The input and output of the second shifter are both connected to a register, and are used to shift the least significant bit of the partial product of the buffer in each clock cycle and feed it back to the register;

[0025] The input of the third shifter is connected to a register and is used to right-shift the accumulated result of the partial product of all clock cycles to eliminate invalid bit width in the low bits, so as to obtain the operation result of the mixed precision multiplier.

[0026] Furthermore, when the activation value input to the mixed-precision arithmetic unit has a precision of 4 bits, the third shifter in the mixed-precision multiplier that performs the multiplication and addition operation of the high 2 bits of the activation value and the weight value in each group needs to be shifted right by two fewer bits than when performing the multiplication and addition operation of the low 2 bits and the weight value.

[0027] When the activation value input to the mixed precision arithmetic unit is 8 bits, the third shifter in the mixed precision multiplier that performs the multiplication and addition operation of the highest 2 bits activation value and weight value and the multiplication and addition operation of the next 2 bits activation value and weight value needs to be shifted right to eliminate invalid bit width in the low bits. This requires two fewer right shifts than when performing the multiplication and addition operation of the second highest 2 bits activation value and weight value and the multiplication and addition operation of the lowest 2 bits activation value and weight value.

[0028] On the other hand, the present invention provides a mixed-precision calculation method, comprising the following steps:

[0029] Four bit-serial-based mixed-precision multipliers are used to perform multiplication and addition operations on 2-bit precision activation values ​​and 1-8 bit continuous precision variable weight values, respectively.

[0030] The results of the two mixed-precision multipliers are added together to obtain the first result;

[0031] The results of the operations of the other two mixed-precision multipliers are added together to obtain the second result;

[0032] The second result is obtained by shifting the input activation value of the mixed precision arithmetic method using a shifter to obtain the third result.

[0033] The first and third results are added together to obtain the result of the mixed precision arithmetic method.

[0034] Furthermore, shifting the second result using a shifter based on the bit precision of the input activation value of the mixed precision arithmetic method includes shifting the value by 0 when the bit precision of the activation value is 2 or 4, and shifting the value by 4 when the bit precision of the activation value is 8.

[0035] Furthermore, each mixed-precision multiplier is used to perform multiplication and addition operations on the 2-bit activation value and the weight value, including:

[0036] A partial product is obtained by using a mux data selector to perform a selection operation based on a 2-bit activation value and a weight value in each clock cycle;

[0037] In each clock cycle, the partial product is accumulated using an adder, and the accumulated result is cached in a register. The two most significant bits are used in the accumulation of the next cycle, and the least significant bit is slowly shifted to the right. The above operation is repeated until the accumulated result of the partial product for all clock cycles is obtained.

[0038] The accumulated result is shifted to the right to cancel out the invalid bit width in the lower bits, thus obtaining the operation result of the mixed precision multiplier;

[0039] The right shifting of the accumulated result includes: when the input activation value precision of the mixed-precision arithmetic method is 2 bits, each mixed-precision multiplier shifts the accumulated result to the right to cancel the invalid bit width in the lower bits; when the input activation value precision of the mixed-precision arithmetic method is 4 bits, the right shift of the mixed-precision multiplier that completes the multiplication and addition operation of the high 2 bits of the activation value and the weight value requires two fewer right shifts than the right shift of the mixed-precision multiplier that completes the multiplication and addition operation of the low 2 bits of the activation value and the weight value; when the input activation value precision of the mixed-precision arithmetic method is 8 bits, the right shift of the mixed-precision multiplier that completes the multiplication and addition operation of the highest 2 bits of the activation value and the weight value and the second 2 bits of the activation value and the weight value requires two fewer right shifts than the right shift of the mixed-precision multiplier that completes the multiplication and addition operation of the second highest 2 bits of the activation value and the weight value and the lowest 2 bits of the activation value and the weight value.

[0040] On the other hand, the present invention provides a mixed-precision computing array, including four mixed-precision computing units, which are arranged in a one-dimensional vector manner. Each computing unit in the array is responsible for performing convolution operation on one output channel, and the four output channels are grouped together for parallel operation.

[0041] The present invention can achieve at least one of the following beneficial effects:

[0042] 1. Each mixed-precision multiplier supports the calculation of 2-bit precision activation and weight values. By splicing and merging, it supports variable multiplication and addition operations with 2, 4, and 8-bit precision activation values. While meeting the needs of different precision operations, it greatly reduces computational bit width redundancy, power consumption, and area overhead, resulting in higher data utilization and lower computational redundancy in convolution operations.

[0043] 2. Each mixed-precision multiplier supports 2-bit precision activation values ​​and 1-8 bit continuous width variable precision weight values. When processing network convolution operations, especially when processing convolution operations of binarized networks, it avoids redundancy in weight value width and reduces the additional area and power consumption overhead brought by the computing unit.

[0044] 3. The mixed-precision arithmetic unit, through the splicing and fusion of four mixed-precision multipliers, supports multiply-add operations with variable precision of 2, 4, and 8 bits for activation values ​​and continuous variable precision of 1-8 bits for weight values. Compared with traditional arithmetic units, the amount of computation is greatly reduced for the same neural network, improving the parallelism of data operations. Furthermore, it can be expanded into an arithmetic array to further improve the parallelism of data operations and greatly enhance the computational efficiency.

[0045] Other features and advantages of the invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained from what is particularly pointed out in the description, claims, and drawings. Attached Figure Description

[0046] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0047] Figure 1 This is a structural diagram of the mixed-precision computing unit of the present invention;

[0048] Figure 2 This is a structural diagram of the mixed-precision multiplier based on bit serial communication according to the present invention;

[0049] Figure 3 This is a schematic diagram of the operation flow of the bit-serial-based mixed-precision multiplier of the present invention;

[0050] Figure 4 This is a schematic diagram illustrating the multiplication principle of the 2-bit precision activation value and the 4-bit precision weight value in this invention.

[0051] Figure 5 This is a flowchart of the operation of the mixed-precision arithmetic unit in 2-bit mode of the present invention;

[0052] Figure 6 This is a flowchart of the operation of the mixed-precision arithmetic unit in 4-bit mode of the present invention;

[0053] Figure 7 This is a flowchart of the operation of the mixed-precision arithmetic unit in 8-bit mode of the present invention;

[0054] Figure 8 This is a schematic diagram illustrating the calculation principle of the 8-bit precision activation value and the 8-bit precision weight value of the present invention;

[0055] Figure 9 This is a schematic diagram of the mixed-precision arithmetic array of the present invention. Detailed Implementation

[0056] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0057] Device Examples

[0058] One specific embodiment of the present invention discloses a mixed-precision arithmetic unit that supports multiplication and addition operations of activation values ​​with variable precision of 2, 4, and 8 bits and weight values ​​with variable precision of 1-8 bits. Its inputs are activation values ​​with 2, 4, and 8-bit precision and weight values ​​with variable precision of 1-8 bits, and its output is a 16-bit multiplication and addition result.

[0059] Specifically, the mixed-precision computing unit structure is as follows: Figure 1 As shown, it includes four bit-serial based mixed-precision multipliers, three adders (first, second, and third adders) and a first shifter.

[0060] Mixed-precision multiplier 1 and mixed-precision multiplier 2 are connected to the first adder, and the outputs of mixed-precision multiplier 1 and mixed-precision multiplier 2 are used as the inputs of the first adder. After addition operation by the first adder, the output of the first adder with a length of 13 bits is obtained.

[0061] Mixed-precision multipliers 3 and 4 are connected to the second adder, respectively. The outputs of mixed-precision multipliers 3 and 4 serve as the inputs of the second adder. After addition operations by the second adder, the output of the second adder with a length of 13 bits is obtained.

[0062] The second adder is connected to the first shifter. The output of the second adder serves as the input of the first shifter. After shifting by the first shifter, a 17-bit output is obtained. Specifically, when the activation value precision is 2 bits or 4 bits, the first shifter shifts by 0 bits; when the activation value precision is 8 bits, the first shifter shifts by 4 bits to the left.

[0063] The first shifter and the first adder are connected to the third adder respectively. The outputs of the first shifter and the first adder are used as the inputs of the third adder. After the addition calculation by the third adder, the highest bit is removed to obtain a 16-bit output result, which is the result of the multiplication and addition operation of the mixed precision arithmetic unit.

[0064] The first shifter performs the corresponding shift based on the bit precision of the activation value input by the mixed precision arithmetic unit, which is applicable to cases where the bit precision of the activation value is 2, 4, or 8.

[0065] Specifically, a bit-serial-based mixed-precision multiplier is used for multiplying and adding 2-bit precision activation values ​​and 1-8 bit width variable precision weight values. Its inputs are 2-bit precision activation values ​​and 1-8 bit width variable precision weight values, and its output is a 12-bit product. The 2-bit precision activation values ​​are the activation values ​​of the neural network; the 1-8 bit width variable precision weight values ​​are the weight values ​​of the neural network. Its structure is as follows... Figure 1 As shown, it includes a mux data selector, a fourth adder, a 12-bit register, and two shifters (the second and third shifters).

[0066] The mux data selector includes an activation value input port and a weight value input port. The activation value input port loads a 2-bit precision activation value. The weight value input port loads a 1-8 bit wide, variable-precision weight value. Specifically, the weight value input is clock-based, using a bit-serial approach. The weight value is sequentially input to the weight input port from least significant bit to most significant bit, with 1 bit of the weight value input per clock cycle. After the most significant bit of the current weight value is input, the current weight value is fully loaded. The input weight value serves as the selection port for the mux. When the input weight value is 1, the mux outputs the activation value; otherwise, it outputs 2 bits of 0. The output of the mux is called the partial product in this bit-serial-based mixed-precision multiplier.

[0067] The second adder is connected to the mux data selector. The output of the mux data selector, i.e., the partial product, is one input of the second adder. The second adder performs a partial product accumulation once per clock cycle and caches the accumulated result in a 12-bit register connected to the second adder. Specifically, in each cycle, the partial product is accumulated to obtain a 3-bit accumulated result, which is cached in bits [11:9] of the register. Bits [11:10] serve as another input to the second adder and participate in the accumulation of the second adder in the next clock cycle. Bits [9:1] are shifted to bits [8:0] by the second shifter connected to the register, and the result is fed back into the register for caching. The first operation of the second adder is the addition of the partial product and a preloaded bias, which is the bias of the neural network.

[0068] The third shifter is connected to a 12-bit register. After a cycle equal to the weight precision, the weight values ​​are loaded. The activation value and weight values ​​are then calculated by the second adder and the second shifter, completing the accumulation of all partial products. The result is the data buffered in the 12-bit shifter. This data is used as the input to the third shifter. By shifting the data right in the third shifter, invalid bits in the lower bits are eliminated, resulting in a 12-bit output, which is the operation result of the mixed-precision multiplier.

[0069] The following describes the operation flow of a bit-serial based mixed-precision multiplier:

[0070] For example, the calculation process for a 2-bit precision activation value and a 4-bit precision weight value is as follows: Figure 3 As shown:

[0071] The weight values ​​are represented from low to high as follows: bit 0, bit 1, bit 2, and bit 3. The weight value input port sequentially inputs bit 0 of the weight value in the first cycle, bit 1 of the weight value in the second cycle, bit 2 of the weight value in the third cycle, and bit 3 of the weight value in the fourth cycle, completing the loading of the weight value in 4 cycles.

[0072] Each time a weight value is loaded, the next weight input port loads the next bit of weight value, and the activation value input port reloads the 2-bit activation value. The input weight value serves as the selection port for the mux. When the input weight value is 1, the mux outputs the activation value; otherwise, it outputs 2 bits of 0. The mux output is the partial product in the mixed-precision multiplier. The second adder performs a partial product accumulation once per cycle and shifts the accumulated result through the second shifter before buffering it in a register. The operation process generates a total of 4 partial products, each 2 bits, and completes the accumulation of all partial products after 4 clock cycles. For example, Figure 4A schematic diagram illustrating the principle of multiplication between 2-bit precision activation values ​​and 4-bit precision weight values.

[0073] After accumulating all partial products, the third shifter right-shifts by a specified number of bits to ensure that there are no invalid low-order bits in the register. This yields the output of the mixed-precision multiplier.

[0074] Specifically, the splicing and fusion of four bit-serial-based mixed-precision multipliers in the mixed-precision arithmetic unit includes three modes: 2-bit mode, 4-bit mode, and 8-bit mode.

[0075] 2-bit mode:

[0076] When the activation value precision is 2 bits, each mixed-precision multiplier performs a multiplication operation between an activation value and a weight value independently. The mixed-precision arithmetic unit performs the multiplication and accumulation operation of 4 sets of data in parallel.

[0077] The outputs of mixed-precision multipliers 1 and 2 are added by the first adder to obtain the output of the first adder, which has a length of 13 bits. The outputs of mixed-precision multipliers 3 and 4 are added by the second adder to obtain the output of the second adder, which has a length of 13 bits. The output of the second adder is used as the input of the first shifter, and after being shifted by the first shifter (the shift value is 0), the output of the first shifter is obtained. The outputs of the first shifter and the first adder are added by the third adder, and the highest bit is removed to obtain a 16-bit output result, which is the result of the multiplication and addition of 4 sets of data in 2-bit mode.

[0078] For example, Figure 5 The flowchart of mixed-precision arithmetic is shown. The activation value is 2 bits and the weight value is 4 bits.

[0079] 4-bit mode:

[0080] When the activation value precision is 4 bits, the first and second mixed-precision multipliers complete the multiplication and addition operation of one set of data by splicing and merging, and the third and fourth mixed-precision multipliers complete the multiplication and addition operation of another set of data by splicing and merging.

[0081] Specifically, taking the multiplication and addition operation of the first set of data as an example, the first mixed-precision multiplier completes the multiplication and addition operation of the lower 2 bits of the activation value and the weight value, and the second mixed-precision multiplier completes the multiplication and addition operation of the higher 2 bits of the activation value and the weight value. The outputs of the two mixed-precision multipliers are then added by the first adder to obtain the output of the first adder. When the second mixed-precision multiplier completes the multiplication and addition operation of the higher 2 bits of the activation value and the weight value, its third shifter needs to be shifted two bits to the right less than when it completes the multiplication and addition operation of the lower 2 bits and the weight value. The multiplication and addition operation of the second set of data is carried out in the same way to obtain the output of the second adder.

[0082] The output of the second adder is used as the input of the first shifter. After being shifted by the first shifter (the shift value is 0), the output of the first shifter is obtained. The outputs of the first shifter and the first adder are added by the third adder, and the highest bit is removed to obtain a 16-bit output result, which is the result of the multiplication and addition of two sets of data in 4-bit mode.

[0083] For example, Figure 6 The flowchart of mixed-precision arithmetic is shown. The activation value is 4 bits and the weight value is 4 bits.

[0084] 8-bit mode:

[0085] When the activation value precision is 8 bits, the four mixed-precision multipliers are spliced ​​and merged to complete the multiplication and addition operation of a set of data.

[0086] Specifically, the fourth mixed-precision multiplier performs the multiplication and addition operation of the highest 2-bit activation value and the weight value, and the third mixed-precision multiplier performs the multiplication and addition operation of the second highest 2-bit activation value and the weight value. The outputs of the two mixed-precision multipliers are then added by the first adder to obtain the output of the first adder. When the fourth mixed-precision multiplier performs the multiplication and addition operation of the highest 2-bit activation value and the weight value, its third shifter needs to be shifted right by two fewer bits than when performing the multiplication and addition operation of the second highest 2-bit activation value and the weight value.

[0087] The second mixed-precision multiplier performs another 2-bit multiplication and addition operation on the activation value and weight value, while the first mixed-precision multiplier performs the lowest 2-bit multiplication and addition operation on the activation value and weight value. The outputs of the two mixed-precision multipliers are then added by the second adder to obtain the output of the second adder. When the fourth mixed-precision multiplier performs another 2-bit multiplication and addition operation on the activation value and weight value, its third shifter needs to be shifted two bits to the right less than when performing the lowest 2-bit multiplication and addition operation on the activation value and weight value.

[0088] The output of the second adder serves as the input of the first shifter. After shifting by the first shifter (shift value of 4), a 17-bit output is obtained. The outputs of the first shifter and the first adder are then added by the third adder, and the most significant bit is removed to obtain a 16-bit output result, completing an 8-bit multiplication and addition operation between the activation value and the weight value. The mixed-precision arithmetic unit completes the multiplication and accumulation operation of one set of data.

[0089] For example, Figure 7 The flowchart of mixed-precision arithmetic is shown. The activation value is 8 bits and the weight value is 4 bits. Figure 8 This diagram illustrates the operation principle of 8-bit precision activation values ​​and 8-bit precision weight values ​​(where each quadrilateral window represents a mixed-precision multiplier).

[0090] This embodiment discloses a mixed-precision arithmetic unit. By splicing and fusing four mixed-precision multipliers, it supports multiply-add operations with variable precision of 2, 4, and 8 bits for activation values ​​and continuous variable precision of 1-8 bits for weight values. While meeting the needs of different precision operations, it greatly reduces computational bit width redundancy, power consumption, and area overhead, resulting in higher data utilization and lower computational redundancy in convolution operations.

[0091] The mixed-precision computing unit disclosed in this embodiment significantly reduces the amount of computation for the same neural network compared to traditional computing units, making the neural network easier to deploy on hardware.

[0092] Method Implementation Examples

[0093] The present invention discloses a mixed-precision arithmetic method, which uses a mixed-precision arithmetic unit to achieve mixed-precision arithmetic with variable precision of 2, 4, and 8 bits for activation values ​​and continuously variable precision of 1-8 bits for weight values.

[0094] When the activation value precision is 2, 4, or 8 bits, and the weight value is any of the 1-8 bit precision cases, the mixed-precision calculation method based on bit serialization and bit fusion includes:

[0095] Based on the activation value precision of 2, 4, and 8 bits, a mixed-precision arithmetic unit is used to select a mode that corresponds one-to-one with the activation value precision for mixed-precision arithmetic.

[0096] When the activation value precision is 2 bits, the mixed-precision arithmetic unit operates in 2-bit mode, and the mixed-precision arithmetic unit performs multiplication and accumulation operations on 4 sets of data in parallel:

[0097] The results of the two mixed-precision multipliers are added together to obtain the first result;

[0098] The results of the operations of the other two mixed-precision multipliers are added together to obtain the second result;

[0099] The third result is obtained by shifting the second result using a third shifter; wherein the shift value of the third shifter is 0.

[0100] The first and third results are added together to obtain the result of the mixed precision arithmetic method.

[0101] When the activation value precision is 4 bits, the mixed-precision arithmetic unit operates in 4-bit mode, and the mixed-precision arithmetic unit performs multiplication and accumulation operations on two sets of data in parallel:

[0102] The first mixed-precision multiplier performs a multiplication and addition operation on the lower 2 bits of a 4-bit activation value and the weight value. The second mixed-precision multiplier performs a multiplication and addition operation on the higher 2 bits of the activation value and the weight value. The two results are then added together to obtain the first result. Note that the mixed-precision multiplier that performs the multiplication and addition operation on the higher 2 bits of the activation value and the weight value needs to shift two bits to the right compared to the one that performs the multiplication and addition operation on the lower 2 bits of the activation value and the weight value.

[0103] The third and fourth mixed-precision multipliers use the same method to calculate the multiplication and addition operation of another 4-bit activation value and weight value, and obtain the second result;

[0104] The third result is obtained by shifting the second result using a third shifter; wherein the shift value of the third shifter is 0.

[0105] The first and third results are added together to obtain the result of the mixed-precision arithmetic method.

[0106] When the activation value precision is 8 bits, the mixed-precision arithmetic unit operates in 8-bit mode, and completes a multiplication-accumulation operation on one set of data:

[0107] The fourth mixed-precision multiplier performs the multiplication and addition operation of the highest 2-bit activation value and the weight value among the 8-bit activation values. The third mixed-precision multiplier performs the multiplication and addition operation of the next 2-bit activation value and the weight value. The two results are then added together to obtain the first result. When the fourth mixed-precision multiplier performs the multiplication and addition operation of the highest 2-bit activation value and the weight value, it needs to shift two bits to the right less than the mixed-precision multiplier that performs the multiplication and addition operation of the next highest 2-bit activation value and the weight value.

[0108] The second and first mixed-precision multipliers perform the multiplication and addition operations of the second 2-bit activation value, the lowest 2-bit activation value and the weight value in the same way, respectively. The two operation results are then added to obtain the second result.

[0109] The third result is obtained by shifting the second result using a third shifter; the shift value of the third shifter is 4.

[0110] The first and third results are added together to obtain the result of the mixed precision arithmetic method.

[0111] During the operation, the shift value of the third shifter is 4.

[0112] This embodiment discloses a hybrid precision arithmetic method based on bit serialization and bit fusion, which supports multiply-accumulate operations with variable precision of 2, 4, and 8 bits for activation values ​​and continuous variable precision of 1-8 bits for weight values. While meeting the needs of different precision operations, it greatly reduces computational bit width redundancy, power consumption, and area overhead, resulting in higher data utilization and lower computational redundancy in convolution operations.

[0113] Array Examples

[0114] This embodiment discloses a mixed-precision computation array, which consists of four mixed-precision computation units arranged in a one-dimensional vector configuration. Each computation unit in the array is responsible for performing a convolution operation on one output channel, and the four output channels are grouped together for parallel computation. The array is represented as follows: Figure 9 As shown.

[0115] The mixed-precision computing array disclosed in this embodiment further improves the parallelism of mixed-precision data computing and greatly enhances computing efficiency.

[0116] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A mixed-precision arithmetic unit, characterized in that, Deployed in hardware, it includes four bit-serial based mixed-precision multipliers, first, second, and third adders, and a first shifter; Each of the mixed-precision multipliers is used to perform a multiplication and addition operation on the activation value and weight value with 2-bit precision; the mux data selector is used to perform a selection operation based on the input activation value and weight value in each clock cycle to obtain a partial product; The two inputs of the fourth adder are connected to the mux data selector and the two highest bits of the register, respectively, to accumulate the partial product and output it to the three highest bits of the register. The three most significant bits of the register are connected to the output of the fourth adder and are used to cache the accumulated result of the partial product each clock cycle. The input and output of the second shifter are both connected to the register, and it is used to shift the least significant bit of the partial product of the buffer for each clock cycle and feed it back to the register; The input of the third shifter is connected to the register and is used to right-shift the accumulated result of the partial product of all clock cycles to eliminate invalid bit width in the low bits, so as to obtain the operation result of the mixed precision multiplier. The four mixed-precision multipliers perform multiplication and addition operations on the 2, 4, or 8-bit precision activation values ​​and weight values ​​input to the mixed-precision arithmetic unit in groups; the weight values ​​are 1-8 bit full-width variable weight values. The first shifter performs a corresponding shift based on the bit precision of the activation value input by the mixed precision arithmetic unit, including: when the bit precision of the activation value input by the mixed precision arithmetic unit is 2 or 4, the shift value of the first shifter is 0; when the bit precision of the activation value input by the mixed precision arithmetic unit is 8, the shift value of the first shifter is 4. The first adder has two input terminals connected to the output terminals of two mixed-precision multipliers, and its output terminal connected to one input terminal of the third adder. The second adder has two input terminals connected to the output terminals of two other mixed-precision multipliers, and its output terminal is connected to the other input terminal of the third adder after passing through the first shifter; the third adder is used to output the operation result of the mixed-precision arithmetic unit.

2. The mixed-precision computing unit according to claim 1, characterized in that, Each of the mixed-precision multipliers is used for multiplying and adding activation and weight values ​​with 2-bit precision; When the activation value input to the mixed precision arithmetic unit is 2 bits, each mixed precision multiplier is a separate group, performing a multiplication and addition operation on a 2-bit activation value and a weight value respectively. The four mixed precision multipliers complete the multiplication and addition operation on four groups of data in parallel. When the activation value input to the mixed precision arithmetic unit has a precision of 4 bits, each pair of mixed precision multipliers forms a group to complete a multiplication and addition operation of a group of data. In each group, one mixed precision multiplier completes a multiplication and addition operation of the lower 2 bits of the activation value and the weight value, and the other mixed precision multiplier completes a multiplication and addition operation of the higher 2 bits of the activation value and the weight value. When the activation value input to the mixed-precision arithmetic unit has a precision of 8 bits, the four mixed-precision multipliers are concatenated and merged into one group to jointly complete the multiplication and addition operation of a set of data. Among them, the fourth mixed-precision multiplier completes the multiplication and addition operation of the highest 2-bit activation value and weight value, the third mixed-precision multiplier completes the multiplication and addition operation of the next 2-bit activation value and weight value, the second mixed-precision multiplier completes the multiplication and addition operation of the next 2-bit activation value and weight value, and the first mixed-precision multiplier completes the multiplication and addition operation of the lowest 2-bit activation value and weight value.

3. The mixed-precision computing unit according to claim 1, characterized in that, When the activation value input to the mixed precision arithmetic unit is 4 bits, the third shifter in the mixed precision multiplier that performs the multiplication and addition operation of the high 2 bits of the activation value and the weight value in each group needs to be shifted right by two bits less than when performing the multiplication and addition operation of the low 2 bits and the weight value. When the activation value input to the mixed precision arithmetic unit is 8 bits, the third shifter in the mixed precision multiplier that performs the multiplication and addition operation of the highest 2 bits activation value and weight value and the multiplication and addition operation of the next 2 bits activation value and weight value needs to be shifted right to eliminate invalid bit width in the low bits. This requires two fewer right shifts than when performing the multiplication and addition operation of the second highest 2 bits activation value and weight value and the multiplication and addition operation of the lowest 2 bits activation value and weight value.

4. A mixed-precision arithmetic method, characterized in that, Deploying on hardware includes the following steps: Four bit-serial-based mixed-precision multipliers are used to perform multiplication and addition operations on 2-bit precision activation values ​​and 1-8 bit continuous precision variable weight values ​​respectively; a mux data selector is used to perform selection operations based on 2-bit activation values ​​and weight values ​​in each clock cycle to obtain partial products; In each clock cycle, the partial product is accumulated using an adder, and the accumulated result is cached in a register. The two most significant bits are used in the accumulation of the next cycle, and the least significant bit is slowly shifted to the right. The above operation is repeated until the accumulated result of the partial product for all clock cycles is obtained. The accumulated result is shifted to the right to cancel out the invalid bit width in the lower bits, thus obtaining the operation result of the mixed precision multiplier; The four mixed-precision multipliers perform multiplication and addition operations on the input activation values ​​and weight values ​​with 2, 4, or 8-bit precision by grouping; the weight values ​​are variable weight values ​​with a full bit width of 1-8 bits. The results of the two mixed-precision multipliers are added together to obtain the first result; The results of the operations of the other two mixed-precision multipliers are added together to obtain the second result; The second result is obtained by shifting the second result using a shifter based on the bit precision of the input activation value of the mixed precision arithmetic method, including: when the bit precision of the activation value is 2 or 4, the shift value is 0; when the bit precision of the activation value is 8, the shift value is 4. The first and third results are added together to obtain the result of the mixed precision arithmetic method.

5. The mixed-precision calculation method according to claim 4, characterized in that, Each mixed-precision multiplier performs a multiplication-addition operation between the 2-bit activation value and the weight value, including: The right shifting of the accumulated result includes: when the input activation value precision of the mixed-precision arithmetic method is 2 bits, each mixed-precision multiplier shifts the accumulated result to the right to cancel the invalid bit width in the lower bits; when the input activation value precision of the mixed-precision arithmetic method is 4 bits, the right shift of the mixed-precision multiplier that completes the multiplication and addition operation of the high 2 bits of the activation value and the weight value requires two fewer right shifts than the right shift of the mixed-precision multiplier that completes the multiplication and addition operation of the low 2 bits of the activation value and the weight value; when the input activation value precision of the mixed-precision arithmetic method is 8 bits, the right shift of the mixed-precision multiplier that completes the multiplication and addition operation of the highest 2 bits of the activation value and the weight value and the second 2 bits of the activation value and the weight value requires two fewer right shifts than the right shift of the mixed-precision multiplier that completes the multiplication and addition operation of the second highest 2 bits of the activation value and the weight value and the lowest 2 bits of the activation value and the weight value.

6. A mixed-precision arithmetic array, characterized in that; It includes three mixed-precision computing units as described in any one of claims 1-3. The four mixed-precision computing units are arranged in a one-dimensional vector manner. Each computing unit in the array is responsible for performing convolution operation on one output channel, and the four output channels are grouped together for parallel operation.

Images