Data representation method, tensor quantization method, and multiply-add computing device

Abstract

The application provides a data representation method, a tensor quantization method and a multiply-add calculation device. The data representation method comprises: obtaining target data, the target data comprising a flag bit and a signed number, the sum of the bit widths of the flag bit and the signed number being equal to a preset bit width, the signed number comprising a high bit and a low bit; obtaining a split bit of the signed number; determining a belonging segment of the target data according to each value in the flag bit and the high bit; and representing the target data according to the belonging segment of the target data. It can be inferred that the dynamic range that can be represented by a k-bit PINT is [‑2 2(k‑2) ,2 2(k‑2) ], which is equivalent to the range that can be represented by a (2k‑3)-bit INT format. Compared with FP32, the complexity in calculation is reduced, and the model accuracy is not affected.

Description

Technical Field

[0001] This application relates to the field of convolutional neural network technology, specifically to a data representation method, a tensor quantization method, and a multiply-accumulate calculation device. Background Technology

[0002] With the continuous development of artificial intelligence (AI), it has evolved from early manual feature engineering to the ability to learn from massive amounts of data, achieving significant breakthroughs in fields such as machine vision, speech recognition, and natural language processing. Deep Neural Networks (DNNs) are gaining increasing popularity in the field of AI. However, as network structures become larger and more complex, they require substantial computing resources for training on high-end GPU server clusters.

[0003] In recent years, with the increasingly widespread application of DNNs in real-world production and daily life, especially the rise of technologies such as online learning, incremental learning, and federated learning, and with people paying more and more attention to data privacy protection, high-efficiency DNN training on edge devices has gradually become an urgent need. Low-bit training is an effective solution. Traditional low-bit training processes use standard 32-bit floating-point numbers (FP32) for computation, which includes 1 sign bit, 8 unsigned exponent bits (exp), and 23 mant bits. The computation method for data represented by FP32 is divided into normal and denormal cases. In the normal case, it is represented by the following formula: (-1) sign ×2 (exp-127) In the case of ×1.mant;denormal, it is represented by the following formula: (-1) sign ×2 (exp-126) ×0.mant. During calculation, different data representation methods are used depending on whether the case is normal or denormal, and then different calculation methods are used accordingly.

[0004] However, due to the high bit width of data represented by FP32, and the fact that floating-point calculations involve operations such as exponent addition, data alignment, and normal / denormal discrimination, the computational complexity is correspondingly increased. Therefore, there is an urgent need to develop a data representation method with lower complexity to achieve lower hardware resource consumption, computational complexity, computational latency, and data storage overhead. Summary of the Invention

[0005] This application provides a data representation method, a tensor quantization method, and a multiply-accumulate calculation device to solve the problem of high computational complexity caused by the FP32 representation method.

[0006] A first aspect of this application provides a data representation method applied to a convolutional neural network, comprising:

[0007] Acquire target data, which includes a flag bit and a signed number. The sum of the bit widths of the flag bit and the signed number is equal to a preset bit width. The signed number includes a high bit and a low bit.

[0008] Obtain the delimiter of the signed number, wherein the delimiter divides the signed number into first data that is close to the flag bit and second data that is far from the flag bit, wherein the combination of the first data and the delimiter is the high bit and the combination of the second data and the delimiter is the low bit;

[0009] Based on the flag bit and the values ​​of each of the high bits, the segment to which the target data belongs is determined, and the segment to which the data belongs is one of the first segment, the second segment, and the third segment;

[0010] The target data is represented according to its segment, wherein if the target data belongs to the first segment, the target data is represented using the least significant bit; if the target data belongs to the second segment, the target data is represented by the value corresponding to the signed number and 2. d The target data is represented by the product of 2; if the target data is the third segment, then the target data is represented by the value corresponding to the signed number and 2. k-2 The product is represented as d, where d is the dividing bit and k is the preset bit width.

[0011] Optionally, determining the segment to which the target data belongs based on the flag bit and the values ​​in the high bits includes:

[0012] Based on whether the values ​​in the higher bits are the same, the potential segment to which the target data belongs is determined. If all the values ​​in the higher bits are the same, the target data is a potential first segment; otherwise, the target data is a potential third segment.

[0013] Based on the value of the flag bit, the segment to which the target data belongs is determined. If the value of the flag bit is 1, the target data is the second segment; if the value of the flag bit is 0, the segment to which the target data belongs is the segment number corresponding to the potential segment.

[0014] A second aspect of this application provides a tensor quantization method applied to a convolutional neural network, comprising:

[0015] Get the first tensor;

[0016] Calculate the maximum absolute value of the first tensor;

[0017] The quotient of the maximum absolute value and the predetermined maximum representation range is used as the scaling factor;

[0018] The first tensor is scaled using the scaling factor to obtain the second tensor;

[0019] Based on the absolute value of the second tensor, determine the segment to which the second tensor belongs. Specifically, if the absolute value of the second tensor is less than 2... d If the absolute value of the second tensor is 2, then the second tensor is the first segment; if the absolute value of the second tensor is 2 d and 2 k-2+d If the absolute value of the second tensor is greater than 2, then the second tensor is the second segment; k-2+d Then the second tensor is the third segment;

[0020] The second tensor is quantized according to the segment to which it belongs;

[0021] The product of the scaling factor and the quantized second tensor is used as the original numerical range of the first tensor.

[0022] Optionally, the second tensor is quantized according to the segment to which it belongs, including:

[0023] If the segment to which the second tensor belongs is the first segment, then the second tensor x is processed according to the following formula. s Quantification:

[0024] x pi =round(x s );

[0025] Where round is the rounding function, x pi This is the second tensor after quantization;

[0026] or,

[0027] If the segment to which the second tensor belongs is the second segment, then the second tensor x is processed according to the following formula. s Quantification:

[0028] x pi =round(x s / 2 d )×2 d ;

[0029] or,

[0030] If the segment to which the second tensor belongs is the third segment, then the second tensor x is processed according to the following formula. s Quantification:

[0031] xpi =round(x s / 2 k-2 )×2 k-2 ;

[0032] Wherein, d is the segmentation bit when representing data using any of the data representation methods provided in the first aspect, and k is the preset bit width.

[0033] A third aspect of this application provides a multiply-accumulate calculation device applied to a convolutional neural network, comprising:

[0034] The acquisition module is used to acquire the first input data and the second input data;

[0035] The multiplier is used to multiply the first signed number corresponding to the first input data and the second signed number corresponding to the second input data to obtain an initial product;

[0036] The first judgment module is used to determine whether all the values ​​of the first high-order bits corresponding to the first input data are the same;

[0037] The second judgment module is used to determine whether all the values ​​of the second high-order bits corresponding to the second input data are the same;

[0038] The decoder is used to determine the first segment to which the first input data belongs and the second segment to which the second input data belongs based on the first flag bit of the first input data, the second flag bit of the second input data, the first judgment result of the first judgment module, and the second judgment result of the second judgment module, and to determine the shift number of the initial product based on the first segment to which the first input data belongs and the second segment to which the second input data belongs;

[0039] A shifter is used to shift the initial product according to the shift number determined by the decoder to obtain a shifted product;

[0040] An adder is used to add the shifted product to the acquired third input data to obtain a multiply-add result.

[0041] Optionally, the first determination module includes: a first AND gate, a first NOR gate, and a first OR gate, wherein the output of the first AND gate and the output of the first NOR gate are both connected to the input of the first OR gate;

[0042] The first AND gate is used to determine that the first output result is high when all the first high bits are high.

[0043] The first NOR gate is used to determine that the second output result is high when all the first high-level bits are low.

[0044] The first OR gate is used to determine that all the values ​​of the first high-order bits corresponding to the first input data are the same when the first output result is high and / or the second output result is high.

[0045] Optionally, the second judgment module includes: a second AND gate, a second NOR gate, and a second OR gate, wherein the output terminals of the second AND gate and the second NOR gate are both connected to the input terminal of the second OR gate;

[0046] The second AND gate is used to determine that the third output result is high when all the second high bits are high.

[0047] The second NOR gate is used to determine that the fourth output result is high when all the second high bits are low.

[0048] The second OR gate is used to determine that all the values ​​of the second high-order bits corresponding to the second input data are the same when the third output result is high and / or the fourth output result is high.

[0049] Optionally, the decoder includes:

[0050] The first determining unit is configured to determine the first segment to which the first input data belongs based on the first flag bit of the first input data and the first judgment result of the first judgment module;

[0051] The second determining unit is used to determine the second segment to which the second input data belongs based on the second flag bit of the second input data and the second judgment result of the second judgment module;

[0052] The summation unit is used to sum the first and second segments to obtain the sum of the segments;

[0053] The third determining unit is used to determine the shift number of the initial product based on the correspondence between the sum of the segments and the shift number of the initial product.

[0054] In this application, the target data is divided into three segments, each represented using a different method. In fact, different segments have different numerical resolutions. The lower the segment number of the target data, the smaller the value of the target data, and correspondingly, a calculation method matching the resolution can be used during multiplication and addition. Furthermore, it can be inferred that the dynamic range that a k-bit PINT can represent is [-2...]. 2(k-2) ,2 2(k-2) This is equivalent to the range that a (2k-3)-bit INT format can represent, meaning that the data representation method provided in this application can represent a larger range.

[0055] The PINT data representation proposed in this application, compared with FP32, can achieve: a) saving 4 times the storage overhead and bandwidth requirements; b) reducing area overhead by 13.7x; c) reducing power consumption by 9.2x; d) increasing processing speed by 3.8x; e) achieving virtually lossless model accuracy. Attached Figure Description

[0056] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 A flowchart illustrating a data representation method provided in this application embodiment;

[0058] Figure 2 This is a schematic diagram of a data structure represented using the data representation method provided in the embodiments of this application;

[0059] Figure 3 for Figure 2 A schematic diagram of the structure of signed numbers;

[0060] Figure 4 A flowchart illustrating a tensor quantization method provided in this application embodiment;

[0061] Figure 5 This is a schematic diagram of the structure of a multiply-accumulate calculation device provided in an embodiment of this application. Detailed Implementation

[0062] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0063] To address the high computational complexity of the FP32 representation method, this application provides a data representation method, a tensor quantization method, and a multiply-accumulate calculation device.

[0064] refer to Figure 1 The flowchart shown illustrates that this application provides a data representation method applied to a convolutional neural network. The method includes the following steps:

[0065] Step 101: Obtain target data, which includes a flag bit and a signed number. The sum of the bit widths of the flag bit and the signed number is equal to a preset bit width. The signed number includes a high bit and a low bit.

[0066] refer to Figure 2 The schematic diagram shown has a preset bit width of k, with the unit being bits, meaning the target data contains k-bit data. The flag bit is usually the first bit, represented by `flag`, and has a bit width of 1-bit; the data outside the flag bit is a signed number, represented by `SI`, and has a bit width of (k-1)-bit.

[0067] Step 102: Obtain the delimiter of the signed number. The delimiter divides the signed number into first data that is close to the flag bit and second data that is far from the flag bit. The combination of the first data and the delimiter is the high bit, and the combination of the second data and the delimiter is the low bit.

[0068] refer to Figure 3 The structural diagram shown assumes k = 8. The target data can be represented by x[7:0], where the rightmost bit is counted as 0, and the bits increase by 1 from right to left. The dividing bit is represented by d. Assuming d = 3, the fourth bit from right to left is the dividing bit. Then x1[6:3] is the high bit, which has a total of kd-1 = 4 bits; x2[3:0] is the low bit, which has a total of d+1 = 4 bits.

[0069] In this embodiment, the high-order bits are represented by HB, and the low-order bits are represented by LB.

[0070] Step 103: Determine the segment to which the target data belongs based on the flag bit and the values ​​of each of the high bits. The segment to which the data belongs is one of the first segment, the second segment, and the third segment.

[0071] In one feasible approach, determining the segment to which the target data belongs based on the flag bit and the values ​​in the high-order bits includes the following steps:

[0072] Step 10301: Determine the potential segment to which the target data belongs based on whether the values ​​in the higher bits are the same. If all the values ​​in the higher bits are the same, the target data is a potential first segment; otherwise, the target data is a potential third segment.

[0073] Step 10302: Determine the segment to which the target data belongs based on the value of the flag bit, wherein if the value of the flag bit is 1, the target data is the second segment; if the value of the flag bit is 0, the segment to which the target data belongs is the segment number corresponding to the potential segment.

[0074] In addition, other methods can be used to determine the segment to which the target data belongs. For example, the following steps can be used to determine this:

[0075] Step 10311: Determine whether all the values ​​in the higher bits are the same;

[0076] Step 10312: If all the values ​​in the higher bits are the same, then the segment to which the target data belongs is the potential first segment;

[0077] Step 10313: Determine whether the value of the flag bit is 0;

[0078] Step 10314: If yes, the target data is the first segment; otherwise, it is the second segment.

[0079] Step 10315: If the values ​​in the higher bits are different, then the segment to which the target data belongs is a potential third segment;

[0080] Step 10316: Determine whether the value of the flag bit is 0;

[0081] Step 10317: If the value of the flag bit is 0, then it is the third segment; otherwise, it is the second segment.

[0082] Alternatively, the following steps can be used to determine this:

[0083] Step 10321: Determine whether the value of the flag bit is 1;

[0084] Step 10322: If the value of the flag bit is 1, then the target data belongs to the second segment;

[0085] Step 10323: If the value of the flag bit is not 1, then determine whether the value in the higher bits is equal to the target value, where the target value is 0 or 1.

[0086] Step 10324: If all the values ​​in the higher bits are equal to the target value, then the target data belongs to the first segment; otherwise, the target data belongs to the second segment.

[0087] The above examples illustrate how to determine the segment to which the target data belongs. Other methods may also be used, and this application does not specifically limit them.

[0088] Step 104: Represent the target data according to the segment to which it belongs. If the target data belongs to the first segment, it is represented using the least significant bit; if the target data belongs to the second segment, it is represented by the value corresponding to the signed number and 2. dThe target data is represented by the product of 2; if the target data is the third segment, then the target data is represented by the value corresponding to the signed number and 2. k-2 The product is represented as d, where d is the dividing bit and k is the preset bit width.

[0089] In this embodiment, the data corresponding to the signed number is represented by SI, and the data corresponding to the lower-order bits is represented by LSI. PINT is used to represent the data representation method provided in this embodiment.

[0090] In this embodiment, the target data is divided into three segments, each represented using a different method. In fact, different segments have different numerical resolutions. The lower the segment number of the target data, the smaller the value of the target data; correspondingly, when performing multiplication and addition calculations, a calculation method matching the resolution can be used. Furthermore, it can be inferred that the dynamic range that a k-bit PINT can represent is [-2...]. 2(k-2) ,2 2(k-2) Compared with the traditional INT format, the dynamic range that a k-bit PINT can represent is equivalent to the range that a (2k-3)-bit INT format can represent. In other words, the data representation method provided in this application embodiment can represent a larger range.

[0091] The PINT data representation proposed in this solution, compared to FP32, can achieve: a) saving 4 times the storage overhead and bandwidth requirements; b) reducing area overhead by 13.7x; c) reducing power consumption by 9.2x; d) increasing processing speed by 3.8x; and e) achieving virtually lossless model accuracy.

[0092] In convolutional neural network computation, data is input in the form of tensors. Therefore, the following examples illustrate tensor quantization using the PINT data representation method.

[0093] refer to Figure 4 The flowchart shown illustrates that this application provides a tensor quantization method applied to a convolutional neural network, comprising:

[0094] Step 201: Obtain the first tensor.

[0095] Step 202: Calculate the maximum absolute value of the first tensor.

[0096] In this step, r = max(X); where X is the first tensor and r is the maximum absolute value of the first tensor.

[0097] Step 203: Use the quotient of the maximum absolute value and the predetermined maximum representation range as the scaling factor.

[0098] In this step, s = r / 2 2(k-2) Where s is the scaling factor, 2 2(k-2) The maximum range of representation is predetermined.

[0099] Step 204: Scale the first tensor using the scaling factor to obtain the second tensor.

[0100] In this step, X s =X / s, where X s For the second tensor, scale it to [-2]. 2(k-2) ,2 2(k-2) Within the numerical range of ].

[0101] Step 205: Determine the segment to which the second tensor belongs based on the absolute value of the second tensor, wherein if the absolute value of the second tensor is less than 2... d If the absolute value of the second tensor is 2, then the second tensor is the first segment; if the absolute value of the second tensor is 2 d and 2 k-2+d If the absolute value of the second tensor is greater than 2, then the second tensor is the second segment; k-2+d Then the second tensor is the third segment.

[0102] Step 206: Quantize the second tensor according to the segment to which the second tensor belongs;

[0103] In one feasible approach, the second tensor is quantized according to the segment to which it belongs, including the following steps:

[0104] Step 2061: If the segment to which the second tensor belongs is the first segment, then apply the following formula to the second tensor x. s Quantification:

[0105] x pi =round(x s );

[0106] Where round is the rounding function, x pi This is the second tensor after quantization;

[0107] or,

[0108] Step 2062: If the segment to which the second tensor belongs is the second segment, then apply the following formula to the second tensor x. s Quantification:

[0109] x pi =round(x s / 2 d )×2 d ;

[0110] or,

[0111] Step 2063: If the segment to which the second tensor belongs is the third segment, then apply the following formula to the second tensor x. s Quantification:

[0112] x pi =round(x s / 2 k-2 )×2 k-2 ;

[0113] Where d is the dividing bit and k is the preset bit width.

[0114] Step 207: The product of the scaling factor and the quantized second tensor is used as the original numerical range of the first tensor.

[0115] In this step, x q =X pi ×s; where x q This represents the original numerical range of the first tensor.

[0116] This application selects PINT for DNN training. The computations in DNN mainly include convolution and matrix multiplication, both of which are accomplished by a large number of multiplication-accumulation operations. In order to realize PINT computation more efficiently, this application designs a dedicated PINT multiplication-accumulation computation device.

[0117] The multiply-accumulate calculation device provided in this application involves logic gate circuits such as AND gates, NOR gates, and OR gates. In these gate circuits, high and low levels are typically used to determine the input or output signal. In logic levels, there is a minimum allowed high input level to ensure the input of a logic gate is high; when the input level is higher than the input high voltage (Vih), the input level is considered high. In digital logic circuits, a low level represents 0, and a high level represents 1.

[0118] The following is a brief introduction to logic gate circuits.

[0119] An AND gate has multiple inputs and one output. The output is high only when all inputs are simultaneously high (logic 1); otherwise, the output is low (logic 0).

[0120] A NOR gate has multiple inputs and one output. The output is high (logic 1) only when both inputs are low (logic 0). Alternatively, it can be understood that if any input is high (logic 1), the output is low (logic 0).

[0121] An OR gate has multiple inputs and one output. The output is high (logic 1) as long as any one of the inputs is high (logic 1); the output is low (logic 0) only when all inputs are low (logic 0).

[0122] Taking PINT(8,3) as an example, its structural diagram is as follows: Figure 5 As shown. The multiply-add calculation device includes an acquisition module 100, a 7-bit input signed integer multiplier 200, a first judgment module 300, a second judgment module 400, a decoder 500, a shifter 600, and a 32-bit adder 700.

[0123] in:

[0124] The acquisition module 100 is used to acquire the first input data and the second input data;

[0125] Multiplier 200 is used to multiply the first signed number corresponding to the first input data and the second signed number corresponding to the second input data to obtain an initial product;

[0126] The first judgment module 300 is used to determine whether all the values ​​of the first high-order bits corresponding to the first input data are the same;

[0127] The second judgment module 400 is used to determine whether all the values ​​of the second high-order bits corresponding to the second input data are the same;

[0128] The decoder 500 is used to determine the first segment to which the first input data belongs and the second segment to which the second input data belongs based on the first flag bit of the first input data, the second flag bit of the second input data, the first judgment result of the first judgment module, and the second judgment result of the second judgment module, and to determine the shift number of the initial product based on the first segment to which the first input data belongs and the second segment to which the second input data belongs;

[0129] Shifter 600 is used to shift the initial product according to the shift number determined by the decoder to obtain a shifted product;

[0130] Adder 700 is used to add the shifted product to the acquired third input data to obtain a multiply-add result.

[0131] Optionally, the first determination module 300 includes: a first AND gate AND1, a first NOR gate NOR1, and a first OR gate OR1, wherein the output of the first AND gate AND1 and the output of the first NOR gate NOR1 are both connected to the input of the first OR gate OR1.

[0132] The first AND gate AND1 includes four inputs, used to determine that the first output result is high when all the first high bits are high.

[0133] The first NOR gate includes four inputs, used to determine that the second output is high when all the first high-order bits are low.

[0134] The first OR gate OR1 is used to determine that all the values ​​of the first high-order bits corresponding to the first input data are the same when the first output result is high and / or the second output result is high.

[0135] Optionally, the second determination module 400 includes: a second AND gate AND2, a second NOR gate NOR2, and a second OR gate OR2, wherein the output terminals of the second AND gate AND2 and the second NOR gate NOR2 are both connected to the input terminal of the second OR gate OR2.

[0136] The second AND gate AND2 includes four inputs, used to determine that the third output is high when all the second high bits are high;

[0137] The second NOR gate NOR2 includes four inputs, used to determine that the fourth output is high when all the second high-order bits are low;

[0138] The second OR gate OR2 is used to determine that all the values ​​of the second high-order bits corresponding to the second input data are the same when the third output result is high and / or the fourth output result is high.

[0139] Optionally, the decoder 500 includes:

[0140] The first determining unit is configured to determine the first segment to which the first input data belongs based on the first flag bit of the first input data and the first judgment result of the first judgment module;

[0141] The second determining unit is used to determine the second segment to which the second input data belongs based on the second flag bit of the second input data and the second judgment result of the second judgment module;

[0142] The summation unit is used to sum the first and second segments to obtain the sum of the segments;

[0143] The third determining unit is used to determine the shift number of the initial product based on the correspondence between the sum of the segments and the shift number of the initial product.

[0144] In this embodiment of the application, PINT(8,3) is taken as an example, such as Figure 5As shown, the first input data is a[7:0], the inputs of AND1 and NOR1 are a[6:3], and the output of OR1 is ea; the second input data is b[7:0], the inputs of AND2 and NOR2 are b[6:3], and the output of OR1 is eb; the inputs of the multiplier are a[6:0] and b[6:0]; the inputs of the shifter are the output of the shifter and c[31:0], and the output of the shifter is z[31:0].

[0145] The following table illustrates the operation process of the decoder 500.

[0146] Table 1. Input and output results of the first and second determination units.

[0147] a[7] ea First segment b[7] eb Second segment 0 1 "one" 0 1 "one" 1 0 "two" 1 0 "two" 1 1 "two" 1 1 "two" 0 0 "three" 0 0 "three"

[0148] Table 2 shows the correspondence between the sum of segments and the number of shifts.

[0149] {a[7],ea,b[7],eb} Sum of segments shift number {0101} 2 0 {0110},{0111},{1001},{1101} 3 3 {0100},{0001},{1010},{1011},{1110,}{1111} 4 6 {1000,}{1100},{0010},{0011} 5 9 {0000} 6 12

[0150] Based on the output results provided in Table 1 and the correspondence provided in Table 2, the first flag bit a[7] of the first input data, the first judgment result ea of ​​the first judgment module, the second flag bit b[7] of the second input data, and the second judgment result eb of the second judgment module can be used to determine the first segment to which the first input data belongs and the second segment to which the second input data belongs; then, the number of shifts is determined according to the sum of the segments. The initial product after shifting is added to the third input data to obtain the result z of a*b+c.

[0151] It is understood that the above embodiment takes PINT(8,3) as an example. When k and d are other values, the corresponding sum of segments and the number of shifts will also change. This application does not make specific limitations on this.

[0152] Using the Design Compiler tool, the multiply-accumulate computing devices of FP32 and PINT(8,3) were synthesized under a 28nm process, and the results are shown in Table 3. The power consumption and area results were achieved at a frequency of 500MHz.

[0153] Table 3 Comparison of Implementation Results of FP32 and PINT(8,3)

[0154] project FP32 PINT(8,3) Improvement rate <![CDATA[Area (square micrometers, um 2 )]]> 5432.87 395.64 13.7 Power consumption (mW) 1.7215 0.1881 9.2 Delay (nanoseconds, ns) 1.88 0.5 3.8

[0155] As shown in Table 3, compared with FP32, PINT(8,3) has improvements in area, power consumption and latency. Specifically, the area improvement rate is 13.7, the power consumption improvement rate is 9.2, and the latency improvement rate is 3.8.

[0156] The present application has been described in detail above with reference to specific embodiments and exemplary examples; however, these descriptions should not be construed as limiting the present application. Those skilled in the art will understand that various equivalent substitutions, modifications, or improvements can be made to the technical solutions and implementation methods of the present application without departing from the spirit and scope of the present application, and all such modifications and improvements fall within the scope of the present application. The scope of protection of the present application is determined by the appended claims.

Claims

1. A data representation method, wherein the data representation method is applied to a convolutional neural network, characterized in that, include: Acquire target data, which includes a flag bit and a signed number. The sum of the bit widths of the flag bit and the signed number is equal to a preset bit width. The signed number includes a high bit and a low bit. Obtain the delimiter of the signed number, wherein the delimiter divides the signed number into first data that is close to the flag bit and second data that is far from the flag bit, wherein the combination of the first data and the delimiter is the high bit and the combination of the second data and the delimiter is the low bit; Based on the flag bit and the values ​​of each of the high bits, the segment to which the target data belongs is determined, and the segment to which the data belongs is one of the first segment, the second segment, and the third segment; The target data is represented according to its segment, wherein if the target data belongs to the first segment, the target data is represented using the least significant bit; if the target data belongs to the second segment, the target data is represented by the value corresponding to the signed number and 2. d The target data is represented by the product of 2; if the target data is the third segment, then the target data is represented by the value corresponding to the signed number and 2. k-2 The product is represented as d, where d is the dividing bit and k is the preset bit width.

2. The data representation method according to claim 1, characterized in that, Determining the segment to which the target data belongs based on the flag bit and the values ​​in the high bits includes: Based on whether the values ​​in the higher bits are the same, the potential segment to which the target data belongs is determined. If all the values ​​in the higher bits are the same, the target data is a potential first segment; otherwise, the target data is a potential third segment. Based on the value of the flag bit, the segment to which the target data belongs is determined. If the value of the flag bit is 1, the target data is the second segment; if the value of the flag bit is 0, the segment to which the target data belongs is the segment number corresponding to the potential segment.

3. A tensor quantization method, wherein the tensor quantization method is applied to a convolutional neural network, characterized in that, include: Get the first tensor; Calculate the maximum absolute value of the first tensor; The quotient of the maximum absolute value and the predetermined maximum representation range is used as the scaling factor; The first tensor is scaled using the scaling factor to obtain the second tensor; Based on the absolute value of the second tensor, determine the segment to which the second tensor belongs. Specifically, if the absolute value of the second tensor is less than 2... d If the absolute value of the second tensor is 2, then the second tensor is the first segment; if the absolute value of the second tensor is 2 d and 2 k-2+d If the absolute value of the second tensor is greater than 2, then the second tensor is the second segment; k-2+d Then the second tensor is the third segment; The second tensor is quantized according to the segment to which it belongs; The product of the scaling factor and the quantized second tensor is used as the original numerical range of the first tensor.

4. The tensor quantization method according to claim 3, characterized in that, Based on the segment to which the second tensor belongs, the second tensor is quantized, including: If the segment to which the second tensor belongs is the first segment, then the second tensor x is processed according to the following formula. s Quantification: x pi =round(x s ); Where round is the rounding function, x pi This is the second tensor after quantization; or, If the segment to which the second tensor belongs is the second segment, then the second tensor x is processed according to the following formula. s Quantification: x pi =round(x s / 2 d )×2 d ; or, If the segment to which the second tensor belongs is the third segment, then the second tensor x is processed according to the following formula. s Quantification: x pi =round(x s / 2 k-2 )×2 k-2 ; Wherein, d is the segmentation bit when using the data representation method described in claim 1 or 2, and k is the preset bit width.

5. A multiply-accumulate calculation device, wherein the multiply-accumulate calculation device is applied to a convolutional neural network, characterized in that, include: The acquisition module is used to acquire the first input data and the second input data; The multiplier is used to multiply the first signed number corresponding to the first input data and the second signed number corresponding to the second input data to obtain an initial product; The first judgment module is used to determine whether all the values ​​of the first high-order bits corresponding to the first input data are the same; The second judgment module is used to determine whether all the values ​​of the second high-order bits corresponding to the second input data are the same; The decoder is used to determine the first segment to which the first input data belongs and the second segment to which the second input data belongs based on the first flag bit of the first input data, the second flag bit of the second input data, the first judgment result of the first judgment module, and the second judgment result of the second judgment module, and to determine the shift number of the initial product based on the first segment to which the first input data belongs and the second segment to which the second input data belongs; A shifter is used to shift the initial product according to the shift number determined by the decoder to obtain a shifted product; An adder is used to add the shifted product to the acquired third input data to obtain a multiply-add result.

6. The multiply-accumulate calculation device according to claim 5, characterized in that, The first judgment module includes: a first AND gate, a first NOR gate, and a first OR gate, wherein the output of the first AND gate and the output of the first NOR gate are both connected to the input of the first OR gate; The first AND gate is used to determine that the first output result is high when all the first high bits are high. The first NOR gate is used to determine that the second output result is high when all the first high-level bits are low. The first OR gate is used to determine that all the values ​​of the first high-order bits corresponding to the first input data are the same when the first output result is high and / or the second output result is high.

7. The multiply-accumulate calculation device according to claim 5, characterized in that, The second judgment module includes: a second AND gate, a second NOR gate, and a second OR gate, wherein the output terminals of the second AND gate and the second NOR gate are both connected to the input terminal of the second OR gate; The second AND gate is used to determine that the third output result is high when all the second high bits are high. The second NOR gate is used to determine that the fourth output result is high when all the second high bits are low. The second OR gate is used to determine that all the values ​​of the second high-order bits corresponding to the second input data are the same when the third output result is high and / or the fourth output result is high.

8. The multiply-accumulate calculation device according to claim 5, characterized in that, The decoder includes: The first determining unit is configured to determine the first segment to which the first input data belongs based on the first flag bit of the first input data and the first judgment result of the first judgment module; The second determining unit is used to determine the second segment to which the second input data belongs based on the second flag bit of the second input data and the second judgment result of the second judgment module; The summation unit is used to sum the first and second segments to obtain the sum of the segments; The third determining unit is used to determine the shift number of the initial product based on the correspondence between the sum of the segments and the shift number of the initial product.

Images