An accelerator device for neural network post-training quantization offset correction

Abstract

The present invention discloses an accelerator device for post-training quantization offset correction of a neural network. The offset correction operation module outputs the data of a convolution output image to an off-chip after performing offset correction calculation. The offset correction operation module includes an accumulation module and a calculation module. The accumulation module calculates the cumulative sum of corresponding input data according to a data stream. The cumulative sum of the input data is temporarily stored in a cache unit of the accumulation module. The convolution calculation result of the input data and weight data is temporarily stored in an on-chip part and cache. The calculation module is responsible for completing multiplication and addition operations on the convolution calculation result and data such as an offset correction parameter and outputting the result to an off-chip. The offset correction calculation of the weight data is equivalent to the correction of the convolution calculation result. By adding the corresponding offset value to each output image result and multiplying it by the corresponding parameter to perform offset correction, the inference accuracy of the post-quantization network model is improved, and the offset correction calculation method is implemented with lower hardware overhead.

Description

Technical Field

[0001] The present invention relates to the field of offset correction calculation technology, and in particular to an accelerator device for post-training quantization offset correction of a neural network. Background Art

[0002] Quantization of convolutional neural networks can effectively reduce the memory requirements of the model. In the field of convolutional neural network accelerated computing hardware design, quantization-friendly dedicated domain architecture processors have been widely studied. When performing model inference on hardware, most of the energy is consumed in the data movement process of the memory. For the same amount of data movement per unit time, the use of a lower-precision quantization strategy can transport more weight data in a single move and calculate more output image pixels per unit time. Low-precision quantization can reduce the chip area required for dedicated hardware design and reduce chip power consumption.

[0003] In 4-bit low-precision post-training quantization, there is a certain gap in the statistical distribution of the weight data before and after quantization, such as the mean and root mean square error. This will lead to a decrease in the inference accuracy of the quantized network model. Generally speaking, models trained with quantization awareness can achieve good inference accuracy at 4-bit precision, but this method consumes a lot of resources during the quantization process. Existing compensation technology uses the difference in mean and root mean square error between the weight data before and after quantization to correct the quantized weight data. The purpose is to make the statistical distribution of the weight after quantization closer to the data before quantization, thereby improving the inference accuracy of the quantized model. Although the use of offset correction technology can improve the accuracy of the quantized model, this offset correction technology for weight data introduces an offset to the uniformly distributed fixed point. In hardware implementation, it will make the low-precision fixed-point multiplier ineffective, offsetting the low-precision computing advantage brought by quantization. Summary of the Invention

[0004] The purpose of the present invention is to provide an accelerator device for post-training quantization offset correction of a neural network, thereby improving the inference accuracy of the quantized model and realizing the offset correction operation with lower hardware overhead.

[0005] An accelerator device for post-training quantization offset correction of a neural network includes an offset correction operation module, which performs offset correction calculation on the data of a convolution output image and outputs it to an off-chip device. The offset correction operation module includes an accumulation module and a calculation module. The accumulation module calculates the cumulative sum of corresponding input data according to a data stream. The accumulation module is provided with a cache unit, and the cumulative sum of the input data is temporarily stored in the cache unit of the accumulation module. The calculation module includes two multipliers and one adder. Convolution calculation is performed on the input data and weight data. When performing offset correction calculation on each convolution calculation result, two multiplication calculations and one addition calculation are required. The formula for performing offset correction calculation on the convolution calculation result is as follows:

[0006]

[0007] Among them, S represents the sum of the input data involved in the calculation, It represents the quantized data of the input image. It represents the weight data after quantization, μ represents the difference between the mean of the weight data before quantization and the mean of the weight data after quantization, and ξ represents the ratio of the two norms of the weight data before quantization to the weight after quantization.

[0008] The quantized weight data is corrected by using the difference in mean and root mean square error between the weight data before and after quantization. This offset correction method applied to the weight data can make the statistical distribution of the quantized weight closer to the data before quantization, thereby improving the inference accuracy of the quantized model.

[0009] Furthermore, the input data and weight data of the offset correction calculation formula are calculated in the original convolution method to obtain the output image result, which is temporarily stored in the cache. When the result is output to the outside of the chip, each output image result in the cache is taken out and added with its corresponding offset value μS, and then multiplied by the corresponding ξ for offset correction.

[0010] In the low-precision post-training quantization with a 4-bit width, there is a certain gap in the statistical distribution of the weight data before and after quantization, such as the mean and root mean square error. By adding the corresponding offset value μS to each output image result and multiplying it by the corresponding ξ to perform offset correction, the inference accuracy of the network model after quantization is improved.

[0011] Furthermore, the accelerator architecture adopted by the present invention includes multiple planes, and a set of accumulation modules is provided for each plane in the offset correction calculation module, and the calculation results of the accumulation modules are multiplexed by multiple output channels of the plane to which they belong.

[0012] Furthermore, the offset correction operation module is suitable for a systolic array-like architecture and an input data multiplexing architecture. The accelerator architecture adopted by the present invention includes an input data delay unit and a weight data delay unit, which are used to adapt to calculations with different convolution kernel sizes and convolution step sizes.

[0013] Furthermore, the offset correction operation module also includes a multiplication operation between S and the parameter μ. In this example, the bit width of the parameter μ is 8 bits, and the bit width of the S value is 16 bits. The multiplication operation of μ and S obtains a 24-bit intermediate result, which is saturated and truncated to 16 bits in the offset correction operation module. This 16-bit intermediate result is added to the corresponding convolution calculation result value, and then the data result of this addition is multiplied by the parameter ξ.

[0014] Furthermore, in the architecture of this example, the offset correction operation module is equipped with 4 groups of calculation modules, each group of calculation modules is responsible for calculating the offset correction of an 8-bit convolution calculation result, and the convolution calculation results of 32 output channels are stored at one address of the partial sum cache. The bit width of each convolution calculation result is 8 bits, so it takes a total of 8 clock cycles to read out all the values ​​at one address of the partial sum cache and complete the offset correction calculation. Within these 8 clock cycles, each clock cycle performs a +1 operation on the read address of the on-chip cache of the offset correction calculation parameter μ and ξ.

[0015] Furthermore, the read address of the on-chip cache of the offset correction calculation parameters μ and ξ includes three operations, a read address +1 operation, a read address -7 operation and a read address clear operation. The read address clear operation of the on-chip cache of the offset correction calculation parameters μ and ξ occurs when the offset correction calculation and the switching part and the cache are started. The read address -7 operation of the on-chip cache of the offset correction calculation parameters μ and ξ occurs when the offset correction calculation of 32 output channels is processed each time (and at this time a row of convolution calculation results in the output image has not been processed). The read address +1 operation of the on-chip cache of the offset correction calculation parameters μ and ξ occurs in the normal stage of the offset correction calculation and when the convolution calculation results of a row of 32 output channels of the output image are processed.

[0016] Furthermore, the parameters μ and ξ are stored in the on-chip cache of the offset correction calculation parameters μ and ξ. In this design, the capacity of the on-chip cache of the offset correction calculation parameters μ and ξ is 0.5kB. The offset correction parameters μ and ξ of each channel used in the offset correction calculation are preloaded into the on-chip cache of the offset correction calculation parameters μ and ξ. After completing the convolution calculation of all input data and weight data in a convolution layer, the partial sum cache and the cumulative cache respectively store the convolution calculation results of this layer and the cumulative sum of the input data. The offset correction operation module performs offset correction calculation on the convolution calculation results, and the output port of the offset correction operation module is connected to the outside of the chip.

[0017] Furthermore, the on-chip cache for the offset correction calculation parameters μ and ξ consists of two static random access memories called MK_SRAM. In this example, the depth of each MK_SRAM is 64 and the bit width is 32. The data bit width of parameters μ and ξ is 8 bits. Each output channel has a set of μξ parameters, so one address in each MK_SRAM can store two sets of μξ parameters. Four groups of calculation modules are configured in the offset correction operation module. When performing offset correction calculations, four groups of μξ parameters will be taken out from the two MK_SRAMs simultaneously for calculation in one cycle.

[0018] Furthermore, the offset correction calculation module also includes a data loading control unit, which is used to load the μξ parameters from outside the chip into the on-chip cache of the offset correction calculation parameters μ and ξ. The bit width of the chip data interface in this design is 32 bits. During the data loading stage, two sets of μξ parameters can be transported from outside the chip to the MK_SRAM in one cycle.

[0019] The advantages of the present invention are that the offset correction calculation of the weight data is equivalent to the correction of the convolution calculation result. By adding the corresponding offset value μS to each output image result and multiplying it by the corresponding ξ to perform offset correction, the inference accuracy of the quantized network model is improved, and the offset correction calculation method is implemented with lower hardware overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic diagram of offset correction calculation.

[0021] Figure 2 This is a schematic diagram of the architecture of an offset correction calculation module.

[0022] Figure 3 It is a control logic diagram for offset correction calculation.

[0023] Figure 4 It is a schematic diagram of the data flow of the offset correction operation module.

[0024] Figure 5 A schematic diagram of an on-chip cache of an offset correction operation module is shown. DETAILED DESCRIPTION

[0025] As an implementation method, Figure 1 In the example shown, the convolution step size is 1, the input image dimension is 4*4*1 (height*width*number of input channels), the convolution kernel dimension is 2*2*1 (width*height*number of output channels), X 11 , X 12 , X 21 , X 22 With W 11 , W 12 , W21 , W 22 The result of the convolution calculation is Y 11 , in the offset correction calculation formula,

[0026]

[0027] The value of Y 11 , S is X 11 , X 12 , X 21 , X 22 The sum of the input data and the weight data is first calculated to obtain the output image result and temporarily stored in the partial and on-chip cache. When the result is output to the outside of the chip, each output image result in the cache is taken out and added with the corresponding offset value μS, and multiplied by the corresponding ξ to achieve the purpose of offset correction.

[0028] Preferably, in the calculation process of the convolution layer, the input data of the first layer is determined, and the S value required for the calculation of the first layer can be accumulated in advance in the preprocessing stage. The input data for the calculation of the remaining convolution layers all come from the output of the previous layer. When the subsequent convolution layers are accelerated, the accumulation process of the input data cannot be completed in the preprocessing stage. Therefore, it is necessary to design the function of accumulating input data during operation on the chip. The offset correction operation module of the present technical solution includes a module for realizing the accumulation calculation of input data. Each multiplication and addition calculation of the weight data and the input data will be accompanied by the accumulation operation of the input data. The accumulation operation realized by combining the architecture and data reuse can save hardware resources and reduce power consumption.

[0029] Preferably, the weight offset correction calculation method focuses on the difference between the mean and the second norm of the weight data before and after quantization, as shown in the following formula:

[0030]

[0031] Among them, the quantization of weights adopts a channel-based quantization method, where W refers to the weight data of a certain output channel. Refers to the quantized data of W, is the mean of the W data, the parameter μ represents the difference between the mean of the weight data before quantization and the weight data after quantization, and the parameter ξ represents the ratio of the two norms of the weight data before quantization and the weight after quantization. The offset correction calculation method uses the parameters μ and ξ to correct the quantized weight. The process is to increase the quantized weight by μ and then multiply it by ξ to achieve the purpose of correcting the quantized weight, as shown in the following formula:

[0032]

[0033] Among them, W bRepresents the corrected weight data, Represents the quantized data of the input image. Represents the convolution calculation of the input data and the corrected weight data.

[0034] As an implementation method, due to the multiplication and addition characteristics of convolution calculation, the offset correction calculation of weight data can be equivalent to the correction of the convolution calculation result, as shown in the following formula:

[0035]

[0036] Among them, S represents the sum of the input data involved in the calculation, It represents the quantized data of the input image. It represents the weight data after quantization, μ represents the difference between the mean of the weight data before quantization and the mean of the weight data after quantization, and ξ represents the ratio of the two norms of the weight data before quantization to the weight after quantization.

[0037] As an implementation method, Figure 2 As shown, this technical solution adopts a 3x7x32 convolution-specific accelerator architecture and adds a special offset correction operation module. The offset correction operation module includes two parts: an accumulation module (ACC) and a calculation module (2M1A). The accumulation module (ACC) is used to calculate the cumulative sum of the corresponding input data according to the data stream. The cumulative sum of the input data will be temporarily stored in the accumulation and cache on-chip cache. The calculation module (2M1A) consists of 2 multipliers and 1 adder. When performing offset correction calculation on each convolution calculation result, a total of 2 multiplication calculations and 1 addition calculation are required. The output of the offset correction operation module is the result after offset correction.

[0038] As an implementation method, Figure 2 As shown, each xy plane in the architecture is equipped with a set of accumulation modules. The calculation results of the accumulation modules can be reused by different output channels of the plane to which they belong. The architecture includes an on-chip cache for offset correction calculation parameters, an on-chip cache for input data, an on-chip cache for weight data, an on-chip cache for partial sums, etc. PE is a multiplication-addition unit. The efficiency of the offset correction calculation module designed in the present invention depends on the data reuse capability of the accelerator architecture. The offset correction calculation module is designed and optimized using a 3x7x32 quasi-systolic array accelerator architecture. The offset correction calculation module designed in the present invention is not only applicable to quasi-systolic array architectures, but also to all architectures that implement input data reuse.

[0039] As an implementation method, Figure 2In the architecture shown, every three cascaded PEs in the y direction form a PE chain, and the seven groups of PE chains in each yz plane share the same weight data memory, realizing the reuse of weight data. The 32 groups of PE chains in each xy plane share the same input data memory, realizing the reuse of input data. The input data delay module (IDelay) and the weight data delay module (W_Delay) are used to adapt to the calculation of different convolution kernel sizes and convolution step sizes. In this example, each weight data is quantized to 4 bits, the input data is quantized to 8 bits, the parameters μ and ξ used for offset correction calculation are 8 bits respectively, the cumulative sum data calculated by ACC is 16 bits, the convolution calculation result calculated by PE is 8 bits, and the final output of the offset correction operation module is 8 bits.

[0040] Preferably, the S value corresponding to each convolution calculation result of the output image in the offset correction operation module can be implemented by an accumulation module (ACC). To improve energy efficiency, the ACC should be designed in combination with the overall accelerator architecture and data flow during implementation. Taking a 3x7x32 PE array and a row-fixed data flow as an example, the input data is reused in the x dimension, that is, the input data will be reused 32 times. Referring to this input data reuse pattern, for a 3x7x32 PE array, only 3x7 ACC modules are required to complete the input cumulative sum calculation, and each cumulative sum result S can also be reused 32 times.

[0041] Preferably, the overall accelerator architecture has a total of seven xy planes. The input data for the PEs on each xy plane when calculating multiplication and addition is the same. Therefore, it is only necessary to equip each xy plane with a set of input data accumulation calculation chains. A set of input data accumulation calculation chains consists of three cascaded accumulation calculation units ACC. Three ACC modules are set in the y direction of each xy plane to implement the input data accumulation function. The accumulated sum of the input data on each plane will be stored in its corresponding accumulation cache. In this example, in order to adapt to the calculation of the offset correction operation module, a total of seven on-chip caches are configured on the chip for storing the accumulated sum of the input data. The capacity of each block is 448B (bytes), and the total capacity of the accumulation cache is 3.06kB.

[0042] Preferably, in addition to the calculation of the S value, the offset correction operation module also includes a multiplication operation between S and the parameter μ. The bit width of the parameter μ is 8 bits, and the bit width of the S value is 16 bits. An 8-bit * 16-bit multiplier is required to support it. The multiplication of μ and S will obtain a 24-bit result, which is saturated and truncated to 16 bits in the offset correction operation module. This result is then added to the value of the corresponding position of the output image, and an adder is required here. Finally, the result of this addition needs to be multiplied by the parameter ξ, and a multiplier is required here. This configuration of two multipliers and one adder is called 2M1A.

[0043] As an implementation method, Figure 3 As shown in the figure, the storage method of the offset correction calculation parameter μξ on the chip and the control logic during the offset correction calculation are shown, where PSUM_Addr refers to the read address of the partial sum cache and BCINFO_Addr refers to the read address of the offset correction parameter cache. Specifically, each convolution output channel has a set of parameters μξ, Figure 3 (a) is a convolution calculation example used to illustrate the offset correction calculation process. The input image size is 4*4*3 (height*width*number of channels), the convolution kernel size is 3*3*3*64 (height*width*number of input channels*number of output channels), the convolution step size is 1, and the output image size of the convolution calculation is 2*2*64 (height*width*number of channels). Figure 3 (c) is the control logic for reading μξ from the offset correction parameter cache to the calculation module. For the read address of the offset correction parameter cache, it includes the following operations: read address +1, read address -7 and read address clear. Figure 3 As shown in (a) and (b), in the output image cube, (0, 0, 0) represents the convolution calculation result of the upper left corner of the 0th output channel, and (1, 1, 63) represents the convolution calculation result of the lower right corner of the 63rd output channel. The values ​​of the output image are stored in the partial and cache in a certain order.

[0044] As an implementation method, Figure 4 As shown, the offset correction operation module is equipped with 4 groups of 2M1A, each group of 2M1A is responsible for calculating the offset correction of an 8-bit convolution calculation result, so it takes a total of 8 clock cycles to read out all the convolution calculation results on an address in the partial and cache.

[0045] Preferably, in these 8 clock cycles, the read address of the offset correction parameter cache needs to be increased by 1 in each clock cycle. Each +1 operation represents the switching of the μξ parameters corresponding to the next 4 output channels. In the 8th clock cycle, when the convolution calculation result of the output image position (0, 0, 31) is read out and the offset correction calculation is completed, the read address PSUM_Addr of the partial sum cache increases from 0 to 1. The data read from the partial sum cache are (0, 1, 0), (0, 1, 1), ..., (0, 1, 31), etc. The output channels corresponding to these convolution calculation results are 0 to 31, respectively. Therefore, In the 8th clock cycle, the read address of the offset correction parameter cache needs to be decremented by 7 to obtain the μξ parameter corresponding to the output image convolution calculation result. In the 16th clock cycle, the offset correction operation module has completed the offset correction calculation for all convolution calculation results in the first row with output channels 0 to 31. Next, the offset correction calculation for the first row of convolution calculation results with output channels 32 to 63 needs to be processed. Therefore, in this clock cycle, BCINFO_Addr needs to be increased by 1, that is, from 7 to 8. BCINFO_Addr = 8 corresponds to the μξ parameter for output channels 32 to 35.

[0046] Preferably, at the 24th clock cycle, the offset correction operation module has completed the offset correction calculation of all convolution calculation results (0, 0, 32) to (0, 0, 63). At this time, BCINFO_Addr has increased to 15. Next, the offset correction calculation of the convolution calculation results (0, 1, 32) to (0, 1, 63) needs to be performed. Therefore, in this clock cycle, BCINFO_Addr needs to be reduced by -7 to reduce it from 15 to 8, which corresponds to the μξ parameters of output channels 32 to 35. At the 32nd clock cycle, the convolution calculation results of all output channels 0 to 63 have completed the offset correction calculation and are output to the outside of the chip. In this clock cycle, it will switch to partial sum cache 2 to perform the offset correction calculation of the second row of convolution calculation results. Therefore, the read address of the offset correction parameter cache needs to be cleared to start from output channel 0 again. The subsequent read address change logic is consistent with the calculation of partial sum cache 1.

[0047] Preferably, it is ensured that each convolution calculation result can correctly enter the offset correction operation module with the μξ parameter of its corresponding output channel when performing the offset correction calculation, the read address clearing operation of the offset correction parameter cache occurs when the offset correction calculation and the switching part and the cache are started, the read address -7 operation of the offset correction parameter cache occurs when the offset correction calculation of 32 output channels is processed each time (and at this time a row of convolution calculation results in the output image has not been processed), and the read address +1 operation of the offset correction parameter cache occurs in the normal stage of the offset correction calculation and when the convolution calculation results of a row of 32 output channels of the output image are processed.

[0048] As an implementation method, Figure 5 As shown, parameters μ and ξ are stored in the offset correction parameter cache BCINFO_MEM. In this example, the capacity of BCINFO_MEM is 0.5kB and is preloaded during the initialization phase of the hardware acceleration calculation. The offset correction parameters μ and ξ for each channel required for a convolution layer are loaded into the offset correction parameter cache. After the convolution calculation of all input data and weight data in a convolution layer is completed, the output result and the cumulative sum of the input data of this layer are prepared in the partial sum cache and the cumulative sum cache, respectively. This generates an enable signal for the offset correction operation module to begin offset correction calculation of the output result. The output port of the offset correction operation module is directly connected to the output data port of the chip, which is the corrected output result.

[0049] Preferably, the offset correction parameter cache is composed of 2 blocks of MK_SRAM. MK_SRAM is an on-chip cache for storing μξ parameters. It is a static random access memory, that is, the on-chip cache for offset correction parameters described in the entire patent. The size of each MK_SRAM is 64*32 (depth*bit width), that is, 2 sets of μξ parameters can be placed on one address. In order to match the output result bit width of the offset correction operation module with the chip output port bit width (32 bits), the offset correction operation module is configured with a total of 4 groups of 2M1A calculation modules. When performing offset correction calculations, 4 groups of μξ parameters will be taken out from the offset correction parameter cache for calculation in one cycle. The reading of μξ parameters from off-chip to the offset correction parameter cache is implemented in the data loading control module. The bit width of the chip data interface is 32 bits. Therefore, in the data loading stage, 2 sets of μξ parameters can be transported from off-chip to on-chip in one cycle.

[0050] Preferably, there are several key signals when writing data to the offset correction parameter cache. When the write enable signal WR_EN of MK_SRAM is valid, the write address of MK_SRAM will increase by 1 every cycle. MK_SRAM_SEL is the chip select signal for the two SRAMs. MK_SRAM_SEL is 1 bit wide and flips on each rising edge of the clock while the write enable signal of MK_SRAM is valid. Specifically, when MK_SRAM_SEL = 0, it represents chip selection MK_SRAM0, and the μξ parameter at this moment will be written to MK_SRAM0. When MK_SRAM_SEL = 1, it represents chip selection MK_SRAM1, and the μξ parameter at this moment will be written to MK_SRAM1.

[0051] Preferably, after the accelerator completes a batch of convolution calculations, the convolution calculation results are stored in the partial and cache. In this example accelerator architecture, a total of 7 partial and caches are equipped, and the bit width of the partial and cache is 256 bits. Each partial and cache stores the convolution calculation results of its corresponding xy plane. The bit width of the multiplication and addition result calculated by PE is 8 bits. 32 convolution calculation results will be stored at one address of each partial and cache, corresponding to 32 output channels respectively. When reading data from the partial and cache, in order to match the bit width of the chip output port, 4 convolution calculation result data will be read from the partial and cache in each cycle when performing offset correction calculation and sent to the offset correction operation module for calculation.

[0052] The beneficial effects of the present invention are as follows: the offset correction calculation of the weight data is equivalent to the correction of the convolution calculation result. By adding the corresponding offset value μS to each output image result and multiplying it by the corresponding ξ to perform offset correction, the inference accuracy of the quantized network model is improved, and the offset correction calculation method is implemented with lower hardware overhead.

[0053] All patents and publications cited in this specification are intended to indicate that they are state of the art and that the present invention may be used. All patents and publications cited herein are incorporated by reference in their entirety, as if each publication were specifically incorporated by reference. The invention described herein may be practiced in the absence of any element or elements, limitation or limitations, unless otherwise specified. For example, in each instance, the terms "comprising," "consisting essentially of," and "consisting of" may be replaced with either of the other two terms. The terms and expressions used herein are intended to be descriptive, not limiting, and there is no intention to exclude any equivalent features from the terms and interpretations herein. However, it is understood that any suitable changes or modifications may be made within the scope of the present invention and the appended claims. It is understood that the embodiments described herein are preferred embodiments and features, and that modifications and variations may be made by persons of ordinary skill in the art based on the spirit of the present invention. Such modifications and variations are considered to be within the scope of the present invention and the scope of the independent and appended claims.

Claims

1. An accelerator device for post-training quantization offset correction of a neural network, characterized by: The offset correction operation module includes an offset correction operation module, which performs offset correction calculation on the data of the convolution output image and outputs the result to the outside of the chip; The offset correction operation module includes an accumulation module and a calculation module. The accumulation module calculates the cumulative sum of the corresponding input data according to the data stream. The accumulation module is provided with a cache unit. The cumulative sum of the input data is temporarily stored in the cache unit of the accumulation module. The calculation module includes two multipliers and one adder. The input data is convolved with the weight data. When performing offset correction calculation on each convolution calculation result, two multiplication calculations and one addition calculation are required. The formula for performing offset correction calculation on the convolution calculation result is as follows: Among them, S represents the sum of the input data involved in the calculation, It represents the quantized data of the input image. It represents the weight data after quantization, μ represents the difference between the mean of the weight data before quantization and the mean of the weight data after quantization, and ξ represents the ratio of the two norms of the weight data before quantization to the weight after quantization; The input data and weight data of the offset correction calculation formula are calculated in the original convolution mode to obtain the output image result, which is temporarily stored in the cache. When outputting the result to the outside of the chip, each convolution calculation result in the cache is taken out and added to the corresponding offset value μS, and then multiplied by the corresponding ξ to perform offset correction; The offset correction calculation module is equipped with four groups of calculation modules, each of which is responsible for calculating the offset correction of an 8-bit convolution calculation result. The convolution calculation results of 32 output channels are stored at one address in the on-chip partial and cache. The bit width of each convolution calculation result is 8 bits. Therefore, it takes a total of 8 clock cycles to read all the values ​​at one address in the partial and on-chip cache and complete the offset correction calculation. During these 8 clock cycles, the offset correction calculation parameter μ is added to the read address of the on-chip cache of ξ by +1 in each clock cycle.

2. The accelerator device for neural network post-training quantization offset correction according to claim 1, characterized in that: The architecture of the accelerator device includes multiple planes. In the offset correction calculation module, each plane is equipped with a set of accumulation modules, and the calculation results of the accumulation modules are multiplexed by multiple output channels of the plane to which they belong.

3. The accelerator device for neural network post-training quantization offset correction according to claim 1, characterized in that: The offset correction operation module is suitable for a systolic array-like architecture and an input data multiplexing architecture. The accelerator architecture includes an input data delay unit and a weight data delay unit to adapt to calculations with different convolution kernel sizes and convolution step sizes.

4. The accelerator device for neural network post-training quantization offset correction according to claim 1, characterized in that: The offset correction operation module further includes a multiplication operation between S and a parameter μ. The bit width of the parameter μ is 8 bits, and the bit width of the S value is 16 bits. The multiplication operation of μ and S obtains a 24-bit intermediate result, which is saturated and truncated to 16 bits in the offset correction operation module. This 16-bit intermediate result is added to the corresponding convolution calculation result value, and then the data result of this addition is multiplied by the parameter ξ.

5. The accelerator device for neural network post-training quantization offset correction according to claim 1, characterized in that: The read address of the on-chip cache of the offset correction calculation parameters μ and ξ includes three operations: a read address +1 operation, a read address -7 operation, and a read address clear operation. The read address clear operation of the on-chip cache of the offset correction calculation parameters μ and ξ occurs when the offset correction calculation and switching part and cache are started. The read address -7 operation of the on-chip cache of the offset correction calculation parameters μ and ξ occurs when the offset correction calculation of 32 output channels is completed each time, and at this time, a row of convolution calculation results in the output image has not been processed. The read address +1 operation of the on-chip cache of the offset correction calculation parameters μ and ξ occurs in the normal stage of the offset correction calculation and when the convolution calculation results of one row of 32 output channels of the output image are processed.

6. The accelerator device for neural network post-training quantization offset correction according to claim 1, characterized in that: The parameters μ and ξ are stored in an on-chip cache of the offset correction calculation parameters μ and ξ. The capacity of the on-chip cache of the offset correction calculation parameters μ and ξ is 0.5kB. The offset correction parameters μ and ξ of each channel used in the offset correction calculation are pre-loaded into the on-chip cache of the offset correction calculation parameters μ and ξ. After completing the convolution calculation of all input data and weight data in a convolution layer, the partial sum cache and the cumulative sum cache respectively store the convolution calculation results of this layer and the cumulative sum of the input data. The offset correction operation module performs offset correction calculation on the convolution calculation results, and the output port of the offset correction operation module is connected to the outside of the chip.

7. The accelerator device for neural network post-training quantization offset correction according to claim 1, characterized in that: The on-chip cache for the offset correction calculation parameters μ and ξ consists of two static random access memories named MK_SRAM. Each MK_SRAM has a depth of 64 and a bit width of 32 bits. The data bit width of parameters μ and ξ is 8 bits. Each output channel has a set of μξ parameters, so one address in each MK_SRAM stores two sets of μξ parameters. The offset correction operation module is configured with four groups of calculation modules. When performing offset correction calculations, four sets of μξ parameters are simultaneously retrieved from the two MK_SRAMs for calculation in one cycle.

8. The accelerator device for neural network post-training quantization offset correction according to claim 1, characterized in that: The offset correction calculation module also includes a data loading control unit, which is used to load the μξ parameters from outside the chip into the on-chip cache of the offset correction calculation parameters μ and ξ. The bit width of the chip data interface is 32 bits. During the data loading phase, two sets of μξ parameters are transported from outside the chip to the offset correction parameter cache in one cycle.

Images