Low-bit neural network training method and accelerator

Abstract

The invention belongs to the technical field of neural network acceleration design, and provides a low-bit neural network training method and an accelerator. The bit neural network training method and the accelerator are mainly characterized in the following three aspects. Firstly, a periodic function is used in the training process to approximate the gradient of an evidence obtaining function; secondly, the low-bit computing array is adjusted when the computing parallelism of the dimensions of the input channel and the output channel can be operated; and thirdly, the low-bit neural network accelerator uses full addition convolution. According to the bit neural network training method and the accelerator, four-bit wide operation is realized, relatively good network precision is maintained, relatively high calculation parallelism and calculation speed are achieved, calculation complexity and resource requirements are reduced, and the method and the accelerator are suitable for deployment of a resource-limited platform.

Description

Technical Field

[0001] The present invention belongs to the technical field of neural network acceleration and proposes a low-bit neural network training method and an accelerator. Background Art

[0002] With the rapid development of artificial intelligence technology, neural network models are increasingly used in various applications. These models usually require a lot of computing resources. In mobile devices and embedded systems, computing resources have become a key factor limiting the application of models. To solve this problem, researchers have proposed a variety of neural network compression and acceleration technologies.

[0003] In existing technologies, there are two main methods to reduce the computational complexity of neural networks: one is network pruning, which reduces the amount of computation by removing unimportant connections in the network; the other is quantization, which reduces the number of bits of weights in the network, thereby reducing storage requirements and computational complexity.

[0004] Although existing quantization techniques can reduce the computational complexity of neural networks to a certain extent, these methods generally have the following problems: existing quantization techniques often lead to a significant decline in neural network performance, especially at low-bit quantization (such as less than four bits); existing neural network computing units are complex in design, making it difficult to achieve high computing efficiency while maintaining low power consumption; existing technologies have limitations in hardware implementation, such as poor flexibility and insufficient scalability.

[0005] In response to the above problems, this application is dedicated to proposing a low-bit neural network training method and accelerator. Summary of the invention

[0006] The purpose of the present invention is to solve the problems of low accuracy of existing low-bit convolutional neural networks and complex design of convolutional neural network computing units, which make it difficult to achieve high computing efficiency while maintaining low power consumption, and propose a low-bit neural network training method and accelerator. The method avoids excessive loss of accuracy of the neural network while quantizing the neural network to a lower bit width; the accelerator provides hardware computing support for the method, and realizes the flexibility of convolution computing by expanding in the feature channel dimension, which can adapt to convolution computing of different sizes; a configurable addition tree realizes configurable computing parallelism at runtime, and improves the versatility and flexibility of the accelerator; the accelerator also proposes a design of full additive convolution, which avoids the need for multipliers for traditional convolution, reduces the need for the accelerator for the hardware platform, and realizes a low-bit convolution accelerator with simple computing logic, low power consumption, high efficiency and high flexibility, which partially solves the problem that it is difficult to deploy neural networks on small resource platforms.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] As a first aspect of the present invention, a low-bit neural network training method is proposed, comprising the following steps:

[0009] S1. Initializing parameters of a full-precision neural network model embedded in a quantization block, including: initializing the full-precision neural network model embedded in a quantization block using parameters of the full-precision neural network model; initializing trainable parameters of the quantization block using random numbers;

[0010] The quantization blocks include a first quantization block and a second quantization block, and both include division, rounding and multiplication;

[0011] S2. Forward inference is performed on the full-precision neural network model embedded with the quantization block. The specific operation process for a certain layer is as follows: the weight of each convolutional layer is quantized by the first quantization block and then convolved with the input feature map; the convolution result is then quantized by the second quantization block to obtain the output feature map;

[0012] S3, back propagates and calculates the gradients of all layer parameters in the full-precision neural network model embedded in the quantization block; S3, when implemented specifically, the operation process for a certain layer is as follows:

[0013] S31. Calculate the gradient of the loss function and back-propagate to obtain the gradient of the output feature map of the convolutional layer;

[0014] S32, calculating the gradient of the multiplication in the second quantization block to obtain a partial gradient of the quantization step length and a gradient of a rounded result;

[0015] S33, using the rounding function gradient approximation function to replace the gradient of the rounding function in the second quantization block, to obtain the gradient of the result after the division;

[0016] S34, calculating the gradient of the division in the second quantization block to obtain the complete gradient of the quantization step length and the gradient of the input of the second quantization block;

[0017] S35, calculating the quantization bit width and the gradient of the quantization range of the second quantization block;

[0018] S36, replacing the second quantization block in S32 to S35 with the first quantization block, repeating S32 to S35, and obtaining the gradient of the first quantization block input and the gradient of the quantization bit width and quantization range of the first quantization block;

[0019] S4. Update all trainable parameters according to the calculated gradients.

[0020] The method can quantize the neural network to a lower bit width, thereby avoiding excessive loss of accuracy of the neural network.

[0021] S1: The full-precision neural network model of the embedded quantization block includes the network structure and the weights of each network layer; the trainable parameters of the quantization block include the quantization bit width and the quantization range.

[0022] Each convolution layer in the full-precision neural network model embedded with quantization blocks described in S1 is embedded with a first quantization block and a second quantization block.

[0023] The first quantization block quantizes the input weight to obtain the quantized input weight, and the weight is then convolved with the input feature map to obtain the input quantized convolution result; the second quantization block quantizes the input quantized convolution result to obtain the output feature map, which is the output of the convolution layer.

[0024] The quantification described in S2 includes the following sub-processes:

[0025] S21, calculating the quantization step size based on the trainable parameters;

[0026] S22. Divide the input of the quantization block by the quantization step size, round the result, and then multiply it by the quantization step size to obtain a quantized parameter.

[0027] The input of the quantization block, for the first quantization block, corresponds to the weight of the convolution layer; for the second quantization block, it is the convolution result of the weight of the convolution layer after quantization and convolution with the input feature map.

[0028] S36: the input of the first quantization block, i.e., the weight of the convolutional layer; S4: all the trainable parameters, including the parameters of the full-precision neural network model embedded in the quantization block and the training parameters of the quantization block.

[0029] A full-additive low-bit neural network accelerator is connected to a unified memory and a general-purpose processor, and includes a hardware data rearrangement module, a cache scheduling module, an input feature map cache, a parallel computing array, and an output feature map cache; the hardware data rearrangement module is connected to the unified memory and the cache scheduling module; the cache scheduling module is connected to the hardware data rearrangement module, the general-purpose processor, the input feature map cache, and the output feature map cache; the input feature map cache is connected to the unified memory, the cache scheduling module, and the parallel computing array; the output feature map cache is connected to the unified memory, the cache scheduling module, and the parallel computing array; the parallel computing array is connected to the cache scheduling module, the input feature map cache, and the output feature map cache.

[0030] The hardware data rearrangement module receives the size and data address information of each layer of input feature map from the cache scheduling module, reads the data information of the input feature map of the current layer from the unified memory according to the size and data address information, converts the arrangement of the data into the order of height, width, and number of feature channels, and stores it back into the unified memory;

[0031] The cache scheduling module receives the input feature map size, input feature map data address, weight size, weight data address, output feature map size, output feature map data address and current layer calculation type information from the general processor, and sends the input feature map size and input feature map data address to the hardware data rearrangement module at the beginning of each layer calculation; calculates the parallelism division information of the parallel computing array according to the input feature map size and weight size and sends it to the parallel computing array; calculates different input cache read addresses and input cache read lengths according to the parallelism division information, input feature map size, input feature map data address, weight size and weight data address, and sends them to the input feature map cache; calculates different output cache write addresses and output cache write lengths according to the parallelism division information, output feature map size and output feature map data address, and sends them to the output feature map cache;

[0032] The input feature map cache receives information such as the input cache read address and the input cache read length from the cache scheduling module, reads the divided input feature map data and weight data from the unified memory, and sends them to the parallel computing array;

[0033] The output feature map cache receives information such as the output cache write address and the output cache write length from the cache scheduling module, and sends the output feature map data of the blocks from the parallel computing array to the unified memory;

[0034] The parallel computing array receives the parallelism division information from the cache scheduling module, and configures the parallelism of the parallel computing of the input channel and the output channel according to the parallelism division information; reads the input feature map data and weight data of the blocks from the input feature map cache for convolution calculation, and obtains the output feature map after the calculation is completed and stores it in the output feature map cache;

[0035] The parallel computing array includes a weight cache array, a first-level adder tree array, a second-level adder tree unit, an accumulator array and a quantization unit array; the weight cache array is connected to the input feature map cache and the first-level adder tree array, the first-level adder tree array is connected to the input feature map cache and the weight cache array; the second-level adder tree unit is connected to the first-level adder tree array and the accumulator array; the accumulator array is connected to the second-level adder tree unit and the quantization unit array; the quantization unit array is connected to the accumulator array and the output feature map cache.

[0036] The weight cache array includes several weight caches; each first-level adder tree array includes several first-level adder trees; each first-level adder tree includes three adders, which can realize low-bit multiplication calculation; the second-level adder tree unit includes an adder level controller and several levels of adders; the adder level controller is used to enable the adder level; the accumulator array includes several accumulators; the quantization unit array includes several quantization units; the accumulator array receives the addition result from the second-level adder tree unit, and after performing the accumulation of the specified number of cycles, passes the accumulation result to the quantization unit array; the quantization unit array receives the accumulation result from the accumulator array, performs the quantization operation, obtains the output feature map result, and sends it to the output feature map cache.

[0037] The weight cache array receives the weight data transmitted by the input feature map cache, temporarily stores the weight data in the weight cache, and transmits it to the first-level adder tree array when needed; the first-level adder tree array reads the input feature map data and the weight data from the input feature map cache and the weight cache, calculates the multiplication result and transmits the multiplication result to the second-level adder tree unit; the second-level adder tree unit receives the multiplication result from the first-level adder tree array, performs the addition operation of the specified number of levels according to the configuration information, and transmits the addition result to the accumulator array;

[0038] The accelerator provides hardware computing support for the method, and realizes the flexibility of convolution computing by expanding in the feature channel dimension, which can adapt to convolution computing of different sizes; the accelerator also proposes a design of full additive convolution, which avoids the need for multipliers in traditional convolution, reduces the demand for hardware platform of the accelerator, realizes a low-bit convolution accelerator with simple computing logic, low power consumption, high efficiency and high flexibility, and partially solves the problem that it is difficult to deploy neural networks on small resource platforms. A configurable addition tree design realizes configurable computing parallelism at runtime, which improves the versatility and flexibility of the accelerator.

[0039] Beneficial effects:

[0040] The present invention proposes a low-bit neural network training method and accelerator, which has the following beneficial effects compared with the existing low-bit neural network:

[0041] 1. The low-bit neural network training method proposes an integer function gradient approximation function, which improves the gradient mismatch problem caused by the approximate direct estimation method used in the traditional low-bit neural network training method and improves the low-bit neural network training accuracy;

[0042] 2. The low-bit neural network accelerator can configure the parallelism of convolution calculation in the input channel dimension and the output channel dimension during operation, which improves the problem of low versatility of traditional neural network accelerators, so that the low-bit neural network accelerator can simultaneously calculate convolutions of different feature map channel numbers and different groups, thereby improving the versatility and calculation speed of the neural network accelerator;

[0043] 3. The bit width of the input feature map data, weight data and output feature map data of the convolution calculation of the low-bit neural network accelerator is four bits. Compared with the traditional neural network accelerator using eight bits, the low-bit neural network accelerator can consume fewer resources and achieve higher parallelism;

[0044] 4. The low-bit neural network accelerator uses full addition operations to calculate convolution, which reduces the demand for computing power of the neural network accelerator, allowing the low-bit neural network accelerator to be deployed on a wider range of hardware devices, increasing the scope of use of neural networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A schematic diagram of a low-bit convolution calculation method during the training process of the present invention;

[0046] Figure 2 It is a schematic diagram of the quantization block calculation method of the present invention;

[0047] Figure 3 It is a schematic diagram of the approximate function of the rounding function gradient of the present invention;

[0048] Figure 4 It is a schematic diagram of the accelerator structure of the present invention;

[0049] Figure 5 A schematic diagram of the parallel computing array structure of the present invention;

[0050] Figure 6 It is a schematic diagram of the structure of the two-level addition tree unit of the present invention. DETAILED DESCRIPTION

[0051] The deployment, practical application process and advantages (beneficiary effects) of a low-bit convolutional neural network computing unit of the present invention are described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of protection of the present invention. After reading the present invention, various equivalent modifications made by those skilled in the art fall within the scope defined by the claims attached to this application.

[0052] Example 1

[0053] This embodiment describes the specific implementation steps of the low-bit neural network training method.

[0054] A low-bit neural network training method is proposed based on the existing low-bit neural network training method, and a rounding function gradient approximation function is proposed, including the following steps:

[0055] Step 1: Use the full-precision neural network model parameters to initialize the parameters of the low-bit neural network training model, and at the same time, use random numbers to initialize the trainable parameters of the quantization block in the low-bit neural network training model. The trainable parameters of the quantization block include the quantization bit width and the quantization range.

[0056] Step 2: Perform forward reasoning of the neural network. Figure 1 As shown in Figure 2, in each convolution process, the weights are quantized by the quantization block and then convolved with the input feature map. The result of the convolution is then quantized by the quantization block to obtain the output feature map. Figure 2 As shown, the quantization process includes the following sub-processes:

[0057] Step 21, calculate the quantization step size based on the trainable parameter quantization bit width and quantization range.

[0058] Step 22: Divide the input parameter by the quantization step size, round the result, and then multiply it by the quantization step size to obtain the quantized parameter.

[0059] Step 3: Perform reverse gradient calculation and parameter update of the neural network. The process includes the following sub-processes:

[0060] Step 31. According to the chain rule, the gradient of the loss function is calculated to obtain the gradient of the output feature map.

[0061] Step 32: According to the chain rule, the gradient of the multiplication in the quantization block is calculated to obtain the partial gradient of the quantization step and the gradient of the rounded result.

[0062] Step 33. Use the integer function gradient approximation function to replace the gradient of the integer function in the quantization block, and calculate the gradient of the result after division according to the chain rule.

[0063] Step 34. According to the chain rule, the gradient of the division in the quantization block is obtained to obtain the gradient of the quantization step size and the gradient of the input parameter.

[0064] Step 35. According to the chain rule, calculate the gradient of the quantization bit width and the quantization range.

[0065] Step 36: Update all trainable parameters according to the calculated gradients.

[0066] A fully additive low-bit neural network accelerator proposes a parallel computing array with configurable computing parallelism at runtime and a fully additive convolution. It is connected to a unified memory and a general-purpose processor and includes a hardware data reordering module, a cache scheduling module, an input feature map cache, an output feature map cache, and a parallel computing array.

[0067] The parallel computing array includes a weight cache array, a first-level addition tree array, a second-level addition tree unit, an accumulator array, and a quantization unit array.

[0068] The first-level addition tree is composed of three adders and can realize low-bit multiplication calculation.

[0069] The two-level addition tree unit is composed of several levels of adders, and the number of adder enable levels can be configured during operation to implement grouped convolution operations.

[0070] The low-bit quantization module includes a shift controller and a shifter.

[0071] With respect to beneficial effect 1, the advantages of the low-bit neural network training method are as follows:

[0072] In low-bit neural network inference calculations, weights and convolution results need to be quantized, and one of the steps is rounding. The operation of the rounding function is to round the input data. The problem is that the rounding function is not a continuous function, so its derivative function cannot be calculated, resulting in the inability to calculate the gradient of the trainable parameters during the training process.

[0073] In traditional low-bit neural network inference calculations, the solution to the above problem is to use an approximate direct estimation method, replacing the derivative of the rounding function with a constant function g(x) = 1. In other words, the gradient of the result after rounding is used as the gradient of the value before rounding. In this way, the gradient of each trainable parameter can be obtained.

[0074] However, this method has a disadvantage, that is, there is a large error between the true derivative and the constant function used in the approximate direct estimation method. By introducing the concept of impulse function, the derivative of the integer function is the sum of the impulse functions at all integers plus 0.5, and the constant function g(x)=1 used in the approximate direct estimation method has a large error with the derivative. As the number of layers of the neural network increases, this error will gradually accumulate, causing a large gradient mismatch problem in the parameters during the training process. This leads to unsatisfactory training results of traditional low-bit neural networks.

[0075] In the low-bit convolutional neural network training method proposed in the present invention, the approximate direct estimation method is no longer used to approximate the gradient of the integer function. Figure 3The gradient approximation function of the rounding function replaces the gradient of the rounding function. Compared with the constant function g(x)=1, the gradient approximation function of the rounding function is closer to the derivative of the real rounding function, and the error caused in the gradient calculation process of back propagation is smaller. The calculated gradient of each training parameter is closer to the real gradient, thereby obtaining a better low-bit neural network training effect.

[0076] Example 2

[0077] This embodiment describes the specific implementation steps of the low-bit neural network accelerator.

[0078] The full-additive low-bit neural network accelerator is connected to a unified memory and a general-purpose processor, and includes a hardware data rearrangement module, a cache scheduling module, an input feature map cache, an output feature map cache, and a parallel computing array, as shown in the attached Figure 4 shown.

[0079] The hardware data rearrangement module is connected to the unified memory and the cache scheduling module. The cache scheduling module is connected to the hardware data rearrangement module, the general processor, the input feature map cache and the output feature map cache. The input feature map cache is connected to the unified memory, the cache scheduling module and the parallel computing array. The output feature map cache is connected to the unified memory, the cache scheduling module and the parallel computing array. The parallel computing array is connected to the cache scheduling module, the input feature map cache and the output feature map cache.

[0080] The hardware data rearrangement module receives the size and data address information of each layer of input feature map from the cache scheduling module, reads the data information of the current layer input feature map from the unified memory according to the information, converts the data arrangement into the order of height, width and number of feature channels, and stores it back into the unified memory.

[0081] The cache scheduling module receives information such as the input feature map size, input feature map data address, weight size, weight data address, output feature map size, output feature map data address, current layer calculation type, etc. from the general processor. At the beginning of each layer of calculation, the input feature map size and input feature map data address are sent to the hardware data rearrangement module. The parallelism division information of the parallel computing array is calculated based on the input feature map size and weight size and sent to the parallel computing array. Different input cache read addresses and input cache read lengths are calculated based on the parallelism division information, input feature map size, input feature map data address, weight size and weight data address, and sent to the input feature map cache. Different output cache write addresses and output cache write lengths are calculated based on the parallelism division information, output feature map size, and output feature map data address, and sent to the output feature map cache.

[0082] The input feature map cache receives information such as the input cache read address and input cache read length from the cache scheduling module, reads the block input feature map data and weight data from the unified memory, and sends them to the parallel computing array.

[0083] The output feature map cache receives information such as the output cache write address and the output cache write length from the cache scheduling module, and sends the block output feature map data from the parallel computing array to the unified memory.

[0084] The parallel computing array receives the parallelism division information from the cache scheduling module, and configures the parallelism of the parallel computing of the input channel and the output channel according to the parallelism division information. The block input feature map data and weight data are read from the input feature map cache for convolution calculation, and the output feature map obtained after the calculation is completed is stored in the output feature map cache.

[0085] The parallel computing array includes a weight cache array, a first-level addition tree array, a second-level addition tree unit, an accumulator array, and a quantization unit array, as shown in the attached Figure 5 shown.

[0086] The weight cache array is connected to the input feature map cache and the first-level addition tree, and includes a plurality of weight caches.

[0087] The weight cache array receives weight data transmitted by the input feature map cache, temporarily stores the weight data in the weight cache, and transmits it to the first-level addition tree array when needed.

[0088] The first-level adder tree array is connected to the input feature map cache and the weight cache array, and is composed of a plurality of first-level adder trees; each first-level adder tree includes three adders, which can realize low-bit multiplication calculation;

[0089] The first-level addition tree array reads input feature map data and weight data from the input feature map cache and the weight cache, calculates the multiplication result, and then passes the multiplication result to the second-level addition tree unit.

[0090] The secondary addition tree unit is connected to the primary addition tree array and the accumulator array, and is composed of configuration information and a plurality of levels of adders, as shown in the attached Figure 6 The configuration information includes the number of enabled stages of the adder.

[0091] The secondary addition tree unit receives the multiplication result from the primary addition tree array, performs addition operations of a specified number of stages according to the configuration information, and then transmits the addition result to the accumulator array.

[0092] The accumulator array is connected to the secondary addition tree unit and the quantization unit array, and is composed of a plurality of accumulators.

[0093] The accumulator array receives the addition results from the secondary addition tree unit, and after performing the accumulation for a specified number of cycles, transmits the accumulation results to the quantization unit array.

[0094] The quantization unit array is connected to the accumulator array and the output feature map buffer, and is composed of a plurality of quantization units.

[0095] The quantization unit array receives the accumulation result from the accumulator array, performs a quantization operation, obtains an output feature map result, and sends it to the output feature map cache.

[0096] In the process of calculating the low-bit neural network, the parameter data of the low-bit neural network, such as the network structure, weights, and quantization scale of each layer, are pre-saved in the unified memory.

[0097] At the beginning of each layer of calculation, the general processor passes the input feature map address, input feature map size, weight address, weight size, output feature map address, output feature map size and other data to the cache scheduling module. The cache scheduling module divides the feature map data into blocks according to the input feature map size and the input feature map size, and controls the parallelism of the parallel computing array in the input channel dimension and the output channel dimension.

[0098] The cache scheduling module controls the hardware data reordering module to read the input feature map data from the unified memory according to the input feature map address and the input feature map size, and changes the arrangement of the input feature map data from the organization form of channel number, height and width to the organization form of height, width and channel number.

[0099] At the same time, the cache scheduling module reads the weight data according to the weight address, weight size and output channel dimension parallelism and stores it in the weight cache array in the parallel computing array.

[0100] After the weight data is read, the cache scheduling module controls the input feature map cache to read the input feature map data from the unified memory according to the input feature map address and input feature map size. After reading in enough data, the parallel computing array starts computing. After reading in the input feature map data once, a portion of the output feature map data is calculated.

[0101] After the parallel computing array completes the calculation, the calculation result is stored in the output feature map cache. After a sufficient amount of data is stored, the output feature map cache stores the output feature map data in the unified memory according to the output feature map address and output feature map size.

[0102] With respect to beneficial effect 2, the advantages of the low-bit neural network accelerator are as follows:

[0103] In traditional neural network accelerators, if the overall parallelism of the parallel computing array is 1024, the parallelism of the input channel dimension is set to 16, and the parallelism of the output channel dimension is set to 64. When calculating the first layer of convolution of the Resnet-18 neural network, the result can be obtained after reading in the weight data once and the input feature map data of the neural network once. The output data requires a total of 50,176 computing clock cycles and two data accesses. When calculating the third layer of convolution of the Resnet-18 neural network, the weight data needs to be read in twice and the input feature map data needs to be read in twice to calculate the result. The output data requires a total of 18,432 computing clock cycles and four data accesses. When calculating the fifth layer of convolution of the Resnet-18 neural network, the weight data needs to be read in four times and the input feature map data needs to be read in four times to calculate the result. The output data requires a total of 18,432 computing clock cycles and eight data accesses.

[0104] The low-bit neural network accelerator performs parallel calculations on the input channel and output channel dimensions. The parallelism on each channel dimension can be configured at runtime. If the overall parallelism of the parallel computing array is 1024, when calculating the first layer convolution of the Resnet-18 neural network, the parallelism of the input channel dimension can be set to 16, and the parallelism of the output channel dimension can be set to 64. After reading in the weight data once and the neural network input feature map data once, the result can be obtained. The output data requires a total of 50176 computing clock cycles and two data accesses, which is the same as the traditional neural network accelerator. When calculating the third layer convolution of the Resnet-18 neural network, the parallelism of the input channel dimension can be set to 8, and the parallelism of the output channel dimension can be set to 128. After reading in the weight data once and the neural network input feature map data once, the result can be obtained. The output data requires a total of 18432 computing clock cycles and two data accesses, which is two less than the traditional neural network accelerator. When calculating the fifth convolution of the Resnet-18 neural network, the parallelism of the input channel dimension can be set to 4, and the parallelism of the output channel dimension can be set to 256. The result can be obtained after reading in the weight data once and the neural network input feature map data once. The output data requires a total of 18432 computing clock cycles and two data memory accesses, which is six data memory accesses less than the traditional neural network accelerator.

[0105] If the parallelism of the input channel dimension of the traditional neural network accelerator is set to 4, and the parallelism of the output channel dimension is set to 256, when calculating the first layer convolution of the Resnet-18 neural network, the result can be obtained after reading the weight data once and reading the neural network input feature map data once, but it will require 200704 computing clock cycles, which is 150528 computing clock cycles more than the low-bit neural network accelerator described in the present invention; when calculating the third layer convolution of the Resnet-18 neural network, the result can be obtained after reading the weight data once and reading the neural network input feature map data once, but it will require 36864 computing clock cycles, which is 18432 computing clock cycles more than the low-bit neural network accelerator described in the present invention.

[0106] However, for the group convolution and depth convolution commonly used in mobileNet neural networks, traditional neural network accelerators cannot calculate these two convolutions at the same time. When using traditional neural network accelerator solutions, it is necessary to deploy accelerators for group convolution and depth convolution at the same time.

[0107] When using the low-bit neural network accelerator described in the present invention, a lower input channel parallelism can be configured when calculating group convolution at runtime, and a higher input channel parallelism can be configured when calculating depth convolution. Only one low-bit neural network accelerator described in the present invention needs to be deployed.

[0108] The low-bit neural network accelerator can configure the parallelism of convolution calculations in the input channel dimension and the output channel dimension during runtime, thereby improving the problem of low versatility of traditional neural network accelerators. The low-bit neural network accelerator can simultaneously calculate convolutions of different numbers of feature map channels and different groups, thereby improving the versatility and computing speed of the neural network accelerator.

[0109] With respect to beneficial effect 3, the advantages of the low-bit neural network accelerator are as follows:

[0110] In the past neural network accelerators with parallel computing in the input channel and output channel dimensions, the bit width of the input feature map data and the output feature map data is generally eight bits, and the feature map data and weight data of each convolution layer have a floating point scaling factor and a floating point zero offset respectively. The input data bit width of the two parts of the multiplication calculation unit is eight bits, and the output data bit width is sixteen bits.

[0111] The advantage of this implementation is that the neural network has high precision calculation and good performance. However, this implementation has two disadvantages: First, eight bits are used to represent each feature map data and each weight data, and the overall data volume is relatively large, which requires high storage resources and bandwidth resources of the platform. Second, the two parts of the input data of the multiplication calculation unit are both eight bits wide, and the required amount of calculation is relatively large, which requires high computing resources of the platform.

[0112] After the low-bit neural network accelerator described in the present invention changes the feature map data bit width and weight data bit width of the convolution layer to four bits, the four-bit weight is obtained by relying on the low-bit neural network training method described in the present invention, which can better maintain the accuracy of the neural network. The size of the network model can be reduced to half of the original size, and the feature map data storage space and weight data storage space required for the entire neural network are reduced to half of the original size, which greatly reduces the storage resource requirements and bandwidth resource requirements of the traditional convolutional neural network computing unit for the platform, and can deploy the same neural network on a platform with more limited resources.

[0113] At the same time, after changing the feature map data width and weight data width of the convolution layer to four bits, the two input data widths of the multiplication unit are reduced to four bits. Compared with the multiplication unit with an eight-bit data width, the multiplication unit with a four-bit data width requires only one-fourth of the computing resources. If the same computing resources are used, the degree of parallelism can be quadrupled.

[0114] Therefore, after changing the feature map data bit width and weight data bit width of the convolutional layer to four bits, the consumption of storage resources, bandwidth resources, and computing resources of the deployment platform has been greatly reduced, solving the problem of excessive demand for storage resources, bandwidth resources, and digital signal processors for neural network accelerators that used to parallelize input and output channel dimensions. On a platform with the same amount of resources, a higher degree of parallelism can be achieved compared to the traditional neural network accelerators that used to parallelize input and output channel dimensions.

[0115] With respect to beneficial effect 4, the advantages of the low-bit neural network accelerator are as follows:

[0116] In traditional neural network accelerators, multiplication calculations are all implemented using multipliers. The advantage of this method is that the multiplier has good timing and a large operation bit width. However, the disadvantage is that it consumes computing resources and has high requirements for the computing platform, which needs to be able to carry multipliers.

[0117] The low-bit neural network accelerator described in the present invention designs a full-additive convolution calculation unit. Figure 5As shown, the parallel computing array in the low-bit neural network accelerator includes a weight cache array, a first-level adder tree array, a second-level adder tree unit, an accumulator array, and a quantization unit array. The first-level adder tree array implements a four-bit multiplication function, replacing each multiplier with three adders. Only addition operations are used to implement the convolution calculation function.

[0118] The calculation method eliminates the reliance of traditional neural network accelerators on multipliers, so that the low-bit neural network accelerator only needs to implement addition operations and can be deployed on a hardware platform without multipliers, thereby improving the versatility of the neural network accelerator.

[0119] In summary, the low-bit neural network accelerator described in the present invention reduces the demand for computing power of the neural network accelerator, so that the low-bit neural network accelerator can be deployed on a wider range of hardware devices, increasing the scope of use of neural networks.

Claims

1. A low-bit neural network training method, characterized in that: The steps include: S1. Initializing a full-precision neural network model embedded in a quantization block, including: initializing the full-precision neural network model embedded in a quantization block using parameters of the full-precision neural network model; initializing trainable parameters of the quantization block using random numbers; The quantization blocks include a first quantization block and a second quantization block, and both include division, rounding and multiplication; S2. Forward inference is performed on the full-precision neural network model embedded with the quantization block. The specific operation process for a certain layer is as follows: the weight of each convolutional layer is quantized by the first quantization block and then convolved with the input feature map; the convolution result is then quantized by the second quantization block to obtain the output feature map; S3, back-propagate and calculate the gradients of all layer parameters in the full-precision neural network model embedded in the quantization block; S3, when implemented, the operation process for a certain layer is as follows: S31. Calculate the gradient of the loss function and back-propagate to obtain the gradient of the output feature map of the convolutional layer; S32, calculating the gradient of the multiplication in the second quantization block to obtain a partial gradient of the quantization step length and a gradient of a rounded result; S33, using the rounding function gradient approximation function to replace the gradient of the rounding function in the second quantization block, to obtain the gradient of the result after the division; S34, calculating the gradient of the division in the second quantization block to obtain the complete gradient of the quantization step length and the gradient of the input of the second quantization block; S35, calculating the quantization bit width and the gradient of the quantization range of the second quantization block; S36, replacing the second quantization block in S32 to S35 with the first quantization block, repeating S32 to S35, and obtaining the gradient of the first quantization block input and the gradient of the quantization bit width and quantization range of the first quantization block; S4. Update all trainable parameters according to the calculated gradients.

2. A low-bit neural network training method according to claim 1, characterized in that: S1: The full-precision neural network model of the embedded quantization block includes the network structure and the weights of each network layer; the trainable parameters of the quantization block include the quantization bit width and the quantization range.

3. A low-bit neural network training method according to claim 1, characterized in that: Each convolution layer in the full-precision neural network model embedded with quantization blocks described in S1 is embedded with a first quantization block and a second quantization block.

4. A low-bit neural network training method according to claim 3, characterized in that: The first quantization block quantizes the input weight to obtain a quantized input weight, and the weight is then convolved with the input feature map to obtain an input quantized convolution result; The second quantization block quantizes the input quantization convolution result to obtain an output feature map, which is the output of the convolution layer.

5. A low-bit neural network training method according to claim 1, characterized in that: The quantification described in S2 includes the following sub-processes: S21, calculating the quantization step size based on the trainable parameters; S22. Divide the input of the quantization block by the quantization step size, round the result, and then multiply it by the quantization step size to obtain a quantized parameter.

6. A low-bit neural network training method according to claim 5, characterized in that: The input of the quantization block is the weight of the convolution layer for the first quantization block; and the input of the quantization block is the convolution result of the weight of the convolution layer after quantization and convolution with the input feature map for the second quantization block.

7. A low-bit neural network training method according to claim 1, characterized in that: S36: the input of the first quantization block, i.e., the weight of the convolutional layer; S4: all the trainable parameters, including the parameters of the full-precision neural network model embedded in the quantization block and the training parameters of the quantization block.

8. A full-additive low-bit neural network accelerator connected to a unified memory and a general-purpose processor, characterized in that: It includes a hardware data reordering module, a cache scheduling module, an input feature map cache, a parallel computing array and an output feature map cache; The hardware data rearrangement module is connected to the unified memory and cache scheduling module; the cache scheduling module is connected to the hardware data rearrangement module, the general processor, the input feature map cache and the output feature map cache; the input feature map cache is connected to the unified memory, the cache scheduling module and the parallel computing array; the output feature map cache is connected to the unified memory, the cache scheduling module and the parallel computing array; the parallel computing array is connected to the cache scheduling module, the input feature map cache and the output feature map cache.

9. The full-addition low-bit neural network accelerator according to claim 8, characterized in that: The parallel computing array comprises a weight cache array, a first-level adder tree array, a second-level adder tree unit, an accumulator array and a quantization unit array; the weight cache array is connected to the input feature map cache and the first-level adder tree array, and the first-level adder tree array is connected to the input feature map cache and the weight cache array; The secondary addition tree unit is connected to the primary addition tree array and the accumulator array; the accumulator array is connected to the secondary addition tree unit and the quantization unit array; the quantization unit array is connected to the accumulator array and the output feature map buffer.

10. The full-addition low-bit neural network accelerator according to claim 8, characterized in that: The weight cache array includes a plurality of weight caches; each first-level adder tree array includes a plurality of first-level adder trees; each first-level adder tree includes three adders, which can realize low-bit multiplication calculation; the second-level adder tree unit includes an adder level controller and a plurality of adders; the adder level controller is used to enable the adder level; the accumulator array includes a plurality of accumulators; the quantization unit array includes a plurality of quantization units; the accumulator array receives the addition result from the second-level adder tree unit, and after performing the accumulation of the specified number of cycles, passes the accumulation result to the quantization unit array; The quantization unit array receives the accumulation result from the accumulator array, performs a quantization operation, obtains an output feature map result, and sends it to the output feature map cache.

Images