Neural network calculation method, device, board and computer-readable storage medium

Abstract

The present disclosure relates to a method, apparatus, board, and computer-readable storage medium for neural network computing. The neural network computing apparatus of the present disclosure is included in an integrated circuit device, which includes a universal interconnect interface and other processing devices. The neural network computing apparatus interacts with the other processing devices to jointly complete user-specified computing operations. The integrated circuit device may also include a storage device, which is connected to the neural network computing apparatus and the other processing devices, respectively, for storing data for the neural network computing apparatus and the other processing devices.

Description

Technical Field

[0001] The present disclosure generally relates to the field of neural networks and more specifically to methods, devices, boards, and computer-readable storage media for neural network computing. Background Art

[0002] In order to process massive amounts of data, modern neural networks are constantly increasing their depth, width, and resolution, which has led to an increasing amount of model storage capacity and computational complexity. This causes neural networks to be extremely resource-intensive during training and inference. Therefore, technicians in this field are all trying their best to reduce energy consumption.

[0003] Quantized neural networks can compress models, reduce computing power, and accelerate model inference and training without sacrificing accuracy. Quantization involves replacing floating-point numbers with low-bit precision numbers in the network, reducing data size from 32 bits (single-precision floating-point numbers) to 8 or 4 bits. Quantized networks offer these advantages and have long been a common technique in this field.

[0004] During training quantization, in addition to quantizing weights and activations, the backpropagation gradient must also be quantized. The computational complexity of the backpropagation gradient is approximately twice that of the forward propagation. Numerous experiments have demonstrated that during fixed-point quantization training, the quantization accuracy requirement for the backpropagation gradient must be higher than that for the forward propagation weights and activations. In other words, the quantization bit width of the backpropagation gradient must be higher than that of the forward propagation weights and activations. Therefore, the current quantization bit width of the backpropagation gradient cannot be sufficiently low, typically remaining at 16 bits. This limits the effectiveness of the compression model and has become a major factor limiting the quantization of fixed-point network training at the edge. Without effective training, the neural network cannot be applied.

[0005] Therefore, there is a technical problem at this stage, that is, the existing quantitative methods cannot effectively reduce energy consumption in both inference and training processes. Summary of the Invention

[0006] In order to at least partially solve the technical problems mentioned in the background technology, the solution of the present disclosure provides a method, device, board and computer-readable storage medium for neural network calculation.

[0007] In one aspect, the present disclosure discloses a neural network computing device, comprising a control unit, a quantization unit, and a computing unit. The control unit is configured to provide a segmentation parameter; the quantization unit is configured to quantize floating-point data according to the segmentation parameter to generate fixed-point data; and the computing unit is configured to compute the neural network using the fixed-point data.

[0008] In another aspect, the present disclosure discloses a multiplication unit comprising a multiplier, an addition module, and a fixed-point to floating-point converter. The multiplier is configured to multiply first fixed-point data and second fixed-point data to generate a fixed-point product; the addition module is configured to add a plurality of quantization offset coefficients corresponding to the first fixed-point data and the second fixed-point data to generate a sum of the quantization offset coefficients; and the fixed-point to floating-point converter is configured to convert the fixed-point product into floating-point data based on the sum of the quantization offset coefficients.

[0009] In another aspect, the present disclosure discloses an integrated circuit device including the aforementioned neural network computing device or multiplication unit, and discloses a board including the aforementioned integrated circuit device.

[0010] In another aspect, the present disclosure discloses a method for calculating a neural network, comprising: receiving floating-point data for calculating the neural network, the floating-point data falling within a numerical distribution; segmenting the numerical distribution into a first interval and a second interval based on a segmentation parameter; determining whether the floating-point data falls within the first interval; if so, quantizing the floating-point data according to the segmentation parameter to generate fixed-point data; and calculating the neural network using the fixed-point data.

[0011] In another aspect, the present disclosure discloses a method for calculating a neural network based on floating-point data, including: presetting multiple quantization intervals, each quantization interval corresponds to a quantization formula, and each quantization formula exhibits a different gradient of quantization; determining whether the floating-point data falls within a specific interval of the multiple quantization intervals; and quantizing the floating-point data according to the quantization formula corresponding to the specific interval to generate fixed-point data; and using the fixed-point data to calculate the neural network.

[0012] In another aspect, the present disclosure discloses a method for calculating a neural network based on floating-point data, comprising: presetting multiple quantization intervals; determining whether the floating-point data falls within a specific interval of the multiple quantization intervals; and quantizing the floating-point data according to the specific interval to generate fixed-point data; setting an N-bit flag in a data structure of the fixed-point data, the flag recording the specific interval, where N is a positive integer; and calculating the neural network using the fixed-point data.

[0013] In another aspect, the present disclosure discloses a method for calculating a neural network, comprising: providing a first segmentation parameter and a second segmentation parameter; quantizing first floating-point data according to the first segmentation parameter to generate first fixed-point data; quantizing second floating-point data according to the second segmentation parameter to generate second fixed-point data; performing a multiplication operation on the first fixed-point data and the second fixed-point data to generate intermediate data; and calculating the neural network based on the intermediate data.

[0014] In another aspect, the present disclosure discloses a method for forward propagation in a neural network, comprising: receiving activation values ​​and weights required for calculating a current layer; providing a first segmentation parameter, a first offset value, and a first segmentation value corresponding to the activation value; providing a second segmentation parameter, a second offset value, and a second segmentation value corresponding to the weight; quantizing the activation value according to the first segmentation parameter, the first offset value, and the first segmentation value to generate first fixed-point data; quantizing the weight according to the second segmentation parameter, the second offset value, and the second segmentation value to generate second fixed-point data; performing multiplication operations on the first fixed-point data and the second fixed-point data to generate intermediate data; performing floating-point calculations on the intermediate data to generate activation values ​​for the next layer; and repeating the above steps to perform calculations on each layer to complete the neural network.

[0015] In another aspect, the present disclosure discloses a method for backpropagation in a neural network, wherein the neural network includes weights and fixed-point data of the weights after the weights are fixed-point processed, including: receiving a next-layer error value; providing a segmentation parameter, an offset value, and a segmentation value corresponding to the next-layer error value; quantizing the next-layer error value according to the segmentation parameter, the offset value, and the segmentation value to generate error value fixed-point data; performing multiplication operation on the error value fixed-point data and the weight fixed-point data to generate a gradient of the weight; performing fixed-point calculation on the gradient to generate a current-layer error value; and adjusting the weight according to the current-layer error value.

[0016] In another aspect, the present disclosure discloses a method for training a neural network, comprising: performing forward propagation; calculating a next-layer error value based on the next-layer activation value; performing backpropagation; and adjusting the weight value based on the current-layer error value. The forward propagation step includes: receiving activation values ​​and weight values ​​required for calculating the current layer; quantizing the activation values ​​based on a first segmentation parameter, a first offset value, and a first segmentation value corresponding to the activation values ​​to generate first fixed-point data; quantizing the weight values ​​based on a second segmentation parameter, a second offset value, and a second segmentation value corresponding to the weight values ​​to generate second fixed-point data; multiplying the first fixed-point data and the second fixed-point data to generate intermediate data; and performing floating-point calculations on the intermediate data to generate the next-layer activation value. The backpropagation step includes: quantizing the next-layer error value based on a third segmentation parameter, a third offset value, and a third segmentation value corresponding to the next-layer error value to generate third fixed-point data; multiplying the second fixed-point data and the third fixed-point data to generate a gradient of the weight value; and performing fixed-point calculations on the gradient to generate the current-layer error value.

[0017] In another aspect, the present disclosure discloses an electronic device, comprising one or more processors and a memory, wherein the memory stores computer-executable instructions, and when the one or more processors execute the computer-executable instructions, the electronic device executes any one of the methods described above.

[0018] In another aspect, the present disclosure discloses a computer-readable storage medium comprising computer-executable instructions. When the computer-executable instructions are executed by one or more processors, any one of the methods described above is performed.

[0019] This paper proposes a technical solution for adaptive non-uniform low-bit fixed-point training. By introducing multiple quantization intervals, non-uniform fixed-point data formats, and corresponding hardware, it addresses the technical problem that existing quantization methods cannot effectively reduce energy consumption during both inference and training. This solution achieves the technical effects of reducing network model memory usage, lowering model training resource consumption, and improving training speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts.

[0021] Figure 1 is a graph showing the input and output of the sigmoid function;

[0022] Figure 2 1 is a schematic diagram showing the four-layer structure of a neural network;

[0023] Figure 3 is a schematic diagram illustrating a neural network computing device according to an embodiment of the present disclosure;

[0024] Figure 4 is a schematic diagram illustrating a control unit of an embodiment of the present disclosure;

[0025] Figure 5 is a graph illustrating possible numerical distribution of input data according to an embodiment of the present disclosure;

[0026] Figure 6 is a schematic diagram illustrating an 8-bit non-uniform fixed-point quantization data structure according to an embodiment of the present disclosure;

[0027] Figure 7 is a flowchart illustrating a method for calculating a neural network according to another embodiment of the present disclosure;

[0028] Figure 8 is a schematic diagram illustrating a multiplication unit according to another embodiment of the present disclosure;

[0029] Figure 9 is a flowchart illustrating a method for executing a multiplication calculation neural network according to another embodiment of the present disclosure;

[0030] Figure 10 is a schematic diagram illustrating a non-uniform fixed-point quantization data structure according to another embodiment of the present disclosure;

[0031] Figure 11 is a graph showing possible numerical distribution of input data according to another embodiment of the present disclosure;

[0032] Figure 12 is a schematic diagram illustrating a neural network computing device according to another embodiment of the present disclosure;

[0033] Figure 13 is a flow chart illustrating a forward propagation method according to another embodiment of the present disclosure;

[0034] Figure 14 is a flowchart illustrating a back propagation method according to another embodiment of the present disclosure;

[0035] Figure 15 is a structural diagram illustrating an integrated circuit device according to another embodiment of the present disclosure; and

[0036] Figure 16 FIG. 4 is a structural diagram of a board according to another embodiment of the present disclosure. DETAILED DESCRIPTION

[0037] The following will be combined with the accompanying drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the embodiments described are part of the embodiments of the present disclosure, not all of them. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0038] It should be understood that the terms "first," "second," "third," and "fourth," etc. in the claims, specification, and drawings of the present disclosure are used to distinguish different objects rather than to describe a specific order. The terms "include" and "comprising" used in the specification and claims of the present disclosure indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0039] It should also be understood that the terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the present disclosure. As used in this disclosure and the claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should further be understood that the term "and / or" as used in this disclosure and the claims refers to any and all possible combinations of one or more of the associated listed items, including and including these combinations.

[0040] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context.

[0041] The specific embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0042] Neural networks are built based on the concept of neurons. Neurons are like perceptrons. The activation function of a perceptron is generally a step function, while the activation function of a neuron is commonly a sigmoid function, which is defined as follows:

[0043]

[0044] The input and output diagram of the sigmoid function is as follows Figure 1 As shown, it can map a real number to the interval between 0 and 1, which is suitable for binary classification.

[0045] A neural network is a system of multiple neurons connected according to certain rules. Taking a convolutional neural network as an example, it is generally composed of the following four layers: input layer, convolution layer, pooling layer, and fully connected layer. Figure 2 2 is a schematic diagram showing a four-layer structure of the neural network 200.

[0046] The input layer 201 extracts part of the information from the input data and converts it into a feature matrix, which contains the features corresponding to the part of the information. The input data here can include but is not limited to image data, voice data or text data.

[0047] The convolution layer 202 is configured to receive the feature matrix from the input layer 201 and extract features from the input data through convolution operations. In actual use, the convolution layer 202 can be constructed into multiple convolution layers. Taking image data as an example, the convolution layer in the first half is used to capture local and detailed information of the image. For example, each pixel of the output image only senses the result of calculation based on a very small range of values ​​in the input image. The range of perception of the subsequent convolution layers increases layer by layer to capture more complex and abstract information of the image. After the operation of multiple convolution layers, an abstract representation of the image at various scales is finally obtained. Although the feature extraction of the input image is completed through the convolution operation, the amount of information in the feature image is too large and the dimensionality is too high. Not only is the calculation time-consuming, but it is also easy to cause overfitting and requires further dimensionality reduction.

[0048] Pooling layer 203 is configured to replace a specific region of the data with a single value, typically the maximum or average value of all values ​​in that region. Using the maximum value is called max pooling, while using the average value is called mean pooling. Pooling can reduce model size and increase computational speed without losing too much information.

[0049] The fully connected layer 204 acts as a classifier in the entire convolutional neural network 200, which is equivalent to feature space transformation, extracting and integrating all the useful information mentioned above, and adding the nonlinear mapping of the aforementioned activation function. The multi-layer fully connected layer can theoretically simulate any nonlinear transformation to perform information comparison based on different classifications, thereby determining whether the input data is similar to the comparison target.

[0050] Neural networks require weights as model parameters, and optimal solutions are determined during the training process. Furthermore, some parameters, such as the neural network's connectivity, the number of layers, and the number of nodes per layer, are not learned but rather set in advance. These artificially set parameters are called hyperparameters.

[0051] Neural networks are divided into forward propagation and back propagation. The so-called forward propagation is like Figure 2 The forward calculation direction is shown, from the input layer to the output layer, and the state value and activation value of each neuron are calculated in turn. Backward propagation is the reverse calculation from the output end to the input end, in order to obtain the gradient value.

[0052] When computing input data within a neural network, the goal is generally to find a solution that minimizes the loss function, representing the result closest to the real world. However, neural network loss functions are complex, making it difficult to find an optimal analytical expression. A common approach is to calculate the negative gradient, as the maximum negative gradient indicates the direction in which the loss function decreases the most. The backpropagation algorithm seeks to determine this gradient, subsequently updating the model parameters based on gradient descent. Therefore, starting from the output layer of the neural network model, the backpropagation algorithm uses the chain rule for function differentiation, working backwards layer by layer to determine the model gradient, hoping to find a solution that minimizes the loss function.

[0053] By calculating the gradient in this way, each neural unit is only calculated once, without repeated calculations. The fundamental reason for this calculation direction's efficiency is that when calculating the gradient, the previous level unit depends on the calculation of the next level unit. By first calculating the gradient value of the next level unit and then calculating the gradient value of the previous level unit, we can fully utilize the already calculated results and avoid repeated calculations.

[0054] Regularization penalties and ReLU activation functions are often used in neural network training. Regularization penalties are used because, when overfitting occurs, poorly generalized higher-order functions will generate significant jitter in order to fit all the data in the test set. This jitter causes the derivatives to become very large, requiring large parameters to fit all the data. Therefore, adding a penalty term can penalize very large parameters and avoid these parameters with excessive jitter.

[0055] The ReLU activation function is used when the model has N layers. In theory, the activation rate of neurons will be reduced by 2 N times, ReLU can better realize the sparse model to mine relevant features and fit the training data. In addition, compared with other activation functions, ReLU has the following advantages: for linear functions, ReLU has stronger expressive power, especially in deep networks; and for nonlinear functions, the gradient of ReLU in the non-negative interval is constant, so there is no gradient vanishing problem, which keeps the convergence speed of the model at a stable state. The gradient vanishing problem means that when the gradient is less than 1, the error between the predicted value and the true value decays once per layer. If the aforementioned sigmoid function is used as the activation function in a deep model, this phenomenon is particularly obvious, which will cause the model convergence to stagnate.

[0056] Due to the effects of regularization penalty and ReLU activation function, the weights, activations and gradients of each layer are not uniform in numerical distribution, and the numerical distribution of weights, activations and gradients shows a characteristic of moving closer to 0 during the entire training process, like a Gaussian distribution. Considering that the values ​​will be concentrated near 0, uniform quantization will quantize all these values ​​close to 0 to 0, which seriously affects the direction of network training. For example, if 7-bit uniform quantization is used, the absolute value of the value less than 0 is 0. The value of will be quantized to 0. This quantization is unacceptable for training. Under the existing uniform quantization scheme, the factor limiting the accuracy of network training mainly comes from the gradient quantization error, especially the quantization error of the gradient near 0 in the value distribution.

[0057] The present disclosure provides an adaptive non-uniform low-bit quantization scheme that is applicable to training and reasoning tasks of various neural networks (such as convolutional neural networks, recurrent neural networks, graph neural networks, etc.). As mentioned above, in the process of improving neural networks, two types of parameters are involved. One is general parameters, that is, parameter data obtained through training, such as the weight w and offset b in y = wx + b; the other is called hyperparameters, which are parameters set before learning begins and parameter data that cannot be obtained through training.

[0058] The devices described in the embodiments of the present disclosure and the various devices, units, modules, etc. described below can be implemented in the form of hardware circuits, such as digital circuits or analog circuits. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. Unless otherwise specified, the artificial intelligence processor mentioned in the embodiments can be any appropriate hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the memory, storage device, and storage unit can be any appropriate magnetic storage medium or magneto-optical storage medium, such as resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high-bandwidth memory (HBM), hybrid memory cube (HMC), etc.

[0059] Optionally, when the following devices, various devices, units, and modules are implemented as ASIC, their advantages over other hardware implementations are in terms of power consumption, reliability, area, etc., especially when used in high-performance, low-power mobile terminals.

[0060] One embodiment of the present disclosure is a neural network computing device, which divides the numerical distribution of floating-point input data into a first interval and a second interval, and uses different quantization methods for floating-point data falling in different intervals. Figure 3 shown.

[0061] The neural network computing device of this embodiment includes a control unit 31, a quantization unit 32, and a computing unit 33. The control unit 31 is used to provide various parameters and hyperparameters required for the quantization process; the quantization unit 32 is used to quantize floating-point data based on the parameters and hyperparameters to generate fixed-point data; and the computing unit 33 is used to calculate the neural network using the fixed-point data. Figure 3 Each unit can be implemented in the form of hardware circuit.

[0062] The control unit 31 is used to provide the segmentation parameter α, the offset value shift and the segmentation value x required in the quantization process. p Etc. Figure 4 As shown, the control unit 31 includes an input / output module 311 , a segmentation parameter generator 312 , an offset value generator 313 and a segmentation value generator 314 .

[0063] The input / output module 311 serves as a channel for signal transmission between the control unit 31 and the external unit. It can output signals when a control request occurs, and can also receive control signals from the external unit and send the control signals to the segmentation parameter generator 312, the offset value generator 313 or the segmentation value generator 314.

[0064] The segmentation parameter generator 312 is used to generate a hyperparameter: the segmentation parameter α. The segmentation parameter α is used to define the first interval and the second interval. If the input data x (floating point number) is distributed in the numerical distribution D, the relationship between the segmentation parameter α and the first interval A and the second interval B is:

[0065] A={x|x<2 α max{abs(D)},x∈D} (1)

[0066] B={x|x≥2 α max{abs(D)},x∈D} (2)

[0067] See Figure 5The Gaussian distribution curve in the figure is the possible value distribution of the input data x. The input data x will basically fall completely within the range of the value distribution D. The first interval A is a positive and negative symmetrical interval including 0, that is, the range between the two dotted lines in the figure, and the range outside the dotted lines is the second interval B. When the input data x is less than 2 α max{abs(D)} indicates that the input data x falls within the first interval A; when the input data x is greater than or equal to 2 α When max{abs(D)}, it means that the input data x falls within the second interval B.

[0068] The segmentation parameter α determines Figure 5 The position of the dashed line determines the range of the first interval A. The smaller the absolute value of the segmentation parameter α, the larger the range of the first interval A. The segmentation parameter α is related to the quantization bit width b, which represents the number of bits of the fixed-point number. The quantization bit width b is a positive integer, and the segmentation parameter α is a negative integer. In this embodiment, the relationship between the two is as follows:

[0069] 50%×b≤abs(α)≤90%×b

[0070] Taking the quantization bit width b as 8 (ie, an 8-bit fixed-point number) as an example, the segmentation parameter α can be optionally one of -4, -5, -6, and -7.

[0071] The offset value generator 313 is used to generate an offset value shift, where the offset value shift represents a quantization offset. The value generator 313 generates the offset value shift according to the following expression:

[0072]

[0073] The ceil function returns the smallest integer greater than or equal to the expression.

[0074] The segmentation value generator 314 is used to generate the segmentation value x p , split value x p Indicates that the first interval A and the second interval B are Figure 5 The dividing line on the horizontal axis, that is, Figure 4 The value generator 314 generates the segmentation value x according to the following expression: p :

[0075] x p =2 α max{abs(D)} (4)

[0076] The aforementioned segmentation parameter α, offset value shift and segmentation value x p These are parameters required in the quantization process, which are generated by the control unit 31 and sent to the quantization unit 32 via the input / output module 311 .

[0077] Optionally, in one embodiment, the control unit 31 may also have other hardware circuit implementations, which will not be described in detail here.

[0078] Back to Figure 3 The quantization unit 32 includes an absolute value operator 321 , a comparator 322 , a two-way selector 323 , an adder 324 , a quantizer 325 and a bus converter 326 .

[0079] The absolute value operator 321 receives the input data x, takes the absolute value of the floating point format input data x, and outputs the absolute value abs(x). The comparator 322 receives the absolute value abs(x) and the division value x. p , the absolute value abs(x) and the segmentation value x p If the split value x p is greater than the absolute value abs(x), referring to formula (1), indicating that the input data x falls within the first interval A, and the comparator 322 sets the flag value to 1. If the segmentation value x p If the absolute value abs(x) is less than or equal to abs(x), referring to formula (2), it means that the input data x falls within the second interval B, and the value of the flag is set to 0. The value of the flag reflects whether the input data x falls within the first interval A or the second interval B.

[0080] The binary selector 323 receives the segmentation parameter α from the control unit 31 and determines whether to output the segmentation parameter α or 0 based on the value of the flag. When the flag is 1, it indicates that the input data x falls within the first interval A. Values ​​within the first interval A, including 0, are easily quantized to 0. This embodiment adjusts the quantization precision for values ​​within the first interval A. Therefore, the binary selector 323 sets the output value to the segmentation parameter α for subsequent precision adjustment. When the flag is 0, it indicates that the input data x falls within the second interval B. Values ​​within this interval do not require precision adjustment, so the binary selector 323 sets the output value to 0.

[0081] Adder 324 receives the output of binary selector 323 and the offset value shift from control unit 31 and adds the offset value shift to the output of binary selector 323 to generate a quantized offset value s. Specifically, when the value of flag is 1, the quantized offset value s is the segmentation parameter α plus the offset value shift; when the value of flag is 0, the quantized offset value s is the offset value shift.

[0082] The quantizer 325 quantizes the input data x into n-bit fixed-point data. The quantization formula is:

[0083]

[0084] Among them, 2 s The quantization interval is called the round function. The round function is the result of the rounding operation. x[n-1:0] is the fixed-point data after the input data x is quantized. However, the output x[n-1:0] of the quantizer 325 is only intermediate data, not the final quantization result.

[0085] The bus converter 326 is used to combine the value of the flag bit flag with the intermediate data x[n-1:0] to generate the fixed-point data x[n:0]. More specifically, the value of the flag bit flag is added to the intermediate data x[n-1:0] so that the final fixed-point data x[n:0] is n+1 bits.

[0086] In another scenario, if the value of the flag is not important to the computing unit 33 , this embodiment may not include the bus converter 326 , and the output x[n−1:0] of the quantizer 325 is the final quantization result, which is directly transmitted to the computing unit 33 .

[0087] Optionally, in one embodiment, the quantization unit 32 may also have other hardware circuit implementations, which will not be described in detail here.

[0088] This embodiment is coupled with a new non-uniform fixed-point quantization data structure. The data format of the non-uniform fixed-point data x[n:0] will be described below. Figure 6 An 8-bit non-uniform fixed-point quantization data structure is shown, which includes a sign bit 61, a value bit 62, and a flag bit 63. The sign bit 61 is 1 bit, which is the most significant bit (MSB), and is used to record the positive and negative signs of the fixed-point data. The flag bit 63 is 1 bit, which is the least significant bit (LSB), and is used to record the value of the flag bit flag. The middle 6 bits are the value bit 62, which are used to record the value of the fixed-point data. The output x[n-1:0] of the quantizer 325 corresponds to the values ​​of the sign bit 61 and the value bit 62 of the non-uniform fixed-point quantization data structure. The bus converter 326 then records the value of the flag bit flag in the flag bit 63 to generate the complete fixed-point data x[n:0]. Based on Figure 6 The relationship between the corresponding floating-point input data x and the sign bit 61, the value bit 62 and the flag bit 63 is as follows:

[0089] x=(-1) sign ×value×2 α·flag ×interval (6)

[0090] Among them, sign is the value of the sign bit 61, and value is the value of the value bit 62.

[0091] If the number of bits of the fixed-point number is not 8 but n, then under this fixed-point data structure, its most significant bit and least significant bit are also the sign bit 61 and the flag bit 63, and the middle n-2 bits are the value bits 62.

[0092] Back to Figure 3 In the quantization unit 32, the floating point input data x is converted into fixed point data x[n:0], and the fixed point data x[n:0] is transmitted to the calculation unit 33 for calculation. The calculation unit 33 can perform specific calculations according to actual needs, such as Figure 2 The convolution calculation of the fully connected layer 204 generates an intermediate result y[n:0] after the calculation is completed. The intermediate result y[n:0] is also fixed-point data. The calculation unit 33 then restores the intermediate result y[n:0] to floating-point data y to complete the entire calculation process.

[0093] The above description takes the input data x as an activation value as an example, but the present disclosure is not limited to this. That is, in a neural network, any data that needs to be quantized (such as weights and gradients, etc.) can be converted into fixed-point data using the quantization unit 32.

[0094] In summary, this embodiment implements a method for calculating a neural network. Figure 7 A flow chart of this method is shown. The process of this embodiment is described based on the aforementioned hardware design. It is understandable that it is not limited to the hardware design of the present disclosure, such as digital circuits or analog circuits. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. The artificial intelligence processor mentioned in the embodiment can be any appropriate hardware processor, such as CPU, GPU, FPGA, DSP and ASIC, etc. Unless otherwise specified, the memory, storage device, storage unit can be any appropriate magnetic storage medium or magneto-optical storage medium, such as resistive random access memory RRAM, dynamic random access memory DRAM, static random access memory SRAM, enhanced dynamic random access memory EDRAM, high bandwidth memory HBM, hybrid memory cube HMC, etc.

[0095] In step 71, the absolute value operator 321 receives floating point data for computing a neural network, wherein the floating point data falls within a numerical distribution. Figure 3 , the absolute value operator 321 receives the input data x, takes the absolute value of the input data x in floating point format, and outputs the absolute value abs(x). It should be emphasized that in this embodiment, a series of floating point data will be received, and each floating point data is calculated based on the absolute value of the floating point data. Figure 7 process to process.

[0096] In step 72, the control unit 31 divides the numerical distribution into a first interval and a second interval based on the segmentation parameter. Figure 4, the segmentation parameter generator 312 of the control unit 31 generates a segmentation parameter α. The segmentation parameter α is used to define the first interval A and the second interval B according to equations (1) and (2).

[0097] In step 73, the comparator 322 determines whether the floating point data falls within the first interval. Figure 3 The comparator 322 receives the absolute value abs(x) and the division value x p , the absolute value abs(x) and the split value x p If the split value x p is greater than the absolute value abs(x), indicating that the input data x falls within the first interval A. The comparator 322 sets the flag value to 1. If the segmentation value x p If it is less than or equal to the absolute value abs(x), it means that the input data x falls within the second interval B, and the value of the flag flag is set to 0.

[0098] If the floating point data falls within the first interval, step 74 is executed, and the two-way selector 323, the adder 324, the quantizer 325 and the bus converter 326 quantize the floating point data according to the segmentation parameter to generate fixed point data. Figure 3 , the two-way selector 323 receives the segmentation parameter α from the control unit 31 and decides to output the segmentation parameter α or 0 based on the value of the flag. Since the floating-point data falls within the first interval, the value of the flag is 1, and the two-way selector 323 sets the output value to the segmentation parameter α. The quantization offset value s output by the adder 324 is the segmentation parameter α plus the offset value shift. The quantizer 325 quantizes the input data x into the intermediate data x based on equation (5) q The bus converter 326 then adds the value of the flag bit flag to the intermediate data x q To generate fixed-point data x[n:0].

[0099] If the floating-point data does not fall within the first interval, step 75 is executed, and the two-way selector 323, the adder 324, the quantizer 325 and the bus converter 326 do not quantize the floating-point data according to the segmentation parameter, but quantize the floating-point data according to the following expression to generate the fixed-point data x q :

[0100]

[0101] interval=2 shift (8)

[0102]

[0103] These expressions are essentially the same as equations (3) and (5). The difference is that since the floating-point data falls within the second interval B, the value of the flag is 0, and the two-way selector 323 sets the output value to 0. The quantization offset value s output by the adder 324 is only the offset value shift. The quantizer 325 quantizes the input data x into the intermediate data x q , and the bus converter 326 then adds the value of the flag bit flag to the intermediate data x q To generate fixed-point data x[n:0].

[0104] In step 76, the calculation unit 33 calculates the neural network using the fixed-point data x[n:0]. Figure 3 The calculation unit 33 can perform specific calculations according to actual needs and generate an intermediate result y[n:0] after the calculation is completed. The intermediate result y[n:0] is also fixed-point data. The calculation unit 33 then restores the intermediate result y[n:0] to floating-point data y to complete the entire calculation process.

[0105] Since this embodiment can generate a flag by the comparator 322 to record whether the input data x falls within the first interval A or the second interval B, the quantization unit 32 can select the appropriate precision for quantization. Therefore, this embodiment is an "adaptive" system. Furthermore, the precision of the first interval A and the second interval B are different. The first interval A including 0 uses 2 α The precision is divided into finer parts to avoid a large amount of data close to 0 from being quantized to 0, so this embodiment is still a "non-uniform" system.

[0106] For a neural network, the computing unit 33 of the aforementioned embodiment can perform an important calculation, namely matrix multiplication. Matrix multiplication involves a large number of fixed-point multiplications. Figure 6 A fixed-point data structure with a special structure of multiplication unit is proposed.

[0107] To have Figure 6 When performing multiplication calculation on two fixed-point data of a fixed-point data structure, assuming that the first fixed-point data x1 falls within the first numerical distribution and the second fixed-point data x2 falls within the second numerical distribution, the first segmentation parameter α1 is used to segment the first numerical distribution into the first interval and the second interval, the second segmentation parameter α2 is used to segment the second numerical distribution into the third interval and the fourth interval, the first flag value flag1 is used to reflect that the first fixed-point data x1 falls within the first interval or the second interval, and the second flag value flag2 is used to reflect that the second fixed-point data x2 falls within the third interval or the fourth interval. Based on formula (6), its floating-point product x1×x2 is:

[0108]

[0109] Another embodiment of the present disclosure is a multiplication unit, the schematic diagram of which is shown in FIG. Figure 8 The multiplication unit 80 of this embodiment includes a multiplier 81 , an addition module 82 and a fixed-point to floating-point converter 83 .

[0110] The multiplier 81 is used to multiply the first fixed-point data x1[n-1:0] and the second fixed-point data x2[n-1:0] to generate a fixed-point product y[2n-1:0], that is, to realize the formula (10) The first fixed-point data x1[n-1:0] and the second fixed-point data x2[n-1:0] can come from Figure 3 The output of quantizer 325 or the output from bus converter 326 is then removed from the least significant bit (flag) in the data structure, leaving only the sign and magnitude bits. Since both the first fixed-point data x1[n-1:0] and the second fixed-point data x2[n-1:0] are n bits, the fixed-point product y[2n-1:0] is 2n bits.

[0111] The adding module 82 is used to add a plurality of quantization offset coefficients corresponding to the first fixed-point data x1[n-1:0] and the second fixed-point data x2[n-1:0] to generate a sum of quantization offset coefficients, that is, to realize the formula (10) As shown in formula (8), formula (10) It is equivalent to Therefore, the quantization offset coefficient involves the segmentation parameter α, the flag value flag and the offset value shift.

[0112] The addition module 82 includes a first selector 821, a second selector 822, and an adder 823. The first selector 821 sets the first output value to the first segmentation parameter α1 or 0 according to the first flag value flag1. When the first flag value flag1 is 1, that is, when the value of the flag bit x1[n] is 1, it indicates that the first fixed-point data x1[n-1:0] falls within the first interval, and the first segmentation parameter α1 needs to be involved in the calculation, so the first segmentation parameter α1 is output; when the first flag value flag1 is 0, that is, when the value of the flag bit x1[n] is 0, it indicates that the first fixed-point data x1[n-1:0] falls within the second interval, and the first segmentation parameter α1 does not participate in the calculation, so 0 is output. Similarly, the second selector 822 sets the second output value to the second segmentation parameter α2 or 0 according to the second flag value flag2. Its specific operation is the same as that of the first selector 821 and will not be repeated here.

[0113] The adder 823 adds the first offset value shift1, the second offset value shift2, the first output value, and the second output value to generate the quantized offset coefficient sum s, i.e., α1·flag1+α2·flag2+shift1+shift2. The first offset value shift1 and the second offset value shift2 are calculated according to equation (3).

[0114] The fixed-point to floating-point converter 83 converts the fixed-point product y[2n-1:0] into floating-point data y based on the quantization offset coefficient and s. The fixed-point to floating-point converter 83 includes a power of 2 calculator 831 and a multiplier 832. The power of 2 calculator 831 generates a quantization offset value 2 based on the quantization offset coefficient and s. s , to achieve Multiplier 832 multiplies the fixed-point product y[2n-1:0] by the quantization offset value 2 s Multiplying them, we get the floating-point product x1×x2.

[0115] Optionally, in one embodiment, the multiplication unit 80 may also have other hardware circuit implementations, which will not be described in detail here.

[0116] In summary, this embodiment implements a method for executing a multiplication calculation neural network. The flowchart of this method is as follows: Figure 9 As shown. The process of this embodiment is described based on the aforementioned hardware design. It is understandable that it is not limited to the hardware design of the present disclosure, such as digital circuits or analog circuits. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. The artificial intelligence processor mentioned in the embodiment can be any appropriate hardware processor, such as CPU, GPU, FPGA, DSP and ASIC, etc. Unless otherwise specified, the memory, storage device, storage unit can be any appropriate magnetic storage medium or magneto-optical storage medium, such as resistive random access memory RRAM, dynamic random access memory DRAM, static random access memory SRAM, enhanced dynamic random access memory EDRAM, high bandwidth memory HBM, hybrid memory cube HMC, etc.

[0117] In step 91 , the control unit 31 provides a first segmentation parameter α1 and a second segmentation parameter α2 . Specifically, the segmentation parameter generator 312 of the control unit 31 generates the first segmentation parameter α1 and the second segmentation parameter α2 .

[0118] In step 92, the quantization unit 32 quantizes the first floating-point data x1 according to the first segmentation parameter α1 to generate the first fixed-point data x1[n:0]. The comparator 322 generates the segmentation value x1[n:0] based on the first segmentation parameter α1 and according to formula (4). p, dividing the first numerical distribution into a first interval A and a second interval B; then determining whether the first floating-point data x1 falls within the first interval A, where the first interval A is defined by equation (1). If the first floating-point data x1 falls within the first interval A, the two-way selector 323, the adder 324, and the quantizer 325 generate the intermediate fixed-point data x1[n-1:0] according to the following expression:

[0119]

[0120]

[0121]

[0122] If the first floating-point data x1 does not fall within the first interval A, the binary selector 323, the adder 324, and the quantizer 325 generate the intermediate fixed-point data x1[n-1:0] according to the following expression:

[0123]

[0124]

[0125]

[0126] Finally, the bus converter 326 fills in the flag bit flag to generate the complete first fixed-point data x1[n:0].

[0127] In step 93, the quantization unit 32 quantizes the second floating-point data x2 according to the second segmentation parameter α2 to generate the second fixed-point data x2[n:0]. Similarly, the comparator 322 generates the segmentation value x2 according to the second segmentation parameter α2 and equation (4). p , dividing the second numerical distribution into a third interval and a fourth interval; then determining whether the second floating-point data x2 falls within the third interval, where the third interval is defined by equation (1). If the second floating-point data x2 falls within the third interval, the two-way selector 323, the adder 324, and the quantizer 325 generate the second fixed-point data x2[n-1:0] according to the following expression:

[0128]

[0129]

[0130]

[0131] If the second floating-point data x2 does not fall within the third interval, the two-way selector 323, the adder 324, and the quantizer 325 generate the intermediate fixed-point data x2[n-1:0] according to the following expression:

[0132]

[0133]

[0134]

[0135] Finally, the bus converter 326 fills in the flag bit flag to generate the complete second fixed-point data x2[n:0].

[0136] In step 94, the multiplication unit 80 performs a multiplication operation on the first fixed-point data x1[n:0] and the second fixed-point data x2[n:0] to generate the intermediate data y. As previously mentioned, the flag bit of the fixed-point data data structure of the present disclosure is used to record the interval of the first fixed-point data x1[n:0] and the second fixed-point data x2[n:0], the sign bit is used to record the positive and negative signs of the first fixed-point data x1[n:0] and the second fixed-point data x2[n:0], and the value bit is used to record the values ​​V1 and V2 of the first fixed-point data x1[n:0] and the second fixed-point data x2[n:0]. The multiplier 81 adds the values ​​of the sign bits of the first fixed-point data x1[n:0] and the second fixed-point data x2[n:0] to generate the signed sum value sign t The task of the first selector 821 is equivalent to multiplying the first segmentation parameter α1 and the value of the flag bit x1[n] of the first fixed-point data to generate a first parameter product value pm1; the task of the second selector 822 is equivalent to multiplying the second segmentation parameter α2 and the value of the flag bit x2[n] of the second fixed-point data to generate a second parameter product value pm2; finally, the fixed-point to floating-point converter 83 executes the following expression to generate the intermediate data:

[0137]

[0138] In step 95, a neural network is calculated based on the intermediate data. Figure 2 In the neural network architecture shown, the input data can be diverse, such as images, speech, and text data. This data undergoes a large number of multiplication operations in the input layer 201, convolutional layer 202, pooling layer 203, and fully connected layer 204. These multiplication operations can all be implemented using the aforementioned steps until the inference process is completed and the image, speech, and text data are ultimately recognized. In addition to the input data, the parameters of the neural network can also be converted into fixed-point data using the aforementioned steps and multiplied with the input data.

[0139] Because data close to zero is quantized to zero during the quantization process, affecting calculations, the aforementioned embodiments divide the numerical distribution of the input data into two intervals: one symmetrical interval containing 0, and the other interval outside of the aforementioned intervals. However, the present disclosure does not limit the number of intervals; as long as the numerical distribution within different ranges requires quantization with different precisions, it can be appropriately divided into multiple intervals.

[0140] When the interval exceeds two, Figure 6 The data structure in the example only needs to adjust the size of the flag bit 63. Taking 3 or 4 intervals as an example, in order to fully record the data falling into one of the 3 or 4 intervals, the flag bit 63 requires 2 bits, such as Figure 10 In other words, if the value distribution is divided into N intervals, the flag bit requires ceil[log2N] bits.

[0141] Another embodiment of the present disclosure is a neural network computing device. The difference from the above embodiment is that the neural network computing device of this embodiment divides the numerical distribution of floating-point data into a first interval, a second interval, and a third interval. Different quantization methods are used for floating-point data falling in different intervals. Figure 11 As shown, the numerical distribution D is divided into a first interval A, a second interval B and a third interval C according to different quantization accuracies.

[0142] This embodiment uses two segmentation parameters to define the three intervals. The segmentation parameter α1 is used to define the first interval A and the second interval B, and the segmentation parameter α2 is used to define the second interval B and the third interval C. The relationship between the segmentation parameters α1, α2 and the first interval A, the second interval B, and the third interval C is:

[0143]

[0144]

[0145]

[0146] When calculating the fixed-point data x that falls in the first interval A q , the following expressions can be used:

[0147]

[0148]

[0149]

[0150] When calculating the fixed-point data x that falls in the second interval B q , the following expressions can be used:

[0151]

[0152]

[0153]

[0154] When calculating the fixed-point data x that falls in the third interval C q , the following expressions can be used:

[0155]

[0156] interval3=2 shift

[0157]

[0158] The schematic diagram of this embodiment is as follows Figure 12 As shown, it is Figure 3 There is no significant difference in the framework of FIG. 1 , and the only difference lies in the control unit 121 , the comparator 122 and the three-way selector 123 .

[0159] Compared to Figure 3 The control unit 31 of the control unit 121 outputs the segmentation parameters α1 and α2 to the three-way selector 123, and outputs the first segmentation value x corresponding to the segmentation parameter α1. p1 and the second segmentation value x corresponding to the segmentation parameter α2 p2 To the comparator 122, its expression is as follows:

[0160]

[0161]

[0162] Compared to Figure 3 The comparator 322 is implemented as a two-stage comparison circuit. The first stage compares the absolute value abs(x) of the input data x with the first segmentation value x. p1 , if the absolute value abs(x) is less than the first segmentation value x p1 , indicating that the input data x falls within the first interval A, so it does not need to be compared with the second segmentation value x p2 Compare and output flag value flag is 00, if the absolute value abs(x) is not less than the first segmentation value x p1 , indicating that the input data x falls in the second interval B or the third interval C, then enters the second stage circuit, compares the absolute value abs(x) of the input data x with the second segmentation value x p2 If the absolute value abs(x) is less than the second division value x p2, indicating that the input data x falls in the second interval B, so the output flag value flag is 01. If the absolute value abs(x) is not less than the second segmentation value x p2 , indicating that the input data x falls within the third interval C, so the output flag value flag is 10.

[0163] Compared to Figure 3 The two-way selector 323 receives the segmentation parameters α1 and α2, and determines whether to output the segmentation parameters α1, α2, or 0 based on the value of the flag. When the value of the flag is 00, it indicates that the input data x falls within the first interval A, so the three-way selector 123 sets the output value to the segmentation parameter α1. When the value of the flag is 01, it indicates that the input data x falls within the second interval B, so the three-way selector 123 sets the output value to the segmentation parameter α2. When the value of the flag is 10, it indicates that the input data x falls within the third interval C, so the three-way selector 123 sets the output value to 0.

[0164] The remaining components operate similar to Figure 3 The corresponding components are the same, so they are not described in detail. Optionally, in one embodiment, Figure 12 The embodiment may also have other hardware circuit implementations, which will not be described here in detail.

[0165] Figure 12 The embodiment is described with three intervals as an example. The present disclosure is not limited to the number of intervals, and those skilled in the art can easily generalize to multiple intervals without creative input.

[0166] Another embodiment of the present disclosure is a method for forward propagation in a neural network, that is, the reasoning process of the neural network, which can be used to Figure 3 or Figure 12 For the convenience of explanation, the following will be used with Figure 3 The embodiment of the invention is described. Figure 13 A flow chart of the method of this embodiment is shown. The process of this embodiment is described based on the aforementioned hardware design. It is understandable that it is not limited to the hardware design of the present disclosure, such as digital circuits or analog circuits. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. The artificial intelligence processor mentioned in the embodiment can be any appropriate hardware processor, such as CPU, GPU, FPGA, DSP and ASIC, etc. Unless otherwise specified, the memory, storage device, storage unit can be any appropriate magnetic storage medium or magneto-optical storage medium, such as resistive random access memory RRAM, dynamic random access memory DRAM, static random access memory SRAM, enhanced dynamic random access memory EDRAM, high bandwidth memory HBM, hybrid memory cube HMC, etc.

[0167] In step 1301, the absolute value operator 321 receives the activation value x1 and weight x2 required for calculating the layer. The activation value x1 and weight x2 in floating point format are input to the absolute value operator 321 as input data, and the absolute value abs(x) is output.

[0168] In step 1302, the control unit 31 provides a first segmentation parameter α1, a first offset value shift1 and a first segmentation value x1 corresponding to the activation value x1. p1 The first segmentation parameter α1, the first offset value shift1 and the first segmentation value x p1 All of them have been explained in the above embodiments and will not be described again. p1 It can be obtained by calculating formula (4).

[0169] In step 1303, the control unit 31 provides a second segmentation parameter α2, a second offset value shift2 and a second segmentation value x corresponding to the weight x2. p2 . The second split value x p2 It can be obtained by calculating formula (4).

[0170] In step 1304, the quantization unit 32 calculates the first segmentation parameter α1, the first offset value shift1 and the first segmentation value x p1 , quantize the activation value x1 to generate the first fixed-point data. The comparator 322 is based on the first segmentation value x p1 , determines whether the activation value x1 falls within the first interval A, which is defined by equation (1). If the activation value x1 falls within the first interval A, the two-way selector 323, the adder 324, and the quantizer 325 generate the intermediate fixed-point data x1[n-1:0] according to the following expression:

[0171]

[0172]

[0173]

[0174] If the activation value x1 does not fall within the first interval A, the binary selector 323, the adder 324, and the quantizer 325 generate the intermediate fixed-point data x1[n-1:0] according to the following expression:

[0175]

[0176]

[0177]

[0178] Finally, the bus converter 326 fills in the flag bit flag to generate the complete first fixed-point data x1[n:0].

[0179] In step 1305, the quantization unit 32 calculates the second segmentation parameter α2, the second offset value shift2 and the second segmentation value x p2 , quantize the weight x2 to generate the second fixed-point data. More specifically, the weight x2 falls within the second numerical distribution, and the second numerical distribution is divided into a third interval and a fourth interval. The comparator 322 is based on the second segmentation value x p2 , determines whether the weight value x2 falls within the third interval, which is also defined by equation (1). If the weight value x2 falls within the third interval, the two-way selector 323, adder 324 and quantizer 325 generate intermediate fixed-point data x2[n-1:0] according to the following expression:

[0180]

[0181]

[0182]

[0183] If the weight value x2 does not fall within the third interval, the binary selector 323, the adder 324, and the quantizer 325 generate the intermediate fixed-point data x2[n-1:0] according to the following expression:

[0184]

[0185]

[0186]

[0187] Finally, the bus converter 326 fills in the flag bit flag to generate the complete second fixed-point data x2[n:0].

[0188] In step 1306, the calculation unit 33 performs a multiplication operation on the first fixed-point data x1[n:0] and the second fixed-point data x2[n:0] to generate intermediate data. In this embodiment, the calculation unit 33 has Figure 8The structure of the multiplication unit 80 performs a multiplication operation on the first fixed-point data x1[n:0] and the second fixed-point data x2[n:0] to generate the intermediate data y. As mentioned above, the flag bit of the fixed-point data data structure of the present disclosure is used to record the interval of the first fixed-point data x1[n:0] and the second fixed-point data x2[n:0], the sign bit is used to record the positive and negative signs of the first fixed-point data x1[n:0] and the second fixed-point data x2[n:0], and the value bit is used to record the values ​​V1 and V2 of the first fixed-point data x1[n:0] and the second fixed-point data x2[n:0]. The multiplier 81 adds the values ​​of the sign bits of the first fixed-point data x1[n:0] and the second fixed-point data x2[n:0] to generate the signed sum value sign t The task of the first selector 821 is equivalent to multiplying the first segmentation parameter α1 and the value of the flag bit x1[n] of the first fixed-point data to generate a first parameter product value pm1; the task of the second selector 822 is equivalent to multiplying the second segmentation parameter α2 and the value of the flag bit x2[n] of the second fixed-point data to generate a second parameter product value pm2; finally, the fixed-point to floating-point converter 83 executes the following expression to generate intermediate data:

[0189]

[0190] This intermediate data is the output of this layer and also the input data for the next layer, i.e., the activation value. The method returns to step 1301, where the activation value of the next layer obtained in this step is input to the next layer, and the process repeats until all layers are fully calculated.

[0191] In step 1307, the neural network reasoning is completed. Figure 2 In the neural network architecture shown, the input data can be diverse, such as images, speech, text data, etc. These data are repeatedly multiplied multiple times in the input layer 201, convolution layer 202, pooling layer 203, and fully connected layer 204 until the inference process is completed, and finally these image, speech, and text data are recognized.

[0192] Another embodiment of the present disclosure is a method for back propagation in a neural network, which quantifies the back propagation gradient by propagating error values. The method can also be used Figure 3 or Figure 12 For the convenience of explanation, the following will be used with Figure 3 The embodiment of the invention is described. Figure 14A flow chart of the method of this embodiment is shown. The process of this embodiment is described based on the aforementioned hardware design. It is understandable that it is not limited to the hardware design of the present disclosure, such as digital circuits or analog circuits. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. The artificial intelligence processor mentioned in the embodiment can be any appropriate hardware processor, such as CPU, GPU, FPGA, DSP and ASIC, etc. Unless otherwise specified, the memory, storage device, storage unit can be any appropriate magnetic storage medium or magneto-optical storage medium, such as resistive random access memory RRAM, dynamic random access memory DRAM, static random access memory SRAM, enhanced dynamic random access memory EDRAM, high bandwidth memory HBM, hybrid memory cube HMC, etc.

[0193] In step 1401, the quantization unit 32 receives the next layer error value x R . Reference Figure 2 , the back propagation is in the direction of the fully connected layer 204, the pooling layer 203, the convolution layer 202, and the input layer 201, and the error value of the output value is propagated back in reverse. For the final output node, the difference between the activation value generated by the network and the actual value is taken as the error value x of the next layer R , the error value x of the next layer R That is Figure 3 The input data x of the device.

[0194] In step 1402, the control unit 31 provides the error value x corresponding to the next layer. R The segmentation parameter α R , offset value shift R and the split value x pR It should be noted that the forward segmentation parameter α, the offset value shift and the segmentation value x p and the reverse segmentation parameter α R , offset value shift R and the split value x pR Different, where the split value x pR The same can be obtained by calculation using formula (4).

[0195] In step 1403, the quantization unit 32 calculates the value of the segmentation parameter α according to the segmentation parameter α. R , offset value shift R and the split value x pR , quantize the next layer error value x R , to generate error value fixed point data. More specifically, the comparator 322 is based on the division value x pR , determine the next layer error value x R Whether it falls within the first interval A, the first interval A is defined by formula (1).R If it falls within the first interval A, the two-way selector 323, the adder 324 and the quantizer 325 generate the intermediate fixed-point data x according to the following expression: R [n-1:0]:

[0196]

[0197]

[0198]

[0199] The next layer error value x R If it does not fall within the first interval A, the two-way selector 323, the adder 324 and the quantizer 325 generate the intermediate fixed-point data x according to the following expression: R [n-1:0]:

[0200]

[0201]

[0202]

[0203] Finally, the bus converter 326 fills in the flag to generate the complete fixed-point data x R [n:0].

[0204] In step 1404, the calculation unit 33 calculates the error value fixed point data x R [n:0] and the weight fixed point number x2[n:0] are multiplied to generate the gradient of the weight x2. The weight fixed point number x2[n:0] can be quantized in the forward propagation process (step 1305). In this embodiment, the calculation unit 33 has Figure 8 The structure of the multiplication unit 80 is to calculate the error value fixed-point data x R [n:0] and the weight fixed-point data x2[n:0] are multiplied to generate the gradient of the weight x2. As mentioned above, the flag bit of the fixed-point data structure of the present disclosure is used to record the error value fixed-point data x R [n:0] and the weight fixed-point data x2[n:0], the sign bit is used to record the error value fixed-point data x R [n:0] and the positive and negative signs of the weighted fixed-point data x2[n:0], and the numerical bit is used to record the error value fixed-point data x R [n:0] and the value V of the weighted fixed-point data x2[n:0] R , V2. Multiplier 81 multiplies the error value fixed point data x R [n:0] and the sign bit of the weighted fixed-point data x2[n:0] are added to generate the sign sum value signt The task of the first selector 821 is equivalent to dividing the parameter α R and the flag bit x of the error value fixed-point data R The task of the second selector 822 is equivalent to multiplying the second segmentation parameter α2 and the value of the flag bit x2[n] of the weight fixed-point data to generate the second parameter multiplication value pm2; finally, the fixed-point to floating-point converter 83 executes the following expression to generate the gradient of the weight x2:

[0205]

[0206] In step 1405, the quantization unit 32 performs fixed-point calculation on the gradient to generate the error value of this layer. The detailed process of performing fixed-point calculation is as described above and will not be repeated here.

[0207] In step 1406, the control unit 31 adjusts the weight x2 according to the error value of the current layer. The back propagation algorithm starts from the output layer of the neural network model and uses the chain rule of function derivation to calculate the model gradient layer by layer to adjust the weight x2.

[0208] Another embodiment of the present disclosure is a method for training a neural network. The training of a neural network generally uses the output value obtained by forward propagation to calculate its error value, and then backpropagates the error value back to the input end, and adjusts the weight of each layer according to the error value of the layer, so that the reasoning structure of the neural network model is closer to the actual situation. In other words, the method for training a neural network of this embodiment includes Figure 13 The forward propagation process is to obtain the activation value of the next layer, and then the error value of the next layer is calculated according to the activation value of the next layer, and then the next layer is executed. Figure 13 The back propagation process is used to generate the error value of this layer, and then the appropriate weights are adjusted according to the error value of this layer. The appropriate weights for each layer are obtained by pushing back all the way forward.

[0209] Figure 15 FIG. 1 is a structural diagram showing an integrated circuit device 1500 according to an embodiment of the present disclosure. Figure 15 As shown, the integrated circuit device 1500 includes a computing device 1502 , which is a neural network computing device in the aforementioned embodiments. In addition, the integrated circuit device 1500 also includes a universal interconnect interface 1504 and other processing devices 1506 .

[0210] Other processing devices 1506 can be one or more general-purpose and / or specialized processors, such as a central processing unit (CPU), a graphics processing unit (GPU), and an artificial intelligence (AI) processor. The number of other processing devices 1506 is not limited and is determined based on actual needs. Other processing devices 1506 serve as an interface between computing device 1502 and external data and control, performing tasks including but not limited to data transfer and basic control of computing device 1502, such as starting and stopping the computing device. Other processing devices 1506 can also collaborate with computing device 1502 to jointly complete computing tasks.

[0211] Universal interconnect interface 1504 can be used to transmit data and control instructions between computing device 1502 and other processing devices 1506. For example, computing device 1502 can obtain required input data from other processing devices 1506 via universal interconnect interface 1504 and write the data to the on-chip storage unit of computing device 1502. Furthermore, computing device 1502 can obtain control instructions from other processing devices 1506 via universal interconnect interface 1504 and write the instructions to the on-chip control cache of computing device 1502. Alternatively or optionally, universal interconnect interface 1504 can also read data from the storage module of computing device 1502 and transmit it to other processing devices 1506.

[0212] Integrated circuit device 1500 further includes a storage device 1508, which can be connected to computing device 1502 and other processing device 1506. Storage device 1508 is used to store data from computing device 1502 and other processing device 1506, and is particularly useful for data that cannot be fully stored in the internal storage of computing device 1502 or other processing device 1506.

[0213] Depending on the application scenario, integrated circuit device 1500 can serve as a system-on-chip (SOC) for devices such as mobile phones, robots, drones, and video capture devices, effectively reducing the core area of ​​the control unit, improving processing speed, and lowering overall power consumption. In this case, the universal interconnect interface 1504 of integrated circuit device 1500 is connected to certain components of the device. These components may be, for example, a camera, display, mouse, keyboard, network card, or Wi-Fi interface.

[0214] The present disclosure further discloses a chip or integrated circuit chip, which includes the integrated circuit device 1500. The present disclosure further discloses a chip package structure, which includes the above chip.

[0215] Another embodiment of the present disclosure is a board card, which includes the above chip packaging structure. Figure 16 In addition to the plurality of chips 1602 described above, the board 1600 may also include other supporting components, including a storage device 1604 , an interface device 1606 and a control device 1608 .

[0216] The memory device 1604 is connected to the chip 1602 in the chip package structure via a bus 1616 for storing data. The memory device 1604 may include multiple groups of memory cells 1610 .

[0217] Interface device 1606 is electrically connected to chip 1602 within the chip package. Interface device 1606 facilitates data transmission between chip 1602 and an external device 1612 (e.g., a server or computer). In this embodiment, interface device 1606 is a standard PCIe interface. Data to be processed is transferred from the server to chip 1602 via the standard PCIe interface, enabling data transfer. Calculation results from chip 1602 are also transmitted back to external device 1612 via interface device 1606.

[0218] The control device 1608 is electrically connected to the chip 1602 to monitor the status of the chip 1602. Specifically, the chip 1602 and the control device 1608 can be electrically connected via an SPI interface. The control device 1608 can include a microcontroller (MCU).

[0219] Another embodiment of the present disclosure is an electronic device or apparatus, which includes the above-mentioned board 1600. Depending on different application scenarios, the electronic device or apparatus may include a data processing device, a robot, a computer, a printer, a scanner, a tablet computer, a smart terminal, a mobile phone, a driving recorder, a navigator, a sensor, a camera, a server, a cloud server, a camera, a video camera, a projector, a watch, a headset, a mobile storage, a wearable device, a vehicle, a household appliance, and / or a medical device. The vehicle includes an airplane, a ship, and / or a vehicle; the household appliance includes a television, an air conditioner, a microwave oven, a refrigerator, an electric rice cooker, a humidifier, a washing machine, an electric light, a gas stove, and a range hood; and the medical device includes an MRI machine, an ultrasound machine, and / or an electrocardiograph.

[0220] Another embodiment of the present disclosure is an electronic device, comprising one or more processors and a memory, wherein the memory stores computer executable instructions. When the computer executable instructions are executed by the one or more processors, the electronic device executes the method as described above, in particular, executes the method as described above. Figure 7 、 Figure 9 、 Figure 13 and Figure 14 The method described.

[0221] Another embodiment of the present disclosure is a computer-readable storage medium having stored thereon computer-executable instructions for computing data in a computing device. When the computer-executable instructions are executed by one or more processors, the method described above is executed, in particular, the method described above is executed. Figure 7 、 Figure 9 、 Figure 13 and Figure 14 The method described.

[0222] This disclosure proposes a technical solution for adaptive non-uniform low-bit fixed-point training. By introducing multiple quantization intervals, non-uniform fixed-point data formats, and corresponding hardware, this solution addresses the technical issue of insufficient quantization bit width to effectively reduce energy consumption, thereby achieving the following technical effects:

[0223] 1. Reduce network model memory usage, reduce model training resource consumption, and increase training speed.

[0224] 2. When using the neural network model disclosed in the present invention for reasoning, the reasoning speed can be accelerated and the accuracy deterioration can be reduced.

[0225] 3. When training the neural network model disclosed herein, problems such as training collapse and non-convergence can be avoided.

[0226] The foregoing content can be better understood in accordance with the following terms:

[0227] Item A1. A multiplication unit comprising: a multiplier for multiplying first fixed-point data and second fixed-point data to generate a fixed-point product; an addition module for adding a plurality of quantization offset coefficients corresponding to the first fixed-point data and the second fixed-point data to generate a sum of quantization offset coefficients; and a fixed-point to floating-point converter for converting the fixed-point product into floating-point data according to the sum of the quantization offset coefficients.

[0228] Item A2. A multiplication unit according to Item A1, wherein the first fixed-point data falls within a first numerical distribution and the second fixed-point data falls within a second numerical distribution, and the quantization offset coefficient includes: a first segmentation parameter for segmenting the first numerical distribution into a first interval and a second interval; a second segmentation parameter for segmenting the second numerical distribution into a third interval and a fourth interval; a first flag value for reflecting that the first fixed-point data falls within the first interval or the second interval; and a second flag value for reflecting that the second fixed-point data falls within the third interval or the fourth interval; the addition module includes: a first selector for setting the first output value to the first segmentation parameter or 0 according to the first flag value; a second selector for setting the second output value to the second segmentation parameter or 0 according to the second flag value.

[0229] Clause A3. The multiplication unit of clause A2, wherein the quantized offset coefficient comprises:

[0230] The first offset value is generated according to the following expression:

[0231]

[0232] The second offset value is generated according to the following expression:

[0233]

[0234] Wherein, x1 is the floating-point data corresponding to the first fixed-point data, x2 is the floating-point data corresponding to the second fixed-point data, the ceil function returns the smallest integer greater than or equal to the expression, and b is the quantization bit width.

[0235] Item A4. The multiplication unit according to Item A3, wherein the addition module further comprises an adder for adding the first offset value, the second offset value, the first output value, and the second output value to generate the quantized offset coefficient sum.

[0236] Item A5. The multiplication unit of Item A1, wherein the fixed-point to floating-point converter comprises a power-of-two calculator, the sum of the quantization offset coefficients being a power of two to generate a quantization offset value.

[0237] Item A6. A multiplication unit according to Item A5, wherein the fixed-point to floating-point converter includes a multiplier that multiplies the fixed-point product by the quantization offset value to obtain the floating-point data.

[0238] Clause A7. An integrated circuit device comprising a multiplication unit according to any one of clauses A1-6.

[0239] Item A8. A board comprising the integrated circuit device according to Item A7.

[0240] Item A9. A method for calculating a neural network, comprising: quantizing first floating-point data according to a first segmentation parameter to generate first fixed-point data; quantizing second floating-point data according to a second segmentation parameter to generate second fixed-point data; performing a multiplication operation on the first fixed-point data and the second fixed-point data to generate intermediate data; and calculating the neural network based on the intermediate data.

[0241] Item A10. A method according to Item A9, wherein the first floating-point data falls within a first numerical distribution, and the step of quantizing the first floating-point data includes: dividing the first numerical distribution into a first interval and a second interval based on the first segmentation parameter; and determining whether the first floating-point data falls within the first interval.

[0242] Clause A11. The method of clause A10, wherein the first interval is:

[0243]

[0244] Among them, x1 is the first floating-point data, α1 is the first segmentation parameter, and D1 is the first data distribution.

[0245] Clause A12. The method of clause A11, wherein if the first floating-point data falls within the first interval, the quantizing the first floating-point data step generates the first fixed-point data according to the following expression:

[0246]

[0247]

[0248]

[0249] in, For the first fixed-point data, the ceil function returns the smallest integer greater than or equal to the expression, and b is the quantization bit width.

[0250] Item A13. A method according to Item A12, wherein the second floating-point data falls within a second numerical distribution, and the step of quantizing the second floating-point data includes: dividing the second numerical distribution into a third interval and a fourth interval based on the second segmentation parameter; and determining whether the second floating-point data falls within the third interval.

[0251] Clause A14. The method of clause A13, wherein the third interval is:

[0252]

[0253] Wherein, x2 is the second floating-point data, α2 is the second segmentation parameter, and D2 is the second data distribution.

[0254] Clause A15. The method of clause A14, wherein if the second floating-point data falls within the third interval, the quantizing the second floating-point data step generates the second fixed-point data according to the following expression:

[0255]

[0256]

[0257]

[0258] in, is the second fixed-point data.

[0259] Item A16. A method according to Item A15, wherein the data structure of the first fixed-point data and the second fixed-point data includes: a flag bit for recording the interval of the first fixed-point data and the second fixed-point data; a sign bit for recording the positive and negative signs of the first fixed-point data and the second fixed-point data; and a value bit for recording the values ​​V1, V2 of the first fixed-point data and the second fixed-point data.

[0260] Item A17. The method according to Item A16, wherein the step of performing the multiplication operation comprises: adding the values ​​of the sign bits of the first fixed-point data and the second fixed-point data to generate a signed sum value sign t ; multiplying the first segmentation parameter and the value of the flag bit of the first fixed-point data to generate a first parameter multiplication value pm1; multiplying the second segmentation parameter and the value of the flag bit of the second fixed-point data to generate a second parameter multiplication value pm2; and executing the following expression to generate the intermediate data:

[0261]

[0262] Item A18. A computer-readable storage medium comprising computer-executable instructions, which, when executed by one or more processors, perform the method as described in any one of items A9-17.

[0263] The above is a detailed introduction to the embodiments of the present disclosure. Specific examples are used herein to illustrate the principles and implementation methods of the present disclosure. The description of the above embodiments is only used to help understand the method and core ideas of the present disclosure. At the same time, for those skilled in the art, based on the ideas of the present disclosure, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present disclosure.

Claims

1. A method for computing a neural network, comprising: quantizing first floating-point data according to a first segmentation parameter to generate first fixed-point data, wherein the first floating-point data falls within a first value distribution, and segmenting the first value distribution into a first interval and a second interval based on the first segmentation parameter, wherein the first interval is: Wherein, x1 is the first floating-point data, α1 is the first segmentation parameter, and D1 is the first numerical distribution; quantizing the second floating-point data according to the second segmentation parameter to generate second fixed-point data; performing a multiplication operation on the first fixed-point data and the second fixed-point data to generate intermediate data; and The neural network is calculated based on the intermediate data, wherein the input data of the neural network includes image data, voice data or text data.

2. The method according to claim 1, wherein the step of quantizing the first floating-point data comprises: Based on the first segmentation parameter, segment the first numerical distribution into a first interval and a second interval; It is determined whether the first floating-point data falls within the first interval.

3. The method according to claim 1 , wherein if the first floating-point data falls within the first interval, the step of quantizing the first floating-point data generates the first fixed-point data according to the following expression: in, For the first fixed-point data, the ceil function returns the smallest integer greater than or equal to the expression, and b is the quantization bit width.

4. The method of claim 3 , wherein the second floating point data falls within a second numerical distribution, the step of quantizing the second floating point data comprising: Based on the second segmentation parameter, segment the second value distribution into a third interval and a fourth interval; as well as It is determined whether the second floating-point data falls within the third interval.

5. The method according to claim 4, wherein the third interval is: in, x2 is the second floating-point data, α2 is the second segmentation parameter, and D2 is the second data distribution.

6. The method according to claim 5 , wherein if the second floating-point data falls within the third interval, the step of quantizing the second floating-point data generates the second fixed-point data according to the following expression: in, is the second fixed-point data.

7. The method according to claim 6, wherein the data structures of the first fixed-point data and the second fixed-point data include: A flag bit, used to record the interval of the first fixed-point data and the second fixed-point data; a sign bit, used to record the positive and negative signs of the first fixed-point data and the second fixed-point data; and The value bit is used to record the values ​​V1 and V2 of the first fixed-point data and the second fixed-point data.

8. The method according to claim 7, wherein the performing the multiplication step comprises: Add the sign bits of the first fixed-point data and the second fixed-point data to generate a signed sum value sign t ; Multiplying the first segmentation parameter and the value of the flag bit of the first fixed-point data to generate a first parameter multiplication value pm1; multiplying the second segmentation parameter and the value of the flag bit of the second fixed-point data to generate a second parameter product value pm2; and The following expression is executed to generate the intermediate data:

9. A computer-readable storage medium comprising computer-executable instructions, wherein when the computer-executable instructions are executed by one or more processors, the method according to any one of claims 1 to 8 is executed.

Images