Method and system for recognizing images

Abstract

A method and system for recognizing an image are provided. The method includes: obtaining a tensor of an artificial neural network; dividing a quantization range of the tensor of the artificial neural network into a first region and a second region; separately quantizing the value of the tensor in the first region and the value of the tensor in the second region; applying the quantized value of the tensor in the first region and the quantized value of the tensor in the second region to the artificial neural network to obtain a quantized artificial neural network; obtaining image data to be recognized; and inputting the image data to be recognized into the quantized artificial neural network to execute the quantized neural network to perform image recognition on the image data to be recognized.

Description

Technical Field

[0001] The present disclosure relates generally to pattern recognition and, more particularly, to methods and systems for recognizing images. Background Art

[0002] Artificial neural networks, such as deep neural networks (DNNs), can be used for image recognition. The accuracy of DNNs can be improved by increasing one or more dimensions, such as network depth, network width, and image resolution, but this can result in greater computational complexity and memory requirements, as well as increased energy consumption and execution time. These increased resource requirements can make DNNs difficult to deploy in resource-constrained applications, such as embedded systems and mobile devices.

[0003] Quantization techniques can be used to reduce the resource requirements of DNNs by converting full-precision weights and activations into low-precision (low-bit-width) representations. For example, 32-bit floating-point numbers (FP32) commonly used when training artificial neural networks can be quantized by converting them into smaller 8-bit integers (INT8), which reduces the number of discrete values used to represent each number. These low-bit-width numbers can take up less memory space and reduce the complexity, cost, and / or energy consumption of the systems that process them (whether in hardware, software, or a combination thereof). However, quantization can introduce quantization errors, which reduce the performance (accuracy) of the DNN.

[0004] To minimize quantization error and maintain the performance of full-precision models, quantized DNNs can be trained from scratch using quantization-aware training. In other approaches, pre-trained floating-point models can be fine-tuned after quantization. However, both approaches can complicate and slow down deployment processing and may require access to the entire training dataset.

[0005] In some applications, post-training quantization (PTQ) using uniform quantization of INT8 numbers can maintain close to the original FP32 pre-trained model performance. However, in some applications, the resource requirements of 8-bit quantized DNNs may still be too high for wider deployment. Low-bitwidth techniques (such as 4-bit uniform PTQ) can enable DNNs to be deployed in a wider range of resource-constrained applications, but low-bitwidth quantization can reduce performance (e.g., image recognition accuracy) to unacceptable levels. Summary of the Invention

[0006] A method for recognizing an image may include: obtaining a tensor of an artificial neural network; dividing a quantization range of the tensor of the artificial neural network into a first region and a second region; separately quantizing a value of the tensor in the first region and a value of the tensor in the second region; applying the quantized value of the tensor in the first region and the quantized value of the tensor in the second region to the artificial neural network to obtain a quantized artificial neural network; obtaining image data to be recognized; and inputting the image data to be recognized into the quantized artificial neural network to execute the quantized neural network to perform image recognition on the image data to be recognized.

[0007] A system for recognizing an image may include: a processor configured to execute a program, wherein when the program is executed, the processor performs a method for recognizing an image.

[0008] A method for quantizing an artificial neural network may include: dividing a quantization range of a tensor of the artificial neural network into a first region and a second region; and separately quantizing values of the tensor in the first region and values of the tensor in the second region. The values of the tensor in the first region may be quantized according to a first quantization function, and the values of the tensor in the second region may be quantized according to a second quantization function.

[0009] The first quantization function and the second quantization function may be different. The first quantization function and the second quantization function may be substantially the same. The first quantization function and the second quantization function may have different scaling factors. Linear quantization may be applied to the values of the tensor in the first region and the second region. The first region may include a positive portion and a negative portion. The second region may include a positive portion and a negative portion. The quantization range may extend from a minimum value to a maximum value of the tensor.

[0010] The step of dividing the quantization range may include locating breakpoints in the first region and the second region. The step of locating the breakpoints may include determining a quantization error over at least a portion of the quantization range. The step of locating the breakpoints may include substantially minimizing the quantization error. The step of minimizing the quantization error may include: formulating the quantization error as a function of the position of the breakpoint; formulating a first-order derivative of the function; and determining a value of the breakpoint such that the first-order derivative is substantially zero. The value of the breakpoint such that the first-order derivative is substantially zero may be determined using a binary search. Regression may be used to approximate the position of the breakpoint. A grid search may be used to substantially minimize the quantization error. The breakpoints may be located offline. The breakpoints may be located online. The breakpoints may be constrained to be less than a midpoint of the quantization range. The breakpoints may be located so that the density of quantized values matches the density of values in the tensor.

[0011] The tensor may include at least two channels, the at least two channels being individually quantized in at least one of a first region and a second region. The tensor may include at least two groups, the at least two groups being individually quantized in at least one of the first region and the second region. The method may also include correcting bias in quantized values of the tensor.

[0012] The tensor may include one or more weights. The tensor may include one or more activations. The artificial neural network may include a deep neural network. The first region and the second region may not overlap. The tensor may include trained values. The values of the tensor may be generated in response to a training dataset, and the quantization range of the tensor may be divided without accessing the entire training dataset or only a portion of the entire training dataset.

[0013] A system may include a processor configured to execute a program comprising: dividing a quantization range of a tensor of an artificial neural network into a first region and a second region; and quantizing values of the tensor in the first region separately from values of the tensor in the second region. The processor may also be configured to execute the program comprising: locating a breakpoint by substantially minimizing a quantization error over at least a portion of the quantization range.

[0014] An artificial neural network device may include: a first accumulator configured to sum activations and weight products of a first region of a quantization range; a second accumulator configured to sum activations and weight products of a second region of the quantization range; and a third accumulator configured to sum activations of the second region. The activations of the second region may be multiplied by an offset. The first accumulator, the second accumulator, and the third accumulator may be implemented in dedicated hardware. The first accumulator, the second accumulator, and the third accumulator may be implemented by one or more processors configured to execute an accumulation program. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings are not necessarily drawn to scale, and for illustrative purposes, elements of similar structure or function are generally represented by the same reference numerals throughout the drawings. The drawings are intended only to facilitate the description of the various embodiments described herein. The drawings do not describe every aspect of the teachings disclosed herein and do not limit the scope of the claims. To prevent the drawings from becoming obscure, not all components, connections, etc. may be shown, and not all components may have reference numerals. However, the mode of component configuration can be easily understood from the drawings. The drawings, together with the description, illustrate example embodiments of the present disclosure, and the drawings, together with the description, serve to explain the principles of the present disclosure.

[0016] Figure 1 Shown is a histogram illustrating an example distribution of weights in a trained deep neural network.

[0017] Figure 2Shown from Figure 1 The distribution of quantized values of the 4-bit uniform post-training quantization of the weights in .

[0018] Figure 3 The distribution of quantized values after piecewise linear training according to the present disclosure is shown.

[0019] Figure 4 An embodiment of a piecewise quantization technique according to the present disclosure is shown.

[0020] Figure 5 An embodiment of an apparatus for processing piecewise quantized values in an artificial neural network according to the present disclosure is shown.

[0021] Figure 6 An embodiment of a computing system that can be used to implement any or all of the methods and / or apparatus described in this disclosure is shown. DETAILED DESCRIPTION

[0022] In some artificial neural networks, the distribution of weights and activations in a pre-trained model can be approximated by Gaussian and / or Laplace functions. Figure 1 An example of a histogram showing an example distribution of weights in a trained deep neural network (DNN) is shown in . Most of the weights may be concentrated around zero, while the reduced number may be spread out in a long tail.

[0023] Figure 2 Shown from Figure 1 The distribution of quantized values of 4-bit uniform post-training quantization (PTQ) of the weights in (shown as thick bars). Figure 2It is clear that despite the uneven distribution of weights, the uniform quantization technique can evenly divide the quantization range. Therefore, particularly in the case of low bit width, the uniform quantization scheme may assign too few quantization levels to a large number of values with small amplitudes, and assign relatively too many quantization levels to a relatively small number of values with large amplitudes. This can degrade the performance of the model by, for example, increasing the quantization error and reducing the accuracy of the results. According to an embodiment of the present invention, a method for recognizing an image may include: obtaining a tensor of an artificial neural network; dividing the quantization range of the tensor of the artificial neural network into a first region and a second region; separately quantizing the values of the tensor in the first region and the values of the tensor in the second region; applying the values of the tensor in the quantized first region and the values of the tensor in the quantized second region to the artificial neural network to obtain a quantized artificial neural network; obtaining image data to be recognized; and inputting the image data to be recognized into the quantized artificial neural network to execute the quantized neural network to perform image recognition on the image data to be recognized. Here, the method may also include: obtaining the artificial neural network by training an initial neural network using a training image.

[0024] In an artificial neural network according to some principles of the present disclosure, the quantization range of a tensor (such as a weight) can be divided into two or more regions that can be quantized separately. Figure 3 As shown in , the piecewise linear quantization technique divides the quantization range of the trained weight distribution into two non-overlapping regions and applies uniform quantization to each region.

[0025] Figure 3 Shown as applied to Figure 1 The distribution of quantized values of the weights shown in FIG. 1 (shown as thick bars) after quantization according to the piecewise linear training of the present disclosure. Figure 3 The height of each thick bar in represents the number of weight samples quantized at that weight value.

[0026] The quantization range [-t, t] can be divided into two regions, each having a positive portion and a negative portion. Region 1 may include a negative portion [-x, 0] and a positive portion [0, x]. Region 1 may be represented as [-x, 0] ∪ [0, x], or alternatively as [-x, x]. Region 2 may include a negative portion [-t, -x] and a positive portion [x, t]. Region 2 may be represented as [-t, -x] ∪ [x, t]. For convenience, the two portions in each of the two regions may also be collectively or individually referred to as the four segments of the quantization range [-t, -x], [-x, 0], [0, x], and [x, t].

[0027] The breakpoint x between region 1 and region 2 can be shown as a dashed line at -x and x, and 2-bit uniform quantization can be used in each segment. Therefore, there can be four quantization values in each segment, and a total of 16 quantization values over the entire quantization range.

[0028] The thick bar at -x may include two co-located bars: one at the upper end of the segment [-t, -x] and the other at the lower end of the segment [-x, 0]. Similarly, the thick bars at 0 and x may each include two co-located bars at the ends of adjacent segments of the quantization range.

[0029] from Figure 3 It is clear that in the region [-x, x] (region 1) where the distribution of weights is denser, the quantized values can be denser. Similarly, in the region [-t, -x] ∪ [x, t] (region 2) where the distribution of weights is less dense, the quantized values can be less dense. According to implementation details, even if Figure 3 The piecewise linear PTQ technique and Figure 2 The uniform PTQ technique shown in FIG can use the same number of quantization values. This density matching can also make Figure 3 The piecewise linear PTQ technique shown in Figure 2 The uniform PTQ technique shown in provides better performance (eg, lower quantization error).

[0030] Furthermore, depending on implementation details, piecewise quantization techniques according to the principles of the present disclosure may provide one or more additional benefits (such as higher speed, lower power consumption, reduced memory requirements, lower complexity and / or cost of hardware and / or software, reduced chip area on integrated circuits, implementation with little or no modification to commodity hardware, etc.). For example, Figure 2 The uniform quantization scheme shown in FIG may require 4-bit additions, 4-bit multiplications, 4-bit accumulations, etc., while Figure 3 The piecewise quantization technique shown in may only require 2-bit operations for at least a portion of the pipeline, which may be faster, more efficient, etc. Furthermore, depending on implementation details, some or all of these benefits may be achieved without access to the entire training dataset.

[0031] In some embodiments, the performance of the piecewise quantization model may depend on one or more breakpoints (e.g., Figure 3

[0045] In one embodiment, one or more breakpoints may be placed at or near one or more optimized locations to reduce or minimize quantization error. Principles of the present disclosure also include techniques for locating one or more optimized breakpoints. These techniques may include formulating the quantization error as a function of the breakpoints and analytically solving for zero derivatives, approximating the optimized breakpoints using linear approximations, and / or performing a grid search, all of which are described in more detail below.

[0032] According to the principles of this disclosure, Figure 3 The embodiments shown in FIG. 5 can be modified in many ways and configured and / or adapted for use in countless applications. Some possible modifications and / or applications are as follows.

[0033] In some embodiments, any bit width can be used to quantize any region or all regions and / or portions of regions of the quantization range. Different bit widths can be used for different regions and / or portions of regions, as well as different layers of an artificial neural network and / or nodes within a layer.

[0034] In some embodiments, piecewise quantization can be applied to any tensor or combination of tensors of an artificial neural network including weights, activations (including activations before and after application of an activation function), and any other tensor or value that can be quantized. Any tensor can be signed or unsigned. Piecewise quantization can be applied to any tensor or combination of tensors with any distribution of values at any location in the artificial neural network. This can include, for example, tensors that generally have any bell-shaped or normal distribution (such as a Gaussian distribution or a Laplace distribution), which distributions generally can have long tails. Piecewise quantization can be applied to tensors with distributions of values that are uniform or non-uniform, symmetric or asymmetric, and the like.

[0035] In some embodiments, piecewise quantization can be applied to any type of artificial neural network (including deep neural networks, convolutional neural networks, feedforward neural networks, etc.).

[0036] In some embodiments, piecewise quantization can be used with any quantization function or combination thereof, including uniform quantization or linear quantization and nonlinear quantization, affine quantization or non-affine quantization, etc. Different quantization functions can be used for different regions and / or portions of regions of the quantization range. Piecewise quantization can be applied per layer, per channel, per group, or any variant or combination thereof.

[0037] In some embodiments, the quantization range may be divided into any number of regions and / or portions of regions, and may have any number of breakpoints between regions and / or portions of regions. Regions and / or portions of regions may be symmetrical or asymmetrical, balanced or unbalanced. Regions and / or portions of regions may have only negative values, only positive values, or a combination thereof. The quantization range used for quantization may span the entire (minimum to maximum) value range of the tensor, or may cover only one or more portions of the entire value range of the tensor. The quantization range may be divided into regions and / or portions of regions that are overlapping or non-overlapping, continuous or discontinuous (touching or spaced apart), or any combination thereof. Depending on implementation details, for example, the use of non-overlapping regions and / or portions of regions may improve the accuracy of the model using piecewise quantization according to the present disclosure.

[0038] In some embodiments, quantization may be applied separately to each region or portion thereof of the quantization range.In some embodiments, operations and / or devices may be combined to fully or partially combine quantization in more than one region or portion thereof.

[0039] In some embodiments, piecewise quantization can be implemented online (real time) or offline, or any combination thereof. Piecewise quantization can be implemented after training, during pre-training, during retraining after adjustment or initial training, during quantization-aware training, or any combination thereof. In some embodiments, piecewise quantization according to the present disclosure can be implemented without access to all of the training data set that may have been used to generate the distribution of tensor values. In some embodiments, part or all of the training data set may be used. For example, a small portion of the training data set may be used to describe a series of activation functions.

[0040] In some embodiments, piece-wise quantization may be implemented in hardware, software, or any combination thereof.

[0041] In some embodiments, piecewise quantization may be implemented using affine quantization in one or more regions or portions thereof of the quantization range. In some embodiments, affine quantization may involve quantization parameters such as scaling factors and offsets or zero points.

[0042] In some embodiments, quantization may be represented as follows:

[0043]

[0044] Among them, x q can be the quantized output value, x can be the input floating point value, x min can be a floating point value at the minimum end of the quantized region, and S can be a scaling factor given as follows:

[0045]

[0046] Among them, x max Can be a floating point value at the maximum end of the quantization region, N levels =2 b , and b can be the bit width used for quantization.

[0047] In the case of symmetric quantization, x max It can be given by:

[0048] x max =-x min =max(|x|) (Equation 3)

[0049] In the case of asymmetric quantization, x max It can be given by:

[0050] x max =max(x) (Equation 4A)

[0051] x min =min(x) (Equation 4B)

[0052] where x can be an array of input floating-point values.

[0053] While the inventive principles are not limited to the use of any particular type of quantization, the use of affine quantization techniques (such as the example described above with reference to Equation 1) may provide one or more advantages depending on the implementation details. This may be understood with reference to some alternative implementation techniques. For example, one possible technique for implementing quantization may be to use numbers represented in fixed point. However, the use of fixed point numbers may limit the minimum and / or maximum numbers that can be represented, and may also limit the options for selecting breakpoints and / or scaling factors. Any of these limitations may reduce the accuracy of the model and / or results. For example, the use of fixed point numbers may limit the minimum and / or maximum values that can be represented, and / or breakpoints and / or scaling factors, to values that can be equal to 2. N , where N can be a positive or negative integer. In contrast, the use of the quantization techniques described above with reference to Equation 1 can ensure the use of floating point or other more flexible numerical representations for scaling factors, offsets or zero points, breakpoints, etc., and can also ensure the representation of smaller numbers, wider ranges of numbers, etc., any of which, depending on implementation details, can improve the accuracy of the model and / or results and other performance metrics.

[0054] A further potential advantage of a quantization technique such as the example described above with reference to Equation 1 is that it may facilitate the implementation of non-overlapping quantization regions. For example, in some embodiments, by min Placed at the minimum end of the piecewise quantization region for larger values, quantization resolution is not wasted for values less than x min The value of is less than x minThe values of may be quantized as parts of separate regions for smaller values.

[0055] Figure 4 Another embodiment of the piecewise quantization technique according to the present disclosure is shown. Figure 4 The embodiment shows how weight values represented in floating point over the entire quantization range [-t, t] (shown as continuous horizontal lines) can be mapped to four different 4-bit (INT4) quantization values in the four segments of the quantization range [-t, -x], [-x, 0], [0, x], and [x, t].

[0056] In some embodiments, the optimized breakpoints can be found by minimizing the expected quantization error. For example, in an embodiment with a quantization range [-t, t] having Gaussian weights and divided into two regions (each region having a positive part and a negative part), the weight distribution can be divided into four segments [-t, -x], [-x, 0], [0, x], and [x, t]. Each segment can have N quantized values representing floating-point weights. levels The Gaussian probability density function (PDF) can be given by:

[0057]

[0058] The cumulative distribution function (CDF) can be given by:

[0059]

[0060] The expected quantization error can then be given by:

[0061]

[0062] Where C can be a constant given by,

[0063]

[0064] And wherein b can be the bit width used for quantization.

[0065] The first derivative of Equation 7 can be given by:

[0066] D(x,t)=x+2t*(F(x)-1)-f(x)*(t 2 -2tx) (Equation 9)

[0067] Wherein, if the solution is 0, an optimized breakpoint may be provided.

[0068] D(x, t) can increase monotonically, so

[0069]

[0070] D(t, t)=t+f(t)*t 2 >0 (Equation 11)

[0071] Therefore, there can be a unique optimal breakpoint x opt Make

[0072] D(x opt , t) = 0 (Equation 12)

[0073] This minimizes quantization error.

[0074] According to the implementation details, the quantization error E(x opt , t) can have a smaller quantization error than a uniform quantization process using the same number of quantization levels. For example, a segment-wise quantization technique using 4-bit quantization for each of the four segments of the quantization range according to the present disclosure can have a smaller quantization error than a conventional quantization process applying uniform 6-bit quantization over the same quantization range.

[0075] One approach to solving the equation D(x, t) = 0 to locate the optimized breakpoint according to the present disclosure is to numerically solve it using, for example, a binary search with complexity O(log t). Depending on the implementation details, this approach can provide highly accurate results and may be particularly suitable for implementations where quantization can be performed offline (e.g., when quantizing weights).

[0076] Another method of solving the equation D(x, t) = 0 according to the present disclosure is to use linear regression with O(1) complexity to approximate the optimized breakpoints. This can be achieved, for example, by first determining the maximum value of the standard deviation of the weights (or other tensors) to estimate the normalized quantization range [-t, t]. Then, a binary search can be used to find the optimized breakpoint based on the normalized quantization range. The binary search results can then be used to find a linear approximation of one or more optimized breakpoints. In an example embodiment, the approximate optimized breakpoint value x can be determined by using the following approximation app ,

[0077] x app (t) / t=ln(0.8614*t+0.6079) (Equation 13)

[0078] For Gaussian weight distribution and perturbation |x app -x opt|<0.05. Depending on the implementation details, this method (which can be viewed as a one-shot approach) can reduce the time required to determine the optimized breakpoint while resulting in little, if any, degradation in quantization error. Therefore, this approximation method can be beneficial in implementations that use, for example, online or dynamic quantization with activation. This method can be used with distributions other than Gaussian distributions. For example, for the Laplace distribution, x can be determined using app The value of

[0079]

[0080] Here, the distribution can be normalized.

[0081] Another method for solving the equation D(x, t) = 0 according to the present disclosure is to use a coarse-to-fine grid search, which does not require any assumptions about the distribution of tensor values. In an example embodiment, a three-stage grid search may use the following stages to search for the ratio between the optimal breakpoint and the maximum value of the tensor to minimize the quantization error.

[0082] Stage 1: Search for the optimal ratio r1 in np.arange(0.1,1.0,0.1).

[0083] Stage 2: Search for the optimal ratio r2 in np.arange(r1-0.05,r1+0.05,0.01).

[0084] Stage 3: Search for the optimal ratio r3 in np.arange(r2-0.005,r2+0.005,0.001).

[0085] Among them, np is a NumPy operator, and the np.arange() function returns a set of sequences with a starting point, an end point, and a fixed step size. The three parameters of the function are the starting point, the end point, and the step size.

[0086] Depending on the implementation details, the grid search approach can provide highly accurate results and may be particularly suitable for implementations where quantization can be performed offline. A potential benefit of the coarse-to-fine grid search is that it can directly reduce and / or minimize the quantization error without requiring any specific knowledge and / or assumptions about the distribution of tensor values.

[0087] Table 1 lists some example pseudocode for implementing a coarse-to-fine grid search to locate breakpoints according to the present disclosure.

[0088]

[0089] Table 1

[0090] In some embodiments, when locating breakpoints for piecewise quantization of a tensor that generally has a bell-shaped distribution according to the present disclosure, the highest density of values may generally be clustered around zero. Therefore, the breakpoints are constrained to be smaller than the midpoint of the quantization range (e.g., x opt In some embodiments, the quantization error may be reduced or minimized by evaluating the quantization error over a portion of the quantization range. In some other embodiments, the quantization error may be reduced or minimized by evaluating the quantization error over a portion of the quantization range and / or any or all regions and / or segments into which the range may be divided.

[0091] As described above, the quantization range can be divided into any number of regions and / or portions thereof, and can have any number of breakpoints between regions and / or portions thereof. In some embodiments with multiple breakpoints, any of the techniques discussed above can be extended to find one or more optimized breakpoint values. For example, in some embodiments, multiple optimized breakpoint values can be determined by minimizing the expected quantization error given in Equation 7 as applied to multiple breakpoints. As another example, the coarse-to-fine grid search can also be extended to locate multiple breakpoints. In some embodiments, determining the number and / or location of optimized breakpoints can involve balancing various considerations. For example, depending on implementation details, utilizing multiple breakpoints can provide a significant improvement in accuracy. However, in other implementations, the improvement in accuracy provided by multiple breakpoints can be offset by the additional hardware and / or software execution time required to implement the additional breakpoints.

[0092] While the inventive principles of this disclosure may not require the use of any particular technique to find one or more optimized breakpoints, any analysis, approximation, and / or search techniques disclosed herein may provide a more easily analyzable, scalable, and / or robust solution.

[0093] In some embodiments, according to the present disclosure, dedicated hardware can be used to implement piecewise quantization. In hardware implementation, piecewise quantization can divide the tensor range into two or more regions, each of which can be implemented using a separate computational path (e.g., a computational path to accommodate different scaling factors and / or offsets). Figure 3In the example embodiment of piecewise linear quantization (as shown in ), the hardware implementation may include some hardware similar to that used for uniform quantization within each region and / or portion thereof (e.g., a segment of the quantization range), while adding some additional equipment to accommodate the piecewise arrangement. For example, uniform quantization within each segment can be implemented using two accumulators: one to sum the products of activations and tensors (e.g., weights), and one to sum the activations. An additional accumulator can be included to sum the activations that are multiplied with a non-zero offset in regions or segments that may be zero offset (e.g., in region 2).

[0094] Thus, in some embodiments, a hardware implementation of the piecewise quantization technique according to the present disclosure may include any or all of the following: adders and multipliers similar to those used for uniform quantization; three accumulator registers (the first for the sum of products in one of the two computation paths, the second for the sum of products in the other of the two computation paths, and the third for activations in the path that handles non-zero offsets); and one or two additional storage bits for each tensor (one to indicate the sign and one to indicate the region to which the tensor value is quantized). The bit indicating the sign may not be included in an implementation with unsigned values. Because the bit indicating the region is only used to find the appropriate accumulator, the bit indicating the region may not be included in the multiply-accumulate (MAC) calculation. Thus, in some embodiments, a hardware implementation of the piecewise quantization technique according to the present disclosure may be implemented with minimal modification to commercial hardware.

[0095] Figure 5 An embodiment of a device for processing piecewise quantized values in an artificial neural network according to the present disclosure is shown. Figure 5In an embodiment of the invention, the weight buffer 112 may include columns of 4-bit quantized weight values, each column may be accompanied by an index bit indicating whether it has been quantized in region 1 (gray) or region 2 (white). Any number of columns may be used to provide parallel processing of multiple values, and the columns may have any suitable depth. In this example, the column depth is 9 weights. The local input feature map (IFM) queue 110 may provide a stream of 4-bit quantized values (ifm_value), each 4-bit quantized value may be accompanied by a most significant nibble / least significant nibble bit (ifm_MSN / LSN) to indicate which region it is associated with. The IFM index (ifm_idx) associated with each value (ifm_value) in the local IFM queue 110 may use a selector 114 (in this example, a 9-to-1 selector) to select a weight block 120 that will be multiplied by the value (ifm_value) from the local IFM queue 110 via a 4-bit multiplier 116. The partial products may be shifted by 4 bits in the shift register 118 depending on the state of the ifm_MSN / LSN bits. Thus, each weight block 120 is associated with a corresponding set 122 of values for the IFM index (ifm_idx) and nibble bits (ifm_MSN / LSN). An offset adder tree 124, which may be associated with each column, may sum the activations if the weights in its corresponding column belong to region 2. Figure 5 In the example state shown in FIG, the first column may be associated with region 1, and thus the corresponding offset adder tree 124 for the first column is disabled, as indicated by the X. Conversely, the second and last columns may be associated with region 2, and thus the corresponding offset adder trees 124 for the second and last columns are enabled, as indicated by the circles.

[0096] The principles of the present disclosure relating to piecewise quantization can be combined with other quantization techniques to create additional embodiments that can produce synergistic results. For example, an embodiment that implements piecewise uniform (linear) quantization can implement bias correction within each segment of the quantization range, either individually or collectively. In such an embodiment, the bias in the quantized values of a tensor can be corrected, for example, by folding one or more correction terms into a scaling and / or offset of one or more segments of the quantization range. Depending on the implementation details, the combination of piecewise quantization bias corrections can further reduce quantization error and improve the performance of the artificial neural network in which it is implemented.

[0097] As another example, an embodiment of implementing piecewise uniform (linear) quantization according to the present disclosure may apply separate quantization to each output channel filter for one or more segments of the quantization range. According to the present disclosure, this technology can be further extended to decompose one or more output channel filters into one or more groups that can be individually quantized for one or more segments of the quantization range. Depending on the implementation details, using a combination of piecewise quantization per channel and / or per group can further reduce quantization error and improve the performance of the artificial neural network. In some embodiments, the change from one group to another can include changing the scaling factor. Depending on the implementation details, applying piecewise quantization on a per-channel (or channel-by-channel) basis can improve the accuracy achieved per layer (or layer-by-layer).

[0098] Any or all of the methods and / or devices described in this disclosure may be deployed in any application where an artificial neural network can be used. As an example, an artificial neural network implementing piecewise quantization according to the present disclosure may be deployed in devices for mobile computing, Internet of Things (IoT), and / or embedded applications (including devices implementing piecewise quantization that can be configured to use image recognition). An artificial neural network implementing piecewise quantization according to the present disclosure may also be deployed in desktop / workstation applications, servers, data centers (including hyperscale data centers, etc.).

[0099] Figure 6 An embodiment of a computing system according to the present disclosure is shown. Figure 6 The system 150 may be used to implement any or all of the methods and / or devices described in this disclosure. The system 150 may include a central processing unit (CPU) 152, a user interface 154, a network interface 156, a memory 158, a storage device 160, a GPU 162, a field programmable gate array (FPGA) 157, and an application specific integrated circuit (ASIC) 163. However, the principles of this disclosure are not limited to the use of Figure 6 The implementation of any of the components shown in the present disclosure may be implemented using any suitable hardware, software, or combination thereof. In various embodiments, the system 150 may omit any of these components, or may include duplicates of any component or any additional number of any component, as well as any other types of components, to implement any of the methods and / or apparatus described in this disclosure.

[0100] CPU 152 may include any number of cores, caches, buses and / or interconnect interfaces, and / or controllers. Memory 158 may include any arrangement of dynamic RAM and / or static RAM, non-volatile memory (e.g., flash memory), and / or the like. Storage 160 may include a hard disk drive (HDD), a solid-state drive (SSD), and / or any other type of data storage device, or any combination thereof. User interface 154 may include any type of human interface device (such as a keyboard, mouse, display, video capture or transmission device, microphone, speaker, touch screen, etc.), as well as any virtual or remote versions of such devices. Network interface 156 may include one or more adapters or other devices for communicating via Ethernet, Wi-Fi, Bluetooth, or any other computer network arrangement to enable components to communicate over a physical and / or logical network (such as an intranet, the Internet, a local area network, a wide area network, etc.). GPU 162, FPGA 157, and ASIC 163 may include specialized hardware (such as programmable multiply-accumulate (MAC) hardware) that can be configured to efficiently implement any or all of the quantization and / or other neural network processing disclosed herein.

[0101] Any or all components of the system 150 may be interconnected via a system bus 164, which may collectively refer to various interfaces including a power bus, an address bus, and a data bus, high-speed interconnects such as Serial AT Attachment (SATA), Peripheral Component Interconnect (PCI), Peripheral Component Interconnect Express (PCI-e), a system management bus (SMB), and any other type of interface that enables components to work together locally at one location and / or distributed between different locations.

[0102] The system 150 may also include various chipsets, interfaces, adapters, glue logic, embedded controllers (such as programmable or non-programmable logic devices or arrays), ASICs, embedded computers, smart cards, etc., arranged to enable the various components of the system 150 to work together to implement any or all of the methods and / or devices described in this disclosure. Any component of the system 150 may be implemented in hardware, software, firmware, or any combination thereof. In some embodiments, any or all components may be implemented in a virtualized form and / or cloud-based implementation, for example, within a data center or distributed across multiple data centers to have flexible resource provisioning.

[0103] The components or steps of the methods or algorithms and functions described in conjunction with the embodiments disclosed herein may be implemented as hardware included in the system 150, as one or more software modules executed by a processor, or as a combination of the two. If implemented in software, the functions may be stored or transmitted as one or more instructions or codes on a tangible, non-transitory computer-readable medium. The software modules may reside in random access memory (RAM), flash memory, read-only memory (ROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disk, removable disk, CD-ROM, or any other form of storage medium. Any system disclosed herein or its component or portion may be implemented as part of a software stack of a larger system. Any system disclosed herein or its component or portion may be implemented as its own software stack.

[0104] The embodiments disclosed above have been described in the context of various implementation details, but the principles of the present disclosure are not limited to these or any other specific details. For example, some functions may be described as being implemented by specific components, but in other embodiments, the functions may be distributed between different systems and components in different locations and have various user interfaces. Specific embodiments have been described as having specific processes, steps, etc., but these terms also include embodiments in which specific processes, steps, etc. can be implemented using multiple processes, steps, etc., or embodiments in which multiple processes, steps, etc. can be integrated into a single process, step, etc. Reference to a component or element may refer to only a portion of the component or element.

[0105] Unless the context makes it clear otherwise, terms such as "first" and "second" used in this disclosure and claims may be used solely for the purpose of distinguishing the things they modify and may not indicate any spatial or temporal order. Reference to a first thing does not imply the existence of a second thing.

[0106] According to the inventive principles disclosed in this patent, additional embodiments can be generated by combining the various details and embodiments described above. Since the inventive principles disclosed in this patent can be modified in arrangement and detail without departing from the inventive concept, such changes and modifications are considered to fall within the scope of the claims.

Claims

1. A method for recognizing an image, the method comprising: Get the tensor of the artificial neural network; Dividing a quantization range of a tensor of an artificial neural network into a first region and a second region; quantizing the values of the tensor in the first region and the values of the tensor in the second region separately using separate computation paths; applying the value of the tensor in the quantized first region and the value of the tensor in the quantized second region to an artificial neural network to obtain a quantized artificial neural network; Obtaining image data to be recognized; The image data to be recognized is input to the quantized artificial neural network to execute the quantized artificial neural network to perform image recognition on the image data to be recognized.

2. The method according to claim 1, wherein The values of the tensor in the first region are quantized according to a first quantization function, and The values of the tensor in the second region are quantized according to a second quantization function.

3. The method according to claim 2, wherein: The first quantization function and the second quantization function have different scaling factors.

4. The method according to claim 3, wherein: The first quantization function and the second quantization function have different offsets.

5. The method according to claim 1, wherein Linear quantization is applied to the values of the tensor in the first region and the second region.

6. The method according to claim 1, wherein The first region includes a positive portion and a negative portion.

7. The method according to claim 1, wherein The second region includes a positive portion and a negative portion.

8. The method according to any one of claims 1 to 7, wherein: The step of dividing the quantization range includes: locating breakpoints of the first area and the second area.

9. The method according to claim 8, wherein The steps to locate a breakpoint include: determining a quantization error over at least a portion of a quantization range; and Minimize quantization error.

10. The method according to claim 9, wherein: The steps to minimize the quantization error include: Formulate the quantization error as a function of the position of the breakpoint; formulating the first derivative of the function; and Determine the value of the breakpoint that makes the first derivative zero.

11. The method according to claim 10, wherein: Use binary search to determine the value of the breakpoint that makes the first derivative zero.

12. The method according to claim 9, wherein Use regression to approximate the location of the breakpoints.

13. The method according to claim 9, wherein: A grid search is used to minimize the quantization error.

14. The method according to claim 8, wherein Position the breakpoints so that the density of the quantized values matches the density of the tensor's values.

15. The method according to any one of claims 1 to 7, wherein A tensor consists of at least two channels, and The at least two channels are individually quantized in at least one of the first region and the second region.

16. The method according to any one of claims 1 to 7, wherein The method further comprises: Dividing the quantization range of the tensor of the artificial neural network into a third region in addition to the first region and the second region; quantizing the value of the tensor separately in the first region, the second region, and the third region using separate computation paths; and applying the value of the tensor in the quantized first region, the value of the tensor in the quantized second region, and the value of the tensor in the quantized third region to an artificial neural network to obtain a quantized artificial neural network, The first breakpoint between the first area and the second area and the second breakpoint between the second area and the third area are located by the following steps: formulating a quantization error over at least a portion of a quantization range; and Minimize quantization error.

17. The method according to any one of claims 1 to 7, wherein: A tensor consists of one or more weights.

18. The method according to any one of claims 1 to 7, wherein The first region and the second region do not overlap.

19. The method according to any one of claims 1 to 7, further comprising: An artificial neural network is obtained by training an initial neural network using training images.

20. A system for recognizing an image, comprising: A processor is configured to execute a program, which, when executed, causes the processor to perform the method according to any one of claims 1 to 19.

21. An artificial neural network device comprising: a first computation path comprising a first accumulator configured to sum activation and weight products for a first region of a quantization range; a second computation path comprising a second accumulator configured to sum activation and weight products for a second region of the quantization range; and A third accumulator is configured to sum the activations of the second region.

Images