Quantization of weights in neural networks

Abstract

The description relates to quantization of weights in neural networks. A circuit is provided that is configured to perform multiplication and accumulation operations in an artificial neural network layer, the operations taking input data values and weights as inputs, and wherein the weights have only values within a finite set, the finite set is formed only by a value 0, a plurality of values equal to 2n (where n is an integer), and a plurality of values each equal to a product of 2n and an odd number greater than or equal to 3.

Description

[0001] Priority declaration

[0002] This application claims priority to European Patent Application No. 23305924.5 filed on June 9, 2023, French Patent Application No. 2309178 filed on September 1, 2023, and U.S. Patent Application No. 18 / 736,783 filed on June 7, 2024, the contents of which are incorporated herein by reference in their entirety to the maximum extent permitted by law. Technical Field

[0003] The present disclosure relates generally to the architecture of artificial neural networks, and more particularly to circuits or methods for performing multiplication and accumulation operations in neural networks. Background Art

[0004] The operations performed in each layer of an artificial neural network (especially in the convolutional layers) imply the use of hardware circuits configured to perform multiplication and accumulation operations (MAC). Such MAC circuits generally consist of a multiplier implemented by combinatorial logic followed by an adder.

[0005] However, MAC circuits are costly in terms of energy consumption and surface area. In particular, multipliers dominate the power consumption and surface area consumption of the processor on which the MAC circuit is implemented.

[0006] Rounding errors occur when parameters, such as the weights of neurons, are quantized to fixed-point values. It is desirable to represent the parameters in quantized values ​​that limit the rounding errors.

[0007] There is also a need to reduce the power consumption and surface area required for hardware implementations of MAC circuits. In particular, there is a need to provide a simplified implementation of a multiplier suitable for quantized value representations. Summary of the invention

[0008] An embodiment provides a circuit configured to perform a multiply and accumulate (MAC) operation in an artificial neural network layer, the MAC operation taking as input an input data value and a weight, wherein the weight has only values ​​within a finite set, the finite set consisting only of the values ​​0, 2, n Multiple values ​​of and each value is equal to 2 n A plurality of values ​​of the product of an odd number greater than or equal to 3 is formed, where n is an integer.

[0009] According to an embodiment, the circuit comprises: a first subcircuit configured to perform a multiplication of an input data value with the odd number, the first subcircuit comprising a first shift circuit and an adder configured to perform a multiplication with 2; and a second subcircuit configured to perform a multiplication with the value 2 n The second sub-circuit includes a second shift circuit configured to perform an n-bit shift.

[0010] According to an embodiment, the first subcircuit further comprises a first multiplexer configured to supply to the adder a value selected from among: an output of the first shift circuit; and a value 0, wherein the first multiplexer is configured to, if the weight and the value are equal to 2 n is equal to one of the multiple values ​​of the product of an odd number greater than or equal to 3, then the output of the first shift circuit is selected.

[0011] According to an embodiment, the circuit further comprises a second multiplexer configured to supply the first subcircuit with a value selected from among: the input data value; and the value 0, wherein the second multiplexer is configured to select the value 0 if the weight is equal to 0.

[0012] According to an embodiment, the circuit further comprises a third sub-circuit configured to selectively apply a negative sign to an output value of the first sub-circuit, the third sub-circuit comprising, for example, an inverter, an adder configured to add 1, and a third multiplexer configured to selectively bypass the inverter and the adder.

[0013] According to an embodiment, each weight is encoded on at least 4 bits, the at least 4 bits including: a bit encoding the sign of the weight; a bit indicating whether the value of the weight is equal to 2; n at least one bit of a value among the multiple values ​​of the product of an odd number greater than or equal to 3; and at least two bits encoding the value of n.

[0014] According to an embodiment, the second shift circuit is configured to perform a multiplication with a power of two based on the value of the at least two bits encoding the exponent of two.

[0015] According to an embodiment, the integer n is less than three.

[0016] According to an embodiment, equal to the plurality of values ​​2 n The plurality of values ​​of the product with an odd number greater than or equal to 3 include:

[0017] Equal to the multiple value 2 n and a first plurality of values ​​of the product of a first odd number; and a value equal to said plurality of values ​​2 n A second plurality of values ​​of the product of a second odd number different from the first odd number.

[0018] According to an embodiment, the odd number is the number 3.

[0019] According to an embodiment, the odd number is the number 3 and / or the number 5 and / or the number 7.

[0020] According to an embodiment, a weight value equal to 0 is encoded by a specific value, such as a binary sequence X111, where X is equal to 0 or 1.

[0021] According to an embodiment, the artificial neural network layer is a convolutional layer.

[0022] Another embodiment provides a method of generating weighted output values ​​of neurons of a neural network layer by a circuit, the method comprising multiplication of an input data value by a weight, wherein the weight has only values ​​within a finite set, the finite set consisting only of values ​​0, 2, n Multiple values ​​of and each value is equal to 2 n A plurality of values ​​of the product of an odd number greater than or equal to 3 is formed, where n is an integer.

[0023] Another embodiment provides a method of training an artificial neural network, comprising determining a weight of a neuron by only selecting a value equal to 0, equal to 2, or n multiple values ​​of (where n is an integer) and each value is equal to 2 n Each weight is selected from a finite set of values ​​that are products of an odd number greater than or equal to 3. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The foregoing features and advantages, as well as other features and advantages, will be described in detail in the remainder of the present disclosure with reference to specific embodiments given by way of illustration and not limitation, with reference to the accompanying drawings, in which:

[0025] Figure 1 illustrates an example of a convolutional neural network;

[0026] Figure 2 The hardware implementation of multiplication and accumulation operations is illustrated;

[0027] Figure 3 The diagram shows the operation of quantization on 8 bits;

[0028] Figure 4A is a histogram of weight values;

[0029] Figure 4B is a graph illustrating Gaussian quantization;

[0030] Figure 5 is a block diagram illustrating a multiplier;

[0031] Figure 6 illustrates the test for verification of the quantization operation;

[0032] Figure 7 is a table including the results of the validation test; and

[0033] Figure 8 is a graph showing various quantization modes for a VGG-type neural network model. DETAILED DESCRIPTION

[0034] Similar features are denoted by similar reference numerals in the various drawings. In particular, common structural and / or functional features between various embodiments may have the same reference numerals and may employ exactly the same structure, dimensions, and material properties.

[0035] For the sake of clarity, only the steps and elements useful for understanding the described embodiments are illustrated and described in detail. In particular, the operation and implementation of artificial neural networks are known to those skilled in the art and are therefore not described in detail.

[0036] Unless otherwise indicated, when referring to two elements being connected together, this means a direct connection without any intermediate elements except conductors, and when referring to two elements being coupled together, this means the two elements may be connected or they may be coupled via one or more other elements.

[0037] In the following description, when reference is made to terms defining absolute positions (such as terms "edge", "back", "top", "bottom", "left", "right", etc.) or relative positions (such as terms "above", "below", "higher", "lower", etc.) or terms defining directions (such as terms "horizontal", "vertical", etc.), unless otherwise specified, this refers to the orientation of the drawing.

[0038] Unless otherwise specified, the expressions "about", "approximately", "substantially" and "around" mean plus or minus 10%, preferably plus or minus 5%.

[0039] Figure 1 An example of a convolutional neural network 100 is illustrated. As an example, the neural network 100 shown is configured to perform image recognition. More specifically, based on an input image 102 (e.g., shown as a dog), the neural network 100 is configured to estimate the probability of belonging to one or more object classes. Thus, the neural network 100 is configured to, for example, generate an output data element 104 (e.g., a vector) indicating the probability that the input image 102 belongs to a class or category (such as, for example, "dog", "cat", "lion" and "bird"). Of course, other examples of image processing and other categories are also possible. Similarly, in other examples, the neural network 100 is configured to generate an output vector 104 that includes the probability that the input image 102 belongs to more than tens or hundreds of classes. The example of a network configured to perform image processing is of course not limiting. In general, the neural network 100 includes at least one convolutional layer.

[0040] As an example, an input image 102 is processed sequentially by a series of layers of a neural network 100. As an example, the first layer of the network 100 is a convolutional layer 106, configured to detect features in the input image 102, such as, for example, contours, shapes, colors, and / or textures. For example, layer 106 is followed by a pooling layer 108, which is configured to reduce the dimensionality of the image and thus enable a reduction in the number of parameters and calculations performed in the network 100. For example, layer 108 is followed by a new convolutional layer 110. For example, layer 110 is configured to detect features in the input image 102, such as, for example, contours, shapes, colors, and / or textures. As an example, the features detected by layer 110 are finer features than the features detected by layer 106. For example, layer 106 would be configured to detect colors and / or contours, while layer 110 would be configured to detect patterns and / or textures in an image. As an example, layer 110 is followed by a new pooling layer 112 , which is itself coupled to a fully connected layer 114 , which is configured to combine features extracted from previous layers to generate an output vector 104 .

[0041] Each of the layers 106, 108, 110, 112 and 114 is associated with one or more parameters, such as, for example, a weight associated with each neuron of the layer. The value of each parameter, in particular each weight, is a value learned during the training or learning of the network 100. Once the values ​​of the parameters are set, i.e. when the training of the network 100 is considered to be finished, the network 100 is used to perform an inference operation via a series of layers 106 to 114. In particular, during the execution of the network 100, operations of the multiplication and accumulation (MAC) type are performed. For example, a MAC operation corresponds to summing, over a plurality of neurons of a layer, the products between the value supplied to a neuron and, for example, the weight associated with this neuron. In other words, performing a MAC operation on N neurons (N being an integer greater than or equal to 1) corresponds to performing the following operations: where w i is the weight associated with the neuron of rank i, and x i is the input value supplied to the neuron of rank i.

[0042] Figure 2 An example of a hardware implementation 200 of a multiply and accumulate operation is illustrated.

[0043] As an example, an implementation or circuit 200 includes a multiplier 202 (×), which is configured to receive two input values ​​and generate an output value wx corresponding to the product of the two input values. As an example, the two input values ​​supplied to the multiplier correspond to the input value x toward the neuron and the weight w associated with this neuron, respectively. The circuit 200 also includes an adder 204 (+). The adder 204 is, for example, coupled to the multiplier 202, and the multiplier 202 is configured to deliver the generated output value to the adder 204. The adder 204 is, for example, also coupled to a memory 206 (ACCU). As an example, the memory 206 is a register. As an example, the adder 204 is also configured to read the value stored in the memory 206 and generate the output value by adding it to the value supplied by the multiplier 202. Then, the adder 204 is, for example, configured to store the generated value in the memory 206. As an example, the storage of the generated value is performed as a result of suppressing the value that has been previously read by the adder 204.

[0044] The values ​​manipulated by a MAC circuit, such as circuit 200, are values ​​that are quantized into a set of values. The quantized values ​​are, for example, fixed-point values.

[0045] Figure 3 The diagram illustrates the quantization operation of the weights of network 100 on 8 bits.

[0046] As an example, the training of the network 100 is performed by a device external to the circuit on which it will be implemented. As an example, the external device is a device with high computing power. After training, the weights are stored, for example, in a memory 300 (cloud AI) external to the circuit on which the network 100 will be implemented. As an example, the memory 300 is a server. As an example, the weights stored in the memory 300 are floating point numbers and are encoded, for example, on 16 or 32 bits.

[0047] As an example, the learned weight values ​​are quantized or transformed into quantized values ​​of integer values ​​or fixed-point values ​​before being stored in the memory of the device to be implemented in the network 100. As an example, the quantization is performed by a compiler for porting to the peripheral circuit 302 (edge ​​AI). As an example, the peripheral circuit 302 is configured to perform operations according to the network edge computing method.

[0048] As an example, the circuit 302 is configured to transform the weight values ​​transmitted by the memory 300 into integer values, and for example on a number less than 16 bits, for example on 8 bits or 4 bits.

[0049] As an example, the circuit 302 is configured to quantize the weight values ​​on 8 bits by performing a linear operation. As an example, for a floating point value r (e.g., encoded on 32 bits or on 16 bits), an integer value q (e.g., encoded on 8 bits) is such that r=S.(qZ), where the value S is a real positive constant representing a scaling factor and the value Z is a zero offset corresponding to the value of the quantized value 0. The linear operation described in this way is known to those skilled in the art and is described, for example, in Part 2 of the publication “Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference” by B. Jacob et al. (incorporated herein by reference). In this example, and with Figure 2 In relation, the input data supplied to the multiplier 202 is data encoded on 8 bits, for example. Then, the multiplier 202 is configured, for example, to generate a value encoded on 16 bits based on the two input values ​​encoded on 8 bits.

[0050] Figure 4A is a histogram 400 of weight values. In particular, the histogram 400 illustrates the values ​​belonging to the set of integer values ​​Q 3 Gaussian distribution of values, where Q 3 : = {0}∪ n∈N {±2 n}∪ n∈N ±3.2 n}. In other words, the set Q 3 Only includes, for example, the value 0, integers equal to powers of 2 (such as, for example, the values ​​-4, -2, -1, 1, 2, 4, etc.), and integers equal to powers of 2 times 3 (such as, for example, -6, -3, 3, 6, etc.). In another example, the value 0 does not belong to the set Q. 3 .

[0051] According to an embodiment, the circuit 302 is configured, for example, to quantize the values ​​of the weights stored in the memory 300 as belonging to the set Q 3 According to an embodiment, for a floating point value r, by applying a The operation is used to perform the so-called simple quantization for the set Q 3 The value q of , where sign(r) = 1 if r is positive and sign(r) = -1 if r is negative, and where, for any floating-point value x, It belongs to Q 3 The value of the absolute value of the closest value x. As an example, for r = -5.3, And q=-6.

[0052] According to an embodiment, each circuit on which the network 100 is implemented is configured to manipulate and store data belonging to the set Q 3 The quantified value of .

[0053] According to other embodiments, the set Q 3 is replaced or supplemented by one or more other sets that include only values ​​that are equal to the product of a power of 2 and an odd number different from 3 (e.g., 5 and / or 7, etc.). As an example, the set Q 3 Q 3 ∪Q 5 Replace, or Q 5 : = {0}∪ n∈N {±2 n}∪ n∈N ±5.2 n} and in the set Q 3 ∪Q 5 In another example, only in the set Q 5 quantization is performed in . In general, only 2p+1 Quantization is performed on a set or combination of sets, where Q 2p+1 : = {0}∪ n∈N {±2 n}∪ n∈N {±(2p+1).2 n}, p is a positive integer. However, the most advantageous hardware combination for the implementation of the MAC circuit is a value that is the sum of two powers of two.

[0054] Figure 4B is a diagram as about Figure 4A A graph depicting the Gaussian quantization alternative to quantization.

[0055] As an example, for Q 3 The quantization of the values ​​in is performed by a so-called Gaussian quantization operation. As an example, Q 3 The quantization in is performed on a number of bits p, for example 4 bits. In other words, the quantized value is encoded on a number of bits p. To discretize the weight values ​​onto p bits, assume that they follow the variance * 2 The standard normal distribution of . As an example, the variance * 2 The value of is obtained from the statistics of the sample of weight values. The number is defined as 2 p -1 interval I k (k∈{1,...,2 p -1}) makes the distribution N(0,σ 2 The probability that a random variable with a probability of 2 is uniform is equal to 1 / (2 p -1). Interval I 1 The form is I1 =]-∞, b 1 ], interval The form is Interval I k (k∈{2,…,2 p -2}) is in the form [b k-1 ,b k [. Endpoint b k It depends on p and * 2 The density of the standard normal distribution is symmetric and 2 p -1 is an odd number, for any k∈{1,...,2 p -1}, The value 0 is the quantized value. k (k∈{2,…,2 p -2}), value l k is defined such that:

[0056]

[0057] where erf is the error function. As an example, for p=4 and * 2 =1, rounded to the nearest hundredth of l k ={-1.83,-1.28,-0.97,-0.73,-0.53,-0.34,-0.17,0,0.17,0.34,0.53,0.73,0.97,1.28,1.83}. To avoid manipulating floating point values, we have to k (k∈{1,2 p -1}) is linearly transformed into a value Among them A G ∈R is an approximation factor and where is a set of quantized values, such as Q = Q 3 For example, we can use linear regression Get value A G .

[0058] Figure 5 is a block diagram illustrating a multiplication and accumulation circuit 500 according to an embodiment of the present disclosure.

[0059] According to the embodiment, belonging to Q 3 The value of is encoded on a maximum of 4 bits. One bit (e.g., the most significant bit) is a sign bit, and another bit (e.g., the second most significant bit) indicates whether the value is a multiple of, for example, 3. Finally, the remaining two bits (e.g., the two least significant bits) encode the value of the exponent n. As an example, if the value is an odd number, then n=0, and if the value is equal to ±2 or ±6, etc., then n=1. The exponent n is restricted to 2 bits, so the set Q 3is limited to the values ​​{-12, -8, -6, -4, -3, -2, -1, 0, 1, 2, 3, 4, 6, 8, 12}. Thus, for example, the value 0000 on 4 bits encodes, for example, the value 1, the value 1101 on 4 bits encodes, for example, -6, and so on. Of course, this example of encoding on 4 bits is disclosed as an example, and other forms of encoding can certainly be envisioned. In another example, the least significant bit is a sign bit, and the two most significant bits encode the value of the exponent n. As an example, the value X111 on 4 bits encodes the value 0, where X is a bit of value 0 or 1.

[0060] Generally speaking, in the form of Q 2p+1 In the case where quantization is performed on a set of , the second most significant bit indicates, for example, whether the value is a multiple of 2p+1.

[0061] The value 0000 then encodes a value different from 0, and where one skilled in the art expects that the weights of the network 100 may be zero, they will be able to specifically encode a weight value of 0.

[0062] according to Figure 5 In the embodiment shown in , MAC circuit 500 includes multiplier circuit 502. Multiplier circuit 502 is configured to receive an activation value, which corresponds to an input value, for example, transmitted to a neuron of network 100. As an example, the input value is supplied to multiplier circuit 502 via multiplexer 504. As an example, the activation value supplied to multiplexer 504 is a value on 8 bits. As an example, multiplexer 504 is configured to supply activation values ​​to circuit 502, which activation values ​​are either equal to value 0 or equal to a non-zero value. As an example, when the activation is encoded with a shifted zero value, multiplexer 504 exists. As an example, when the activation is not encoded with a shifted zero value, multiplexer 504 is omitted. In this case, the activation is supplied as an input to multiplexer 502.

[0063] In particular, the multiplexer 504 is coupled to the multiplexer 506 of the circuit 502. The multiplexer 506 is configured to select the value 0 or the activation value supplied by the multiplexer 504 based on the value of three bits indicating whether the weight value is a multiple of 3 (M) and indicating the value of the exponent n (EE). As an example, if at least one of the three bits is different from 1 (≠111), the multiplexer 506 is configured to: select the activation value, corresponding to the case where the weight value is different from the value 0; and, select the value 0 in the case where the three bits are encoded as the value 1. Then, the value selected by the multiplexer 506 is, for example, a value on 8 bits and is transmitted to the adder 508. The value selected by the multiplexer 506 is concurrently supplied to the shift circuit 510 (<<1). Then, the shift circuit 510 generates a value encoded, for example, on 9 bits, which is the value supplied by the multiplexer 506 shifted one bit to the left, thus corresponding to its multiplication by 2. The circuit 510 is also coupled to a multiplexer 512, which is configured to select the output of the circuit 510 or the value 0 based on the value of the bit (M) indicating whether the value of the weight is a multiple of 3. As an example, if the weight value is not a multiple of 3, the multiplexer 512 is configured to select the value 0, and if the weight value is a multiple of 3, the multiplexer 512 is configured to select the output value of the circuit 510. The selected value is then transmitted to the adder 508. The adder 508 is then configured to add the output values ​​of the multiplexers 506 and 512. As an example, the circuit 510 includes a ground line and is configured to add the bit "0" as the least significant bit to the input value. The shift circuit 510 is, for example, a sign component.

[0064] Therefore, in the case where the weight value is a multiple of 3, the output value of the adder 508 corresponds to three times the input value. In the case where the weight value is not a multiple of 3, if the weight value is not 0, the output value of the adder 508 corresponds to the activation value. As an example, the value generated by the adder 508 is a value on 10 bits.

[0065] The value generated by the adder 508 is then supplied to the multiplexer 514 and in parallel to the inverter 516. Therefore, the inverter 516 is configured to switch the bit having the value 1 to the value 0, inverting the value supplied from the adder 508. The inverted value is then supplied to the adder 518. The adder 518 is an adder that adds 1, and is configured to add 1 to the output value of the inverter 516. Therefore, the inverter 516 and the adder 518 are configured to generate a value opposite to the output value of the adder 508 according to the 2's complement method. Therefore, the output value of the adder 518 corresponds to a value that generates a series of 0s when it is added to the output value of the adder 508. In other words, the output value of the adder 518 corresponds to the output value of the adder 508 multiplied by -1.

[0066] Then, the multiplexer 514 is configured to select the output value of the adder 508 or the adder 518 based on the value of the bit (S) indicating whether the weight value is positive or negative. For example, if the weight value is positive, the multiplexer 514 selects the output value of the adder 508, and if the weight value is negative, the multiplexer 514 selects the output value of the adder 518.

[0067] Multiplexer 514 is then configured to supply the selected value (e.g., at bit 11) to a shift circuit or shifter 520. Shift circuit 520 is then configured to shift the received value to the left based on the value of the two bits (EE) indicating the exponent. As an example, if the two bits are equal to 0, then circuit 520 simply restores the output value of multiplexer 514 without shifting it. In general, circuit 520 is configured to shift the supplied value by a number of bits equal to the value of n. The value restored by circuit 520 is then compared with the output value of multiplexer 514 and 2 n The product of corresponds to As an example, the output value of the circuit 520 is a value on 21 bits.

[0068] The multiplier 502 is also coupled to an adder 522, which is configured to read the output value of the circuit 520. The adder 522 is also configured to add the output value of the circuit 520 to a value stored in a memory 524 (ACCU). The adder 522 is also configured to transmit the generated value to the memory 524. As an example, the memory 524 is similar to the memory 206. As an example, the value manipulated by the adder 522 is encoded on 21 bits.

[0069] The number of bits manipulated by the elements of multiplier 502 and adder 522 is given as an example only. For example, in the case where the exponent n is encoded on a number of bits greater than 2, a person skilled in the art will be able to adjust the number of bits manipulated by the different elements of circuit 500 to the number of bits used to encode the weight values ​​and the shifts performed by circuit 520.

[0070] The weight value belongs to type Q 2p+1 (p≥2) or belongs to type Q 2p+1 In the case of a combination of multiple sets of (p ≥ 1), a person skilled in the art will be able to adjust the portion of circuit 502 including elements 506, 510, 512 and 508 to replace them, or add other elements to perform multiplication with the desired odd number. As an example, if the set under consideration has the form Q 3 ∪Q 5, then another multiplexer can be added to select whether the weight value is a multiple of 5, and this other multiplexer can be coupled to an adder and a shift circuit that is configured to shift the received value to the left by at least one bit. The number of bits implicit in the shift depends on the number of bits used to encode the exponent value of 2. Depending on the position of this other multiplexer, the circuit can be adjusted to the set Q G , without adding another additional hardware component. In this case, the set Q G will correspond to (1xA+2xA)+4xA, where the first multiplexer will select whether to add the value 2xA in the same manner as multiplexer 512, and the second multiplexer will select whether to add the value 4xA. In general, the types Q included in the set under consideration 2p+1 The more combinations of sets, the closer the representation is to a floating point description. In general, the number of adders included in circuit portion 502 is equal to the number of "1"s in the binary representation of the odd number minus one. For example, the binary representations of the odd numbers 3 and 5 are "011" and "101", which include two values ​​of "1", so the adjustment is Q 3 Or adjust to Q 5 The portion 502 includes an adder. For example, the binary representation of an odd number is "111", which includes three values ​​of "1", so Q is adjusted. G The portion 502 includes two adders.

[0071] Figure 6 The diagram shows the set Q 3 Tests for quantitative validation.

[0072] As an example, the validation test shown is performed as a result of learning the weights of the convolutional layers of the network, such as with respect to Figure 1 The neural network 100 described in is similar. As an example, the network is a convolutional neural network (CNN) type network.

[0073] Histogram 600 illustrates the distribution of weights of the second layer of the network obtained during training (full distribution). More specifically, the weights shown in histogram 600 are floating point values. The weight distribution then forms a Gaussian curve.

[0074] Histogram 602 illustrates the distribution of quantized weights. As an example, the quantized weights are the weights of the second layer of the network obtained during training and then quantized to the set Q 3 As an example, based on the floating point weight r, the quantized weight q is such that Therefore, Q 3 The distribution of the quantized weights in forms a Gaussian curve. In addition, it can be observed that all quantized weights belong to the interval [-12,12] and can therefore be encoded on 4 bits, as shown in Figure 5described.

[0075] Figure 7 Tables 700 and 702 are illustrated, each including the results of a verification test. In particular, Tables 700 and 702 include the results for linear quantization (Linear) and Q quantization to 4 bits. 3 (Simple Q 3 ) power consumption (power (μW)), surface area (area (μm 2 )) and the cost comparison in terms of network accuracy (accuracy). In particular, Table 700 illustrates, for example, Figure 3 Describes the linear quantization on 8 bits (Linear) and quantization to 4 bits Q 3 Table 702 illustrates the comparison between linear quantization on 4 bits and quantization to 4 bits Q 3 For example, the comparison is performed for a convolutional layer of a neural network similar to network 100 and having data for activating the convolutional layer encoded therein by linear quantization on 8 bits. The weights of the neurons of the layer are then quantized and based on floating point values ​​obtained, for example, during training of the network.

[0076] As an example, linear quantization on 8 bits can quantize floating point values ​​to, for example, belong to the interval Linear quantization on 4 bits can quantize floating point values ​​into integers belonging to the interval More specifically, linear quantization on 8 bits includes 256 representations of the value of the weight, while linear quantization on 4 bits includes 16 representations. Figure 5 described, quantized to 4 bits on Q 3 There are 15 representations, including the value 0. The gain column (Gain) in each of Tables 700 and 702 includes the values ​​quantized to Q by linear quantization. 3 The gains in power consumption, surface area, and accuracy provided. As an example, by quantizing to 4 bits on Q 3 The surface area used is based on Figure 5 The described implementation of the MAC circuit 500 is calculated.

[0077] As an example, for the tested 40nm technology, linear quantization on 8 and 4 bits requires 28.1μW and 20.6μW of power consumption respectively, while quantization to Q 3 For example, 17.9 μW is required, which corresponds to a gain of 36% in the first case and 13.1% in the second case. Similarly, linear quantization on 8 and 4 bits requires 566.3 μm, respectively. 2 and 369 μm 2 The surface area is quantified to Q 3 For example, 358μm is required 2, which corresponds to a gain of 37% in the first case and 3% in the second.

[0078] For example, the accuracy in the case of linear quantization on 8 bits is 99.06%, and the accuracy in the case of linear quantization on 4 bits is 99.05%. 3 The accuracy is 99.37%, ie, a gain of 0.31% over linear quantization on 8 bits and a gain of 0.32% over linear quantization on 4 bits.

[0079] Although quantized to Q 3 Quantization on 4 bits allows encoding the weight values ​​as a set of 14 values ​​(or 16 values ​​if the values ​​-24 and 24 are included, or 15 values ​​if the value 0 is included), but the latter are more scattered for a more sensible and clever coverage of the Gaussian distribution (which leads to higher accuracy). In fact, the 16 values ​​described by the linear representation on 4 bits are too few and equidistant from each other, which is not ideally suitable for describing values ​​distributed according to a normal distribution. In addition, the 256 values ​​described by the linear representation on 8 bits, although enabling a faithful description of the Gaussian distribution, are too many. In fact, the tail values ​​(i.e., the high and low values) are rarely used, considering that they actually increase the surface area and power consumption, but do not provide a significant accuracy gain.

[0080] Figure 8 800 is a graph showing various quantization modes for a neural network model of the Visual Geometry Group (VGG) type. This model is characterized by successive blocks formed by two to three convolutional layers followed by a layer called a max pooling layer. Each convolutional layer includes a 3×3 filter, and the max pooling layer has a size of 2×2. As an example, the VGG16 model has a depth of 16 layers and the VGG19 model has a depth of 19 layers. In particular, the graph 800 includes a plurality of points, each of which is associated with a quantization model. The abscissa of each point represents a power ratio, and the ordinate of each point represents the accuracy obtained by the VGG network. The power ratio is defined with respect to the power consumption of a linear quantization with 8 weight bits and 8 activation bits. As an example, a ratio of 65% means that the tested quantization consumes 35% less energy than a linear quantization on 8 bits. As an example, the parameters of the VGG network are obtained by training and are essentially floating point values ​​(floating point). For example, the different quantizations shown are performed on the weights of the convolutional layers of the network. As an example, a first training of the network is performed using floating point values, and then a second training is performed based on the weights obtained during the first training to simulate quantization. Those skilled in the art refer to this technique as quantization aware training (QAT). As an example, for each point (between 10 and 20, e.g. 15), training of the network is performed and the maximum accuracy is recovered.

[0081] As an example, points 801 (Linear_a8_w8), 802 (Linear_a8_w4), 804 (Linear_a6_w8), and 806 (Linear_a6_w4) represent the power ratio and accuracy of the VGG network for linear quantization. In particular, point 801 is associated with linear quantization on 8 bits, and the activation point is also on 8 bits. Point 802 is associated with linear quantization on 4 bits, and the activation point is also on 8 bits. The accuracy provided by these two quantizations is similar, i.e., approximately 99.05%, but quantization on 4 bits is less expensive in terms of power consumption. Points 804 and 806 are associated with linear quantization on 8 bits and 4 bits, respectively, and the activation point is also on 6 bits.

[0082] Points 808 (Log2_a8_w4), 810 (Log2_a6_w4), and 812 (Log2_a4_w4) are associated with logarithmic quantization on 4 bits, and the activation points are on 8, 6, and 4 bits, respectively.

[0083] Point 814 (Simple_Q 3 _a8_w4)、816(Simple_Q 3 _a6_w4) and 818 (Simple_Q 3 _a4_w4) and Q on 4 3 In simple quantification, such as Figure 4A As described, the activation points are at positions 8, 6 and 4 respectively.

[0084] Point 820 (Gauss_Q 3 _a8_w4)、822(Gauss_Q 3 _a6_w4) and 824 (Gauss_Q 3 _a4_w4) is associated with Gaussian quantization, such as Figure 4B Q on the 4-bit described 3 , and the activation points are at positions 8, 6, and 4 respectively.

[0085] Graph 800 shows that, for the tests performed, linear quantization results in worse accuracy than retaining floating point values. 3 , especially with activation points on 8 and / or 6 bits, gives better accuracy than when keeping floating point values.

[0086] An advantage of embodiments is that they enable implementation of MAC circuits suitable for operations performed by one or more layers of a neural network and having a reduced surface area.

[0087] Various embodiments and variations have been described. It will be appreciated by those skilled in the art that certain features of these various embodiments and variations may be combined, and that other variations will occur to those skilled in the art. In particular, the set on which quantization is performed may vary. In fact, although the set Q on 4 bits is primarily described, 3 , but those skilled in the art will be able to adapt the implementation of the MAC circuit to Q 2p+1 Type collection, or adjust to multiple Q 2p+1 Similarly, when encoding values ​​on more than 4 bits, one skilled in the art will be able to adjust Q 3 The encoding of the value and its processing by the MAC circuit.

[0088] Finally, based on the functional indications given above, the practical implementation of the described embodiments and variants is within the capabilities of a person skilled in the art. In particular, a person skilled in the art will be able to adapt the implementation of circuit 502 to perform multiplications with one or more odd numbers different from 3.

Claims

1. A circuit comprising: an artificial neural network layer, configured to perform multiplication and accumulation (MAC) operations; Wherein the MAC operation receives the input data value and the weight as input; as well as where the weights have values ​​within a finite set consisting only of the values ​​0, 2, n Multiple values ​​of, and each value is equal to 2 n A plurality of values ​​of the product of an odd number greater than or equal to 3 is formed, where n is an integer.

2. The circuit according to claim 1, comprising: a first subcircuit configured to perform a multiplication of the input data value with the odd number, the first subcircuit comprising a first shift circuit configured to perform a multiplication with 2 and an adder; as well as The second subcircuit is configured to perform AND with the value 2 n The second sub-circuit includes a second shift circuit configured to perform an n-bit shift.

3. The circuit of claim 2, wherein the first subcircuit further comprises a first multiplexer configured to supply the adder with a value selected from among: an output of the first shift circuit; and a value of 0; and The first multiplexer is configured so that if the weight is equal to 2 n is equal to one of the multiple values ​​of the product of an odd number greater than or equal to 3, then the output of the first shift circuit is selected.

4. The circuit of claim 2, further comprising a second multiplexer configured to supply the first subcircuit with a value selected from among: an input data value; and a value of 0; and Wherein the second multiplexer is configured to select the value 0 if the weight is equal to 0.

5. The circuit of claim 2 , further comprising a third subcircuit configured to selectively apply a negative sign to the output value of the first subcircuit, the third subcircuit comprising an inverter, an adder configured to add 1, and a third multiplexer configured to selectively bypass the inverter and the adder.

6. The circuit of claim 2, wherein each weight is encoded on at least 4 bits, the at least 4 bits comprising: a single bit encoding the sign of the weight; Indicates whether the value of weight is equal to 2 n at least one bit of a value among the plurality of values ​​of a product of an odd number greater than or equal to 3; as well as At least two bits that encode the value of n. 7 . The circuit of claim 6 , wherein the second shift circuit is configured to perform a multiplication with a power of 2 based on the at least two-bit value encoding an exponent of 2. 8 . The circuit of claim 1 , wherein the integer n is less than 3.

9. The circuit of claim 1, wherein the plurality of values ​​2 n The plurality of values ​​of the product with an odd number greater than or equal to 3 include: Equal to the multiple value 2 n a first plurality of values ​​of the product with the first odd number; as well as Equal to the multiple value 2 n A second plurality of values ​​of the product of a second odd number different from the first odd number.

10. The circuit of claim 1, wherein the odd number is the number 3.

11. The circuit of claim 1, wherein the odd number is selected from the group consisting of: 3, 5, and 7.

12. The circuit of claim 1, wherein a weight value equal to 0 is encoded by a specific value comprising a binary sequence X111, where X is equal to 0 or 1.

13. The circuit of claim 1, wherein the artificial neural network layer is a convolutional layer.

14. A method for generating weighted output values ​​of neurons of a neural network layer by a circuit, the method comprising: The multiplication of the input data value and the weight, where the weight has a value within a finite set consisting only of the values ​​0, 2, n Multiple values ​​of, and each value is equal to 2 n A plurality of values ​​of the product of an odd number greater than or equal to 3 is formed, where n is an integer.

15. The method of claim 14, wherein the multiplication comprises: a first multiplication of the input data value and the odd number, the first multiplication comprising a first shift and add configured to perform a multiplication by 2; as well as With value 2 n The second multiplication includes a second shift of n bits.

16. The method of claim 15, wherein the first multiplication supplies the addition with a value selected from among: the output of the first shift; and the value 0; and Also includes when the weight is equal to 2 n When one of the plurality of values ​​of the multiplication with an odd number greater than or equal to 3 is equal, the output of the first shift is selected.

17. The method of claim 15, further comprising selecting for the first multiplication a value selected from among: an input data value; and a value of 0; and It also includes selecting a value of 0 when the weight is equal to 0.

18. The method of claim 15, further comprising selectively applying a negative sign to an output value of the first multiplication by performing a 2's complement operation.

19. A method for training an artificial neural network, comprising: Determine the weights of neurons; The determination is performed by the following operation: in the case of only the value 0, equal to 2 n Multiple values ​​of, and each value is equal to 2 n Each weight is selected from a finite set of multiple values ​​formed by products of odd numbers greater than or equal to 3, where n is an integer.

20. The method of claim 19, wherein each weight is encoded on at least 4 bits, the at least 4 bits comprising: a single bit encoding the sign of the weight; Indicates whether the value of weight is equal to 2 n at least one bit of a value among the plurality of values ​​of a product of an odd number greater than or equal to 3; as well as At least two bits that encode the value of n.

21. The method according to claim 20, Which is equal to the multiple value 2 n The plurality of values ​​of the product with an odd number greater than or equal to 3 include: Equal to the multiple value 2 n a first plurality of values ​​of the product with the first odd number; as well as Equal to the multiple value 2 n A second plurality of values ​​of the product of a second odd number different from the first odd number.

22. The method of claim 20, wherein the odd number is the number 3.

23. The method of claim 20, wherein the odd number is selected from the group consisting of: 3, 5, and 7.

24. The method of claim 20, wherein a weight value equal to 0 is encoded by a specific value comprising a binary sequence X111, where X is equal to 0 or 1.

Images