Weight quantification in a neural network

Abstract

Weight Quantification in a Neural Network This description concerns a circuit (500) configured to perform a multiplication and accumulation operation in a layer of an artificial neural network. The operation takes as input an input data value and a weight, and the weight is uniquely represented in a limited set consisting only of the value 0, a plurality of values ​​equal to , where is an integer, and a plurality of values, each equal to the product of by an odd number greater than or equal to 3. Figure for the abstract: Fig. 5

Description

Title of the invention: Weight quantization in a neural network Technical field

[0001] The present description relates generally to the architecture of artificial neural networks, and in particular to a circuit or method for performing multiplication and accumulation operations in a neural network. Prior art

[0002] The operations performed in each layer of artificial neural networks, and in particular in the convolution layers, involve the use of a hardware circuit configured to perform the multiplication and accumulation (MAC) operations. Such MAC circuits generally comprise a multiplier, implemented by combinational logic, followed by an adder.

[0003] Nevertheless, MAC circuits have a high power and area consumption cost. In particular, the multiplier plays an important role in the power and area consumption of the processor on which the MAC circuit is implemented.

[0004] When parameters, such as neuron weights, are quantized to fixed-point values, rounding errors occur. There is a need to provide a representation of the parameters in quantized values ​​that can limit rounding errors.

[0005] There is a need to reduce the power consumption, and the area, required for the hardware implementation of MAC circuits. In particular, there is a need to provide a simplified implementation of a multiplier suitable for representation in quantized values. Summary of the invention

[0006] One embodiment provides a circuit configured to perform a multiply and accumulate operation in a layer of an artificial neural network, the operation taking, as input, an input data value and a weight, and wherein the weight is only a value in a limited set consisting of only the value 0, a plurality of values ​​equal to 2”, where n is an integer, and a plurality of values, each equal to the product of 2” and an odd number greater than or equal to 3.

[0007] According to one embodiment, the circuit comprises: - a first sub-circuit configured to perform a multiplication of an input data value by said odd number, the first sub-circuit comprising a first shift circuit configured to perform a multiplication by 2 and an ad- editor; and - a second subcircuit configured to perform a multiplication by the value 2”, the second subcircuit comprising a second shift circuit configured to perform a shift of n bits.

[0008] According to one embodiment, the first subcircuit further comprises a first multiplexer configured to provide the adder with a value selected from: an output of the first shift circuit; and the value 0, wherein the first multiplexer is configured to select the output of the first shift circuit if the weight is equal to one of said plurality of values ​​equal to the multiplication of 2” by an odd number greater than or equal to 3.

[0009] According to one embodiment, the circuit further comprises a second multiplexer configured to provide the first subcircuit with a value selected from: the input data value; and the value 0, wherein the second multiplexer is configured to select the value 0 if the weight is equal to 0.

[0010] According to one embodiment, the circuit further comprises a third subcircuit configured to selectively apply a negative sign to an output value of the first subcircuit, the third subcircuit comprising, for example, an inverter, an adder configured to add 1, and a third multiplexer configured to selectively bypass the inverter and the adder.

[0011] According to one embodiment, each weight is encoded on at least 4 bits, the at least 4 bits comprising: - a bit encoding the sign of the weight; - at least one bit indicating whether the weight has as its value one of the plurality of values ​​equal to the product of 2” by an odd number greater than or equal to 3; and - at least two bits encoding the value of n.

[0012] According to one embodiment, the second shift circuit is configured to perform a multiplication by a power of 2, based on the value of the at least two bits encoding an exponent of 2.

[0013] According to one embodiment, the integer n is less than 3.

[0014] According to one embodiment, said plurality of values ​​equal to the product of the plurality of values ​​2" by an odd number greater than or equal to 3 includes: - a first plurality of values ​​equal to the product of the plurality of values ​​2” by a first odd number; and - a second plurality of values ​​equal to the product of the plurality of values ​​2W by a second odd number, different from the first odd number.

[0015] According to one embodiment, the odd number is the number 3.

[0016] According to one embodiment, the odd number is the number 3 and / or the number 5 and / or the number 7.

[0017] According to one embodiment, the weight value equal to 0 is encoded by a specific value, for example the binary sequence XI11, where X is equal to 0 or 1.

[0018] According to one embodiment, the layer is a convolution layer.

[0019] Another embodiment provides a method of generating, by a circuit, a A weighted output value of a neuron of a layer of a neural network, the method comprising multiplying an input data value by a weight, wherein the weight is only a value in a limited set consisting of only the value 0, a plurality of values ​​equal to 2”, where n is an integer, and a plurality of values, each equal to the product of 2W and an odd number greater than or equal to 3.

[0020] Another embodiment provides a method of training an artificial neural network, comprising determining weights of neurons, the determination being performed by selecting each weight only from a limited set consisting of only the value 0, a plurality of values ​​equal to 2”, where n is an integer, and a plurality of values, each equal to the product of 2” and an odd number greater than or equal to 3. Brief description of the drawings

[0021] These characteristics and advantages, as well as others, will be explained in detail in the following description of particular embodiments given without limitation in relation to the attached figures among which:

[0022] [Fig.l] illustrates an example of a convolutional neural network;

[0023] [Fig.2] illustrates the hardware implementation of a multiplication operation and ac accumulation;

[0024] [Fig.3] illustrates an 8-bit quantization operation;

[0025] [Fig.4A] is a histogram of weight values, according to one embodiment of this description;

[0026] [Fig.4B] is a graph illustrating a Gaussian quantization, according to an embodiment of the present description;

[0027] [Fig.5] is a block diagram illustrating a multiplier, according to an embodiment of the present description;

[0028] [Fig.6] illustrates a validation test of the quantification operation, according to an embodiment of the present description;

[0029] [Fig.7] is a table comprising the results of the validation tests; and

[0030] [Fig.8] is a graph representing several quantization modes for a VGG type neural network model. Description of the embodiments

[0031] The same elements have been designated by the same references in the different figures. In particular, the structural and / or functional elements common to the different embodiments may have the same references and may have identical structural, dimensional and material properties.

[0032] For the sake of clarity, only the steps and elements useful for understanding the described embodiments have been shown and are detailed. In particular, the operation and implementation of artificial neural networks are known to those skilled in the art and are not described in detail.

[0033] Unless otherwise specified, when referring to two elements connected to each other, this means directly connected without intermediate elements other than conductors, and when referring to two elements connected (in English "coupled") to each other, this means that these two elements can be connected or be connected by means of one or more other elements.

[0034] In the following description, when reference is made to absolute position qualifiers, such as the terms "front", "back", "top", "bottom", "left", "right", etc., or relative position qualifiers, such as the terms "above", "below", "upper", "lower", etc., or to orientation qualifiers, such as the terms "horizontal", "vertical", etc., reference is made unless otherwise specified to the orientation of the figures.

[0035] Unless otherwise specified, the expressions "about", "approximately", "substantially", and "of the order of" mean to within 10%, preferably to within 5%.

[0036] [Fig.l] illustrates an example of a convolutional neural network 100. By way of example, the illustrated neural network 100 is configured to perform image recognition. More particularly, from an input image 102, illustrating for example a dog, the neural network 100 is configured to estimate probabilities of belonging to one or more classes of objects. The neural network 100 is therefore configured to, for example, generate output data 104, for example a vector, indicating probabilities of belonging, of the input image 102, to classes, or categories, such as, for example, “dog”, “cat”, “lions” and “bird”. Other examples of image processing and other categories are of course possible.Similarly, in other examples, the neural network 100 is configured to generate an output vector 104 comprising the probabilities of the input image 102 belonging to more than ten, or a hundred classes. The example of a network configured to perform image processing is of course not limiting. Generally, the neural network 100 comprises at least one convolution layer.

[0037] By way of example, the input image 102 is successively processed by a series of layers of the neural network 100. By way of example, the first layer of the network 100 is a convolution layer 106 configured to detect characteristics in the input image 102, such as, for example, contours, shapes, colors and / or textures. Layer 106 is for example followed by a pooling layer 108 configured to reduce the dimensions of the image and thus reduce the number of parameters and calculations performed in the network 100. Layer 108 is for example followed by a new convolution layer 110. Layer 110 is for example configured to detect characteristics in the input image 102, such as for example, contours, shapes, colors and / or textures. By way of example, the characteristics detected by layer 110 are finer characteristics than those detected by layer 106. For example, layer 106 will be configured to detect colors and / or contours and layer 110 will be configured to detect patterns and / or textures in the image.For example, layer 110 is followed by a new pooling layer 112, itself connected to a fully connected layer 114 configured to combine the features extracted in the previous layers to generate the output vector 104.

[0038] Each of the layers 106, 108, 110, 112 and 114 is associated with one or more parameters, such as, for example, weights associated with each neuron of a layer. The value of each parameter, and in particular of each weight, is a value learned during the training, or learning, of the network 100. Once the values ​​of the parameters are fixed, that is to say when the training of the network 100 is considered complete, the network 100 is used to perform, via the sequence of layers 106 to 114, inference operations. In particular, during the execution of the network 100, operations of the multiplication and accumulation type (MAC) are performed. MAC operations correspond, for example, to the summation, over several neurons in a layer, of the products between a value supplied to a neuron and, for example, the weight associated with this neuron.In other words, a MAC operation, on a number N of neurons, N being an integer greater than or equal to 1, corresponds to performing the following operation: , where wi is the weight associated with . neuron of rank i and where xi is the input value provided to the neuron of rank L

[0039] [Fig.2] illustrates an example of hardware implementation 200 of a multiplication and accumulation operation.

[0040] As an example, the implementation, or circuit, 200 comprises a multiplier 202 (x ) configured to receive two input values ​​and to generate an output value WJC corresponding to the product of the two input values. As an example, the two input values ​​provided to the multiplier correspond respectively to an input value x to a neuron and the weight w associated with this neuron. The circuit 200 further comprises an adder 204 (+). The adder 204 is for example connected to the multiplier 202, and the multiplier 202 is configured to provide the generated output value to the adder 204. The adder 204 is, for example, further connected to a memory 206 (ACCU). For example, the memory 206 is a register. For example, the adder 204 is further configured to read the value stored in the memory 206 and to generate an output value by adding it to the value provided by the multiplier 202. The adder 204 is then, for example, configured to store the generated value in the memory 206. For example, the storage of the generated value is carried out following the deletion of the value which was previously read by the adder 204.

[0041] The values ​​manipulated by MAC circuits such as circuit 200 are values ​​quantized in sets of values. The quantized values ​​are, for example, fixed point.

[0042] [Fig.3] illustrates an operation of quantization of the weights of the network 100 on 8 bits.

[0043] As an example, the learning of the network 100 is carried out by an external device to the circuit on which it will be implemented. For example, the external device is a device with a large computing capacity. Following the learning, the weights are, for example, stored in a memory 300 (CLOUD AI), external to the circuit on which the network 100 will be implemented. For example, the memory 300 is a server. For example, the weights stored in the memory 300 are floating point numbers and are for example coded on 16 or 32 bits.

[0044] For example, before being stored in a memory of a device in which the network 100 will be implemented, the learned weight values ​​are quantized, or transformed, into quantized values ​​being integer values ​​or fixed-point values. For example, the quantization is performed by a compiler for porting to a peripheral circuit 302 (EDGE AI). For example, the peripheral circuit 302 is configured to perform operations according to edge computing methods.

[0045] By way of example, the circuit 302 is configured to transform the weight values ​​transmitted by the memory 300 into integer values, and for example into a number less than 16 bits, for example into 8 or 4 bits.

[0046] As an example, the circuit 302 is configured to quantize the weight values ​​on 8 bits by performing a linear operation. For example, for a floating point value f, for example coded on 32 or 16 bits, an integer value for example coded on 8 bits, is such that r = 5.(q - Z), where the value S is a real and positive constant representing a scaling factor, the value Z is an offset of the zero value corresponding to the value quantifying the value 0. The linear operation thus described is known to those skilled in the art and is for example described in part 2 of the publication “Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference” by

[0047]

[0048]

[0049]

[0050] B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam and D. Kale-nichenko. In this example, and in relation to [Fig. 2], the input data supplied to the multiplier 202 are, for example, 8-bit coded data. The multiplier 202 is then, for example, configured to generate, from the two 8-bit coded input values, a 16-bit coded value. Figure 4A is a histogram 400 of weight values, according to an embodiment of the present description. In particular, the histogram 400 illustrates a Gaussian distribution of values ​​belonging to a set of integer values ​​Q3 where f) ■= / ni II [ + 7"l| | | + 37”]. In other words, the set Q, comprises v3- 1 J --- only, for example the value 0, integers equal to powers of 2, such as for example the values ​​-4, -2, -1, 1, 2, 4, etc., and integers equal to powers of 2, multiplied by 3, such as for example, -6, -3, 3, 6 etc. In another example, the value 0 does not belong to the set Q3. According to one embodiment, the circuit 302 is for example configured to quantize the value of the weights stored in the memory 300 into values ​​belonging to the set Q3. According to one embodiment, for a floating point value 1, a so-called simple quantization towards a value ? belonging to the set Q3 is carried out by applying an operation such as $ _ jô-” where sign(r) = 1 if r is positive and sign(r) = - 1 if r is negative and where, for any floating point x value । iG3 is the value belonging to Q3 closest to the absolute value of the x value. For example, for r = -5.3, 6 and "=-6. According to one embodiment, each circuit, on which the network 100 is implemented, is configured to manipulate and store the quantized values, belonging to the set Q3. According to other embodiments, the set Q3 is replaced, or supplemented, by one or more other sets comprising only values ​​equal to the product of the power of 2 by an odd number other than 3, for example by 5 and / or 7, etc. For example, the set Q3 is replaced by the set Q3 UQ where fis = {°! IL .4 1 LUed + 52"} and the quantification is carried out in the set Q3 U g In another example, the quantification is carried out only in the set Q^. Generally, the quantification is carried out only in a set, or in a union of sets, of the form where =M LL 4 1L LL 41 dp + LO'p being a positive integer-However, the most interesting combinations at the material level of the implementation of the MAC circuit are the values ​​which are the sums of two powers of two.

[0051] [Fig.4B] is a graph illustrating Gaussian quantization as an alternative to the quantization described in connection with [Fig.4A], according to one embodiment of the present disclosure.

[0052] For example, quantization to values ​​in is performed by so-called Gaussian quantization operations. For example, quantization in Q3 is performed on a number P of bits, for example 4 bits. That is to say, the quantized values ​​are encoded on a number P of bits. To discretize the weight values ​​on P-bits, it is assumed that they follow a centered normal distribution and variance o2. For example, the value of the variance o2 is obtained statistically from a sample of weight values. A number 2 / ' - 1 of interval Ik, fa g [ £ _ 2P - 1} is defined so that the probability of belonging of a random variable of distribution N ( 0, a2 ) to an interval is uniform, that is equal to whatever the interval. The interval is then of the form = ]-«>, the interval j of the form / p ; . The intervals h, k G {2, ■ ■ ■, 2P - 2} are of the form [b^, b^- The limits bk are then values ​​dependent on P and <?2. La densité d’une loi normale centrée étant symétrique et 2P -1 étant un nombre impair, pour tout £ e {1, 2P - 1J, 'a valeur 0 est une valeur quantifiée. Pour each interval & e {2, ■ ■ ■, 2P - 2}, a value is defined such that: [Math 1] erf(bkl0^2)+erf(bkJa^2 ) \ where erf is the error function. As 2 I ' for example, for p=4 and = 1, the lk values ​​rounded to the nearest hundredth are {-1.83,-1.28,-0.97,-0.73,-0.53,-0.34,-0.17,0,0.17,0.34,0.53, 0.73,0.97,1.28,1.83}. To avoid handling floating point values, the values ​​1^, & e 2P- 1| are transformed linearly into values ​​„ _ 11 4 12 where AG e R is a factor Ll k L g-''g J of approximation and where Q c Z is a set of quantized values, for example Q — Qy As an example, the value AG can be obtained by linear regression, You"

[0053] [Fig.5] is a block diagram illustrating a multiplier and accumulator circuit 500, according to an embodiment of the present description.

[0054] According to one embodiment, the values ​​belonging to are coded on a maximum of 4 bits. A bit, for example the most significant bit, is a sign bit, a another bit, for example the second most significant bit indicates whether the value is, for example, a multiple of 3. Finally, the remaining two bits, for example the two least significant bits, encode the value of the exponent n. For example, n — 0 if the value is odd, n = 1 if the value is equal to ± 2 or ± 6, etc. Since the exponent n is limited to 2 bits, the set is then limited to the values {-12, -8, -6, -4, -3, -2, -1, 0,1, 2, 3, 4, 6, 8, 12). So, for example, the 4-bit value 0000 encodes, for example, the value 1, the 4-bit value 1101 encodes, for example, -6, etc. This example of 4-bit encoding is of course presented as an example and other forms of encoding can of course be envisaged. In another example, the least significant bit is the sign bit and the two most significant bits encode the value of the exponent. As an example, the 4-bit value XI11 encodes the value 0, where X is a bit with a value of 0 or 1.

[0055] Generally speaking, in the case where the quantification is carried out only in a set of form Q^p+x, the second most significant bit indicates for example whether the value is a multiple of 2p + 1 or not.

[0056] The value 0000 then encoding a value other than 0, in the case where the person skilled in the art wishes for the weights of the network 100 to have the possibility of being zero, he will know how to encode, in a specific manner, the value of weight 0.

[0057] According to one embodiment, the MAC circuit 500 comprises a multiplier circuit 502. The multiplier circuit 502 is configured to receive an activation value, corresponding to an input value transmitted for example to a neuron of the network 100. For example, the input value is provided to the multiplier circuit 502 via a multiplexer 504. For example, the activation value provided to the multiplexer 504 is an 8-bit value. For example, the multiplexer 504 is configured to provide the circuit 502 with an activation value, being equal either to the value 0 or to a non-zero value. For example, the multiplexer 504 is present when the activations are encoded with an offset of the zero value. For example, multiplexer 504 is omitted when activations are not encoded with an offset of 0. In this case the activations are provided as inputs to multiplexer 502.

[0058] In particular, the multiplexer 504 is connected to a multiplexer 506 of the circuit 502. The multiplexer 506 is configured to select the value 0, or the activation value provided by the multiplexer 504, on the basis of the values ​​of the three bits indicating whether the weight value is a multiple of 3 (M) and indicating the value of the exponent (EE). For example, if at least one of the three bits is different from 1 ( * 111), the multiplexer 506 is configured: to select the activation value, corresponding to the case where the weight value is different from the value 0; and to select the value 0 in the case where the three bits are encoded to the value 1. The value selected by the multiplexer 506 is then, for example, an 8-bit value and is transmitted to an adder 508. The value selected by the multiplexer 506 is in parallel supplied to a shift circuit 510 (“1). The shift circuit 510 then generates a value encoded on, for example, 9 bits, which is the value provided by the multiplexer 506 shifted one bit to the left, thus corresponding to its multiplication by 2. The circuit 510 is further connected to a multiplexer 512 configured to select, on the basis of the value of the bit (M) indicating whether the weight value is a multiple of 3, the output of the circuit 510 or the value 0. For example, if the weight value is not a multiple of 3, the multiplexer 512 is configured to select the value 0 and, if the weight value is a multiple of 3, the multiplexer 512 is configured to select the value at the output of the circuit 510.The selected value is then transmitted to adder 508. Adder 508 is then configured to add the output values ​​of multiplexers 506 and 512. For example, circuit 510 includes a wire connected to ground and configured to add bit "0" as the least significant bit to the input value. Shift circuit 510 is, for example, a symbolic component.

[0059] Thus, in the case where the weight value is a multiple of 3, the output value of the adder 508 corresponds to three times the input value. In the case where the weight value is not a multiple of 3, without being the value 0, the output value of the adder 508 corresponds to the activation value. For example, the value generated by the adder 508 is a 10-bit value.

[0060] The value generated by the adder 508 is then supplied to a multiplexer 514 and, in parallel, to an inverter 516. The inverter 516 is therefore configured to change the bits having the value 1 to the value 0, and vice versa, from the value supplied by the adder 508. The inverted value is then supplied to an adder 518. Adder 518 is a 1-by-one adder, configured to add 1 to the output value of inverter 516. Inverter 516 and adder 518 are therefore configured to generate, using the 2's complement method, the opposite value of the output value of adder 508. The output value of adder 518 therefore corresponds to the value which, when added to the output value of adder 508, results in a series of 0s. In other words, the output value of adder 518 corresponds to multiplying the output value of adder 508 by -1.

[0061] The multiplexer 514 is then configured to select, based on the value of the bit (S) indicating whether the weight value is positive or negative, the output value of the adder 508 or the adder 518. For example, if the weight value is positive, the multiplexer 514 selects the output value of the adder 508, and if the weight value is negative, the multiplexer 514 selects the output value of the adder 518.

[0062] The multiplexer 514 is then configured to provide the selected value, for example on 11 bits, to a shift circuit, or shifter 520. The shift circuit 520 is then configured to shift the received value to the left based on the value of the two bits (EE) indicating the exponent. For example, if the two bits are equal to 0, the circuit 520 simply restores the value at the output of the multiplexer 514, without shifting it. Generally, the circuit 520 is configured to shift the supplied value by a number of bits equal to the value of n. The value restored by the circuit 520 then corresponds to the product of the value at the output of the multiplexer 514 by 2”. For example, the value at the output of the circuit 520 is a 21-bit value.

[0063] The multiplier 502 is further connected to an adder 522 configured to read the output value of the circuit 520. The adder 522 is further configured to add the output value of the 520 with a value stored in a memory 524 (ACCU). The adder 522 is further configured to transmit the generated value to the memory 524. For example, the memory 524 is similar to the memory 206. For example, the values ​​manipulated by the adder 522 are coded on 21 bits.

[0064] The numbers of bits manipulated by the elements of the multiplier 502 and by the adder 522 are given by way of example. For example, in the case where the exponent n is coded on a number of bits greater than 2, the person skilled in the art will know how to adapt the number of bits manipulated by the different elements of the circuit 500 to the number of bits encoding the weight value as well as the shift carried out by the circuit 520.

[0065] In the case where the weight value belongs to a set of type P - or to the union of several sets of type Ô2p+i' P - My person skilled in the art will know how to adapt the part of the circuit 502 comprising the elements 506, 510, 512 and 508 in order to replace them, or to add other elements so as to carry out a multiplication by the desired odd number(s). For example, if the set considered is of the form Q3 U ô5, another multiplexer could be added in order to select whether the weight value is a multiple of 5 or not, and this other multiplexer could be connected to an adder and to a shift circuit configured to shift the received value by at least one bit to the left. The number of bits involved in the shift depends on the number of bits used to encode the value of the exponent of 2. Depending on the position of this other multiplexer, the circuit can be adapted to the whole without having to add any additional hardware component.In this case, the set will correspond to (1xA+2xA)+4xA, where a first multiplexer will choose to add the value 2xA or not, in the same way as the multiplexer 512, and a second multiplexer will choose to add the value 4xA or not. Generally speaking, the more the set considered includes set unions of type <22p+n the more the rePre . will approximate a floating-point description. Generally, the circuit portion 502 includes a number of adders equal to the number of "1"s in the binary representation of the odd number, minus one. For example, the binary representations of odd 3 and 5 are "011" and "101", comprising two "1" values, so the portion 502 adapted to or to Q5 includes one adder. For example, the binary representation of odd is "111" comprising three "1" values, so the portion 502 adapted to includes two adders.

[0066] Figure 6 illustrates a validation test of the quantification in the set

[0067] By way of example, the illustrated validation test is carried out following the learning of the weights of a convolution layer of a network for example similar to the neural network 100 described in relation to [Fig.l]. By way of example, the network is a CNN type network (from the English “Convolutional Neural Network”).

[0068] A histogram 600 illustrates the distribution of weights (WEIGHT DISTRIBUTION) obtained, during training, for the second layer of the network. More particularly, the weights illustrated in the histogram 600 are floating point values. The distribution of weights then forms a Gaussian.

[0069] A histogram 602 illustrates the distribution of quantized weights. For example, the quantized weights are the weights obtained, during training, for the second layer of the network, then quantized in the set Qy K for example, from a floating point weight 7, a quantized weight ? is such that _ ( r ) [ 5 rj ^3' æ" partition of the quantized weights also forms a Gaussian. Moreover, we note that all the quantized weights belong to the interval [-12,12] and can therefore be coded on 4 bits as described in relation to [Fig.5].

[0070] Figure 7 illustrates a table 700 and a table 702 each comprising the results of validation tests. In particular, tables 700 and 702 include comparisons of the costs in energy consumption (POWER (^W)), in surface area (AREA ( pm2)) as well as the precision of the network (ACCURACY) for a linear quantization (LINEAR) and a quantization in Qv on 4 bits (SIMPLE Q^. In particular, table 700 illustrates a comparison between a linear quantization (LINEAR), for example described in relation to figure 3, on 8 bits and the quantization in on 4 bits. Table 702 illustrates a linear quantization on 4 bits and the quantization in on 4 bits. The comparison is carried out for example for a convolution layer of a neural network similar to the network 100 and in which the activation data of the layer are coded, by a linear quantization, on 8 bits.The quantifications are then carried out for the weights of neurons of said layer, and from floating point values, obtained for example during the learning of the . network.

[0071] As an example, 8-bit linear quantization makes it possible to quantize floating-point values ​​into integers belonging, for example, to the interval |J 0.255 J • 4-bit linear quantization makes it possible to quantize floating-point values ​​into integers belonging, for example, to the interval [ 0.15 ] • More particularly, 8-bit linear quantization has 256 value representations for the weights and 4-bit linear quantization has 16 representations. 4-bit quantization in Q3 has, as described in relation to FIG. 5, 15 representations, including the value 0. A gain column (GAIN) in each table 700 and 702 includes the gains in power consumption, area, and precision provided by quantization in Q3 compared to linear quantization.As an example, the area used by the quantization in Q3 on 4 bits is calculated from the implementation of the MAC 500 circuit described in relation to [Fig.5].

[0072] As an example, on a tested 40nm technology, a linear quantization on 8 bits, respectively on 4 bits, requires for example 28.1 UIV of consumption, respectively 20.6 g W, while the quantization in Q3 requires for example 17.9 pW, which represents a gain of 36% in the first case and 13.1% in the second case. Similarly, the linear quantization on 8 bits, respectively on 4 bits, requires for example 566.3 ^m2 of surface, respectively 369 pm2, while the quantization in Q3 requires for example 358 pw2. which represents a gain of 37% in the first case and 3% in the second case.

[0073] The accuracy in the case of 8-bit linear quantization is for example 99.06% and 99.05% in the case of 4-bit linear quantization. The accuracy for quantization on Q3 is for example 99.37%, i.e. a gain of 0.31% compared to 8-bit linear quantization and 0.32% compared to 4-bit linear quantization.

[0074] Although the quantization in Q3 and on 4 bits allows the coding of the weight values ​​only in a set of 14 values, or 16 if the values ​​-24 and 24 are included, or 15 including the value 0, these are more distributed so as to cover the Gaussian distribution more judiciously and cleverly, which results in a high precision. Indeed, the 16 values ​​described by the linear representation on 4 bits are too few in number and all at equal distance from each other which is not ideally suited to describing values ​​distributed according to a normal law. In addition, the 256 values ​​described by the linear representation on 8 bits, although allowing the Gaussian distribution to be faithfully described, are too numerous. Indeed, the tail values, that is to say the high and low values ​​are very little used and the fact of considering them increases the surface and the energy consumption without providing a significant gain in precision.

[0075] [Fig.8] is a graph 800 representing several quantization modes for a VGG (Visual Geometry Group) type neural network model. This model is characterized by a succession of blocks composed of two to three convolution layers followed by a so-called maxpooling layer. Each convolution layer has a 3 by 3 filter and the maxpooling layers are 2 by 2 in size. For example, a VGG16 model has a depth of 16 layers and a VGG 19 model has a depth of 19 layers. In particular, the graph 800 comprises several points, each associated with a quantization model. The abscissa of each point represents the consumption ratio (POWER RATIO) and the ordinate of each point represents the precision obtained by the VGG network. The consumption ratio is defined relative to the consumption of linear quantization with 8 weight bits and 8 activation bits.For example, a ratio of 65% means that the tested quantization consumes 35% less energy than the 8-bit linear quantization. For example, the parameters of the VGG network were obtained by training and are, by default, floating point values ​​(FLOAT). The different quantizations illustrated are carried out, for example, on the weights of the convolution layers of the network. For example, a first training of the network is carried out with floating point values, then a second training is carried out, from the weights obtained during the first training, in order to simulate a quantization. This technique is known by the person skilled in the art as QAT (from the English "Quantization Aware Training"). For example, for each point, between 10 and 20, for example 15, trainings of the network are carried out and the maximum precision is recovered.

[0076] For example, points 801 (Linear_a8_w8), 802 (Linear_a8_w4), 804 (Linear_a6_w8) and 806 (Linear_a6_w4) represent the consumption ratio and the precision of the VGG network for linear quantizations. In particular, point 801 is associated with an 8-bit linear quantization, and for which the activation points are also on 8 bits. Point 802 is associated with a 4-bit linear quantization, and for which the activation points are also on 8 bits. The precision provided by these two quantizations is similar, i.e. approximately 99.05%, but the 4-bit quantization is less expensive in terms of energy consumption. Points 804 and 806 are associated with a linear quantization on 8 and 4 bits respectively, and for which the activation points are also on 6 bits.

[0077] Points 808 (Log2_a8_w4), 810 (Log2_a6_w4) and 812 (Log2_a4_w4) are associated with 4-bit logarithmic quantizations, and for which the activation points are, respectively, on 8, 6 and 4 bits.

[0078] Points 814 (Simple_C23_a8_w4), 816 (Simple_ô-_a6_w4) and 818 (Simple_<23 _a4_w4) are associated with simple quantizations in 4 bits, as described in relation to [Fig.4A], and for which the activation points are, respectively, on 8, 6 and 4 bits.

[0079] Points 820 (Gauss_Ô3_a8_w4), 822 (Gauss_(23_a6_w4) and 824 (Gauss_Ô3 _a4_w4) are associated with Gaussian quantizations, as described in relation to FIG. 4B, in 4 bits, and for which the activation points are, respectively, on 8, 6 and 4 bits.

[0080] Graph 800 shows that for the tests performed, linear quantizations result in lower accuracy than keeping floating point values. However, quantizations in Qv in particular having activation points on 8 and / or 6 bits show better accuracy than keeping floating point values.

[0081] An advantage of the embodiments is that they make it possible to implement a MAC circuit adapted to the operations executed by one or more layers of a neural network and having a reduced surface area.

[0082] Various embodiments and variants have been described. The person skilled in the art will understand that certain features of these various embodiments and variants could be combined, and other variants will appear to the person skilled in the art. In particular, the set on which the quantization is carried out may vary. Indeed, although the set on 4 bits is mainly described, the person skilled in the art will know how to adapt the implementation of the MAC circuit to a set of type Q^p+i, or to the union of several sets of type Ô2 / >+i- Similarly, the person skilled in the art will know how to adapt the encoding of the values ​​of and their processing by the MAC circuit, when these are encoded on a number greater than 4 bits.

[0083] Finally, the practical implementation of the embodiments and variants described is within the reach of the person skilled in the art from the functional indications given above. In particular, the person skilled in the art will know how to adapt the implementation of the circuit 502 in order to carry out multiplications by one or more odd numbers other than 3.

Claims

Claims

1. A circuit (500) configured to perform a multiply and accumulate operation in a layer of an artificial neural network (100), the circuit comprising a multiplier circuit (502), an adder (522) and a memory (524), configured to perform the operation, the operation taking, as input, an input data value and a weight, and wherein the weight is only a value in a limited set consisting of only the value 0, a plurality of values ​​equal to 2”, where n is an integer, and a plurality of values, each equal to the product of 2” and an odd number greater than or equal to 3.

2. Circuit (500) according to claim 1, comprising: - a first sub-circuit configured to perform a multiplication of an input data value by said odd number, the first sub-circuit comprising a first shift circuit (510) configured to perform a multiplication by 2 and an adder (508); and - a second sub-circuit configured to perform a multiplication by the value 2W, the second sub-circuit comprising a second shift circuit (520) configured to perform a shift of n bits.

3. The circuit (500) of claim 2, wherein the first subcircuit further comprises a first multiplexer (512) configured to provide to the adder (508) a value selected from: an output of the first shift circuit (510); and the value 0, wherein the first multiplexer (512) is configured to select the output of the first shift circuit if the weight is equal to one of said plurality of values ​​equal to multiplying 2” by an odd number greater than or equal to 3.

4. The circuit (500) of claim 2 or 3, further comprising a second multiplexer (506) configured to provide the first subcircuit with a value selected from: the input data value; and the value 0, wherein the second multiplexer is configured to select the value 0 if the weight is equal to 0.

5. The circuit (500) of any one of claims 2 to 4, further comprising a third subcircuit configured to selectively apply a negative sign to an output value of the first subcircuit, the third subcircuit comprising, for example, an inverter (516), an adder (518) configured to add 1, and a third multiplexer (514) configured to selectively bypass the inverter and the adder.

6. Circuit (500) according to any one of claims 2 to 5, in which each weight is encoded on at least 4 bits, the at least 4 bits comprising: - one bit (S) encoding the sign of the weight; - at least one bit (M) indicating whether the weight has as its value one of the plurality of values ​​equal to the product of 2” by an odd number greater than or equal to 3; and - at least two bits (EE) encoding the value of n.

7. The circuit (500) of claim 6, wherein the second shift circuit (520) is configured to perform a multiplication by a power of 2, based on the value of the at least two bits (EE) encoding an exponent of 2.

8. A circuit (500) according to any one of claims 1 to 7, wherein the integer n is less than 3.

9. A circuit (500) according to any one of claims 1 to 8, wherein said plurality of values ​​equal to the product of the plurality of 2” values ​​by an odd number greater than or equal to 3 comprises: - a first plurality of values ​​equal to the product of the plurality of 2” values ​​by a first odd number; and - a second plurality of values ​​equal to the product of the plurality of 2” values ​​by a second odd number, different from the first odd number.

10. A circuit (500) according to any one of claims 1 to 9, wherein the odd number is the number 3.

11. A circuit (500) according to any one of claims 1 to 9, wherein the odd number is the number 3 and / or the number 5 and / or the number 7.

12. A circuit (500) according to any one of claims 1 to 11, wherein the weight value equal to 0 is encoded by a specific value, for example the binary sequence XI11, where X is equal to 0 or 1.

13. The circuit (500) of any one of claims 1 to 12, wherein the layer is a convolution layer.

14. A method of generating, by a circuit (500), a weighted output value of a neuron of a layer of a neural network, the circuit comprising a multiplier circuit (502), an adder (522) and a memory (524), the method comprising multiplying, by the multiplier circuit, an input data value by a weight, wherein the weight is only a value in a limited set consisting of only the value 0, a plurality of values ​​equal to 2”, where n is an integer, and a plurality of values, each equal to the product of 2” and an odd number greater than or equal to 3.

15. A method of training an artificial neural network, comprising determining weights of neurons, the determination being performed by selecting each weight only from a limited set consisting of only the value 0, a plurality of values ​​equal to 2”, where n is an integer, and a plurality of values, each equal to the product of 2” and an odd number greater than or equal to 3.

Images