Circuit and procedure for performing a mathematical operation

Abstract

The invention relates to a circuit (1) for performing a mathematical operation in which several input values ​​(10) are mapped to an output value (16), wherein the input values ​​are encoded as bit sequences of a predetermined number format in which different bit lengths are provided. The circuit comprises a decoder (2) configured to adapt the bit lengths of the bit sequences of the input values ​​(10) so that they have a predetermined processing bit length, and to decode the adapted bit sequences having the processing bit length according to the number format in order to determine numerical components (12) of the corresponding input value, and a processing unit (4) configured to perform the mathematical operation using the numerical components (12) of the input values ​​in order to determine numerical components (14) of the output values.

Description

[0001] The present invention relates to a circuit for performing a mathematical operation and an associated method. Background of the invention

[0002] Many computer applications, including neural networks, use approximations of real numbers in the form of floating-point numbers. Depending on the application, a specific level of floating-point precision can be used. For example, the IEEE 754-2008 standard specifies floating-point numbers with half precision (16 bits), single precision (32 bits), double precision (64 bits), and quadruple precision (128 bits). Disclosure of the invention

[0003] According to the invention, a circuit for performing a mathematical operation and an associated method, as well as a computing unit for performing the method, are proposed, comprising the features of the independent claims. Advantageous embodiments are the subject of the dependent claims and the following description.

[0004] The circuit for performing a mathematical operation, in which multiple input values ​​encoded as bit sequences of a predetermined numerical format are mapped to an output value, includes a decoder configured to adjust the bit lengths of the input bit sequences to a predetermined processing bit length. The adjusted bit sequences are decoded by the decoder and processed by a processing unit that implements the mathematical operation to determine numerical components of the output values. Adjusting the bit lengths enables the direct, co-processing of input values ​​with different bit lengths within a single circuit (i.e., in hardware). Since the bit lengths correspond to a certain precision of the input values, mixed-precision processing (of the input values) is enabled. This mixed precision is useful in applications (e.g.,(in neural networks) where large amounts of data are processed, this is useful to reduce the required storage space and memory bandwidth, since parts of such an application (e.g., a layer of a multilayer neural network) for which lower accuracy is sufficient can be implemented with smaller bit lengths.

[0005] The (mathematical) operation in question is, for example, a multiply-add operation or a fused multiply-add operation. Such operations occur, for instance, in neural networks during convolution. In a multiply-add operation, factors are multiplied and their product is added to form a continuous summand. A fused multiply-add operation refers to the implementation of the multiply-add operation without intermediate rounding (after the product is formed).

[0006] In particular, the number format is POSIT, i.e., the POSIT number format. This is well-known and will be briefly explained below.

[0007] POSIT numbers, i.e., numbers in the so-called POSIT number format, can be specified in the form of bit sequences as "POSIT<n, es> numbers", where n denotes the total number of bits and es denotes the maximum number of bits that an exponent can have.

[0008] The structure as a bit sequence generally has several bit fields as follows. S R E F

[0009] Here, S is a sign bit, where the sign of the positive number is represented as sign = (-1) S The sign is determined. If the sign is negative (S = 1), the two's complement is determined before further decoding.

[0010] R are so-called regime bits, from which a value k can be determined. The numerical value of k results from the number of consecutive 0 bits (bits with the value 0) or 1 bits (bits with the value 1) following the sign bit. If m consecutive 0 bits occur, k = -m. If m consecutive 1 bits occur, k = m - 1. Here, m ≤ n - 1, so the other bit fields E and F can also be omitted. The regime bit field R is terminated with a bit that has the opposite value of each successive bit in the sequence. If this terminating bit is not present (i.e., if m = n - 1), the bit fields E and F are omitted.

[0011] The bit field E, if present, specifies the exponent exp (as a binary value, where the least significant bit in bit field E is on the right relative to the bit sequence representation above). This can have a maximum of e bits. Whether bit field E is present and, if so, how many bits it contains, is determined by comparing the known number of bits used for bit fields S and R to the total number n of bits.

[0012] A fraction f (decimal part) is encoded in the bit field F, if present (i.e., if the total number n of bits has not yet been exhausted, even though the exponent contains n bits). The bit field F contains fractional bits F i or fractional bits. Implicitly, a 1 is also included at the beginning, which is omitted when encoding the bit sequence. If the number of fractional bits is denoted by b, then: f=2−b⋅∑i=0b−1Fi⋅2i

[0013] The least significant fractional bit, F0, is located to the right, or at the end, of the bit sequence as shown above. The corresponding bit field F is F. b-1 F b-2 ... F0.

[0014] The encoded numerical value is calculated as follows: (−1)sign⋅2k⋅2es⋅2exp⋅(1+f)

[0015] It follows in particular that adding bits with a value of 0 to the end (or to the right) of the bit sequence does not change the numerical value.

[0016] The bit fields R and E (if present), from which k and es are derived, together encode the order of magnitude (in the sense of a power of 2) of the positive number. The order of magnitude (as a binary value) is approximately the combined exponent of 2 in the formula above (i.e., k · 2). es + exp).

[0017] Bit sequences in which all bits except the sign bit have the value 0 encode special values ​​or have special meanings. If the sign bit is 1 and all other bits are 0, this is interpreted as an indefinite or non-real number (NaR). If the sign bit is 0 and all other bits are 0, the positive number is assigned the numerical value 0.

[0018] The sign bit, the order of magnitude, and the fractional bits represent numerical components obtained in the decoding of a positive number, suitable for processing or calculations (especially by hardware processing units). These are also referred to as the (numerical) components of the positive number. Furthermore, a binary code (i.e., one or more bits) can be used as an additional numerical component of the positive number (also determined during decoding) to indicate the special values ​​or meanings mentioned above.

[0019] The accuracy with which real numbers can be approximated decreases for POSIT numbers with a specific total number n of bits, starting from zero. Here, the accuracy corresponds to the distance between two POSIT numbers that are adjacent in terms of encoding, or the density of POSIT numbers. This is what distinguishes POSIT numbers from, for example, conventional floating-point numbers (IEEE 754). This property makes POSIT numbers particularly suitable for certain applications where the data values ​​to be processed exhibit a statistical distribution with a large number of small-valued data values ​​and a small number of large-valued data values. This is the case, for example, with neural networks.

[0020] In one embodiment of the circuit, POSIT is used as the number format; the decoder is configured to append one or more bits with the value 0 to the bit sequence of an input value if the input value has a smaller bit length than the processing bit length, in order to adjust the bit length. This adjustment can be implemented simply, without loss of accuracy.

[0021] In one embodiment, the circuit includes an encoder configured to encode the numerical components of the output values ​​according to the numerical format in order to determine the output value as a bit sequence with a processing bit length. The output value can then be further processed or stored by a higher-level control unit of the circuit.

[0022] According to one embodiment, the encoder is further configured to adjust the bit length of the output value's bit sequence so that it has a predetermined output bit length or a configurable output bit length. The output values ​​can thus be adjusted to a desired output bit length, which is particularly smaller than the processing bit length. This adjustment by the encoder can be achieved, in particular, by configuring the encoder to adjust the output value's bit sequence by removing bits when the processing bit length is larger than the output bit length.

[0023] In one embodiment of the circuit, POSIT is used as the number format, with the decoder configured to remove one or more bits from the bit sequence of an input value if it has a larger bit length than the processing bit length, in order to adjust the bit length. This embodiment allows the use of input values ​​with a large bit length.

[0024] According to a further aspect of the invention, a method for performing a mathematical operation using a circuit according to the invention is proposed, wherein the mathematical operation is to be applied to several input values ​​encoded as bit sequences of a predetermined number format. The bit sequence of at least one input value has a bit length that is smaller than the processing bit length. In the method, the input values ​​are transmitted as bit sequences of the number format to an input register of the circuit, the decoder of the circuit is controlled to adjust the bit lengths of the bit sequences of the input values ​​and to decode the adjusted bit sequences in order to determine numerical components of the corresponding input value, and the processing unit is controlled to determine numerical components of an output value from the numerical components of the input values.

[0025] A computing unit according to the invention, e.g. an accelerator computing unit for accelerating AI applications, comprises a circuit according to the invention and is, in particular in terms of programming, configured to carry out a method according to the invention.

[0026] Further advantages and embodiments of the invention will become apparent from the description and the accompanying drawing.

[0027] The invention is schematically illustrated in the drawing using exemplary embodiments and is described below with reference to the drawing. Brief description of the drawings Fig. Figure 1 illustrates a circuit for performing a mathematical operation according to an exemplary embodiment. Fig. Figure 2 shows a decoder according to an exemplary embodiment. Fig. Figure 3 shows an encoder according to an exemplary embodiment. Fig. Figure 4 shows, by way of example, the sequence of a method for performing a mathematical operation using a circuit according to the invention. embodiment(s) of the invention

[0028] Fig. Figure 1 illustrates a circuit 1 for performing a mathematical operation according to an exemplary embodiment. The circuit 1 comprises a decoder 2, a processing unit 4, and optionally an encoder 6.

[0029] Circuit 1, or rather its decoder 2, is configured to receive one or more input values ​​10. The input values ​​10 are encoded as bit sequences according to a specific number format, in particular in the POSIT number format. The input values ​​10 can have different bit lengths.

[0030] Decoder 2, see also Fig. Decoder 2 is configured to adapt the bit sequences of input values ​​to a predetermined bit length, referred to as the processing bit length. Furthermore, decoder 2 is configured to decode bit sequences with the processing bit length in order to determine components, or numeric components 12, corresponding to the input values. The encoding is performed according to the number format. Decoder 2 can be configured to decode multiple input values ​​simultaneously, i.e., to determine their numeric components at the same time. Alternatively, decoder 2 can be configured to decode a single input value, whereby, if the mathematical operation uses input values ​​(i.e., multiple operands), these can be decoded sequentially.

[0031] The bit lengths of the input bit sequences are, in particular, less than or equal to the processing bit length, which can be, for example, 8, 12, 16, 24, or 32 (or any other bit length). If the processing bit length is denoted by N, the input bit sequences could, for example, have bit lengths of N, N-4, or N-8.

[0032] Processing unit 4 is configured to perform the mathematical operation; that is, processing unit 4 is configured to map the numerical components of the input values ​​to numerical components 14 of an output value, or a result of the mathematical operation. The mathematical operation can be, for example, a multiply-add operation, i.e., an operation in which two input values ​​are multiplied and added together to form a running addend. Thus, the numerical components of this addend, which represents the output value, are determined.

[0033] Encoder 6 is configured to encode the numerical components of the output value according to the numerical format, so that the output value in numerical format is determined as a bit sequence with the processing bit length. Furthermore, encoder 6 can be configured to adjust the bit length of the output value so that it has a specific output bit length. This corresponds to the output value 16 of circuit 1. The output bit length can be predetermined. Alternatively, it can be provided that the output bit length is predefinable or configurable, in particular selectable from a predetermined set of output bit lengths (e.g., by means of a corresponding control signal). The output bit length is, in particular, equal to or less than the processing bit length.

[0034] The circuit can also be implemented without an encoder, e.g., in the case that downstream processing elements in the circuit are intended to process the output values ​​directly based on their numerical components.

[0035] Fig. Figure 2 shows a decoder 2 according to an exemplary embodiment. This decoder is configured to receive an input value 10 in the form of a bit sequence corresponding to the number format used, e.g., in an input register 21 (which can also be considered an input register of the circuit). The decoder 2 includes a (first) bit length adjustment stage 22, which is configured to adjust the bit length of the bit sequence of a received input value so that it is equal to the processing bit length. The adjusted bit sequence is stored, for example, in a (first) intermediate register 23 of the decoder (which can be the same as the input register if the bit length adjustment stage operates directly on it). In particular, the bit lengths of the bit sequences of the input values ​​are less than or equal to the processing bit length.

[0036] The preferred number format is the POSIT number format. If the processing bit length is again denoted by N, the input value can, for example, be a POSIT number with a bit length of N-4, i.e., a POSIT < N - 4, es > number. The bit sequence of the input value (in input register 21) is therefore symbolic: Posit< N - 4, es >

[0037] This bit sequence changes to a < N, es > number (in intermediate register 23) by adding 0 bits (through the bit length adjustment stage): Posit< N - 4, es > 0 0 0 0

[0038] The corresponding numerical value does not change due to the structure of numbers in the POS format, as already explained.

[0039] Decoder 2 further comprises a decoding stage 24, which is configured to determine the numerical components 12 of the adapted bit sequence (in intermediate register 23) of the input value. The numerical components 12 thus correspond to the input value. In the case of the positive number format, the numerical components are, specifically, a sign component 12s, an order-of-magnitude component 12g, a fractional component 12f, and an additional component 12z, which indicates whether the adapted bit sequence has a special value or meaning (zero or indeterminate). These numerical components 12 are passed, e.g., via corresponding registers, to the processing unit 4 (see...). Fig. 1) passed, which then performs the mathematical operation and determines numerical components of a result or output value of the operation, which are passed, e.g. via corresponding registers, to subsequent circuit elements, in particular the encoder 6.

[0040] Fig. Figure 3 shows an encoder 6 according to an exemplary embodiment. The encoder 6 is configured to receive numerical components 14, which in particular, see Figure 3. Fig. 1, as numerical components of an output value were determined by processing unit 4, e.g., via corresponding registers. In the positive number format, the numerical components are, for example, a sign component 14s, an order-of-magnitude component 14g, a fraction component 14f, and an additional component 14z, which indicates whether a special value or meaning applies (zero or indeterminate).

[0041] The encoder 6 has an encoding stage 62, which is configured to generate a bit sequence in numerical format from the number components 14, where the bit sequence has a bit length equal to the processing bit length. The generated bit sequence is temporarily stored, for example, in a (second) intermediate register 63.

[0042] A (second) bit length adjustment stage 64 may also be provided, which is configured to adapt the bit length of the bit sequence stored in the intermediate register 63 to the output bit length. The adapted bit sequence (which corresponds in particular to the output value of the processing unit) is output, for example, in an output register 65 (which may coincide with the intermediate register 63; and which can also be considered an output register of the circuit) (e.g., to further or calling hardware components). Referring to Fig. 1. The initial value 16 (result of the mathematical operation) is therefore output as a bit sequence in number format.

[0043] Fig. Figure 4 shows, by way of example, the sequence of a method for performing a mathematical operation using a circuit according to the invention.

[0044] The method involves applying a mathematical operation to multiple input values ​​encoded as bit sequences of a predetermined numerical format. The bit sequence of at least one input value has a bit length smaller than the processing bit length. For example, in a multi-layer neural network, the input values ​​could include the weights of a layer, which already have the processing bit length, and the values ​​calculated from a previous layer, which have a bit length smaller than the processing bit length. The steps of the method are initiated or controlled, for example, by a controller. The controller and the circuitry (which implements the mathematical operation) are integrated, for example, in a processing unit. The processing unit is, for example, an AI accelerator processing unit.A computing unit for accelerating calculations that occur in connection with applications of artificial intelligence (AI), e.g., neural networks.

[0045] In step 110, the input values ​​101, 102, ... 10M are transmitted as bit sequences of the number format to an input register of the circuit.

[0046] In step 120, the circuit's decoder is controlled to adjust the bit lengths of the input value bit sequences and to decode the adjusted bit sequences to determine numerical components of the corresponding input value.

[0047] In step 130, the processing unit is controlled to determine numerical components of an output value from the numerical components of the input values.

[0048] In an optional step 140, the encoder is controlled to encode the numerical components of the output values ​​according to the number format in order to determine the output value 16 as a bit sequence with the processing bit length. Specifically, the encoder is controlled to adjust the bit length of the output value's bit sequence so that it equals the output bit length.

Claims

[1] Circuit (1) for performing a mathematical operation in which a plurality of input values ​​(10) are mapped to an output value (16), wherein the input values ​​are coded as bit sequences of a predetermined number format in which different bit lengths are provided, comprising a decoder (2) which is arranged to adapting the bit lengths of the bit sequences of the input values ​​(10) so that they have a predetermined processing bit length, and decoding the adapted bit sequences having the processing bit length according to the number format to determine number components (12) of the corresponding input value; and a processing unit (4) configured to perform the mathematical operation using the numerical components (12) of the input values ​​to determine numerical components (14) of the output values. [2] A circuit according to claim 1, wherein Posit is used as the number format; wherein the decoder (2) is arranged to add one or more bits with the value 0 to the bit sequence of an input value if it has a smaller bit length than the processing bit length in order to adapt the bit length. [3] A circuit according to claim 1 or 2, further comprising an encoder (6) arranged to encode the number components (14) of the output values ​​according to the number format in order to determine the output value (16) as a bit sequence having the processing bit length. [4] A circuit according to claim 3, wherein the encoder (6) is further configured to adapt the bit length of the bit sequence of the output value (16) so that it has a predetermined output bit length or a configurable output bit length. [5] A circuit according to claim 4, wherein Posit is used as the number format; wherein the encoder (6) is arranged to adapt the bit sequence of the output value (16) by removing bits if the processing bit length is greater than the output bit length. [6] A circuit according to any preceding claim, wherein the operation is a multiply-add operation or a fused multiply-add operation. [7] A circuit according to any preceding claim, wherein Posit is used as the number format; wherein the decoder (2) is arranged to remove one or more bits from the bit sequence of an input value (10) if it has a bit length greater than the processing bit length in order to adjust the bit length. [8] A circuit according to any preceding claim, wherein Posit is used as the number format; wherein the number components (12, 14) of the input values ​​and the output values ​​respectively comprise a sign component (12s, 14s), a magnitude component (12g, 14g), a fractional component (12f, 14g) and an additional component (12z, 14z) indicating whether a specific value is present. [9] A method for performing a mathematical operation by means of a circuit (1) according to one of the preceding claims, wherein the mathematical operation is to be applied to a plurality of input values ​​(101, 102, ..., 10M) which are encoded as bit sequences of the predetermined numerical format, wherein the bit sequence of at least one input value has a bit length which is smaller than the processing bit length, wherein the input values ​​(101, 102, ..., 10M) are transmitted (110) as bit sequences of the numerical format to an input register (21) of the circuit; wherein the decoder (2) of the circuit is controlled (120) to adapt the bit lengths of the bit sequences of the input values ​​and to decode the adapted bit sequences in order to determine numerical components of the corresponding input value; wherein the processing unit (4) is controlled (130) to determine numerical components of an output value (16) from the numerical components of the input values. [10] Method according to claim 9, wherein the circuit (1) comprises the features of claim 3; wherein the encoder (2) is controlled (140) to encode the number components of the output values ​​(16) according to the number format in order to determine the output value as a bit sequence having the processing bit length; wherein, if the circuit (1) comprises the features of claim 4, the encoder (2) is controlled (140) to adapt the bit length of the bit sequence of the output value (16) so that it is equal to the output bit length. [11] Computing unit comprising a circuit (1) according to one of claims 1 to 8 and configured to carry out all method steps of a method according to claim 9 or 10.

Images