Methods and apparatus for floating point processing

Abstract

A method and an apparatus for floating point processing are provided. The method includes receiving a first floating point operand and a second floating point operand, each having an n-bit format including a sign field, an exponent field, and a significand field; normalizing a binary value obtained by performing an arithmetic operation on fields corresponding to each other in the first floating point operand and the second floating point operand for an n-bit multiplication operation; determining whether the normalized binary value is a number representable in the n-bit format or an extended normal number not representable in the n-bit format; encoding the normalized binary value using an extended bit format in which an extension code identifying whether the normalized binary value is the extended normal number is added to the n-bit format according to a result of the determination; and outputting the encoded binary value using the extended bit format as a result of the n-bit multiplication operation.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to Korean Patent Application No. 10-2019-0178509, filed on December 30, 2019, with the Korean Intellectual Property Office, the entire disclosure of which is incorporated herein by reference for all purposes. Technical Field

[0003] The following description relates to methods and apparatus for floating-point processing. Background Technology

[0004] A neural network can be a computing system implemented based on a reference hardware computing architecture. Neural network technology can analyze input data and extract useful information from it.

[0005] Neural network devices typically require extensive computation on complex input data. To enable a typical neural network device to handle such computations, it is often necessary to perform operations that read or write large amounts of data from or to memory, which can consume significant amounts of energy due to frequent memory accesses. Low-power and high-performance systems, such as mobile or Internet of Things (IoT) devices, typically have limited resources and therefore often require techniques to reduce the energy consumption needed to process large amounts of data. Summary of the Invention

[0006] This summary is provided to introduce some concepts in a simplified form, which will be further described below in detail in the specific embodiments. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to help define the scope of the claimed subject matter.

[0007] In one general aspect, a processor-implemented method includes: receiving a first floating-point operand and a second floating-point operand, both having an n-bit format including a sign field, an exponent field, and a significant digit field, where n is a natural number; normalizing a binary value obtained by performing arithmetic operations on corresponding fields of the first and second floating-point operands for n-bit multiplication; determining whether the normalized binary value is a number that can be represented in the n-bit format or an extended normal number that cannot be represented in the n-bit format; encoding the normalized binary value using an extended bit format based on the determination result, wherein an extension code indicating whether the normalized binary value is an extended normal number is added to the n-bit format; and outputting the encoded binary value using the extended bit format as the result of the n-bit multiplication operation between the first and second floating-point operands.

[0008] Numbers that can be represented in n-bit format can be normal numbers or subnormal numbers, and extended normal numbers may not be included in the dynamic range of normal numbers or the dynamic range of subnormal numbers.

[0009] Determining may include: determining whether the normalized binary value is a normal number, a subnormal number, or an extended normal number based on the exponent of the normalized binary value.

[0010] Arithmetic operations can include convolution operations.

[0011] When the normalized binary value is a normal or subnormal number that can be represented in n-bit format, the extended code can have a first value, and when the normalized binary value is an extended normal number, the extended code can have a second value.

[0012] The dynamic range of extended normal numbers can be a range of positive or negative numbers whose absolute values ​​are less than the absolute values ​​of subnormal numbers that can be represented in n-bit format.

[0013] The determination may include: determining that the normalized binary value is a normal number when the exponent of the normalized binary value is included in the dynamic range of a normal number that can be represented in n-bit format; determining that the normalized binary value is a subnormal number when the exponent of the normalized binary value is not included in the dynamic range of a normal number but is included in the dynamic range of a subnormal number; and determining that the normalized binary value is an extended normal number when the exponent of the normalized binary value is not included in the dynamic range of either a normal number or a subnormal number.

[0014] The dynamic range of extended normal numbers can be a range that represents positive or negative numbers, where the absolute value of a positive or negative number is greater than the absolute value of a subnormal number that can be represented in n-bit format, and less than the absolute value of a normal number that can be represented in n-bit format.

[0015] The determination may include: determining that the normalized binary value can be a normal number when the exponent of the normalized binary value is included in the dynamic range of a normal number that can be represented in n-bit format; determining that the normalized binary value is an extended normal number when the exponent of the normalized binary value is not included in the dynamic range of a normal number but is included in the dynamic range of an extended normal number; and determining that the normalized binary value can be a subnormal number when the exponent of the normalized binary value is not included in the dynamic range of a normal number and is not included in the dynamic range of an extended normal number.

[0016] The value of n can be 16, the n-bit format can be half-precision floating-point format, the n-bit multiplication operation can be a 16-bit multiplication operation using a 16-bit multiplier, and in the n-bit format, the sign field can include 1 bit, the exponent field can include 5 bits, and the significant digits field can include 10 bits.

[0017] Extended normal numbers can be and Positive numbers between, or and Negative numbers between.

[0018] Normalization may include: performing addition on the exponent field of the first floating-point operand and the exponent field of the second floating-point operand; performing subtraction by subtracting the deviation corresponding to the n-bit format from the result of the addition; performing multiplication on the significant digit field of the first floating-point operand and the significant digit field of the second floating-point operand; and normalizing the binary value obtained based on the result of the subtraction and the result of the multiplication.

[0019] An n-bit multiplication operation can correspond to a portion of the multiplication-accumulation (MAC) operation in a neural network, and the encoded binary value can be provided to the accumulation operation with an m-bit third floating-point operand, where m can be a natural number greater than n.

[0020] In another general aspect, a non-transitory computer-readable storage medium stores instructions that, when executed by one or more processors, cause one or more processors to perform the methods described above.

[0021] In another general aspect, an apparatus includes one or more processors configured to: receive a first floating-point operand and a second floating-point operand, both having an n-bit format including a sign field, an exponent field, and a significant digit field, where n is a natural number; normalize a binary value obtained by performing arithmetic operations on corresponding fields of the first and second floating-point operands for n-bit multiplication; determine whether the normalized binary value is a number that can be represented in n-bit format or an extended normal number that cannot be represented in n-bit format; based on the determination, encode the normalized binary value using an extended bit format, in which an extension code indicating whether the normalized binary value is an extended normal number is added to the n-bit format; and output the encoded binary value using the extended bit format as the result of an n-bit multiplication operation between the first and second floating-point operands.

[0022] The device may also include a memory that stores instructions that, when executed by one or more processors, configure one or more processors to perform normalization, determination, encoding, and output of encoded binary values.

[0023] Numbers that can be represented in n-bit format can be normal numbers or subnormal numbers, and extended normal numbers may not be included in the dynamic range of normal numbers or the dynamic range of subnormal numbers.

[0024] One or more processors can also be configured to determine whether a normalized binary value is a normal number, a subnormal number, or an extended normal number based on the exponent of the normalized binary value.

[0025] Arithmetic operations can include convolution operations.

[0026] When the normalized binary value is a normal or subnormal number that can be represented in n-bit format, the extended code can have a first value, and when the normalized binary value is an extended normal number, the extended code can have a second value.

[0027] The dynamic range of extended normal numbers can be a range of positive or negative numbers whose absolute values ​​are less than the absolute values ​​of subnormal numbers that can be represented in n-bit format.

[0028] The dynamic range of extended normal numbers can be a range that represents positive or negative numbers, where the absolute value of a positive or negative number is greater than the absolute value of a subnormal number that can be represented in n-bit format, and less than the absolute value of a normal number that can be represented in n-bit format.

[0029] The value of n can be 16. The n-bit format can be half-precision floating-point format. An n-bit multiplication operation can be a 16-bit multiplication operation using a 16-bit multiplier. In the n-bit format, the sign field can include 1 bit, the exponent field can include 5 bits, the significant digits field can include 10 bits, and the extended normal number can be... and Positive numbers between, or and Negative numbers between.

[0030] One or more processors may include a multiply-accumulate (MAC) unit configured to perform MAC operations, wherein the MAC unit may include an n-bit multiplier and an m-bit accumulator for performing n-bit multiplication, where m may be a natural number greater than n, wherein the n-bit multiplier may include: an adder configured to perform addition on the exponent fields of a first floating-point operand and a second floating-point operand; a subtractor configured to perform subtraction by subtracting a deviation corresponding to the n-bit format from the result of the addition; a multiplier configured to perform multiplication on the significant digit fields of a first floating-point operand and a second floating-point operand; and a normalizer configured to normalize the binary value obtained based on the result of the subtraction and the result of the multiplication, and wherein the MAC unit may be configured to perform an accumulation operation on the encoded binary value and the m-bit third floating-point operand using the accumulator.

[0031] In another general aspect, a processor implementation method includes: normalizing a binary value obtained by performing arithmetic operations on corresponding fields of a first and second floating-point operands for an n-bit multiplication operation, wherein the n-bit multiplication operation corresponds to part of a multiply-accumulate (MAC) operation of a neural network; encoding the normalized binary value using an extended bit format based on determining whether the normalized binary value is a number that can be represented in an n-bit format or an extended normal number that cannot be represented in an n-bit format; in the extended bit format, an extension code for identifying whether the normalized binary value is an extended normal number is added to the n-bit format; and outputting the encoded binary value using the extended bit format as the result of the n-bit multiplication operation between the first and second floating-point operands for use in a neural network.

[0032] Numbers that can be represented in n-bit format can be normal numbers or subnormal numbers, and extended normal numbers may not be included in the dynamic range of normal numbers or the dynamic range of subnormal numbers.

[0033] When the normalized binary value is a normal or subnormal number that can be represented in n-bit format, the extended code can have a first value, and when the normalized binary value is an extended normal number, the extended code can have a second value.

[0034] The dynamic range of extended normal numbers can be a range of positive or negative numbers whose absolute values ​​are less than the absolute values ​​of subnormal numbers that can be represented in n-bit format.

[0035] The method may also include generating a first floating-point operand and a second floating-point operand, both of which have an n-bit format, including a sign field, an exponent field, and a significant digit field, where n can be a natural number.

[0036] The first and second floating-point operands can be used to generate the activation output of the previous layer of the neural network.

[0037] Other features and aspects will become clear from the following detailed description, drawings, and claims. Attached Figure Description

[0038] Figure 1 This is a diagram illustrating the architecture of a neural network according to one or more embodiments.

[0039] Figure 2 This is a diagram illustrating a neural network according to one or more embodiments.

[0040] Figure 3 This is a diagram illustrating the use of mixed precision to perform training in a neural network according to one or more embodiments.

[0041] Figure 4 This is a diagram illustrating a mixed-precision multiply-accumulate (MAC) operation according to one or more embodiments.

[0042] Figure 5 This is a diagram showing the format and decoding table of half-precision floating-point (FP16) as defined in the Institute of Electrical and Electronics Engineers (IEEE) 754 standard.

[0043] Figure 6 This is a diagram showing a histogram of activation gradients according to one or more embodiments.

[0044] Figure 7 This is a block diagram illustrating a processing apparatus according to one or more embodiments.

[0045] Figure 8 This is a diagram illustrating mixed-precision MAC operations using a half-precision floating-point (FP16) multiplier according to one or more embodiments.

[0046] Figure 9 This is a diagram illustrating an extended bit format method according to one or more embodiments.

[0047] Figure 10 This is a diagram illustrating an FP multiplier according to one or more embodiments.

[0048] Figure 11 This is a flowchart illustrating the encoding of the result of an FP multiplication operation using an extended bit format method according to one or more embodiments.

[0049] Figure 12 This is a diagram illustrating an extended bit format method according to one or more embodiments.

[0050] Figure 13 This is a flowchart illustrating the encoding of the result of an FP multiplication operation using an extended bit format method according to one or more embodiments.

[0051] Figure 14 This is a diagram illustrating an extension of the FP range, which can be represented by an FP16 multiplier, using an extended bit format according to one or more embodiments.

[0052] Figure 15 This is a flowchart of a method for processing using FP operations performed by a processing device according to one or more embodiments.

[0053] Figure 16 This is a block diagram illustrating an electronic system according to one or more embodiments.

[0054] Throughout the accompanying drawings and detailed description, the same reference numerals refer to the same elements. The drawings may not be drawn to scale, and for clarity, illustration, and convenience, the relative dimensions, scale, and depiction of elements in the drawings may be enlarged. Detailed Implementation

[0055] The following detailed description is provided to aid the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding the disclosure of this application. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but may be significantly altered upon understanding the disclosure of this application, except for operations that must be performed in a certain order. Furthermore, for clarity and conciseness, descriptions of known features may be omitted upon understanding the disclosure of this application.

[0056] The features described herein may be implemented in various forms and are not to be construed as limited to the examples described herein. Rather, the examples described herein are provided merely to illustrate some of the many feasible ways of implementing the methods, apparatus, and / or systems described herein, which will become apparent upon understanding the disclosure of this application.

[0057] Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings, wherein the same reference numerals refer to the same elements throughout the drawings. In this respect, embodiments may take different forms and should not be construed as being limited to the description set forth herein. Therefore, embodiments are described below only by reference to the accompanying drawings to explain various aspects. The term “and / or” as used herein includes any and all combinations of one or more of the associated listed items. Expressions such as “at least one of…” modify the entire list of elements when following a list of elements, rather than modifying individual elements in the list.

[0058] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as would be commonly understood by one of ordinary skill in the art to which this disclosure pertains based on an understanding of the disclosure herein. Terms such as those defined in common dictionaries shall be interpreted as having the same meaning as in the context of the relevant technology and / or this application, and shall not be interpreted as having an ideal or overly formal meaning, unless expressly defined herein.

[0059] Terms such as “comprising” or “including” used in the embodiments should not be construed as necessarily including all of the various components or operations described in the specification, but should be construed as potentially excluding some of the components or operations, or potentially including additional components or operations. In this document, the use of the term “may” with respect to examples or embodiments (e.g., what an example or embodiment may include or implement) means that there exists at least one example or embodiment in which such features are included or implemented, and that all examples are not limited thereto.

[0060] The present disclosure will now be described more fully with reference to the accompanying drawings. However, the present disclosure may be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein.

[0061] Figure 1 This is a diagram illustrating the architecture of a neural network according to one or more embodiments.

[0062] refer to Figure 1 Neural network 1 can have an architecture of a deep neural network (DNN) or an n-layer neural network. As a non-limiting example, a DNN or an n-layer neural network can correspond to one of the following: a convolutional neural network (CNN), a recurrent neural network (RNN), a deep belief network, a restricted Boltzmann machine, etc. For example, neural network 1 can be implemented as a CNN, but the type of neural network 1 is not limited to CNN.

[0063] Relative to the input image, the neural network 1 can be implemented with a multi-layered computational architecture. In the neural network 1, convolution operations using filters (also called weight kernels) can be performed on the input image. The result is the generation of an output feature map. This generated output feature map can then be used as the input feature map for the next layer. Convolution operations with corresponding kernels can be performed on the corresponding output feature maps of different convolutional layers to generate new output feature maps. As a result of performing the convolution operations, the final output can be the result of recognizing the characteristics of the input image through the neural network 1.

[0064] For example, when an image with a size of 24×24 pixels is input into... Figure 1 In neural network 1, the input image can be output as four-channel feature maps, each with a size of 20×20 pixels, by performing convolution operations using four kernels. Subsequently, the size of each of the 20×20 feature maps can be gradually reduced by repeating convolution operations using the corresponding kernels, ultimately outputting features of size 1×1 pixels. In one or more such embodiments, neural network 1 can filter and output robust features representing various aspects of the entire image from the input image by performing multiple convolution operations with corresponding pooling or subsampling operations in each layer, and then obtain a recognition result about the input image based on the final output features.

[0065] Figure 2 This is a diagram illustrating operations performed in a neural network according to one or more embodiments.

[0066] refer to Figure 2 Neural network 2 can have a structure including an input layer, one or more hidden layers, and an output layer. Neural network 2 can perform operations based on received input data (e.g., I1 and I2) and can generate output data (e.g., O1 and O2) based on the results of the operations.

[0067] In one or more embodiments, as described above, the neural network 2 can be a DNN or an n-layer neural network including two or more hidden layers. For example, as... Figure 2 As shown, neural network 2 can be a DNN comprising an input layer (layer 1), two hidden layers (layers 2 and 3), and an output layer (layer 4). Compared to neural networks with a single layer, neural network 2 can handle more complex datasets because, when implemented as a DNN architecture, it can include more layers capable of processing information. Furthermore, neural network 2... Figure 2 It is shown as including four layers, but Figure 2 These are merely non-limiting examples, and neural network 2 can also be represented as a neural network with fewer or more than 4 layers, and can also include more than... Figure 2One or more embodiments of the neural network 2 may show fewer or more channels, each channel having one or more artificial nodes. Therefore, the neural network 2 may include a variety of numbers of layers.

[0068] Each layer included in neural network 2 may include multiple channels (e.g., each channel represents one or more nodes, also referred to as corresponding artificial neurons, processing elements (PEs), or units). Figure 2 As shown, for example, the input layer "Layer 1" may include two channels (nodes) that interact with the hidden layers "Layer 2" and "Layer 3", each of which may include three channels (nodes). However, this is only a non-limiting example, and in one or more embodiments, each layer of the neural network 2 may include a variety of numbers of channels (and corresponding one or more nodes).

[0069] The channels (nodes) included in each layer of neural network 2 can be connected to other channels (nodes), and all of these channels are capable of processing data. For example, any channel (node) can receive data from other channels or nodes (such as channels (nodes) in the previous layer) to compute data, and output the result of such computation to the channels (nodes) in the next layer.

[0070] For a given channel (node), the input and output of each channel (node) can be referred to as the input activation and output activation, respectively. That is, the output activation can be the output of a channel (node) and simultaneously a parameter serving as the input activation corresponding to one or more channels (nodes) included in the subsequent layer. Furthermore, each channel (node) can determine and output its activation based on the input activations received from the channels (nodes) included in the previous layer and the connection weights. Each weight is a parameter used to calculate the output activation and the corresponding input activation in each channel (node), and can be a value assigned to the connection relationships between channels (or, corresponding nodes).

[0071] Each channel in the channel (or each node or multiple nodes represented by each shown channel of the layer) can be processed by a computing unit or processing element that receives input and activates output. Such a computing unit or processing element can be implemented in various ways by one or more hardware processors or one or more hardware computers. The input and output of each channel (node) can be appropriately mapped by such processor. For example, when σ represents the activation function, This represents the weight between the k-th channel (or node) included in the (i-1)-th layer and the j-th channel (or node) included in the (i-1)-th layer. This represents the bias of the j-th channel included in the i.1-th layer, and The activation of the j-th channel (or node) in the i-th layer can be calculated as shown in Equation 1 below.

[0072] Equation 1:

[0073]

[0074] like Figure 2 As shown in one or more embodiments, the activation of the first channel CH1 (or the first node) of the hidden layer "Layer 2" can be represented as Moreover, according to Equation 1, It can have a value However, Equation 1 is merely a non-limiting example for calculating activation and its application to weights in processing data in Neural Network 2, but the equations for activation are not limited to Equation 1. Activation can refer to the value obtained by passing the value of the activation function applied to the sum of activations received from the previous layer to the rectified linear unit (ReLU) (e.g., through appropriate weighted connections).

[0075] As described above, in Neural Network 2, large datasets are exchanged between multiple interconnected channels (or nodes) and undergo extensive computational processing through multiple incremental layers. In such an example, because the data can correspond to floating-point values ​​of various precisions, Neural Network 2 can advantageously implement techniques that minimize the loss of accuracy while also reducing the computational burden required to process complex input data such as floating-point values.

[0076] Figure 3 This is a diagram illustrating the training performed in a neural network using mixed precision, based on an example.

[0077] refer to Figure 3 As a non-limiting example, neural networks (e.g., Figure 1 Neural network 1 or Figure 2 The neural network 2) can perform feedforward propagation (FWD) 302 and backpropagation (BWD) 303 and 304 for neural network training. Here, although FWD 302 is referenced in the context of feedforward propagation (whether feedforward, recursive, bidirectional, convolution, pooling, etc.), FWD 302 is also applicable (and represents) inference operation embodiments in the training of the neural network, for example, with respect to training input data, compared to backpropagation errors on the layers where BWD 303 / 304 operations are performed, where such forward propagation is similarly preferred for non-training data.

[0078] FWD 302 can be a process that performs convolution operations between input activations and their corresponding weights. Through FWD 302, the output activations of each layer can be obtained, and ultimately, the output of the neural network can be obtained. The input activations and corresponding weights of FWD 302 can be in half-precision floating-point (FP) format, and the output activations of each layer can also be in half-precision FP format. Half-precision FP can correspond to the format of 16-bit floating-point values ​​(FP16).

[0079] Neural networks can support mixed precision. For example, while a convolution operation between activations (FP16) and weights (FP16) can be performed during FWD 302, the intermediate arithmetic results of the convolution operation can be obtained in a floating-point format with different precision (e.g., single precision (FP32)). As described in detail above, the final output activation can be output in FP16 format.

[0080] The loss function can then be generated by comparing the output with the expected value. Therefore, BWD 303 and 304 can be used to perform training to minimize the loss of the loss function.

[0081] BWD (BWD-Activation) 303 for activation can be a process of propagating the FP16 activation gradient and FP16 weights calculated based on the loss function. BWD (BWD-Weight) 304 for weights can be a process of propagating the FP16 activation gradient Activation Grad and FP16 activation calculated based on the loss function. As a result of performing BWD 303 and 304, the FP16 activation gradient Activation Grad and the FP16 weight gradient Weight Grad can be output.

[0082] The FP16 weight gradient (Weight Grad) output by BWD-weight 304 can be used for weight updates 305 for each layer of the neural network. Weight updates 305 can be a process of updating the corresponding sovereign weights (e.g., corresponding to the weights applied in FWD 302) based on the FP16 weight gradient (Weight Grad) output by BWD-weight 304. In such an example, both the sovereign weight and the updated sovereign weight can correspond to parameters in FP32 format.

[0083] The FP32 sovereign weight can be quantized into FP16 format in Operation 301 and provided to each layer.

[0084] Alternatively, the forward and backward propagation processes performed during neural network training can be executed through operations on parameters with mixed precision. Higher precision can improve the accuracy of the neural network, but increases the computational cost and processing time. Conversely, lower precision can reduce accuracy, but reduces computational cost and processing time. Therefore, neural networks supporting mixed precision can achieve efficient computation while helping to ensure a certain level of accuracy.

[0085] In addition, Figure 3 For ease of illustration, as a non-limiting example, a format utilizing mixed precision of FP16 and FP32 will be used, but there are also examples that support the use of mixed precision with other precision values.

[0086] Figure 4 This is a diagram illustrating a mixed-precision multiply-accumulate (MAC) operation 400 according to one or more embodiments.

[0087] refer to Figure 4 In each layer of the neural network, a MAC operation 400 can be performed to perform a corresponding convolution between the corresponding input activation and weights. In such an example, the MAC operation 400 can be a mixed-precision operation. The MAC operation 400 can support mixed-precision formats such as FP16 and FP32.

[0088] Specifically, in a non-limiting example, the MAC operation 400 may include an FP multiplication operation 401, an FP accumulation operation 402, and a quantization operation 403. The FP multiplication operation 401 may be a multiplication operation between FP16 input activations and FP16 weights. The FP multiplication operation 401 may be performed by an FP32 multiplier. The multiplication result can then be output from the FP32 multiplier in FP32 format. The FP accumulation operation 402 may be performed on the FP32 format multiplication result using an FP32 accumulator, and the accumulation result can be output in FP32 format. The FP32 format accumulation result is then quantized to FP16 format by the quantization operation 403, and such a quantization result may correspond to the example output activations of the corresponding layer (e.g., multiple output activations).

[0089] As described above, FP32 multipliers and FP32 accumulators can be used to process MAC operations 400 that support mixed precision, such as FP16 and FP32. In such an example, the FP32 multiplier can correspond to a full-precision multiplier relative to FP16 operands (e.g., FP16 input activation and FP16 weights). Therefore, the multiplication result of FP16 operands can be obtained without loss of precision. However, FP32 multipliers or single-precision multipliers may have a larger circuit area and higher power consumption than FP16 multipliers or half-precision multipliers. As a non-limiting example, in experiments, the area of ​​an FP16 multiplier can be reduced to about 37.5% of the area of ​​an FP32 multiplier, and the power consumption of an FP16 multiplier can be reduced to about 61.0% of the power consumption of an FP32 multiplier. However, because the precision of an FP16 multiplier may be lower than that of an FP32 multiplier, the accuracy of the multiplication operation may be reduced as a result. Therefore, in an implementation of a MAC arithmetic unit for supporting mixed precision (e.g., FP16 and FP32) MAC arithmetic 400, a solution that achieves efficiency in area and power consumption while ensuring the accuracy of the MAC arithmetic unit may be desired.

[0090] Figure 5 This is a diagram showing the format of half-precision floating-point (FP16) 510 as defined in the Institute of Electrical and Electronics Engineers (IEEE) 754 standard and the corresponding decoding table 520.

[0091] refer to Figure 5 The FP16510 format is defined by the IEEE 754 standard and corresponds to the binary bit format used for storing 16-bit floating-point data in memory. Figure 5 In the FP16510 format, 1 bit

[15] corresponds to the sign field, 5 bits [14:10] correspond to the exponent field, and 10 bits [9:0] correspond to the significant and (or, fractional) field. For example, when a floating-point value is expressed as "a×2 b When expressed in the form of "b", the sign of "a" can correspond to the sign field, the absolute value of "a" can correspond to the significant number, and "b" can correspond to the exponent. In such an example, "a" can refer to the significant number used in floating-point values.

[0092] The numbers that can be represented in the FP16510 format can be further interpreted relative to the decoding table 520.

[0093] According to decoding table 520, when the exponent of a floating-point value is in the range of "000012 to 111102", the floating-point value can correspond to a normal number. The expression used to derive a normal number can be (-1). signbit ×2exponent-15 ×1.significantbits2.

[0094] When the exponent of a floating-point value is 111112, depending on the significant number, this floating-point value can be interpreted as corresponding to an infinite or not-a-number (NaN).

[0095] A floating-point value can correspond to 0 when the exponent is 000002 and the significant digits are 0. However, when the exponent is 000002 and the significant digits are not 0, the floating-point value can correspond to a subnormal number, also known as a nonnormal number. An expression representing a subnormal number can be (-1). signbit ×2 -14 ×0.significantbits2.

[0096] Therefore, according to decoding table 520, the maximum and minimum values ​​of positive normal numbers and positive subnormal numbers that can be represented in FP16 510 format are as follows.

[0097] The maximum value of a positive normal number can be represented as:

[0098] The minimum value of a positive normal number can be expressed as:

[0099] The maximum value of a positive normal number can be expressed as:

[0100] The minimum value of a positive normal number can be expressed as:

[0101] Figure 6 This is a diagram showing a histogram 600 of activation gradients according to one or more embodiments.

[0102] refer to Figure 6 In one or more embodiments, histogram 600 shows the activation gradient magnitude, which is shown using a log2 (magnitude) scale and is obtained during FP32 training of a multi-box SSD network such as a single-shot multi-box detector (SSD).

[0103] Referring to histogram 600, in the activation gradient magnitude, the magnitude of 0 is distributed at approximately 64%.

[0104] Based on the above reference Figure 5The described decoding table 520, within the FP 16 representation range 610, has a dynamic range of 2 for positive normal numbers that can be represented in FP16 510 format. -14 Up to 2 15 The dynamic range of positive normal numbers that can be represented in FP16 510 format is 2. -24 Up to 2 -15 .

[0105] Therefore, when converting the activation gradient magnitudes obtained during FP32 training to FP16, all FP32 floating-point values ​​(approximately 31%) included in the dynamic range of 630 can be converted to the value 0. This is because the values ​​included in the dynamic range of 630 are... Figure 6 The FP32 floating-point values ​​in the dynamic range 630 part cannot be represented in the FP16 510 format, so the above conversion can be performed, which can then simply truncate them to have a value of 0.

[0106] In the following, a low-precision FP multiplier, such as an FP16 FP multiplier, will be described in more detail according to one or more embodiments, wherein the FP multiplier is capable of representing floating-point values ​​that cannot be represented in low precision (e.g., FP32 floating-point values ​​included in dynamic range 630 in the example above).

[0107] Figure 7 This is a block diagram illustrating the hardware configuration of a processing apparatus 100 according to one or more embodiments.

[0108] refer to Figure 7 The processing device 100 may include at least one processor 110 and at least one memory 120. Figure 7 The processing apparatus 100 shown herein is illustrated with only some components from one or more embodiments, while in other embodiments, components are shown in addition to or in place of others. Figure 7 The processing device 100 may include other general-purpose components in addition to the non-limiting components.

[0109] Processing device 100 may correspond to a computing device that implements a neural network, such as by performing the example operations described above on a neural network. For example, processing device 100 may be a personal computer (PC), server device, mobile device, etc., or alternatively may correspond to an accelerator (such as a hardware accelerator) configured to perform neural network operations in such a device. Furthermore, processing device 100 may be a large system or may be set in the context of a large system, such as an autonomous vehicle, robot, smartphone, tablet device, augmented reality (AR) device, Internet of Things (IoT) device, etc., and uses one or more neural networks to perform speech recognition and / or image recognition. However, processing device 100 is not limited to these specific examples, and as a non-limiting example, it may correspond to various other devices or various types of processing devices, and perform neural network operations in such devices.

[0110] Processor 110 may be hardware configured to perform overall control functions for controlling the operation of processing device 100. For example, processor 110 may typically control processing device 100 by processing or executing instructions and / or data stored in memory 120 of processing device 100. Such processor 110 disposed in processing device 100 may be implemented as a central processing unit (CPU), graphics processing unit (GPU), application processor (AP), neural processing unit (NPU), tensor processing unit (TPU), etc., but is not limited to these listed embodiments, and processor 110 may be implemented with other types of processors in other embodiments. Moreover, processor 110 is not limited to a single processor, but may also be implemented using multiple processors or multi-core processors.

[0111] Memory 120 may be hardware that stores various neural network data and / or parameters to be processed in processor 110. For example, memory 120 may store input / output feature map data, convolutional data, and other similar data processed in the neural network. Additionally, memory 120 may store various applications to be driven by processor 110. As another example, memory 120 may store instructions that, when executed by processor 1120, configure processor 110 to perform one or more of the operations or methods described herein.

[0112] The memory 120 may correspond to one or more memory devices, such as random access memory (RAM), read-only memory (ROM), hard disk drive (HDD), solid-state drive (SSD), compact flash memory (CF), secure digital (SD), micro-secure digital (micro-SD), mini-secure digital (Mini-SD), extreme digital (xD), memory stick, etc., and the type of memory 120 may be various types of memory and is not limited to the examples listed above.

[0113] The processor 110 may be implemented as at least one processor core for performing inference and / or training processing of the neural network as described above. For example, the processor 110 may include an FP multiplier 1110 and an FP accumulator 1120 for handling MAC operations of convolution operations. Alternatively, the processor 110 may include a logic circuit configuration that uses FP to perform inference and / or training processing. Furthermore, the processor 110 may perform various operations for implementing the neural network, while supporting mixed-precision operations.

[0114] The FP multiplier 1110 configured for MAC operations in processor 110 can be a half-precision floating-point (FP16) multiplier. That is, with... Figure 5 Compared to the FP32 (high-precision or higher-precision) multiplier described herein, the FP multiplier 1110 can correspond to low (or lower) precision, such as an FP16 multiplier. Therefore, compared to operations performed using high precision (e.g., an FP32 multiplier), the area occupied by the FP multiplier 1110 in the processor 110 can be reduced, and the power consumption of the operation can be relatively reduced. When the FP multiplier 1110 is implemented as a low-precision (e.g., FP16) multiplier, an extended bit format as the FP format can be used, which adds an extension pin, described in more detail below, to prevent, for example, inclusion of... Figure 6 The loss of FP32 floating-point values ​​in the dynamic range of 630.

[0115] In addition, such as Figure 7 As shown, as a non-limiting example, the FP multiplier 1110 can be integrated by configuring the MAC arithmetic unit 1100 (i.e., the fused multiplication-addition arithmetic unit) together with the FP accumulator 1120 in the processor 110. However, one or more embodiments are not limited thereto, and the FP multiplier 1110 can be implemented as a FP multiplier separately located in the processor 110, independent of the MAC arithmetic unit 1100. That is, the implementation of the FP multiplier in the processor 110 is not limited to any particular example, and various implementations exist in various embodiments.

[0116] The process of performing multiplication operations using the FP multiplier 1110 to perform MAC operations will be described in more detail when the FP multiplier 1110 of the processor 110 corresponds to a low-precision (e.g., FP16) multiplier.

[0117] Processor 110 can receive a first FP operand and a second FP operand in n-bit format to be subjected to n-bit multiplication, where n is a natural number. In such an example, the n-bit formatted FP operand may include a sign field, an exponent field, and a significant digit field. In the following description, for ease of description, as a non-limiting example, it is assumed that n = 16, the n-bit format may be a half-precision FP format, the n-bit multiplication operation may be a 16-bit multiplication using a 16-bit multiplier (i.e., FP multiplier 1110), and in the n-bit multiplication operation, the sign field may be 1 bit, the exponent field may be 5 bits, and the significant digit field may be 10 bits. However, the embodiment is not limited to this example, and in other examples, n may correspond to a number of bits other than 16.

[0118] Processor 110 can normalize the binary value obtained by arithmetic operations on corresponding fields of FP operands used for n-bit multiplication. More specifically, FP multiplier 1110 of processor 110 can perform addition on the exponent fields of the first FP operand and the second FP operand, and can perform subtraction on the result of the addition by subtracting the deviation corresponding to the n-bit format. In such an example, when the FP operand is in FP16 format, the deviation can be 011112 = 15. Next, FP multiplier 1110 of processor 110 can perform multiplication on the significant digit fields of the first FP operand and the second FP operand. Based on the result of the subtraction and the result of the multiplication, FP multiplier 1110 of processor 110 can normalize the obtained binary value. Binary value normalization refers to the process of converting the significant digits of the binary value to the form 0.significantbits2 or 1.significantbits2, and then converting the exponent of the binary value to a power of 2.

[0119] Processor 110 can determine whether the normalized binary value is a number that can be represented in n-bit format or an extended normal number that cannot be represented in n-bit format. In such an example, a number that can be represented in n-bit format can refer to a normal number or a subnormal number. An extended normal number can refer to a number that is not included in the dynamic range of normal numbers or the dynamic range of subnormal numbers. That is, an extended normal number can be... Figure 6 The number corresponding to a single-precision FP or FP32 value with a dynamic range of 630, which cannot be represented in the format of a half-precision FP or FP16 value as defined by, for example, the IEEE 754 standard.

[0120] Based on the determined result, processor 110 can encode the normalized binary value using an extended bit format, in which an extension code for identifying whether the normalized binary value is an extended normal number is added to the n-bit format. In other words, processor 110 can extend the previous FP16 format by using the extension code, thereby enabling the representation of FPs in the dynamic range that cannot be represented by normal or subnormal numbers when using the previous FP16 format.

[0121] Processor 110 can output a binary value encoded in an extended bit format as the result of an n-bit multiplication operation between FP operands. Subsequently, processor 110 can perform other operations, such as accumulation, on the result of the n-bit multiplication operation obtained as described above. For example, the result of the n-bit multiplication operation corresponding to the encoded binary value can be provided to an accumulation operation with a third FP operand having m bits, where m is a natural number greater than n.

[0122] Thus, even when using an n-bit multiplier instead of a 2×n-bit multiplier to handle multiplication of n-bit FP operands, the processor 110 can minimize the accuracy loss of n-bit multiplication operations, reduce the area of ​​the circuit implementation for the multiplier in the processor 110, and also reduce the power consumption of the multiplier.

[0123] Figure 8 This is a diagram illustrating the use of a half-precision FP (FP16) multiplier to perform a mixed-precision MAC operation 800 according to an example.

[0124] As a non-limiting example, processors (e.g.) Figure 7 The processor 110 may include a MAC unit configured to perform MAC operations on a neural network. The MAC unit may include an n-bit multiplier and an m-bit accumulator for performing n-bit multiplication operations. For example, in Figure 8 In one or more embodiments, n = 16, and m is a natural number greater than n.

[0125] refer to Figure 8 The multiplication operation 810 between the FP16 activation and the FP weights can correspond to a portion of the MAC operation 800 to be processed in the neural network. In one or more such examples, with, for example Figure 4 Unlike single-precision FP or FP32 multipliers, half-precision FP or FP16 multipliers can be used to perform multiplication operations 810. As a result of multiplication operation 810, the FP16 multiplier can output a binary value encoded in FP16 format and an extended bit format including a 1-bit extension code, wherein the FP16 format includes a sign bit (1 bit), an exponent bit (5 bits), and a significant bit (10 bits) field.

[0126] Subsequently, an accumulation operation 820 can be performed on the multiplication result of the extended bit format of FP16+1b with the multiplication result of different FP32 formats using mixed precision. That is, according to one or more embodiments, this operation can be performed using an m-bit accumulator assuming m=32. Therefore, an accumulation operation 820 can be performed on the encoded binary value and the m-bit third FP operand in one or more embodiments.

[0127] The accumulated result in FP32 format can be quantized to FP16 format using quantization operation 830. The corresponding quantization result can be propagated to the next layer and used as the output activation of the corresponding layer.

[0128] Figure 9 and Figure 11 This is a detailed diagram illustrating an extended bit format method according to one or more embodiments. Figure 12 and Figure 13 This is a detailed diagram illustrating an extended bit format method according to one or more embodiments.

[0129] Figure 9 This is a diagram illustrating an extended bit format method according to one or more embodiments.

[0130] refer to Figure 9 The extended bit format method can be a format for representing positive or negative numbers whose absolute value is less than the absolute value of a subnormal number that can be represented in n-bit (e.g., n = 16) format. In other words, an extended normal number according to this method's extended bit format can correspond to a value less than a positive subnormal number or greater than a negative subnormal number.

[0131] Referring to the decoding table 910 of the extended bit format according to this method, when the binary value normalized by the FP16 multiplier is a normal or subnormal number that can be represented in FP16 format, the value of the extension code can be "0". When the binary value normalized by the FP16 multiplier is an extended normal number, the value of the extension code can be "1". Alternatively, the extension code can be a 1-bit identifier or flag that can distinguish whether the binary value normalized by the FP16 multiplier is an extended normal number.

[0132] When the extension code is "0", it can be used in conjunction with, for example, the above-mentioned... Figure 5 The decoding table 520 is interpreted in a similar manner to the decoding table defining 0, subnormal numbers, normal numbers, infinity, and NaN. However, unlike decoding table 520, in one or more embodiments, according to this extended bit format method, the extended code is "1" and can be added to decoding table 910.

[0133] The FP value with a spread code of "1" can correspond to an extended normal number, and can correspond to a range of values ​​less than a positive normal number or greater than a negative normal number. An expression representing an extended normal number can be (-1). signbit ×2 exponent-25 ×1.significantbits2. Therefore, the extended normal number can be... and Positive numbers between these ranges can also be and Negative numbers between.

[0134] The dynamic range of this extended bit format method, 920, can be from 2... -14 to 2 15 The dynamic range of positive normal numbers, from 2 -24 to 2 -15 The dynamic range of positive normal numbers, and from 2 -56 to 2 -25 The dynamic range of positive extended normal numbers. That is, the extended bit format, which can be used as an extended FP16 format, can represent the FP value between the minimum of a positive normal number and the maximum of a negative normal number by adding one bit of extension code. Therefore, even though half-precision FP or FP16 operand multiplication in MAC operations can be performed by an FP16 multiplier, the accuracy loss relative to the full-precision method can still be minimized, and for example, it can be achieved in... Figure 7 The processor 110 uses a more efficient FP multiplier 1110.

[0135] Figure 10 This is a diagram illustrating the architecture of an FP multiplier 1000 according to one or more embodiments.

[0136] refer to Figure 10 The FP multiplier 1000 can be set in the processor (such as...) Figure 7 The processor 110) contains an FP16 multiplier to perform multiplication operations on half-precision FP or FP16 operands. For example, the multiplier can utilize... Figure 9 The extended bit format method. In a non-limiting example, the FP multiplier 1000 may include XOR logic circuitry 1010, adder 1021, subtractor 1022, multiplier 1030, and normalizer 1040.

[0137] The FP multiplier 1000 can receive a first FP operand and a second FP operand in FP16 format. The FP multiplier 1000 can perform arithmetic operations on corresponding fields in the first FP operand and the second FP operand used for 16-bit multiplication.

[0138] Specifically, in the example, XOR logic circuit 1010 can perform an XOR operation between the sign field a

[15] of the first FP operand and the sign field b

[15] of the second FP operand. Adder 1021 can perform a 5-bit addition operation between the exponent field a[14:10] of the first FP operand and the sign field b[14:10] of the second FP operand. Subtractor 1022 can perform a subtraction operation by subtracting the deviation 011112 (decimal form 15) corresponding to the FP16 format from the result of the 5-bit addition operation.

[0139] Additionally, multiplier 1030 can perform 11-bit multiplication between the significant number field a[9:0] of the first FP operand and the significant number field b[9:0] of the second FP operand. In such an example, since the significant number of the FP operand can be represented as 1.significantbits2, one bit, for example, the value "1", can be added to correspond to a total of 11 bits.

[0140] The normalizer 1040 can normalize a binary value, which can be obtained based on the result of a subtraction operation performed by the subtractor 1022 and the result of a multiplication operation performed by the multiplier 1030. The normalizer 1040 can determine whether the normalized binary value is an extended normal number and can output an extension code (ext_pin[0]) to identify whether the normalized binary value is an extended normal number. In this respect, the processor 110 can determine whether the normalized binary value is a normal number, a subnormal number, or an extended normal number based on the exponent of the normalized binary value.

[0141] The binary value normalized by the normalizer 1040 can be encoded by using an extended bit format including the extended code ext_pin[0] and the FP16 format, and the extended code ext_pin[0] and the encoded binary value p[15:0] can be output based on the result of the FP16 multiplication operation between the first FP operand and the second FP operand.

[0142] Figure 11 This is a flowchart illustrating a method for encoding the result of an FP multiplication operation using an extended bit format method according to one or more embodiments. For example, the extended bit format method may correspond to... Figure 9 and / or Figure 10 The extended bit format method.

[0143] refer to Figure 11 In operation 1101, when a multiplication operation is performed between the first FP operand and the second FP operand in FP 16 format, the FP multiplier 1110 (such as, Figure 7One or more embodiments of the FP multiplier 1110, i.e., corresponding to, for example, the FP16 multiplier included in processor 110, can be obtained by the normalizer 1040 (such as, Figure 10 The exponent of the normalizer (1040) normalizes the binary value, note that although it references... Figure 7 and Figure 10 Examples of components are provided, but the examples are not limited to these.

[0144] In operation 1102, processor 110 can determine whether the exponent of the normalized binary value satisfies the dynamic range condition of "exponent < -14". That is, when the exponent of the normalized binary value is within the dynamic range of normal numbers that can be represented in FP16 format, processor 110 can determine that the normalized binary value is a normal number. Accordingly, when the exponent is greater than or equal to -14, processor 110 can execute operation 1103. However, when the exponent is less than -14, processor 110 can execute operation 1104.

[0145] In operation 1103, when the exponent is greater than or equal to -14, processor 110 can encode the normalized binary value into an extended bit format corresponding to the normal number. In this case, the value of the extended code in the extended bit format can be "0", so ext_pin[0] = 0.

[0146] In operation 1104, processor 110 can determine whether the exponent satisfies the dynamic range condition of "exponent < -24". That is, when the exponent is not included in the dynamic range of normal numbers but is included in the dynamic range of subnormal numbers, processor 110 can determine that the normalized binary value is a subnormal number. Accordingly, when the exponent is greater than or equal to -24, processor 110 can execute operation 1105. However, when the exponent is less than -24, processor 110 can execute operation 1106.

[0147] In operation 1105, when the exponent is greater than or equal to -24, processor 110 can encode the normalized binary value into an extended bit format corresponding to the subnormal number. In this case, the value of the extension code in the extended bit format can be "0", so ext_pin[0] = 0.

[0148] In operation 1106, when the exponent is less than -24, processor 110 can determine that the normalized binary value corresponds to an extended normal number. Alternatively, when the exponent is not included in the dynamic range of the normal number or the dynamic range of the subnormal number, processor 110 can determine that the normalized binary value is an extended normal number. Accordingly, processor 110 can encode the normalized binary value in an extended bit format corresponding to the extended normal number. In this case, in the extended bit format, the value of the extension code can be "1", so ext_pin[0] = 1.

[0149] Figure 12 This is a diagram illustrating an extended bit format method according to one or more embodiments.

[0150] refer to Figure 12 The extended bit format method can be a format for representing positive or negative numbers whose absolute value is greater than the absolute value of the subnormal number and less than the absolute value of a normal number that can be represented in an n-bit format, for example, where n = 16 (as a non-limiting example). That is, an extended normal number according to this extended bit format method can correspond to a value between the subnormal number and the normal number.

[0151] Referring to decoding table 1210, according to this extended bit format method, when the binary value normalized by the FP16 multiplier is a normal number or subnormal number that can be represented in FP16 format, the value of the extended code can be "0", and when the binary value normalized by the FP16 multiplier is an extended normal number, the value of the extended code can be "1".

[0152] For example, when the extension code is "0", the decoding tables for normal numbers, infinity, and NaN can be similar to those above. Figure 5 The decoding table is 520. However, when the extended code is "0", the decoding tables defining 0 and subnormal numbers can also be similar to those above. Figure 5 The decoding table is 520. The reason for this property is that the extended normal number is defined as the value between the subnormal number and the normal number.

[0153] According to decoding table 1210, the expression representing 0 and the second normal number can be (-1). signbit ×2 -46 ×0.significantbits2.

[0154] The FP value with the extension code "1" can correspond to an extended normal number and can have a range of values ​​between the subnormal number and the normal number. The expression for the extended normal number is (-1). signbit ×2 exponent-46 ×1.significantbits2. That is to say, this expression can differ from, for example, representing... Figures 9 to 11The extended normal number expression in the extended bit format of any of the methods. Therefore, return to Figure 12 And with this extended bit format method, the extended normal number can be and Positive numbers between, or and Negative numbers between.

[0155] The dynamic range of this extended bit format method, 1220, can correspond to a range from 2... -14 to 2 15 The dynamic range of positive normal numbers, from 2 -46 to 2 -15 The dynamic range of positively extended normal numbers and from 2 -56 to 2 -47 The dynamic range of positive subnormal numbers. That is, according to one or more embodiments, the extended bit format (also known as the extended FP16 format) can represent the FP value between subnormal numbers and normal numbers by adding 1 bit of extension code.

[0156] Figure 13 This is a flowchart illustrating a method for encoding the result of an FP multiplication operation using an extended bit format method according to one or more embodiments.

[0157] refer to Figure 13 In operation 1301, when a multiplication operation is performed between the first FP operand and the second FP operand in FP 16 format, the FP multiplier 1110 (such as...) Figure 7 One or more embodiments of the FP multiplier 1110, i.e., with, for example, included in, such as Figure 7 The FP16 multiplier in processor 110 (corresponding to the multiplier) can be obtained by normalizer 1040 (such as, Figure 10 The exponent of the normalizer (1040) normalizes the binary value.

[0158] In operation 1302, processor 110 can determine whether the exponent of the normalized binary value satisfies the dynamic range condition of "exponent < -14". That is, processor 110 can determine that the normalized binary value is a normal number when the exponent of the normalized binary value is within the dynamic range of normal numbers that can be represented in FP16 format. Accordingly, when the exponent is greater than or equal to -14, processor 110 can execute operation 1303. However, when the exponent is less than -14, processor 110 can execute operation 1304.

[0159] In operation 1303, when the exponent is greater than or equal to -14, processor 110 can encode the normalized binary value into an extended bit format corresponding to the normal number. In this case, the value of the extension code in the extended bit format can be "0", so ext_pin[0] = 0.

[0160] In operation 1304, processor 110 can determine whether the exponent satisfies the dynamic range condition of "exponent < -46". That is, when the exponent is included in the dynamic range of an extended normal number but not in the dynamic range of a normal number, processor 110 can determine that the normalized binary value is an extended normal number. Accordingly, when the exponent is greater than or equal to -46, processor 110 can execute operation 1305. However, when the exponent is less than -46, processor 110 can execute operation 1306.

[0161] In operation 1305, when the exponent is greater than or equal to -46, processor 110 can encode the normalized binary value into an extended bit format corresponding to the extended normal number. In this case, the value of the extension code in the extended bit format can be "1", such that ext_pin[0] = 1.

[0162] In operation 1306, when the exponent is less than -46, processor 110 can determine that the normalized binary value corresponds to a subnormal number. That is, when the exponent is not included in the dynamic range of the normal number or the dynamic range of the extended normal number, processor 110 can determine that the normalized binary value is a subnormal number. Accordingly, processor 110 can encode the normalized binary value in an extended bit format corresponding to the subnormal number. In this case, in the extended bit format, the value of the extension code can be "0", such that ext_pin[0] = 0.

[0163] In addition, refer to Figures 7 to 13 One or more embodiments of the FP multiplier architecture can be implemented for processing extended bit formats, thereby achieving compatibility with existing FP multiplier architectures. Specifically, the FP multiplier architecture for processing extended bit formats in one or more embodiments may include additional logic circuitry that identifies 1 bit of the extended code data ext_pin[0] from the output of the normalizer in an existing FP multiplier architecture, and can be implemented without otherwise modifying the existing FP multiplier architecture, thus enabling compatibility with existing FP multiplier architectures in one or more embodiments. For example, for example, in addition to in Figures 7 to 13 The FP16 multiplier architecture described in any or all of these documents can support any or all of the extended bit formats described herein, in addition to those described herein. Figures 7 to 13The FP16 multiplier architecture described in one or more embodiments is compatible with existing FP16 multiplier architectures. Additionally, the FP multiplier architecture supporting extended bit formats according to one or more embodiments can be implemented to be compatible with existing FP multiplier architectures of other precisions.

[0164] Figure 14 This is a diagram illustrating an extension of the FP range that can be represented by an FP16 multiplier using an extended bit format according to one or more embodiments.

[0165] refer to Figure 14 One or more embodiments, as referenced above Figure 6 As stated above, when a multiplication operation is performed by an FP16 multiplier, the result of a typical multiplication operation corresponding to an FP included in the dynamic range 630 of histogram 600 is output as 0. However, when a multiplication operation is performed by an FP16 multiplier using an extended bit format defined in one or more embodiments, it is possible to represent an FP in the dynamic range 1410, which may include a portion of the dynamic range 630, as defined in the decoding tables 910 or 1210 discussed in more detail above, by using an extended bit format including an extended code or ext_pin[0] and the FP16 format.

[0166] Figure 15 This is a flowchart of a method for processing FP operations performed by a processing device according to one or more embodiments. (See reference) Figure 15 Processed by processing device 100 (such as, Figure 7 The method of performing FP operations by the processing device 100 can be related to one or more embodiments described in the above figures, and therefore the description provided in the above figures can be applied even if not repeated below. Figure 15 The method.

[0167] In operation 1501, processor 10 (such as, Figure 7 The processor 110 of one or more embodiments can receive a first FP operand and a second FP operand having an n-bit format, the n-bit format including a sign field, an exponent field and a significant digit field, where n is a natural number.

[0168] In operation 1502, processor 110 can normalize the binary value obtained by performing arithmetic operations on corresponding fields of the FP operands used for n-bit multiplication.

[0169] In operation 1503, processor 110 can determine whether the normalized binary value is a number that can be represented in n-bit format or an extended normal number that cannot be represented in n-bit format.

[0170] In operation 1504, based on the determined result, processor 110 can encode the normalized binary value using an extended bit format in which an extension code for identifying whether the normalized binary value is an extended normal number is added to the n-bit format.

[0171] In operation 1505, processor 110 can output a binary value encoded in extended bit format as the result of an n-bit multiplication operation between FP operands.

[0172] Furthermore, the above description has described a non-limiting example of using an FP16 multiplier to perform multiplication of FP16 operands in the MAC operation of a neural network. That is, the architecture of the FP multiplier 1000, the extended bit format of the decoding table 910 or 1210, and the dynamic ranges 920 and 1220 are all described as being defined relative to the FP16 format. However, similar modifications can be made to the example, and similar principles can be applied to other FP formats of precision (such as lower or higher precision). For example, embodiments can be modified and applied to implement the architecture, extended bit format, dynamic range of an FP8 multiplier, and similar constructions are suitable for multiplication operations on FP8 operands. In one or more embodiments, various examples exist that are applied to implement the architecture of an FP32 multiplier to perform multiplication operations on higher precision operands (e.g., FP32), extended bit formats, dynamic ranges, etc.

[0173] Figure 16 This is a block diagram illustrating an electronic system 1600 according to one or more embodiments.

[0174] refer to Figure 16 The electronic system 1600 can extract useful information by analyzing input data in real time based on a neural network. The electronic system 1600 can also determine the status based on the extracted information or the control configuration of the electronic device on which the electronic system 1600 is installed. The electronic device can also be the electronic system 1600. For example, the electronic system 1600 can be or can be applied to robotic devices such as drones, advanced driver assistance systems (ADAS) and similar systems, as well as smart TVs, smartphones, medical devices, mobile devices, image display devices, measuring devices, IoT devices, etc., and can be at least one of these various electronic devices, or can be installed on or coupled to at least one of these electronic devices.

[0175] According to non-limiting examples, electronic system 1600 may include processor 1610, RAM 1620, neural network device 1630, memory 1640, sensor module 1650, and communication module 1660. In addition to or instead of the examples listed above, electronic system 1600 may also include other components such as input / output modules, security modules, power control devices, etc. For example, some hardware configurations of electronic system 1600 may be mounted on or be at least one semiconductor chip. Neural network device 1630 may be included in the above references. Figures 7 to 15 The device includes the processing apparatus described in one or more embodiments, and may be a hardware accelerator dedicated to performing neural networks or a device including such a hardware accelerator, wherein the hardware accelerator may include hardware dedicated to performing neural networks or for more effectively supporting the operation of neural networks. Figures 7 to 15 The processing device can also be an electronic system 1600.

[0176] Processor 1610 can control the overall operation of electronic system 1600. Processor 1610 may include a processor core, a single-core processor, multiple processor cores, or a multi-core processor. Processor 1610 can process or execute instructions and / or data stored in memory 1640. In one or more embodiments, processor 1610 can control the functionality of neural network device 1630 by executing instructions stored in memory 1640. As a non-limiting example, processor 1610 may be implemented as a CPU, GPU, AP, or another type of processor. Through the execution of instructions, processor 1610 can be configured to perform one or more or all of the operations and methods described herein.

[0177] RAM 1620 can temporarily store instructions, data, or applications. For example, programs and / or data stored in memory 1640 can be temporarily stored in RAM 1620 under the control of or based on the startup code of processor 1610. For example, as a non-limiting example, RAM 1620 can be implemented as a memory such as dynamic RAM (DRAM) or static RAM (SRAM).

[0178] The neural network device 1630 can perform neural network operations based on received input data and can generate information signals based on the results of the operations. As a non-limiting example, the neural network may include, but is not limited to, convolutional neural networks (CNNs), recurrent neural networks (RNNs), deep belief networks, and restricted Boltzmann machines. The neural network device 1630 may include hardware that uses a neural network to perform processing and may correspond to, for example, reference devices specifically designed for neural networks. Figures 7 to 15 Hardware accelerators, including the described processing device.

[0179] As a non-limiting example, the information signal may include one of various types of recognition signals, such as speech recognition signals, object recognition signals, image recognition signals, and biometric recognition signals. For example, the neural network device 1630 may receive frame data included in a video stream as input data and may generate a recognition signal about an object included in an image represented by the frame data based on the frame data. However, the neural network device 1630 is not limited to these specific examples, and the neural network device 1630 may receive various types of input data depending on the type or function of the electronic system 1600 or the electronic device coupled to the electronic system 1600, and may generate a recognition signal based on the input data.

[0180] Memory 1640 can be a storage location for storing data, and can store an operating system (OS), various programs, and various types of data. In one or more embodiments, memory 1640 can store neural network data generated during processing that performs the operation of neural network device 1630, such as FP-type or fixed-point type input / output activations, weights, etc. In one or more embodiments, as described in more detail above, memory 1640 can store parameters used for mixed-precision MAC operations. Additionally, memory 1640 can also store parameters encoded in an extended bit format.

[0181] The memory 1640 may be DRAM, but is not limited thereto. The memory 1640 may include at least one of volatile memory and non-volatile memory. For example, as a non-limiting example, non-volatile memory may include ROM, PROM, EPROM, EEPROM, flash memory, PRAM, MRAM, RRAM, FRAM, etc. As a non-limiting example, volatile memory may include DRAM, SRAM, SDRAM, PRAM, MRAM, RRAM, FeRAM, etc. In embodiments, the memory 1640 may also include at least one of HDD, SSD, CF, SD, Micro-SD, Mini-SD, xD, and Memory Stick memory that provides persistent storage capabilities.

[0182] In this example, sensor module 1650 can collect information about the vicinity of an electronic device on which electronic system 1600 is installed, or collect information about the vicinity of an electronic system (where the electronic system is an electronic device). For example, sensor module 1650 can sense or receive signals (e.g., image signals, voice signals, magnetic signals, biometric signals, touch signals, etc.) from outside the electronic device, and can convert the sensed or received signals into data. For this purpose, sensor module 1650 can include at least one of various types of sensing devices. For example, as a non-limiting example, sensing devices can include microphones, imaging devices, image sensors, light detection and ranging (LIDAR) sensors, ultrasonic sensors, infrared sensors, biometric sensors, and touch sensors.

[0183] Sensor module 1650 can provide the converted data to neural network device 1630 as input data. For example, as a non-limiting example, sensor module 1650 may include an image sensor that can generate a video stream by capturing the external environment of an electronic device, and can sequentially provide consecutive data frames of the video stream to neural network device 1630 as input data. However, in other examples, sensor module 1650 is not limited to this particular example, and sensor module 1650 can provide various other types of data to neural network device 1630.

[0184] The communication module 1660 may include various wired or wireless interfaces for communicating with external devices. For example, the communication module 1660 may include communication interfaces capable of accessing the following: mobile cellular networks such as wired local area networks (LANs), wireless local area networks (WLANs) such as Wi-Fi networks, wireless personal area networks (WPANs) such as Bluetooth, wireless universal serial buses (USB), Zigbee, near field communication (NFC), radio frequency identification (RFID), power line communication (PLC), or third-generation (3G), fourth-generation (4G), long-term evolution (LTE), fifth-generation (5G), etc.

[0185] In one or more examples, the communication module 1260 can receive data about the quantized neural network from an external source. Here, "external source" can refer to a device that can perform neural network training based on a large amount of data, quantize the trained neural network into a fixed-point type, and provide the quantized neural network data to the electronic system 1200. The received quantized neural network data can be stored in the memory 1240.

[0186] Perform the operations described in this application Figures 1 to 16The processing device (processing device 100), processor (processor 110), memory (memory 120), MAC arithmetic unit (MAC arithmetic unit 1100), FP multiplier (FP multiplier 1000), XOR logic circuit 1010, adder 1021, subtractor 1022, multiplier 1030, normalizer 1040, FP multiplier 1110, FP accumulator 1120, electronic system 1600, processor 1610, RAM 1620, neural network device 1630, memory 1640, sensor module 1650, and communication module 1660 are implemented by hardware components configured to perform the operations described in this application, which are performed by the hardware components. Examples of hardware components that may be used to perform the operations described in this application, where appropriate, include controllers, sensors, generators, drivers, memories, comparators, arithmetic logic units, adders, subtractors, multipliers, dividers, integrators, and any other electronic components configured to perform the operations described in this application. In other examples, one or more hardware components for performing the operations described in this application are implemented by computing hardware (e.g., by one or more processors or computers). A processor or computer may be implemented by one or more processing elements (e.g., logic gate arrays, controllers and arithmetic logic units, digital signal processors, microcomputers, programmable logic controllers, field-programmable gate arrays, programmable logic arrays, microprocessors, or any other device or combination of devices configured to respond to and execute instructions in a defined manner to achieve a desired result). In one example, the processor or computer includes (or is connected to) one or more memories storing instructions or software executed by the processor or computer. The hardware components implemented by the processor or computer can execute instructions or software, such as an operating system (OS) and one or more software applications running on the OS, to perform the operations described in this application. Hardware components can also access, manipulate, process, create, and store data in response to the execution of instructions or software. For brevity, the singular terms "processor" or "computer" may be used in the description of the examples described in this application, but multiple processors or computers may be used in other examples, or a processor or computer may include multiple processing elements, or multiple types of processing elements, or both. For example, a single hardware component or two or more hardware components may be implemented by a single processor, or two or more processors, or a processor and a controller. One or more hardware components may be implemented by one or more processors, or a processor and a controller, and one or more other hardware components may be implemented by one or more other processors or another processor and another controller. One or more processors or a processor and a controller may implement a single hardware component, or two or more hardware components.The hardware components can have any one or more different processing configurations, examples of which include single processor, discrete processor, parallel processor, single instruction single data (SISD) multiprocessing, single instruction multiple data (SIMD) multiprocessing, multiple instruction single data (MISD) multiprocessing, and multiple instruction multiple data (MIMD) multiprocessing.

[0187] Perform the operations described in this application Figures 1 to 16 The methods illustrated are executed by computing hardware, such as one or more processors or a computer, wherein the computing hardware is implemented as described above to execute instructions or software to perform the operations performed by these methods as described in this application. For example, a single operation or two or more operations may be executed by a single processor, or two or more processors, or a processor and a controller. One or more operations may be executed by one or more processors or a processor and a controller, and one or more other operations may be executed by one or more other processors or another processor and another controller. One or more processors or a processor and a controller may execute a single operation or two or more operations.

[0188] Instructions or software for controlling computing hardware (e.g., one or more processors or computers) to implement hardware components and perform the methods described above can be written as computer programs, code segments, instructions, or any combination thereof, for individually or collectively instructing or configuring one or more processors or computers to operate as machines or special-purpose computers to perform the operations performed by the hardware components and methods described above. In one example, the instructions or software include machine code that is directly executed by one or more processors or computers, such as machine code generated by a compiler. In another example, the instructions or software include higher-level code that is executed by one or more processors or computers using an interpreter. The instructions or software can be written using any programming language based on the block diagrams and flowcharts shown in the accompanying drawings and the corresponding descriptions in the specification (which disclose algorithms for performing the operations performed by the hardware components and the methods described above).

[0189] Instructions or software for controlling computing hardware (e.g., one or more processors or computers) to implement hardware components and perform the methods described above, along with any associated data, data files, and data structures, may be recorded, stored, or fixed in or on one or more non-transitory computer-readable storage media. Examples of non-transitory computer-readable storage media include read-only memory (ROM), random access memory (RAM), flash memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store instructions or software and any associated data, data files, and data structures in a non-transitory manner and to provide instructions or software and any associated data, data files, and data structures to one or more processors or computers, enabling the one or more processors or computers to execute the instructions. In one example, instructions or software, along with any associated data, data files, and data structures, are distributed across a networked computer system, enabling one or more processors or computers to store, access, and execute the instructions and software, along with any associated data, data files, and data structures, in a distributed manner.

[0190] Although this disclosure includes specific examples, it will be apparent upon understanding the disclosure of this application that various changes in form and detail may be made to these examples without departing from the spirit and scope of the claims and their equivalents. The examples described herein are to be considered descriptive only and not for limiting purposes. The description of features or aspects in each example is to be considered applicable to similar features or aspects in other examples. Suitable results may be achieved if the described techniques are performed in a different order and / or if components in the described system, architecture, device, or circuit are combined in a different manner and / or replaced or supplemented by other components or their equivalents. Therefore, the scope of this disclosure is not limited by the specific embodiments but by the claims and their equivalents, and all variations within the scope of the claims and their equivalents are to be construed as included in this disclosure.

Claims

1. A method for floating-point processing, the method comprising: The system receives a first floating-point operand and a second floating-point operand, both of which have an n-bit format. The n-bit format includes a sign field, an exponent field, and a significant digit field, where n is a natural number. The binary value obtained by performing arithmetic operations on corresponding fields of the first and second floating-point operands used for n-bit multiplication is normalized, wherein the normalization is performed by a normalizer in the floating-point multiplier. Determine whether the normalized binary value is a number that can be represented in the n-bit format or an extended normal number that cannot be represented in the n-bit format; Based on the determined result, the normalized binary value is encoded using an extended bit format, in which an extension code indicating whether the normalized binary value is the extended normal number is added to the n-bit format; as well as As the result of the n-bit multiplication operation between the first floating-point operand and the second floating-point operand, the encoded binary value is output using the extended bit format.

2. The method according to claim 1, wherein, The number that can be represented in the n-bit format is a normal number or a subnormal number, and The extended normal number is not included in the dynamic range of the normal number, nor is it included in the dynamic range of the subnormal number.

3. The method of claim 2, wherein the determination includes: Based on the exponent of the normalized binary value, determine whether the normalized binary value is the normal number, the subnormal number, or the extended normal number.

4. The method according to claim 1, wherein the arithmetic operation is a convolution operation.

5. The method of claim 1, wherein the extended code has a first value when the normalized binary value is a normal or subnormal number that can be represented in the n-bit format, and the extended code has a second value when the normalized binary value is the extended normal number.

6. The method according to claim 1, wherein the dynamic range of the extended normal number is a range representing positive or negative numbers, the absolute value of which is less than the absolute value of a subnormal number that can be represented in the n-bit format.

7. The method of claim 6, wherein the determination comprises: The normalized binary value is determined to be a normal number when the exponent of the normalized binary value is within the dynamic range of a normal number that can be represented in the n-bit format. When the exponent of the normalized binary value is not included in the dynamic range of the normal number but is included in the dynamic range of the subnormal number, the normalized binary value is determined to be the subnormal number; and The normalized binary value is determined to be the extended normal number when the exponent of the normalized binary value is not included in the dynamic range of the normal number or the dynamic range of the subnormal number.

8. The method according to claim 1, wherein the dynamic range of the extended normal number is a range representing positive or negative numbers, the absolute value of which is greater than the absolute value of a subnormal number that can be represented in the n-bit format, and less than the absolute value of a normal number that can be represented in the n-bit format.

9. The method of claim 8, wherein the determination comprises: The normalized binary value is determined to be the normal number when the exponent of the normalized binary value is within the dynamic range of the normal number that can be represented in the n-bit format. The normalized binary value is determined to be the extended normal number when the exponent of the normalized binary value is not included in the dynamic range of the normal number but is included in the dynamic range of the extended normal number; and The normalized binary value is determined to be the subnormal number when the exponent of the normalized binary value is not included in the dynamic range of the normal number or the dynamic range of the extended normal number.

10. The method according to claim 1, wherein, The value of n is 16. The n-bit format is a half-precision floating-point format. The n-bit multiplication operation is a 16-bit multiplication operation using a 16-bit multiplier, and In the n-bit format, the sign field contains 1 bit, the exponent field contains 5 bits, and the significant digits field contains 10 bits.

11. The method according to claim 10, wherein, The extended normal number is and Positive numbers between, or and Negative numbers between.

12. The method of claim 1, wherein the normalization comprises: Perform an addition operation on the exponent field of the first floating-point operand and the exponent field of the second floating-point operand; The subtraction operation is performed by subtracting the deviation corresponding to the n-bit format from the result of the addition operation; Perform a multiplication operation on the significant number field of the first floating-point operand and the significant number field of the second floating-point operand; as well as The binary value obtained based on the result of the subtraction operation and the result of the multiplication operation is normalized.

13. The method according to claim 1, wherein, The n-bit multiplication operation corresponds to a part of the multiplication-accumulation (MAC) operation in a neural network, and The encoded binary value is provided to an accumulation operation with a third floating-point operand of m bits, where m is a natural number greater than n.

14. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the method of claim 1.

15. An apparatus for floating-point processing, the apparatus comprising: One or more processors are configured as follows: The system receives a first floating-point operand and a second floating-point operand, both of which have an n-bit format. The n-bit format includes a sign field, an exponent field, and a significant digit field, where n is a natural number. The binary value obtained by performing arithmetic operations on corresponding fields of the first and second floating-point operands used for n-bit multiplication is normalized, wherein the normalization is performed by a normalizer in the floating-point multiplier. Determine whether the normalized binary value is a number that can be represented in the n-bit format or an extended normal number that cannot be represented in the n-bit format; Based on the determined result, the normalized binary value is encoded using an extended bit format, in which an extension code for identifying whether the normalized binary value is the extended normal number is added to the n-bit format; as well as As the result of the n-bit multiplication operation between the first floating-point operand and the second floating-point operand, the encoded binary value is output using the extended bit format.

16. The apparatus of claim 15, further comprising a memory storing instructions that, when executed by the one or more processors, configure the one or more processors to perform the normalization, the determination, the encoding, and the output of the encoded binary value.

17. The apparatus according to claim 15, wherein, The number that can be represented in the n-bit format is a normal number or a subnormal number, and The extended normal number is not included in the dynamic range of the normal number, nor is it included in the dynamic range of the subnormal number.

18. The apparatus of claim 17, wherein the one or more processors are further configured to: determine, based on the exponent of the normalized binary value, whether the normalized binary value is the normal number, the subnormal number, or the extended normal number.

19. The apparatus of claim 15, wherein the arithmetic operation is a convolution operation.

20. The apparatus of claim 15, wherein the extended code has a first value when the normalized binary value is a normal or subnormal number that can be represented in the n-bit format, and the extended code has a second value when the normalized binary value is the extended normal number.

21. The apparatus of claim 15, wherein the dynamic range of the extended normal number is a range representing positive or negative numbers, the absolute value of which is less than the absolute value of a subnormal number that can be represented in the n-bit format.

22. The apparatus of claim 15, wherein the dynamic range of the extended normal number is a range representing positive or negative numbers, the absolute value of which is greater than the absolute value of a subnormal number that can be represented in the n-bit format and less than the absolute value of a normal number that can be represented in the n-bit format.

23. The apparatus according to claim 15, wherein, The value of n is 16. The n-bit format is a half-precision floating-point format. The n-bit multiplication operation is a 16-bit multiplication operation using a 16-bit multiplier. In the n-bit format, the sign field contains 1 bit, the exponent field contains 5 bits, and the significant digits field contains 10 bits. The extended normal number is and Positive numbers between, or and Negative numbers between.

24. The apparatus according to claim 15, wherein, Each of the one or more processors includes a MAC arithmetic unit configured to perform a multiply-accumulate MAC operation. The MAC arithmetic unit includes an n-bit multiplier that performs the n-bit multiplication operation and an m-bit accumulator, where m is a natural number greater than n. The n-bit multiplier includes: An adder is configured to perform an addition operation on the exponent field of the first floating-point operand and the exponent field of the second floating-point operand; The subtractor is configured to perform a subtraction operation by subtracting the deviation corresponding to the n-bit format from the result of the addition operation; The multiplier is configured to perform a multiplication operation on the significant number field of the first floating-point operand and the significant number field of the second floating-point operand; and The normalizer is configured to normalize the binary value obtained based on the result of the subtraction operation and the result of the multiplication operation, and The MAC processor is configured as follows: The accumulator is used to perform an accumulation operation on the encoded binary value and the m-bit third floating-point operand.

25. A method for floating-point processing, the method comprising: The binary value obtained by performing arithmetic operations on corresponding fields in the first and second floating-point operands used for n-bit multiplication is normalized, wherein the normalization is performed by a normalizer in the floating-point multiplier, and wherein the n-bit multiplication operation corresponds to a part of the multiply-accumulate (MAC) operation of a neural network. Based on determining whether the normalized binary value is a number that can be represented in an n-bit format or an extended normal number that cannot be represented in the n-bit format, the normalized binary value is encoded using an extended bit format, in which an extension code for identifying whether the normalized binary value is the extended normal number is added to the n-bit format; as well as As the result of the n-bit multiplication operation between the first floating-point operand and the second floating-point operand, the extended bit format is used to output the encoded binary value for use in a neural network.

26. The method of claim 25, wherein the number that can be represented in the n-bit format is a normal number or a subnormal number, and The extended normal number is not included in the dynamic range of the normal number, nor is it included in the dynamic range of the subnormal number.

27. The method of claim 25, wherein the extended code has a first value when the normalized binary value is a normal or subnormal number that can be represented in the n-bit format, and the extended code has a second value when the normalized binary value is the extended normal number.

28. The method of claim 25, wherein the dynamic range of the extended normal number is a range representing positive or negative numbers, the absolute value of which is less than the absolute value of a subnormal number that can be represented in the n-bit format.

29. The method according to claim 25, further comprising: Generate a first floating-point operand and a second floating-point operand, both of which have an n-bit format. The n-bit format includes a sign field, an exponent field, and a significant digit field, where n is a natural number.

30. The method of claim 29, wherein the first floating-point operand and the second floating-point operand are generated as the activation output of the previous layer of the neural network.

Images