Calculation method and apparatus

Abstract

To provide an arithmetic method for efficiently processing operations related to an artificial neural network.SOLUTION: According to an embodiment, an arithmetic method for efficiently processing operations related to an artificial neural network comprises: receiving a plurality of sets of input data expressed in floating point format (S310); adjusting the bit-width of mantissa by performing masking on the mantissa of each set of the input data based on a size of an exponent of each set of input data (S320); and performing operations between the sets of input data with the adjusted bit-width (S330).SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The following embodiments relate to a calculation method and apparatus. [Background technology]

[0002] Artificial neural networks (ARTNs) are realized by referencing computer science architectures. Active research is being conducted on using ARTNs in various types of electronic systems to analyze input data and extract useful information. Devices that process ARTNs require massive computations on complex input data. Therefore, in order to analyze large amounts of input data in real time using ARTNs and extract desired information, technologies that can efficiently process ARTNs are needed. [Overview of the project] [Problems that the invention aims to solve]

[0003] The object of the present invention is to provide a calculation method and apparatus. [Means for solving the problem]

[0004] A calculation method embodied in a processor according to one embodiment includes the steps of: receiving a plurality of input data expressed in floating-point form; adjusting the bit width of the mantissa by masking the mantissa of each input data based on the magnitude of the exponent of each input data; and performing calculations between the input data whose bit widths have been adjusted.

[0005] The step of adjusting the bit width of the mantissa for each of the input data may include the step of adjusting the bit width of the mantissa in proportion to the size of each of the input data.

[0006] The step of adjusting the bit width of the mantissa for each of the input data may include the steps of comparing each of the input data with a threshold and adjusting the bit width of the mantissa based on the comparison result.

[0007] The threshold can be determined based on the distribution of the input data and the acceptable error range.

[0008] A calculation method according to one embodiment may further include the steps of receiving the distribution of the plurality of input data and determining a threshold corresponding to each of the plurality of input data based on the distribution of the plurality of input data.

[0009] The step of performing the calculation may include a step of controlling the position and timing of the arithmetic unit to which the bit-width adjusted input data is input.

[0010] The step of performing an operation between the bit-width adjusted input data may include the steps of determining the number of cycles of the operation to be performed by a predetermined number of arithmetic units based on the adjusted bit width of each of the input data, and inputting the bit-width adjusted input data to the arithmetic units in accordance with the determined number of cycles.

[0011] The step of determining the number of cycles for the calculation may include determining the number of cycles for the calculation based on the adjusted bit width of each of the input data and the number of bits that the arithmetic unit can process in one cycle.

[0012] The arithmetic unit may include a multiplier that performs integer multiplication of the mantissa of the input data, a shifter that shifts the result of the multiplier, and an accumulator that accumulates the shifted result.

[0013] The step of performing an operation between the input data with the adjusted bit width may include a step of determining the number of arithmetic units for performing the operation within a predetermined number of operation cycles based on the adjusted bit width of each of the input data, and a step of inputting the input data with the adjusted bit width to the arithmetic units corresponding to the determined number of arithmetic units.

[0014] The step of determining the number of arithmetic units may include a step of determining the number of arithmetic units based on the adjusted bit width of each of the input data and the number of bits that the arithmetic unit can process in one cycle.

[0015] An arithmetic device according to an embodiment includes one or more processors that receive a plurality of input data represented in floating point, perform masking on the mantissa of each of the input data based on the magnitude of the exponent of each of the input data to adjust the bit width of the mantissa, and perform an operation between the input data with the adjusted bit width.

[0016] The processor may adjust the bit width of the mantissa of each of the input data in proportion to the magnitude of each of the input data.

[0017] The processor may compare each of the input data with a threshold value, and adjust the bit width of the mantissa of each of the input data based on the comparison result.

[0018] The threshold value may be determined based on the distribution of the input data and an allowable error range.

[0019] The processor may receive the distribution of the plurality of input data, and determine a threshold value corresponding to each of the plurality of input data based on the distribution of the plurality of input data.

[0020] The processor may control the position and timing of the arithmetic units to which the input data with the adjusted bit width is input.

[0021] The processor determines the number of cycles for the calculation to be performed by a predetermined number of arithmetic units based on the adjusted bit width of each of the input data, and can input the input data with the adjusted bit width to the arithmetic units in accordance with the determined number of cycles.

[0022] The processor can determine the number of cycles for the operation based on the adjusted bit width of each of the input data and the number of bits that the arithmetic unit can process in one cycle.

[0023] The arithmetic unit may include a multiplier that performs integer multiplication of the mantissa of the input data, a shifter that shifts the result of the multiplier, and an accumulator that accumulates the shifted result.

[0024] The processor determines, based on the adjusted bit width of each of the input data, the number of arithmetic units to perform the calculation within a predetermined number of calculation cycles, and can input the bit-width adjusted input data to the arithmetic units in accordance with the determined number of arithmetic units.

[0025] The processor can determine the number of arithmetic units based on the adjusted bit width of each of the input data and the number of bits that each arithmetic unit can process in one cycle.

[0026] An arithmetic unit according to one embodiment includes a central processing unit that receives a distribution of a plurality of input data expressed in floating-point form and determines a threshold corresponding to each of the plurality of input data based on the distribution of the plurality of input data, and a hardware accelerator that receives the plurality of input data, adjusts the bit width of the mantissa by masking the mantissa of each of the input data based on the magnitude of the exponent of each of the input data, and performs calculations between the input data whose bit widths have been adjusted.

[0027] A calculation method embodied in a processor according to one embodiment includes the steps of: receiving input data expressed in floating-point form; comparing the magnitude of the exponent of the input data with a threshold and adjusting the bit width of the mantissa of the input data; and performing calculations on the input data whose bit width has been adjusted.

[0028] The step of adjusting the bit width of the mantissa of the input data may include the step of assigning a bit width greater than the threshold or less than the same exponent to an exponent smaller than the threshold.

[0029] The step of performing the calculation includes the step of performing the calculation using an arithmetic unit, wherein the adjusted bit width is less than or equal to the number of bits that the arithmetic unit can process in a single period.

[0030] The step of adjusting the bit width of the mantissa of the input data may include the step of maintaining the bit width of the mantissa if the exponent of the input data is greater than or equal to the threshold.

[0031] The threshold includes a plurality of threshold ranges corresponding to each bit width, and the step of adjusting the bit width of the mantissa of the input data may include adjusting the bit width in response to input data corresponding to one of the plurality of threshold ranges so that the bit width of the mantissa corresponds to one of the plurality of threshold ranges.

[0032] The step of performing the aforementioned calculation may include the step of performing multiplication and accumulation calculations using an arithmetic unit.

[0033] A calculation method embodied in a processor according to one embodiment includes the steps of: receiving weighted value data and feature map data of a neural network expressed in floating-point form; comparing the magnitude of the exponent of the weighted value data with a threshold, comparing the magnitude of the exponent of the feature map data with another threshold, and adjusting the bit width of the mantissa of the weighted value data and the mantissa of the feature map data; and performing a neural network operation between the weighted value data with the adjusted bit width and the feature map data with the adjusted bit width. [Effects of the Invention]

[0034] According to the present invention, a calculation method and apparatus can be provided. [Brief explanation of the drawing]

[0035] [Figure 1A] This is a diagram illustrating a deep learning computation method using an artificial neural network. [Figure 1B] This diagram illustrates the data and filters of the input feature maps provided as input in deep learning operations. [Figure 1C] This diagram illustrates the process of performing convolution operations on a deep learning platform. [Figure 2] This is a block diagram of a neural network device according to one embodiment. [Figure 3] This is a flowchart illustrating a calculation method according to one embodiment. [Figure 4] This figure illustrates a method for efficiently operating a dynamic floating-point system according to one embodiment. [Figure 5] This figure illustrates a specific method for adjusting the bit width of the mantissa using a threshold according to one embodiment. [Figure 6] This figure illustrates a method for performing dynamic floating-point mantissa multiplication by dynamically adjusting the number of calculation cycles using a fixed number of arithmetic units according to one embodiment. [Figure 7] This diagram illustrates a method for performing dynamic floating-point mantissa multiplication by dynamically adjusting the number of arithmetic units participating in an operation within a fixed operation cycle. [Figure 8] This is a block diagram illustrating a computing device according to one embodiment. [Modes for carrying out the invention]

[0036] The specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and can be modified in various ways. Therefore, the embodiments are not limited to the specific disclosure, and the scope of this specification includes modifications, equivalents, or substitutions that are part of the technical idea.

[0037] Terms such as "first" or "second" may be used to describe multiple components, but such terms should be interpreted solely for the purpose of distinguishing one component from others. For example, the first component may be named the second component, and similarly, the second component may also be named the first component.

[0038] When it is mentioned that one component is “linked” or “connected” to another component, it should be understood that it is directly linked to or connected to the other component, but that other components may be present in between.

[0039] A singular expression includes plural expressions unless the context clearly indicates otherwise. In this specification, terms such as “includes” or “has” indicate the presence of features, figures, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood not to presuppose the existence or addition of one or more other features, figures, steps, actions, components, parts, or combinations thereof.

[0040] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as those generally understood by a person of ordinary skill in the art to which this embodiment belongs. Commonly used, predefined terms should be interpreted as having the meaning consistent with their meaning in the context of the relevant art, and not as ideal or overly formal unless expressly defined herein.

[0041] The embodiments can be implemented in various forms of products such as data centers, servers, personal computers, laptop computers, tablet computers, smartphones, televisions, smart home appliances, intelligent vehicles, kiosks, and wearable devices. The embodiments will be described in detail below with reference to the attached drawings. The same reference numerals shown in each drawing indicate the same component. The neural network can be called an "artificial" neural network, but this is not intended to associate it with the way in which the neural network computationally maps or intuitively recognizes information or the way in which the human brain operates. In other words, the term "artificial neural network" is merely a technical term referring to a neural network embodied in hardware.

[0042] Figure 1A is a diagram illustrating a deep learning computation method using an artificial neural network (ANN).

[0043] Artificial intelligence (AI) algorithms, including deep learning, can input data 10 into an artificial neural network, learn output data 30 through operations such as convolution, and extract features using the learned artificial neural network. An artificial neural network refers to a computer science architecture that models the biological brain. Within the artificial neural network, nodes corresponding to neurons in the brain are interconnected and work collectively to process input data. Various types of neural networks exist, but are not limited to, convolutional neural networks (CNNs), recurrent neural networks (RNNs), deep belief networks (DBNs), and restricted Boltzmann machines (RBMs). In a feed-forward neural network, neurons have links with other neurons. Such links may be extended unidirectionally through the neural network, for example, in the forward direction.

[0044] Figure 1A shows the structure of an artificial neural network (e.g., a convolutional neural network) 20 that receives input data 10 and outputs output data 130. The artificial neural network may also be a deep neural network with two or more layers.

[0045] The convolutional neural network 20 is used to extract "features" such as frames and line colors from the input data 10. The convolutional neural network 20 may include multiple layers. Each layer receives data, processes the data input to that layer, and generates output data from that layer. The output data from a layer may be an image input to the convolutional neural network 20 or a feature map generated by convolutional operation with filter weight values ​​on the input feature map. The initial layer of the convolutional neural network 20 may operate to extract low-level features such as edges or gradients from the input. Subsequent layers of the convolutional neural network 20 can extract increasingly complex features in the image, such as eyes and noses.

[0046] Figure 1B is a diagram illustrating the data and filters of the input feature map provided as input in a deep learning operation.

[0047] Referring to Figure 1B, the input feature map 100 is, but is not limited to, a set of pixel values ​​or numerical data of an image input to the artificial neural network. In Figure 1B, the input feature map 100 is the pixel values ​​of the image that must be learned or recognized via the artificial neural network. For example, the input feature map 100 may have 256 × 256 pixels and a depth of K.

[0048] Filters 110-1 to 110-n are formed in N units. Each of the filters 110-1 to 110-n may contain n by n (n × n) weight values. For example, each of the filters 110-1 to 110-n may have 3 × 3 pixels and a depth value of K.

[0049] Figure 1C is a diagram illustrating the process of performing a deep learning convolution operation.

[0050] Referring to FIG. 1C, the process of performing a convolution operation in an artificial neural network means a process of generating output values by performing multiplication and addition operations on an input feature map 100 and a filter 110 at each layer, and generating an output feature map 120 by accumulating and summing the output values.

[0051] The convolution operation execution process is a process of performing multiplication and addition operations by applying a filter 110 of a fixed size, that is, an n×n size, from the upper left side to the lower right side of the input feature map 100 at the current layer. Hereinafter, the process of performing a convolution operation when the size of the filter 110 is 3×3 will be described.

[0052] For example, first, in the first region 101 on the upper left side of the input feature map 100, a total of 9 data X including 3×3, that is, 3 data in the first direction and 3 data in the second direction 11 ~X 33 are respectively subjected to an operation of multiplying with the weight values W of the filter 110. 11 ~W 33 After that, the output values of the multiplication operation, that is, X 11 ×W 11 , X 12 ×W 12 , X 13 ×W 13 , X 21 ×W 21 , X 22 ×W 22 , X 23 ×W 23 , X 31 ×W 31 , X 32 ×W 32 , X 33 ×W 33 are all accumulated and summed, whereby the 1-1-th output data Y11 of the output feature map 120 is generated.

[0053] Subsequently, the data is moved from the first region 101 in the upper left of the input feature map 100 to the second region 102 by the unit of data, and the operation is performed. Here, the number of data points moved within the input feature map 100 during the convolution operation is called the stride, and the size of the generated output feature map 120 is determined according to the size of the stride. For example, if the stride is 1, the total of 9 input data X12~X34 contained in the second region 102 is multiplied by the weight values ​​W11~W33 of the filter 110, and the output value of the multiplication operation is X 12 ×W 11 , X 13 ×W 12 , X 14 ×W 13 , X 22 ×W 21 , X 23 ×W 22 , X 24 ×W 23 , X 32 ×W 31 , X 33 ×W 32 , X 34 ×W 33 By accumulating and summing all of these, the first and second output data Y12 of the output feature map 120 is generated.

[0054] Figure 2 is a block diagram of a neural network device according to one embodiment.

[0055] Referring to Figure 2, a neural network device 200 according to one embodiment includes a host 210, a hardware accelerator 230, and a memory 220. Only the components relating to this embodiment are shown in the neural network device 200 shown in Figure 2. Therefore, it will be obvious to those skilled in the art that the neural network device 200 includes other general-purpose components in addition to those shown in Figure 2.

[0056] The host 210 performs the overall functions for controlling the neural network device 200. The host 210 controls the neural network device 200 by executing programs stored in the memory 220 within the neural network device 200. The host 210 is implemented using, but is not limited to, the CPU (central processing unit), GPU (graphics processing unit), AP (application processor) provided within the neural network device 200.

[0057] Host 210 can output a calculation result regarding whether the input data belongs to one of the classes using a neural network trained for classification. Specifically, the neural network for classification can output the calculation result regarding the probability of the input data belonging to each class as a result value for each class. The neural network for classification may also include a softmax layer and a loss layer. The softmax layer converts the result values ​​for each class into probability values, and the loss layer can calculate the loss as an objective function for training the neural network.

[0058] Memory 220 is hardware that stores data processed and data to be processed within the neural network device 200. Memory 220 also stores applications, drivers, etc., that are driven by the neural network device 200. Memory 220 may include volatile memory such as DRAM or non-volatile memory.

[0059] The neural network device 200 includes a hardware accelerator 230 for driving the neural network. The hardware accelerator 230 is, for example, a dedicated module for driving the neural network, such as an NPU (neural processing unit), TPU (Tensor Processing Unit), or Neural Engine.

[0060] Figure 3 is a flowchart illustrating a calculation method according to one embodiment.

[0061] The operations shown in Figure 3 may be performed in the order and manner indicated, but the order of some operations may be changed or some operations may be omitted, without exceeding the concept and scope of the illustrated embodiment. The operations shown in Figure 3 may be performed in parallel or simultaneously. The blocks and combinations of blocks shown in Figure 3 can be realized by a special-purpose hardware-based computer that performs a specific function, or by a combination of special-purpose hardware and computer instructions.

[0062] Conventional devices that perform deep learning calculations consume a large amount of computation because they have to process iterative multiplication and addition operations on many layers. On the other hand, as will be explained in detail below, one embodiment of a computing device can reduce the amount of deep learning computation by converting data represented in floating-point numbers.

[0063] In step S310, the hardware accelerator receives multiple input data represented as floating-point numbers. The hardware accelerator may also be referred to as a computing unit. As a different example, a neural network device (e.g., neural network device 200) may be referred to as a computing unit and may include a hardware accelerator. The input data may include weight values ​​and input feature map data as described with reference to Figures 1A to 1C.

[0064] Before describing the calculation method according to one embodiment, we will first explain how to represent data as a floating-point number. Floating-point numbers are a method of representing real numbers by dividing them into a mantissa, which indicates the significant figures, and an exponent, which indicates the position of the decimal point, without fixing the position of the decimal point. For example, if 263.3 expressed in decimal is expressed in binary, it becomes 100000111.0100110…, which can be represented as 1.0000011101 × 28. Furthermore, if this is expressed as a 16-bit floating-point number, the sign bit (1 bit) is 0 (positive), the exponent bit (5 bits) is 11000 (8 + 16 (bias)), and the mantissa bit is 0000011101 (10 bits), and finally it can be represented as 0110000000011101.

[0065] In step S320, the hardware accelerator adjusts the bit width of the mantissa based on the magnitude of each input data. By dynamically setting the bit width of the floating-point mantissa to vary according to the magnitude of the input data during deep learning calculations, the hardware accelerator can minimize accuracy loss and reduce the complexity of calculations by reducing the bit width required for multiplication and addition operations between floating-point input data. The representation of input data adjusted by the method according to one embodiment is called dynamic floating-point number.

[0066] As mentioned above, deep learning operations involve iterative multiplication and addition across many layers, resulting in a large amount of computation time. To process these operations efficiently and at low cost, various quantization schemes and supporting hardware accelerators have been proposed. One method of increasing computation speed by lowering the precision of the artificial neural network parameters is to convert 32-bit floating-point data into 8-bit integer data.

[0067] However, while quantizing data can increase computation speed, it reduces computation accuracy, and a retraining process is required to maintain computation accuracy. Furthermore, when performing real-time calculations, each time an operation with large quantization errors, such as batch normalization, is performed, dequantization (for example, converting 8-bit integer data to 32-bit floating-point data) is performed, and then calculations are performed using the floating-point data. Subsequently, dequantization and quantization processes are required to reduce the complexity of the calculations that follow. The benefits obtained through this are limited. On the other hand, a hardware accelerator according to one embodiment can reduce both accuracy loss and computational complexity simultaneously.

[0068] As the size of the data being approximated increases, the overall accuracy of the calculation decreases. Therefore, hardware accelerators can simultaneously reduce accuracy loss and computational complexity based on a method of allocating the mantissa bit width in proportion to the size of the data.

[0069] In one embodiment, a hardware accelerator can adjust the bit width of the mantissa using only the exponent, without utilizing the entire input data. For example, the hardware accelerator can adjust the bit width of the mantissa in proportion to the size of the exponent, and this method is advantageous in terms of access speed and calculation speed compared to using the entire input data. Furthermore, since the exponent represents the position of the decimal point, and the size of the exponent has an absolute effect on the size of the input data, adjusting the bit width of the mantissa using the size of the exponent does not significantly reduce the accuracy of the overall calculation. The smaller the size of the exponent of the input data, the smaller the impact of the mantissa of the input data on the accuracy of the overall calculation. The hardware accelerator can allocate a larger bit width to the mantissa of input data with a large exponent, and a smaller bit width to the mantissa of input data with a small exponent, thereby simultaneously reducing accuracy loss and calculation complexity.

[0070] More specifically, the hardware accelerator uses a comparator to compare the exponent of the input data with a threshold, assigning a larger bit width to the mantissa of input data with a larger exponent (e.g., greater than or equal to the threshold) and a smaller bit width to the mantissa of input data with a smaller exponent (e.g., less than the threshold). A specific method for adjusting the bit width of the mantissa using the threshold according to the embodiment will be explained with reference to Figure 5 below.

[0071] In step S330, the hardware accelerator performs operations between bit-width adjusted input data. The hardware accelerator performs multiplication and addition operations between bit-width adjusted input data. For example, the hardware accelerator may perform multiplication and addition operations between bit-width adjusted weight values ​​and bit-width adjusted input feature maps.

[0072] The hardware accelerator can perform multiplication operations after normalizing the input data by multiplying the exponents and mantissas of each input data. Multiplication between exponents is equivalent to addition because the lower part is multiplication between the same exponents, and multiplication between mantissas is performed in the same way as integer multiplication. A specific method of operations between input data with adjusted bit width according to one embodiment will be explained with reference to Figures 6 and 7 below.

[0073] The hardware accelerator can repeat steps S310 to S330 for each layer. For each layer, the hardware accelerator receives the input data input to that layer, adjusts the bit width of the mantissa of each input data, and performs operations between the input data with adjusted bit widths. Furthermore, a threshold, which is compared with the exponent of the input data, may be determined for each layer.

[0074] Figure 4 illustrates a method for efficiently operating dynamic floating-point arithmetic according to one embodiment.

[0075] Referring to Figure 4, a hardware accelerator according to one embodiment includes a dynamic floating-point conversion module 410 and a mixed precision arithmetic module 420.

[0076] The hardware accelerator receives a first input 401 and a second input 402, expressed as floating-point numbers. For example, the first input 401 may be a weight value and the second input 402 may be an input feature map, or vice versa.

[0077] The hardware accelerator's dynamic floating-point conversion module 410 can adjust the bit width of the mantissas of the first input 401 and the second input 402 based on the magnitude of the exponents of the first input 401 and the second input 402, respectively. In other words, the dynamic floating-point conversion module 410 can convert the first input 401 and the second input 402, which are represented in floating-point form, into dynamic floating-point form, which has an even smaller bit width for the mantissa.

[0078] The dynamic floating-point conversion module 410 includes a comparator, which is used to compare the exponent of the input data with a threshold, assigning a larger bit width to the mantissa of input data with a larger exponent and a smaller bit width to the mantissa of input data with a smaller exponent. Because the dynamic floating-point conversion module 410 performs threshold comparison using only the exponent and not the entire input data, it can convert floating-point data at a low cost and with a loss of accuracy.

[0079] The dynamic floating-point conversion module 410 can output information about the bit width assigned to the input data mantissa along with the input data whose bit width has been adjusted. For example, the dynamic floating-point conversion module 410 can output information 403 about the bit width of the first input 401 and information 404 about the bit width of the second input 402 along with the first input data 405 and the second input data 406, which are represented in dynamic floating-point format.

[0080] The mixed precision arithmetic module 420 of the hardware accelerator performs operations between first input data 405 and second input data 406, which are represented as dynamic floating-point numbers. The mixed precision arithmetic module 420 may also include an arithmetic unit that performs MAC operations.

[0081] Based on information 403 regarding the bit width of the first input 401 and information 404 regarding the bit width of the second input 402, the hardware accelerator can control the timing at which the first input data 405 and the second input data 406, represented in dynamic floating-point format, are input to the mixed precision arithmetic module 420, the number of arithmetic cycles performed in the mixed precision arithmetic module 420, and the number of arithmetic units operating.

[0082] The mixed precision arithmetic module 420 supports mixed precision arithmetic using a spatial fusion method and / or a temporal fusion method, and can achieve even higher processing capacity when the mantissa bit width is reduced. As a result, through the reduction of the mantissa bit width, a hardware accelerator according to one embodiment can improve the speed of hardware operation and power consumption compared to existing floating-point arithmetic. A detailed explanation of the spatial fusion method and the temporal fusion method is given with reference to Figures 6 and 7 below.

[0083] Figure 5 illustrates a specific method for adjusting the bit width of the mantissa using a threshold according to one embodiment.

[0084] A neural network device according to one embodiment can receive the distribution of input data for each layer and determine thresholds corresponding to the input data of each layer. For example, referring to Figure 5, the neural network device according to one embodiment can determine four thresholds corresponding to the input data of the relevant layer. (outside 1) TIFF0007933577000001.tif819, (outside 2) TIFF0007933577000002.tif1119, (Outside 3) TIFF0007933577000003.tif1119, (outside 4) The data TIFF0007933577000004.tif1119 can be determined. The neural network device is the neural network device 200 shown in Figure 2.

[0085] In Figure 5, the threshold (outside 1) TIFF0007933577000005.tif819 and (outside 4) TIFF0007933577000006.tif1119 and (outside 2) TIFF0007933577000007.tif1119 and (Outside 3) TIFF0007933577000008.tif1119 has a symmetric relationship where each threshold has the same magnitude but different signs, but depending on the embodiment, the thresholds do not need to have a symmetric relationship with each other. Also, although four thresholds are explained as examples in Figure 5, the number of thresholds is not limited to this and can be adopted in various ways. Depending on the embodiment, the host of the neural network device or the hardware accelerator of the neural network device can determine the thresholds. For example, in the case of learned weight values, the host of the neural network device determines the threshold for the weight values, and the hardware accelerator of the neural network device receives the threshold determined from the host. Also, in the case of input feature map values, the hardware accelerator of the neural network device can determine the threshold for the input feature map.

[0086] The input data distribution refers to the weight distribution of the learned artificial neural network in the case of weight values, and to the distribution of the sampled input set in the case of input feature maps.

[0087] The neural network device can determine weight values ​​and thresholds corresponding to input feature maps using a brute-force algorithm. The neural network device can calculate the computational complexity and accuracy of each combination of thresholds and determine a combination of thresholds that satisfies predetermined criteria.

[0088] More specifically, for each combination of thresholds, the neural network device can use a brute-force algorithm to find the candidate threshold combination that has the lowest computational complexity and an average value-wise error smaller than a predetermined maximum average value-wise error. However, the method for determining the thresholds is not limited to the examples presented above and may include any algorithm that can determine the threshold combination that maximizes the ratio of computational accuracy to computational complexity.

[0089] After determining a threshold, the neural network device can classify the distribution of input data into multiple regions based on that threshold and assign a bit width corresponding to each region to the mantissa of the input data. For example, the neural network device may dynamically assign 10 bits, 8 bits, or 4 bits to the mantissa of the input data depending on the magnitude of the exponent.

[0090] More specifically, the neural network device has an input data index of 510 in the first domain.

[0091]

number

[0092]

number

[0093]

number

[0094]

number

[0095]

number

[0096] Figure 6 illustrates a method for performing dynamic floating-point mantissa multiplication by dynamically adjusting the number of calculation cycles using a fixed number of arithmetic units according to one embodiment.

[0097] Referring to Figure 6, Figure 610 is a diagram showing a method for performing mantissa multiplication between DFP16_L data and DFP16_S data using a temporal fusion method, and Figure 620 is a diagram showing a method for performing mantissa multiplication between DFP16_L data and DFP16_S data using a temporal fusion method.

[0098] The mixed precision arithmetic unit is the mixed precision arithmetic unit 420 shown in Figure 4, and includes a 4x4 multiplier, a shifter, and an accumulator. One set of arithmetic units (4x4 multiplier, shifter, and accumulator) can operate in multiple cycles to perform mantissa multiplication operations.

[0099] As illustrated with reference to Figure 620, when performing mantissa multiplication of two data represented as 16-bit floating-point numbers using a single 4x4 multiplier, 9 cycles are always consumed because 10 bits are fixedly allocated to the mantissa of the input data. However, when using input data converted to dynamic floating-point numbers according to one embodiment, performance can be improved by up to 9 times (for example, in the case of calculations between DFP16_S data and DFP16_S data, only 1 cycle is consumed).

[0100] For example, as illustrated in Figure 610, the mantissa multiplication operation between DFP16_L data and DFP16_S data consumes only 3 cycles. Specifically, the mixed precision arithmetic unit can complete the mantissa multiplication operation between DFP16_L data and DFP16_S data by performing the multiplication operation between the lower 4 bits of DFP16_L data and the 4 bits of DFP16_S data in the first cycle, the multiplication operation between the middle 4 bits of DFP16_L data and the 4 bits of DFP16_S data in the second cycle, and the multiplication operation between the upper 2 bits of DFP16_L data and the 4 bits of DFP16_S data in the third cycle.

[0101] Figure 7 illustrates a method for performing dynamic floating-point mantissa multiplication by dynamically adjusting the number of arithmetic units participating in the operation within a fixed arithmetic cycle.

[0102] Referring to Figure 7, Figure 710 shows a method for mantissa multiplication between DFP16_L data and DFP16_S data using a spatial fusion method.

[0103] The mixed precision arithmetic unit is the mixed precision arithmetic unit 420 shown in Figure 4, and includes multiple arithmetic units consisting of a 4x4 multiplier, a shifter, and an accumulator. For example, the mixed precision arithmetic unit may include nine arithmetic units.

[0104] For example, referring to Figure 710, in the case of mantissa multiplication between DFP16_L data and DFP16_S data, nine arithmetic units may perform three operations per cycle. That is, in this case, the mixed precision arithmetic unit can group three arithmetic units into one set of arithmetic units and perform operations on one DFP16_L data and three DFP16_S data (first DFP16_S data, second DFP16_S data, third DFP16_S data) in one cycle.

[0105] Specifically, the mixed precision arithmetic unit can perform multiplication operations between one DFP16_L data and three DFP16_S data (first DFP16_S data, second DFP16_S data, and third DFP16_S data) by performing multiplication operations between DFP16_L data and first DFP16_S data with the first set of arithmetic units, multiplication operations between DFP16_L data and second DFP16_S data with the second set of arithmetic units, and multiplication operations between DFP16_L data and third DFP16_S data with the third set of arithmetic units.

[0106] Figure 8 is a block diagram illustrating a computing device according to one embodiment.

[0107] Referring to Figure 8, an arithmetic unit 800 according to one embodiment includes a processor 810, a memory 830, and a communication interface 850. The processor 810, the memory 830, and the communication interface 850 can communicate via a communication bus 805.

[0108] The processor 810 receives multiple input data represented as floating-point numbers, adjusts the bit width of the mantissa of each input data based on the magnitude of the exponent of each input data, and performs operations between the input data with adjusted bit widths.

[0109] Memory 830 may be volatile memory or non-volatile memory.

[0110] In addition, the processor 810 may perform the methods or algorithms corresponding to the methods described above, with reference to Figures 1A to 7. The processor 810 can execute programs and control the arithmetic unit 800. The program code executed by the processor 810 is stored in memory 830. The arithmetic unit 800 can be connected to external devices (e.g., personal computers or networks) via input / output devices (not shown) to exchange data. The arithmetic unit 800 can be installed in various computing devices and / or systems such as smartphones, tablet computers, laptop computers, desktop computers, televisions, wearable devices, security systems, and smart home systems.

[0111] The embodiments described above are embodied in hardware components, software components, or combinations of hardware and software components. For example, the devices and components described in these embodiments are embodied using one or more general-purpose or special-purpose computers, such as a processor, controller, ALU (arithmetic logic unit), digital signal processor, microcomputer, FPA (field programmable array), PLU (programmable logic unit), microprocessor, or different devices that execute and respond to instructions. The processing device executes an operating system (OS) and one or more software applications that run on the OS. The processing device also accesses, stores, manipulates, processes, and generates data in response to the execution of the software. For convenience of understanding, the processing device may sometimes be described as being used as a single unit, but a person with ordinary skill in the art will understand that the processing device includes multiple processing elements and / or multiple types of processing elements. For example, the processing device includes multiple processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

[0112] Software includes computer programs, code, instructions, or a combination of one or more of these, which can configure a processing unit to operate as desired, or instruct the processing unit independently or in combination. Software and / or data can be permanently or temporarily embodied in any type of machine, component, physical device, virtual device, computer storage medium or device, or transmitted signal wave, for interpretation by a processing unit or for providing instructions or data to a processing unit. Software can be distributed across a network of computer systems and stored and executed in a distributed manner. Software and data can be stored on a recording medium readable by one or more computers.

[0113] The method according to this embodiment is embodied in the form of program instructions that are implemented via various computer means and recorded on a computer-readable recording medium. The recording medium includes program instructions, data files, data structures, etc., individually or in combination. The recording medium and program instructions may be specifically designed and configured for the purposes of the present invention, or they may be known and usable by those skilled in the art who have technology in the field of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floppy disks, and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, and flash memory. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code executed by a computer using an interpreter or the like.

[0114] The hardware device described above may be configured to operate as one or more software modules to perform the operations shown in the present invention, and vice versa.

[0115] As described above, although embodiments have been illustrated with limited drawings, a person with ordinary skill in the art can apply various technical modifications and variations based on the above description. For example, the described technique may be performed in a different order than described, and / or the described system, structure, apparatus, circuit, and other components may be combined or assembled in a different manner than described, or replaced or substituted by other components or equivalents, and still achieve the desired results.

[0116] Therefore, other embodiments, other embodiments, and claims equivalent to those described below also fall within the scope of the claims. [Explanation of Symbols]

[0117] 10: Input data 20: Artificial Neural Network 30: Output data 210: Host 220: Memory 230: Hardware accelerator

Claims

1. The steps include receiving multiple input data represented as floating-point numbers, A step of adjusting the bit width of the mantissa by masking the mantissa of each input data based on the magnitude of the exponent of each input data, The steps include performing operations between the input data whose bit width has been adjusted, A method of computation embodied by a processor, including [specific details omitted].

2. The arithmetic method embodied in the processor according to claim 1, wherein the step of adjusting the bit width of the mantissa of each of the input data includes the step of adjusting the bit width of the mantissa in proportion to the size of each of the input data.

3. The step of adjusting the bit width of the mantissa is: The steps include comparing each of the aforementioned input data with a threshold, Based on the results of the above comparison, the steps include adjusting the bit width of the mantissa, A calculation method embodied by the processor according to claim 1, including the above.

4. The calculation method embodied by the processor according to claim 3, wherein the threshold is determined based on the distribution of the input data and the acceptable error range.

5. The steps include receiving the distribution of the multiple input data, The steps include determining a threshold corresponding to each of the multiple input data based on the distribution of the multiple input data, A method of calculation embodied by the processor according to claim 1, further comprising:

6. The calculation method embodied in the processor according to claim 1, wherein the step of performing the calculation includes a step of controlling the position and timing of the arithmetic unit to which the bit-width adjusted input data is input.

7. The step of performing operations between the input data whose bit width has been adjusted is: A step of determining the number of cycles of the calculation to be performed by a predetermined number of arithmetic units based on the adjusted bit width of each of the input data, The steps include inputting the input data with the bit width adjusted in accordance with the determined number of cycles to the arithmetic unit, A calculation method embodied by the processor according to claim 1, including the above.

8. The calculation method embodied in the processor according to claim 7, wherein the step of determining the number of cycles of the calculation includes determining the number of cycles of the calculation based on the adjusted bit width of each of the input data and the number of bits that the arithmetic unit can process in one cycle.

9. The aforementioned arithmetic unit, A multiplier that performs integer multiplication of the mantissa of the input data, A shifter that shifts the result of the multiplier, A cumulative unit that accumulates the shifted results, A calculation method embodied by the processor according to claim 7, including the above.

10. The step of performing operations between the input data whose bit width has been adjusted is: A step of determining the number of arithmetic units to perform the calculation within a predetermined number of calculation cycles based on the adjusted bit width of each of the input data, The steps include inputting the bit-width adjusted input data to the arithmetic unit in accordance with the number of arithmetic units determined above, A calculation method embodied by the processor according to claim 1, including the above.

11. The arithmetic method embodied in the processor according to claim 10, wherein the step of determining the number of arithmetic units includes determining the number of arithmetic units based on the adjusted bit width of each of the input data and the number of bits that the arithmetic unit can process in one cycle.

12. A computer-readable storage medium storing one or more programs that include instruction words for performing the calculation method described in any one of claims 1 to 11.

13. An arithmetic unit comprising one or more processors configured to execute a program, wherein the program causes the one or more processors to execute the arithmetic method described in any one of claims 1 to 11.

14. A central processing unit receives a distribution of multiple input data expressed in floating-point numbers and determines a threshold corresponding to each of the multiple input data based on the distribution of the multiple input data. A hardware accelerator that receives the aforementioned multiple input data, masks the mantissa of each input data based on the magnitude of the exponent of each input data to adjust the bit width of the mantissa, and performs calculations between the input data with adjusted bit widths, Includes, Adjusting the bit width of the aforementioned mantissa means An arithmetic device that includes comparing each of the input data with the threshold and adjusting the bit width of the mantissa based on the result of the comparison.

15. The steps include receiving input data represented as a floating-point number, The steps include comparing the magnitude of the exponent of the input data with a threshold and adjusting the bit width of the mantissa of the input data, The steps include performing operations on the input data whose bit width has been adjusted, Includes, A processor-implemented arithmetic method comprising the step of adjusting the bit width of the mantissa of the input data, the step of maintaining the bit width of the mantissa if the exponent of the input data is greater than or equal to the threshold.

16. A step of receiving input data expressed in floating-point numbers, The steps include comparing the magnitude of the exponent of the input data with a threshold and adjusting the bit width of the mantissa of the input data, The steps include performing operations on the input data whose bit width has been adjusted, Includes, The threshold includes a plurality of threshold ranges corresponding to each bit width, A processor-implemented arithmetic method comprising the step of adjusting the bit width of the mantissa of the input data, the step of adjusting the bit width in response to input data corresponding to one of the plurality of threshold ranges such that the bit width of the mantissa corresponds to one of the plurality of threshold ranges.

17. The arithmetic method embodied in the processor according to claim 15 or 16, wherein the step of adjusting the bit width of the mantissa of the input data includes the step of assigning a bit width greater than the threshold or less than the same exponent to an exponent smaller than the threshold.

18. The step of performing the calculation includes the step of performing the calculation using an arithmetic unit, The arithmetic method embodied in the processor according to claim 15 or 16, wherein the adjusted bit width is less than or equal to the number of bits that the arithmetic unit can process in a single period.

19. The calculation method embodied in the processor according to claim 15 or 16, wherein the step of performing the calculation includes a step of performing multiplication and accumulation calculations using an arithmetic unit.

20. The steps include receiving weighted data and feature map data of a neural network expressed in floating-point format, The steps include comparing the magnitude of the exponent of the weighted value data with a threshold, comparing the magnitude of the exponent of the feature map data with other thresholds, and adjusting the bit width of the mantissa of the weighted value data and the bit width of the mantissa of the feature map data. The steps include performing a neural network operation between the weighted value data with adjusted bit width and the feature map data with adjusted bit width, A method of computation embodied by a processor, including [specific details omitted].

Images