Method and apparatus for quantizing parameters of a neural network

Abstract

A method of quantizing parameters of a neural network includes: calculating a bit shift value for each parameter, the bit shift value indicating a degree outside a bit range of a fixed-point format used to quantize the parameter; updating the fixed-point format based on the calculated bit shift value of the parameter; and quantizing the parameters updated in a learning or inference process according to the updated fixed-point format.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the priority of Korean Patent Application No. 10-2018-0123927 filed on October 17, 2018 in the Korean Intellectual Property Office, the disclosure of which is incorporated herein in its entirety by reference. Technical Field

[0003] The following description relates to methods and apparatus for quantizing parameters of a neural network. Background Art

[0004] Neural networks refer to computing architectures that mimic biological brains. With recent advances in neural network technology, a great deal of research has been conducted on using neural network devices in various electronic systems to analyze input data and extract useful information.

[0005] Neural network devices perform a large number of operations on input data. A technique is needed to efficiently process such neural network operations. Summary of the invention

[0006] This summary is provided to introduce a selection of concepts in a simplified form that are further described in the detailed description below. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0007] In one general aspect, a method of quantizing parameters of a neural network includes: for each parameter, calculating a bit shift value indicating a degree outside a bit range of a fixed-point format used to quantize the parameter; updating the fixed-point format using the calculated bit shift value of the parameter; and quantizing the parameters updated during learning or inference according to the updated fixed-point format.

[0008] Calculating the bit shift value may include: for each parameter, detecting the most significant bit having a value of "1"; and for each parameter, determining the difference in the number of bits between the detected most significant bit and the most significant bit of the integer part of the fixed-point format as the bit shift value.

[0009] Detecting the most significant bit may include searching for a bit within a specific range based on the most significant bit of the integer part of the fixed-point format for each parameter, and detecting the most significant bit having a value of '1'.

[0010] Updating the fixed-point format may include: determining a number of occurrences of overflow and a maximum bit shift value according to the calculated bit shift value; and updating the fixed-point format based on the number of occurrences of overflow and the maximum bit shift value.

[0011] Updating the fixed-point format may include updating the fixed-point format by reducing a fractional length of the fixed-point format by a maximum bit shift value if the number of occurrences of the overflow is greater than a specific value.

[0012] The specific value may be based on the number of parameters.

[0013] The updated parameters may be parameters updated during the t+1th learning or inference process, the parameters may be parameters updated during the tth learning or inference process, the fixed-point format may be a fixed-point format updated based on the parameters updated during the t-1th learning or inference process, and t may be a natural number greater than or equal to 2.

[0014] Calculating the bit shift value may include calculating the bit shift value for each parameter during quantization of the parameter according to the fixed point format.

[0015] Parameters can be weights or activations on the same layer in a neural network.

[0016] A computer-readable recording medium may store a program for causing a computer to execute the method.

[0017] In another general aspect, an apparatus for quantizing parameters of a neural network includes: a memory storing at least one program; and a processor configured to: calculate a bit shift value for each parameter by executing the at least one program, the bit shift value indicating a degree outside a bit range of a fixed-point format used to quantize the parameter, update the fixed-point format using the calculated bit shift value of the parameter; and quantize the parameters updated in a learning or inference process according to the updated fixed-point format.

[0018] The processor may detect a most significant bit having a value of "1" for each parameter; and determine a difference in the number of bits between the detected most significant bit and a most significant bit of the integer part of the fixed-point format as the bit shift value for each parameter.

[0019] The processor may search for bits within a specific range based on the most significant bit of the integer part of the fixed-point format, and detect the most significant bit having a value of '1'.

[0020] The processor may determine the number of occurrences of overflow and a maximum bit shift value according to the calculated bit shift value; and update the fixed-point format using the number of occurrences of overflow and the maximum bit shift value.

[0021] The processor may update the fixed-point format by reducing the fractional length of the fixed-point format by the maximum bit shift value if the number of occurrences of the overflow is greater than a predetermined value.

[0022] The processor may calculate the bit shift value for each parameter during quantization of the parameters according to the fixed point format.

[0023] In another general aspect, a method includes: for each parameter updated in a t-th learning or inference process of a neural network, calculating a bit shift value based on a fixed-point format used to quantize the parameter; determining a number of occurrences of overflow and a maximum bit shift value according to the calculated bit shift value; updating the fixed-point format based on the number of occurrences of overflow and the maximum bit shift value; and quantizing the parameter in a t+1-th learning or inference process of the neural network based on the updated fixed-point format, where t is a natural number greater than or equal to 2.

[0024] Determining the number of occurrences of overflow may include determining whether the bit shift value of each parameter is greater than 0, and increasing the number of occurrences of overflow by 1 for each bit shift value greater than 0.

[0025] Determining the maximum bit shift value may include comparing the calculated bit shift values ​​of the parameters updated in the t-th learning or inference process with each other, and determining the maximum value among the bit shift values ​​updated in the t-th learning or inference process as the maximum bit shift value.

[0026] In another general aspect, a computer-readable recording medium has recorded thereon one or more programs including instructions for executing a method of quantizing parameters of a neural network.

[0027] Other features and aspects will be apparent from the following detailed description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a diagram for explaining a neural network learning device and a neural network inference device according to an example.

[0029] Figure 2 is a diagram illustrating an example of a neural network according to an example.

[0030] Figure 3A is a diagram showing floating point format parameters according to an example, Figure 3B is a diagram showing fixed-point format parameters according to an example, and Figure 3C is a diagram illustrating fixed-point format parameters according to another example.

[0031] Figure 4 is a block diagram of a hardware configuration of a neural network device according to an example.

[0032] Figure 5 An example of a processor operating in an iterative learning or inference process is shown.

[0033] Figure 6 An example of a processor updating a fixed-point format using a bit-shifted value of a parameter is shown.

[0034] Figure 7 An example of a processor calculating a bit-shifted value of a parameter is shown.

[0035] Figure 8 An example of a most significant bit having "1" in the processor detection parameter is shown.

[0036] Fig. 9 An example of a processor determining a maximum bit shift value and a number of occurrences of overflow is shown.

[0037] Fig.10 is a diagram illustrating an algorithm for a processor to update a fractional length of a fixed-point format according to an example.

[0038] Fig.11 is a diagram illustrating an algorithm for a processor to update a fractional length of a fixed-point format according to another example.

[0039] Fig.12 is a block diagram of an electronic system according to an example.

[0040] Fig.13 is a diagram for explaining a method of operating a neural network device according to an example.

[0041] Throughout the drawings and detailed description, unless otherwise described or provided, the same reference numerals will be understood to refer to the same elements, features, and structures. The drawings may not be drawn to scale, and the relative sizes, proportions, and depictions of elements in the drawings may be exaggerated for clarity, illustration, and convenience. DETAILED DESCRIPTION

[0042] Now will be detailed with reference to embodiment, the example of embodiment is shown in the accompanying drawings, wherein, similar reference numerals throughout the accompanying drawings refer to similar elements. In this regard, the embodiment presented can have different forms, and should not be construed as being limited to the description set forth herein. Therefore, the embodiment is described below only by reference to the accompanying drawings to explain various aspects. The term "and / or" used herein includes any and all combinations of one or more items in the relevant listed items. Statements such as "at least one of ... " modify the entire element list when it is before the element list, rather than modifying the individual elements in the list.

[0043] The terms “include” or “comprising” and the like used herein should not be interpreted as necessarily including all the various elements or operations described in the specification, and should exclude some of them, or may be interpreted as including additional components or operations.

[0044] In addition, as used herein, terms including ordinal numbers such as "first" or "second" may be used to describe various elements, but these elements should not be limited by these terms. These terms are only used to distinguish one element from another element.

[0045] The examples relate to a method and apparatus for processing parameters of a neural network, and a detailed description thereof will be omitted regarding matters widely known to those of ordinary skill in the art.

[0046] Figure 1 2 is a diagram for explaining a neural network learning device 10 and a neural network inferring device 20 according to an example.

[0047] refer to Figure 1 , the neural network learning device 10 may correspond to a computing device having various processing functions, such as the following functions: generating a neural network, training or learning a neural network, quantizing a floating point format neural network to a fixed point format neural network, quantizing a fixed point format neural network to another fixed point format neural network, or retraining a neural network. For example, the neural network learning device 10 may be implemented as various types of devices, such as a personal computer (PC), a server device, a mobile device, etc. Meanwhile, parameter quantization may represent conversion of a floating point format parameter to a fixed point format parameter, or may represent conversion of a fixed point format parameter having a specific bit width to another fixed point format parameter having another bit width.

[0048] The neural network learning device 10 can generate a trained neural network 11 by repeatedly training (learning) a given initial neural network. In this state, the initial neural network can have floating point format parameters, for example, parameters of 32-bit floating point precision, in terms of ensuring the processing accuracy of the neural network. The parameters may include various types of data input to or output from the neural network, for example, input / output activations, weights, biases, etc. of the neural network. As the neural network is repeatedly trained, the floating point parameters of the neural network can be adjusted to output a more accurate output relative to a given input.

[0049] The neural network learning device 10 can process parameters according to the fixed-point format in the process of repeatedly learning (training) the initial neural network. Specifically, the neural network learning device 10 can process parameters according to the 8-bit or 16-bit fixed-point format so as to learn the neural network within the allowable accuracy loss while substantially reducing the number of operations. Therefore, the neural network learning device 10 can be implemented in a smartphone, tablet computer, or wearable device with relatively low processing power for device-side learning.

[0050] The neural network learning device 10 may send the trained neural network 11 to a hardware accelerator (e.g., a neural network inference device 20). The neural network inference device 20 may be included in a mobile device, an embedded device, etc. The neural network inference device 20 is a dedicated hardware for driving a quantized neural network 21. Since the neural network inference device 20 is implemented with relatively low power or low performance, the neural network inference device 20 may be implemented to be more suitable for fixed-point operations rather than floating-point operations. The neural network inference device 20 may correspond to (but not limited to) a tensor processing unit (TPU), a neural engine, etc., which is a dedicated module for driving a neural network.

[0051] The neural network inference device 20 for driving the quantized neural network 21 may be implemented in a separate device from the neural network learning device 10. However, the present disclosure is not limited thereto, and the neural network inference device 20 may be implemented in the same device as the neural network learning device 10.

[0052] The neural network inference device 20 deploying the quantized neural network 21 may be included in, for example, an autonomous driving vehicle, a robot, a smart phone, a tablet device, an augmented reality (AR) device, an Internet of Things (IoT) device, etc., which performs speech recognition, image recognition, etc. by using a neural network, but the present disclosure is not limited thereto.

[0053] Figure 2 is a diagram showing an example of a neural network 2.

[0054] refer to Figure 2 The neural network 2 may have a structure including an input layer, a hidden layer, and an output layer, and may be based on the received input data (e.g., I 1 and I 2 ) to perform the operation, and can generate output data (e.g., 1 and O 2 ).

[0055] The neural network 2 may be a deep neural network (DNN) or an n-layer neural network including one or more hidden layers. Figure 2 As shown, the neural network 2 may be a DNN including an input layer 1, two hidden layers (layer 2 and layer 3), and an output layer 4. The DNN may include a convolutional neural network (CNN), a recurrent neural network (RNN), a deep belief network, a restricted Boltzmann machine, etc., but is not limited thereto.

[0056] Although the neural network 2 is shown as including four layers, this is merely exemplary, and the neural network 2 may include more or fewer layers or more or fewer nodes. Figure 2The structures shown are different layers of various structures. For example, the neural network 2 may include a convolutional layer, a pooling layer, and a fully connected layer as a DNN.

[0057] Each layer included in the neural network 2 may include a plurality of artificial nodes referred to as neurons, processing elements (PEs), units, or similar terms. Figure 2 As shown, layer 1 may include two nodes, and layer 2 may include three nodes. However, this is merely exemplary, and each layer included in the neural network 2 may include various numbers of nodes.

[0058] The nodes included in each layer of the neural network 2 can be connected to each other to process data. For example, one node can receive data from other nodes and process the data, and can output operation results to other nodes.

[0059] The output value of each node may be referred to as an activation. An activation may be the output of one node and the input value of a node included in the next layer. Each node may determine its own activation based on the activation received from the node included in the previous layer and the weight. The weight is a parameter used to calculate the activation in each node and may be a value assigned to a connection relationship between nodes.

[0060] Each node can be processed by a computational unit that receives input and outputs activation, and can map input and output. In σ, the activation function is represented. represents the weight from the kth node included in the (i-1)th layer to the jth node included in the i-th layer, represents the bias of the jth node included in the i-th layer and When we denote the activation of the jth node in the i-th layer, we can calculate the activation by using the following equation 1:

[0061] Equation 1

[0062]

[0063] like Figure 2 As shown, the activation of the first node in the second layer 2 can be expressed as Furthermore, according to Equation 1, Can have value However, the above equation 1 is only an example for describing activation, weight, and bias for processing data in the neural network 2, and the present disclosure is not limited thereto. The activation may be a value obtained by allowing a value obtained by applying an activation function to a weighted sum of activations received from a previous layer to pass through a rectified linear unit (ReLU).

[0064] As described above, in the neural network 2, a large amount of data sets are exchanged between a plurality of interconnected nodes and these data sets undergo a number of calculation processes in each layer.

[0065] Figure 3A is a diagram illustrating floating point format parameters 30 according to an example. Figure 3B is a diagram illustrating fixed-point format parameters 35 according to an example. Figure 3C is a diagram illustrating fixed-point format parameters according to another example.

[0066] refer to Figure 3A , the floating point format parameter 30 may include a sign bit 310, an exponent part 320, a mantissa part 330, and a bias 340. The floating point indication divides a number into a part indicating a significant figure part (ie, mantissa) and a part indicating a decimal point position.

[0067] The mantissa part 330 may correspond to a part indicating a significant digit part. The exponent part 320 may correspond to a part indicating a decimal point position. The sign bit 310 may determine the sign of the floating point format parameter 30. The bias 340 may be a value added to the exponent part 320 or a value subtracted from the exponent part 320 and determined as a value representing a negative exponent. The floating point format parameter 30 may include the sign bit 310, a bit corresponding to the exponent part 320, and a bit corresponding to the mantissa part 330. The bias 340 may be predetermined with respect to the floating point format parameter 30 and stored separately.

[0068] When the sign bit 310 indicates a sign, the exponent part 320 indicates an exponent, the mantissa part 330 indicates a mantissa, and the bias 340 indicates a bias, the floating point format parameter 30 may be a value according to the following Equation 2.

[0069] Equation 2

[0070] Floating point value = (-1) sign 2 exponent-bias ·mantissa

[0071] refer to Figure 3B , the fixed-point format parameter 35 may include a sign bit 315, an integer part 325, a fractional part 335, and a decimal point 345. The fixed-point indicates a notation that uses a decimal point to represent a fixed number of decimal places.

[0072] The sign bit 315 may determine the sign of the fixed-point format parameter 35. The integer portion 325 may correspond to a portion representing an integer of the fixed-point format parameter 35. The fractional portion 335 may correspond to a portion representing a fraction of the fixed-point format parameter 35. The decimal point 345 may indicate a point as a reference for distinguishing the integer portion 325 and the fractional portion 335 of the fixed-point format parameter 35.

[0073] You can refer to Figure 3C To describe the value represented by the fixed-point format parameter 35. Figure 3C The fixed-point format parameter 35 may be an 8-bit fixed-point value. The fixed-point format parameter 35 may also include an integer part 325, a fractional part 335, and a decimal point 345.

[0074] Each bit representing the integer part 325 and the fractional part 335 can have a value of 1 or 0. Figure 3C As shown, the bits representing the integer part 325 and the fractional part 335 may sequentially have values ​​of -8, +4, +2, +1, +0.5, +0.25, +0.125, and +0.0625. When the most significant bit of the integer part 325 is 1, since the value represented by the most significant bit of the integer part 325 is -8, even if other bits included in the integer part 325 and the fractional part 335 have specific values, the value represented by the fixed-point format parameter 35 may be negative. The most significant bit of the integer part 325 may correspond to the sign bit 315 that determines the sign of the fixed-point format parameter 35.

[0075] However, Figure 3C This is only an example, and the fixed-point format parameter 35 may be a 16-bit fixed-point value, and may be a fixed-point value having any suitable number of bits. In addition, the fixed-point format parameter 35 may represent a negative number by any one of a coding representation method, a 1's complement method, and a 2's complement method.

[0076] Figure 4 is a block diagram of a hardware configuration of neural network device 100 according to an example.

[0077] The neural network device 100 may be operated by being included in at least one of the neural network learning device 10 and the neural network inference device 20, or may be operated as an independent third hardware accelerator.

[0078] refer to Figure 4 , the neural network device 100 may include a processor 110 and a memory 120. Figure 4 In FIG. 1 , only the components of the neural network device 100 of this example are shown. Therefore, it is obvious to a person skilled in the art that the neural network device 100 may also include Figure 4Common constituent elements other than the constituent elements shown in .

[0079] Processor 110 performs all functions to control neural network device 100. For example, processor 110 controls all functions of neural network device 100 by executing a program stored in memory 120 in neural network device 100. Processor 110 may be implemented by a central processing unit (CPU), a graphics processing unit (GPU), an application processor (AP), etc. provided in neural network device 100. However, the present disclosure is not limited thereto.

[0080] Memory 120 is hardware for storing various pieces of data processed in neural network device 100. For example, memory 120 may store data processed in neural network device 100 or to be processed in neural network device 100. In addition, memory 120 may store applications, drivers, etc. to be driven by neural network device 100. Memory 120 may be (dynamic random access memory) DRAM, but the present disclosure is not limited thereto. Memory 120 may include at least one of a volatile memory or a non-volatile memory.

[0081] The processor 110 may generate a trained neural network by repeatedly learning an initial neural network. The processor 110 may repeatedly update the parameters of the neural network by repeatedly learning the neural network. For example, the weights in the neural network may be repeatedly updated during the learning process, and the activation may also be repeatedly updated due to the operation using the weights. Each time the parameters of the neural network are repeatedly updated, the processor 110 may quantize the updated parameters in a fixed-point format. In addition, whenever the parameters of the neural network are repeatedly updated, the processor 110 may repeatedly update the fixed-point format. When the variable parameters are quantized in a fixed-point format, since a loss of accuracy may occur during the learning process, the processor 110 may update the fixed-point format to quantize the parameters, thereby reducing the loss of accuracy while reducing the number of operations. Specifically, the processor 110 may update the fixed-point format to correspond to the distribution of the fixed-point values ​​of the updated parameters. For example, the processor 110 may update the decimal point position in the fixed-point format to correspond to the parameter with the maximum value in the updated parameters.

[0082] The processor 110 may repeatedly update the parameters of the neural network even during an inference process of driving the learned neural network to obtain a result value. For example, during an inference process, data may be repeatedly input to the learned neural network, and thus activations in the learned neural network may be repeatedly updated. Therefore, similar to the learning process, the processor 110 may quantize the updated parameters according to a fixed-point format each time the parameters of the neural network are repeatedly updated, even during the inference process. Furthermore, similar to the learning process, the processor 110 may repeatedly update the fixed-point format each time the parameters of the neural network are repeatedly updated, even during the inference process.

[0083] The processor 110 may update the parameters during the t-th learning or inference process, and then may quantize the updated parameters. At this time, the processor 110 may quantize the updated parameters during the t-th learning or inference process according to a specific first fixed-point format. For example, the first fixed-point format may be an updated fixed-point format based on the parameters updated during the t-1-th learning or inference process. The processor 110 may update the existing first fixed-point format to a second fixed-point format based on the parameters updated during the t-th learning or inference process. Subsequently, the processor 110 may quantize the parameters updated during the t+1-th learning or inference process according to the second fixed-point format.

[0084] Therefore, when quantizing the parameters updated during the tth learning or inference process, the processor 110 can quantize the parameters according to a specific fixed-point format, thereby reducing the time for scanning all parameters updated during the tth learning or inference process and the hardware overhead for determining the fixed-point format.

[0085] Figure 5 An example of processor 110 operating in an iterative learning or inference process is shown.

[0086] The processor 110 may be configured to generate a number having a fractional length frac_length. t-1 The fixed-point format is used to quantize the parameters updated during the t-th learning or inference process. t , to generate the quantized parameter q_parameter t In other words, the processor 110 may update the parameter parameter in the t-1th learning or inference process based on the parameter parameter t-1 To predetermine the fractional length frac_length t-1 fixed-point format, and in the t-th learning or inference process, according to the fractional length frac_length t-1 The fixed-point format for parameter parameter t Quantify.

[0087] The processor 110 may be based on the parameter parameter t Set the existing fractional length frac_length t-1 Updated to fractional length frac_length t In other words, the processor 110 may predetermine the parameter parameter used to quantify the update in the t+1th learning or inference process. t+1 Fractional length frac_length t .

[0088] The processor 110 may set the parameter parameter t Update to parameter parameter t+1 In addition, the processor 110 may be configured to generate a number having a fractional length frac_length. t The fixed-point format for parameter parameter t+1 Quantization is performed to generate a quantized parameter q_parameter t+1 .

[0089] exist Figure 5 In the example, the parameter t-1 、parameter t and parameter t+1 It may be a data tensor, which is, for example, a weight or input / output activation on the same layer in a neural network, and according to another example, a data tensor is a weight or input / output activation on the same channel in a neural network. In addition, t may be a natural number of 1 or greater, and when t is 1, a fixed-point format for quantizing parameters updated during the first learning or inference process may be preset by a user.

[0090] Reference again Figure 4, the processor 110 may calculate a bit shift value for each parameter, the bit shift value indicating the degree of being outside the bit range of the fixed-point format used to quantize the parameter. The bit shift value may be a value indicating the degree to which the bit range representing the parameter value is outside the bit range that can be covered by the fixed-point format. According to an example, in the case where the fixed-point format can cover up to 3 bits of integer part, when up to 6 bits of integer part are required to represent the parameter value with the fixed-point format, the bit shift value may be 3. As another example, in the case where the fixed-point format can cover up to 4 bits of fractional part, when up to 6 bits of fractional part are required to represent the parameter value with the fixed-point format, the bit shift value may be -2. The bit shift value also indicates the degree to which overflow or underflow occurs when quantizing the parameter with the fixed-point format. For example, when the bit shift value is 3, it can be seen that a 3-bit overflow occurs, and when the bit shift value is -2, it can be seen that a 2-bit underflow occurs. Therefore, the processor 110 may calculate the bit shift value of the parameter to determine the total number of occurrences of overflow or underflow related to the parameter. Furthermore, the processor 110 may store the bit shift value, the parameter, the number of occurrences of overflow, and the number of occurrences of underflow in the memory 120 .

[0091] The processor 110 may update the fixed-point format using the bit shift value of the parameter. According to an example, when the number of occurrences of the overflow is greater than a predetermined value, the processor 110 may update the fixed-point format by reducing the length of the fractional part in the fixed-point format by the maximum bit shift value in the bit shift value. According to another example, when the number of occurrences of the overflow is greater than a predetermined value, the processor 110 may update the fixed-point format using the maximum bit shift value and the minimum bit shift value in the bit shift value. In addition, the processor 110 may store information related to the updated fixed-point format in the memory 120.

[0092] Then, the processor 110 may quantize the parameters updated in the learning or inference process according to the updated fixed-point format. Specifically, the processor 110 may quantize the parameters updated in the t+1th learning or inference process according to the fixed-point format updated based on the bit shift value of the parameter updated in the tth learning or inference process.

[0093] Therefore, the processor 110 may update the fixed-point format using the number of occurrences of overflow of the parameter and the maximum bit shift value, thereby updating the fixed-point format through a relatively simple operation with fewer operations, and thus hardware overhead may be reduced.

[0094] Figure 6 An example is shown where the processor 110 updates a fixed-point format using a bit-shifted value of a parameter.

[0095] The processor 110 may calculate the parameter parameter updated during the t-th learning or inference process.t (1) To parameter t (i) (where i is a natural number greater than or equal to 2). Specifically, the processor 110 may calculate the bit shift value of each parameter parameter t (1) To parameter t Each parameter in (i) (where i is a natural number greater than or equal to 2) is relative to a predetermined fractional length frac_length t-1 The bit-shift value of the bit range in fixed-point format.

[0096] Figure 7 The processor 110 calculates the i-th parameter parameter t (i) Examples of bit shift values.

[0097] The processor 110 may calculate the i-th parameter parameter among the parameters updated in the t-th learning or inference process. t (i) is the bit shift value of

[0098] The processor 110 may detect the i-th parameter parameter t The most significant bit in (i) has the value "1". Figure 7 , the processor 110 may convert the i-th parameter parameter t The sixth bit of the integer part in (i) is detected as the most significant bit.

[0099] Then, the processor 110 may use the parameter used to quantize the i-th parameter t (i) has a fractional length of frac_length t-1 The most significant bit of the integer part of the fixed-point format is matched to the previously detected i-th parameter parameter t The difference in the number of bits between the most significant bits of (i) is determined as the i-th parameter parameter t (i) Bit shift value. Figure 7 , due to the fractional length frac_length t-1 The third bit of the integer part of the fixed-point format of is the most significant bit, so the difference in the number of bits between the sixth bit and the third bit is 3, and thus the processor 110 can set the i-th parameter parameter t The bit shift value of (i) is calculated to be 3. A bit shift detector for detecting a difference in the number of bits may be included in the processor 110. In addition, the processor 110 may determine the bit shift value relative to the i-th parameter parameter t (i) A 3-bit overflow occurred.

[0100] The processor 110 may be configured to generate a number having a fractional length frac_length. t-1 The fixed-point format is used to quantize the i-th parameter parameter t (i) To generate a quantized parameter q_parameter t (i), thus quantizing the i-th parameter parameter t (i) process calculates the i-th parameter parameter t In other words, the processor 110 may calculate the i-th parameter parameter t The process of bit shifting the value of (i) and quantizing the i-th parameter parameter t (i) is performed together.

[0101] exist Figure 7 In the tth learning or inference process, the i-th parameter parameter updated t (i) is shown as a 48-bit fixed point value and the quantized parameter q_parameter t (i) is shown as a fixed-point value with a bit width of 8 and a fraction length of 4, but is not limited to such.

[0102] Figure 8 The processor 110 detects the parameter parameter t Example with the most significant bit being "1" in (i).

[0103] The processor 110 can obtain the parameter parameter t The bit values ​​are read sequentially from the most significant bit in (i) in the bit decreasing direction to detect the parameter parameter t The most significant bit in (i) has "1".

[0104] According to another example, the processor 110 may read the parameter parameter t (i) bits within a predetermined range related to the predetermined bit in order to detect the parameter parameter t (i) has the most significant bit of "1". Figure 8 As shown, the processor 110 can read the fractional length frac_length t-1 The most significant bit of the integer part of the fixed-point format is related to the bits within the 8-bit or 16-bit range to detect the parameter parameter t(i) has the most significant bit of "1". Therefore, the processor 110 can read the bits within the predetermined range without reading the parameter parameter t (i), thereby reducing the hardware overhead for scanning.

[0105] Reference again Figure 6 , the processor 110 may be configured to generate a t (1) To parameter t Specifically, the processor 110 may determine the maximum value of the bit shift value as the maximum bit shift value, and determine the number of bit shift values ​​corresponding to positive numbers in the bit shift value as the frequency of overflow occurrence.

[0106] Then, the processor 110 may update the fixed-point format based on the maximum bit shift value and the number of overflow occurrences. In other words, the processor 110 may update the fixed-point format with a fractional length frac_length based on the maximum bit shift value and the number of overflow occurrences. t-1 The fixed-point format is updated to have a fractional length of frac_length t Specifically, when the number of overflows is greater than a certain value, the processor 110 may set the fraction length frac_length to t-1 Reduce the maximum bit shift value and increase the fraction length from the existing fraction length frac_length t-1 Updated to the new fraction length frac_length t .

[0107] Therefore, the processor 110 may be configured to generate a number having a fractional length frac_length. t The fixed-point format is used to quantize the parameters updated during the t+1th learning or inference process.

[0108] Fig. 9 An example is shown in which the processor 110 determines a maximum bit shift value and a number of occurrences of overflow.

[0109] Similar to Fig. 9 The processor 110 may determine the maximum bit shift value and the number of overflow occurrences by performing logical operations on a total of N parameters, where the N parameters are parameters updated during the t-th learning or inference process. The processor 110 may include a processor for executing Fig. 9 The logical operation shown in the logic operator.

[0110] In operation s910, the processor 110 may determine whether the bit shift value bit_shift(i) of the i-th parameter in the parameters is greater than 0. In other words, the processor 110 may determine whether the i-th parameter is a parameter corresponding to an overflow. When the bit shift value bit_shift(i) of the i-th parameter is greater than 0, the processor 110 may increase the number of occurrences of the overflow by 1 (s920). Thereafter, the processor 110 may determine whether the bit shift value bit_shift(i+1) of the i+1-th parameter is greater than 0 to determine the number of occurrences of the overflow. As a result, the processor 110 may sequentially determine whether each bit shift value of the N parameters updated during the t-th learning or inference process is greater than 0, and determine the total number of occurrences of the overflow during the t-th learning or inference process.

[0111] In operation s930, the processor 110 may compare the bit shift value bit_shift(i) of the i-th parameter with the existing maximum bit shift value max_bit_shift t When the bit shift value bit_shift(i) is greater than the existing maximum bit shift value max_bit_shift t When the processor 110 updates the bit shift value bit_shift(i) to the maximum bit shift value max_bit_shift t (s940). Then, the processor 110 can compare the bit shift value bit_shift(i+1) of the i+1th parameter with the updated maximum bit shift value max_bit_shift t Compare to determine whether to update the maximum bit shift value max_bit_shift t As a result, the processor 110 may compare the bit shift values ​​of the N parameters updated in the t-th learning or inference process with each other, thereby determining the maximum value among the bit shift values ​​as the maximum bit shift value max_bit_shift t .

[0112] The processor 110 may determine a minimum bit shift value corresponding to an underflow among the bit shift values ​​of the N parameters. Specifically, the processor 110 may compare the bit shift values ​​of the N parameters with each other to determine a minimum bit shift value, which is a minimum value among the bit shift values ​​having a value less than or equal to 0.

[0113] Fig.10 FIG. 1 is a diagram showing a method for the processor 110 to update the fractional length frac_length of a fixed-point format according to an example. t-1 Graph of the algorithm.

[0114] In operation s1010, the processor 110 may obtain a maximum bit shift value max_bit_shift t , the total number of overflow occurrences #of overflow t , the length of the fixed-point format fraction used to quantize the parameters updated during the tth learning or inference process, frac_length t-1 , the total number of parameters N updated during the t-th learning or inference process, and the abnormal value data rate TH. The processor 110 may Fig. 9 The logical operation shown determines the maximum bit shift value max_bit_shift t , the total number of overflow occurrences #of overflow t In addition, the user can set the outlier data rate TH to any number between 0 and 1.

[0115] In operation s1020, when the total number of overflows occurs #of overflow t When it is greater than Nx TH, the processor 110 may update the fractional length frac_length by equation 1030 t-1 In equation 1030, a represents a specific weight. For example, when a is 1, the processor 110 may perform t-1 Subtract the maximum bit shift value max_bit_shift from t Operation to update the fraction length frac_length t-1 In other words, the processor 110 may execute the operation from the fraction length frac_length t-1 Subtract the maximum bit shift value max_bit_shift from t Operation to increase the length of the fraction from the existing fraction length frac_length t-1 Update to the new fraction length frac_length t Therefore, the processor 110 may determine the fractional length frac_length in a fixed-point format used to quantize the parameters updated during the t+1th learning or inference process. t .

[0116] When the total number of overflow t When the value is not greater than Nx TH, the processor 110 may not update and maintain the fractional length frac_length t-1 In other words, the processor 110 may use the fractional length frac_length used to quantize the parameters updated in the t-th learning or inference process to t-1Determines the length of the fraction used to quantize the parameters updated during the t+1th learning or inference process, frac_length t As a result, the processor 110 may set the outlier data rate TH to a value between 0 and 1, and thus when determining the total number of overflows #of overflow t If the length of the fraction frac_length is relatively small compared to the total number of parameters N, the fraction length may not be updated. t-1 .

[0117] Fig.11 FIG. 1 is a diagram showing a method for the processor 110 to update the fractional length frac_length of a fixed-point format according to another example. t-1 Graph of the algorithm.

[0118] Fig.11 Operations s1110 and s1120 may correspond to Fig.10 Operations s1010 and s1020 are described below, and thus redundant descriptions thereof will be omitted.

[0119] In operation s1110, in addition to Fig.10 In addition to the information obtained in operation s1010, the processor 110 may also obtain a minimum bit shift value min_bit_shift t In other words, the processor 110 may obtain a minimum bit shift value min_bit_shift corresponding to an underflow in a bit shift value of a parameter updated in the t-th learning or inference process. t .

[0120] In operation s1120, when the total number of overflows #of overflow t When it is greater than NXTH, the processor 110 may update the fraction length frac_length by equation 1130 t-1 In equation 1130, a and b are values ​​representing specific weights, and the symbol [x] is the Gaussian function symbol representing the largest integer not exceeding x. For example, a user may set a larger weight to a than to b to determine the fractional length frac_length. t .

[0121] Fig.12 is a block diagram of an electronic system 1200 according to an example.

[0122] refer to Fig.12The electronic system 1200 can extract effective information by analyzing input data in real time based on a neural network, determine a condition based on the extracted information, or control a component of an electronic device on which the electronic system 1200 is installed. For example, the electronic system 1200 can be applied to a robot device (e.g., a drone, an advanced driver assistance system (ADAS)), a smart TV, a smart phone, a medical device, a mobile device, an image display device, a measuring device, an IoT device, etc., and can also be installed on at least one electronic device among various types of electronic devices.

[0123] The electronic system 1200 may include a processor 1210, a RAM 1220, a neural network device 1230, a memory 1240, a sensor module 1250, and a communication (TX / RX) module 1260. The electronic system 1200 may also include an input / output module, a security module, a power control device, etc. Some hardware components of the electronic system 1200 may be mounted on at least one semiconductor chip. The neural network device 1230 may include the above-mentioned neural network device 100 or a neural network dedicated hardware accelerator or a device including the same.

[0124] The processor 1210 controls all operations of the electronic system 1200. The processor 1210 may include one processor core (single core) or multiple processor cores (multi-core). The processor 1210 may process or execute programs and / or data stored in the memory 1240. The processor 1210 may control the functions of the neural network device 1230 by executing the programs stored in the memory 1240. The processor 1210 may be implemented by a CPU, a GPU, an AP, etc.

[0125] The RAM 1220 may temporarily store programs, data, or instructions. For example, the programs and / or data stored in the memory 1240 may be temporarily stored in the RAM 1220 according to the control of the boot code or the processor 1210. The RAM 1220 may be implemented by a memory such as a dynamic RAM (DRAM) or a static RAM (SRAM).

[0126] The neural network device 1230 may perform a neural network operation based on the received input data, and generate an information signal based on the result of the operation. The neural network may include a CNN, an RNN, a deep belief network, a restricted Boltzmann machine, etc., but the present disclosure is not limited thereto. The neural network device 1230 is hardware that drives the above-mentioned neural network for classification, and may correspond to a neural network dedicated hardware accelerator.

[0127] The information signal may include one of various types of recognition signals, such as a voice recognition signal, an object recognition signal, an image recognition signal, a biometric information recognition signal, etc. For example, the neural network device 1230 may receive frame data included in a video stream as input data, and generate a recognition signal for an object included in an image indicated by the frame data based on the frame data. However, the present disclosure is not limited thereto, and depending on the type or function of an electronic device on which the electronic system 1200 is installed, the neural network device 1230 may receive various types of input data and generate a recognition signal based on the input data.

[0128] The memory 1240 is a storage device for storing data (e.g., an operating system (OS), various programs, and various data). In an embodiment, the memory 1240 may store intermediate results generated during the execution of operations of the neural network device 1230, such as output feature maps, as output feature lists or external feature matrices. In an embodiment, the memory 1240 may store compressed output feature maps. In addition, the memory 1240 may store quantized neural network data used by the neural network device 1230, such as parameters, weight maps, or weight lists.

[0129] The memory 1240 may be a DRAM, but the present disclosure is not limited thereto. The memory 1240 may include at least one of a volatile memory and a non-volatile memory. The non-volatile memory may include a ROM, a PROM, an EPROM, an EEPROM, a flash memory, a PRAM, an MRAM, an RRAM, a FRAM, etc. The volatile memory may include a DRAM, a SRAM, a SDRAM, a PRAM, an MRAM, an RRAM, a FeRAM, etc. In an embodiment, the memory 1240 may include at least one of a HDD, a SSD, a CF, a SD, a micro SD, a mini SD, an xD, and a memory stick.

[0130] The sensor module 1250 may collect information about the periphery of the electronic device on which the electronic system 1200 is mounted. The sensor module 1250 may sense or receive a signal from the outside of the electronic device, such as an image signal, a voice signal, a magnetic signal, a biometric signal, a touch signal, etc., and convert the sensed or received signal into data. To this end, the sensor module 1250 may include at least one of various types of sensing devices, such as a microphone, an imaging device, an image sensor, a light detection and ranging (LIDAR) sensor, an ultrasonic sensor, an infrared sensor, a biosensor, a touch sensor, etc.

[0131] The sensor module 1250 may provide the converted data as input data to the neural network device 1230. For example, the sensor module 1250 may include an image sensor, and may generate a video stream by photographing the external environment of the electronic device, and provide continuous data frames of the video stream to the neural network device 1230 in the order of the input data. However, the present disclosure is not limited thereto, and the sensor module 1250 may provide various types of data to the neural network device 1230.

[0132] The communication module 1260 may include various wired or wireless interfaces capable of communicating with external devices. For example, the communication module 1260 may include a local area network (LAN), a wireless local area network (WLAN) such as wireless fidelity (Wi-Fi), a wireless personal area network (WPAN) such as Bluetooth, a wireless universal serial bus (USB), ZigBee, near field communication (NFC), radio frequency identification (RFID), power line communication (PLC), or a communication interface capable of connecting to a mobile cellular network (e.g., 3rd generation (3G), 4th generation (4G), long term evolution (LTE), etc.).

[0133] Fig.13 is a diagram for explaining a method of operating the neural network device 100 according to an embodiment.

[0134] Fig.13 The method shown can be Figure 4 The neural network device 100 or Fig.12 The electronic system 1200 is executed by each component, and redundant explanations are omitted.

[0135] In operation s1310, the neural network device 100 may calculate a bit shift value for each parameter, the bit shift value indicating the degree outside the bit range of the fixed-point format used to quantize the parameter. Specifically, the neural network device 100 may calculate the bit shift value of each parameter updated in the t-th learning or inference process relative to the bit range of the predetermined fixed-point format based on the parameter updated in the t-1-th learning or inference process. The neural network device 100 may calculate the bit shift value of the parameter to determine the total number of occurrences of overflow or underflow of the parameter.

[0136] For each parameter, neural network device 100 may detect the most significant bit having a value of "1", and calculate the difference in the number of bits between the detected most significant bit and the most significant bit of the integer part of the fixed-point format for each parameter as a bit shift value. In addition, for each parameter, neural network device 100 may search for a bit in a predetermined range based on the most significant bit of the integer part of the fixed-point format, and detect the most significant bit having a value of "1".

[0137] In operation s1320, the neural network device 100 may update the fixed-point format using the bit shift value of the parameter. The neural network device 100 may update the existing fixed-point format used to quantize the parameter in the t-th learning or inference process to a new fixed-point format. For example, when the number of occurrences of overflow is greater than a predetermined value, the neural network device 100 may update the fixed-point format by reducing the fractional length of the fixed-point format by the maximum bit shift value of the bit shift value.

[0138] In operation s1330, the neural network device 100 may quantize the parameters updated in the learning or inference process according to the updated fixed-point format. Specifically, the neural network device 100 may quantize the parameters updated in the t+1th learning or inference process according to the fixed-point format updated by the bit shift value of the parameter updated in the tth learning or inference process.

[0139] According to an embodiment, a neural network device may update a fixed-point format for quantizing a parameter updated in a learning or inference process according to a change in an updated parameter, thereby reducing the amount of calculation and reducing accuracy loss. In addition, when quantizing a parameter updated in a t-th learning or inference process, the neural network device may quantize the parameter according to a predetermined fixed-point format, thereby reducing the time for scanning all parameters updated in the t-th learning or inference process and the hardware overhead for determining the fixed-point format. In addition, since the neural network device updates the fixed-point format using the number of occurrences of overflow and the maximum bit shift value of the parameter, the neural network device may update the fixed-point format through relatively simple and small-scale calculations, thereby reducing hardware overhead.

[0140] The apparatus described herein may include: a processor, a memory (such as a permanent memory such as a disk drive) for storing program data to be executed by the processor, a communication port for handling communication with external devices, and a user interface device (including a display, buttons, etc.). When it comes to software modules, these software modules can be stored as program instructions or computer-readable codes that can be executed by the processor on a non-transitory computer-readable medium, such as a read-only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. The computer-readable recording medium can also be distributed on a networked computer system so that the computer-readable code is stored and executed in a distributed manner. The medium can be read by a computer, can be stored in a memory, and can be executed by a processor.

[0141] Embodiments can be described in the sense of functional block components and various processing steps. Such functional blocks can be implemented by any number of hardware and / or software components configured to perform specified functions. In an embodiment, the present disclosure can adopt various integrated circuit components that can perform various functions under the control of one or more microprocessors or other control devices, such as memory elements, processing elements, logic elements, lookup tables, etc. Similarly, in the case of using software programming or software elements to implement the elements of the present disclosure, any programming or scripting language (e.g., C, C++, Java, assembly language, etc.) and various algorithms can be used to implement the present disclosure, wherein any combination of data structures, objects, processes, routines or other programming elements is used to implement the various algorithms. Functional aspects can be implemented as algorithms executed on one or more processors. In addition, the present disclosure can adopt any number of technologies according to the relevant fields for electronic configuration, signal processing and / or control, data processing, etc. The words "mechanism" and "element" are used broadly and are not limited to mechanical or physical embodiments, and can include software routines combined with processors, etc.

[0142] The specific implementations shown and described herein are illustrative examples of the present disclosure and are not intended to further limit the scope of the present disclosure in any way. For the sake of brevity, the electronics, control systems, software development and other functional aspects of the system according to the relevant art (and the components of the various operating components of the system) may not be described in detail. In addition, the connecting lines or connectors shown in the various drawings presented are intended to represent the exemplary functional relationships and / or physical or logical couplings between the various elements. It should be noted that: many alternative or additional functional relationships, physical connections or logical connections may exist in actual devices.

[0143] The use of the terms "one" and "an" and "said" and similar indicators in the context of describing the present disclosure (especially in the context of the appended claims) should be interpreted as covering both the singular and the plural. In addition, unless otherwise indicated herein, the description of the range of values ​​herein is only intended to be used as a convenient method for referring to each individual value falling within the range, and each individual value is incorporated into this specification as if it is described separately in this article. Finally, the steps of all methods described herein can be performed in any suitable order, unless otherwise indicated herein or the context clearly indicates otherwise. The use of any and all examples or exemplary language (e.g., "such as (for example)") provided herein is only intended to better illustrate the present disclosure and not to impose restrictions on the scope of the present disclosure, unless otherwise required. Without departing from the spirit and scope of the present disclosure, multiple modifications and adaptations will be apparent to those of ordinary skill in the art.

[0144] It should be understood that the embodiments described herein should be considered in a descriptive sense only and not for purposes of limitation. Descriptions of features or aspects in each embodiment should generally be considered as available for other similar features or aspects in other embodiments.

[0145] Although one or more embodiments have been described with reference to the drawings, it will be understood by those skilled in the art that various changes in form and details may be made without departing from the spirit and scope of the invention as defined by the following claims.

[0146] Although the present disclosure includes specific examples, it will be apparent to those of ordinary skill in the art that various changes in form and detail may be made in these examples without departing from the spirit and scope of the claims and their equivalents. The examples described herein should be considered to be merely descriptive and not for purposes of limitation. The description of the features or aspects in each example should be considered to apply to similar features or aspects in other examples. Suitable results may be achieved if the described techniques are performed in a different order and / or if the components in the described systems, architectures, devices, or circuits are combined in different ways and / or replaced or supplemented by other components or their equivalents. Therefore, the scope of the present disclosure is not limited by specific embodiments, but by the claims and their equivalents, and all changes within the scope of the claims and their equivalents should be interpreted as being included in the present disclosure.

Claims

1. A processor-implemented method for quantizing parameters of a neural network, the neural network being used to perform speech recognition and / or image recognition, the method comprising: For each parameter, calculating a bit shift value indicating a degree outside a bit range of a fixed point format used to quantize the parameter; updating the fixed-point format based on the calculated bit shift value of the parameter; as well as quantizes the parameters updated during learning or inference according to the updated fixed-point format, Wherein, calculating the bit shift value comprises: For each parameter, detecting the most significant bit having a value of "1"; and For each parameter, the difference in the number of bits between the detected most significant bit and the most significant bit of the integer part of the fixed-point format is determined as the bit shift value.

2. The method according to claim 1, wherein: Detecting the most significant bit comprises: For each parameter, bits within a predetermined range are searched based on the most significant bit of the integer part of the fixed-point format, and the most significant bit having a value of "1" is detected.

3. The method according to claim 1, wherein: Updating the fixed-point format includes: determining the number of overflow occurrences and the maximum bit shift value according to the calculated bit shift value; and The fixed-point format is updated based on the number of occurrences of the overflow and the maximum bit shift value.

4. The method according to claim 3, wherein: Updating the fixed-point format includes: In a case where the number of occurrences of overflow is greater than a predetermined value, the fixed-point format is updated by reducing the fractional length of the fixed-point format by the maximum bit shift value.

5. The method according to claim 4, wherein: The predetermined value is based on the number of parameters.

6. The method according to claim 1, in, The updated parameters are the parameters updated during the t+1th learning or inference process. Wherein, the parameters are the parameters updated during the t-th learning or inference process, Wherein, the fixed-point format is a fixed-point format updated based on the parameters updated during the t-1th learning or inference process, and t is a natural number greater than or equal to 2.

7. The method according to claim 1, wherein: Calculating the bit shift value includes: A bit shift value for each parameter is calculated during quantization of the parameter according to the fixed point format.

8. The method according to claim 1, wherein: The parameters are the weights or activations at the same layer in the neural network.

9. A computer-readable recording medium storing a program for causing a computer to execute the method according to claim 1.

10. A device for quantizing parameters of a neural network, the neural network being used to perform speech recognition and / or image recognition, the device comprising: A memory storing at least one program; as well as A processor configured to: For each parameter, calculating a bit shift value indicating a degree outside a bit range of a fixed point format used to quantize the parameter; updating the fixed-point format using the calculated bit shift value of the parameter; and quantizes the parameters updated during learning or inference according to the updated fixed-point format, Wherein, the processor is configured to: For each parameter, detecting the most significant bit having a value of "1"; and For each parameter, the difference in the number of bits between the detected most significant bit and the most significant bit of the integer part of the fixed-point format is determined as the bit shift value.

11. The device according to claim 10, wherein: The processor is configured to search for bits within a predetermined range based on a most significant bit of an integer part of the fixed-point format, and detect the most significant bit having a value of "1".

12. The device according to claim 10, wherein: The processor is configured to: determine a number of occurrences of overflow and a maximum bit shift value according to the calculated bit shift value; and update the fixed-point format using the number of occurrences of overflow and the maximum bit shift value.

13. The device according to claim 12, wherein: The processor is configured to update the fixed-point format by reducing the fractional length of the fixed-point format by the maximum bit shift value if the number of occurrences of overflow is greater than a predetermined value.

14. The device according to claim 13, wherein: The predetermined value is based on the number of parameters.

15. The device according to claim 10, in, The updated parameters are the parameters updated during the t+1th learning or inference process. Wherein, the parameters are the parameters updated during the t-th learning or inference process, Wherein, the fixed-point format is a fixed-point format updated based on the parameters updated during the t-1th learning or inference process, and t is a natural number greater than or equal to 2.

16. The device according to claim 10, wherein: The processor is configured to calculate the bit shift value for each parameter during quantization of the parameters according to the fixed point format.

17. The device according to claim 10, wherein: The parameters are the weights or activations at the same layer in the neural network.

Images