Computing Device Performing Quantization of a Neural Network and Operating Method of the Computing Device

Abstract

A computing device and a method of operating the computing device are disclosed. The method of operating the computing device may include the step of dividing a layer of a neural network into chunks; the step of determining a scale factor (SF) for each layer divided into chunks; and the step of quantizing the number of weights or activation functions of the layers divided into chunks based on the scale factor determined for each chunk.

Description

Technology Field

[0001] The present invention relates to the quantization of a neural network, and more specifically, to an apparatus and method for quantizing by dividing the layers of a neural network into chunks and determining a scale factor (SF) for each divided layer. Background Technology

[0002] Modern deep learning architectures can achieve remarkable accuracy in many real-world applications at the expense of massive memory bandwidth and computational costs. Recently, there has been a surge in extending deep learning networks to Internet-of-Things (IoT) devices equipped with limited resources, such as sensors and batteries.

[0003] IoT devices have supported various IoT applications by processing sensor data with cloud analysis tools. However, this approach can place a burden on network bandwidth. Furthermore, low-power IoT devices face difficulties in maintaining connections with cloud resources, which can also lead to latency issues in IoT applications.

[0004] To address these issues and deploy deep learning networks on IoT devices, there are several network compression techniques that can reduce the size of neural networks. Among them, quantization of deep learning networks has been proven to be an efficient technique for reducing network memory bandwidth. The problem to be solved

[0005] The present invention aims to provide a quantized network with accuracy corresponding to a full-precision network by dividing the layers of a deep learning network, that is, a neural network, into chunks and determining a scale factor (SF) for each divided layer to perform quantization.

[0006] However, technical challenges are not limited to the technical challenges described above, and other technical challenges may exist. means of solving the problem

[0007] A method of operation of a computing device according to an embodiment of the present invention may include the step of dividing a layer of a neural network into chunks; the step of determining a scale factor (SF) for each layer divided into chunks; and the step of quantizing a weight or activation function of the layer divided into chunks based on the scale factor determined for each chunk.

[0008] The above-mentioned splitting step may divide the layer of the neural network into at least two or more chunks of the same size.

[0009] The above-mentioned splitting step may divide the layer of the neural network into at least two or more chunks of different sizes based on the distribution structure of the weights or activation functions.

[0010] The above-mentioned splitting step can divide the chunk size into smaller chunks as the density of the weight or activation function in the layer of the neural network increases, and divide the chunk size into larger chunks as the density of the weight or activation function decreases.

[0011] The above determining step can determine a scale factor for each layer divided into chunks based on the real value range of the weight or activation function included in the layer divided into chunks and the bit size for quantization.

[0012] A computing device according to one embodiment of the present invention comprises one or more processors; and a memory for loading or storing a program executed by said processors, wherein the program may include instructions for performing operations such as dividing a layer of a neural network into chunks, determining a scale factor (SF) for each layer divided into chunks, and quantizing the number of weights or activation functions of the layers divided into chunks based on the scale factor determined for each chunk.

[0013] The processor can divide the layer of the neural network into at least two or more chunks of the same size.

[0014] The processor can divide the layer of the neural network into at least two or more chunks of different sizes based on the distribution structure of the weight or activation function.

[0015] The processor can divide the chunk size into smaller chunks as the density of the weight or activation function in the layer of the neural network increases, and divide the chunk size into larger chunks as the density of the weight or activation function decreases.

[0016] The processor can determine a scale factor for each layer divided into chunks based on the real value range of weights or activation functions included in the layer divided into chunks and the bit size for quantization. Effects of the invention

[0017] The present invention can provide a quantization method having accuracy corresponding to a full-precision network by dividing the layers of a neural network into chunks and determining a scale factor (SF) for each divided layer to perform quantization. Brief explanation of the drawing

[0018] FIG. 1 is a diagram showing a conceptual diagram of a quantization method according to an embodiment of the present invention. FIG. 2 is a diagram illustrating the configuration of a computing device that performs a novel quantization method according to an embodiment of the present invention. FIG. 3 is a flowchart illustrating the operation method of a computing device according to an embodiment of the present invention. FIG. 4 is a diagram illustrating a method for dividing a layer of a neural network according to an embodiment of the present invention. FIG. 5 is a diagram showing a comparison between a conventional quantization method based on layer-by-layer granularity according to an embodiment of the present invention and a novel quantization method proposed in the present invention. Specific details for implementing the invention

[0019] Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, actual implementations are not limited to the specific embodiments disclosed, and the scope of this specification includes modifications, equivalents, or substitutions included in the technical concept described by the embodiments.

[0020] Terms such as "first" or "second" may be used to describe various components, but these terms should be interpreted solely for the purpose of distinguishing one component from another. For example, the first component may be named the second component, and similarly, the second component may be named the first component.

[0021] When it is stated that a component is "connected" to another component, it should be understood that it may be directly connected to or joined to that other component, or that there may be other components in between.

[0022] Singular expressions include plural expressions unless the context clearly indicates otherwise. In this document, phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B or C,” “at least one of A, B and C,” and “at least one of A, B, or C” may each include any one of the items listed together with the corresponding phrase, or all possible combinations thereof. In this specification, terms such as “comprising” or “having” are intended to designate the existence of the described feature, number, step, action, component, part, or combination thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0023] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.

[0024] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are given the same reference numeral regardless of the drawing number, and redundant descriptions thereof will be omitted.

[0026] FIG. 1 is a diagram showing a conceptual diagram of a quantization method according to an embodiment of the present invention.

[0027] Existing quantization methods can generally be classified into per-layer granularity and per-channel granularity. Per-layer granularity allows for quantization to be performed by identifying the quantization range for each layer constituting the neural network. In this case, per-layer granularity can quantize the weights or activation functions of multiple channels contained within a layer using a scale factor of one per layer. Consequently, due to the limited representation range, the accuracy of the neural network to which this quantization method is applied may be reduced.

[0028] In contrast, channel-specific granularity can perform quantization by identifying the quantization range for each channel of the layers constituting the neural network. In this case, channel-specific granularity can perform quantization using a single scale factor per channel. That is, compared to layer-specific granularity, channel-specific granularity can improve the accuracy of the neural network by quantizing the weights or activation functions of multiple channels contained within a layer using scale factors corresponding to the number of channels for each layer. However, channel-specific granularity suffers from increased computational cost because it uses multiple scale factors for each layer.

[0029] The present invention can provide a novel quantization method that can improve the accuracy of a neural network while paying low computational costs by compensating for the disadvantages of such layer-by-layer granularity and channel-by-channel granularity.

[0030] The novel quantization method of the present invention can provide a method for quantizing a layer of a neural network by dividing it into chunk units, as shown by reference numeral 100 in FIG. 1. More specifically, the novel quantization method of the present invention can provide a method for improving the accuracy of a neural network by determining a scale factor for each layer divided into chunk units and performing quantization using the scale factor determined for each layer divided into chunk units. Through this, the novel quantization method of the present invention can provide accuracy corresponding to a full-precision network despite a low quantization bit size.

[0031] The new quantization method will be explained in more detail through the following diagrams.

[0033] FIG. 2 is a diagram illustrating the configuration of a computing device that performs a novel quantization method according to an embodiment of the present invention.

[0034] As illustrated in FIG. 2, the computing device (200) may include one or more processors (210) and a memory (220) for loading or storing a program (230) executed by the processors (210). The components included in the computing device (200) of FIG. 2 are merely examples, and a person skilled in the art to which the present invention pertains will know that other general-purpose components may be included in addition to the components shown in FIG. 2.

[0035] The processor (210) controls the overall operation of each component of the computing device (200). The processor (210) may be configured to include at least one of a CPU (Central Processing Unit), MPU (Micro Processor Unit), MCU (Micro Controller Unit), GPU (Graphic Processing Unit), NPU (Neural Processing Unit), DSP (Digital Signal Processor), or any other type of processor well known in the art of the present invention. Additionally, the processor (210) may perform operations for at least one application or program for executing a method / operation according to various embodiments of the present invention. The computing device (200) may have one or more processors.

[0036] Memory (220) stores one or more combinations of various data, instructions, and information used by a component (e.g., processor (210)) included in the computing device (200). Memory (220) may include volatile memory and / or non-volatile memory.

[0037] The program (230) may include one or more actions in which methods / actions according to various embodiments of the present invention are implemented, and may be stored in memory (220) in the form of software. Here, the action corresponds to instructions implemented in the program (230). For example, the program (230) may include instructions to perform an action of dividing a layer of a neural network into chunks, an action of determining a scale factor for each layer divided into chunks, and an action of quantizing the weights of the layers divided into chunks based on the scale factor determined for each chunk.

[0038] When the program (230) is loaded into memory (220), the processor (210) can perform methods / operations according to various embodiments of the present invention by executing a plurality of operations to implement the program (230).

[0039] The execution screen of the program (230) can be displayed through the display (240). In the case of FIG. 2, the display (240) is depicted as a separate device connected to the computing device (200), but in the case of a computing device (200) such as a terminal that a user can carry, such as a smartphone or tablet, the display (240) can be a component of the computing device (200). The screen displayed on the display (240) may be the result of the execution of the program or before information is input into the program.

[0041] FIG. 3 is a flowchart illustrating the operation method of a computing device according to an embodiment of the present invention.

[0042] The operation method illustrated in FIG. 3 can be performed by a processor (210) of a computing device (200). First, in step (S210), the processor (210) divides the layers of the neural network into chunks, in step (S220), determines a scale factor for each layer divided into chunks, and in step (S230), can quantize the weights of the layers divided into chunks based on the scale factor determined for each chunk.

[0043] More specifically, the processor (210) can divide a layer of the neural network into at least two or more chunks of the same size. For example, FIG. 4(a) shows an example in which a layer of the neural network is divided into four chunks of the same size. The processor (210) can determine a scale factor for each layer divided into four chunks of the same size in this way.

[0044] General quantization can be defined as shown in Equation 1 below.

[0045] <Equation 1>

[0046]

[0047] Here, represents the real value of the weight or activation function to be quantized, and represents the scale factor, and represents a quantized integer value of a real-valued weight or activation function, and represents the zero point value.

[0048] The quantization according to Equation 1 above is known as symmetric quantization. It can be simplified by arranging it, and through this, the computational cost required to perform quantization as in Equation 2 can be reduced.

[0049] <Equation 2>

[0050]

[0051] Here, scale factor is a factor used to scale the real values ​​of a weight or activation function into quantized values, and can be defined as shown in Equation 3 below.

[0052] <Equation 3>

[0053]

[0054] Here, represents the range of real values ​​of the weights or activation functions to be quantized, and represents the bit size for quantization. In this case, the bit size Since is determined before the quantization process starts, the processor (210) is a range of real values ​​of the weight or activation function to be quantized. The scale factor can be determined by optimizing it.

[0055] Accordingly, the processor (210) has the highest value for the real value of the weight or activation function corresponding to the first chunk (ⓐ). ) and minimum value( )'s car By setting it to and substituting it into Equation 3, the scale factor for the first chunk (ⓐ) can decide.

[0056] Additionally, the processor (210) has the highest value for the real value of the weight or activation function corresponding to the second chunk (ⓑ). ) and minimum value( )'s car By setting it to and substituting it into Equation 3, the scale factor for the second chunk (ⓑ) can decide.

[0057] Additionally, the processor (210) has the highest value for the real value of the weight or activation function corresponding to the third chunk (ⓒ). ) and minimum value( )'s car By setting it to and substituting it into Equation 3, the scale factor for the third chunk (ⓒ) can decide.

[0058] Finally, the processor (210) determines the maximum value for the real value of the weight or activation function corresponding to the fourth chunk (ⓓ). ) and minimum value( )'s car By setting it to and substituting it into Equation 3, the scale factor for the fourth chunk (ⓓ) can decide.

[0059] In this way, the processor (210) determines a scale factor for each layer divided into chunks and quantizes the weights or activation functions of the layers divided into chunks based on the scale factor of the corresponding chunk, thereby providing accuracy corresponding to a full-precision network despite the low quantization bit size.

[0060] Meanwhile, the layer statistics of a pre-trained neural network, namely the weights or activation functions, tend to follow a non-uniform distribution. For example, the weights may follow a bell-shaped distribution as shown in Figures 4 (a) and (b). In this case, when the layers of the neural network are divided into chunks of the same size as in Figure 4 (a), there is a concern that the quantization performance of the weights or activation functions of the second chunk (b) and the third chunk (c), which contain important information, may be degraded compared to the first chunk (a) and the fourth chunk (c).

[0061] Accordingly, the processor (210) can divide the layer of the neural network into at least two or more chunks of different sizes based on the distribution structure of the weights or activation functions. More specifically, the processor (210) can divide the chunks into smaller sizes as the density of the weights or activation functions in the layer of the neural network increases, and divide the chunks into larger sizes as the density of the weights or activation functions decreases.

[0062] For example, FIG. 4(b) shows an example in which a layer of a neural network is divided into four chunks of different sizes. The processor (210) can perform quantization by determining a scale factor for each layer divided into four chunks of different sizes in this way. At this time, the second chunk (ⓑ) and the third chunk (ⓒ) have a narrower range than the first chunk (ⓐ) and the fourth chunk (ⓓ). Since the scale factor is calculated through ), quantization accuracy can be improved.

[0064] FIG. 5 is a diagram showing a comparison between a conventional quantization method based on layer-by-layer granularity according to an embodiment of the present invention and a novel quantization method proposed in the present invention.

[0065] Figure 5(a) illustrates a quantization method based on the granularity of each layer. Looking at Figure 5(a), a scale factor can be applied to an entire layer, and the interval of the quantized weights or activation functions can be determined based on the bit size for quantization according to Equation 3 above. That is, when the bit sizes are 3, 4, and 5, respectively, the weights or activation functions of multiple channels included within the same layer are quantized into 7, 15, and 31 numbers, respectively, so the interval of the quantized weights or activation functions may vary.

[0066] In contrast, FIG. 5(b) relates to a novel quantization method proposed in the present invention, wherein a single layer is divided into four chunks of equal size, and a scale factor can be applied to each layer divided into four chunks. Accordingly, when the bit sizes are 3, 4, and 5 respectively according to Equation 3 above, the weights or activation functions of multiple channels included in each chunk are quantized into 7, 15, and 31 numbers, respectively, and thus the weights or activation functions of multiple channels included in the entire layer are quantized into a total of 28, 60, and 124 numbers, thereby improving the quantization accuracy.

[0068] The embodiments described above may be implemented as hardware components, software components, and / or combinations of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0069] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or instruct the processing unit independently or collectively. Software and / or data may be stored on any type of machine, component, physical device, virtual equipment, computer storage medium, or device so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on computer-readable recording media.

[0070] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may store program instructions, data files, data structures, etc., either individually or in combination, and the program instructions recorded on the medium may be those specifically designed and configured for the embodiment or those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0071] The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0072] Although the embodiments have been described above with reference to the limited drawings, those skilled in the art can apply various technical modifications and variations based thereon. For example, suitable results may be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0073] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below. Explanation of the symbols

[0074] 200: Computing device 210 : Processor 220 : Memory 230 : Program 240:Display

Claims

Claim 1 A method of operation of a computing device comprising: a step of dividing a real value range of weight values ​​included in a layer of a neural network or a real value range of output values ​​of an activation function into at least two chunks having different sizes based on a distribution structure of said weight values ​​or said activation function output values; a step of determining a scale factor (SF) corresponding to each chunk based on said weight values ​​or said activation function output values ​​included in each of said chunks and a bit size for quantization; and a step of quantizing said weight values ​​or said activation function output values ​​included in each chunk based on said scale factor. Claim 2 delete Claim 3 delete Claim 4 A method of operation according to claim 1, wherein the splitting step divides the chunk size into smaller chunks as the density of the weight value or the output value of the activation function in the layer of the neural network increases, and divides the chunk size into larger chunks as the density of the weight value or the output value of the activation function decreases. Claim 5 delete Claim 6 A computer program stored on a computer-readable recording medium in combination with hardware to execute the method of either claim 1 or claim 4. Claim 7 A computing device comprising: at least one processor; and a memory including one or more storage media for storing instructions, wherein when the instructions are executed individually or collectively by the at least one processor, the computing device causes the computing device to divide a real value range of weight values ​​included in a layer of a neural network or a real value range of output values ​​of an activation function into at least two chunks having different sizes based on a distribution structure of said weight values ​​or said activation function output values, determine a scale factor (SF) corresponding to each chunk based on said weight values ​​or said activation function output values ​​included in each chunk and a bit size for quantization, and quantize said weight values ​​or said activation function output values ​​included in each chunk based on the scale factor corresponding to each chunk. Claim 8 delete Claim 9 delete Claim 10 In claim 7, the processor divides the chunk size into smaller chunks as the density of the weight value or the output value of the activation function in the layer of the neural network increases, and divides the chunk size into larger chunks as the density of the weight value or the output value of the activation function decreases. Claim 11 delete

Images