Method for quantizing neural network model and device for performing same

Abstract

An electronic device, according to one embodiment, may operate in the order of: acquiring an input; quantizing at least one parameter included in a first layer among a plurality of layers included in a neural network model; measuring a first reconstruction error for the input, in a state where at least one parameter included in the remaining layers other than the first layer among the plurality of layers is fixed; and on the basis of the first reconstruction error, adjusting the quantized at least one parameter included in the first layer.

Description

Method for quantizing a neural network model and a device for performing the same

[0001] The present disclosure relates to a method for quantizing a neural network model and a device for performing the same, and more particularly, to a method for accelerating quantization of a neural network model and reducing errors resulting from quantization and a device for performing the same.

[0002] When running an unquantized artificial intelligence model for at least one parameter included in a neural network on an electronic device with hardware constraints in terms of memory and computing resources, problems such as latency in a series of processes running on the electronic device may occur.

[0003] Therefore, in order to efficiently operate an artificial intelligence model in an electronic device with hardware constraints, it is necessary to quantize at least one parameter included in the neural network included in the artificial intelligence model.

[0004] A method according to one embodiment may include obtaining an input.

[0005] A method according to one embodiment may include a step of quantizing at least one parameter included in a first layer among a plurality of layers included in the neural network model, and measuring a first reconstruction error for the input while fixing at least one parameter included in the remaining layers excluding the first layer among the plurality of layers.

[0006] A method according to one embodiment may include adjusting at least one quantized parameter included in the first layer based on the first reconstruction error.

[0007] An electronic device according to one embodiment may include a memory storing a program or at least one instruction and at least one processor executing at least one instruction stored in the memory.

[0008] The electronic device can obtain input by having at least one processor execute at least one instruction stored in the memory according to one embodiment.

[0009] According to one embodiment, at least one processor executes at least one command stored in the memory, thereby allowing the electronic device to quantize at least one parameter included in a first layer among a plurality of layers included in the neural network model, and measure a first reconstruction error for the input while fixing at least one parameter included in the remaining layers excluding the first layer among the plurality of layers.

[0010] According to one embodiment, at least one processor executes at least one instruction stored in the memory, thereby allowing the electronic device to adjust at least one quantized parameter included in the first layer based on the first reconstruction error.

[0011] Figure 1 is a diagram illustrating an example of an artificial intelligence model to which quantization of a neural network model is applied.

[0012] FIG. 2 is a diagram illustrating an example of an artificial intelligence model to which an operating method of an electronic device according to one embodiment can be applied.

[0013] FIG. 3 is a flowchart illustrating an operating method of an electronic device according to one embodiment.

[0014] FIG. 4 is a diagram illustrating a method for measuring a first reconstruction error by an electronic device according to one embodiment.

[0015] FIG. 5 is a flowchart illustrating an operating method of an electronic device according to one embodiment.

[0016] FIG. 6 is a flowchart illustrating an operation method of an electronic device according to one embodiment.

[0017] Figure 7 is a flowchart for explaining an operating method of an electronic device according to one embodiment.

[0018] Figure 8 is a flowchart for explaining an operating method of an electronic device according to one embodiment.

[0019] FIG. 9 is a flowchart illustrating an operation method of an electronic device according to one embodiment.

[0020] FIG. 10 is a diagram illustrating a method for measuring a second reconstruction error by an electronic device according to one embodiment.

[0021] Fig. 11 is a flowchart for explaining an operating method of an electronic device according to one embodiment.

[0022] FIG. 12 is a diagram illustrating a method for measuring a third reconstruction error by an electronic device according to one embodiment.

[0023] FIG. 13 is a diagram illustrating a method for an electronic device according to one embodiment to quantize a neural network model based on a first reconstruction error, a second reconstruction error, and a third reconstruction error.

[0024] FIG. 14 is a block diagram illustrating components of an electronic device according to one embodiment.

[0025] The terms used in this specification will be briefly explained, and the present invention will be described in detail.

[0026] The terms used in this invention have been selected from widely used, current terms, taking into account the functions of the invention. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, in which case their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this invention should not be defined simply as names, but rather based on their inherent meanings and the overall content of the invention.

[0027] When a part of the specification is said to "include" a component, this does not exclude other components, but rather implies the inclusion of other components, unless otherwise specifically stated. Furthermore, terms such as "part," "module," etc., used throughout the specification refer to a unit that processes at least one function or operation, which may be implemented in hardware, software, or a combination of hardware and software.

[0028] Additionally, the description 'at least one of A, B, and C' means that it can be any one of 'A', 'B', 'C', 'A and B', 'A and C', 'B and C', and 'A, B, and C'.

[0029] It should be understood that the combinations of blocks and sequence diagrams in each flowchart can be performed by one or more computer programs containing computer-executable instructions. The one or more computer programs may be stored entirely in a single memory or may be divided and stored in multiple different memories.

[0030] All functions or operations described in this document may be performed by a single processor or a combination of processors. A single processor or a combination of processors is a circuitry that performs processing, and may include circuitry such as an Application Processor (AP), a Communication Processor (CP), a Graphical Processing Unit (GPU), a Neural Processing Unit (NPU), a Microprocessor Unit (MPU), a System on Chip (SoC), or an Integrated Chip (IC).

[0031] A processor may include various processing circuits and / or multiple processors. For example, the term “processor” as used herein, including in the claims, may include various processing circuits, including at least one processor. At least one processor, one or more processors, may be individually and / or collectively configured to perform the various functions described herein in a distributed fashion. As used herein, “processor,” “at least one processor,” and “one or more processors” may be configured to perform multiple functions. However, these terms encompass, without limitation, situations where one processor performs some of the functions and other processor(s) perform other parts of the functions, and situations where a single processor may perform all of the functions. Furthermore, the at least one processor may include a combination of processors that perform various of the disclosed functions in a distributed manner. At least one processor may execute program instructions to achieve or perform various functions.

[0032] Below, with reference to the attached drawings, embodiments of the present invention are described in detail so that those skilled in the art can easily implement the present invention. However, the present invention can be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description have been omitted to clearly explain the present invention, and similar parts have been designated with similar reference numerals throughout the specification.

[0033] Figure 1 is a diagram illustrating an example of an artificial intelligence model to which quantization of a neural network model is applied.

[0034] The artificial intelligence model (100) may include at least one layer (120, 130, 140).

[0035] Here, at least one layer (120, 130, 140) may mean a neural network.

[0036] Neural networks can be trained by receiving training data. The trained neural network then receives input data as input, and the output can analyze the input data and perform operations to produce the desired output data. Operations performed by neural networks can be performed through hidden layers. These hidden layers can be composed of multiple layers.

[0037] The artificial intelligence model (100) can obtain an output (180) through a series of processes (150, 160, 170) in which learning is completed by applying the acquired input (110) to at least one layer (120, 130, 140).

[0038] Meanwhile, when operating an artificial intelligence model in which quantization has not been performed on at least one parameter included in a neural network on an electronic device with hardware limitations in terms of memory and computing resources, problems such as latency occurring in a series of processes operated on the electronic device may occur.

[0039] Therefore, in order to efficiently operate an artificial intelligence model in an electronic device with hardware constraints, it is necessary to quantize at least one parameter included in the neural network included in the artificial intelligence model.

[0040] According to one embodiment of the present disclosure, a neural network can be quantized by expressing at least one parameter expressed in floating point as a fixed point. However, this is not limited to this, and parameters included in the neural network can be quantized in various ways.

[0041] However, if at least one parameter included in the neural network included in the artificial intelligence model is quantized through repetitive learning, quantization may take an excessively long time, and the accuracy of the quantized artificial intelligence model may be lost.

[0042] Hereinafter, the present disclosure will describe a method and device for accelerating quantization of a neural network model and reducing errors due to quantization by performing quantization and freezing on at least one layer included in an artificial intelligence model by an electronic device according to one embodiment.

[0043] FIG. 2 is a diagram illustrating an example of an artificial intelligence model to which an operating method of an electronic device according to one embodiment can be applied.

[0044] The method of operating an electronic device according to one embodiment may be applicable to various types of neural network models including multiple layers, but in order to more clearly explain the invention, the following description will be based on an attention mechanism.

[0045] FIG. 2 is a drawing for explaining an attention mechanism corresponding to an example of an artificial intelligence model to which an operating method of an electronic device according to one embodiment can be applied.

[0046] An artificial intelligence model (200) according to one embodiment including an attention mechanism may include a first layer (220), a second layer (230), and a third layer (240).

[0047] Here, the first layer (220), the second layer (230), and the third layer (240) may refer to a neural network.

[0048] Neural networks can be trained by receiving training data. The trained neural network then receives input data as input, and the output can analyze the input data and perform operations to produce the desired output data. Operations performed by neural networks can be performed through hidden layers. These hidden layers can be composed of multiple layers.

[0049] An artificial intelligence model (200) according to one embodiment can obtain an input (210).

[0050] An artificial intelligence model (200) according to one embodiment can obtain a query matrix (251) by applying the acquired input (210) to the first layer (220).

[0051] An artificial intelligence model (200) according to one embodiment can obtain a key matrix (252) by applying the acquired input (210) to a second layer (230).

[0052] An artificial intelligence model (200) according to one embodiment can obtain a value matrix (253) by applying the acquired input (210) to the third layer (240).

[0053] An artificial intelligence model (200) according to one embodiment can obtain an attention score matrix (254) based on a query matrix (251) and a key matrix (252).

[0054] An artificial intelligence model (200) according to one embodiment can obtain an attention weight matrix (255) by performing scaling and normalization on an attention score matrix (254).

[0055] An artificial intelligence model (200) according to one embodiment can obtain an output (256, 260) by multiplying an attention weight matrix (255) by a value matrix (253).

[0056] FIG. 3 is a flowchart illustrating an operating method of an electronic device according to one embodiment.

[0057] At step 310, the electronic device can obtain input.

[0058] The input can be an input sequence in the form of a vector and / or matrix.

[0059] Here, the input sequence refers to the actual input applied to the model, and can refer to the entire input that the model must process. The input sequence can refer to the entire data passed to the model's input layer.

[0060] In step 320, the electronic device quantizes at least one parameter included in a first layer among a plurality of layers included in the neural network model, and measures a first reconstruction error for the input while fixing at least one parameter included in the remaining layers excluding the first layer among the plurality of layers.

[0061] The plurality of layers may include a first layer, a second layer, and a third layer.

[0062] In one embodiment of the present disclosure, quantization may mean expressing at least one parameter expressed in floating point as a fixed point. Here, expressing in fixed point includes assigning an integer value to at least one parameter expressed in floating point.

[0063] A first layer according to one embodiment may include at least one weight matrix used for operations on the acquired input.

[0064] An electronic device according to one embodiment may maintain the dimension of an input obtained based on at least one weight matrix included in a first layer, while changing the form of the input to perform a series of operations (see FIG. 2) included in an artificial intelligence model according to one embodiment.

[0065] A specific method of measuring a first reconstruction error for an input, in which an electronic device according to one embodiment quantizes at least one parameter included in a first layer among a plurality of layers included in a neural network model, and at least one parameter included in the remaining layers excluding the first layer among the plurality of layers is fixed, will be described with reference to FIG. 4.

[0066] FIG. 4 is a diagram illustrating a method for measuring a first reconstruction error by an electronic device according to one embodiment.

[0067] An electronic device according to one embodiment may obtain a query matrix by applying the acquired input to a first layer. Here, the first layer may include a weight matrix for generating the query matrix. The weight matrix for generating the query matrix may include at least one parameter. At least one parameter included in the first layer may be expressed in floating-point form.

[0068] An electronic device according to one embodiment may obtain a key matrix by applying the acquired input to a second layer. Here, the second layer may include a weight matrix for generating the key matrix. The weight matrix for generating the key matrix may include at least one parameter. At least one parameter included in the second layer may be expressed in floating point form.

[0069] An electronic device according to one embodiment may obtain a value matrix by applying the acquired input to a third layer. Here, the third layer may include a weight matrix that generates the value matrix. The weight matrix that generates the value matrix may include at least one parameter. At least one parameter included in the third layer may be expressed in floating-point form.

[0070] An electronic device according to one embodiment can quantize at least one parameter included in a first layer (410).

[0071] In one embodiment of the present disclosure, quantization may mean expressing at least one parameter expressed in floating point as a fixed point. Expressing in fixed point includes assigning an integer to at least one parameter expressed in floating point.

[0072] An electronic device according to one embodiment can fix the second layer and the third layer, excluding the first layer, among the first layer, the second layer, and the third layer (420, 430).

[0073] Here, the fixation of the second and third layers means that at least one parameter included in the second and third layers is expressed as a floating point. In other words, it can mean a state in which at least one parameter included in the second and third layers is maintained in floating point form.

[0074] Accordingly, an electronic device according to one embodiment can express at least one parameter included in a first layer as an integer based on quantization, and can express at least one parameter included in a second layer and a third layer excluding the first layer as a floating point.

[0075] An electronic device according to one embodiment can measure a first reconstruction error for an input by quantizing at least one parameter included in a first layer among a plurality of layers included in a neural network model, and fixing at least one parameter included in layers other than the first layer among the plurality of layers.

[0076] The first reconstruction error may refer to an error that occurs when the electronic device according to one embodiment quantizes at least one parameter included in the first layer. In other words, the first reconstruction error may refer to a difference between an output obtained by the electronic device using the first to third layers that are not quantized and an output obtained by quantizing only the parameters included in the first layer while keeping the second and third layers fixed.

[0077] For example, a first reconstruction error may mean an error between an output obtained by applying inputs to a first layer, a second layer, and a third layer including at least one parameter expressed in floating point, and an output obtained by applying inputs to a first layer including at least one quantized parameter and a second layer and a third layer including at least one parameter expressed in floating point, according to an embodiment of the present invention.

[0078] To explain in a more general sense, the reconstruction error according to one embodiment of the present disclosure may mean the difference between the output (original output) obtained from the neural network model in its original state that has not been quantized, and the output (quantized output) obtained by quantizing only one layer among a plurality of layers included in the neural network model and keeping the remaining layers fixed.

[0079] Referring again to Figure 3,

[0080] At step 330, the electronic device can adjust at least one quantized parameter included in the first layer based on the first reconstruction error.

[0081] FIG. 5 is a flowchart illustrating an operating method of an electronic device according to one embodiment.

[0082] Steps 310, 320 and 330 of FIG. 5 correspond to steps 310, 320 and 330 of FIG. 3, respectively.

[0083] In step 321, the electronic device can obtain a first output by applying an input to a neural network model before quantizing at least one parameter included in the first layer.

[0084] The first output means an output obtained by applying an input to the first layer, the second layer and the third layer, wherein the electronic device according to one embodiment includes at least one parameter expressed in floating point.

[0085] First output ( ) can be expressed by the following [Mathematical Formula 1].

[0086] [Mathematical Formula 1]

[0087]

[0088] Q stands for query matrix.

[0089] K stands for key matrix.

[0090] V stands for value matrix.

[0091] A represents the attention weight matrix.

[0092] d represents the dimension of the query matrix and key matrix.

[0093] In step 323, the electronic device can obtain a second output by applying an input to a neural network model after quantizing at least one parameter included in the first layer.

[0094] The second output means an output obtained by applying input to the first layer including at least one quantized parameter and the second and third layers including at least one parameter expressed in floating point.

[0095] Second output ( ) can be expressed by the following [Mathematical Formula 2].

[0096] [Equation 2]

[0097]

[0098] stands for quantized query matrix.

[0099] K stands for key matrix.

[0100] V stands for value matrix.

[0101] d represents the dimension of the query matrix and key matrix.

[0102] refers to an attention weight matrix based on a quantized query matrix.

[0103] At step 325, the electronic device can measure a first reconstruction error based on the first output and the second output.

[0104] An electronic device according to one embodiment may measure a first reconstruction error based on comparing a first output with a second output. Here, the first reconstruction error may refer to an error that occurs when the electronic device according to one embodiment quantizes at least one parameter included in a first layer.

[0105] An electronic device according to one embodiment may measure a first reconstruction error in the form of a mean square error.

[0106] First reconstruction error ( ) can be expressed by the following [Mathematical Formula 3].

[0107] [Equation 3]

[0108]

[0109] Components included in [Mathematical Formula 3] but not separately defined correspond to the components defined in [Mathematical Formula 1] and [Mathematical Formula 2], respectively.

[0110] An electronic device according to one embodiment includes [Mathematical Formula 3] The term can be measured as the first reconstruction error.

[0111] An electronic device according to one embodiment can adjust at least one quantized parameter included in a first layer such that a measured first reconstruction error is minimized.

[0112] FIG. 6 is a flowchart illustrating an operation method of an electronic device according to one embodiment.

[0113] Steps 310, 320 and 330 of FIG. 6 correspond to steps 310, 320 and 330 of FIG. 3, respectively.

[0114] At step 322, the electronic device can obtain an objective function for measuring the first reconstruction error.

[0115] Hereinafter, a method for deriving an objective function from [Mathematical Formula 3] measured by a first reconstruction error according to an electronic device according to one embodiment will be described.

[0116] Here, the objective function means a function and / or term to be optimized to minimize an error that occurs when an electronic device according to one embodiment quantizes at least one parameter included in a first layer.

[0117] Included in [Mathematical Formula 3] The components can be expressed by the following [Mathematical Formula 4].

[0118] [Equation 4]

[0119]

[0120] Components included in [Mathematical Formula 4] but not separately defined correspond to the components defined in [Mathematical Formula 1] to [Mathematical Formula 3], respectively.

[0121] By applying Taylor expansion, [Equation 3] can be expressed as [Equation 5] below.

[0122] [Equation 5]

[0123]

[0124] stands for the Jacobian matrix for the softmax function.

[0125] X represents a matrix for the input sequence.

[0126] refers to the weight matrix that generates the query matrix.

[0127] Components included in [Mathematical Formula 5] but not separately defined correspond to the components defined in [Mathematical Formula 1] to [Mathematical Formula 4].

[0128] If an upper bound is derived for the terms included in [Mathematical Formula 5], [Mathematical Formula 5] can be expressed as [Mathematical Formula 6] below.

[0129] [Equation 6]

[0130]

[0131] An electronic device according to one embodiment is affected by the quantization included in [Mathematical Formula 6]. The term can be obtained as an objective function.

[0132] Components included in [Mathematical Formula 6] but not separately defined correspond to the components defined in [Mathematical Formula 1] to [Mathematical Formula 5], respectively.

[0133] At step 324, the electronic device can transform the objective function such that at least one component included in the objective function is separated from the quantization-independent component.

[0134] Hereinafter, a method for inducing a transformation of an objective function such that a component independent of quantization among at least one component included in the objective function is separated will be described according to an embodiment of the present invention.

[0135] Objective function obtained from [Equation 6] If the Kronecker product is applied to the term, [Equation 6] can be expressed as [Equation 7] below.

[0136] [Equation 7]

[0137]

[0138] By applying the Kronecker product to [Equation 6], the equation included in [Equation 6] is It refers to an item whose components have been transformed.

[0139] Components included in [Mathematical Formula 7] but not separately defined correspond to the components defined in [Mathematical Formula 1] to [Mathematical Formula 6].

[0140] In [Equation 7] Port and A term corresponds to a component that is independent of the quantization of at least one parameter included in the first layer.

[0141] An electronic device according to one embodiment is Port and The term can be judged as a component independent of the quantization of at least one parameter included in the first layer.

[0142] Here, an independent component may mean a component that is not affected by the quantization of at least one parameter included in the first layer.

[0143] Accordingly, an electronic device according to one embodiment can transform an objective function so that a component independent of quantization among at least one component included in the objective function is separated based on [Mathematical Equation 6] and [Mathematical Equation 7].

[0144] In step 326, the electronic device can measure a first reconstruction error from the transformed objective function by applying a value calculated according to the training data of the neural network model to the quantization-independent component.

[0145] An electronic device according to one embodiment comprises a component corresponding to the quantization of at least one parameter included in a first layer obtained from an objective function and independent of the component. Port and By applying the value calculated according to the learning data of the neural network model to the term, the first reconstruction error can be measured from the transformed objective function.

[0146] Here, the learning data of the neural network model may mean data used for learning an artificial intelligence model and / or a neural network model operated in an electronic device according to one embodiment.

[0147] Figure 7 is a flowchart for explaining an operating method of an electronic device according to one embodiment.

[0148] Steps 320 and 330 of FIG. 7 correspond to steps 320 and 330 of FIG. 3, respectively.

[0149] At step 332, the electronic device can determine at least one quantization parameter including a scale factor and a zero point.

[0150] A quantization parameter may mean a variable related to an interval for quantizing at least one parameter included in the first layer.

[0151] The scale factor may mean the ratio between the range of continuous real values ​​of at least one parameter to be quantized and the range of integer values ​​to be quantized.

[0152] For example, if the continuous real value range of at least one parameter to be quantized is [-1.0, 1.0] and the range of integer values ​​to be quantized is [0, 255], the scale factor may be determined as 2.0 / 255. However, the present invention is not limited thereto.

[0153] A zero point may mean a parameter that includes reference point information about which value of 0 in at least one quantized parameter corresponds to which value among the range of real values ​​of at least one parameter before quantization.

[0154] An electronic device according to one embodiment can determine a quantization parameter used in adjusting at least one parameter included in a first layer based on a quantization parameter used in quantizing at least one parameter included in a first layer.

[0155] An electronic device according to one embodiment can group at least one parameter included in a first layer into at least one group.

[0156] An electronic device according to one embodiment can determine a quantization parameter including a scale exponent and a zero point based on a range of at least one parameter included in at least one group.

[0157] An electronic device according to one embodiment can determine quantization parameters, including scale indices and zero points, used for each group based on a range of at least one parameter included in at least one group.

[0158] In step 334, the electronic device can adjust at least one parameter included in the first layer based on at least one quantization parameter.

[0159] Figure 8 is a flowchart for explaining an operating method of an electronic device according to one embodiment.

[0160] Steps 332 and 334 of FIG. 8 correspond to steps 332 and 334 of FIG. 7, respectively.

[0161] At step 336, the electronic device can learn at least one quantization parameter such that the first reconstruction error is reduced.

[0162] An electronic device according to one embodiment may learn at least one quantization parameter such that a first reconstruction error is reduced based on a grid search technique, convex optimization, or backpropagation.

[0163] In step 338, the electronic device can adjust at least one quantized parameter included in the first layer based on at least one quantized parameter for which learning has been completed.

[0164] FIG. 9 is a flowchart illustrating an operation method of an electronic device according to one embodiment.

[0165] Step 330 of FIG. 9 corresponds to step 330 of FIG. 3.

[0166] In step 910, the electronic device can measure a second reconstruction error for the input by quantizing at least one parameter included in a second layer among a plurality of layers included in the neural network model, and keeping at least one parameter included in the remaining layers except for the second layer among the plurality of layers fixed.

[0167] The plurality of layers may include a first layer, a second layer, and a third layer.

[0168] In one embodiment of the present disclosure, quantization may mean expressing at least one parameter expressed in floating point as a fixed point. Here, expressing in fixed point includes assigning an integer value to at least one parameter expressed in floating point.

[0169] A second layer according to one embodiment may include at least one weight matrix used for operations on the acquired input.

[0170] An electronic device according to one embodiment may maintain the dimension of an input obtained based on at least one weight matrix included in a second layer, while changing the form of the input to perform a series of operations (see FIG. 2) included in an artificial intelligence model according to one embodiment.

[0171] A specific method of measuring a second reconstruction error for an input, in which an electronic device according to one embodiment quantizes at least one parameter included in a second layer among a plurality of layers included in a neural network model, and at least one parameter included in the remaining layers except for the second layer among the plurality of layers is fixed, will be described with reference to FIG. 10.

[0172] FIG. 10 is a diagram illustrating a method for measuring a second reconstruction error by an electronic device according to one embodiment.

[0173] An electronic device according to one embodiment may obtain a query matrix by applying the acquired input to a first layer. Here, the first layer may include a weight matrix for generating the query matrix. The weight matrix for generating the query matrix may include at least one parameter. At least one parameter included in the first layer may be expressed in floating-point form.

[0174] An electronic device according to one embodiment may obtain a key matrix by applying the acquired input to a second layer. Here, the second layer may include a weight matrix for generating the key matrix. The weight matrix for generating the key matrix may include at least one parameter. At least one parameter included in the second layer may be expressed in floating point form.

[0175] An electronic device according to one embodiment may obtain a value matrix by applying the acquired input to a third layer. Here, the third layer may include a weight matrix that generates the value matrix. The weight matrix that generates the value matrix may include at least one parameter. At least one parameter included in the third layer may be expressed in floating-point form.

[0176] An electronic device according to one embodiment can quantize at least one parameter included in a second layer (1020).

[0177] An electronic device according to one embodiment can fix the first layer and the third layer, excluding the second layer, among the first layer, the second layer, and the third layer (1010, 1030).

[0178] Here, the fact that the first and third layers are fixed means that at least one parameter included in the first and third layers is expressed as a floating point. In other words, it can mean a state in which at least one parameter included in the first and third layers is maintained in floating point form.

[0179] Accordingly, an electronic device according to one embodiment can express at least one parameter included in a second layer as an integer based on quantization, and can express at least one parameter included in a first layer and a third layer excluding the second layer as a floating point.

[0180] An electronic device according to one embodiment can measure a second reconstruction error for an input by quantizing at least one parameter included in a second layer among a plurality of layers included in a neural network model, and fixing at least one parameter included in the remaining layers except for the second layer among the plurality of layers.

[0181] The second reconstruction error may refer to an error that occurs when the electronic device according to one embodiment quantizes at least one parameter included in the second layer. In other words, the second reconstruction error may refer to the difference between an output obtained by the electronic device using the first to third layers that are not quantized and an output obtained by fixing the first and third layers and quantizing only the parameters included in the second layer.

[0182] For example, the second reconstruction error may mean an error between an output obtained by applying inputs to the first layer, the second layer, and the third layer, which include at least one parameter expressed in floating point, and an output obtained by applying inputs to the second layer, which includes at least one quantized parameter, and the first layer and the third layer, which include at least one parameter expressed in floating point, according to an embodiment of the present invention.

[0183] To explain in a more general sense, the reconstruction error according to one embodiment of the present disclosure may mean the difference between the output (original output) obtained from the neural network model in its original state that has not been quantized, and the output (quantized output) obtained by quantizing only one layer among a plurality of layers included in the neural network model and keeping the remaining layers fixed.

[0184] An electronic device according to one embodiment can obtain a third output by applying an input to a neural network model before quantizing at least one parameter included in a second layer.

[0185] The third output means an output obtained by applying an input to the first layer, the second layer and the third layer, wherein the electronic device according to one embodiment includes at least one parameter expressed in floating point.

[0186] An electronic device according to one embodiment can obtain a fourth output by applying an input to a neural network model after quantizing at least one parameter included in a second layer.

[0187] The fourth output means an output obtained by applying input to the second layer including at least one quantized parameter and the first layer and the third layer including at least one parameter expressed in floating point.

[0188] An electronic device according to one embodiment can measure a second reconstruction error based on the third output and the fourth output.

[0189] An electronic device according to one embodiment may measure a second reconstruction error based on comparing a third output with a fourth output. Here, the second reconstruction error may refer to an error that occurs when the electronic device according to one embodiment quantizes at least one parameter included in the second layer.

[0190] An electronic device according to one embodiment can adjust at least one quantized parameter included in a second layer such that a measured second reconstruction error is minimized.

[0191] An electronic device according to one embodiment can obtain an objective function for measuring a second reconstruction error.

[0192] The objective function refers to a function and / or term that is the target of optimization in order to minimize an error that occurs when an electronic device according to one embodiment quantizes at least one parameter included in a second layer.

[0193] A specific method by which an electronic device according to one embodiment obtains an objective function for measuring a second reconstruction error is symmetrical to a method by which an electronic device according to one embodiment obtains an objective function for measuring a first reconstruction error (see FIG. 6), and therefore, a redundant description is omitted.

[0194] An electronic device according to one embodiment is The term can be obtained as an objective function. Here, refers to the weight matrix that generates the key matrix.

[0195] An electronic device according to one embodiment can transform an objective function such that a component independent of quantization among at least one component included in the objective function is separated.

[0196] A specific method by which an electronic device according to one embodiment transforms an objective function so that a quantization-independent component among at least one component included in the objective function is separated in relation to a second reconstruction error is symmetrical to a method by which an electronic device according to one embodiment transforms an objective function so that a quantization-independent component among at least one component included in the objective function is separated in relation to a first reconstruction error (see FIG. 6), and therefore, a redundant description is omitted.

[0197] An electronic device according to one embodiment is Port and The term can be judged as a component independent of the quantization of at least one parameter included in the second layer.

[0198] Here, an independent component may mean a component that is not affected by the quantization of at least one parameter included in the second layer.

[0199] An electronic device according to one embodiment can measure a second reconstruction error from a transformed objective function by applying a value calculated according to learning data of a neural network model to a component independent of quantization.

[0200] An electronic device according to one embodiment corresponds to a component independent of the quantization of at least one parameter included in a second layer obtained from an objective function. Port and By applying the values ​​calculated according to the learning data of the neural network model to the term, the second reconstruction error can be measured from the transformed objective function.

[0201] Here, the learning data of the neural network model may mean data used for learning an artificial intelligence model operated in an electronic device according to one embodiment.

[0202] Referring again to Figure 9,

[0203] At step 920, the electronic device can adjust at least one quantized parameter included in the second layer based on the second reconstruction error.

[0204] An electronic device according to one embodiment can determine at least one quantization parameter including a scale factor and a zero point.

[0205] A quantization parameter may mean a variable related to an interval for quantizing at least one parameter included in a second layer.

[0206] The scale factor may mean the ratio between the range of continuous real values ​​of at least one parameter to be quantized and the range of integer values ​​to be quantized.

[0207] For example, if the continuous real value range of at least one parameter to be quantized is [-1.0, 3.0] and the range of integer values ​​to be quantized is [0, 255], the scale factor may be determined as 4.0 / 255. However, the present invention is not limited thereto.

[0208] A zero point may mean a parameter that includes reference point information about which value of 0 in at least one quantized parameter corresponds to which value among the range of real values ​​of at least one parameter before quantization.

[0209] An electronic device according to one embodiment can determine a quantization parameter used in adjusting at least one parameter included in a second layer based on a quantization parameter used in quantizing at least one parameter included in a second layer.

[0210] An electronic device according to one embodiment can group at least one parameter included in a second layer into at least one group.

[0211] An electronic device according to one embodiment can determine a quantization parameter including a scale exponent and a zero point based on a range of at least one parameter included in at least one group.

[0212] An electronic device according to one embodiment can determine quantization parameters, including scale indices and zero points, used for each group based on a range of at least one parameter included in at least one group.

[0213] An electronic device according to one embodiment can learn at least one quantization parameter such that a second reconstruction error is reduced.

[0214] An electronic device according to one embodiment may learn at least one quantization parameter to reduce a second reconstruction error based on a grid search technique, convex optimization, or backpropagation.

[0215] An electronic device according to one embodiment can adjust at least one quantized parameter included in a second layer based on at least one quantized parameter for which learning has been completed.

[0216] Fig. 11 is a flowchart for explaining an operating method of an electronic device according to one embodiment.

[0217] Step 330 of FIG. 11 corresponds to step 330 of FIG. 3.

[0218] In step 1110, the electronic device quantizes at least one parameter included in a third layer among a plurality of layers included in the neural network model, and measures a third reconstruction error for the input while keeping at least one parameter included in the remaining layers except for the third layer among the plurality of layers fixed.

[0219] The plurality of layers may include a first layer, a second layer, and a third layer.

[0220] A third layer according to one embodiment may include at least one weight matrix used for operations on the acquired input.

[0221] An electronic device according to one embodiment may maintain the dimension of an input obtained based on at least one weight matrix included in a third layer, while changing the form of the input to perform a series of operations (see FIG. 2) included in an artificial intelligence model according to one embodiment.

[0222] A specific method of measuring a third reconstruction error for an input, in which an electronic device according to one embodiment quantizes at least one parameter included in a third layer among a plurality of layers included in a neural network model, and at least one parameter included in the remaining layers except for the third layer among the plurality of layers is fixed, will be described with reference to FIG. 12.

[0223] FIG. 12 is a diagram illustrating a method for measuring a third reconstruction error by an electronic device according to one embodiment.

[0224] An electronic device according to one embodiment may obtain a query matrix by applying the acquired input to a first layer. Here, the first layer may include a weight matrix for generating the query matrix. The weight matrix for generating the query matrix may include at least one parameter. At least one parameter included in the first layer may be expressed in floating-point form.

[0225] An electronic device according to one embodiment may obtain a key matrix by applying the acquired input to a second layer. Here, the second layer may include a weight matrix for generating the key matrix. The weight matrix for generating the key matrix may include at least one parameter. At least one parameter included in the second layer may be expressed in floating point form.

[0226] An electronic device according to one embodiment may obtain a value matrix by applying the acquired input to a third layer. Here, the third layer may include a weight matrix that generates the value matrix. The weight matrix that generates the value matrix may include at least one parameter. At least one parameter included in the third layer may be expressed in floating-point form.

[0227] An electronic device according to one embodiment can quantize at least one parameter included in a third layer (1230).

[0228] An electronic device according to one embodiment can fix the first layer and the second layer, excluding the third layer, among the first layer, the second layer, and the third layer (1210, 1220).

[0229] Here, the fact that the first and third layers are fixed means that at least one parameter included in the first and second layers is expressed as a floating point. In other words, it can mean a state in which at least one parameter included in the first and second layers is maintained in floating point form.

[0230] Accordingly, an electronic device according to one embodiment can express at least one parameter included in a third layer as an integer based on quantization, and can express at least one parameter included in a first layer and a second layer excluding the third layer as a floating point.

[0231] An electronic device according to one embodiment can measure a third reconstruction error for an input by quantizing at least one parameter included in a third layer among a plurality of layers included in a neural network model, and keeping at least one parameter included in the remaining layers except for the third layer among the plurality of layers fixed.

[0232] The third reconstruction error may refer to an error that occurs when the electronic device according to one embodiment quantizes at least one parameter included in the third layer. In other words, the third reconstruction error may refer to the difference between an output obtained by the electronic device using the first to third layers that are not quantized and an output obtained by quantizing only the parameters included in the third layer while keeping the first and second layers fixed.

[0233] For example, a third reconstruction error may mean an error between an output obtained by applying inputs to a first layer, a second layer, and a third layer including at least one parameter expressed in floating point, and an output obtained by applying inputs to a third layer including at least one quantized parameter and the first layer and the second layer including at least one parameter expressed in floating point, according to an embodiment of the present invention.

[0234] To explain in a more general sense, the reconstruction error according to one embodiment of the present disclosure may mean the difference between the output (original output) obtained from the neural network model in its original state that has not been quantized, and the output (quantized output) obtained by quantizing only one layer among a plurality of layers included in the neural network model and keeping the remaining layers fixed.

[0235] An electronic device according to one embodiment can obtain a fifth output by applying an input to a neural network model before quantizing at least one parameter included in a third layer.

[0236] The fifth output means an output obtained by applying an input to the first layer, the second layer, and the third layer, wherein the electronic device according to one embodiment includes at least one parameter expressed in floating point.

[0237] An electronic device according to one embodiment can obtain a sixth output by applying an input to a neural network model after quantizing at least one parameter included in a third layer.

[0238] The sixth output means an output obtained by applying input to the third layer including at least one quantized parameter and the first and second layers including at least one parameter expressed in floating point.

[0239] An electronic device according to one embodiment can measure a third reconstruction error based on the fifth output and the sixth output.

[0240] An electronic device according to one embodiment may measure a third reconstruction error based on comparing the fifth output and the sixth output. Here, the third reconstruction error may refer to an error that occurs when the electronic device according to one embodiment quantizes at least one parameter included in the third layer.

[0241] An electronic device according to one embodiment can adjust at least one quantized parameter included in a third layer such that a measured third reconstruction error is minimized.

[0242] An electronic device according to one embodiment can obtain an objective function for measuring a third reconstruction error.

[0243] The objective function refers to a function and / or term that is the target of optimization in order to minimize an error that occurs when an electronic device according to one embodiment quantizes at least one parameter included in a third layer.

[0244] An electronic device according to one embodiment of the present invention measures a third reconstruction error based on the following [Mathematical Formula 8]: ) can be obtained.

[0245] [Equation 8]

[0246]

[0247] stands for quantized value matrix.

[0248] refers to the weight matrix that generates the value matrix.

[0249] Components included in [Mathematical Formula 8] but not separately defined correspond to the components defined in [Mathematical Formula 1] to [Mathematical Formula 7], respectively.

[0250] An electronic device according to one embodiment can transform an objective function such that a component independent of quantization among at least one component included in the objective function is separated.

[0251] Objective function obtained from [Equation 8] If the trace (tr) operation is applied to the term, [Equation 8] can be expressed as [Equation 9] below.

[0252] [Equation 9]

[0253]

[0254] Components included in [Mathematical Formula 9] but not separately defined correspond to the components defined in [Mathematical Formula 1] to [Mathematical Formula 8], respectively.

[0255] An electronic device according to one embodiment is The term can be considered as a component independent of the quantization of at least one parameter included in the third layer.

[0256] Here, an independent component may mean a component that is not affected by the quantization of at least one parameter included in the third layer.

[0257] An electronic device according to one embodiment can measure a third reconstruction error from a transformed objective function by applying a value calculated according to learning data of a neural network model to a component independent of quantization.

[0258] An electronic device according to one embodiment corresponds to a component independent of the quantization of at least one parameter included in a third layer obtained from an objective function. By applying the values ​​calculated according to the learning data of the neural network model to the term, the third reconstruction error can be measured from the transformed objective function.

[0259] Here, the learning data of the neural network model may mean data used for learning an artificial intelligence model operated in an electronic device according to one embodiment.

[0260] Referring again to Figure 11,

[0261] At step 1120, the electronic device can adjust at least one quantized parameter included in the third layer based on the third reconstruction error.

[0262] An electronic device according to one embodiment can determine at least one quantization parameter including a scale factor and a zero point.

[0263] A quantization parameter may mean a variable related to an interval for quantizing at least one parameter included in the third layer.

[0264] The scale factor may mean the ratio between the range of continuous real values ​​of at least one parameter to be quantized and the range of integer values ​​to be quantized.

[0265] For example, if the continuous real value range of at least one parameter to be quantized is [-2.0, 2.0] and the range of integer values ​​to be quantized is [0, 255], the scale factor may be determined as 4.0 / 255. However, the present invention is not limited thereto.

[0266] A zero point may mean a parameter that includes reference point information about which value of 0 in at least one quantized parameter corresponds to which value among the range of real values ​​of at least one parameter before quantization.

[0267] An electronic device according to one embodiment can determine a quantization parameter used in adjusting at least one parameter included in a third layer based on a quantization parameter used in quantizing at least one parameter included in a third layer.

[0268] An electronic device according to one embodiment can group at least one parameter included in a third layer into at least one group.

[0269] An electronic device according to one embodiment can determine a quantization parameter including a scale exponent and a zero point based on a range of at least one parameter included in at least one group.

[0270] An electronic device according to one embodiment can determine quantization parameters, including scale indices and zero points, used for each group based on a range of at least one parameter included in at least one group.

[0271] An electronic device according to one embodiment can learn at least one quantization parameter such that a third reconstruction error is reduced.

[0272] An electronic device according to one embodiment can learn at least one quantization parameter to reduce a third reconstruction error based on a grid search technique, convex optimization, or backpropagation.

[0273] An electronic device according to one embodiment can adjust at least one quantized parameter included in a third layer based on at least one quantized parameter for which learning has been completed.

[0274] FIG. 13 is a diagram illustrating a method for an electronic device according to one embodiment to quantize a neural network model based on a first reconstruction error, a second reconstruction error, and a third reconstruction error.

[0275] In an artificial intelligence model (1300) according to one embodiment including a first layer, a second layer, and a third layer, an embodiment (1310) of quantizing at least one parameter included in the first layer and fixing the second layer and the third layer, an embodiment (1320) of quantizing at least one parameter included in the second layer and fixing the first layer and the third layer, and an embodiment (1330) of quantizing at least one parameter included in the third layer and fixing the first layer and the second layer may be performed serially and / or in parallel.

[0276] In FIGS. 3 to 12, the present disclosure has been described on the premise that the first layer includes a weight matrix for generating a query matrix, the second layer includes a weight matrix for generating a key matrix, and the third layer includes a weight matrix for generating a value matrix. However, this is only to define each weight matrix included in each layer in order to more clearly explain the invention, and each weight matrix included in each layer may be included regardless of the order of the layers.

[0277] For example, a first layer may include a weight matrix that generates a value matrix, a second layer may include a weight matrix that generates a key matrix, and a third layer may include a weight matrix that generates a query matrix.

[0278] For example, a first layer may include a weight matrix that generates a value matrix, a second layer may include a weight matrix that generates a query matrix, and a third layer may include a weight matrix that generates a key matrix.

[0279] For example, the first layer may include a weight matrix that generates a key matrix, the second layer may include a weight matrix that generates a value matrix, and the third layer may include a weight matrix that generates a query matrix, but is not limited thereto.

[0280] FIG. 14 is a block diagram illustrating components of an electronic device according to one embodiment.

[0281] An electronic device (1400) according to one embodiment may include a memory (1420) storing a program or at least one instruction and at least one processor (1410) executing at least one instruction stored in the memory (1420).

[0282] The memory (1420) may store various data, programs, or applications for driving and controlling the electronic device (1400) according to one embodiment. The memory (1420) may include, for example, a non-volatile memory including at least one of a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., an SD or XD memory, etc.), a ROM (Read-Only Memory), and an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), and a volatile memory such as a RAM (Random Access Memory) or an SRAM (Static Random Access Memory).

[0283] The memory (1420) may store instructions, data structures, and program codes that can be read by the processor (1410). In the following embodiments, the processor (1410) may be implemented by executing instructions or codes of a program stored in the memory (1420).

[0284] The processor (1410) is a configuration that controls a series of processes to allow the electronic device (300) to operate according to the embodiments described below, and may be composed of one or more processors.

[0285] The processor (1410) may be composed of hardware components that perform arithmetic, logic, and input / output operations and signal processing. One or more processors included in the processor (1410) may be circuitry such as a System on Chip (SoC), an Integrated Circuit (IC), etc. The processor (210) may be composed of at least one of, for example, a Central Processing Unit (CPU), a microprocessor, a Graphic Processing Unit (GPU), Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), and Field Programmable Gate Arrays (FPGAs), but is not limited thereto.

[0286] The processor (1410) can write data to the memory (1420), read data stored in the memory (1420), and process data according to predefined operation rules, particularly by executing a program or at least one command stored in the memory (1420).

[0287] According to one embodiment, the electronic device (1400) can obtain input by having at least one processor (1410) execute at least one instruction stored in the memory (1420).

[0288] The input can be an input sequence in the form of a vector and / or matrix.

[0289] Here, the input sequence refers to the actual input supplied to the model, and can refer to the entire input that the model must process. The input sequence can refer to the entire data passed to the model's input layer.

[0290] According to one embodiment, at least one processor (1410) executes at least one command stored in a memory (1420), so that the electronic device (1400) can quantize at least one parameter included in a first layer among a plurality of layers included in a neural network model, and measure a first reconstruction error for an input while at least one parameter included in the remaining layers excluding the first layer among the plurality of layers is fixed.

[0291] The plurality of layers may include a first layer, a second layer, and a third layer.

[0292] In one embodiment of the present disclosure, quantization may mean expressing at least one parameter expressed in floating point as a fixed point. Here, expressing in fixed point includes assigning an integer value to at least one parameter expressed in floating point.

[0293] A first layer according to one embodiment may include at least one weight matrix used for operations on the acquired input.

[0294] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), so that the electronic device (1400) can change the form of the input to perform a series of operations (see FIG. 2) included in an artificial intelligence model according to one embodiment while maintaining the dimension of the input obtained based on at least one weight matrix included in the first layer.

[0295] According to one embodiment, at least one processor (1410) executes at least one command stored in a memory (1420), so that the electronic device (1400) can apply the acquired input to a first layer to obtain a query matrix. Here, the first layer may include a weight matrix that generates the query matrix. The weight matrix that generates the query matrix may include at least one parameter. At least one parameter included in the first layer may be expressed in floating point form.

[0296] According to one embodiment, at least one processor (1410) executes at least one command stored in a memory (1420), so that the electronic device (1400) can apply the acquired input to a second layer to obtain a key matrix. Here, the second layer may include a weight matrix that generates the key matrix. The weight matrix that generates the key matrix may include at least one parameter. At least one parameter included in the second layer may be expressed in floating point form.

[0297] According to one embodiment, at least one processor (1410) executes at least one command stored in a memory (1420), so that the electronic device (1400) can apply the acquired input to a third layer to obtain a value matrix. Here, the third layer may include a weight matrix that generates the value matrix. The weight matrix that generates the value matrix may include at least one parameter. At least one parameter included in the third layer may be expressed in floating point form.

[0298] According to one embodiment, the electronic device (1400) can quantize at least one parameter included in the first layer by having at least one processor (1410) execute at least one instruction stored in the memory (1420).

[0299] According to one embodiment, at least one processor (1410) executes at least one command stored in a memory (1420), thereby enabling the electronic device (1400) to fix the second layer and the third layer, excluding the first layer, among the first layer, the second layer, and the third layer.

[0300] Here, the fixation of the second and third layers means that at least one parameter included in the second and third layers is expressed as a floating point. In other words, it can mean a state in which at least one parameter included in the second and third layers is maintained in floating point form.

[0301] Accordingly, an electronic device according to one embodiment can express at least one parameter included in a first layer as an integer based on quantization, and can express at least one parameter included in a second layer and a third layer excluding the first layer as a floating point.

[0302] According to one embodiment, at least one processor (1410) executes at least one command stored in a memory (1420), so that the electronic device (1400) can quantize at least one parameter included in a first layer among a plurality of layers included in a neural network model, and measure a first reconstruction error for an input while at least one parameter included in the remaining layers excluding the first layer among the plurality of layers is fixed.

[0303] The first reconstruction error may refer to an error that occurs when the electronic device according to one embodiment quantizes at least one parameter included in the first layer. That is, the first reconstruction error may refer to an error between an output obtained by applying an input to the first layer, the second layer, and the third layer, which include at least one parameter expressed in floating point, and an output obtained by applying an input to the first layer, which includes at least one quantized parameter, and the second layer and the third layer, which include at least one parameter expressed in floating point.

[0304] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), so that the electronic device (1400) can adjust (update) at least one quantized parameter included in the first layer based on the first reconstruction error.

[0305] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), thereby allowing the electronic device (1400) to obtain a first output by applying an input to a neural network model prior to quantizing at least one parameter included in a first layer.

[0306] The first output means an output obtained by applying an input to the first layer, the second layer and the third layer, wherein the electronic device according to one embodiment includes at least one parameter expressed in floating point.

[0307] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), so that the electronic device (1400) can obtain a second output by applying an input to a neural network model after quantizing at least one parameter included in a first layer.

[0308] The second output means an output obtained by applying input to the first layer including at least one quantized parameter and the second and third layers including at least one parameter expressed in floating point.

[0309] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), thereby enabling the electronic device (1400) to measure a first reconstruction error based on the first output and the second output.

[0310] According to one embodiment, at least one processor (1410) executes at least one instruction stored in the memory (1420), and the electronic device (1400) can measure a first reconstruction error based on comparing a first output with a second output. Here, the first reconstruction error may refer to an error generated when the electronic device according to one embodiment quantizes at least one parameter included in the first layer.

[0311] An electronic device according to one embodiment may measure a first reconstruction error in the form of a mean square error.

[0312] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), thereby enabling the electronic device (1400) to adjust at least one quantized parameter included in the first layer so that the measured first reconstruction error is minimized.

[0313] According to one embodiment, at least one processor (1410) may execute at least one instruction stored in a memory (1420) so that the electronic device (1400) can obtain an objective function for measuring a first reconstruction error.

[0314] Here, the objective function means a function and / or term to be optimized to minimize an error that occurs when an electronic device according to one embodiment quantizes at least one parameter included in a first layer.

[0315] A specific method of deriving an objective function from a first reconstruction error measured by an electronic device (1400) by having at least one processor (1410) execute at least one instruction stored in a memory (1420) according to one embodiment has been described above (see FIG. 5), and therefore, a redundant description thereof will be omitted.

[0316] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), thereby enabling the electronic device (1400) to transform the objective function so that at least one component included in the objective function is separated from a quantization-independent component.

[0317] The method of transforming an objective function so that at least one component included in the objective function is separated from a quantization-independent component by having at least one processor (1410) execute at least one instruction stored in a memory (1420) according to one embodiment has been described above (see FIG. 5), and therefore, a redundant description will be omitted.

[0318] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), so that the electronic device (1400) can measure a first reconstruction error from the transformed objective function by applying a value calculated according to the learning data of the neural network model to a component independent of quantization.

[0319] Here, the learning data of the neural network model may mean data used for learning an artificial intelligence model and / or a neural network model operated in an electronic device according to one embodiment.

[0320] According to one embodiment, at least one processor (1410) may execute at least one instruction stored in a memory (1420) so that the electronic device (1400) can determine at least one quantization parameter including a scale factor and a zero point.

[0321] A quantization parameter may mean a variable related to an interval for quantizing at least one parameter included in the first layer.

[0322] The scale factor may mean the ratio between the range of continuous real values ​​of at least one parameter to be quantized and the range of integer values ​​to be quantized.

[0323] For example, if the continuous real value range of at least one parameter to be quantized is [-1.0, 1.0] and the range of integer values ​​to be quantized is [0, 255], the scale factor may be determined as 2.0 / 255. However, the present invention is not limited thereto.

[0324] A zero point may mean a parameter that includes reference point information about which value of 0 in at least one quantized parameter corresponds to which value among the range of real values ​​of at least one parameter before quantization.

[0325] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), so that the electronic device (1400) can determine a quantization parameter used in adjusting at least one parameter included in the first layer based on the quantization parameter used in quantizing at least one parameter included in the first layer.

[0326] According to one embodiment, at least one processor (1410) executes at least one command stored in a memory (1420), thereby allowing the electronic device (1400) to group at least one parameter included in the first layer into at least one group.

[0327] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), thereby enabling the electronic device (1400) to determine a quantization parameter including a scale index and a zero point based on a range of at least one parameter included in at least one group.

[0328] According to one embodiment, at least one processor (1410) executes at least one instruction stored in the memory (1420), so that the electronic device (1400) can determine quantization parameters, including scale indices and zero points, used for each group based on the range of at least one parameter included in at least one group.

[0329] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), thereby enabling the electronic device (1400) to adjust at least one parameter included in the first layer based on at least one quantization parameter.

[0330] According to one embodiment, at least one processor (1410) may execute at least one instruction stored in a memory (1420) so that the electronic device (1400) can learn at least one quantization parameter so that the first reconstruction error is reduced.

[0331] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), thereby enabling the electronic device (1400) to learn at least one quantization parameter to reduce a first reconstruction error based on a grid search technique, convex optimization, or backpropagation.

[0332] According to one embodiment, at least one processor (1410) executes at least one command stored in a memory (1420), thereby allowing the electronic device (1400) to adjust at least one quantized parameter included in the first layer based on at least one quantized parameter for which learning has been completed.

[0333] According to one embodiment, at least one processor (1410) executes at least one command stored in a memory (1420), so that the electronic device (1400) can quantize at least one parameter included in a second layer among a plurality of layers included in a neural network model, and measure a second reconstruction error for an input while at least one parameter included in the remaining layers except for the second layer among the plurality of layers is fixed.

[0334] The plurality of layers may include a first layer, a second layer, and a third layer.

[0335] In one embodiment of the present disclosure, quantization may mean expressing at least one parameter expressed in floating point as a fixed point. Here, expressing in fixed point includes assigning an integer value to at least one parameter expressed in floating point.

[0336] A second layer according to one embodiment may include at least one weight matrix used for operations on the acquired input.

[0337] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), so that the electronic device (1400) can change the form of the input to perform a series of operations (see FIG. 2) included in an artificial intelligence model according to one embodiment while maintaining the dimension of the input obtained based on at least one weight matrix included in the second layer.

[0338] According to one embodiment, the electronic device (1400) can quantize at least one parameter included in the second layer by having at least one processor (1410) execute at least one instruction stored in the memory (1420).

[0339] According to one embodiment, at least one processor (1410) executes at least one command stored in a memory (1420), thereby enabling the electronic device (1400) to fix the first layer and the third layer, excluding the second layer, among the first layer, the second layer, and the third layer.

[0340] Here, the fact that the first and third layers are fixed means that at least one parameter included in the first and third layers is expressed as a floating point. In other words, it can mean a state in which at least one parameter included in the first and third layers is maintained in floating point form.

[0341] Accordingly, an electronic device according to one embodiment can express at least one parameter included in a second layer as an integer based on quantization, and can express at least one parameter included in a first layer and a third layer excluding the second layer as a floating point.

[0342] According to one embodiment, at least one processor (1410) executes at least one command stored in a memory (1420), so that the electronic device (1400) can quantize at least one parameter included in a second layer among a plurality of layers included in a neural network model, and measure a second reconstruction error for an input while at least one parameter included in the remaining layers except for the second layer among the plurality of layers is fixed.

[0343] The second reconstruction error may refer to an error that occurs when the electronic device according to one embodiment quantizes at least one parameter included in the second layer. That is, the second reconstruction error may refer to an error between an output obtained by applying an input to the first layer, the second layer, and the third layer, which include at least one parameter expressed in floating point, and an output obtained by applying an input to the second layer, which includes at least one quantized parameter, and the first layer and the third layer, which include at least one parameter expressed in floating point.

[0344] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), thereby allowing the electronic device (1400) to obtain a third output by applying an input to a neural network model prior to quantizing at least one parameter included in a second layer.

[0345] The third output means an output obtained by applying an input to the first layer, the second layer and the third layer, wherein the electronic device according to one embodiment includes at least one parameter expressed in floating point.

[0346] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), so that the electronic device (1400) can obtain a fourth output by applying an input to a neural network model after quantizing at least one parameter included in the second layer.

[0347] The fourth output means an output obtained by applying input to the second layer including at least one quantized parameter and the first layer and the third layer including at least one parameter expressed in floating point.

[0348] According to one embodiment, at least one processor (1410) executes at least one instruction stored in the memory (1420), so that the electronic device (1400) can measure the second reconstruction error based on the third output and the fourth output.

[0349] According to one embodiment, at least one processor (1410) executes at least one instruction stored in the memory (1420), and the electronic device (1400) can measure a second reconstruction error based on comparing the third output and the fourth output. Here, the second reconstruction error may refer to an error that occurs when the electronic device according to one embodiment quantizes at least one parameter included in the second layer.

[0350] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), thereby enabling the electronic device (1400) to adjust at least one quantized parameter included in the second layer so as to minimize the second reconstruction error.

[0351] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), thereby enabling the electronic device (1400) to obtain an objective function for measuring a second reconstruction error.

[0352] The objective function refers to a function and / or term that is the target of optimization in order to minimize an error that occurs when an electronic device according to one embodiment quantizes at least one parameter included in a second layer.

[0353] A specific method for obtaining an objective function for measuring a second reconstruction error by having at least one processor (1410) execute at least one instruction stored in a memory (1420) according to one embodiment of the present invention has been described above (see FIG. 10), and therefore, a redundant description thereof will be omitted.

[0354] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), thereby enabling the electronic device (1400) to transform the objective function so that at least one component included in the objective function is separated from a quantization-independent component.

[0355] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), so that the electronic device (1400) can measure a second reconstruction error from the transformed objective function by applying a value calculated according to the learning data of the neural network model to a component independent of quantization.

[0356] The training data of the neural network model may refer to data used for training an artificial intelligence model operated in an electronic device according to one embodiment.

[0357] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), thereby allowing the electronic device (1400) to adjust at least one quantized parameter included in the second layer based on the second reconstruction error.

[0358] An electronic device according to one embodiment can determine at least one quantization parameter including a scale factor and a zero point.

[0359] A quantization parameter may mean a variable related to an interval for quantizing at least one parameter included in a second layer.

[0360] The scale factor may mean the ratio between the range of continuous real values ​​of at least one parameter to be quantized and the range of integer values ​​to be quantized.

[0361] For example, if the continuous real value range of at least one parameter to be quantized is [-1.0, 3.0] and the range of integer values ​​to be quantized is [0, 255], the scale factor may be determined as 4.0 / 255. However, the present invention is not limited thereto.

[0362] A zero point may mean a parameter that includes reference point information about which value of 0 in at least one quantized parameter corresponds to which value among the range of real values ​​of at least one parameter before quantization.

[0363] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), so that the electronic device (1400) can determine a quantization parameter used in adjusting at least one parameter included in the second layer based on the quantization parameter used in quantizing at least one parameter included in the second layer.

[0364] According to one embodiment, at least one processor (1410) executes at least one command stored in a memory (1420), thereby allowing the electronic device (1400) to group at least one parameter included in the second layer into at least one group.

[0365] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), thereby enabling the electronic device (1400) to determine a quantization parameter including a scale index and a zero point based on a range of at least one parameter included in at least one group.

[0366] According to one embodiment, at least one processor (1410) executes at least one instruction stored in the memory (1420), so that the electronic device (1400) can determine quantization parameters, including scale indices and zero points, used for each group based on the range of at least one parameter included in at least one group.

[0367] According to one embodiment, at least one processor (1410) may execute at least one instruction stored in a memory (1420) so that the electronic device (1400) can learn at least one quantization parameter so that the second reconstruction error is reduced.

[0368] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), thereby enabling the electronic device (1400) to learn at least one quantization parameter to reduce a second reconstruction error based on a grid search technique, convex optimization, or backpropagation.

[0369] According to one embodiment, at least one processor (1410) executes at least one command stored in a memory (1420), thereby allowing the electronic device (1400) to adjust at least one quantized parameter included in the second layer based on at least one quantized parameter for which learning has been completed.

[0370] According to one embodiment, at least one processor (1410) executes at least one command stored in a memory (1420), so that the electronic device (1400) can quantize at least one parameter included in a third layer among a plurality of layers included in a neural network model, and measure a third reconstruction error for an input while at least one parameter included in the remaining layers except for the third layer among the plurality of layers is fixed.

[0371] The plurality of layers may include a first layer, a second layer, and a third layer.

[0372] A third layer according to one embodiment may include at least one weight matrix used for operations on the acquired input.

[0373] An electronic device according to one embodiment may maintain the dimension of an input obtained based on at least one weight matrix included in a third layer, while changing the form of the input to perform a series of operations (see FIG. 2) included in an artificial intelligence model according to one embodiment.

[0374] According to one embodiment, the electronic device (1400) can quantize at least one parameter included in the third layer by having at least one processor (1410) execute at least one instruction stored in the memory (1420).

[0375] According to one embodiment, at least one processor (1410) executes at least one command stored in a memory (1420), thereby enabling the electronic device (1400) to fix the first layer and the second layer, excluding the third layer, among the first layer, the second layer, and the third layer.

[0376] Here, the fact that the first and third layers are fixed means that at least one parameter included in the first and second layers is expressed as a floating point. In other words, it can mean a state in which at least one parameter included in the first and second layers is maintained in floating point form.

[0377] Accordingly, an electronic device according to one embodiment can express at least one parameter included in a third layer as an integer based on quantization, and can express at least one parameter included in a first layer and a second layer excluding the third layer as a floating point.

[0378] According to one embodiment, at least one processor (1410) executes at least one command stored in a memory (1420), so that the electronic device (1400) can quantize at least one parameter included in a third layer among a plurality of layers included in a neural network model, and measure a third reconstruction error for an input while at least one parameter included in the remaining layers except for the third layer among the plurality of layers is fixed.

[0379] The third reconstruction error may refer to an error that occurs when the electronic device according to one embodiment quantizes at least one parameter included in the third layer. That is, the third reconstruction error may refer to an error between an output obtained by applying an input to the first layer, the second layer, and the third layer, which include at least one parameter expressed in floating point, and an output obtained by applying an input to the third layer, which includes at least one quantized parameter, and the first layer and the second layer, which include at least one parameter expressed in floating point.

[0380] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), so that the electronic device (1400) can obtain a fifth output by applying an input to a neural network model before quantizing at least one parameter included in the third layer.

[0381] The fifth output means an output obtained by applying an input to the first layer, the second layer, and the third layer, wherein the electronic device according to one embodiment includes at least one parameter expressed in floating point.

[0382] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), so that the electronic device (1400) can obtain a sixth output by applying an input to a neural network model after quantizing at least one parameter included in the third layer.

[0383] The sixth output means an output obtained by applying input to the third layer including at least one quantized parameter and the first and second layers including at least one parameter expressed in floating point.

[0384] According to one embodiment, at least one processor (1410) executes at least one instruction stored in the memory (1420), so that the electronic device (1400) can measure the third reconstruction error based on the fifth output and the sixth output.

[0385] According to one embodiment, at least one processor (1410) executes at least one instruction stored in the memory (1420), and the electronic device (1400) can measure a third reconstruction error based on comparing the fifth output and the sixth output. Here, the third reconstruction error may refer to an error that occurs when the electronic device according to one embodiment quantizes at least one parameter included in the third layer.

[0386] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), thereby allowing the electronic device (1400) to adjust at least one quantized parameter included in the third layer so that the measured third reconstruction error is minimized.

[0387] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), thereby enabling the electronic device (1400) to obtain an objective function for measuring a third reconstruction error.

[0388] Here, the objective function means a function and / or term to be optimized to minimize an error that occurs when an electronic device according to one embodiment quantizes at least one parameter included in a third layer.

[0389] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), thereby enabling the electronic device (1400) to transform the objective function so that at least one component included in the objective function is separated from a quantization-independent component.

[0390] A specific method for transforming an objective function so that at least one component included in the objective function is separated from a quantization-independent component by having at least one processor (1410) execute at least one instruction stored in a memory (1420) according to one embodiment has been described above (see FIG. 12), and therefore, a redundant description will be omitted.

[0391] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), so that the electronic device (1400) can measure a third reconstruction error from the transformed objective function by applying a value calculated according to the learning data of the neural network model to a component independent of quantization.

[0392] Here, the learning data of the neural network model may mean data used for learning an artificial intelligence model operated in an electronic device according to one embodiment.

[0393] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), so that the electronic device (1400) can adjust at least one quantized parameter included in the third layer based on the third reconstruction error.

[0394] According to one embodiment, at least one processor (1410) may execute at least one instruction stored in a memory (1420) so that the electronic device (1400) can determine at least one quantization parameter including a scale factor and a zero point.

[0395] A quantization parameter may mean a variable related to an interval for quantizing at least one parameter included in the third layer.

[0396] The scale factor may mean the ratio between the range of continuous real values ​​of at least one parameter to be quantized and the range of integer values ​​to be quantized.

[0397] For example, if the continuous real value range of at least one parameter to be quantized is [-2.0, 2.0] and the range of integer values ​​to be quantized is [0, 255], the scale factor may be determined as 4.0 / 255. However, the present invention is not limited thereto.

[0398] A zero point may mean a parameter that includes reference point information about which value of 0 in at least one quantized parameter corresponds to which value among the range of real values ​​of at least one parameter before quantization.

[0399] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), so that the electronic device (1400) can determine a quantization parameter used in adjusting at least one parameter included in the third layer based on the quantization parameter used in quantizing at least one parameter included in the third layer.

[0400] According to one embodiment, at least one processor (1410) executes at least one command stored in a memory (1420), thereby allowing the electronic device (1400) to group at least one parameter included in the third layer into at least one group.

[0401] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), thereby enabling the electronic device (1400) to determine a quantization parameter including a scale index and a zero point based on a range of at least one parameter included in at least one group.

[0402] According to one embodiment, at least one processor (1410) executes at least one instruction stored in the memory (1420), so that the electronic device (1400) can determine quantization parameters, including scale indices and zero points, used for each group based on the range of at least one parameter included in at least one group.

[0403] According to one embodiment, at least one processor (1410) executes at least one instruction stored in a memory (1420), thereby enabling the electronic device (1400) to learn at least one quantization parameter to reduce a third reconstruction error based on a grid search technique, convex optimization, or backpropagation.

[0404] According to one embodiment, at least one processor (1410) executes at least one command stored in a memory (1420), so that the electronic device (1400) can adjust at least one quantized parameter included in the third layer based on at least one quantized parameter for which learning has been completed.

[0405] A method according to one embodiment may include obtaining an input.

[0406] A method according to one embodiment may include a step of quantizing at least one parameter included in a first layer among a plurality of layers included in the neural network model, and measuring a first reconstruction error for the input while fixing at least one parameter included in the remaining layers excluding the first layer among the plurality of layers.

[0407] A method according to one embodiment may include adjusting at least one quantized parameter included in the first layer based on the first reconstruction error.

[0408] A method according to one embodiment may include a step of obtaining a first output by applying the input to the neural network model before quantizing at least one parameter included in the first layer.

[0409] A method according to one embodiment may include a step of applying the input to the neural network model after quantizing at least one parameter included in the first layer to obtain a second output.

[0410] A method according to one embodiment may include a step of measuring the first reconstruction error based on the first output and the second output.

[0411] A method according to one embodiment may include obtaining an objective function for measuring the first reconstruction error.

[0412] A method according to one embodiment may include a step of transforming the objective function such that a component independent of the quantization is separated from at least one component included in the objective function.

[0413] A method according to one embodiment may include a step of measuring the first reconstruction error from the transformed objective function by applying a value calculated according to training data of the neural network model to a component independent of the quantization.

[0414] A method according to one embodiment may include determining at least one quantization parameter including a scaling factor and a zero point.

[0415] A method according to one embodiment may include adjusting at least one quantized parameter included in the first layer based on the at least one quantization parameter.

[0416] A method according to one embodiment may include a step of learning the at least one quantization parameter such that the first reconstruction error is reduced.

[0417] A method according to one embodiment may include adjusting at least one quantized parameter included in the first layer based on at least one quantized parameter for which learning has been completed.

[0418] A first layer according to one embodiment may include at least one weight matrix used in operations on the input.

[0419] According to one embodiment, a state in which at least one parameter included in the remaining layers excluding the first layer among the plurality of layers is fixed may include a state in which a floating point is maintained for at least one parameter included in the remaining layers excluding the first layer among the plurality of layers.

[0420] A method according to one embodiment may include a step of quantizing at least one parameter included in a second layer among a plurality of layers included in the neural network model, and measuring a second reconstruction error for the input while fixing at least one parameter included in the remaining layers except for the second layer among the plurality of layers.

[0421] A method according to one embodiment may include adjusting at least one quantized parameter included in the second layer based on the second reconstruction error.

[0422] A method according to one embodiment may include a step of quantizing at least one parameter included in a third layer among a plurality of layers included in the neural network model, and measuring a third reconstruction error for the input while fixing at least one parameter included in the remaining layers except for the third layer among the plurality of layers.

[0423] A method according to one embodiment may include adjusting at least one quantized parameter included in the third layer based on the third reconstruction error.

[0424] An electronic device according to one embodiment may include a memory storing a program or at least one instruction and at least one processor executing at least one instruction stored in the memory.

[0425] The electronic device can obtain input by having at least one processor execute at least one instruction stored in the memory according to one embodiment.

[0426] According to one embodiment, at least one processor executes at least one command stored in the memory, thereby allowing the electronic device to quantize at least one parameter included in a first layer among a plurality of layers included in the neural network model, and measure a first reconstruction error for the input while fixing at least one parameter included in the remaining layers excluding the first layer among the plurality of layers.

[0427] According to one embodiment, at least one processor executes at least one instruction stored in the memory, thereby allowing the electronic device to adjust at least one quantized parameter included in the first layer based on the first reconstruction error.

[0428] According to one embodiment, at least one processor executes at least one instruction stored in the memory, thereby allowing the electronic device to obtain a first output by applying the input to the neural network model before quantizing at least one parameter included in the first layer.

[0429] According to one embodiment, at least one processor executes at least one instruction stored in the memory, thereby allowing the electronic device to apply the input to the neural network model after quantizing at least one parameter included in the first layer to obtain a second output.

[0430] According to one embodiment, at least one processor may execute at least one instruction stored in the memory, thereby allowing the electronic device to measure the first reconstruction error based on the first output and the second output.

[0431] According to one embodiment, at least one processor may execute at least one instruction stored in the memory, thereby enabling the electronic device to obtain an objective function for measuring the first reconstruction error.

[0432] According to one embodiment, at least one processor may execute at least one instruction stored in the memory, thereby transforming the objective function such that a component independent of the quantization among at least one component included in the objective function is separated.

[0433] According to one embodiment, at least one processor executes at least one instruction stored in the memory, thereby allowing the electronic device to measure the first reconstruction error from the transformed objective function by applying a value calculated according to learning data of the neural network model to the quantization-independent component.

[0434] According to one embodiment, at least one processor may execute at least one instruction stored in the memory, thereby enabling the electronic device to determine at least one quantization parameter including a scaling factor and a zero point.

[0435] According to one embodiment, at least one processor executes at least one instruction stored in the memory, thereby allowing the electronic device to adjust at least one quantized parameter included in the first layer based on the at least one quantization parameter.

[0436] According to one embodiment, at least one processor may execute at least one instruction stored in the memory, thereby causing the electronic device to learn at least one quantization parameter such that the first reconstruction error is reduced.

[0437] According to one embodiment, at least one processor executes at least one command stored in the memory, thereby allowing the electronic device to adjust at least one quantized parameter included in the first layer based on at least one quantized parameter for which learning has been completed.

[0438] A first layer according to one embodiment may include at least one weight matrix used in operations on the input.

[0439] According to one embodiment, a state in which at least one parameter included in the remaining layers excluding the first layer among the plurality of layers is fixed may include a state in which a floating point is maintained for at least one parameter included in the remaining layers excluding the first layer among the plurality of layers.

[0440] According to one embodiment, at least one processor executes at least one command stored in the memory, thereby allowing the electronic device to quantize at least one parameter included in a second layer among a plurality of layers included in the neural network model, and measure a second reconstruction error for the input while keeping at least one parameter included in the remaining layers excluding the second layer among the plurality of layers fixed.

[0441] According to one embodiment, at least one processor executes at least one instruction stored in the memory, thereby allowing the electronic device to adjust at least one quantized parameter included in the second layer based on the second reconstruction error.

[0442] According to one embodiment, at least one processor executes at least one command stored in the memory, thereby allowing the electronic device to quantize at least one parameter included in a third layer among a plurality of layers included in the neural network model, and measure a third reconstruction error for the input while keeping at least one parameter included in the remaining layers except for the third layer among the plurality of layers fixed.

[0443] According to one embodiment, at least one processor executes at least one instruction stored in the memory, thereby allowing the electronic device to adjust at least one quantized parameter included in the third layer based on the third reconstruction error.

[0444] An operating method of an electronic device according to one embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., either singly or in combination. The program commands recorded on the medium may be those specially designed and configured for the present invention or may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands, such as ROMs, RAMs, and flash memories. Examples of program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc.

[0445] Additionally, the operating method of the electronic device according to the disclosed embodiments may be provided as a computer program product. The computer program product may be traded as a product between a seller and a buyer.

[0446] A computer program product may include a software program and a computer-readable storage medium on which the software program is stored. For example, a computer program product may include a product in the form of a software program (e.g., a downloadable app) distributed electronically by an electronic device manufacturer or through an electronic marketplace (e.g., Google Play Store, App Store). For electronic distribution, at least a portion of the software program may be stored on a storage medium or temporarily created. In this case, the storage medium may be a storage medium of a manufacturer's server, an electronic marketplace server, or a relay server that temporarily stores the software program.

[0447] In a system comprising a server and a client device, the computer program product may include a storage medium of the server or a storage medium of the client device. Alternatively, if a third device (e.g., a smartphone) exists that is communicatively connected to the server or the client device, the computer program product may include a storage medium of the third device. Alternatively, the computer program product may include a software program itself that is transmitted from the server to the client device or the third device, or from the third device to the client device.

[0448] In this case, one of the server, the client device, and the third device may execute the computer program product to perform the method according to the disclosed embodiments. Alternatively, two or more of the server, the client device, and the third device may execute the computer program product to perform the method according to the disclosed embodiments in a distributed manner.

[0449] For example, a server (e.g., a cloud server or an artificial intelligence server, etc.) may execute a computer program product stored on the server, thereby controlling a client device in communication with the server to perform a method according to the disclosed embodiments.

[0450] Although the embodiments have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concept of the present invention defined in the following claims also fall within the scope of the present invention.

Claims

1. In the method of quantizing a neural network model, Step of obtaining input; A step of measuring a first reconstruction error for the input, wherein at least one parameter included in a first layer among a plurality of layers included in the neural network model is quantized, and at least one parameter included in the remaining layers except for the first layer among the plurality of layers is fixed; and A method comprising: adjusting at least one quantized parameter included in the first layer based on the first reconstruction error.

2. In the first paragraph, the step of measuring the first reconstruction error comprises: A step of obtaining a first output by applying the input to the neural network model before quantizing at least one parameter included in the first layer; A step of obtaining a second output by applying the input to the neural network model after quantizing at least one parameter included in the first layer; and A method comprising: measuring the first reconstruction error based on the first output and the second output.

3. In the first or second paragraph, the step of measuring the first reconstruction error comprises: A step of obtaining an objective function for measuring the first reconstruction error; A step of transforming the objective function so that a component independent of the quantization is separated from at least one component included in the objective function; and A method comprising: a step of measuring the first reconstruction error from the transformed objective function by applying a value calculated according to the learning data of the neural network model to the component independent of the quantization; 4. In any one of paragraphs 1 to 3, A method in which at least one parameter included in the remaining layers excluding the first layer among the plurality of layers is fixed, including a state in which a floating point is maintained for at least one parameter included in the remaining layers excluding the first layer among the plurality of layers.

5. In any one of paragraphs 1 to 4, the method, A step of measuring a second reconstruction error for the input, wherein at least one parameter included in a second layer among a plurality of layers included in the neural network model is quantized, and at least one parameter included in the remaining layers except for the second layer among the plurality of layers is fixed; and A method further comprising: adjusting at least one quantized parameter included in the second layer based on the second reconstruction error.

6. In any one of paragraphs 1 to 5, the method, A step of measuring a third reconstruction error for the input while quantizing at least one parameter included in a third layer among a plurality of layers included in the neural network model and fixing at least one parameter included in the remaining layers except for the third layer among the plurality of layers; and A method further comprising: adjusting at least one quantized parameter included in the third layer based on the third reconstruction error.

7. A computer-readable recording medium having recorded thereon a program for performing the method of any one of clauses 1 to 6 on a computer.

8. A computer program stored in a recording medium to perform the method of any one of claims 1 to 6, and performed by a computing device.

9. In the electronic device (1400), a memory (1420) storing a program or at least one instruction; and At least one processor (1410) that executes at least one instruction stored in the memory (1420), The electronic device (1400) executes at least one instruction stored in the memory (1420) by the at least one processor (1410). Obtain input, Quantizing at least one parameter included in a first layer among a plurality of layers included in the neural network model, and measuring a first reconstruction error for the input while fixing at least one parameter included in the remaining layers except for the first layer among the plurality of layers, An electronic device that adjusts at least one quantized parameter included in the first layer based on the first reconstruction error.

10. In the 9th paragraph, the electronic device (1400) executes at least one instruction stored in the memory (1420) by the at least one processor (1410). Applying the input to the neural network model before quantizing at least one parameter included in the first layer to obtain a first output, Applying the input to the neural network model after quantizing at least one parameter included in the first layer to obtain a second output, An electronic device that measures the first reconstruction error based on the first output and the second output.

11. In the 9th or 10th paragraph, the electronic device (1400) executes at least one instruction stored in the memory (1420) by the at least one processor (1410). Obtain an objective function for measuring the first reconstruction error, Transforming the objective function so that a component independent of the quantization is separated from at least one component included in the objective function, An electronic device that measures the first reconstruction error from the transformed objective function by applying a value calculated according to the learning data of the neural network model to the component independent of the quantization.

12. In any one of the 9th to 11th paragraphs, the electronic device (1400) executes at least one instruction stored in the memory (1420) by the at least one processor (1410). Determine at least one quantization parameter including a scaling factor and a zero point, An electronic device that adjusts at least one quantized parameter included in the first layer based on at least one quantization parameter.

13. In any one of paragraphs 9 to 12, An electronic device, wherein a state in which at least one parameter included in the remaining layers excluding the first layer among the plurality of layers is fixed includes a state in which a floating point is maintained for at least one parameter included in the remaining layers excluding the first layer among the plurality of layers.

14. In any one of the 9th to 13th paragraphs, the electronic device (1400) executes at least one instruction stored in the memory (1420) by the at least one processor (1410). Quantizing at least one parameter included in a second layer among a plurality of layers included in the neural network model, and measuring a second reconstruction error for the input while keeping at least one parameter included in the remaining layers except for the second layer among the plurality of layers fixed, An electronic device that adjusts at least one quantized parameter included in the second layer based on the second reconstruction error.

15. In any one of the 9th to 14th paragraphs, the electronic device (1400) executes at least one instruction stored in the memory (1420) by the at least one processor (1410). Quantizing at least one parameter included in a third layer among a plurality of layers included in the neural network model, and measuring a third reconstruction error for the input while fixing at least one parameter included in the remaining layers except for the third layer among the plurality of layers, An electronic device that adjusts at least one quantized parameter included in the third layer based on the third reconstruction error.

Images