Electronic apparatus and control method therefor

Abstract

Provided are an electronic apparatus and a control method therefor. The electronic apparatus comprises: a memory for storing at least one instruction; and at least one processor. The at least one processor executes the at least one instruction to: obtain a first quantization matrix expressed as a product of a first vector and a second vector for each of a plurality of layers on the basis of a weight for each of the plurality of layers included in a neural network model; obtain a second quantization matrix for each of the plurality of layers by multiplying each of first vectors of first quantization matrices corresponding to the plurality of layers by a second vector of a first quantization matrix corresponding to a previous layer; and quantize the neural network model by using the second quantization matrix.

Description

Electronic device and method of controlling the same

[0001] The present disclosure relates to an electronic device and a control method thereof, and more particularly, to an electronic device for quantizing weights of a neural network model and a control method thereof.

[0002] To efficiently run large neural network models such as LLM (Large Language Model) on servers or edge devices (e.g., TVs, mobile terminals, etc.), compression of the neural network model is essential.

[0003] Quantizing neural network weights has recently been developed as a means of compressing neural network models. Quantization is a model compression technique that converts trained neural network models using high-precision floating-point values ​​(e.g., FP32) into fixed-point models, thereby reducing memory footprint and latency during inference.

[0004] In particular, quantization techniques can be broadly categorized into Quantization Aware Training (QAT) and Post-Training Quantization (PTQ). For large-scale neural network models, QAT, which requires a large amount of data and computing resources, is difficult to apply. Therefore, PTQ, which allows for quantization at a low cost, is primarily applied.

[0005] However, PTQ suffers from a significant loss of accuracy in the compressed neural network model compared to QAT. Therefore, when performing PTQ, the search for a quantization method with minimal accuracy loss is needed.

[0006] According to one embodiment of the present disclosure, an electronic device includes a memory storing at least one instruction; and at least one processor, wherein the at least one processor obtains a first quantization matrix expressed as a product of a first vector and a second vector for each of a plurality of layers based on weights for each of a plurality of layers included in a neural network model by executing the at least one instruction, and obtains a second quantization matrix for each of the plurality of layers by multiplying each of the first vectors of the first quantization matrices corresponding to the plurality of layers by the second vector of the first quantization matrix corresponding to a previous layer, and quantizes the neural network model using the second quantization matrix.

[0007] The at least one processor may, if there is correction data for correcting the weights of the neural network model, input the correction data into the neural network model to obtain intermediate result values ​​for each of the plurality of layers, calculate a sample covariance matrix through the intermediate result values ​​for each of the plurality of layers to estimate a Hessian matrix, and fine-tune the weights of the neural network model using the estimated Hessian matrix.

[0008] The at least one processor may perform model inversion to generate synthetic data if the correction data does not exist, and may fine-tune the weights of the neural network model using the generated synthetic data.

[0009] The at least one processor can obtain the first quantization matrix by calculating the first vector and the second vector such that the difference between the weights included in the plurality of layers and the quantized weights is minimized.

[0010] The first vector may be a vector in the in-channel direction, and the second vector may be a vector in the out-channel direction.

[0011] The above-described one processor obtains the second quantization matrix by multiplying each of the first vectors of the first quantization matrices corresponding to the first layer by the second vector of the first quantization matrix corresponding to the previous layer, when the previous layer is a preset layer, and the preset layer may be a layer including an activation function satisfying scale equivariance or a layer including an affine transform or a scale transform.

[0012] The above quantization may be post-training quantization (PTQ).

[0013] According to one embodiment of the present disclosure, a control method of an electronic device includes: obtaining a first quantization matrix expressed as a product of a first vector and a second vector for each of a plurality of layers based on weights for each of a plurality of layers included in a neural network model; obtaining a second quantization matrix for each of the plurality of layers by multiplying each of the first vectors of the first quantization matrices corresponding to the plurality of layers by the second vector of the first quantization matrix corresponding to a previous layer; and quantizing the neural network model using the second quantization matrix.

[0014] The above control method may include, if there is correction data for correcting the weights of the neural network model, a step of inputting the correction data into the neural network model to obtain intermediate result values ​​for each of the plurality of layers; a step of calculating a sample covariance matrix through the intermediate result values ​​for each of the plurality of layers to estimate a Hessian matrix; and a step of fine-tuning the weights of the neural network model using the estimated Hessian matrix.

[0015] The above control method may include a step of generating synthetic data by performing model inversion if the correction data does not exist; and a step of fine-tuning the weights of the neural network model using the generated synthetic data.

[0016] The step of obtaining the first quantization matrix may obtain the first quantization matrix by calculating the first vector and the second vector such that the difference between the weights included in the plurality of layers and the quantized weights is minimized.

[0017] The first vector may be a vector in the in-channel direction, and the second vector may be a vector in the out-channel direction.

[0018] The step of obtaining the second quantization matrix comprises, when the previous layer is a preset layer, obtaining the second quantization matrix by multiplying each of the first vectors of the first quantization matrices corresponding to the first layer by the second vector of the first quantization matrix corresponding to the previous layer, and the preset layer may be a layer including an activation function that satisfies scale equivariance or a layer including an affine transform or a scale transform.

[0019] The above quantization may be post-training quantization (PTQ).

[0020] FIG. 1 is a block diagram illustrating a configuration of an electronic device according to one embodiment of the present disclosure;

[0021] FIG. 2 is a block diagram illustrating a configuration for quantizing weights of a neural network model according to one embodiment of the present disclosure;

[0022] FIG. 3A is a diagram for explaining a first quantization matrix according to one embodiment of the present disclosure;

[0023] FIG. 3b is a diagram for explaining an algorithm for obtaining a first quantization matrix expressed as a product of a first vector and a second vector, according to one embodiment of the present disclosure;

[0024] FIG. 4 is a diagram illustrating an algorithm for fine-tuning weights of a neural network model according to one embodiment of the present disclosure;

[0025] FIGS. 5A to 5C are diagrams for explaining a method of obtaining a second quantization matrix according to various embodiments of the present disclosure, and

[0026] FIG. 6 is a flowchart illustrating a control method of an electronic device for quantizing weights of a neural network model according to one embodiment of the present disclosure.

[0027] The present embodiments may be modified and have various embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope to specific embodiments, but should be understood to encompass various modifications, equivalents, and / or alternatives of the embodiments of the present disclosure. In connection with the description of the drawings, similar reference numerals may be used for similar components.

[0028] In describing the present disclosure, if it is determined that a specific description of a related known function or configuration may unnecessarily obscure the gist of the present disclosure, a detailed description thereof will be omitted.

[0029] Additionally, the following embodiments may be modified in various other forms, and the scope of the technical concepts of the present disclosure is not limited to the following embodiments. Rather, these embodiments are provided to further faithfully and completely convey the technical concepts of the present disclosure to those skilled in the art.

[0030] The terminology used in this disclosure is for the purpose of describing specific embodiments only and is not intended to limit the scope of the rights. Singular expressions include plural expressions unless the context clearly dictates otherwise.

[0031] In this disclosure, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a corresponding feature (e.g., a component such as a number, function, operation, or part), and do not exclude the presence of additional features.

[0032] In this disclosure, expressions such as “A or B,” “at least one of A and / or B,” or “one or more of A or / and B” can include all possible combinations of the listed items. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” can all refer to (1) including at least one A, (2) including at least one B, or (3) including both at least one A and at least one B.

[0033] The expressions “first,” “second,” “first,” or “second,” etc., used in this disclosure can describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0034] When it is said that a component (e.g., a first component) is “(operatively or communicatively) coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that said component may be directly coupled to said other component, or may be coupled via another component (e.g., a third component).

[0035] On the other hand, when it is said that a component (e.g., a first component) is "directly connected" or "directly connected" to another component (e.g., a second component), it can be understood that no other component (e.g., a third component) exists between said component and said other component.

[0036] The expression "configured to" as used in the present disclosure may be used interchangeably with, for example, "suitable for," "having the capacity to," "designed to," "adapted to," "made to," or "capable of." The term "configured to" may not necessarily mean only "specifically designed to" in terms of hardware.

[0037] Instead, in some contexts, the phrase "a device configured to" may mean that the device, in conjunction with other devices or components, is "capable of" performing A, B, and C. For example, the phrase "a processor configured (or set) to perform A, B, and C" may refer to a dedicated processor (e.g., an embedded processor) for performing those operations, or a general-purpose processor (e.g., a CPU or application processor) that can perform those operations by executing one or more software programs stored in a memory device.

[0038] In the embodiments, a 'module' or 'part' performs at least one function or operation, and may be implemented as hardware or software, or as a combination of hardware and software. Furthermore, a plurality of 'modules' or 'parts' may be integrated into at least one module and implemented as at least one processor, except for a 'module' or 'part' that needs to be implemented as a specific hardware.

[0039] Meanwhile, the various elements and areas in the drawings are schematically drawn. Therefore, the technical concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.

[0040] Hereinafter, with reference to the attached drawings, embodiments according to the present disclosure will be described in detail so that a person having ordinary knowledge in the technical field to which the present disclosure pertains can easily implement the present disclosure.

[0041]

[0042] FIG. 1 is a diagram illustrating the configuration of an electronic device according to one embodiment of the present disclosure. As illustrated in FIG. 1, the electronic device (100) may include a memory (110) and at least one processor (120). In this case, the electronic device (100) may be a server. However, this is merely an example, and the electronic device (100) may also be a user terminal. Meanwhile, the configuration of the electronic device (100) is not limited to the configuration illustrated in FIG. 1, and it goes without saying that additional configurations apparent to those skilled in the art may be added.

[0043] The memory (110) can store an operating system (OS) for controlling the overall operation of components of the electronic device (100) and at least one instruction or data related to the components of the electronic device (100). In addition, the memory (110) can store data required for a module for controlling the operation of the electronic device (100) to perform various operations.

[0044] In particular, the memory (110) can store information (e.g., information about weights) regarding a neural network model for performing a specific function. While the neural network model according to one embodiment of the present disclosure may be an LLM, this is merely an example and may be implemented using other neural network models (e.g., a neural network model for object recognition, etc.).

[0045] The configuration for controlling the operation of the electronic device (100) to quantize the weights of the neural network model may include a first quantization matrix acquisition module (210), a weight fine-tuning module (220), a second quantization matrix acquisition module (230), and a quantization module (240), as illustrated in FIG. 2.

[0046] Meanwhile, the memory (110) may include non-volatile memory capable of maintaining stored information even when power is cut off, and volatile memory requiring a continuous power supply to maintain stored information. A configuration for controlling the operation of quantizing the parameters of a neural network model may be stored in the non-volatile memory.

[0047] Additionally, the memory (110) may include learning data for learning a neural network model, calibration data for fine-tuning the weights of the neural network model, and information about quantization options.

[0048] At least one processor (120) can control the electronic device (100) according to at least one instruction stored in the memory (110).

[0049] In particular, at least one processor (120) may include multiple processors. Specifically, the multiple processors may include one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an Accelerated Processing Unit (APU), a Many Integrated Core (MIC), a Digital Signal Processor (DSP), a Neural Processing Unit (NPU), a hardware accelerator, or a machine learning accelerator. The one or more processors may control one or any combination of other components of the electronic device and perform operations related to communication or data processing. The one or more processors may execute one or more programs or instructions stored in a memory. For example, the multiple processors may perform a method according to an embodiment of the present disclosure by executing one or more instructions stored in a memory.

[0050] When a method according to an embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one processor or by a plurality of processors. That is, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor (e.g., a general-purpose processor) and the third operation may be performed by the second processor (e.g., an artificial intelligence-dedicated processor).

[0051] At least one processor (120) may be implemented as one or more multi-core processors including multiple cores (e.g., homogeneous multi-cores or heterogeneous multi-cores). When at least one processor (120) is implemented as a multi-core processor, each of the multiple cores included in the multi-core processor may include internal processor memory, such as cache memory or on-chip memory, and a common cache shared by the multiple cores may be included in the multi-core processor. In addition, each of the multiple cores (or some of the multiple cores) included in the multi-core processor may independently read and execute a program instruction for implementing a method according to an embodiment of the present disclosure, or all (or some) of the multiple cores may be linked to read and execute a program instruction for implementing a method according to an embodiment of the present disclosure.

[0052] When a method according to an embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one core among a plurality of cores included in a multi-core processor, or may be performed by a plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by a first core included in the multi-core processor, or the first operation and the second operation may be performed by a first core included in the multi-core processor, and the third operation may be performed by a second core included in the multi-core processor.

[0053] In embodiments of the present disclosure, a processor may mean a system on a chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, a GPU, an APU, a MIC, a DSP, an NPU, a hardware accelerator, or a machine learning accelerator, but embodiments of the present disclosure are not limited thereto.

[0054] In particular, at least one processor (120) obtains a first quantization matrix expressed as a product of a first vector and a second vector for each of a plurality of layers based on weights for each of a plurality of layers included in a neural network model by executing at least one instruction stored in a memory (110). At this time, the neural network model may be a pre-trained neural network model, and the quantization of the neural network model may be post-training quantization (PTQ). In addition, the weight may be a configuration included in the neural network model, and may be a value applied to input data for inference. At this time, the weight may be called a parameter, etc.

[0055] In particular, at least one processor (120) can obtain a first quantization matrix by generating a first vector and a second vector that minimize the difference between the weights included in the plurality of layers and the quantized weights. At this time, the first vector may be a vector in the in-channel direction, and the second vector may be a vector in the out-channel direction.

[0056] At least one processor (120) can determine whether there is correction data for correcting the weights of the neural network model. If there is correction data for correcting the weights of the neural network model, the at least one processor (120) can input the correction data into the neural network model to obtain intermediate result values ​​for each of a plurality of layers, calculate a sample covariance matrix using the intermediate result values ​​for each of the plurality of layers to estimate a Hessian matrix, and fine-tune the weights of the neural network model using the estimated Hessian matrix. If there is no correction data, the at least one processor (120) can perform model inversion to generate synthetic data, and fine-tune the weights of the neural network model using the generated synthetic data.

[0057] At least one processor (120) obtains a second quantization matrix for each of the plurality of layers by multiplying each of the first vectors of the first quantization matrices corresponding to the plurality of layers by the second vector of the first quantization matrix corresponding to the previous layer. At this time, if the previous layer is a preset layer, the at least one processor (120) may obtain a second quantization matrix by multiplying each of the first vectors of the first quantization matrices corresponding to the first layer by the second vector of the first quantization matrix corresponding to the previous layer. At this time, the preset layer may be a layer including an activation function that satisfies scale equivariance, or a layer including an affine transform or a scale transform.

[0058] And, at least one processor (120) can quantize the neural network model using the second quantization matrix.

[0059] Hereinafter, the present disclosure will be described in more detail with reference to FIGS. 2 to 5c.

[0060] FIG. 2 is a block diagram illustrating a configuration for quantizing weights of a neural network model according to one embodiment of the present disclosure. The electronic device (100) may include a first quantization matrix acquisition module (210), a weight fine-tuning module (220), a second quantization matrix acquisition module (230), and a quantization module (240).

[0061] The first quantization matrix acquisition module (210) can acquire a neural network model including multiple layers. At this time, the neural network model may be a pre-learned neural network model.

[0062] In particular, the first quantization matrix acquisition module (210) acquires a first quantization matrix expressed as a product of a first vector and a second vector for each of a plurality of layers based on weights for each of a plurality of layers included in the neural network model.

[0063] Specifically, in the past, in order to quantize a neural network model, quantization was performed using a single vector (e.g., a scale for each out-channel). However, according to one embodiment of the present disclosure, the first quantization matrix may include not only the existing scaling factor α but also a new scaling factor ζ. In this case, ζ may be expressed as a first vector in the in-channel direction, and α may be expressed as a second vector in the out-channel direction.

[0064] Specifically, the first quantization matrix acquisition module (210) may use the following mathematical formula to obtain the first quantization matrix (S) expressed as the product of the first vector (Z) and the second vector (A) using the weight (W) included in the neural network model.

[0065]

[0066]

[0067]

[0068] At this time, and can be Hadamard division and Hadamard product, respectively. n and p can represent the upper and lower bounds of the quantized weights, respectively.

[0069] That is, as described above, the first quantization matrix (S) for obtaining the quantized weight (Wq) can be expressed as the product of the first vector (Z) and the second vector (A).

[0070] For example, as illustrated in FIG. 3a, when the first weight matrix (310) (W') of the first layer is quantized to obtain the second weight matrix (320) (Wq), the quantization matrix (S) multiplied by the first weight matrix (310) can be expressed as the product of a first vector (ζ) in the in-channel direction including a first scaling factor (ζ0, ζ1) and a second vector (α) in the out-channel direction including a second scaling factor (α1, α2, α3).

[0071] In the past, quantization was performed by obtaining a quantization matrix using only the second vector (α) in the out-channel direction, but in the present disclosure, by calculating not only the second vector (α) in the out-channel direction but also the first vector (ζ) in the in-channel direction, there is an effect of further increasing the accuracy of the quantized neural network model.

[0072] In particular, the first quantization matrix acquisition module (210) can obtain the first quantization matrix by calculating the first vector and the second vector so that the difference between the weights included in the multiple layers and the quantized weights is minimized. For example, the first quantization matrix acquisition module (210) can calculate the first vector (ζ) and the second vector (α) using an algorithm as illustrated in FIG. 3b.

[0073] Specifically, the first quantization matrix acquisition module (210) initializes the first vector (ζ) and the second vector (α), as illustrated in line 2 of FIG. 3b. At this time, the first vector (ζ) may be initialized as an all-1 vector, and the second vector (α) may be initialized by applying Min-Max quantization as in the following formula.

[0074]

[0075] And, the first quantization matrix acquisition module (210) can repeatedly update the first vector (ζ) and the second vector (α) until convergence (i.e., when the difference between the weight and the quantized weight is minimized) using a least square solution, as in line 3-8 of FIG. 3b.

[0076] In particular, as shown in lines 12-14 of FIG. 3b, since the column-wise least square method provides loop-level parallelism, the first quantization matrix acquisition module (210) is capable of parallel processing through batch matrix-matrix multiplication (i.e., bmm).

[0077] As another example, the first quantization matrix acquisition module (210) obtains a weight (W) and a quantized weight (W q The first vector (ζ) and the second vector (α) can be derived through gradient-based optimization so that the difference (or loss) between them is minimized.

[0078] The weight fine-tuning module (220) can perform fine-tuning on the weights included in the neural network model using correction data.

[0079] At this time, the weight fine-tuning module (220) can identify whether calibration data is stored in the memory (110). If calibration data does not exist in the memory (110), the weight fine-tuning module (220) can perform model inversion on the neural network model to generate synthetic data. In addition, the memory (110) can use the synthetic data as calibration data.

[0080] In addition, the weight fine-tuning module (220) can input correction data into the neural network model to obtain intermediate result values ​​for each of multiple layers. The weight fine-tuning module (220) can estimate a Hessian matrix by calculating a sample covariance matrix using the intermediate result values ​​for each of multiple layers. The weight fine-tuning module (220) can fine-tune the weights of the neural network model using the estimated Hessian matrix.

[0081] For example, the weight fine-tuning module (220) can perform fine-tuning of the weights of each layer using an algorithm as illustrated in FIG. 4. Specifically, FIG. 4 is an algorithm for explaining a method of fine-tuning weights when correction data is given. At this time, a quantized neural network model can be initialized. At this time, as illustrated in lines 2-3 of FIG. 4, in order to minimize degradation of the loss, the weight fine-tuning module (220) can use a Hessian matrix for the second vector (α). In addition, as illustrated in lines 4-9 of FIG. 4, the weight fine-tuning module (220) can further minimize the loss using optimized product tree quantization (OPTQ).

[0082] In particular, the weight fine-tuning module (220) can perform fine-tuning not only on the weights of the neural network model, but also on the scaling factors included in the first vector (ζ) and the second vector (α). For example, as illustrated in lines 8-9 of FIG. 4, fine-tuning can be performed on the scaling factor (ζi) of the first vector (ζ).

[0083] The second quantization matrix acquisition module (230) can acquire a second quantization matrix for each of the plurality of layers by multiplying each of the first vectors of the first quantization matrices corresponding to the plurality of layers by the second vector of the first quantization matrix corresponding to the previous layer. At this time, the operation of multiplying each of the first vectors of the first quantization matrices corresponding to the plurality of layers by the second vector of the first quantization matrix corresponding to the previous layer may be called a folding operation.

[0084] Specifically, as illustrated in FIG. 5A, the scaling factor of the first vector (ζ) may be multiplied by the weights (510-1) of the in-channel, and the scaling factor of the second vector (α) may be multiplied by the weights (520-2) of the out-channel. In this way, when multiple vectors are multiplied by the weights included in one layer, the amount of calculation may increase, resulting in a problem of reduced computational performance.

[0085] According to one embodiment of the present disclosure, in order to solve the problem as described above, a first vector (ζ) multiplied by the weights (510-1) of the in-channel n ) as the second vector (α) of the previous layer n-1 ) can be multiplied. By this, while maintaining high accuracy through the quantization matrix including the first vector and the second vector, the amount of calculation and parameters are the same as before through the folding operation, so that a fast calculation speed can be maintained.

[0086] According to one embodiment of the present disclosure, the second quantization matrix acquisition module (230) may perform the folding operation described above when the previous layer is a preset layer. For example, the preset layer may be a layer including an activation function that satisfies scale equivariance, or a layer including an affine transform or a scale transform.

[0087] In one embodiment, as illustrated in FIG. 5b, if the previous layer is a layer including an activation function that satisfies scale equivariance (i.e., a function that satisfies f(ax) = af(x) for any constant a, e.g., a ReLU (Rectified Linear Unit) function), the second quantization matrix acquisition module (230) may perform a folding operation as in Equation 5 below.

[0088]

[0089] That is, the second quantization matrix acquisition module (230) obtains the first vector ( ) and the second vector of the previous layer (520-1) ( ) is multiplied by the second quantization matrix of the previous layer (520-1). ) can be obtained.

[0090] In one embodiment, as illustrated in FIG. 5c, when the previous layer is a layer (530-1) including an affine transform, the second quantization matrix acquisition module (230) can perform a folding operation as in mathematical expression 6 below.

[0091]

[0092] That is, the second quantization matrix acquisition module (230) obtains the first vector ( ) and the second vector of the previous layer (530-1) ( ) is multiplied by the second quantization matrix of the previous layer (530-1). ) can be obtained.

[0093] The second quantization matrix acquisition module (230) can acquire the second quantization matrix for multiple layers through the method described above.

[0094] The quantization module (240) can perform a quantization operation on the neural network model through the second quantization matrix. At this time, the quantization operation refers to the process of converting a weight expressed in a high-precision unit into a weight of relatively low precision. In other words, the quantization operation refers to the process of converting a weight expressed in a first bit range into a weight expressed in a second bit range smaller than the first bit. At this time, the quantization operation may be uniform quantization or non-uniform quantization.

[0095] Meanwhile, in one embodiment of the present disclosure, it has been described that the electronic device (100) directly performs quantization through the second quantization matrix, but this is only one embodiment, and it is of course possible to transmit the second quantization matrix to another electronic device and have the neural network model quantized by the other electronic device.

[0096]

[0097] FIG. 6 is a flowchart illustrating a control method of an electronic device for quantizing weights of a neural network model according to one embodiment of the present disclosure.

[0098] First, the electronic device (100) obtains a first quantization matrix expressed as a product of a first vector and a second vector for each of a plurality of layers based on weights for each of a plurality of layers included in the neural network model (S610). Specifically, the electronic device (100) can obtain the first quantization matrix by calculating the first vector and the second vector such that the difference between the weights included in the plurality of layers and the quantized weights is minimized. At this time, the first vector may be a vector in the in-channel direction, and the second vector may be a vector in the out-channel direction.

[0099] The electronic device (100) can identify whether correction data exists (S620).

[0100] If correction data for correcting the weights of the neural network model exists (S620-Y), the electronic device (100) can fine-tune the weights of the neural network model using the correction data (S630). If correction data for correcting the weights of the neural network model does not exist (S620-N), the electronic device (100) can perform model inversion to generate synthetic data (S640). Then, the electronic device (100) can fine-tune the weights of the neural network model using the generated synthetic data as correction data (S630).

[0101] The electronic device (100) obtains a second quantization matrix for each of the plurality of layers by multiplying each of the first vectors of the first quantization matrices corresponding to the plurality of layers by the second vector of the first quantization matrix corresponding to the previous layer (S650). At this time, if the previous layer is a preset layer, the electronic device (100) can obtain the second quantization matrix by multiplying each of the first vectors of the first quantization matrices corresponding to the first layer by the second vector of the first quantization matrix corresponding to the previous layer. The preset layer may be a layer including an activation function that satisfies scale equivariance, or a layer including an affine transform or a scale transform.

[0102] The electronic device (100) quantizes the neural network model using the second quantization matrix (S660). At this time, the quantization may be post-training quantization (PTQ) that quantizes a pre-trained neural network model.

[0103] As described above, by calculating the first vector and the second vector and performing quantization, quantization can be performed with less error than the existing method that uses only the second vector. Accordingly, the electronic device (100) can perform a more accurate and faster inference operation using a quantized neural network model with a small error. In addition, since a neural network model with a higher compression ratio while maintaining the same accuracy can be implemented, a large neural network model can be efficiently operated. Alternatively, since quantization with a lower bit rate while maintaining the same accuracy becomes possible, the design of an efficient neural network accelerator (e.g., NPU) that uses a lower bit precision can be enabled.

[0104]

[0105] The artificial intelligence-related functions according to the present disclosure are operated through the processor and memory of the electronic device (100).

[0106] The processor may be composed of one or more processors. In this case, the one or more processors may include at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an NPU (Neural Processing Unit), but is not limited to the examples of the processors described above.

[0107] CPUs are general-purpose processors capable of performing not only general calculations but also artificial intelligence calculations. Their multi-layered cache structure allows for the efficient execution of complex programs. CPUs are advantageous for serial processing, enabling organic linking of previous and subsequent calculation results through sequential calculations. General-purpose processors are not limited to the examples described above, except where specifically identified as CPUs.

[0108] A GPU is a processor designed for large-scale computations, such as floating-point operations used in graphics processing. It integrates a large number of cores to perform large-scale computations in parallel. In particular, GPUs may be advantageous over CPUs in parallel processing methods, such as convolution operations. Furthermore, GPUs can be used as coprocessors to supplement the functions of CPUs. Processors for large-scale computations are not limited to the examples described above, except in cases where they are specifically referred to as GPUs.

[0109] An NPU is a processor specialized in artificial intelligence computation using artificial neural networks, and each layer of the artificial neural network can be implemented in hardware (e.g., silicon). Since NPUs are designed specifically according to the company's specifications, they have less freedom than CPUs or GPUs, but can efficiently process the AI ​​computations requested by the company. Meanwhile, as a processor specialized in AI computation, an NPU can be implemented in various forms, such as a Tensor Processing Unit (TPU), an Intelligence Processing Unit (IPU), or a Vision Processing Unit (VPU). Except as specifically designated as an NPU, an AI processor is not limited to the examples described above.

[0110] Additionally, one or more processors may be implemented as a System on Chip (SoC). In this case, in addition to one or more processors, the SoC may further include memory and a network interface, such as a bus, for data communication between the processor and the memory.

[0111] When a System on Chip (SoC) included in an electronic device includes multiple processors, the electronic device may use some of the multiple processors to perform operations related to artificial intelligence (e.g., operations related to learning or inference of an artificial intelligence model). For example, the electronic device may use at least one of a GPU, NPU, VPU, TPU, or hardware accelerator specialized in artificial intelligence operations, such as convolution operations or matrix multiplication operations, among the multiple processors to perform operations related to artificial intelligence. However, this is merely an example, and it is of course possible to process operations related to artificial intelligence using a CPU or other general-purpose processor.

[0112] Additionally, electronic devices can perform computations related to artificial intelligence (AI) functions by utilizing multiple cores (e.g., dual cores, quad cores, etc.) contained within a single processor. In particular, electronic devices can perform AI operations, such as convolution operations and matrix multiplication operations, in parallel by utilizing the multiple cores contained within the processor.

[0113] One or more processors are controlled to process input data according to predefined operating rules or artificial intelligence models stored in memory. The predefined operating rules or artificial intelligence models are characterized by being created through learning.

[0114] Here, "created through learning" means that a predefined set of behavioral rules or an AI model with desired characteristics is created by applying a learning algorithm to a large number of learning data. This learning may be performed on the device itself, where the AI ​​according to the present disclosure is implemented, or through a separate server / system.

[0115] An artificial intelligence model may be composed of multiple neural network layers. At least one layer has at least one weight value and performs its operation through the operation result of the previous layer and at least one defined operation. Examples of neural networks include a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), deep Q-networks, and a transformer. The neural networks in the present disclosure are not limited to the above-described examples unless otherwise specified.

[0116] A learning algorithm is a method for training a target device (e.g., a robot) using a large amount of learning data, enabling the target device to make decisions or predictions on its own. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. Unless otherwise specified, the learning algorithms in this disclosure are not limited to the aforementioned examples.

[0117] Meanwhile, the methods according to various embodiments of the present disclosure may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0118] The methods according to various embodiments of the present disclosure may be implemented as software including commands stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The device is a device that can call commands stored in the storage medium and operate according to the called commands, and may include an electronic device (e.g., a TV) according to the disclosed embodiments.

[0119] Meanwhile, a device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory storage medium" simply means a tangible device that does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is permanently stored in the storage medium and cases where data is temporarily stored. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0120] When the above instruction is executed by the processor, the processor may perform the function corresponding to the instruction directly or by using other components under the control of the processor. The instruction may include code generated or executed by a compiler or interpreter.

[0121] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person having ordinary skill in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. In electronic devices, memory that stores at least one instruction; and Contains at least one processor, The at least one processor, by executing the at least one instruction, A first quantization matrix expressed as a product of a first vector and a second vector for each of the plurality of layers is obtained based on weights for each of the plurality of layers included in the neural network model. Each of the first vectors of the first quantization matrices corresponding to the plurality of layers is multiplied by the second vector of the first quantization matrix corresponding to the previous layer to obtain a second quantization matrix for each of the plurality of layers. An electronic device for quantizing the neural network model using the second quantization matrix.

2. In paragraph 1, At least one processor of the above, If there is correction data for correcting the weights of the above neural network model, the correction data is input into the above neural network model to obtain intermediate result values ​​for each of the multiple layers, The sample covariance matrix is ​​calculated through the intermediate result values ​​of each of the above multiple layers, and the Hessian matrix is ​​estimated. An electronic device for fine-tuning the weights of the neural network model using the estimated Hessian matrix.

3. In paragraph 2, At least one processor of the above, If the above correction data does not exist, model inversion is performed to generate synthetic data. An electronic device that fine-tunes the weights of the neural network model using the generated synthetic data.

4. In paragraph 1, At least one processor of the above, An electronic device that obtains the first quantization matrix by calculating the first vector and the second vector such that the difference between the weights included in the plurality of layers and the quantized weights is minimized.

5. In paragraph 1, The above first vector is a vector in the in-channel direction, An electronic device wherein the second vector is a vector in the out-channel direction.

6. In paragraph 1, As mentioned above, one processor, If the previous layer is a preset layer, the second quantization matrix is ​​obtained by multiplying each of the first vectors of the first quantization matrices corresponding to the first layer by the second vector of the first quantization matrix corresponding to the previous layer. The above preset layers are, An electronic device, comprising a layer including an activation function satisfying scale equivariance, or a layer including an affine transform or a scale transform.

7. In paragraph 1, The above quantization is, An electronic device characterized by post-training quantization (PTQ).

8. In a method for controlling an electronic device, A step of obtaining a first quantization matrix expressed as a product of a first vector and a second vector for each of the plurality of layers based on weights for each of the plurality of layers included in the neural network model; A step of obtaining a second quantization matrix for each of the plurality of layers by multiplying each of the first vectors of the first quantization matrices corresponding to the plurality of layers by the second vector of the first quantization matrix corresponding to the previous layer; and A control method comprising: a step of quantizing the neural network model using the second quantization matrix.

9. In paragraph 8, The above control method is, If there is correction data for correcting the weights of the neural network model, a step of inputting the correction data into the neural network model to obtain intermediate result values ​​for each of the plurality of layers; A step of estimating a Hessian matrix by calculating a sample covariance matrix through the intermediate result values ​​of each of the above multiple layers; and A control method comprising: a step of fine-tuning weights of the neural network model using the estimated Hessian matrix.

10. In paragraph 9, The above control method is, If the above correction data does not exist, a step of performing model inversion to generate synthetic data; and A control method comprising: a step of fine-tuning the weights of the neural network model using the generated synthetic data.

11. In paragraph 8, The step of obtaining the first quantization matrix is: A control method for obtaining the first quantization matrix by calculating the first vector and the second vector so that the difference between the weights included in the plurality of layers and the quantized weights is minimized.

12. In paragraph 8, The above first vector is a vector in the in-channel direction, A control method wherein the second vector is a vector in the out-channel direction.

13. In paragraph 8, The step of obtaining the second quantization matrix is: If the previous layer is a preset layer, the second quantization matrix is ​​obtained by multiplying each of the first vectors of the first quantization matrices corresponding to the first layer by the second vector of the first quantization matrix corresponding to the previous layer. The above preset layers are, A control method, wherein the layer includes an activation function satisfying scale equivariance, or a layer including an affine transform or a scale transform.

14. In paragraph 8, The above quantization is, A control method characterized by post-training quantization (PTQ).

Images