Electronic device for quantizing neural network model and control method thereof

Abstract

An electronic device is disclosed. The electronic device comprises: a memory storing first weight data including a plurality of weights for each of a plurality of layers of a neural network model; and a processor connected to the memory and controlling the electronic device. The processor obtains second weight data by quantizing the first weight data using a plurality of quantization intervals, trains the neural network model such that smallest target quantization intervals among the plurality of quantization intervals converge on a preset value for each of the plurality of layers, obtains third weight data by treating at least one quantization interval satisfying a preset condition among the target quantization intervals for each of the plurality of layers as 0 on the basis of a training result of the neural network model, and obtains a quantized neural network model on the basis of the third weight.

Description

Electronic device for quantizing neural network models and its control method

[0001] The present disclosure relates to an electronic device and a control method thereof, and more particularly, to an electronic device and a control method thereof for quantizing a neural network model by determining an optimal bit width for weights per layer of the neural network model.

[0002] Recently, technologies for lightweighting neural network models have been advancing, and these advancements are particularly driven by ongoing efforts to efficiently implement neural network models on-device. In particular, technologies for quantizing neural network model weight data, typically expressed in high-precision units, into relatively low-precision weight data have emerged as a key challenge: improving computational efficiency while maintaining the accuracy required by neural network models.

[0003] For example, when weight data expressed in a 32-bit floating point (FP32) format is quantized and converted to weight data expressed in 8 bits, a quantization error may occur between the weight data before and after quantization, which may lead to a decrease in the accuracy of the neural network model. In other words, while a specific layer of the neural network model may exhibit sufficient accuracy based on weight data expressed in 8 bits, it may be difficult for other layers to exhibit sufficient accuracy based on weight data expressed in 8 bits. Therefore, there is a need for a technology to quantize weight data by considering the accuracy of each layer included in the neural network model.

[0004] Meanwhile, conventional techniques for quantizing weight data exist, such as those based on single precision, which quantize weight data and perform computations based on neural network models. While single precision techniques can reduce quantization errors by providing sufficient bit-width across all layers of a neural network model, they significantly increase computational complexity and may limit optimization in terms of memory resources.

[0005] Meanwhile, in the case of a technology that performs quantization and calculation according to a neural network model for weight data based on mixed-precision as a conventional technology, hardware resources can be minimized because quantization and calculation are performed based on the optimal bit width for each layer of the neural network model, but there is a limitation in that the time and resource consumption for searching for the optimal bit width for each layer may be large.

[0006] An electronic device according to one or more embodiments of the present disclosure comprises: a memory storing first weight data including a plurality of weights for each of a plurality of layers of a neural network model; and a processor connected to the memory and controlling the electronic device, wherein the processor quantizes the first weight data using a plurality of quantization intervals to obtain second weight data, trains the neural network model such that smallest target quantization intervals among the plurality of quantization intervals for each of the plurality of layers converge to a preset value, and, based on a result of training the neural network model, treats at least one quantization interval satisfying a preset condition among the target quantization intervals for each of the plurality of layers as 0, thereby obtaining third weight data, and obtains a quantized neural network model based on the third weight.

[0007] A control method of an electronic device according to one or more embodiments of the present disclosure includes: obtaining first weight data including a plurality of weights for each of a plurality of layers of a neural network model; obtaining second weight data by quantizing the first weight data using a plurality of quantization intervals; training the neural network model such that smallest target quantization intervals among the plurality of quantization intervals for each of the plurality of layers converge to a preset value; obtaining third weight data by treating at least one quantization interval satisfying a preset condition among the target quantization intervals for each of the plurality of layers as 0 based on a result of training the neural network model; and obtaining a quantized neural network model based on the third weight.

[0008] A non-transitory computer-readable recording medium including a program for executing a control method of an electronic device according to one or more embodiments of the present disclosure, the control method comprising: obtaining first weight data including a plurality of weights for each of a plurality of layers of a neural network model; obtaining second weight data by quantizing the first weight data using a plurality of quantization intervals; training the neural network model such that smallest target quantization intervals among the plurality of quantization intervals for each of the plurality of layers converge to a preset value; obtaining third weight data by treating at least one quantization interval satisfying a preset condition among the target quantization intervals for each of the plurality of layers as 0 based on a result of training the neural network model; and obtaining a quantized neural network model based on the third weight.

[0009] FIG. 1 is a block diagram illustrating a configuration of an electronic device according to one or more embodiments of the present disclosure.

[0010] FIG. 2 is a flowchart illustrating a method for controlling an electronic device according to one or more embodiments of the present disclosure.

[0011] FIG. 3 is a diagram illustrating a plurality of quantization intervals according to one or more embodiments of the present disclosure.

[0012] FIGS. 4, 5, and 6 are diagrams illustrating bit widths of neural network models according to one or more embodiments of the present disclosure.

[0013] The present embodiments may be modified and have various embodiments. Therefore, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope to specific embodiments, but should be understood to encompass various modifications, equivalents, and / or alternatives of the embodiments of the present disclosure. In connection with the description of the drawings, similar reference numerals may be used for similar components.

[0014] In describing the present disclosure, if it is determined that a specific description of a related known function or configuration may unnecessarily obscure the gist of the present disclosure, a detailed description thereof will be omitted.

[0015] Additionally, the following embodiments may be modified in various other forms, and the scope of the technical concepts of the present disclosure is not limited to the following embodiments. Rather, these embodiments are provided to further faithfully and completely convey the technical concepts of the present disclosure to those skilled in the art.

[0016] The terminology used in this disclosure is for the purpose of describing specific embodiments only and is not intended to limit the scope of the rights. Singular expressions include plural expressions unless the context clearly dictates otherwise.

[0017] In this disclosure, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a corresponding feature (e.g., a component such as a number, function, operation, or part), and do not exclude the presence of additional features.

[0018] In this disclosure, expressions such as “A or B,” “at least one of A and / or B,” or “one or more of A or / and B” can include all possible combinations of the listed items. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” can all refer to (1) including at least one A, (2) including at least one B, or (3) including both at least one A and at least one B.

[0019] The expressions “first,” “second,” “first,” or “second,” etc., used in this disclosure can describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0020] When it is said that a component (e.g., a first component) is “(operatively or communicatively) coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that said component may be directly coupled to said other component, or may be coupled via another component (e.g., a third component).

[0021] On the other hand, when it is said that a component (e.g., a first component) is "directly connected" or "directly connected" to another component (e.g., a second component), it can be understood that no other component (e.g., a third component) exists between said component and said other component.

[0022] The expression "configured to" used in the present disclosure may be used interchangeably with, for example, "suitable for," "having the capacity to," "designed to," "adapted to," "made to," or "capable of." The term "configured to" may not necessarily mean only "specifically designed to" in terms of hardware.

[0023] Instead, in some contexts, the phrase "a device configured to" may mean that the device, in conjunction with other devices or components, is "capable of" performing A, B, and C. For example, the phrase "a processor configured (or set) to perform A, B, and C" may refer to a dedicated processor (e.g., an embedded processor) for performing those operations, or a general-purpose processor (e.g., a CPU or application processor) that can perform those operations by executing one or more software programs stored in a memory device.

[0024] In the embodiments, a 'module' or 'part' performs at least one function or operation, and may be implemented as hardware or software, or as a combination of hardware and software. Furthermore, a plurality of 'modules' or 'parts' may be integrated into at least one module and implemented as at least one processor, except for a 'module' or 'part' that needs to be implemented as a specific hardware.

[0025] Meanwhile, the various elements and areas in the drawings are schematically drawn. Therefore, the technical concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.

[0026] Hereinafter, with reference to the attached drawings, embodiments according to the present disclosure will be described in detail so that a person having ordinary knowledge in the technical field to which the present disclosure pertains can easily implement the present disclosure.

[0027] FIG. 1 is a block diagram illustrating a configuration of an electronic device according to one or more embodiments of the present disclosure.

[0028] The electronic device (100) may include at least one of a memory (110), a communication interface (120), and a processor (130). The electronic device (100) may further include other components in addition to the above components.

[0029] The electronic device (100) may be implemented as a server, but this is only one embodiment, and the electronic device (100) may be implemented in various forms such as a smartphone or tablet PC.

[0030] The memory (110) can store at least one instruction regarding the electronic device (100). The memory (110) can store an operating system (O / S) for driving the electronic device (100). In addition, the memory (110) can store various software programs or applications for operating the electronic device (100) according to various embodiments of the present disclosure. In addition, the memory (110) can include a semiconductor memory such as a flash memory or a magnetic storage medium such as a hard disk.

[0031] Specifically, the memory (110) can store various software modules for operating the electronic device (100) according to various embodiments of the present disclosure, and the processor (130) can control the operation of the electronic device (100) by executing various software modules stored in the memory (110). That is, the memory (110) is accessed by the processor (130), and data reading / recording / modifying / deleting / updating, etc. can be performed by the processor (130).

[0032] Meanwhile, in the present disclosure, the term memory (110) may be used to mean a memory (110), a ROM (not shown), a RAM (not shown) in a processor (130), or a memory card (not shown) (e.g., a micro SD card, a memory stick) mounted on an electronic device (100).

[0033] And, the communication interface (120) includes a circuitry and is a configuration capable of communicating with external devices and servers. The communication interface (120) can communicate with external devices or servers based on a wired or wireless communication method. The communication interface (120) may include a Bluetooth module (not shown), a Wi-Fi module (not shown), an IR (infrared) module, a LAN (Local Area Network) module, an Ethernet module, etc. Here, each communication module may be implemented in the form of at least one hardware chip. In addition to the above-described communication method, the wireless communication module may include at least one communication chip that performs communication according to various wireless communication standards such as Zigbee, USB (Universal Serial Bus), MIPI CSI (Mobile Industry Processor Interface Camera Serial Interface), 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), 5G (5th Generation), etc. However, this is only one embodiment, and the communication interface (120) can utilize at least one communication module among various communication modules.

[0034] The processor (130) can control the overall operation and function of the electronic device (100). Specifically, the processor (130) is connected to the configuration of the electronic device (100) including the memory (110), and can control the overall operation of the electronic device (100) by executing at least one command stored in the memory (110) as described above.

[0035] The processor (130) may be implemented in various ways. For example, the processor (130) may be implemented as at least one of an application specific integrated circuit (ASIC), a logic integrated circuit, an embedded processor, a microcomputer (Micom), a microprocessor, hardware control logic, a hardware finite state machine (FSM), and a digital signal processor (DSP).

[0036] In particular, the processor (130) may include one or more processors. Specifically, the one or more processors may include one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an Accelerated Processing Unit (APU), a Many Integrated Core (MIC), a Digital Signal Processor (DSP), a Neural Processing Unit (NPU), a Main Processing Unit (MPU), a hardware accelerator, or a machine learning accelerator. The one or more processors may control one or any combination of other components of the electronic device, and may perform operations related to communication or data processing. The one or more processors may execute one or more programs or instructions stored in a memory. For example, the one or more processors may perform a method according to an embodiment of the present disclosure by executing one or more instructions stored in a memory.

[0037] When a method according to one or more embodiments of the present disclosure includes multiple operations, the multiple operations may be performed by one processor or by multiple processors. That is, when a first operation, a second operation, and a third operation are performed by a method according to one or more embodiments, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor (e.g., a general-purpose processor) and the third operation may be performed by the second processor (e.g., an artificial intelligence-specific processor).

[0038] One or more processors may be implemented as a single core processor including one core, or may be implemented as one or more multicore processors including multiple cores (e.g., homogeneous multicores or heterogeneous multicores). When one or more processors are implemented as a multicore processor, each of the multiple cores included in the multicore processor may include internal processor memory, such as cache memory or on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. In addition, each of the multiple cores (or some of the multiple cores) included in the multicore processor may independently read and execute program instructions for implementing a method according to one or more embodiments of the present disclosure, or all (or some) of the multiple cores may be linked to read and execute program instructions for implementing a method according to one or more embodiments of the present disclosure.

[0039] When a method according to one or more embodiments of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one core among the plurality of cores included in a multi-core processor, or may be performed by the plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to one or more embodiments, the first operation, the second operation, and the third operation may all be performed by a first core included in the multi-core processor, or the first operation and the second operation may be performed by a first core included in the multi-core processor, and the third operation may be performed by a second core included in the multi-core processor.

[0040] In embodiments of the present disclosure, the processor (130) may mean a system on a chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, a GPU, an APU, a MIC, a DSP, an NPU, a hardware accelerator, or a machine learning accelerator, but embodiments of the present disclosure are not limited thereto.

[0041] The operation of the processor (130) for implementing various embodiments of the present disclosure may be implemented through a plurality of modules.

[0042] Specifically, data for a plurality of modules according to the present disclosure can be stored in a memory (110), and the processor (130) can access the memory (110) to load the data for the plurality of modules into a memory or buffer within the processor (130), and then implement various embodiments according to the present disclosure using the plurality of modules.

[0043] However, at least one of the plurality of modules according to the present disclosure may be implemented in hardware and included in the processor (130) in the form of a system on chip.

[0044] Alternatively, at least one of the plurality of modules according to the present disclosure may be implemented as a separate external device, and the electronic device (100) and each module may communicate and perform operations according to the present disclosure.

[0045] Hereinafter, the operation of the processor (130) according to the present disclosure will be described in detail with reference to the attached drawings.

[0046] FIG. 2 is a drawing for explaining a control method of an electronic device (100) according to one or more embodiments of the present disclosure.

[0047] The processor (130) may obtain first weight data including a plurality of weights for each of a plurality of layers constituting a neural network model (S210). At this time, the first weight data may be data in which the weights of each of the plurality of layers constituting the neural network model are expressed in the same bit width (e.g., 6 bits).

[0048] Specifically, the memory (110) can store data for a neural network model. Specifically, the memory (110) can store first weight data including weights for each of a plurality of layers of the neural network model. In addition, the memory (110) can store training data for training the neural network model.

[0049] At this time, the neural network model stored in the memory (110) may be a model optimized so that each of the multiple layers constituting the neural network model is expressed with the same bit width using the PTQ (Post-Training Quantization) or QAT (Quantization-Aware Training) method.

[0050] The processor (130) can obtain the first weight data by loading the first weight data stored in the memory (110) into a memory or buffer within the processor (130).

[0051] Alternatively, the processor (130) may obtain first weight data by quantizing the weights of each of the plurality of layers constituting the neural network model stored in the memory (110).

[0052] At this time, the neural network model stored in the memory (110) may be a pre-trained neural network model. At this time, the weights constituting the multiple layers included in the neural network model may be data expressed in a 32-bit floating point (FP32) format, but are not limited thereto.

[0053] In addition, the processor (130) can quantize the neural network model stored in the memory (110) using the PTQ (Post-Training Quantization) or QAT (Quantization-Aware Training) method to obtain first weight data in which the weights of each of the multiple layers are expressed with the same bit width.

[0054] The processor (130) can quantize the first weight data using multiple quantization intervals to obtain second weight data (S220). Specifically, the processor (130) can quantize the first weight data using multiple quantization intervals and multiple integer tensors to obtain second weight data. Here, the quantization interval may mean a scaling factor or a step size.

[0055] The method by which the processor (130) quantizes the weight data of the kth layer among multiple layers into multiple quantization intervals can be expressed according to the following mathematical expression 1.

[0056] Mathematical formula 1

[0057]

[0058] may mean the weight data of the k-th layer quantized using multiple quantization intervals. can mean the ith quantization interval. may mean weight data expressed in integer form for the i-th quantization interval. is a weight expressed in integer form for the i-th quantization interval. It can mean a compensation value to represent the difference between the original weight and the original weight.

[0059] Here, one quantization interval and one integer tensor can represent 2-bit data. Furthermore, to maintain a uniform spacing between quantization points, multiple quantization intervals can satisfy the relationship described in Equation 2 below.

[0060] Mathematical formula 2

[0061]

[0062] FIG. 3 is a diagram illustrating a plurality of quantization intervals according to one or more embodiments of the present disclosure.

[0063] Referring to FIG. 3, when n=2, the processor (130) can express 4-bit data using the first quantization interval s1 and the second quantization interval s2. At this time, z1 may be 1 and z2 may be 3, but is not limited thereto.

[0064] Although not shown in the drawing, when n=3, the processor (130) can express 6-bit data using the first quantization interval s1, the second quantization interval s2, and the third quantization interval s3.

[0065] The processor (130) can obtain second weight data by quantizing the weights of each of the plurality of layers into a plurality of quantization intervals according to the above-described method.

[0066] For example, referring to FIG. 4, the neural network model (10) may include four layers (11, 12, 13, 14). At this time, the bit width of the weight of each of the four layers (11, 12, 13, 14) may be 6 bits. That is, the second weight data may express 6 bits of data using the first quantization interval s1, the second quantization interval s2, and the third quantization interval s3.

[0067] And, the processor (130) can set a target quantization interval among a plurality of quantization intervals for each of a plurality of layers from the second weight data (S230).

[0068] Specifically, the processor (130) can set the smallest quantization interval among the plurality of quantization intervals for each of the plurality of layers as the target quantization interval.

[0069] At this time, the processor (130) can set the smallest quantization interval excluding 0 among the plurality of quantization intervals for each of the plurality of layers as the target quantization interval.

[0070] For example, if the first layer is expressed as 6-bit data using the first quantization interval s1, the second quantization interval s2, and the third quantization interval s3, the processor can set the third quantization interval s3 as the target quantization interval.

[0071] The processor (130) can set a target quantization interval for each of the second layer, the third layer, and the fourth layer, as described above.

[0072] In addition, the processor (130) can train a neural network model so that multiple target quantization intervals for multiple layers converge to preset values ​​(S240). Here, the preset value may be 0, but this is only an example, and the preset value may be a value close to 0 or an arbitrary value.

[0073] Specifically, when the second weight data is acquired, the processor (130) can train the neural network model so that the first loss value according to the output value of the neural network model acquired based on the second weight data is minimized and each of the plurality of target quantization intervals converges to a preset value.

[0074] Specifically, the neural network model can be trained to minimize the loss value described below, and accordingly, multiple quantization intervals included in the second weight data can be updated. Hereinafter, the term "loss value" refers to the functional value of the loss function and can be replaced with the term "loss."

[0075] The first loss value may be defined based on the difference between the first output value of the neural network model obtained based on the first weight data and the second output value of the neural network model obtained based on the second weight data. Specifically, the electronic device (100) may train the neural network model according to a knowledge distillation technique in which the neural network model based on the first weight data serves as a teacher and the neural network model based on the second weight data serves as a student, but is not limited thereto.

[0076] Specifically, the processor (130) can train a neural network model based on the first weight data and obtain a first output value corresponding to an input value through the trained neural network model. When the second weight data is obtained, the processor (130) can obtain a second output value of the neural network model based on the second weight data. In addition, the processor (130) can obtain a first loss value based on the difference between the first output value and the second output value.

[0077] The second loss value may be defined such that each of the plurality of target quantization intervals converges to 0. Specifically, the second loss value may include a regularization term for the plurality of target quantization intervals. Specifically, the second loss value may serve to ensure that, when the plurality of target quantization intervals are updated during the training process of the neural network model, they are updated in a direction toward an extreme value closer to 0 among the plurality of extreme values.

[0078] The processor (130) can train the neural network model by performing backpropagation on the neural network model based on a third loss value defined as the sum of the first loss value and the second loss value. When the neural network model is trained in a direction that minimizes the third loss value, the second weight data can be updated so that each of the plurality of target quantization intervals converges at a value close to 0 while minimizing an error according to the output value of the neural network model.

[0079] In the learning process of a neural network model, various optimization methods such as batch gradient descent, stochastic gradient descent, mini-batch gradient descent, and Adam optimizer can be used as optimization methods (optimizers) to find parameters that minimize the loss function. Hereinafter, in order to specifically explain the learning process of the neural network model according to the present disclosure, reference will be made to Mathematical Expression 3.

[0080] Mathematical formula 3

[0081]

[0082] Mathematical expression 3 represents a first loss value (L1), a second loss value (L2), and a third loss value (L) according to the present disclosure. As shown in Mathematical expression 3, the third loss value (L) may be formed by the sum of the first loss value (L1) and the second loss value (L2). The first loss value (L1) may be defined in the form of various loss functions, such as L1 loss, L2 loss, MSE (mean squared error), and RMSE (root mean squared error). Meanwhile, the second loss value (L2) may include a term representing a normalization constant k and a size of a quantization interval, as in Mathematical expression 2.

[0083] Mathematical formula 4

[0084]

[0085] Mathematical expression 4 illustrates the process of updating the quantization interval during the learning process of a neural network model. Referring to Mathematical expression 4, the quantization interval St+1 at time t+1 can be calculated by subtracting the product of the learning rate and the derivative for the third loss value (L) from the quantization interval St at time t.

[0086] Specifically, as shown in Equations 5 and 6 below, regardless of the sign of the quantization interval, in the process of calculating the quantization interval st+1 at time t+1, a process of subtracting the product of the gradient of the first loss value (L1) and the learning rate from the scaling factor st at time t can be performed.

[0087] Mathematical Formula 5

[0088] , if s > 0

[0089] Mathematical formula 6

[0090] , if s > 0

[0091] Meanwhile, if the quantization interval is greater than 0, in the process of calculating the quantization interval St+1 at time t+1, a process of subtracting rk from the quantization interval St at time t may be performed. And, if the quantization interval is less than 0, in the process of calculating the quantization interval St+1 at time t+1, a process of adding rk to the quantization interval St at time t may be performed. In other words, if the quantization interval is greater than 0, in the process of calculating the quantization interval St+1 at time t+1, the quantization interval St at time t may be moved towards 0 by -rk, and if the quantization interval is less than 0, in the process of calculating the quantization interval St+1 at time t+1, the quantization interval St at time t may be moved towards 0 by +rk.

[0092] Meanwhile, as described above, based on the least squares method and binary search method, when the second weight data is changed, the electronic device (100) can train the neural network model based on the changed second weight data.

[0093] As a result of learning the neural network model, the processor (130) can set a target quantization interval that satisfies a preset condition among a plurality of target quantization intervals to 0 (S250).

[0094] Specifically, as a result of learning the neural network model, the processor (130) can identify at least one target quantization interval that satisfies a preset condition among a plurality of target quantization intervals.

[0095] Here, the preset condition may be a condition in which the target quantization interval is less than or equal to a preset value. That is, the processor (130) may identify at least one target quantization interval among a plurality of target quantization intervals whose value is less than or equal to the preset value.

[0096] Alternatively, the preset condition may be a condition in which the value is a preset number in descending order among a plurality of target quantization intervals. Specifically, the processor (130) may identify at least one target quantization interval that is a preset number in descending order among a plurality of target quantization intervals. For example, the processor (130) may identify the target quantization interval of the second layer and the target quantization interval of the third layer, which are two target quantization intervals in descending order among the target quantization interval of the first layer, the target quantization interval of the second layer, the target quantization interval of the third layer, and the target quantization interval of the fourth layer.

[0097] When at least one target quantization interval is identified, the processor (130) can obtain third weight data by setting the identified at least one target quantization interval to 0 from the second weight data.

[0098] For example, the target quantization interval of the second layer and the target quantization interval of the third layer may be treated as 0. Accordingly, in the third weight data, the bit width of the weights of the second layer and the third layer may be 4 bits. And, in the third weight data, the bit width of the weights of the first layer and the fourth layer may be 6 bits.

[0099] And, based on the third weight data, the processor (130) can obtain a quantized neural network model. For example, referring to FIG. 5, the quantized neural network model (10) may have a bit width of 6 bits for the weights of the first layer (11) and the fourth layer (14), and a bit width of 4 bits for the weights of the second layer (12) and the third layer (13).

[0100] And, the processor (130) can train the neural network model so that the performance of the neural network model satisfies preset conditions (S260).

[0101] Specifically, the processor (130) can train the quantized neural network model until the accuracy of the quantized neural network model becomes a preset value or higher.

[0102] When the training of the quantized neural network model is completed, the processor (130) can identify whether the performance of the quantized neural network model for which training has been completed satisfies the required performance conditions (S270).

[0103] At this time, the processor can identify whether the performance of the quantized neural network model for which learning has been completed satisfies the required performance conditions. However, this is only an example, and the learning process of the quantized neural network model may be omitted. That is, the processor (130) can omit step S260 and perform the operation according to step S270 after step S250.

[0104] Accordingly, the processor (130) can identify whether the performance of the quantized neural network model satisfies the required performance conditions without performing an operation of training the quantized neural network model.

[0105] The required performance conditions may include at least one of the inference delay time, memory usage, power consumption, and the size of the quantized neural network model while the quantized neural network model performs inference. Information regarding the required performance conditions may be stored in the memory (110).

[0106] For example, the required performance condition may be a condition in which the inference delay time of the quantized neural network model is less than or equal to a preset first value and the memory consumption of the quantized neural network model is less than or equal to a preset second value.

[0107] If the performance of the quantized neural network model satisfies the required performance condition (S270-Y), the processor (130) can complete the quantization operation of the neural network model (S280). That is, the processor (130) can identify the third weight data as the optimal weight data for quantizing the neural network model. In the present disclosure, the optimal weight data is weight data obtained when the condition for stopping learning of the neural network model is satisfied, and the processor (130) can store the weight data identified as the optimal weight data in the memory (110).

[0108] If the performance of the quantized neural network model does not meet the required performance conditions (S270-N), the processor may continue to perform operations below S230.

[0109] Specifically, if the performance of the quantized neural network model does not satisfy the required performance condition, the processor may set the smallest quantization interval among the plurality of quantization intervals for each of the plurality of layers as the target quantization interval (S230). At this time, the processor (130) may set the smallest quantization interval among the plurality of quantization intervals for each of the plurality of layers as the target quantization interval from the weight data obtained in step S250. For example, the processor (130) may set the smallest quantization interval among the plurality of quantization intervals for each of the plurality of layers as the target quantization interval from the third weight data.

[0110] As illustrated in FIG. 5, in the third weight data, when the bit widths of the weights of the first layer (11) and the fourth layer (14) are 6 bits, and the bit widths of the weights of the second layer (12) and the third layer (13) are 4 bits, the target quantization interval of the first layer (11) and the target quantization interval of the fourth layer (14) may be s3. In addition, the target quantization interval of the second layer (12) and the target quantization interval of the third layer (13) may be s2. At this time, the target quantization interval s3 of the first layer (11) and the target quantization interval s3 of the fourth layer (14) may be different. In addition, the target quantization interval s2 of the second layer (12) and the target quantization interval s2 of the third layer (13) may be different.

[0111] Once the target quantization interval is set, the processor (130) can train a neural network model so that multiple target quantization intervals converge to the preset value (S240). At this time, the processor (130) can train the neural network model so that multiple target quantization intervals converge to the preset value in the third weight data.

[0112] As a result of learning the neural network model, the processor (130) can set a target quantization interval that satisfies a preset condition among a plurality of target quantization intervals to 0 (S250). At this time, the processor (130) can set a target quantization interval that satisfies a preset condition among a plurality of target quantization intervals to 0 in the third weight data. That is, the processor (130) can obtain the fourth weight data by setting a target quantization interval that satisfies a preset condition among a plurality of target quantization intervals to 0 in the third weight data.

[0113] Accordingly, the bit width of the weight of the neural network model may be as illustrated in FIG. 6. For example, among the plurality of target quantization intervals, the target quantization intervals that satisfy the preset conditions may be the target quantization interval of the first layer (11) and the target quantization interval of the third layer (13). Accordingly, the target quantization interval s3 of the first layer (11) may be set to 0, and the target quantization interval s2 of the third layer (13) may be set to 0. At this time, the bit width of the weight of the first layer (11) may be 4 bits, the bit width of the weight of the second layer (12) may be 4 bits, the bit width of the weight of the third layer (13) may be 2 bits, and the bit width of the weight of the fourth layer (14) may be 6 bits.

[0114] Meanwhile, if at least one target quantization interval among a plurality of target quantization intervals that satisfy a preset condition is s1, the processor (130) may not set the target quantization interval corresponding to s1 to 0 and may only set the remaining target quantization intervals that satisfy the preset condition to 0.

[0115] When the target quantization interval satisfying the preset conditions is set to 0, the processor (130) trains the neural network model so that the performance of the neural network model satisfies the preset conditions (S260), and can identify whether the performance of the neural network model satisfies the required performance conditions (S270). As described above, the operation according to step S260 may be omitted.

[0116] If the performance of the neural network model satisfies the required performance condition (S270-Y), the processor (130) can complete the quantization operation of the neural network model (S280).

[0117] And, if the performance of the neural network model does not satisfy the required performance condition (S270-N), the processor (130) can repeatedly perform the operations below S230 to obtain a neural network model that satisfies the required performance condition.

[0118] According to the various embodiments of the present disclosure described above, the electronic device (100) can identify an optimal bit range for each layer of a neural network model while minimizing the time and resource consumption required to search for the optimal bit range. Accordingly, an effective mixed-precision neural network model can be obtained.

[0119] The artificial intelligence-related function according to the present disclosure is operated through the processor (130) and memory (110) of the electronic device (100).

[0120] The processor (130) may be composed of one or more processors (130). At this time, the one or more processors (130) may include at least one of a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), and an NPU (Neural Processing Unit), but is not limited to the examples of the processors (130) described above.

[0121] The CPU is a general-purpose processor (130) capable of performing not only general calculations but also artificial intelligence calculations. Its multi-layer cache structure allows for the efficient execution of complex programs. The CPU is advantageous in a serial processing method, enabling organic linking of previous and subsequent calculation results through sequential calculations. The general-purpose processor (130) is not limited to the aforementioned examples, except in cases where it is specifically designated as a CPU.

[0122] A GPU is a processor (130) for large-scale operations such as floating-point operations used in graphic processing, and can perform large-scale operations in parallel by integrating a large number of cores. In particular, a GPU may be advantageous compared to a CPU in parallel processing methods such as convolution operations. In addition, a GPU may be used as a co-processor (130) to supplement the functions of a CPU. The processor (130) for large-scale operations is not limited to the examples described above, except in cases where it is specified as a GPU as described above.

[0123] An NPU is a processor (130) specialized in artificial intelligence operations using an artificial neural network, and each layer constituting the artificial neural network can be implemented with hardware (e.g., silicon). At this time, since the NPU is designed specifically according to the required specifications of the company, it has a lower degree of freedom compared to a CPU or GPU, but it can efficiently process the artificial intelligence operations requested by the company. Meanwhile, as a processor (130) specialized in artificial intelligence operations, the NPU can be implemented in various forms such as a Tensor Processing Unit (TPU), an Intelligence Processing Unit (IPU), a Vision Processing Unit (VPU), etc. The artificial intelligence processor (130) is not limited to the above-described examples, except in cases where it is specified as the above-described NPU.

[0124] Additionally, one or more processors (130) may be implemented as a SoC (System on Chip). In this case, the SoC may further include, in addition to one or more processors (130), a memory (110), and a network interface such as a bus for data communication between the processor (130) and the memory (110).

[0125] When a plurality of processors (130) are included in a SoC (System on Chip) included in an electronic device (100), the electronic device (100) may perform operations related to artificial intelligence (e.g., operations related to learning or inference of an artificial intelligence model) by using some of the plurality of processors (130). For example, the electronic device (100) may perform operations related to artificial intelligence by using at least one of a GPU, an NPU, a VPU, a TPU, and a hardware accelerator specialized in artificial intelligence operations such as convolution operations and matrix multiplication operations among the plurality of processors (130). However, this is merely an example, and it is of course possible to process operations related to artificial intelligence by using a CPU or a general-purpose processor (130).

[0126] In addition, the electronic device (100) can perform operations related to functions related to artificial intelligence by utilizing multiple cores (e.g., dual cores, quad cores, etc.) included in one processor (130). In particular, the electronic device (100) can perform artificial intelligence operations such as convolution operations, matrix multiplication operations, etc. in parallel by utilizing multiple cores included in the processor (130).

[0127] One or more processors (130) are controlled to process input data according to predefined operation rules or artificial intelligence models stored in the memory (110). The predefined operation rules or artificial intelligence models are characterized by being created through learning.

[0128] Here, "created through learning" means that a predefined set of behavioral rules or an AI model with desired characteristics is created by applying a learning algorithm to a large number of learning data. This learning may be performed on the device itself, where the AI ​​according to the present disclosure is implemented, or through a separate server / system.

[0129] An artificial intelligence model may be composed of multiple neural network layers. At least one layer has at least one weight value and performs its operation through the operation result of the previous layer and at least one defined operation. Examples of neural networks include a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), deep Q-networks, and a transformer. The neural networks in the present disclosure are not limited to the above-described examples unless otherwise specified.

[0130] A learning algorithm is a method for training a target device (e.g., a robot) using a large amount of learning data, enabling the target device to make decisions or predictions on its own. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. Unless otherwise specified, the learning algorithms in this disclosure are not limited to the aforementioned examples.

[0131] A device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory storage medium" simply means a tangible device that does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is permanently stored in the storage medium and cases where data is temporarily stored. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0132] According to one embodiment, the method according to the various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., downloadable app) may be temporarily stored or temporarily generated in a machine-readable storage medium, such as a memory (110) of a manufacturer's server, an application store's server, or a relay server.

[0133] Each of the components (e.g., modules or programs) according to the various embodiments of the present disclosure as described above may be composed of a single or multiple entities, and some of the sub-components described above may be omitted, or other sub-components may be further included in the various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the respective components prior to integration.

[0134] According to various embodiments, operations performed by a module, program or other component may be executed sequentially, in parallel, iteratively or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0135] Meanwhile, the terms "part" or "module" used in the present disclosure include units composed of hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A "part" or "module" may be an integrally composed component, a minimum unit performing one or more functions, or a portion thereof. For example, a module may be composed of an application-specific integrated circuit (ASIC).

[0136] Various embodiments of the present disclosure may be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The device, which is a device capable of calling instructions stored in the storage medium and operating according to the called instructions, may include an electronic device (100) according to the disclosed embodiments. When the instructions are executed by a processor, the processor may directly or under the control of the processor perform a function corresponding to the instructions using other components. The instructions may include code generated or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory" means that the storage medium does not contain signals and is tangible, but does not distinguish between data being stored semi-permanently or temporarily in the storage medium.

[0137] According to one or more embodiments, the methods according to the various embodiments disclosed herein may be provided as a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0138] Each component (e.g., a module or a program) according to various embodiments may be composed of one or more entities, and some of the aforementioned sub-components may be omitted, or other sub-components may be further included in various embodiments. Alternatively or additionally, some components (e.g., a module or a program) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the respective components prior to integration. Operations performed by a module, program, or other component according to various embodiments may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

Claims

1. In electronic devices, A memory storing first weight data including a plurality of weights for each of a plurality of layers of a neural network model; and A processor connected to said memory and controlling said electronic device, The above processor, The first weight data is quantized using multiple quantization intervals to obtain the second weight data, The neural network model is trained so that the smallest target quantization intervals among the plurality of quantization intervals for each of the plurality of layers converge to a preset value, Based on the learning result of the neural network model, by treating at least one quantization interval satisfying a preset condition among the target quantization intervals for each of the plurality of layers as 0, the third weight data is obtained, An electronic device for obtaining a quantized neural network model based on the third weight.

2. In paragraph 1, The above processor, Based on the third weight, the quantized neural network model is trained until the accuracy of the quantized neural network model becomes higher than a preset value, An electronic device that identifies the quantized neural network model as an optimal neural network model if the performance of the quantized neural network model satisfies a required performance condition.

3. In paragraph 2, The above required performance conditions are an electronic device in which the inference delay time of the quantized neural network model is equal to or less than a preset first value and the memory consumption of the quantized neural network model is equal to or less than a preset second value.

4. In paragraph 2, The above processor, If the performance of the above quantized neural network model does not satisfy the required performance condition, the quantization intervals corresponding to the highest bit width for each of the plurality of layers in the third weight data are identified as the second target quantization intervals, By treating at least one second quantization interval satisfying a preset condition among the above second target quantization intervals as 0, the fourth weight data is obtained, An electronic device that obtains the quantized neural network model based on the fourth weight data.

5. In paragraph 1, The above processor, An electronic device that trains the neural network model so that a loss value according to an output value of the neural network model obtained based on the second weight data is minimized and the target quantization intervals converge to 0.

6. In paragraph 1, The above processor, An electronic device that treats at least one target quantization interval that is less than or equal to a preset value among the above target quantization intervals as 0.

7. In paragraph 1, The above processor, An electronic device that, based on the learning result of the neural network model, treats at least one quantization interval from among a preset number of target quantization intervals in descending order as 0 in each of the plurality of layers.

8. In a method for controlling an electronic device, A step of obtaining first weight data including a plurality of weights for each of a plurality of layers of a neural network model; A step of obtaining second weight data by quantizing the first weight data using a plurality of quantization intervals; A step of training the neural network model so that the smallest target quantization intervals among the plurality of quantization intervals for each of the plurality of layers converge to a preset value; A step of obtaining third weight data by treating at least one quantization interval satisfying a preset condition among the target quantization intervals for each of the plurality of layers as 0 based on the learning result of the neural network model; and A control method, comprising: a step of obtaining a quantized neural network model based on the third weight; 9. In paragraph 8, The above control method is, A step of training the quantized neural network model until the accuracy of the quantized neural network model becomes a preset value or higher based on the third weight; and A control method, comprising: a step of identifying the quantized neural network model as an optimal neural network model if the performance of the quantized neural network model satisfies a required performance condition; 10. In paragraph 9, The above required performance conditions are a control method in which the inference delay time of the quantized neural network model is less than or equal to a preset first value and the memory consumption of the quantized neural network model is less than or equal to a preset second value.

11. In paragraph 9, The above control method is, If the performance of the above quantized neural network model does not satisfy the required performance condition, a step of identifying quantization intervals corresponding to the highest bit width for each of the plurality of layers in the third weight data as second target quantization intervals; A step of obtaining fourth weight data by treating at least one second quantization interval satisfying a preset condition among the second target quantization intervals as 0; and A control method, comprising: a step of obtaining the quantized neural network model based on the fourth weight data; 12. In paragraph 8, The step of training the above neural network model is: A control method for training the neural network model so that a loss value according to an output value of the neural network model obtained based on the second weight data is minimized and the target quantization intervals converge to 0.

13. In paragraph 8, The step of obtaining the third weight data is: A control method for treating at least one target quantization interval that is less than or equal to a preset value among the above target quantization intervals as 0.

14. In paragraph 8, The step of obtaining the third weight data is: A control method for treating at least one quantization interval in a predetermined number in descending order of values ​​among target quantization intervals as 0, based on the learning result of the neural network model in each of the plurality of layers.

15. A non-transitory computer-readable recording medium including a program for executing a method of controlling an electronic device, The above control method is, A step of obtaining first weight data including a plurality of weights for each of a plurality of layers of a neural network model; A step of obtaining second weight data by quantizing the first weight data using a plurality of quantization intervals; A step of training the neural network model so that the smallest target quantization intervals among the plurality of quantization intervals for each of the plurality of layers converge to a preset value; A step of obtaining third weight data by treating at least one quantization interval satisfying a preset condition among the target quantization intervals for each of the plurality of layers as 0 based on the learning result of the neural network model; and A non-transitory computer-readable recording medium, comprising: a step of obtaining a quantized neural network model based on the third weight;

Images